CN111090392A - Cold and hot data separation method based on feature codes - Google Patents
Cold and hot data separation method based on feature codes Download PDFInfo
- Publication number
- CN111090392A CN111090392A CN201911144275.1A CN201911144275A CN111090392A CN 111090392 A CN111090392 A CN 111090392A CN 201911144275 A CN201911144275 A CN 201911144275A CN 111090392 A CN111090392 A CN 111090392A
- Authority
- CN
- China
- Prior art keywords
- data
- cold
- hot
- feature codes
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a cold and hot data separation method based on a feature code, which judges whether the written data has the feature code or not, if so, stores the data in a corresponding interval according to the feature code, if not, separates the cold and hot data according to the read-write frequency, and simultaneously evaluates and updates the feature code and adjusts the feature code in the above process. According to the cold and hot data separation method based on the feature codes, data are classified in advance, the data are stored in different regions according to classification, the data regions are divided according to the attributes of the data regions from the source, the erasing times of each block are uniform, and the service life of Nand storage equipment is prolonged.
Description
Technical Field
The invention relates to the technical field of memories, in particular to a cold and hot data separation method based on feature codes.
Background
Currently, Nand storage devices use Nand Flash as the basic storage medium. The Nand Flash requires that blocks are erased before data is written, the writing sequence in each block has a strict sequence requirement, the minimum unit of reading and writing is a page, and when the erasing frequency (abrasion) of each block exceeds a certain value, the data is easy to lose.
In order to meet the basic characteristics of Nand and the random reading and writing requirements of upper-Layer users, a Flash Translation Layer (FTL) will implement the conversion of the process.
Based on the characteristics of Nand Flash, the FTL manages data written by a user, and when addresses are repeatedly written, an original Nand physical address is not immediately written, but a new address is selected for writing, and the old address is set as invalid. When Nand is to be written to, FTL will move some blocks of valid data away and erase the whole block to ensure enough space to store data. This process is referred to as a garbage recovery operation (GC).
The service life of the Nand storage device is affected by the wear times and wear balance, frequently written data (namely hot write data) and rarely written data are respectively stored, and the data moved away every time are less, so that the write efficiency can be improved, and the service life of the storage device can be prolonged due to the fact that the moving times are less.
In some existing separation methods, the FTL can only "predict" the cold and hot properties of the data after the written data, and such separation cannot identify the change of the "cold and hot" properties of the data, nor can determine the "cold and hot" properties of the data written for the first time.
Therefore, how to distinguish cold and hot intervals according to the cold and hot characteristics of data to realize the distinction of accurate 'cold and hot' attributes is an urgent problem to be solved.
Disclosure of Invention
The invention aims to provide a cold and hot data separation method based on a feature code, which judges whether the feature code exists in written data or not, stores the data in a corresponding interval according to the feature code if the feature code exists, separates the cold and hot data according to the read-write frequency if the feature code does not exist, and simultaneously evaluates and updates the feature code and adjusts the feature code. According to the cold and hot data separation method based on the feature codes, data are stored in a distinguishing mode, the erasing times of each block are uniform, and the service life of the Nand storage device is prolonged.
The above object of the present invention is achieved by the following technical solutions:
a cold and hot data separation method based on feature codes comprises the following steps:
s1, inputting a feature code;
s2, inputting data;
s3, judging whether the data are the data which accord with the feature codes, if so, turning to S8, and if not, entering the next step;
s4, preparing to store data;
s5, separating cold and hot data according to the read-write frequency;
s6, respectively storing cold and hot data according to the separation result;
s7, evaluating the feature code, and turning to S11;
s8, preparing to store data;
s9, separating cold and hot data according to the feature codes;
s10, respectively storing cold and hot data according to the separation result;
s11, updating the feature code rule;
and S12, ending.
The invention is further configured to: the feature codes comprise hot-write data feature codes, hot-read data feature codes and cold data feature codes.
The invention is further configured to: the feature code comprises a region start address, a region length, a data cold and hot type and a reserved field of cold data or hot data.
The invention is further configured to: in step S5, a threshold is set for the read/write frequency, and when the read/write frequency is equal to or higher than the threshold, the data is classified as hot data, and when the read/write frequency is lower than the threshold, the data is classified as cold data.
The invention is further configured to: in step S7, feature codes are extracted based on the separation result of the cold and hot data, and new feature codes are obtained.
The invention is further configured to: in step S9, the data separation according to the feature code includes the following steps:
a1, start
A2, identifying the area;
a3, judging whether the data are the data which accord with the characteristic code of the hot-written data, if so, turning to A6, and if not, entering the next step;
a4, judging whether the data is the data which accords with the characteristic code of the thermal reading data, if so, turning to A7, and if not, entering the next step;
a5, storing the data in a cold data area, and turning to A8;
a6, storing the data in a hot data area, and turning to A8;
a7, storing data in a cold data area which is checked regularly;
a8, counting the access frequency;
a9, evaluating a feature code;
and A10, ending.
The invention is further configured to: and evaluating the feature codes according to the access frequency of the data, and redefining the cold and hot features of the data.
The invention is further configured to: in step S1, the system file of the operating system is defined as hot read data.
The invention is further configured to: in step S1, the data in the frequently read/write section in the application program is defined as the hot write data.
The invention is further configured to: in step S1, the backup data is defined as cold data.
Compared with the prior art, the invention has the beneficial technical effects that:
1. according to the method, the characteristic codes are set, the cold and hot properties of the data are judged according to the characteristic codes, and the data are stored in different areas according to the data properties, so that the erasing times of each block of Nand Flash are uniform, the service life of equipment is effectively prolonged, and the reading and writing efficiency is effectively improved;
2. furthermore, according to the data property, the corresponding feature codes are given to the data, and the data property is timely modified according to the change of the situation in the storage process, so that the dynamic modification of the data property is realized;
3. furthermore, the characteristic codes are evaluated and updated in the operation process, so that the cold and hot properties of the data are updated in time according to the change of the condition, and the phenomenon that the erasing times are uneven due to the change is avoided.
Drawings
FIG. 1 is a schematic flow chart of a method for separating cold and hot data according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a method for separating cold and hot data based on feature codes according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The invention relates to a cold and hot data separation method based on feature codes, which comprises the following steps as shown in figure 1:
s1, inputting a feature code;
s2, inputting data;
s3, judging whether the data are the data which accord with the feature codes, if so, turning to S8, and if not, entering the next step;
s4, preparing to store data;
s5, separating cold and hot data according to the read-write frequency;
s6, respectively storing cold and hot data according to the separation result;
s7, evaluating the feature code, and turning to S11;
s8, preparing to store data;
s9, separating cold and hot data according to the feature codes;
s10, respectively storing cold and hot data according to the separation result;
s11, updating the feature code rule;
and S12, ending.
Specifically, the feature code is a hot-write data feature code, a hot-read data feature code, and a cold data feature code.
The expression mode of the feature code comprises the following steps: area start address, area length, data hot and cold type, reserved field of cold data or hot data.
In one embodiment of the present application, in step S1, the system file of the operating system is defined as the hot read data.
And defining the data of the frequently read-write interval in the application program as the hot-write data.
The backup data is defined as cold data.
In one embodiment of the present application, the mobile phone installs the operating system for the first time, the operating system is installed in the interval of [0,2G ], and the operating system is installed in the interval of [0,2G ]:
SignatureCode sc；
lb = 0; // data area start Address
Len =2 × 1024 × 2; // zone length
Type = 0; // Heat data
In a specific embodiment of the present application, the device is re-partitioned, and the original space [1G,2G ] will be changed from the cold data interval to the hot data interval, using the following feature codes:
SignatureCode sc；
lba =1 × 1024 × 2; // data area start Address
Len =1 × 1024 × 2; // zone length
Type = 0; // Heat data
The original [18G,20G ] space, which will change from hot data interval to cold data interval, uses the following signature:
SignatureCode sc; // data area start Address
Lba =18 × 1024 × 2; // zone length
sc.len=2*1024*1024*2；
Type = 1; // Cold data
In one embodiment of the present application, a newly installed application will frequently write data in [25G,30G ], using the following feature codes:
SignatureCode sc；
lba =25 × 1024 × 2; // data area start Address
Len =30 × 1024 × 2; // zone length
Type = 0; // Heat data
In one embodiment of the present application, after an application exits, data will not be written to the application until the occupied data interval [24G,31G ] is restarted.
SignatureCode sc；
Lba =24 × 1024 × 2; // data area start Address
Len =7 × 1024 × 2; // zone length
Type = 1; // Cold data
In step S2, the written data is either with or without a signature.
In step S3, it is determined whether the written data has a feature code, and if so, the data is further distinguished by entering a determination mode with a feature code; if the feature code is not contained, entering a judgment mode without the feature code.
In step S5, a threshold is set for the read/write frequency, and when the read/write frequency of a certain data is equal to or higher than the threshold, the data is classified as hot data, and when the read/write frequency of a certain data is lower than the threshold, the data is classified as cold data.
In step S6, the data with the hot and cold attributes separated is stored in a partitioned manner for each attribute. And at different periods, the storage areas of the cold data and the hot data are exchanged to even the times of erasing the storage areas, and the probability of damage to the storage equipment caused by excessive erasing of a certain block is reduced.
In step S7, according to the separation result of the cold and hot data, the data is tracked again to see whether the data keeps the original cold and hot attributes, and if the cold and hot attributes change, the feature codes are extracted again to obtain new feature codes.
In step S9, the data separation according to the feature code includes the following steps:
a1, start
A2, identifying the area;
a3, judging whether the data are the data which accord with the characteristic code of the hot-written data, if so, turning to A6, and if not, entering the next step;
a4, judging whether the data is the data which accords with the characteristic code of the thermal reading data, if so, turning to A7, and if not, entering the next step;
a5, storing the data in a cold data area, and turning to A8;
a6, storing the data in a hot data area, and turning to A8;
a7, storing data in a cold data area which is checked regularly;
a8, counting the access frequency;
a9, evaluating a feature code;
and A10, ending.
And for the data with the feature codes, the data is stored in a partitioned mode according to the feature codes as long as the attribute of the feature codes is judged.
After the data are stored, the feature code evaluation is carried out according to the access frequency of the data, and the cold and hot features of the data are redefined.
And updating the feature codes according to the evaluation results of the feature codes in the steps S7 and S11, so as to realize dynamic management of cold and hot data.
The embodiments of the present invention are preferred embodiments of the present invention, and the scope of the present invention is not limited by these embodiments, so: all equivalent changes made according to the structure, shape and principle of the invention are covered by the protection scope of the invention.
Claims (10)
1. A cold and hot data separation method based on feature codes is characterized in that: the method comprises the following steps:
s1, inputting a feature code;
s2, inputting data;
s3, judging whether the data are the data which accord with the feature codes, if so, turning to S8, and if not, entering the next step;
s4, preparing to store data;
s5, separating cold and hot data according to the read-write frequency;
s6, respectively storing cold and hot data according to the separation result;
s7, evaluating the feature code, and turning to S11;
s8, preparing to store data;
s9, separating cold and hot data according to the feature codes;
s10, respectively storing cold and hot data according to the separation result;
s11, updating the feature code rule;
and S12, ending.
2. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: the feature codes comprise hot-write data feature codes, hot-read data feature codes and cold data feature codes.
3. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: the feature code comprises a region start address, a region length, a data cold and hot type and a reserved field of cold data or hot data.
4. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: in step S5, a threshold is set for the read/write frequency, and when the read/write frequency is equal to or higher than the threshold, the data is classified as hot data, and when the read/write frequency is lower than the threshold, the data is classified as cold data.
5. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: in step S7, feature codes are extracted based on the separation result of the cold and hot data, and new feature codes are obtained.
6. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: in step S9, the data separation according to the feature code includes the following steps:
a1, start
A2, identifying the area;
a3, judging whether the data are the data which accord with the characteristic code of the hot-written data, if so, turning to A6, and if not, entering the next step;
a4, judging whether the data is the data which accords with the characteristic code of the thermal reading data, if so, turning to A7, and if not, entering the next step;
a5, storing the data in a cold data area, and turning to A8;
a6, storing the data in a hot data area, and turning to A8;
a7, storing data in a cold data area which is checked regularly;
a8, counting the access frequency;
a9, evaluating a feature code;
and A10, ending.
7. A cold-hot data separation method based on feature codes according to claim 1 or 6, characterized in that: and evaluating the feature codes according to the access frequency of the data, and redefining the cold and hot features of the data.
8. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: in step S1, the system file of the operating system is defined as hot read data.
9. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: in step S1, the data in the frequently read/write section in the application program is defined as the hot write data.
10. A cold-hot data separation method based on feature codes according to claim 1, characterized in that: in step S1, the backup data is defined as cold data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911144275.1A CN111090392A (en) | 2019-11-20 | 2019-11-20 | Cold and hot data separation method based on feature codes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911144275.1A CN111090392A (en) | 2019-11-20 | 2019-11-20 | Cold and hot data separation method based on feature codes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111090392A true CN111090392A (en) | 2020-05-01 |
Family
ID=70393553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911144275.1A Pending CN111090392A (en) | 2019-11-20 | 2019-11-20 | Cold and hot data separation method based on feature codes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111090392A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111930848A (en) * | 2020-09-17 | 2020-11-13 | 阿里云计算有限公司 | Data partition storage method, device and system |
| CN112379842A (en) * | 2020-11-18 | 2021-02-19 | 深圳安捷丽新技术有限公司 | Method and device for predicting cold and hot properties of data |
| CN113094004A (en) * | 2021-05-13 | 2021-07-09 | 深圳市得一微电子有限责任公司 | Mirror image mapping optimization processing method for specific regular data in storage equipment |
| CN113900597A (en) * | 2021-11-30 | 2022-01-07 | 深圳市安信达存储技术有限公司 | Data storage method, system, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7155467B1 (en) * | 2000-04-07 | 2006-12-26 | Hewlett-Packard Development Company, L.P. | Adaptive type-partitioned garbage collection |
| CN102902628A (en) * | 2012-09-18 | 2013-01-30 | 记忆科技(深圳)有限公司 | Flash memory and method and system for automatically separating cold and hot data based on same |
| US20150347029A1 (en) * | 2014-05-30 | 2015-12-03 | Sandisk Enterprise Ip Llc | Identification of Hot Regions to Enhance Performance and Endurance of a Non-Volatile Storage Device |
| CN107562378A (en) * | 2017-08-28 | 2018-01-09 | 记忆科技(深圳)有限公司 | A kind of method for lifting the solid storage device life-span |
-
2019
- 2019-11-20 CN CN201911144275.1A patent/CN111090392A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7155467B1 (en) * | 2000-04-07 | 2006-12-26 | Hewlett-Packard Development Company, L.P. | Adaptive type-partitioned garbage collection |
| CN102902628A (en) * | 2012-09-18 | 2013-01-30 | 记忆科技(深圳)有限公司 | Flash memory and method and system for automatically separating cold and hot data based on same |
| US20150347029A1 (en) * | 2014-05-30 | 2015-12-03 | Sandisk Enterprise Ip Llc | Identification of Hot Regions to Enhance Performance and Endurance of a Non-Volatile Storage Device |
| CN107562378A (en) * | 2017-08-28 | 2018-01-09 | 记忆科技(深圳)有限公司 | A kind of method for lifting the solid storage device life-span |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111930848A (en) * | 2020-09-17 | 2020-11-13 | 阿里云计算有限公司 | Data partition storage method, device and system |
| CN112379842A (en) * | 2020-11-18 | 2021-02-19 | 深圳安捷丽新技术有限公司 | Method and device for predicting cold and hot properties of data |
| CN113094004A (en) * | 2021-05-13 | 2021-07-09 | 深圳市得一微电子有限责任公司 | Mirror image mapping optimization processing method for specific regular data in storage equipment |
| CN113094004B (en) * | 2021-05-13 | 2023-11-03 | 得一微电子股份有限公司 | Data mirror image mapping optimization processing method aiming at specific rule in storage equipment |
| CN113900597A (en) * | 2021-11-30 | 2022-01-07 | 深圳市安信达存储技术有限公司 | Data storage method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111090392A (en) | Cold and hot data separation method based on feature codes | |
| US9747202B1 (en) | Storage module and method for identifying hot and cold data | |
| CN102511044B (en) | Method for deleting the data and device thereof | |
| US8028121B2 (en) | Method and apparatus for detecting static data area, wear-leveling, and merging data units in nonvolatile data storage device | |
| US9582416B2 (en) | Data erasing method, memory control circuit unit and memory storage apparatus | |
| EP2115595B1 (en) | Memory device performance enhancement through pre-erase mechanism | |
| CN106874213B (en) | Solid state disk hot data identification method fusing multiple machine learning algorithms | |
| CN109558075B (en) | Method and device for storing data by using data cold and hot attributes | |
| WO2011095516A1 (en) | Method and system for mass storage on flash memory | |
| KR20130075018A (en) | Data update apparatus for flash memory file system and method thereof | |
| CN115129262A (en) | Solid state disk garbage recycling method, device, equipment and medium | |
| US8954646B2 (en) | Method for managing a plurality of blocks of a flash memory, and associated memory device and controller thereof | |
| CN108733577A (en) | Storage management method, memorizer control circuit unit and memory storage apparatus | |
| CN115114180A (en) | Method and related controller for performing wear leveling operation in flash memory and storage system | |
| CN105988950A (en) | Memory management method, memory control circuit unit and memory storage device | |
| CN112347001B (en) | Verification method and device for flash memory garbage collection and electronic equipment | |
| CN113282521A (en) | Method and device for prolonging service life of smart card | |
| CN101894077B (en) | Data storage method and system | |
| CN103902468A (en) | Data elimination method and device for storage system | |
| CN113010091B (en) | Method for writing data into solid state disk, method and device for recycling garbage | |
| CN107122124B (en) | Data processing method and device | |
| CN110058938B (en) | Memory processing method and device, electronic equipment and readable medium | |
| CN106021124B (en) | A kind of storage method and storage system of data | |
| CN110308876A (en) | Storage management method, memory storage apparatus and memorizer control circuit unit | |
| CN111949569A (en) | Storage device and control method and control device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000 09-2, 10-11 unit, 6 building, Changhong science and technology building, 18 South Road, science and Technology Park, Nanshan District, Shenzhen, Guangdong, China 18 Applicant after: Deyi Microelectronics Co.,Ltd. Address before: 518000 09-2, 10-11 unit, 6 building, Changhong science and technology building, 18 South Road, science and Technology Park, Nanshan District, Shenzhen, Guangdong, China 18 Applicant before: YEESTOR MICROELECTRONICS Co.,Ltd. |