CN117389484B - Data storage processing method, device, equipment and storage medium - Google Patents

Data storage processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN117389484B
CN117389484B CN202311696475.4A CN202311696475A CN117389484B CN 117389484 B CN117389484 B CN 117389484B CN 202311696475 A CN202311696475 A CN 202311696475A CN 117389484 B CN117389484 B CN 117389484B
Authority
CN
China
Prior art keywords
data
data block
address range
value
address
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311696475.4A
Other languages
Chinese (zh)
Other versions
CN117389484A (en
Inventor
翁大平
周辉
孙兴
余威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Dapu Microelectronics Co Ltd
Original Assignee
Shenzhen Dapu Microelectronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Dapu Microelectronics Co Ltd filed Critical Shenzhen Dapu Microelectronics Co Ltd
Priority to CN202311696475.4A priority Critical patent/CN117389484B/en
Publication of CN117389484A publication Critical patent/CN117389484A/en
Application granted granted Critical
Publication of CN117389484B publication Critical patent/CN117389484B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of flash memory error correction, and discloses a data storage processing method, a device, equipment and a storage medium. When data written in a first data area of a memory space is brushed down, a data block address range of the data is written in a scanning chain, then the scanning range is split according to the largest data block, the split address range is scanned to judge whether the data block exists, and if the data block exists, the value of a valid data bit identifier corresponding to all the data blocks in the address range of the current scanning is updated according to the address range corresponding to each data block; based on the value of the valid data bit identification, it is possible to determine whether a data hole exists in the currently scanned address range and the location of the data hole. According to the method and the device, the positions of the data holes in the data blocks to be flushed are identified for processing the data holes, so that the data written in the first data area can be flushed to the flash memory successfully according with the ZNS partition sequential writing requirement.

Description

Data storage processing method, device, equipment and storage medium
Technical Field
The present invention relates to the field of flash memory error correction technologies, and in particular, to a data storage processing method, apparatus, device, and storage medium.
Background
Partition namespaces solid state drives (Zone Namespaces Solid STATE DISK, ZNS SSDs) divide namespaces into partitions (zones), each of which is linearly distributed in the namespaces. Each partition can be read in any order, but must be written in order to ensure that each logical block address (Logical Block Address, LBA) has data within the data range after the flush, and that logical block addresses without data are not allowed to appear. Therefore, the existing write data based on the ZNS technology does not exist in the situation of data holes.
The method of writing data by adopting ZNS technology can reduce the write amplification and read-write delay in the solid state disk to a certain extent, but is not flexible enough. To enhance the data writing flexibility, techniques supporting Random writing of data, such as Zone Random WRITE AREA, ZRWA (Zone Random WRITE AREA, ZRWA) techniques, have been developed. The ZRWA protocol allows the user to randomly overwrite the ZRWA area formed after the ZNS partition enables ZRWA function, which promotes the flexibility of data writing, but the writing characteristic may cause a situation of data holes when data is written in the ZRWA area because the data is not written in the strict sequence, so that the data written in the ZRWA area cannot meet the requirement of strict sequence writing of the ZNS partition.
Disclosure of Invention
The invention mainly aims to solve the technical problem that data written in certain areas of a memory space can have data holes, so that the data cannot be flushed down to a flash memory.
In order to solve the above technical problem, a first aspect of the present invention provides a data storage processing method, including:
when the data written in the first data area of the memory space is brushed down, a first address range corresponding to a data block of the written data on the scanning chain table is scanned;
Judging whether the first address range is larger than a reference address range corresponding to the maximum data block;
if the first address range is larger than the reference address range, splitting the first address range into a plurality of second address ranges according to the reference address range;
Scanning each second address range in turn and judging whether a data block exists or not;
If the data blocks exist in the second address range which is currently scanned, updating the values of the valid data bit identifications corresponding to all the data blocks in the second address range according to a third address range corresponding to each data block, wherein the third address range is the address range corresponding to the data block;
and determining the position of the data hole in the second address range of the current scanning according to the value of the valid data bit identifier.
In a first implementation manner of the first aspect of the present invention, after the scanning each of the second address ranges in turn and determining whether a data block exists, the method further includes:
if no data block exists in the second address range of the current scanning, determining that all the second address ranges of the current scanning are data holes, and updating the value of the valid data bit identification corresponding to the second address range of the current scanning.
In a second implementation manner of the first aspect of the present invention, the sequentially scanning and determining whether a data block exists in each of the second address ranges includes:
scanning data blocks in the second address range in sequence;
if the tag value of the data block exists in the second address range, determining that the data block does not exist in the second address range;
and if the tag value of the data block does not exist in the second address range, determining that the data block exists in the second address range.
In a third implementation manner of the first aspect of the present invention, the valid data bit identifier is composed of a plurality of bits, each bit corresponds to one logical partition in the address range of the data block, and a value of each bit represents that valid data exists or not in the logical partition in the address range of the corresponding data block.
In a fourth implementation manner of the first aspect of the present invention, the data storage processing method further includes:
when the data written in the first data area is brushed, acquiring a value of a valid data bit identifier corresponding to a data block to be brushed;
Supplementing 0 to address spaces corresponding to positions of all data holes of the data block to be brushed according to a value of a valid data bit identifier corresponding to the data block to be brushed, so as to obtain a new data block of the first data area;
And updating the value of the valid data bit identifier corresponding to the new data block, and hanging the new data block into the linked list.
In a fifth implementation manner of the first aspect of the present invention, the data area processing method further includes:
When a power-down event is detected, storing the values of the valid data bit identifications of all the non-aligned data blocks of the first data area into a nonvolatile readable storage medium;
Determining a logic partition corresponding to the non-aligned data block, in which data is not written, according to the value of the valid data bit identifier, and supplementing 0 to a memory address corresponding to the logic partition not written with data and updating the value of the valid data bit identifier corresponding to the non-aligned data block;
When a power-on event is detected, judging whether the current recovered data block is a non-aligned data block before power failure;
and if the currently recovered data block is a non-aligned data block before power failure, updating the value of the valid data bit identifier corresponding to the data block to the value corresponding to the data block before power failure.
In a sixth implementation manner of the first aspect of the present invention, a one-dimensional array is used to record a partition ID of a partition where each first data area is located, and a two-dimensional array is used to record a value of a valid data bit identifier before all non-aligned data blocks in each first data area are powered down.
A second aspect of the present invention provides a data storage processing apparatus comprising:
The first scanning module is used for scanning a first address range corresponding to a data block of the data written in the linked list when the data written in the first data area of the memory space is brushed down;
the judging module is used for judging whether the first address range is larger than a reference address range corresponding to the maximum data block;
The splitting module is used for splitting the first address range into a plurality of second address ranges according to the reference address range if the first address range is larger than the reference address range;
the second scanning module is used for scanning each second address range in turn and judging whether a data block exists or not;
the identification module is used for updating the values of the effective data bit identifications corresponding to all the data blocks in the second address range according to the third address range corresponding to each data block if the data block exists in the second address range of the current scanning; and determining the position of a data hole in a second address range of the current scanning according to the value of the valid data bit identifier, wherein the third address range is the address range corresponding to the data block.
In a first implementation manner of the second aspect of the present invention, the identification module is further configured to:
if no data block exists in the second address range of the current scanning, determining that all the second address ranges of the current scanning are data holes, and updating the value of the valid data bit identification corresponding to the second address range of the current scanning.
In a second implementation manner of the second aspect of the present invention, the scanning module is specifically configured to:
scanning data blocks in the second address range in sequence;
if the tag value of the data block exists in the second address range, determining that the data block does not exist in the second address range;
and if the tag value of the data block does not exist in the second address range, determining that the data block exists in the second address range.
In a third implementation manner of the second aspect of the present invention, the valid data bit identifier is composed of a plurality of bits, each bit corresponds to one logical partition in the address range of the data block, and a value of each bit represents that valid data exists or not in the logical partition in the address range of the corresponding data block.
In a fourth implementation manner of the second aspect of the present invention, the data storage processing apparatus further includes:
The first zero padding module is used for acquiring a value of a valid data bit identifier corresponding to a data block to be swiped when the data written in the first data area is swiped; supplementing 0 to address spaces corresponding to positions of all data holes of the data block to be brushed according to a value of a valid data bit identifier corresponding to the data block to be brushed, so as to obtain a new data block of the first data area; and updating the value of the valid data bit identifier corresponding to the new data block, and hanging the new data block into the linked list.
In a fifth implementation manner of the second aspect of the present invention, the data storage processing apparatus further includes:
The storage module is used for storing the values of the valid data bit identifiers of all the non-aligned data blocks in the first data area into a nonvolatile readable storage medium when a power-down event is detected;
The second zero padding module is used for determining a logic partition corresponding to the non-aligned data block in which data is not written according to the value of the valid data bit identifier, and adding 0 to a memory address corresponding to the logic partition in which the data is not written and updating the value of the valid data bit identifier corresponding to the non-aligned data block;
the judging module is further used for: when a power-on event is detected, judging whether the current recovered data block is a non-aligned data block before power failure;
And the recovery module is used for updating the value of the valid data bit identifier corresponding to the data block into the value corresponding to the data block before power failure if the current recovered data block is a non-aligned data block before power failure.
In a sixth implementation manner of the second aspect of the present invention, a one-dimensional array is used to record a partition ID of a partition where each first data area is located, and a two-dimensional array is used to record a value of a valid data bit identifier before all non-aligned data blocks in each first data area are powered down.
A third aspect of the present invention provides a computer apparatus comprising: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the computer device to perform the data storage processing method of any of the first aspects.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon instructions which, when executed by a processor, implement the data storage processing method of any of the first aspects.
Different from the situation of the related art, the method, the device, the equipment and the storage medium for data storage provided by the embodiment of the application introduce the effective data bit identifier for identifying the logical partition where data is written and the logical partition where data is not written in the address range of each data block, and when the data written in the first data area of the memory space is to be flushed, in order to avoid the occurrence of data holes, the positions of the data holes in the data blocks to be flushed need to be identified first. The application scans the address range corresponding to the data block of the data written in the linked list; if the scanning range exceeds the address range represented by the maximum data block, splitting the scanning range according to the maximum data block, and then scanning the split address range to judge whether the data block exists. If the data blocks exist in the address range of the current scanning, updating the values of the valid data bit identifications corresponding to all the data blocks in the address range of the current scanning according to the address range corresponding to each data block; and finally, determining the position of a data hole in the current scanning address range according to the updated value of the valid data bit identifier, facilitating the subsequent processing of the data hole, and further ensuring that the data written in the first data area meets the requirement of ZNS partition sequential writing and can be successfully flushed into the flash memory.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to scale, unless expressly stated otherwise.
FIG. 1 is a schematic diagram of an embodiment of a data storage processing method according to an embodiment of the present invention;
FIG. 2 is a diagram of an embodiment of a bitmap layout according to an embodiment of the present invention;
FIG. 3 is a diagram of another embodiment of a bitmap layout according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another embodiment of a data storage processing method according to an embodiment of the present invention;
FIG. 5 is a diagram of one embodiment of a one-dimensional array in accordance with an embodiment of the present invention;
FIG. 6 is a schematic diagram of one embodiment of a two-dimensional array according to an embodiment of the present invention;
FIG. 7 is a diagram of one embodiment of a bitmap for finding unaligned blocks of data in an implementation of the present invention;
FIG. 8 is a schematic diagram of an embodiment of a data storage processing apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of one embodiment of a computer device in an embodiment of the invention.
Detailed Description
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
Each partition of the partition naming space solid state disk can be read in any order, but must be written in order, so that the write data based on the ZNS technology is free of data holes. Although the method can reduce the write amplification and the read-write delay in the solid state disk to a certain extent, the method is not flexible enough. The partition random writing area technology can solve the problem of inflexibility in writing data. The ZRWA area formed after the ZNS partition enables ZRWA functions can be randomly overwritten, and this way improves the flexibility of data writing, but the writing characteristic may cause a situation that a data hole occurs when data is written in the ZRWA area because data is not written in a strict sequence, so that the data written in the ZRWA area cannot meet the strict sequence writing requirement of the ZNS partition and cannot be flushed down the flash memory. In order to avoid that data written in a ZRWA area cannot be normally flushed into a flash memory, the invention provides a data block processing method of a ZRWA area, which is characterized in that the data written in a ZRWA area can be normally flushed into the flash memory by identifying the position where a data hole exists and filling (supplementing 0) data when the data is written in a ZRWA area, thereby realizing the purpose that the ZNS sequential writing requirement is satisfied in form.
In order to facilitate understanding of the present invention, technical terms related to the embodiments are described below.
SSD, solid STATE DISK, solid state drive.
The Zone refers to a logical partition, and refers to dividing the solid state disk into a plurality of independent logical areas;
the abbreviation of ZNS, zoned Namespaces, partition namespace, a new feature of Nvme protocol based on Zone size management space, each logical partition can be read in any order, but must be written in order;
The LBA, logical Block Address is abbreviated as logical block address, which describes the general mechanism of the block where data is located on a computer storage device, and may refer to the address of a certain data block or the data block to which a certain address points;
ZRWA, zone Random WRITE AREA, a partitioned Random write area, allowing Random writing and in-place overlay of data in SSD cache;
A data block, in an area formed after writing data in the logical partition, one Zone may contain one or more data blocks, and one data block may contain one or more LBAs;
Aligned and unaligned data blocks are two types of data blocks, with respect to data blocks that are aligned with 4K, the difference is whether they are written according to the rule of 4K sectors, i.e. 4096 bytes per sector. If the data block range is an integer multiple of 4K then it is a 4K aligned data block, otherwise it is a non-4K aligned data block, and it is determined whether to generate a 4K aligned data block or a non-4K aligned data block, specifically by writing the range.
FTL, flash Translation Layer, a flash translation layer, for mapping host logical address space to flash physical address space. The SSD records the mapping relationship of the logical address to the physical address each time the user logical data is written into the flash memory address space. When the host wants to read the data, the SSD reads the data from flash memory according to the mapping and returns the data to the user.
Referring to fig. 1, an embodiment of the present invention provides a data storage processing method, where the data storage processing method includes:
101. When the data written in the first data area of the memory space is brushed down, a first address range corresponding to a data block of the written data on the scanning chain table is scanned;
because the data written in the first data area (such as ZRWA area) of the memory space may have a data hole, the embodiment needs to identify the data block with the data hole first and identify the position of the data hole in the data block, so as to ensure that the data is not subjected to hole generation when being flushed down to meet the ZNS sequential writing requirement.
For convenience of description, the following description specifically uses the first data area as ZRWA area and uses the linked list as hash linked list, where it should be noted that, the first data area of the present application may be other memory space data areas where data holes exist, and the linked list may also be other linked lists that may be used to store addresses of data blocks.
In this embodiment, in order to facilitate identifying and recording a data hole of a data block, the data block of the first data area (such as ZRWA area) that identifies different address ranges needs to be hung on a linked list (such as a hash linked list) according to address distribution. The hash chain table needs to be updated correspondingly every time data is written in ZRWA areas. The address of the hash chain table refers to LMA, that is Logic Media Unit Address, the logical media unit address, the address takes 4K as a unit, the address distribution refers to LMA address distribution, the data block address in the hash chain table is in LMA format, according to the SSD in 4K format converted by the LBA, lma=lba, and SSD in 512 format, lma=8lba, it should be noted that only the data block of the written data will be hung in the chain table. The hash chain table is used for storing the data block addresses, a plurality of data blocks can be scattered in a hash mode, the data blocks with the same hash result are hung on the same chain table, and the problem that the query efficiency is low due to the fact that the number of data block nodes on the chain table is too large can be avoided.
In this embodiment, a scan range in the hash chain table needs to be determined first, specifically, when the data written in the ZRWA area is swiped down, a first address range corresponding to the ZRWA area data block to be scanned on the hash chain is determined according to the implicit or explicit swipe range, where the first address range is the address range that needs to be scanned for the currently swiped data.
102. Judging whether the first address range is larger than a reference address range corresponding to the maximum data block;
103. If the first address range is larger than the reference address range, splitting the first address range into a plurality of second address ranges according to the reference address range;
In this embodiment, the effective data bit identification mode is identified by a bitmap with a certain fixed bit number, and the range of the data block identified by the bitmap is relatively small, so that in order to match the data block scanning mode with the effective data bit identification mode, and in order to facilitate the data hole position identification, the scanning range needs to be split, and then the hole position identification is performed on the split data blocks. Namely, the method is equivalent to splitting a ZRWA-area data block with a large address range into a plurality of data blocks with small address ranges, and indirectly obtaining the data hole positions in the ZRWA-area data block by identifying the data hole positions in each split data block.
Because the data block structure is organized according to the maximum 128K, namely the address range represented by the maximum data block is 128K, the address range marked by the maximum data block is taken as a splitting reference, and the first address range to be scanned is split to obtain a plurality of second address ranges. The first address range is split with 128K as a reference address range if the first address range exceeds the address range represented by the largest data block, and if the first address range is smaller than the address range represented by the largest data block, the splitting is not required. It should be noted that, the scan range may be an integer multiple of 128K, or may not be an integer multiple of 128K, for example, the scan range is 300K, and may be split into two 128K data blocks and one 44K data block. In addition, the data block management is strictly processed according to the address range, that is, the split data block needs to meet the management requirement of the SSD on the data block. For example, taking a disk in 4K format as an example, the address range corresponding to the data block ranges LBA0-63 is 256K, and splitting can only be performed according to the data blocks of LBA 0-31 and the data blocks of LBA 32-63.
104. Scanning each second address range in turn and judging whether a data block exists or not;
in this embodiment, if the data block address range is split, each split second address range is scanned, and if the data block address range is not split, the first address range is scanned. Since the scanning method and the data hole identification method corresponding to the two cases are the same, the embodiment only describes the case of splitting the address range of the data block.
In one embodiment, the determination of whether a block of data exists in the second address range is made by:
scanning data blocks in the second address range in sequence;
if the tag value of the data block exists in the second address range, determining that the data block does not exist in the second address range;
and if the tag value of the data block does not exist in the second address range, determining that the data block exists in the second address range.
In this embodiment, the scanning object is an address corresponding to a data block on the hash chain table, and the address of the data block corresponding to the written LBA is all suspended on the chain table. If there is no data block in a certain address range on the linked list, the linked list in the address range is empty, and the tag value corresponding to the read data block will be a special value: 0xFFFFUL, which indicates that this address range belongs to ZRWA area and is not written with data, the address space represented by all LBAs of this address range is a hole location. If there is a data block in a certain address range on the linked list, the tag value corresponding to the read data block is a non-special value of 0xFFFFUL, which means that there is a data block in this range, a plurality of data blocks exist in the form of linked list, the address space represented by the scanned LBA in this address range is written with data, and the address space represented by the non-scanned LBA is not written with data, i.e. a data hole.
105. If the data blocks exist in the second address range which is currently scanned, updating the values of the valid data bit identifications corresponding to all the data blocks in the second address range according to a third address range corresponding to each data block, wherein the third address range is the address range corresponding to the data block;
In this embodiment, if a data block exists in a certain second address range, each data block in the second address range needs to be scanned further, and only the data hole position in each data block is identified, the data hole position in the data block in the whole ZRWA area can be obtained. And updating the values of the valid data bit identifications corresponding to all the data blocks in the second address range according to the third address range corresponding to each data block, wherein the values of the valid data bit identifications are used for identifying whether the hole data and the hole data positions exist in the second address range. And if no data block exists in a certain second address range, determining that all the second address ranges currently scanned are data holes.
In an embodiment, in order to facilitate identifying the hole location in the address range of the data block, a valid data bit identifier bitmap is introduced to manage the data block, that is, all the data blocks for writing data can be represented by using the valid data bit identifier. Considering that there may be data holes in both aligned and non-aligned data blocks, two different variable correspondences may be used for identification purposes. For example, a data hole is identified by variable fvalidbmp for aligned data blocks and uaBmp for non-aligned data blocks.
The valid data bit identifier is composed of a plurality of bits, each bit corresponds to a logical partition LBA in the address range of the data block, and it should be noted that the bit arrangement sequence of the bitmap is consistent with the arrangement sequence of the LBAs in the address range of the corresponding data block. For example, bits 1-8 of a bitmap (arranged from right to left) correspond to logical partition LBAs 0-LBA7 of a data block, respectively, and bits 1-8 of another bitmap correspond to logical partition LBAs 8-LBAs 15 of another data block, respectively.
The value of each bit represents that valid data exists or no valid data exists in the logical partition LBA in the address range of the corresponding data block, for example, if a certain LBA exists written data, the bitmap bit corresponding to the LBA is set to 1, otherwise, 0 is set. The value of the valid data bit identification (i.e., the value of the bitmap) is represented in hexadecimal. Valid data specifically refers to data written in the LBA based on a write command.
The bit number of the valid data bit identification is related to the partition format of the ZNS solid state disk. For example, for a 4K format disk, each LBA represents a 4K size, if each data block can be managed with a U32 type parameter bitmap, such as using variables fvalidbmp, and each bit represents 4K, then fvalidbmp represents an address range of 128KB for the largest data block that can be identified; of course, management can be realized through U8 or U16, but for 4K format disk integer U32, the number of data blocks on the linked list can be reduced by U32, and the efficiency of batch processing is increased. While for a 512 format disk, each LBA represents a 512B size, if each data block can be managed with a U8 type bitmap, such as using variable uaBmp, each bit represents 512B, uaBmp corresponds to an address range of 4KB for the largest data block that can be identified.
Because the size of the range corresponding to the address unit (one LMA represents 4K) adopted by the hash chain table is the same as the size of the range corresponding to one LBA of the 4K format disk, the 4K format disk generates all aligned data blocks; the disc in the 512 format may be an aligned data block or a non-aligned data block, and is specifically determined according to the writing range of the data, if the writing range is an integer multiple of 4K (for example, 8 LBAs), the aligned data block is generated, and if the writing range is not an integer multiple of 4K, the non-aligned data block is generated.
For a disc in 4K format, assuming that the starting address of a certain aligned data block is 0 and the length is 32, that is, the address range corresponding to the data block is LBA0-LBA31, where the address range where valid data exists is LBA0-LBA7 and LBA24-LBA31, the valid data bit identifier is expressed as using bitmap: bit0-bit7 and bit24-bit31 of the write data are set to 1, bit8-bit23 without the write data is set to 0, that is, the value of bitmap is expressed as 0xff0000ff in hexadecimal system, and the valid data bit identification corresponding to the data block is shown in fig. 2. Wherein the numerals represent the address ranges LBA0-LBA31, the dark portions LBA0-LBA7 and LBA24-LBA31 represent the written data.
To facilitate an understanding of ZRWA area partition random writing, the writing of corresponding data blocks of fig. 2 is illustrated below.
Taking a 4K format disc as an example, assuming that the address range of the first write data is LBA0-LBA31, a data block 1 is generated, the bit0-bit31 of the bitmap value corresponding to the data block 1 is set to 1, that is, 0xffffffff, the address range of the second write data is LBA8-LBA23 (not writing from LBA32, that is, partition random writing), a data block 2 is generated, the bit0-bit7 of the bitmap value corresponding to the data block 2 is set to 0, the bit8-bit23 is set to 1, the bit24-bit31 is set to 0, that is, 0x00ffff00, and because the writing range of the data block 2 and the writing range of the data block 1 overlap, the address range 8-23 of the data block 1 is set to invalid (that is, an invalid address) when the data block 2 is generated, the bitmap value corresponding to the data block 1 needs to be updated to 0xff0000ff, and the bitmap layout corresponding to the data block 1 is shown in fig. 2. It should be noted that, at this time, the bitmap value corresponding to the data block 1 is 0xff0000ff, which does not indicate that no write data exists between the LBAs 8 to 23, but the write data between the LBAs 8 to 23 is located in the data block 2, that is, each of the LBAs 0 to 31 has valid data, and no data hole exists.
For a disc in the 512 format, assuming that the starting address of a certain unaligned data block is 0 and the length is 8, that is, the address range corresponding to the data block is LBA0-LBA7, where the address range where valid data exists is LBA0-LBA1 and LBA5-LBA6, the valid data bit identifier is expressed as using bitmap: bit0-bit1 and bit5-bit6 of the write data are set to 1, bit2-4 and bit7 of the write data are set to 0, that is, the value of bitmap is expressed as 0x63 in hexadecimal system, and the valid data bit identification corresponding to the data block is shown in fig. 3.
106. And determining the position of the data hole in the second address range of the current scanning according to the value of the valid data bit identifier.
In this embodiment, the value of the valid identification bit calculated in step 105 is obtained by integrating LBAs of the written data in all the data blocks in the second address range, so that the location of the data hole in the second address range currently scanned can be identified according to the value of the valid data bit identification. It should be noted that, by the valid data bit identifier, whether a data hole exists in the second address range currently scanned, that is, a position where no data hole exists in the second address range may also be identified.
The data block processing method of ZRWA area provided by the embodiment of the application can identify the data hole generated when the ZRWA area partitions randomly write data, and the application introduces the effective data bit mark for marking the logic partition where the data is written and the logic partition where the data is not written in the address range of each data block, and when the data written in the ZRWA area is brushed down, the position of the data hole in the data block to be brushed down needs to be identified first in order to avoid the occurrence of the data hole. The method comprises the steps of firstly determining an address range corresponding to a ZRWA area data block to be scanned on a hash chain; if the scanning range exceeds the address range represented by the maximum data block, splitting the scanning range according to the maximum data block, and then scanning the split address range to judge whether the data block exists. If the data blocks exist in the address range of the current scanning, updating the values of the valid data bit identifications corresponding to all the data blocks in the address range of the current scanning according to the address range corresponding to each data block; and finally, determining the position of a data hole in the current scanning address range according to the updated value of the valid data bit identifier, facilitating the subsequent processing of the data hole, and further ensuring that the data written in the ZRWA area meets the requirement of ZNS partition sequential writing and can be successfully flushed into the flash memory.
For a better understanding of the solution according to the present invention, the specific implementation of steps 101-106 is further illustrated based on the above description of the valid data bit identification.
(1) Assuming a disk in 4K format, the scan range of the hash chain is first determined, for example, the address range of the scanned data block is LBA0-LBA63 (corresponding to the first address range), which is 256K, greater than the reference address range 128K, and thus needs to be split into two address ranges: the first 128K corresponds to LBA0-LBA31 (corresponding to a second address range), and the second 128K corresponds to LBA32-LBA63 (corresponding to another second address range); then, firstly scanning whether a data block exists in the first 128K, and if the aligned data block 1 is firstly scanned, and the address range corresponding to the aligned data block 1 is LBA0-LBA7 and LBA24-LBA31 (which are equivalent to a third address range) has valid data, updating the value of the valid data bit identification bitmap of all the currently scanned data blocks (the aligned data block 1) to 0xff0000ff, and then continuing to scan other data blocks in the first 128K range;
If the next scan to another aligned data block 2 is performed, the address range LBA8-LBA23 (corresponding to another third address range) corresponding to the aligned data block 2 has valid data, the value of the valid data bit identification bitmap of all the currently scanned data blocks (aligned data block 1+aligned data block 2) is updated to 0xffffffff, and when all the data blocks in the first 128K address range are scanned, whether a data hole exists in the first 128K address range and the position where the data hole exists can be determined according to the value of the current valid data bit identification bitmap. Since the latest value of the valid data bit identification bitmap corresponding to all data blocks in the first 128K is 0 xffffffffff, the address range corresponding to the first 128K is as follows: LBA0-LBA31, if 0xffffffff is converted into binary, each binary bit is 1 and corresponds to LBA0-LBA31 respectively, that is, the valid data bit mark 0xffffff represents that each LBA of LBA0-LBA31 has write data, that is, the first 128K has no data hole;
If no other data block representing the address range LBA8-LBA23 is scanned subsequently, the first 128K is represented as a data hole, the corresponding data hole position is LBA8-LBA23, and the value of the valid data bit identification bitmap is 0xff0000ff; after the first 128K is scanned, the second 128K is scanned again, and the specific processing manner is the same as that of the first 128K, and if the data block label read during the second 128K scanning is a special value of 0xFFFFUL, it indicates that no data block exists in the address range of the second 128K, and all the address ranges of the second 128K are data holes.
(2) Assuming that a disk in 512 format is used, a scan range of the hash table is first determined, for example, the scan range is LBA0-LABA7 (corresponding to the first address range), the address range is 4K, which is smaller than the reference address range 128K, so that address range splitting is not required, and LBA0-LABA7 belongs to the first 128K range, whether a data block exists in the first 128K range is scanned first, assuming that a non-aligned data block 1 is scanned, the address range corresponding to the non-aligned data block 1 is LBA0-LBA1 and LBA5-LBA6 has valid data, and no other data block exists in the first 128K address range, the value of the valid data bit identifier corresponding to all the data blocks in the first 128K address range is the bitmap value of the current non-aligned data block 1: and 0x63, and obtaining the bitmap value of the data hole position of the scanning range LBA0-LBA7 after inverting the bitmap value: 0x9C, i.e., scan ranges LBA2-LBA4 and LBA7 are data holes.
In an embodiment, in order to ensure that the data written in the first area meets the requirement of ZNS partition sequential writing, the data may be successfully written into the flash memory, and after the data hole position of the first area is identified, the data hole needs to be further processed, where a specific processing manner is as follows:
when the data written in the first data area is brushed, acquiring a value of a valid data bit identifier corresponding to a data block to be brushed;
Supplementing 0 to address spaces corresponding to positions of all data holes of the data block to be brushed according to a value of a valid data bit identifier corresponding to the data block to be brushed, so as to obtain a new data block of the first data area;
And updating the value of the valid data bit identifier corresponding to the new data block, and hanging the new data block into the linked list.
In this embodiment, when the data written in the first area is to be flushed, the value of the valid data bit identifier corresponding to the data block to be flushed is obtained first, and based on the value, the data hole position in the address range corresponding to the data block to be flushed can be determined.
For example, through steps 101-106 in the above embodiment, a value of the valid data bit identifier corresponding to the data block to be flushed may be obtained, and assuming that the value is 0xff0000ff, if the value is converted into a binary representation, a bit of 1 indicates that valid data exists for the corresponding LBA, and a bit of 0 indicates that the corresponding LBA is a data hole, and assuming that the address range of the data block to be flushed is LAB0-LBA31, a data hole exists for the LBA8-LBA23 may be identified by 0xff0000 ff.
In order to avoid that data holes exist in the data blocks to be brushed, address spaces corresponding to positions of all the data holes of the data blocks to be brushed need to be supplemented with 0 in advance, so that new data blocks of the first area are obtained, the new data blocks also need to be hung into a hash chain table, and meanwhile, the values of valid data bit identifiers of the corresponding updated data blocks are as follows: 0x00fffff00.
In this embodiment, after identifying the positions of all the data holes in the address range of the data block to be flushed, 0 is further added to the data holes, so that the data holes in the address range of the data block to be flushed become formally non-data holes, and the data written in the first area is ensured to meet the requirement of ZNS partition sequential writing and can be flushed into the flash memory successfully.
The random write data characteristic of ZRWA area partitions causes data holes, which not only affects the data to be flushed down into the flash memory, but also generates unaligned data blocks, thereby affecting the data recovery in the power-down scenario. Under the power failure scene, whether normal power failure or abnormal power failure occurs, the data in the ZRWA area needs to be all sent to the FTL module in an integral multiple of 4K before power failure so as to be recovered after power failure, and because the minimum unit of the FTL management data is LMA, namely the minimum unit is 4K, the non-aligned data block can be mistakenly regarded as the aligned data block by the FTL module before being sent to the FTL. Therefore, for aligned data blocks, correct recovery of data can be ensured, but for non-aligned data blocks, since the non-aligned data blocks before power failure can be mistakenly regarded as aligned data blocks by the FTL module, the non-aligned data blocks can not be ensured to be correctly recovered after power-up.
Aiming at the problem that the unaligned data block cannot be recovered correctly under the power-off field, the invention saves the bitmap of the unaligned data block in advance before power-off, and simultaneously complements the unaligned data block into an aligned data block and then sends the aligned data block to the FTL module, and the unaligned data block can be recovered correctly based on the prestored bitmap after power-on.
Referring to fig. 4, fig. 4 is another embodiment of the data block storage processing method according to the present invention, where the data storage processing method further includes:
201. When a power-down event is detected, storing the values of the valid data bit identifications of all the non-aligned data blocks of the first data area into a nonvolatile readable storage medium;
in this embodiment, a power-down event is detected in real time, and in view of the problem that the power-down and unaligned data blocks cannot be recovered correctly, when the power-down event is detected, the values of the latest valid data bit identifications of all unaligned data blocks in the first data area (such as the ZRWA area) need to be saved to a nonvolatile readable storage medium (such as NAND FLASH) so as to avoid power-down loss.
It should be noted that, normally, the electronic device has enough time to perform some transaction, such as saving the buffered data. Abnormal power loss typically also takes several tens of ms to do some transactions, such as writing all non-aligned data blocks of the ZRWA area to a non-volatile readable storage medium at power loss and sending ZRWA area data to the FTL module at power loss. Therefore, the power-down processing logic is not required to be executed in advance under the normal power-down or abnormal power-down conditions.
202. Determining a logic partition corresponding to the non-aligned data block, in which data is not written, according to the value of the valid data bit identifier, and supplementing 0 to a memory address corresponding to the logic partition not written with data and updating the value of the valid data bit identifier corresponding to the non-aligned data block;
In this embodiment, before sending the unaligned data blocks to the FTL module, it is necessary to perform padding on each unaligned data block to become an aligned data block, and specifically, determine a logical partition included in the unaligned data block according to a value of a valid data bit identifier bitmap of the unaligned data block. For example, assuming that before the power-down event occurs, the bitmap value of a certain non-aligned data block a is 0x63, the logical partitions included in the non-aligned data block a are LBA0-LBA1 and LBA5-LBA6, and correspondingly, the logical partitions missing for the non-aligned data block a to be the aligned data block are LBA2-LBA4 and LBA7, if the memory addresses corresponding to LBA2-LBA4 and LBA7 are complemented with 0, the bitmap value of the non-aligned data block a will be updated to 0xff, that is, the transition from the non-aligned data block to the aligned data block is completed.
203. When a power-on event is detected, judging whether the current recovered data block is a non-aligned data block before power failure;
The embodiment also detects a power-up event in real time, and because the unaligned data block in the ZRWA area is already converted into an aligned data block when a power-down event occurs, when the data block is recovered, the data block converted from the unaligned data block into the aligned data block when power-down needs to be recovered again into the unaligned data block before power-down. Because all non-aligned data blocks sent to the FTL module are complemented, it cannot be determined whether the recovered data block was a non-aligned data block before the power down according to the data block size. It should be noted that, the 4K format disc does not have unaligned data blocks, and only the 512 format disc does have unaligned data blocks.
In one embodiment, step 203 specifically includes: and judging whether the current recovered data block is a non-aligned data block before power failure according to the valid data bit identifiers of all the pre-stored data blocks and the position information of each data block in the partition.
For a 512 format disk, the valid data bit identification bitmap values corresponding to the aligned data blocks are all the same, specifically 0xff, and all other values except the value 0xff correspond to non-aligned data blocks. And when a power failure event occurs, saving bitmaps of all the data blocks and position information of each data block in different partition Zone into a nonvolatile readable storage medium. When power is on, the position information of each data block in the Zone is read, the corresponding bitmap value is found according to the position information, then whether the data block is 0xff is judged, if the data block is 0xff, the current recovered data block is determined to be an aligned data block, and if the data block is not 0xff, the current recovered data block is determined to be a non-aligned data block.
204. And if the currently recovered data block is a non-aligned data block before power failure, updating the value of the valid data bit identifier corresponding to the data block to the value corresponding to the data block before power failure.
In this embodiment, when determining that the currently recovered data block is a non-aligned data block before power failure, in order to ensure that the non-aligned data block can be recovered correctly, it is necessary to update the value of the valid data bit identifier corresponding to the non-aligned data block to the value corresponding to the data block before power failure, and the value of the valid data bit identifier corresponding to the data block before power failure may find the corresponding bitmap value according to the location information of the data block in the Zone.
It should be noted that, in step 202, the memory address corresponding to the logical partition to which the data is not written in the non-aligned data block is supplemented with 0, so that the transition from the non-aligned data block to the aligned data block is completed in a form, but in step 204, the memory address corresponding to the logical partition to which 0 is supplemented originally may be cleared, or the clearing process may not be performed, and the corresponding memory address may not be cleared, so that the subsequent read/write operation of the non-aligned data block may not be affected.
In one embodiment, to ensure that unaligned data blocks can be correctly recovered in the power down scenario, two types of data structures are used to manage unaligned data blocks.
(1) And the linked list is used for recording index information of all the unaligned data blocks. In order to facilitate the rapid centralized finding of all non-aligned data blocks in the abnormal power-down situation, before power-down, each time a new non-aligned data block is generated in ZRWA areas, the index information of the non-aligned data block is recorded in a linked list. Preferably, the index information is a subscript of an array of the save data blocks.
(2) And the array is used for recording bitmaps of the unaligned data blocks. When there are multiple first data areas of the ZNS (such as ZRWA areas), in order to reduce the size of the array, only the partition ID of the partition where the writable ZRWA area is located is recorded by using a one-dimensional array, and the writable ZRWA area is generally 16. Recording the values of the effective data bit identifiers before power failure of all non-aligned data blocks in each ZRWA area by adopting a two-dimensional array; the first column of the two-dimensional array is used to record the subscript of the one-dimensional array and the first row of the two-dimensional array is used to record the offset of the ZRWA area within each partition relative to the write pointer. Generally, when writing data in the ZONE ZRWA area, each time the sequential writing is completed, a write pointer is used as a flag bit to record the address location of the logical block where the data has been written.
The one-dimensional array is used to record ZoneID that has a corresponding partition of ZRWA areas where unaligned blocks exist. The size of the one-dimensional array represents the maximum number of Zone that can be concurrently opened. As shown in fig. 5 and 7, a one-dimensional zone lookup table is formed, wherein the first data of one-dimensional data in the table is ZoneID, and the corresponding array subscript is 0; the third data is 13 and the corresponding array subscript is 2.
The two-dimensional array is used for recording bitmaps of the unaligned data blocks, and the first column of the two-dimensional array is used for recording subscripts of the one-dimensional array and for carrying out ZoneID indexes; the first row of the two-dimensional array is used to record the offset of the ZRWA area within each partition relative to the write pointer. As shown in fig. 6 and fig. 7, a two-dimensional bitmap lookup table is formed, for example, a bitmap in a two-dimensional array is 0x04, and according to a corresponding ZoneID index value 2, searching data with subscript 2 in the one-dimensional array is 13, and the address offset relative to the write pointer is 1, where the bitmap is 0x 04: a Zone of ZoneID is provided with a non-aligned data block having an address of write pointer + address offset 1 and a bitmap of 0x04. In this embodiment, all non-aligned data blocks in each ZRWA area need to be scanned before power-down, zoneID of the corresponding partition in each ZRWA area is saved in a one-dimensional array, address positions and corresponding bitmaps of each non-aligned data block in ZRWA area are written into a two-dimensional array, and the one-dimensional array and the two-dimensional array are saved in a storage medium which is not lost when power-down.
For a better understanding of the solution of the present invention, the following further illustrates the specific implementation of steps 201-204 based on the above description of the one-dimensional array and the two-dimensional array.
(1) Whenever a ZRWA area generates a new unaligned data block, the index information of the new unaligned data block is recorded in the linked list, for example, the bitmap of the unaligned data block a is 0x04, its subscript in the data block array (system auto-generated) is 3, and then the array subscript index=3 of the unaligned data block a is written in the linked list.
(2) When a power-down event is detected, writing ZoneID corresponding to each ZRWA area into a one-dimensional array; and writing the values of the valid data bit identifiers bitmaps of all the unaligned data blocks in each ZRWA area into a two-dimensional array, taking the subscript of the one-dimensional array as the index of the two-dimensional array, and distinguishing the bitmap values of different unaligned data blocks in the same ZRWA area according to the offset of the ZRWA area relative to the write pointer. And finally, storing the one-dimensional array and the two-dimensional array into a nonvolatile readable storage medium.
In addition, in order to ensure that the unaligned data can be correctly recovered during power-up, the type (aligned or unaligned) of the recovered data block needs to be determined, and when a power-down event occurs, bitmaps of all the data blocks and position information of each data block in different partition Zone are stored in a nonvolatile readable storage medium. Wherein the location information includes: zoneID where each data block is located, the LMA of each data block, and the write pointer for each data block.
(3) When power failure occurs, in order to meet the data block format requirement of the FTL module, the non-aligned data block a needs to be supplemented with 0 and then changed into an aligned data block, and specifically, the non-aligned data block is found out to be supplemented with 0 for the memory address according to the bit maps of all the pre-stored data blocks.
For example, before power failure, the bitmap of the unaligned data block a is 0x04, when power failure occurs, the memory address corresponding to the logical partition where the unaligned data block a is not written with data is first complemented with 0, and then the bitmap is updated to 0xff. It should be noted that, after the unaligned data block a is changed to an aligned data block, the data of the one-dimensional array and the two-dimensional array will not be modified.
(4) When a power-up event is detected, since all the data blocks are aligned data blocks, a judgment cannot be made according to the data block size. Firstly, the position information of each data block in different Zone zones is read, wherein the position information comprises ZoneID of each data block, LMA of each data block and corresponding write pointer WP of each data block, and then the one-dimensional array and the two-dimensional array are inquired based on the position information of each data block.
For example, zoneID of the data block a is 13, the offset of the non-aligned data block a in the Zone is LMA-wp=1, the zoneid queries the one-dimensional array, the index corresponding to ZoneID is 2, then the two-dimensional array is queried under the condition that the index 2+the write pointer offset 1 is a query condition, if the corresponding value exists in the two-dimensional array, the data block a is determined to be the non-aligned data block before the power is turned off (the two-dimensional array only stores the bitmap of the non-aligned data block). And if the corresponding value does not exist in the two-dimensional array, determining that the data block A is an aligned data block before power failure.
(5) If the current recovered data block is a non-aligned data block before power failure, directly taking the bitmap value obtained by inquiring the two-dimensional array as the bitmap value after power-on recovery of the data block. For example, the bitmap value before the power-down of the data block a is 0x04, the bitmap value after the power-down is updated to 0xff, and when the power-up is recovered again, the bitmap value is updated to 0x04 again.
The invention further discloses a data storage processing device corresponding to the method on the basis of the detailed description of the embodiments corresponding to the data storage processing method, and fig. 8 is a functional structure schematic diagram of the data storage processing device according to the embodiment of the invention. As shown in fig. 8, the data storage processing apparatus includes
A first scanning module 701, configured to determine a first address range corresponding to a data block of the data written on the linked list when the data written in the first data area of the memory space is to be flushed;
a judging module 702, configured to judge whether the first address range is greater than a reference address range corresponding to a maximum data block;
A splitting module 703, configured to split the first address range into a plurality of second address ranges according to the reference address range if the first address range is greater than the reference address range;
A second scanning module 704, configured to scan each of the second address ranges in turn and determine whether a data block exists;
The identification module 705 is configured to update, if a data block exists in a second address range currently scanned, values of valid data bit identifications corresponding to all data blocks in the second address range according to a third address range corresponding to each data block; and determining the position of a data hole in a second address range of the current scanning according to the value of the valid data bit identifier, wherein the third address range is the address range corresponding to the data block.
In an embodiment, the identification module 705 is further configured to:
if no data block exists in the second address range of the current scanning, determining that all the second address ranges of the current scanning are data holes, and updating the value of the valid data bit identification corresponding to the second address range of the current scanning.
In one embodiment, the second scanning module 704 is specifically configured to:
scanning data blocks in the second address range in sequence;
if the tag value of the data block exists in the second address range, determining that the data block does not exist in the second address range;
and if the tag value of the data block does not exist in the second address range, determining that the data block exists in the second address range.
In one embodiment, the valid data bit identifier is composed of a plurality of bits, each bit corresponds to a logical partition in the address range of the data block, and the value of each bit represents that valid data exists or is not present in the logical partition in the address range of the corresponding data block.
In an embodiment, the data block processing apparatus of the ZRWA area further includes:
a first zero padding module 706, configured to obtain a value of a valid data bit identifier corresponding to a data block to be swiped when swiped with the data written in the first data area; supplementing 0 to address spaces corresponding to positions of all data holes of the data block to be brushed according to a value of a valid data bit identifier corresponding to the data block to be brushed, so as to obtain a new data block of the first data area; and updating the value of the valid data bit identifier corresponding to the new data block, and hanging the new data block into the linked list.
In an embodiment, the data block processing apparatus of the ZRWA area further includes:
a saving module 707, configured to save, when a power-down event is detected, values of valid data bit identifiers of all non-aligned data blocks in the first data area to a non-volatile readable storage medium;
A second zero padding module 708, configured to determine a logical partition corresponding to the non-aligned data block in which data is not written according to the value of the valid data bit identifier, and add 0 to a memory address corresponding to the logical partition in which data is not written and update the value of the valid data bit identifier corresponding to the non-aligned data block;
The judging module 702 is further configured to: when a power-on event is detected, judging whether the current recovered data block is a non-aligned data block before power failure;
And a recovery module 709, configured to clear a memory address corresponding to a logical partition of the data block with 0 complementary according to a value of a valid data bit identifier of the data block if the current recovered data block is a non-aligned data block before power failure, and update a value of a valid data bit identifier corresponding to the data block.
In an embodiment, a one-dimensional array is used to record the partition ID of the partition where each first data area is located, and a two-dimensional array is used to record the value of the valid data bit identifier before all the unaligned data blocks in each first data area are powered down.
Since the embodiments of the device portion correspond to the embodiments of the method, the description of the data storage processing device provided by the present invention refers to the embodiments of the method, and the present invention is not repeated herein, and has the same advantages as the data storage processing method.
Fig. 8 above is a detailed description of the data storage processing apparatus in the embodiment of the present invention from the point of view of the modularized functional entity, and the detailed description of the computer device in the embodiment of the present invention from the point of view of the hardware processing is described below.
Fig. 9 is a schematic diagram of a computer device according to an embodiment of the present invention, where the computer device 500 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (e.g., one or more processors) and a memory 520, and one or more storage mediums 530 (e.g., one or more mass storage devices) storing application programs 533 or data 532. Wherein memory 520 and storage medium 530 may be transitory or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in the computer device 500. Still further, the processor 510 may be arranged to communicate with a storage medium 530 to execute a series of instruction operations in the storage medium 530 on the computer device 500.
The computer device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input/output interfaces 560, and/or one or more operating systems 531, such as Windows Serve, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the computer device structure shown in FIG. 9 is not limiting of the computer device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The present invention also provides a computer device including a memory and a processor, the memory storing computer readable instructions which, when executed by the processor, cause the processor to perform the steps of the data storage processing method in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, or may be a volatile computer readable storage medium, having stored therein instructions which, when executed on a computer, cause the computer to perform the steps of the data storage processing method.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (9)

1. A data storage processing method, the method comprising:
when the data written in the first data area of the memory space is brushed down, a first address range corresponding to a data block of the written data on the scanning chain table is scanned;
Judging whether the first address range is larger than a reference address range corresponding to the maximum data block;
if the first address range is larger than the reference address range, splitting the first address range into a plurality of second address ranges according to the reference address range;
Scanning each second address range in turn and judging whether a data block exists or not;
If the data blocks exist in the second address range which is currently scanned, updating the values of the valid data bit identifications corresponding to all the data blocks in the second address range according to a third address range corresponding to each data block, wherein the third address range is the address range corresponding to the data block;
And determining the position of a data hole in the second address range of the current scanning according to the value of the valid data bit identifier, wherein the valid data bit identifier consists of a plurality of bits, each bit corresponds to one logic partition in the address range of the data block, and the value of each bit represents that valid data exists or is not present in the logic partition in the address range of the corresponding data block.
2. The data storage processing method according to claim 1, further comprising, after said scanning each of said second address ranges in turn and determining whether a data block exists, the steps of:
And if the data block does not exist in the second address range of the current scanning, determining that all the second address ranges of the current scanning are data holes.
3. The data storage processing method of claim 1, wherein sequentially scanning and determining whether a data block exists in each of the second address ranges comprises:
scanning data blocks in the second address range in sequence;
if the tag value of the data block exists in the second address range, determining that the data block does not exist in the second address range;
and if the tag value of the data block does not exist in the second address range, determining that the data block exists in the second address range.
4. The data storage processing method according to claim 1, characterized in that the data storage processing method further comprises:
when the data written in the first data area is brushed, acquiring a value of a valid data bit identifier corresponding to a data block to be brushed;
Supplementing 0 to address spaces corresponding to positions of all data holes of the data block to be brushed according to a value of a valid data bit identifier corresponding to the data block to be brushed, so as to obtain a new data block of the first data area;
And updating the value of the valid data bit identifier corresponding to the new data block, and hanging the new data block into the linked list.
5. The data storage processing method according to claim 1, characterized in that the data storage processing method further comprises:
When a power-down event is detected, storing the values of the valid data bit identifications of all the non-aligned data blocks of the first data area into a nonvolatile readable storage medium;
Determining a logic partition corresponding to the non-aligned data block, in which data is not written, according to the value of the valid data bit identifier, and supplementing 0 to a memory address corresponding to the logic partition not written with data and updating the value of the valid data bit identifier corresponding to the non-aligned data block;
When a power-on event is detected, judging whether the current recovered data block is a non-aligned data block before power failure;
and if the currently recovered data block is a non-aligned data block before power failure, updating the value of the valid data bit identifier corresponding to the data block to the value corresponding to the data block before power failure.
6. The data storage processing method according to claim 5, wherein the partition ID of the partition where each of the first data areas is located is recorded using a one-dimensional array, and the value of the valid data bit identification before power-down of all the non-aligned data blocks of each of the first data areas is recorded using a two-dimensional array.
7. A data storage processing apparatus, characterized in that the data storage processing apparatus comprises:
The first scanning module is used for scanning a first address range corresponding to a data block of the data written in the linked list when the data written in the first data area of the memory space is brushed down;
the judging module is used for judging whether the first address range is larger than a reference address range corresponding to the maximum data block;
The splitting module is used for splitting the first address range into a plurality of second address ranges according to the reference address range if the first address range is larger than the reference address range;
the second scanning module is used for scanning each second address range in turn and judging whether a data block exists or not;
The identification module is used for updating the values of the effective data bit identifications corresponding to all the data blocks in the second address range according to the third address range corresponding to each data block if the data block exists in the second address range of the current scanning; and determining the position of a data hole in the second address range of the current scanning according to the value of the valid data bit identifier, wherein the third address range is the address range corresponding to the data block, the valid data bit identifier consists of a plurality of bits, each bit corresponds to one logic partition in the address range of the data block, and the value of each bit represents the existence or absence of valid data in the logic partition in the address range of the corresponding data block.
8. A computer device, the computer device comprising: a memory and at least one processor, the memory having instructions stored therein;
The at least one processor invokes the instructions in the memory to cause the computer device to perform the data storage processing method of any of claims 1-6.
9. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the data storage processing method of any of claims 1-6.
CN202311696475.4A 2023-12-12 2023-12-12 Data storage processing method, device, equipment and storage medium Active CN117389484B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311696475.4A CN117389484B (en) 2023-12-12 2023-12-12 Data storage processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311696475.4A CN117389484B (en) 2023-12-12 2023-12-12 Data storage processing method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN117389484A CN117389484A (en) 2024-01-12
CN117389484B true CN117389484B (en) 2024-04-26

Family

ID=89468826

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311696475.4A Active CN117389484B (en) 2023-12-12 2023-12-12 Data storage processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117389484B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102364474A (en) * 2011-11-17 2012-02-29 中国科学院计算技术研究所 Metadata storage system for cluster file system and metadata management method
CN104462549A (en) * 2014-12-25 2015-03-25 瑞斯康达科技发展股份有限公司 Data processing method and device
US9454990B1 (en) * 2015-03-30 2016-09-27 HGST Netherlands B.V. System and method of conducting in-place write operations in a shingled magnetic recording (SMR) drive
CN107947799A (en) * 2017-11-28 2018-04-20 郑州云海信息技术有限公司 A kind of data compression method and apparatus
CN111435288A (en) * 2019-01-14 2020-07-21 阿里巴巴集团控股有限公司 Data processing method and device
WO2022262381A1 (en) * 2021-06-16 2022-12-22 华为技术有限公司 Data compression method and apparatus
CN115629711A (en) * 2022-10-31 2023-01-20 济南浪潮数据技术有限公司 Packet writing alignment method, system, device and medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102364474A (en) * 2011-11-17 2012-02-29 中国科学院计算技术研究所 Metadata storage system for cluster file system and metadata management method
CN104462549A (en) * 2014-12-25 2015-03-25 瑞斯康达科技发展股份有限公司 Data processing method and device
US9454990B1 (en) * 2015-03-30 2016-09-27 HGST Netherlands B.V. System and method of conducting in-place write operations in a shingled magnetic recording (SMR) drive
CN107947799A (en) * 2017-11-28 2018-04-20 郑州云海信息技术有限公司 A kind of data compression method and apparatus
CN111435288A (en) * 2019-01-14 2020-07-21 阿里巴巴集团控股有限公司 Data processing method and device
WO2022262381A1 (en) * 2021-06-16 2022-12-22 华为技术有限公司 Data compression method and apparatus
CN115629711A (en) * 2022-10-31 2023-01-20 济南浪潮数据技术有限公司 Packet writing alignment method, system, device and medium

Also Published As

Publication number Publication date
CN117389484A (en) 2024-01-12

Similar Documents

Publication Publication Date Title
US10620846B2 (en) Enhancing flash translation layer to improve performance of databases and filesystems
TWI499909B (en) Hierarchical immutable content-addressable memory processor
KR100449708B1 (en) Flash memory management method
US7702844B2 (en) Address mapping method and mapping information managing method for flash memory, and flash memory using the same
CN105975399B (en) Method for managing a memory device and related memory device
US7461198B2 (en) System and method for configuration and management of flash memory
US6477616B1 (en) Storage device, storage system, memory management method, recording medium, and computer data signal
US11782632B2 (en) Selective erasure of data in a SSD
CN107656875B (en) Method and system for shortening power-on time of solid state disk serving as system disk
US20140244935A1 (en) Storage system capable of managing a plurality of snapshot families and method of snapshot family based read
US7287117B2 (en) Flash memory and mapping control apparatus and method for flash memory
JP2007012060A (en) File system having inverted hierarchical structure
US7284085B2 (en) Managing configuration data in a flash configuration space in flash memory within a host interface port
CN105446664B (en) A kind of pair of memory carries out file memory method and storage device
EP3336702B1 (en) Metadata recovery method and device
CN112597072A (en) Data updating method and device of Flash memory
JP2007220107A (en) Apparatus and method for managing mapping information of nonvolatile memory
CN117389484B (en) Data storage processing method, device, equipment and storage medium
CN117370222A (en) Storage mapping method, storage mapping device, computer readable storage medium and electronic equipment
US9535796B2 (en) Method, apparatus and computer for data operation
CN109992535B (en) Storage control method, device and system
CN116027973A (en) Data writing method and device based on flash memory
WO2007048272A1 (en) Method of realizing commands synchronization in supporting multi-threading non-volatile memory file system
CN113050891B (en) Method and device for protecting deduplication data
CN107832234B (en) Quick log block hit method for NAND FLASH

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant