CN114968838A

CN114968838A - Data compression method and flash memory device

Info

Publication number: CN114968838A
Application number: CN202210594149.1A
Authority: CN
Inventors: 韩斌; 陈祥; 黄运新; 孟鹏涛
Original assignee: Shenzhen Dapu Microelectronics Co Ltd
Current assignee: Shenzhen Dapu Microelectronics Co Ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2022-08-30

Abstract

The embodiment of the application relates to the field of storage device application, and discloses a data compression method and a flash memory device, wherein a write command sent by a host is obtained through a hardware compression module, host data corresponding to the write command is divided into a plurality of first data blocks in the same space, each first data block is compressed to obtain a data chain formed by a plurality of second data blocks so as to generate the first command, the first command is sent to a firmware module, the firmware module writes the data chain included by the first command into a cache module, and the cache module combines the plurality of second data blocks in the data chain so as to generate a plurality of minimum units in preset space; the firmware module is combined with the minimum units to obtain the lower brushing units, and the lower brushing units are written into the flash memory medium of the flash memory device.

Description

Data compression method and flash memory device

Technical Field

The present application relates to the field of storage device applications, and in particular, to a data compression method and a flash memory device.

Background

Flash memory devices, such as: solid State Drives (SSD), which are hard disks made of Solid State electronic memory chip arrays, include a control unit and a memory unit (FLASH memory chip or DRAM memory chip).

The solid state disk mainly has two key indexes: the service life and the performance of the solid state disk are mainly influenced by the Nand erasing times, and the Nand erasing times in one disk are directly related to the total written data quantity. The longer the Nand can be used and the longer the disc can be used if the smaller the amount of data written at the same time. After the GC is started, the performance of the solid state disk is mainly affected by Write Amplification (WA), and the larger the write amplification, the worse the host write performance. Write amplification is directly affected by the size of the reserved space (OP) within the disc. Therefore, it is generally necessary to compress the original data to reduce the data writing amount of the solid state disk.

At present, the written data is compressed by hardware of the host, and the compressed data is written into the solid state disk, but the compression is completed on the host side, so that the data writing amount of the solid state disk cannot be substantially reduced, and the writing performance of the solid state disk cannot be improved.

Based on this, there is a need for improvement in the art.

Disclosure of Invention

The embodiment of the application provides a data compression method and a flash memory device, which can perform data compression on the flash memory device side, thereby reducing write amplification and prolonging the service life of the flash memory device.

In order to solve the above technical problem, an embodiment of the present application provides the following technical solutions:

in a first aspect, an embodiment of the present application provides a data compression method, which is applied to a flash memory device, where the flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium, and the method includes:

the hardware compression module acquires a write command sent by a host, wherein the write command corresponds to host data;

the hardware compression module divides the host data into a plurality of first data blocks in the same space;

the hardware compression module compresses each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sends the first command to the firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block;

the firmware module writes the data chain into the cache module, and combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of preset space;

the firmware module combines a plurality of minimum units to obtain a plurality of lower brushing units, and writes the lower brushing units into a flash memory medium of the flash memory device.

In some embodiments, combining the second data blocks in the data chain to generate the minimum units of the preset spaces includes:

acquiring a space corresponding to the current second data block;

if the space of the current second data block is equal to the space of the minimum unit, directly taking the current second data block as the minimum unit;

if the space of the current second data block is smaller than the space of the minimum unit, searching the next second data block, further judging whether the sum of the spaces of the current second data block and the next second data block is smaller than the space of the minimum unit, if so, combining the spaces of the current second data block and the next second data block to serve as the current second data block, and repeating the steps until all the second data blocks are traversed;

after traversing all the second data blocks, if the space sum of a plurality of second data blocks is equal to the space of the minimum unit, taking the plurality of combined second data blocks as the minimum unit; if the sum of the spaces of the second data blocks is not equal to the space of the minimum unit, supplementing invalid data so that the sum of the spaces of the second data blocks and the sum of the spaces of the invalid data are equal to the space of the minimum unit.

In some embodiments, the method further comprises:

if the space of the current second data block is smaller than the space of the minimum unit, judging whether the current search length exceeds a preset search length threshold value, wherein the search length is the distance between the current second data block and the second data block which is searched first;

if so, supplementing invalid data so that the sum of the space of the current second data block and the space of the invalid data is equal to the space of the minimum unit;

if not, searching the next second data block.

In some embodiments, the method further comprises:

when garbage collection is started for the flash memory device, a firmware module acquires a source physical block in a flash memory medium of the flash memory device;

reading a plurality of minimum units from a source physical block to a cache module;

determining the position and the length of effective data in a plurality of minimum units according to the metadata of the flash memory device;

merging the effective data in the minimum units to obtain the merged effective data;

and writing the combined effective data into the target physical block determined by garbage collection until the garbage collection exit condition is met.

In some embodiments, merging valid data in the minimum units to obtain valid data after merging includes:

obtaining the space of the effective data of the current minimum unit;

searching a next minimum unit, and judging whether the sum of the space of the effective data of the current minimum unit and the space of the effective data of the next minimum unit is equal to a preset space, wherein the preset space is the space of one minimum unit;

if so, combining the effective data of the current minimum unit and the effective data of the next minimum unit to obtain a new minimum unit;

if not, merging the effective data of the current minimum unit and the effective data of the next minimum unit, further searching the next minimum unit, and so on until all the minimum units are traversed;

after traversing all the minimum units, if the sum of the spaces of a plurality of minimum units is equal to a preset space, taking the combined minimum units as a new minimum unit; if the sum of the spaces of the minimum units does not exist and is equal to the preset space, supplementing invalid data so that the sum of the spaces of the minimum units and the sum of the spaces of the invalid data are equal to the preset space.

In some embodiments, the method further comprises:

after writing the compressed host data to the cache module, the cache module returns a write success message to the host.

In some embodiments, the method further comprises:

after the flash unit is written into the flash memory medium, the firmware module deletes the minimum unit corresponding to the flash unit from the cache module.

In some embodiments, the method further comprises:

when a read command sent by a host is received, if data corresponding to the read command is stored in the cache module, the firmware module reads the data corresponding to the read command from the cache module;

and if the data corresponding to the read command is stored in the flash memory medium, the firmware module reads the data corresponding to the read command from the flash memory medium.

In some embodiments, the space of the minimum unit is multiple of the space of the flush unit, and the space of the first data block is equal to the mapping management granularity of the firmware of the flash memory device; the space of the lower brush unit is related to the size of a data page of a flash medium of the flash memory device.

In a second aspect, an embodiment of the present application provides a flash memory device, including: a hardware compression module, a firmware module, a cache module, and a flash memory medium, wherein,

the hardware compression module is used for acquiring a write command sent by a host, wherein the write command corresponds to host data; dividing host data into a plurality of first data blocks with the same space, compressing each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sending the first command to a firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block;

the firmware module is used for writing the data chain into the cache module, and combining a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of preset space; and combining the plurality of minimum units to obtain a plurality of lower brushing units, and writing the lower brushing units into a flash memory medium of the flash memory device.

In a third aspect, embodiments of the present application further provide a non-volatile computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to enable a flash memory device to execute the command processing method according to the first aspect.

The beneficial effects of the embodiment of the application are that: different from the prior art, a data compression method provided in an embodiment of the present application is applied to a flash memory device, where the flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium, and the method includes: the hardware compression module acquires a write command sent by a host, wherein the write command corresponds to host data; the hardware compression module divides the host data into a plurality of first data blocks in the same space; the hardware compression module compresses each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sends the first command to the firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block; the firmware module writes the data chain into the cache module, and combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of preset space; the firmware module combines a plurality of minimum units to obtain a plurality of lower brushing units, and writes the lower brushing units into a flash memory medium of the flash memory device.

The method comprises the steps that a write command sent by a host is obtained through a hardware compression module, host data corresponding to the write command are divided into a plurality of first data blocks in the same space, each first data block is compressed to obtain a data chain formed by a plurality of second data blocks so as to generate a first command, the first command is sent to a firmware module, the data chain included in the first command is written into a cache module through the firmware module, and the plurality of second data blocks in the data chain are combined in the cache module so as to generate a plurality of minimum units in the preset space; the firmware module is combined with the minimum units to obtain the lower brushing units, and the lower brushing units are written into the flash memory medium of the flash memory device.

Drawings

One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.

Fig. 1 is a schematic structural diagram of a flash memory device according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a data compression method according to an embodiment of the present application;

FIG. 3 is a diagram illustrating processing of a write command of a host according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of data compression provided by an embodiment of the present application;

FIG. 5 is a block diagram of a firmware module according to an embodiment of the present disclosure;

FIG. 6 is a schematic flow chart illustrating the generation of a minimum unit according to an embodiment of the present disclosure;

fig. 7 is a schematic diagram of combining second data blocks to obtain a minimum unit according to an embodiment of the present application;

FIG. 8 is a schematic diagram of supplemental invalid data provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a minimum unit combined into a lower brush unit according to an embodiment of the present disclosure;

FIG. 10 is a schematic view of garbage collection provided by an embodiment of the present application;

FIG. 11 is a schematic flow chart of garbage recycling according to an embodiment of the present disclosure;

FIG. 12 is a schematic view of garbage collection provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of data integration in a garbage collection process according to an embodiment of the present application;

fig. 14 is an overall schematic diagram of a data compression method provided in an embodiment of the present application;

fig. 15 is a timing diagram of a data compression method according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of a flash memory device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that, if not conflicted, the various features of the embodiments of the present application may be combined with each other within the scope of protection of the present application. Additionally, while functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, steps shown or described may be performed in sequences other than block divisions in apparatus or flowcharts. In addition, the words "first", "second", "third", and the like used herein do not limit the data and execution order, but merely distinguish the same items or similar items having substantially the same functions and actions.

Before the present application is explained in detail, terms and expressions referred to in the embodiments of the present application are explained, and the terms and expressions referred to in the embodiments of the present application are applied to the following explanations:

write Amplification (WA) refers to the ratio of the amount of data written to flash memory to the amount of data written by a user. Due to the existence of Garbage Collection (GC), a user writes certain data, and the flash memory device needs to additionally move some data in order to make room for writing the data, that is, additional writing, so that the amount of data written into the flash memory medium by the flash memory device is larger than the amount of data written into the flash memory device by an actual user, and the write amplification is increased.

The technical scheme of the application is specifically described in the following with the attached drawings of the specification:

referring to fig. 1, fig. 1 is a schematic structural diagram of a flash memory device according to an embodiment of the present disclosure;

as shown in fig. 1, the flash memory device 100 includes a flash medium 110 and a controller 120 connected to the flash medium 110. The flash memory device 100 is communicatively connected to the host 200 in a wired or wireless manner, so as to realize data interaction.

The Flash memory medium 110, which is a storage medium of the Flash memory device 100 and is also called a Flash memory, a Flash memory or a Flash granule, belongs to one of memory devices, and is a nonvolatile memory capable of storing data for a long time without current supply, and has a storage characteristic equivalent to a hard disk, so that the Flash memory medium 110 can become a basis of a storage medium of various portable digital devices.

The controller 120 includes a data converter 121, a processor 122, a buffer 123, a flash controller 124, and an interface 125.

The data converter 121 is connected to the processor 122 and the flash memory controller 124, respectively, and the data converter 121 is configured to convert binary data into hexadecimal data and convert the hexadecimal data into binary data. Specifically, when the flash memory controller 124 writes data to the flash memory medium 110, the binary data to be written is converted into hexadecimal data by the data converter 121 and then written to the flash memory medium 110. When the flash controller 124 reads data from the flash medium 110, hexadecimal data stored in the flash medium 110 is converted into binary data by the data converter 121, and then the converted data is read from the binary data page register. The data converter 121 may include a binary data register and a hexadecimal data register, among others. The binary data register may be used to store data converted from hexadecimal to binary, and the hexadecimal data register may be used to store data converted from binary to hexadecimal.

The processor 122 is connected to the data converter 121, the buffer 123, the flash controller 124 and the interface 125, respectively, wherein the processor 122, the data converter 121, the buffer 123, the flash controller 124 and the interface 125 may be connected by a bus or other methods, and the processor is configured to run the nonvolatile software program, the instructions and the modules stored in the buffer 123, so as to implement any method embodiment of the present application.

The buffer 123 is mainly used for buffering read/write commands sent by the host 200 and read data or write data acquired from the flash memory 110 according to the read/write commands sent by the host 200.

A flash memory controller 124 connected to the flash memory medium 110, the data converter 121, the processor 122 and the buffer 123, for accessing the flash memory medium 110 at the back end and managing various parameters and data I/O of the flash memory medium 110; or, an interface and a protocol for providing access, implementing a corresponding SAS/SATA target protocol end or NVMe protocol end, acquiring an I/O instruction sent by the host 200, decoding, and generating an internal private data result to wait for execution; or, the core processing responsible for Flash Translation Layer (FTL).

The interface 125 is connected to the host 200, the data converter 121, the processor 122, and the buffer 123, and configured to receive data sent by the host 200, or receive data sent by the processor 122, so as to implement data transmission between the host 200 and the processor 122, where the interface 125 may be a SATA-2 interface, a SATA-3 interface, a SAS interface, a MSATA interface, a PCI-E interface, an NGFF interface, a CFast interface, an SFF-8639 interface, and an m.2nvme/SATA protocol.

Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a data compression method according to an embodiment of the present disclosure;

the data compression method is applied to the flash memory device, and the flash memory device comprises a hardware compression module, a firmware module, a cache module and a flash memory medium.

As shown in fig. 2, the data compression method includes:

step S201: the hardware compression module acquires a write command sent by a host, wherein the write command corresponds to host data;

specifically, please refer to fig. 3 again, wherein fig. 3 is a schematic diagram illustrating processing of a write command of a host according to an embodiment of the present disclosure;

as shown in fig. 3, after the write command of the host passes through the NVMe module, the write command is converted into an NVMe command conforming to the NVMe protocol by the NVMe module, and then the NVMe command is forwarded to the hardware compression module.

Step S202: the hardware compression module divides the host data into a plurality of first data blocks in the same space;

as shown in fig. 3, the hardware compression module includes a hardware cache unit, a data splitting unit, a data compression unit, and a data output unit, where the hardware cache unit is connected to the data splitting unit, the data splitting unit is connected to the data compression unit, and the data compression unit is connected to the data output unit.

The hardware caching unit is connected with the NVMe module and used for caching an NVMe command sent by the NVMe module, wherein the NVMe command comprises host data. After the hardware compression module receives the NVMe command, the host data contained in the NVMe command is stored by the hardware cache unit.

The data splitting unit is connected with the hardware cache unit, and is used for splitting host data cached in the hardware cache unit, wherein the data splitting unit divides the host data into a plurality of first data blocks in the same space, that is, the space size of each first data block is equal, for example: the spatial size of each first data block is 4KB, 8KB, 16 KB. Preferably, the spatial size of each first data block in the embodiment of the present application is 4 KB.

The data compression unit is connected with the data splitting unit and used for compressing a first data block obtained after the data splitting unit is split so as to obtain compressed data.

The data output unit is connected with the data compression unit and used for outputting data compressed by the data compression unit, specifically, the compressed data information, the host data and the original NVMe command are combined to generate a first command, and the first command is sent to the firmware module.

In an embodiment of the present application, the hardware cache unit includes a memory, for example: dynamic Random Access Memory (DRAM); the data splitting unit includes a processor, for example: a microprocessor; the data compression unit comprises a hardware compression engine; the data output unit comprises an IO interface.

Step S203: the hardware compression module compresses each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sends the first command to the firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block;

specifically, a data compression unit in the hardware compression module compresses each first data block to obtain a plurality of second data blocks, and combines the plurality of second data blocks to obtain a data chain composed of the plurality of second data blocks.

Referring to fig. 4 again, fig. 4 is a schematic diagram of data compression according to an embodiment of the present disclosure;

as shown in fig. 4, assuming that the space size of the host data corresponding to the write command of the host is 16KB, the host data of 16KB is divided into 4 first data blocks of 4KB, and each first data block of 4KB is compressed to obtain a second data block corresponding to each first data block, that is, each first data block corresponds to one second data block, for example: the first data block of 4KB is compressed into second data blocks of 1KB, 3KB, 2KB and 3KB respectively, the 4 second data blocks are combined to generate a data chain, and a first command is generated by the data chain, wherein the first command is a new write command, and the first command comprises the data chain. It is understood that the compressed data length is determined by the compression algorithm, and different compression algorithms or different data patterns may result in different compressed data lengths. Due to the compression, the length of each second data block is smaller than the length of the first data block, i.e. the space of each second data block is smaller than the space of the first data block.

In the embodiment of the present application, the first command includes a set of host data, an original NVMe command and compressed data information, where the original NVMe command is an NVMe command generated after a write command of the host passes through the NVMe module, and the NVMe command includes a starting logical address (startlba) and a data length (len); the compressed data information includes a compressed data length per 4K data unit.

Step S204: the firmware module writes the data chain into the cache module, and combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of preset space;

specifically, after generating the first command, the hardware compression module sends the first command to the firmware module, and the firmware module writes the data chain into the Cache module, where the Cache module includes a Cache memory (Cache) of the flash memory device.

Referring to fig. 5 again, fig. 5 is a schematic structural diagram of a firmware module according to an embodiment of the present disclosure;

the firmware module is used for connecting a HOST (HOST) and a flash memory array (NAND) to realize the processing of data IO.

As shown in fig. 5, the firmware module 50, the connection host 200 and the hardware module 60 include:

a Front-End module 501, namely (Front End, FE), configured to obtain a Host command to generate an IO operation, where the Front-End module is further configured to take charge of operations such as a communication protocol with a Host (Host), parsing of the Host command, and a solid state disk command;

a Flash memory algorithm module 502, namely a Flash Translation Layer (FTL), connected to the data processing module, for performing mapping processing on the IO operation to determine a delivered Flash memory array; wherein, the Flash Translation Layer (FTL) sends IO operation to the Back End module (Back End, BE) of the solid state hard disk controller, so that the Back End module of the solid state hard disk controller receives the IO operation sent by the Flash Translation Layer;

the Back End module 503, namely (Back End, BE), is connected to the flash memory algorithm module, and is configured to receive the IO operation sent by the flash memory algorithm module, so as to control the hardware module to perform read/write/erase operation on the flash memory array;

in the embodiment of the present application, the hardware module 60, i.e. (HW Op Nand Mode), refers to a module for operating the FLASH memory, and is connected to the back-end module 503, and is controlled by the back-end module 503 to operate the FLASH memory, for example: and according to the IO operation, operating the corresponding FLASH memory array or FLASH memory medium, namely finishing the operation processing of the data to the FLASH FLASH memory, wherein the operation comprises read operation, write operation or erase operation.

The front-end module processes the host command to generate IO operation after acquiring the host command, and operates the flash memory array sequentially through the flash memory algorithm module, the rear-end module and the hardware module. For example: when the Host reads data, the Host (Host) sends a Host command to the flash memory device (device), a front end module (FE) of the flash memory device receives the Host command, processes the Host command and then distributes the Host command to a flash memory algorithm module (FTL), the flash memory algorithm module receives the Host command, performs conversion processing from logic to physics, then sends a NAND read request operation to a back end module (BE), the back end module receives the read operation and then sends a hardware command to a hardware module (HW Opnd Mode), and the hardware module performs NAND operation in parallel.

In this embodiment of the present application, a first command sent by a hardware compression module is received by a front-end module, the front-end module forwards the first command to a flash memory algorithm module, the flash memory algorithm module parses a data chain in the first command, and writes the data chain into a cache module, and the flash memory algorithm module combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of a preset space.

Specifically, combining a plurality of second data blocks in the data chain to generate a minimum unit of a plurality of preset spaces includes:

acquiring a space corresponding to the current second data block;

if the space of the current second data block is smaller than the space of the minimum unit, searching the next second data block, further judging whether the sum of the spaces of the current second data block and the next second data block is smaller than the space of the minimum unit, if so, combining the two data blocks to serve as the current second data block, and repeating the steps until all the second data blocks are traversed;

after traversing all the second data blocks, if the space sum of the plurality of second data blocks is equal to the space of the minimum unit, taking the plurality of merged second data blocks as the minimum unit; if the sum of the spaces of the second data blocks is not equal to the space of the minimum unit, supplementing invalid data so that the sum of the spaces of the second data blocks and the sum of the spaces of the invalid data are equal to the space of the minimum unit.

Referring to fig. 6, fig. 6 is a schematic flow chart illustrating a process of generating a minimum unit according to an embodiment of the present application;

as shown in fig. 6, the process of generating the minimum unit includes:

step S601: acquiring the current space size of the first second data block and the second data block S1;

step S602: determining S1 whether it is equal to the space size of the minimum cell;

specifically, if S1 is equal to the space size of the minimum unit, that is, the space of the current second data block is equal to the space of the minimum unit, the current second data block is directly used as the minimum unit, and the search is ended;

if not, that is, the space of the current second data block is smaller than the space of the minimum unit, the step S803 is entered;

step S603: judging whether the current search length exceeds a preset search length threshold value or not;

specifically, if the space of the current second data block is smaller than the space of the minimum unit, determining whether the current search length exceeds a preset search length threshold, where the search length is a distance between the current second data block and a second data block for which a search is started first;

if not, searching the next second data block.

Step S604: sequentially traversing other second data blocks, and acquiring the size Sx of the currently searched second data block;

step S605: judging whether S1+ Sx is less than or equal to the space size of the minimum unit;

specifically, the next second data block is searched, whether the sum of the spaces of the current second data block and the next second data block is smaller than the space of the minimum unit is further judged, and if yes, the step 606 is executed; if not, returning to the step S603;

step S606: combining S1 and Sx into a new data block, wherein S1 is S1+ Sx;

if S1+ Sx is less than or equal to the space size of the minimum unit, merging the two to serve as the current second data block, and so on until all the second data blocks are traversed;

step S607: supplementing invalid data to constitute a space size of a minimum cell;

specifically, if the current search length exceeds the preset search length or search distance, the invalid data is supplemented to form the space size of the minimum unit, and the data block after the invalid data is supplemented is taken as the minimum unit.

The invalid data, i.e. Dummy data, is invalid data supplemented by the firmware, for example: all 0x0 or all 0xF data. For example, nand requires 16KB for a write, the firmware must transfer 16KB of data length. The supplementary invalid data Dummy is a piece of data initialized to all 0x0 or 0xF to supplement the pieced data.

And (6) ending.

Referring to fig. 7, fig. 7 is a diagram illustrating a minimum unit obtained by combining second data blocks according to an embodiment of the present disclosure;

as shown in fig. 7, the host data corresponding to the write commands of the two hosts are respectively 16KB and 8KB, and are respectively split into 3 first data blocks of 4KB and 2 first data blocks of 4KB, and each first data block is compressed, and the sizes of the compressed second data blocks are respectively 1KB, 3KB, 2KB, and 1KB, at this time, several second data blocks are combined to obtain a minimum unit, where the space size of the minimum unit is preset, and the space size of the minimum unit is related to the mapping management granularity in the firmware, that is, the space size of the minimum unit is equal to the mapping management granularity in the firmware, for example: the mapping management granularity is 4KB, 8KB, 16KB, and the space of the minimum unit is preset to 4KB, 8KB, or 16KB, which is not limited herein. Preferably, the preset space of the minimum unit in the embodiment of the present application is 4 KB.

Referring to fig. 8 again, fig. 8 is a schematic diagram illustrating supplemental invalid data according to an embodiment of the present disclosure;

as shown in fig. 8, when several second data blocks cannot be combined to obtain the space size of the minimum unit, i.e. during the data splicing process, a situation that one 4K cannot be spliced may also occur, and at this time, invalid data (dummy) needs to be supplemented. For example, all data is compressed to 3KB in the Cache, and no arbitrary multiple data can be combined into one 4KB, and the final data needs to be supplemented with invalid data to make up the size of 4 KB.

Step S205: the firmware module combines a plurality of minimum units to obtain a plurality of lower brushing units, and writes the lower brushing units into a flash memory medium of the flash memory device.

Specifically, the firmware module combines a plurality of minimum units in the cache module to obtain a plurality of flushing units, and writes the plurality of flushing units into a flash memory medium of the flash memory device.

In the embodiment of the present application, the spatial size of the lower brush unit is greater than or equal to the spatial size of the minimum unit, and the spatial size of the lower brush unit is an integral multiple of the spatial size of the minimum unit. For example: if the space size of the lower brushing unit is equal to that of the minimum unit, each minimum unit is directly used as a lower brushing unit to be written into the flash memory medium; if the space size of the lower brushing unit is larger than the space size of the minimum unit, and the space size of the lower brushing unit is an integral multiple of the space size of the minimum unit, for example: and combining the minimum units of the proportionality coefficients to obtain a lower brushing unit, repeating the steps to obtain a plurality of lower brushing units, and writing the lower brushing units into the flash memory medium.

Referring to fig. 9, fig. 9 is a schematic diagram of a minimum unit combining a lower brush unit according to an embodiment of the present disclosure;

as shown in fig. 9, the two minimum units are 4KB each, the size of the data chain formed by the two minimum units is 8KB, and the size of one lower brushing unit is 8KB, and the two minimum units are combined to form one lower brushing unit.

In the embodiment of the present application, the space size of the lower brushing unit is related to the size of the data page of the flash memory medium.

It is understood that in a solid state disk, written data cannot be directly overwritten in its physical location of storage due to the read and write principles of NAND FLASH. The need to perform an erase operation first in units of physical blocks (blocks) and then allow a write operation to be performed on the pages of data (pages) contained in the Block makes SSDs a key weakness of very inefficient update-in-place operations. Therefore, the SSD system usually adopts a remote update operation, that is, data is copied to a buffer for updating, and then is stored in a new location after the update, an old location containing original data is marked as "Garbage", and all physical pages Page contained in one physical Block can be converted into an available space again only through Garbage Collection (GC).

Garbage Collection (GC) is to copy the data of a valid page in NAND FLASH in one physical block into another free block (unused physical block) and then erase this physical block completely, wherein when there is data to be modified in any one physical page in NAND FLASH, the physical page is identified as an invalid page, the data in the invalid page becomes invalid data, and the size of page space is applied for data writing in other physical blocks, and the physical page which is not identified as an invalid page and is used is called as a valid page. The Program/Erase cycle (PE) of a Solid State Drive (SSD) is simply referred to as the Erase cycle, and determines the lifetime of the Solid State drive.

Referring to fig. 10, fig. 10 is a schematic view illustrating garbage recycling according to an embodiment of the present disclosure;

as shown in fig. 10, the garbage collection process copies the data of the valid page in one physical block into another free block (unused physical block), and then completely erases the physical block, for example: valid data of physical block a (blocka) is transferred to physical block b (blockb). It is understood that there may be invalid data and valid data on one physical Block (Block). The effective data in one physical Block (Block) is read out and written into the other physical Block (Block), so that the purpose of releasing the space of the whole source physical Block can be achieved.

Referring to fig. 11 again, fig. 11 is a schematic view illustrating a garbage recycling process according to an embodiment of the present disclosure;

as shown in fig. 11, the garbage collection process includes:

step S111: starting garbage recovery;

step S112: selecting a source physical block needing garbage recovery;

specifically, after garbage collection is performed, the garbage collection module of the flash memory device selects a source physical block requiring garbage collection according to a starting condition of garbage collection, where the starting condition includes that a ratio of valid data is smaller than a preset ratio threshold.

In this embodiment, the garbage collection module belongs to hardware or firmware of the flash memory device, and is not limited herein.

Step S113: reading a plurality of minimum units from the flash memory medium to a cache module;

step S114: determining the position and the length of effective data in a plurality of minimum units according to the metadata of the flash memory device;

step S115: forming a linked list by the addresses and the lengths of the effective data in the minimum units;

it is understood that one of the smallest units stored on the Nand, for example, the smallest unit is 4KB, and in the space of the smallest unit of 4KB, it may be the result of compressing 4K data of the original plurality of hosts. So that it is true that multiple host 4K data corresponds to one copy of 4K data on Nand. Such as LBA0, LBA1, LBA2, LBA3, are eventually compressed to exist in a 4KB space.

When the host writes the LBA0 again, the LBA0 may be compressed with the LBA11, LBA12, and LBA13 into another 4K data. So that a substantial part of the original 4K data is invalid data. If the GC migrates the entire 4K, a portion of invalid data is also migrated, increasing write amplification.

It should be noted that if only 4K data of one host is valid in 4K data of each NAND that the GC moves, other data are compressed together with other data when the host rewrites. If the GC directly moves the 4K data on the NAND, the compression effect when the entire data is written is gradually "degraded". Therefore, in the garbage recycling process, the data is integrated again, namely effective data is found out from each read 4K data, the effective data in each 4K is spliced into a new 4K again, and then the effective data is written into another physical Block (Block), so that the write amplification is reduced.

Step S116: traversing the linked list to piece up a new minimum unit;

specifically, the valid data in the minimum units are merged to obtain merged valid data, wherein the size of the merged valid data takes the space size of the minimum unit as granularity; if the minimum unit cannot be pieced together, invalid data is supplemented to piece together the space size of the minimum unit.

It should be noted that the firmware of the flash memory device includes metadata, and the metadata is used to maintain valid data in 4K data on each NAND, that is, the metadata is used to determine which data of the location is valid data.

Step S117: writing the minimum unit obtained by splicing into a target physical block;

specifically, writing each minimum unit obtained by the mosaicing into the target physical block is equivalent to writing the valid data after the merging into the target physical block determined by garbage collection.

Referring to fig. 12 again, fig. 12 is a schematic view of garbage recycling according to an embodiment of the present disclosure;

the garbage collection module comprises a data reading module and a data writing module.

Specifically, as shown in fig. 12, the read data module is configured to select a source physical block requiring garbage collection from a flash memory medium (NAND), and the write data module is configured to write a minimum unit obtained by splicing into a target physical block in the flash memory medium (NAND).

Referring to fig. 13, fig. 13 is a schematic diagram illustrating data integration in a garbage collection process according to an embodiment of the present disclosure;

it can be understood that, in the garbage recycling process, only the valid data in the source physical block needs to be moved, so the positions and lengths of the valid data in the minimum units in the source physical block are determined, the valid data in the minimum units are combined to be spliced into a new minimum unit, as shown in fig. 13, the space size of the minimum unit is 4KB, the valid data are combined to obtain a new minimum unit, and the space size of the combined minimum unit is also 4 KB. In the figure, the red part is the rewritten invalid data, and the green part is the valid data. If two 4K data in BlockA read by GC are valid for only 3K total actual data, then only 3K data are extracted and integrated into the buffer to be written to BlockB. This reduces the write amount of the GC, and also saves the result of compression at the time of data writing.

Step S118: continuing to cycle the data reading and writing process until the garbage recycling exit condition is met;

step S119: ending the garbage recovery;

in the embodiment of the application, the bandwidth performance requirement of the SSD can be met by a scheme of compressing and matching special hardware with firmware logic. And moreover, by means of the algorithm of compressing the splicing data and GC splicing data, the write amplification of the flash memory device is reduced, and the service life of the flash memory device is indirectly prolonged. In addition, the scheme in the application is a compression scheme that the host is completely insensitive, and when the host reads and writes the flash memory device normally, the functions of decompressing and compressing data in the flash memory device are automatically completed, so that the interaction stability of the flash memory device and the host is improved.

Referring to fig. 14 again, fig. 14 is an overall schematic diagram of a data compression method according to an embodiment of the present application;

as shown in fig. 14, the Host (Host) sends a Host command (Host cmd) including original data (originalaldata), i.e., Host data. After the write command of the host passes through the NVMe module, the NVMe module converts the write command into an NVMe command conforming to the NVMe protocol, and then forwards the NVMe command to a hardware compression module (hwcompresseengine). The NVMe module and the hardware compression module belong to hardware modules of the flash memory device.

The hardware compression module divides host data into a plurality of first data blocks with the same space, compresses each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sends the first command to the firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block;

the firmware module writes the data chain into the cache module, combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units, and combines the minimum units to obtain a plurality of lower brushing units (Cacheflushunit) to write the lower brushing units into a flash memory medium of the flash memory device.

In the embodiment of the present application, the host command (HostCmd) includes: the host command's identification (Cmd ID), Logical Block Address (Lba), and the length of the Address to read (Len), where Logical Block Address is typically 512 bytes, 4k, etc. relative to the host.

Referring to fig. 15 again, fig. 15 is a timing diagram of a data compression method according to an embodiment of the present disclosure;

as shown in fig. 15, the sequence of the data compression method includes:

step S151: the host writes data into the hardware compression module;

step S152: the hardware compression module performs data compression on data written in by the host to obtain compressed host data;

step S153: the hardware compression module writes the compressed host data into the cache module;

step S154: after the hardware compression module writes the compressed host data into the cache module, the cache module returns a write success message to the host.

The writing success message is sent to the host by the firmware module through the cache module, or the writing success message is directly sent by the cache module.

Step S155: the cache module combines the compressed data;

step S156: writing into a flash memory;

specifically, the firmware module writes the flash memory with the flash unit obtained after the compressed data is combined.

Step S157: writing into the flash memory is successful;

specifically, the firmware module returns a write flash success message to the cache module.

Step S158: deleting data in the cache module;

specifically, after the flush unit is written into the flash memory medium, the firmware module deletes the minimum unit corresponding to the flush unit from the cache module, and specifically, after the flush unit is successfully written into the flash memory (NAND) by the firmware module, the firmware module notifies the cache module to enable the cache module to delete the corresponding data stored in the memory of the cache module, where the memory of the cache module includes a cache.

In some embodiments, the method further comprises:

In some embodiments, the space of the minimum unit is multiple of the space of the flush unit, the space of the first data block is equal to the mapping management granularity of the firmware of the flash memory device; the space of the lower brush unit is related to the size of a data page of a flash medium of the flash memory device.

In an embodiment of the present application, a data compression method is provided, which is applied to a flash memory device, where the flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium, and the method includes: the hardware compression module acquires a write command sent by a host, wherein the write command corresponds to host data; the hardware compression module divides the host data into a plurality of first data blocks in the same space; the hardware compression module compresses each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sends the first command to the firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block; the firmware module writes the data chain into the cache module, and combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of preset space; the firmware module combines a plurality of minimum units to obtain a plurality of lower brushing units, and writes the lower brushing units into a flash memory medium of the flash memory device.

Referring to fig. 16 again, fig. 16 is a schematic structural diagram of a flash memory device according to an embodiment of the present disclosure;

as shown in fig. 16, the flash memory device 160 includes: a hardware compression module 161, a cache module 162, a firmware module 163, and a flash media 164, wherein,

the hardware compression module 161 is configured to obtain a write command sent by a host, where the write command corresponds to host data; and dividing the host data into a plurality of first data blocks with the same space, compressing each first data block to obtain a data chain consisting of a plurality of second data blocks, so as to generate a first command, and sending the first command to the firmware module 163, wherein the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block.

And a cache module 162 for storing the data written by the hardware compression module 161, or storing the data written by the firmware module 163, and combining the data.

A firmware module 163, configured to write the data chain into the cache module, and combine a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of the preset space; and combines the plurality of minimum units to obtain a number of scrub-down units, and writes the scrub-down units to the flash medium 164 of the flash memory device 160.

And a flash medium 164 for storing data written by the firmware module.

In the embodiment of the application, a write command sent by a host is acquired through a hardware compression module, host data corresponding to the write command is divided into a plurality of first data blocks in the same space, each first data block is compressed to obtain a data chain formed by a plurality of second data blocks so as to generate the first command, the first command is sent to a firmware module, the data chain included in the first command is written into a cache module by the firmware module, and the plurality of second data blocks in the data chain are combined in the cache module so as to generate a plurality of minimum units in a preset space; the firmware module is combined with the minimum units to obtain the lower brushing units, and the lower brushing units are written into the flash memory medium of the flash memory device.

Embodiments of the present application also provide a non-volatile computer storage medium storing computer-executable instructions, which are executed by one or more processors, for example, the one or more processors may execute the data compression method in any of the method embodiments, for example, execute the steps described above.

The above-described embodiments of the apparatus or device are merely illustrative, wherein the unit modules described as separate parts may or may not be physically separate, and the parts displayed as module units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or partially contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method of the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; within the context of the present application, features from the above embodiments or from different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the present application as described above, which are not provided in detail for the sake of brevity; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and these modifications or substitutions do not depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data compression method applied to a flash memory device, wherein the flash memory device comprises a hardware compression module, a firmware module, a cache module and a flash memory medium, and the method comprises the following steps:

and the firmware module combines a plurality of the minimum units to obtain a plurality of the lower brushing units, and writes the lower brushing units into a flash memory medium of the flash memory device.

2. The method of claim 1, wherein the combining the second data blocks in the data chain to generate the minimum units of the preset spaces comprises:

acquiring a space corresponding to the current second data block;

after traversing all the second data blocks, if the sum of the spaces of a plurality of second data blocks is equal to the space of the minimum unit, taking the plurality of merged second data blocks as the minimum unit; and if the sum of the spaces of the plurality of second data blocks is not equal to the space of the minimum unit, supplementing invalid data so that the sum of the spaces of the plurality of second data blocks and the sum of the spaces of the invalid data are equal to the space of the minimum unit.

3. The method of claim 2, further comprising:

if not, searching the next second data block.

4. The method of claim 1, further comprising:

when garbage collection is started for the flash memory device, the firmware module acquires a source physical block in a flash memory medium of the flash memory device;

reading a number of minimum units from the source physical block to the cache module;

determining the positions and the lengths of effective data in a plurality of minimum units according to the metadata of the flash memory device;

and writing the merged effective data into a target physical block determined by garbage collection until a garbage collection exit condition is met.

5. The method according to claim 4, wherein the merging the valid data in the minimum units to obtain the valid data after merging comprises:

obtaining the space of the effective data of the current minimum unit;

after traversing all the minimum units, if the sum of the spaces of a plurality of minimum units is equal to the preset space, taking the combined minimum units as a new minimum unit; and if the sum of the spaces of the minimum units does not exist and is equal to the preset space, supplementing invalid data so that the sum of the spaces of the minimum units and the sum of the spaces of the invalid data are equal to the preset space.

6. The method of claim 1, further comprising:

7. The method of claim 1, further comprising:

8. The method of claim 1, further comprising:

9. The method of any of claims 1-8, wherein the space of the minimum unit is a multiple of the space of the flush unit, and the space of the first data block is equal to a mapping management granularity of firmware of the flash memory device; the space of the lower brush unit is related to the size of a data page of a flash medium of the flash memory device.

10. A flash memory device, comprising: a hardware compression module, a firmware module, a cache module, and a flash memory medium, wherein,

the hardware compression module is used for acquiring a write command sent by a host, wherein the write command corresponds to host data; dividing the host data into a plurality of first data blocks with the same space, compressing each first data block to obtain a data chain consisting of a plurality of second data blocks so as to generate a first command, and sending the first command to the firmware module, wherein the first command comprises the data chain, and the space of each second data block is smaller than that of the first data block;

the firmware module is configured to write the data chain into the cache module, and combine a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of a preset space; and combining a plurality of the minimum units to obtain a plurality of lower brushing units, and writing the lower brushing units into a flash memory medium of the flash memory device.