CN117193660A - Data processing method, device and system, electronic equipment and storage medium - Google Patents
Data processing method, device and system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117193660A CN117193660A CN202311257647.8A CN202311257647A CN117193660A CN 117193660 A CN117193660 A CN 117193660A CN 202311257647 A CN202311257647 A CN 202311257647A CN 117193660 A CN117193660 A CN 117193660A
- Authority
- CN
- China
- Prior art keywords
- data
- sub
- mapping
- mapping data
- compressed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000013507 mapping Methods 0.000 claims abstract description 214
- 239000000872 buffer Substances 0.000 claims abstract description 100
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000013144 data compression Methods 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 6
- 239000007787 solid Substances 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 5
- 238000013519 translation Methods 0.000 claims description 3
- 230000001680 brushing effect Effects 0.000 abstract description 5
- 229920001485 poly(butyl acrylate) polymer Polymers 0.000 description 21
- 238000010586 diagram Methods 0.000 description 16
- 238000012005 ligant binding assay Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 108700041286 delta Proteins 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of computers, and discloses a data processing method, a device, a system, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a plurality of pieces of mapping data to be written into an increment buffer; according to the physical block address continuous information represented by the plurality of pieces of mapping data, carrying out data compression on the mapping data to obtain compressed mapping data; and writing the compressed mapping data into an increment buffer area. According to the method provided by the scheme, the mapping data is compressed before being written into the increment buffer zone, and finally the compressed mapping data is written into the increment buffer zone, so that the use efficiency of the increment buffer zone is improved, the brushing times of the mapping data are reduced, and the performance of the SSD is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, system, electronic device, and storage medium.
Background
With the advent and wide use of NAND flash memory, solid State Disk (SSD) based on NAND flash memory has become a development hot spot in the storage field due to the characteristics of higher reliability, better performance, lower energy consumption, and the like. Wherein, the SSD generally constructs a mapping relation between a logical block address (Logic Block Address, abbreviated as LBA) of the host and a physical block address (Physical Block Address, abbreviated as PBA) of the SSD based on the L2P mapping table, so how to process the L2P mapping table data directly affects SSD performance.
In the prior art, the mapping table is usually stored in several parts in the base buffer and the increment buffer, and one increment buffer and one base buffer are merged and stored on the NAND when the increment buffer is filled.
However, due to the limited capacity of the increment buffer, mapping data is frequently brushed down to the NAND during running of the SSD, which reduces the performance of the SSD and is not beneficial to guaranteeing the NAND service life of the SSD.
Disclosure of Invention
The application provides a data processing method, a device, a system, electronic equipment and a storage medium, which are used for solving the defects that the performance of SSD is reduced in the prior art.
The first aspect of the present application provides a data processing method, comprising:
acquiring a plurality of pieces of mapping data to be written into an increment buffer;
according to the physical block address continuous information represented by the mapping data, carrying out data compression on the mapping data to obtain compressed mapping data;
and writing the compressed mapping data into the increment buffer.
Optionally, the mapping data includes a logical block address, an old physical block address and a new physical block address which are in one-to-one correspondence, and the data compression is performed on the mapping data according to the physical block address continuous information represented by the plurality of mapping data, so as to obtain compressed mapping data, including:
splitting the mapping data into first mapping data and second mapping data; the first mapping data comprises one-to-one corresponding logic block addresses and old physical block addresses, and the second mapping data comprises one-to-one corresponding logic block addresses and new physical block addresses;
splitting the first mapping data into a plurality of segments of first sub-data according to the old physical block address continuous information represented by the first mapping data;
for any one of the first sub data, generating first compressed sub data corresponding to the first sub data according to the length of the first sub data, the initial logic block address and the first initial sub data of the first sub data;
combining the first compressed sub data corresponding to each first sub data to obtain first compressed data corresponding to the first mapping data;
splitting the second mapping data into a plurality of segments of second sub-data according to the new physical block address continuous information represented by the second mapping data;
generating second compressed sub-data corresponding to any one of the second sub-data according to the length of the second sub-data, the initial logic block address and the first initial sub-data of the first sub-data;
combining the second compressed sub-data corresponding to each second sub-data to obtain second compressed data corresponding to the second mapping data;
and merging the first compressed data and the second compressed data to obtain compressed mapping data.
Optionally, the first mapping data includes a sequence number, and splitting the first mapping data into a plurality of segments of first sub-data according to the old physical block address continuous information characterized by the first mapping data includes:
traversing the data in the first mapping data from front to back, and taking the first data in the first mapping data as first initial sub-data;
if the address difference of the old physical block between the traversed data and the first initial sub-data is equal to the serial number difference between the traversed data and the first initial sub-data, determining that the data and the first initial sub-data are continuous data of the old physical block address;
when traversing to the data with the address difference of the old physical block between the first initial data and the data which is not equal to the serial number difference between the two, taking the data as new first initial sub-data;
splitting the first mapping data into a plurality of segments of first sub-data according to each first starting sub-data;
the data in each first sub data is continuous data of the old physical block address.
Optionally, the second mapping data includes a sequence number, and splitting the second mapping data into a plurality of segments of second sub-data according to new physical block address continuous information characterized by the second mapping data includes:
traversing the data in the second mapping data from front to back, and taking the first data in the second mapping data as second initial sub-data;
if the new physical block address difference between the traversed data and the second initial sub-data is equal to the sequence number difference between the traversed data and the second initial sub-data, determining that the data and the second initial sub-data are new physical block address continuous data;
when traversing to the data with the new physical block address difference between the second initial data and the data which is not equal to the serial number difference between the second initial data and the data, taking the data as new second initial sub data;
splitting the second mapping data into a plurality of segments of second sub-data according to each second initial sub-data;
wherein, the data in each second sub data is new physical block address continuous data.
Optionally, the compressed mapping data at least includes a start logical block address, a length, start sub-data and a sub-data type corresponding to each sub-data.
Optionally, the method further comprises:
when the data storage amount of the increment buffer reaches a preset threshold value, extracting the compressed mapping data in the increment buffer;
extracting base data in a base buffer;
carrying out data combination on the compressed mapping data and the base data to obtain combined mapping data;
and storing the combined mapping data to a flash memory of the solid state disk.
A second aspect of the present application provides a data processing apparatus comprising:
the acquisition module is used for acquiring a plurality of pieces of mapping data to be written into the increment buffer;
the compression module is used for carrying out data compression on the mapping data according to the physical block address continuous information represented by the plurality of pieces of mapping data to obtain compressed mapping data;
and the processing module is used for writing the compressed mapping data into the increment buffer area.
A third aspect of the application provides a data processing system comprising:
the flash memory comprises a random access memory and a flash memory conversion layer controller, wherein the random access memory comprises a plurality of increment buffer areas and a base buffer area;
the flash translation layer controller processes the mapping data to be written into the delta buffer using the data processing method described in the first aspect and the various possible designs of the first aspect.
A fourth aspect of the present application provides an electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes the computer-executable instructions stored by the memory such that the at least one processor performs the method as described above in the first aspect and the various possible designs of the first aspect.
A fifth aspect of the application provides a computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method as described above for the first aspect and the various possible designs of the first aspect.
The technical scheme of the application has the following advantages:
the application provides a data processing method, a device, a system, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a plurality of pieces of mapping data to be written into an increment buffer; according to the physical block address continuous information represented by the plurality of pieces of mapping data, carrying out data compression on the mapping data to obtain compressed mapping data; and writing the compressed mapping data into an increment buffer area. According to the method provided by the scheme, the mapping data is compressed before being written into the increment buffer zone, and finally the compressed mapping data is written into the increment buffer zone, so that the use efficiency of the increment buffer zone is improved, the brushing times of the mapping data are reduced, and the performance of the SSD is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description will be given below of the drawings required for the embodiments or the prior art descriptions, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a schematic diagram of a base buffer area according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an incremental buffer structure according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a structure of consolidated mapping data according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an incremental data structure according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a compressed sub-data structure according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a code execution flow of a data processing method according to an embodiment of the present application;
FIG. 8 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a data processing system according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. These drawings and the written description are not intended to limit the scope of the disclosed concept in any way, but to illustrate the inventive concept to those skilled in the art by reference to specific embodiments.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. In the following description of the embodiments, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the prior art, an SSD system forms a mapping relation between LBAs of a host and PBAs of the SSD through an L2P table, updates and maintains the mapping relation through an FTL controller, and realizes functions of reading, writing, erasing and the like of host transmission data on NAND. Typically the L2P mapping table is about one thousandth of the SSD disk capacity, e.g. the capacity of a 1T disk L2P mapping table is about 1G. In view of the large capacity of the mapping table, the mapping table is not likely to be stored in whole, but is divided into a plurality of parts to be stored in a column of basic Buffer areas (Base Buffer) with equal length on the RAM, and a column of incremental Buffer areas (Delta Buffer) with equal length is allocated on the RAM, wherein, as shown in fig. 1, the basic Buffer area structure schematic diagram provided by the implementation of the present application is shown in fig. 2, the basic Buffer areas (Base Buffer) store PBAs corresponding to each LBA, lba_0 is PBAs corresponding to lba=0, and lba_1 is PBAs corresponding to lba=1; the Delta Buffer (Delta Buffer) stores Delta entries (Delta Entry), and delta_0, delta_1 and the like are Delta entries. When the system generates a new mapping relation, the new mapping relation of L2P is updated to a basic quantity buffer area, the new mapping relation is saved to an increment buffer area, after one increment buffer area is filled, head information (Header) is added to the increment buffer area data, one basic quantity buffer area is combined, the combined mapping data is obtained, and the combined mapping data is written on NAND in a brushing mode. Fig. 3 is a schematic structural diagram of the merged mapping data according to the embodiment of the present application. The Header information (Header) includes characteristic information of the combined base buffer and delta buffer.
The information to be saved in a mapped increment generally includes an original mapping relationship and a New mapping relationship, and a common storage structure is shown in fig. 4, where the common storage structure includes an LBA, an Old PBA, and a New PBA, and fig. 4 is a schematic diagram of an increment data structure provided in an embodiment of the present application, where one mapping relationship represents a 4KB space on a host, one Entry occupies 12 bytes, and by adopting this structure, a Delta Buffer of 10KB stores 853 Delta Entry as a maximum, and represents a mapping relationship of 3412 KB. The representation mode is simple and direct, but for an enterprise SSD system, a large amount of incremental information is generated when data is written in a large amount and frequently, the load is brought to the caching of the data, the NAND performance is affected, and the space occupation condition of each structure in the incremental data Delta Entry A is shown in the following table:
in view of the above problems, the method for processing data provided by the embodiment of the present application includes: acquiring a plurality of pieces of mapping data to be written into an increment buffer; according to the physical block address continuous information represented by the plurality of pieces of mapping data, carrying out data compression on the mapping data to obtain compressed mapping data; and writing the compressed mapping data into an increment buffer area. According to the method provided by the scheme, the mapping data is compressed before being written into the increment buffer zone, and finally the compressed mapping data is written into the increment buffer zone, so that the use efficiency of the increment buffer zone is improved, the brushing times of the mapping data are reduced, and the performance of the SSD is improved.
The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
The embodiment of the application provides a data processing method which is used for compressing mapping data to be written into an increment buffer area. The execution body of the embodiment of the application is electronic equipment, such as a server, a desktop computer, a notebook computer, a tablet computer and other electronic equipment which can be used for compressing mapping data to be written into an increment buffer.
Fig. 5 is a schematic flow chart of a data processing method according to an embodiment of the present application, where the method includes:
in step 501, several pieces of mapping data to be written into the incremental buffer are obtained.
Specifically, a plurality of pieces of mapping data to be written into the incremental buffer area by the host computer can be collected based on a preset data collection device.
Step 502, performing data compression on the mapping data according to the physical block address continuous information represented by the plurality of pieces of mapping data, and obtaining compressed mapping data.
It should be noted that, according to the characteristics of the host write-in data and the NAND memory space allocation, the LBA, old PBA, new PBA in the New mapping relationship are continuous with high probability, so that the corresponding data compression operation can be performed on the plurality of pieces of mapping data according to the physical block address continuous information represented by the mapping data obtained at present, so as to obtain the compressed mapping data.
Step 503, writing the compressed mapping data into the increment buffer.
The data volume of the compressed mapping data is far smaller than that of the mapping data to be written into the increment buffer area, which is acquired at the beginning, so that the compressed mapping data is written into the increment buffer area, the space occupation amount of the mapping data to the increment buffer area is reduced, and the use efficiency of the increment buffer area is improved.
Further, in an embodiment, when the data storage amount of the incremental buffer reaches a preset threshold, extracting compressed mapping data in the incremental buffer; extracting base data in a base buffer; data combination is carried out on the compressed mapping data and the base data to obtain combined mapping data; and storing the combined mapping data to a flash memory of the solid state disk.
The preset threshold value can be set according to the capacity of the increment buffer zone, when the data storage capacity of the increment buffer zone reaches the preset threshold value, the increment buffer zone is determined to be full, compressed mapping data is extracted from the increment buffer zone, then a base buffer zone is selected, the compressed mapping data stored in the increment buffer zone and the base data stored in the base buffer zone are subjected to data combination, combined mapping data are obtained, and finally the combined mapping data are stored to the NAND flash memory of the solid state disk through a communication interface between the NAND flash memory and the NAND flash memory.
Specifically, in an embodiment, when the data storage amount of the incremental buffer reaches a preset threshold, the compressed mapping data currently stored in the incremental buffer may be compressed for the second time, so as to further release the storage space of the incremental buffer, thereby improving the utilization rate of the incremental buffer.
On the basis of the above embodiment, the mapping data includes a logical block address, an old physical block address and a new physical block address which are in one-to-one correspondence, so as to improve the compression efficiency of the cache data, and realize the maximum data compression on the premise of ensuring the data validity.
Step 5021, splitting the mapping data into first mapping data and second mapping data; the first mapping data comprises one-to-one corresponding logic block addresses and old physical block addresses, and the second mapping data comprises one-to-one corresponding logic block addresses and new physical block addresses;
step 5022, splitting the first mapping data into a plurality of segments of first sub-data according to the old physical block address continuous information represented by the first mapping data;
step 5023, for any first sub data, generating first compressed sub data corresponding to the first sub data according to the length of the first sub data, the initial logic block address and the first initial sub data of the first sub data;
step 5024, merging the first compressed sub-data corresponding to each first sub-data to obtain first compressed data corresponding to the first mapping data;
step 5025, splitting the second mapping data into a plurality of segments of second sub-data according to the new physical block address continuous information represented by the second mapping data;
step 5026, for any second sub-data, generating second compressed sub-data corresponding to the second sub-data according to the length of the second sub-data, the initial logic block address and the first initial sub-data of the first sub-data;
step 5027, merging the second compressed sub-data corresponding to each second sub-data to obtain second compressed data corresponding to the second mapping data;
and 5028, combining the first compressed data and the second compressed data to obtain compressed mapping data.
Fig. 6 is a schematic structural diagram of compressed sub-data according to the embodiment of the present application. The compressed mapping data at least comprises a start logic block address, a length, start sub data and a sub data type corresponding to each sub data, and one compressed sub data occupies 12 bytes. After the Delta Entry A is subjected to data compression, the space occupation conditions of all structures in the obtained compressed sub-data are shown in the following table:
specifically, in an embodiment, the first mapping data includes a sequence number, and the data in the first mapping data may be traversed from front to back, and the first data in the first mapping data is used as first start sub-data; if the address difference of the old physical block between the traversed data and the first starting sub-data is equal to the serial number difference between the traversed data and the first starting sub-data, determining that the data and the first starting sub-data are continuous data of the address of the old physical block; when traversing to the data with the address difference of the old physical block between the data and the first initial data not equal to the serial number difference between the data and the first initial sub data, taking the data as new first initial sub data; according to each first starting sub-data, the first mapping data is split into a plurality of segments of first sub-data.
The data in each first sub-data is continuous data of the old physical block address.
Similarly, in an embodiment, the second mapping data includes a sequence number, and the data in the second mapping data is traversed from front to back, and the first data in the second mapping data is used as second initial sub-data; if the new physical block address difference between the traversed data and the second initial sub-data is equal to the sequence number difference between the traversed data and the second initial sub-data, determining that the data and the second initial sub-data are new physical block address continuous data; when traversing to the data with the new physical block address difference between the data and the second initial data not equal to the serial number difference between the data and the second initial data, taking the data as new second initial sub-data; and splitting the second mapping data into a plurality of segments of second sub-data according to each second initial sub-data.
The data in each second sub-data is continuous data of the new physical block address.
Exemplary, as shown in the following table, the sequence numbers of 20 mapping data to be written into the increment buffer provided in the embodiment of the present application are 1-20.
Further, the 20 pieces of mapping data are split into first mapping data and second mapping data as follows:
specifically, 1-6 in the first mapping data is a first sub-data, the length of the first sub-data is 6, the initial logical block address is 0x161bf6, the first initial sub-data is 0x4a84ca, 7-20 is another first sub-data, the length of the first sub-data is 14, the initial logical block address is 0x161bfc, and the first initial sub-data is 0x4a86b0. In the second mapping data, 1-16 is a second sub-data, the length of the second sub-data is 16, the initial logical block address is 0x161bf6, the second initial sub-data is 0x509220, 17-20 is another second sub-data, the length of the second sub-data is 4, the initial logical block address is 0x161c07, and the second initial sub-data is 0x509410. The structure of the compressed mapping data obtained by compressing the 20 pieces of mapping data is shown in the following table:
| Start LBA | type(s) | Length of | Start PBA | |
| 1 | 0x161bf6 | 0 | 16 | 0x509220 |
| 2 | 0x161c07 | 0 | 4 | 0x509410 |
| 3 | 0x161bf6 | 1 | 6 | 0x4a84ca |
| 4 | 0x161bfc | 1 | 14 | 0x4a86b0 |
Wherein, start LBA represents a Start logical block address, start PBA represents Start sub data (first Start sub data or second Start sub data), the type is used to characterize whether it is for the first sub data or the second sub data, the first sub data (Old PBA) is represented as 1, and the second sub data (New PBA) is represented as 0. If the 20 mapping data are directly stored in the increment buffer based on the conventional technology, the increment buffer takes 12×20=240 bytes, and the method provided by the embodiment of the application is used for storing the increment only takes 12×4=48 bytes, so that the use efficiency of the increment buffer is greatly improved.
In order to facilitate reproduction of the data processing method provided by the embodiment of the present application by a person skilled in the art, as shown in fig. 7, a code execution flow chart of the data processing method provided by the embodiment of the present application is shown, in which PBA _in is a PBA list (mapping data) to be compressed in a segmented manner is input, list_long is a length of the PBA list (for example, the list length is 20), index records a position (a sequence number corresponding to start sub-data) of a starting PBA of each segment of continuous PBA in the list, and when PBA _in is an Old PBA list (first sub-data) and list_long is 20, index= {4,16}, is output; the input PBA _in is a New PBA column (second sub data), and when list_long is 20, the index= {16,4}, and the flow shown in fig. 7 is an exemplary implementation of the flow shown in fig. 5, and the two implementation principles are the same and are not repeated.
According to the data processing method provided by the embodiment of the application, a plurality of pieces of mapping data to be written into the increment buffer area are obtained; according to the physical block address continuous information represented by the plurality of pieces of mapping data, carrying out data compression on the mapping data to obtain compressed mapping data; and writing the compressed mapping data into an increment buffer area. According to the method provided by the scheme, the mapping data is compressed before being written into the increment buffer zone, and finally the compressed mapping data is written into the increment buffer zone, so that the use efficiency of the increment buffer zone is improved, the brushing times of the mapping data are reduced, and the performance of the SSD is improved. The L2P increment continuous mapping relation is expressed by the mode of the initial address and the length, so that the utilization rate of the increment buffer area is improved, the space occupied by the increment buffer area in the RAM is reduced on one hand, the frequency of writing the L2P mapping table on the NAND during the system operation is reduced on the other hand, the influence on writing performance is reduced, and the service life of the NAND is prolonged.
The embodiment of the application provides a data processing device for executing the data processing method provided by the embodiment.
Fig. 8 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. The data processing device 80 includes: an acquisition module 801, a compression module 802, and a processing module 803.
The acquisition module is used for acquiring a plurality of pieces of mapping data to be written into the increment buffer; the compression module is used for carrying out data compression on the mapping data according to the physical block address continuous information represented by the plurality of pieces of mapping data to obtain compressed mapping data; and the processing module is used for writing the compressed mapping data into the increment buffer area.
The specific manner in which the respective modules perform the operations in the data processing apparatus in this embodiment has been described in detail in the embodiments concerning the method, and will not be described in detail here.
The data processing device provided by the embodiment of the present application is configured to execute the data processing method provided by the foregoing embodiment, and its implementation manner and principle are the same and will not be described again.
The embodiment of the application provides a data processing system for executing the data processing method provided by the embodiment.
FIG. 9 is a schematic diagram of a data processing system according to an embodiment of the present application. The data processing system includes: a Random Access Memory (RAM) and a flash translation layer controller (FTL controller), the random access memory comprising a number of delta buffers and a base buffer.
The flash memory conversion layer controller processes the mapping data to be written into the increment buffer by adopting the data processing method provided by the embodiment.
Specifically, the host device sends a plurality of pieces of mapping data to be written into the increment buffer zone to the flash memory conversion layer controller through the interface, performs data compression based on the data processing method provided by the embodiment after obtaining the mapping data to the flash memory conversion layer controller, and finally saves the compressed mapping data to the increment buffer zone, and extracts the compressed mapping data in the increment buffer zone based on the flash memory conversion layer controller when the data storage amount of the increment buffer zone reaches a preset threshold value; extracting base data in a base buffer; data combination is carried out on the compressed mapping data and the base data to obtain combined mapping data; and finally, storing the combined mapping data to the NAND flash memory of the solid state disk through an interface.
The data processing system provided by the embodiment of the present application is configured to execute the data processing method provided by the foregoing embodiment, and its implementation manner and principle are the same and will not be described again.
The embodiment of the application provides electronic equipment for executing the data processing method provided by the embodiment.
Fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 100 includes: at least one processor 1001 and memory 1002.
The memory stores computer-executable instructions; at least one processor executes computer-executable instructions stored in the memory, causing the at least one processor to perform the data processing method as provided in the above embodiments.
The electronic device provided by the embodiment of the present application is configured to execute the data processing method provided by the foregoing embodiment, and its implementation manner and principle are the same and will not be described again.
The embodiment of the application provides a computer readable storage medium, wherein computer executable instructions are stored in the computer readable storage medium, and when a processor executes the computer executable instructions, the data processing method provided by any embodiment is realized.
The storage medium including the computer executable instructions provided in the embodiments of the present application may be used to store the computer executable instructions of the data processing method provided in the foregoing embodiments, and the implementation manner and principle of the storage medium are the same, and are not repeated.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform part of the steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above. The specific working process of the above-described device may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.
Claims (10)
1. A method of data processing, comprising:
acquiring a plurality of pieces of mapping data to be written into an increment buffer;
according to the physical block address continuous information represented by the mapping data, carrying out data compression on the mapping data to obtain compressed mapping data;
and writing the compressed mapping data into the increment buffer.
2. The method according to claim 1, wherein the mapping data includes a logical block address, an old physical block address, and a new physical block address, the mapping data is data-compressed according to the physical block address continuous information represented by the plurality of pieces of mapping data, and the obtaining the compressed mapping data includes:
splitting the mapping data into first mapping data and second mapping data; the first mapping data comprises one-to-one corresponding logic block addresses and old physical block addresses, and the second mapping data comprises one-to-one corresponding logic block addresses and new physical block addresses;
splitting the first mapping data into a plurality of segments of first sub-data according to the old physical block address continuous information represented by the first mapping data;
for any one of the first sub data, generating first compressed sub data corresponding to the first sub data according to the length of the first sub data, the initial logic block address and the first initial sub data of the first sub data;
combining the first compressed sub data corresponding to each first sub data to obtain first compressed data corresponding to the first mapping data;
splitting the second mapping data into a plurality of segments of second sub-data according to the new physical block address continuous information represented by the second mapping data;
generating second compressed sub-data corresponding to any one of the second sub-data according to the length of the second sub-data, the initial logic block address and the first initial sub-data of the first sub-data;
combining the second compressed sub-data corresponding to each second sub-data to obtain second compressed data corresponding to the second mapping data;
and merging the first compressed data and the second compressed data to obtain compressed mapping data.
3. The method according to claim 2, wherein the first mapping data includes a sequence number, and the splitting the first mapping data into a plurality of segments of first sub-data according to the old physical block address continuation information characterized by the first mapping data includes:
traversing the data in the first mapping data from front to back, and taking the first data in the first mapping data as first initial sub-data;
if the address difference of the old physical block between the traversed data and the first initial sub-data is equal to the serial number difference between the traversed data and the first initial sub-data, determining that the data and the first initial sub-data are continuous data of the old physical block address;
when traversing to the data with the address difference of the old physical block between the first initial data and the data which is not equal to the serial number difference between the two, taking the data as new first initial sub-data;
splitting the first mapping data into a plurality of segments of first sub-data according to each first starting sub-data;
the data in each first sub data is continuous data of the old physical block address.
4. The method according to claim 2, wherein the second mapping data includes a sequence number, the splitting the second mapping data into a plurality of segments of second sub-data according to new physical block address continuous information characterized by the second mapping data includes:
traversing the data in the second mapping data from front to back, and taking the first data in the second mapping data as second initial sub-data;
if the new physical block address difference between the traversed data and the second initial sub-data is equal to the sequence number difference between the traversed data and the second initial sub-data, determining that the data and the second initial sub-data are new physical block address continuous data;
when traversing to the data with the new physical block address difference between the second initial data and the data which is not equal to the serial number difference between the second initial data and the data, taking the data as new second initial sub data;
splitting the second mapping data into a plurality of segments of second sub-data according to each second initial sub-data;
wherein, the data in each second sub data is new physical block address continuous data.
5. The method of claim 2, wherein the compressed mapping data includes at least a start logical block address, a length, a start sub-data, and a sub-data type corresponding to each sub-data.
6. The method according to claim 1, wherein the method further comprises:
when the data storage amount of the increment buffer reaches a preset threshold value, extracting the compressed mapping data in the increment buffer;
extracting base data in a base buffer;
carrying out data combination on the compressed mapping data and the base data to obtain combined mapping data;
and storing the combined mapping data to a flash memory of the solid state disk.
7. A data processing apparatus, comprising:
the acquisition module is used for acquiring a plurality of pieces of mapping data to be written into the increment buffer;
the compression module is used for carrying out data compression on the mapping data according to the physical block address continuous information represented by the plurality of pieces of mapping data to obtain compressed mapping data;
and the processing module is used for writing the compressed mapping data into the increment buffer area.
8. A data processing system, comprising: the flash memory comprises a random access memory and a flash memory conversion layer controller, wherein the random access memory comprises a plurality of increment buffer areas and a base buffer area;
the flash translation layer controller processes the mapping data to be written into the delta buffer using the data processing method according to any one of claims 1 to 6.
9. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the method of any one of claims 1 to 6.
10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311257647.8A CN117193660A (en) | 2023-09-26 | 2023-09-26 | Data processing method, device and system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311257647.8A CN117193660A (en) | 2023-09-26 | 2023-09-26 | Data processing method, device and system, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117193660A true CN117193660A (en) | 2023-12-08 |
Family
ID=88992328
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311257647.8A Pending CN117193660A (en) | 2023-09-26 | 2023-09-26 | Data processing method, device and system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117193660A (en) |
-
2023
- 2023-09-26 CN CN202311257647.8A patent/CN117193660A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107481762B (en) | Trim processing method and device of solid state disk | |
| US10761731B2 (en) | Array controller, solid state disk, and method for controlling solid state disk to write data | |
| CN105573681A (en) | Method and system for establishing RAID in SSD | |
| US20200183831A1 (en) | Storage system and system garbage collection method | |
| CN109918352B (en) | Memory system and method of storing data | |
| CN114676072A (en) | Data processing method and device | |
| KR20070031647A (en) | Space-Efficient Management Method of Compressed Data in Flash Memory Storages | |
| KR101077901B1 (en) | Apparatus and method for managing flash memory using log block level mapping algorithm | |
| KR101936364B1 (en) | Memory management system using flash memory and method thereof | |
| CN112052193B (en) | Garbage recycling method and device, readable storage medium and electronic equipment | |
| CN103645995B (en) | Write the method and device of data | |
| CN112905496A (en) | Garbage recycling method and device, readable storage medium and electronic equipment | |
| US20230142948A1 (en) | Techniques for managing context information for a storage device | |
| CN117193660A (en) | Data processing method, device and system, electronic equipment and storage medium | |
| CN115587050A (en) | Mapping table garbage recovery efficiency method and device, computer equipment and medium | |
| CN110658999B (en) | Information updating method, device, equipment and computer readable storage medium | |
| CN114625318A (en) | Data writing method, device and equipment applied to solid state disk | |
| CN110389706B (en) | Fingerprint recovery method and storage system | |
| CN111796757B (en) | Solid state disk cache region management method and device | |
| CN107526548B (en) | Multi-namespace solid state disk and data deleting method and device | |
| KR100994052B1 (en) | Data management method in flash translation layer and flash memory apparatus performing the same | |
| CN116010113B (en) | Method for managing memory of smart card and smart card | |
| CN111611179B (en) | Metadata hit rate improving method and device, storage medium and electronic equipment | |
| CN106201326A (en) | Information processor | |
| CN102789421B (en) | Method and device for improving read-write performance of NAND flash memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |