CN103645991B - Data processing method and device - Google Patents

Abstract

The invention discloses a data processing method and device. The method includes: obtaining the first writing instruction sent by the operating system, then determining the first heat value of the first storage address according to the first writing instruction, and then determining the first heat value of the first storage address according to the The first heat value, write data into the second storage address, so as to realize the storage of data with close heat value in the same physical block, thereby reducing the movement of valid data, improving the performance of the system, and reducing the system Additional loss increases the life of the system.

Description

Data processing method and device

technical field

The invention relates to the field of computer technology, in particular to a data processing method and device.

Background technique

Through the statistics of online business data and personal computer data visits, it can be known that 80% of the visits are concentrated in 20% of the space.

In the existing technology, the reading and writing of the storage medium NAND FLASH is in the unit of page, and the erasure is in the unit of block. When the data is accessed once, the old data at the original physical address is marked as invalid, and the system finds a new one. A blank page that has not been written is used to store the updated data, and then the logical address is pointed to the new page, thus completing the data update process. As the data is continuously updated, the system can start the garbage collection mechanism (GC) to integrate the valid data that has not been updated on each block, and write the data block in the page into another new blank block, and then , to erase the data block as a new blank block for the next write.

However, the storage system can start the garbage collection mechanism (GC) to integrate the valid data that has not been updated on each block. There are both invalid data and valid data in the data block. As a result, when the garbage collection mechanism is started, valid Data is moved to other free blocks, resulting in additional read and write operations in the system, resulting in loss of the storage system.

Contents of the invention

Embodiments of the present invention provide a data processing method and device for reducing loss of a storage system.

In the first aspect, the embodiment of the present invention provides a data processing method, including:

Obtaining a first write instruction sent by an operating system, the first write instruction is an instruction for preparing to write data into a storage medium, and the first write instruction carries the data;

According to the first write instruction, determine the first temperature value of the first storage address, the first storage address is the address where the first write instruction prepares to write the data into the storage medium, the The first heat value identifies the frequency of operations on the first storage address;

Writing the data into a second storage address of the storage medium according to the first heat value, where the first heat value is within a range of the second heat value corresponding to the second storage address.

In a first possible implementation manner, the determining the first temperature value of the storage address according to the first write instruction includes:

determining the first storage address according to the first write instruction;

According to the first storage address, determine a first heat value of the first storage address.

With reference to the first aspect or the first possible implementation of the first aspect, in the second possible implementation, the first write instruction includes the start address of the data to be written and the length of said data;

The determining the first heat value of the first storage address according to the first write instruction includes:

According to the start address of the data and the length of the read-write data, determine the address of the page unit to write the data into the logical page unit in the storage medium;

A first temperature value of the logical page unit is determined according to the page unit address.

With reference to the second possible implementation of the first aspect, in a third possible implementation, the determining the first heat value of the logical page unit according to the address of the page unit includes:

Add the address of the page unit to the position of the link head in the thermal weight linked list, the thermal weight linked list includes a plurality of subsections, and each subsection corresponds to a thermal weight, and each subsection includes each The information of the logical page unit operated by an operation instruction, wherein the hot weight linked list includes the link head and the link tail;

Adding and summing the thermal weight corresponding to each page unit address in the thermal weight linked list to obtain the first thermal value.

With reference to the third possible implementation of the first aspect, in a fourth possible implementation, the adding the page address information of the logical page unit before the link head position in the heat weight linked list ,Also includes:

Delete the page unit address stored in the chain tail in the hot weight linked list;

Moving the address of the page unit that is not deleted in the hot weight linked list to the position of the chain tail by one node.

With reference to any one of the fourth possible implementation manners from the first aspect to the first aspect, in a fifth possible implementation manner, according to the first thermal value, writing the data into the second storage After the address, also include:

Send the information of the second storage address to the operating system.

In a second aspect, an embodiment of the present invention provides a data processing device, including:

An acquisition module, configured to acquire a first write instruction sent by an operating system, the first write instruction is an instruction to write data into a storage medium, and the first write instruction carries the data;

A processing module, configured to determine a first heat value of a first storage address according to the first write instruction, and the first storage address is prepared for the first write instruction to write the data into the storage medium address, the first heat value identifies the frequency of operations on the first storage address;

A writing module, configured to write the data into a second storage address of the storage medium according to the first temperature value, the first temperature value is at the second temperature corresponding to the second storage address within the value range.

In a first possible implementation manner, the processing module is specifically configured to determine the first storage address according to the first write instruction; determine the first storage address according to the first storage address The first heat value of .

With reference to the second aspect or the first possible implementation of the second aspect, in the second possible implementation, the first write instruction includes the start address of the data to be written and the length of said data;

The processing module is specifically configured to determine, according to the start address of the data and the length of the read/write data, a page unit address to be written into a logical page unit in the storage medium; according to the The address of the page unit is used to determine the first thermal value of the logical page unit.

With reference to the second possible implementation of the second aspect, in a third possible implementation, the processing module is further configured to add the address of the page unit to the position of the link head in the heat weight linked list, The thermal weight linked list includes multiple subsections, and each subsection corresponds to a thermal weight, and each subsection includes information on the logical page unit operated by each operation instruction, wherein the thermal weight The value link table includes the link head and the link tail; the heat weight corresponding to each page unit address in the heat weight list is added and summed to obtain the first heat value.

With reference to the third possible implementation of the second aspect, in a fourth possible implementation, the processing module is further configured to delete the page stored in the chain tail in the hot weight linked list Unit address: move the unit address of the page that has not been deleted in the hot weight linked list to the position of the end of the chain by one node.

Combining the second aspect to any one of the fourth possible implementation manners of the second aspect, in a fifth possible implementation manner, further comprising: a sending module;

The sending module is configured to send the information of the second storage address to the operating system.

In the data processing method and device provided by the embodiments of the present invention, by obtaining the first write command sent by the operating system, and then determining the first heat value of the first storage address according to the first write command, and then, according to the first heat value value, and write the data into the second storage address, thereby realizing the storage of data with close heat values in the same physical block, improving the efficiency of garbage collection by the system, and improving the reliability of each physical block, thereby reducing loss of the storage system.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a flowchart of a data processing method provided by an embodiment of the present invention;

FIG. 2 is a flowchart of another data processing method provided by an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a heat weight linked list provided by an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a data processing device provided by an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of another data processing device provided by an embodiment of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The data processing method provided in this embodiment of the present invention can be applied to the process of storing written data in a storage system, wherein the data processing method provided in this embodiment can be specifically executed by a data processing device, and the data processing device can use software and /or implemented by hardware. The data processing method and device provided in this embodiment will be described in detail below.

FIG. 1 is a flowchart of a data processing method provided by an embodiment of the present invention. As shown in Figure 1, the data processing method provided in this embodiment includes:

S101. Acquire a first write instruction sent by an operating system.

In this embodiment, when the operating system sends the first write instruction, the data processing device may obtain the first write instruction sent by the operating system at a flash translation layer (Flash translation layer, hereinafter referred to as FTL). Wherein, the first write instruction is an instruction for preparing to write data into the storage medium, and the first write instruction includes the data, that is, the data to be written into the storage medium.

S102. Determine a first heat value of the first storage address according to the first write instruction.

In this embodiment, the data processing device may determine the first heat value of the first storage address according to the first write instruction, wherein the first storage address is the address where the first write instruction prepares to write data into the storage medium, The first popularity value identifies the frequency of operations on the first storage address. That is to say, within a preset time period, or within a preset number of operations on the storage medium, the data processing device can determine the first heat value of the operation on the first storage address, that is, the operating system's operation on the first storage address. Operation frequency, the operation can be understood as reading the data stored in the first storage address.

The first storage address in this embodiment may be at least one address in the storage medium, and the address may be a physical address or a logical address. It should be noted that the logical address is visible to the user, and the physical address corresponding to the logical address is obtained through the correspondence between the logical address and the physical address, that is, through the mapping table between the logical address and the physical address. The physical address is The actual address of the storage medium.

S103. Write data into a second storage address of the storage medium according to the first heat value.

The first heat value in this embodiment is within the range of the second heat value corresponding to the second storage address.

Specifically, the data processing device may determine the frequency of operations on the first address by the instructions sent by the operating system within a preset time or within a preset number of times, that is, the first heat value, and then write the first The data to be written is instructed to be written into the second address corresponding to the first heat value. Further, the heat value of the written data corresponding to the first address may be determined according to the lifetime of the memory block where the second address is located. For example, if the first heat value is higher, the data may be written into an address with a shorter lifetime. It realizes storing data with similar thermal values in the same physical block, so that when the system erases and writes the same physical block, it does not need to move part of the data, so that the erasing and writing life of the same physical block is close.

The data processing method provided in this embodiment obtains the first write command sent by the operating system, and then determines the first heat value of the first storage address according to the first write command, and then, according to the first heat value, writes the The data is written into the second storage address, which realizes the storage of data with close heat value in the same physical block, thereby reducing the movement of valid data and improving the performance of the system, thereby reducing the additional loss of the system and improving the system performance. life.

On the basis of the foregoing embodiments, after S103, it may further include:

Send the information of the second storage address to the operating system.

In this embodiment, the second storage address to which the data is actually written may be the same as or different from the first storage address to be written by the first write instruction. After the second storage address, the information of the second storage address needs to be sent to the operating system, so as to determine the actual storage address of the data when the data is read in the future.

Further, on the basis of the foregoing embodiments, S102 may specifically include:

According to the first write instruction, the first storage address is determined.

According to the first storage address, a first popularity value of the first storage address is determined.

For example, the first write instruction may include the first address of the data to be written into the storage medium and the length of the data. For example, the first address is a logical address. If it is necessary to write the data into multiple In the logical address, according to the first address and the length of the data, it is possible to determine the logical address of the data to be written into the storage medium, that is, the first storage address, and then, the data processing device can determine the first storage address according to the first storage address. Stores the first heat value of the address.

FIG. 2 is a flowchart of another data processing method provided by an embodiment of the present invention. FIG. 3 is a schematic structural diagram of a heat weight linked list provided by an embodiment of the present invention. As shown in Figure 2, the data processing method provided in this embodiment may specifically include:

S201. Acquire a first write instruction sent by an operating system.

In this embodiment, the first write instruction is an instruction for preparing to write data into the storage medium, and the first write instruction includes the data, that is, the data to be written into the storage medium.

S202. Determine, according to the start address of the data and the length of the data, a page unit address to be written into a logical page unit in the storage medium.

In this embodiment, the first write instruction may include the first address of the data to be written into the storage medium and the length of the data. The address is a logical address. In the address, the logical address of the data to be written into the storage medium, that is, the page unit address of the logical page unit, can be determined according to the first address and the length of the data.

It should be noted that the storage medium in this embodiment includes at least one data block for storing data, and the size of the data block that can be used for storing data is fixed, and each data block includes at least one page (page), and, Each page includes multiple logical page units, and each logical page unit is represented by a logical address, that is, the page unit address. For example, each page in NAND FLASH includes 16 page units, each page is 8K, and each A page unit (sector) is 512B, and the addresses of the 16 page units are represented by 16 consecutive logical LBA addresses.

S203. Add the address of the page unit to the position of the link head in the heat weight linked list.

In this embodiment, as shown in FIG. 3 , the heat weight linked list includes a plurality of subsections, and each subsection may correspond to a heat weight value, and each subsection includes the address of the page unit to be operated by each operation instruction, wherein, The hot weight linked list includes a chain head and a chain tail.

For example, if there are multiple page unit addresses, the multiple addresses included in the page unit address are respectively added to the subsections of the chain header. As shown in FIG. 3, page unit addresses 70, 71, 72 are added in subsections of the chain header.

S204. Add and sum the heat weights corresponding to each page unit address in the heat weight linked list to obtain a first heat value.

In this embodiment, the page unit address is an address where the first write command prepares to write data into the storage medium.

For example, the thermal weights corresponding to the page unit address 70 are 4, 6, and 7 respectively. Therefore, the thermal weight of the page unit address 70 is 4+6+7, that is, the thermal weight is 17; and so on, Calculate the thermal weight of the page unit address.

It should be noted that each heat weight in the heat weight linked list of this embodiment can be set according to the distance from the chain head or the chain tail, or can be set according to the arithmetic sequence or the geometric sequence, and the technology in the art It can be set in any way that personnel can think of, and there is no limit to the heat weight setting here.

S205. Write data into the second storage address of the storage medium according to the first temperature value.

The first heat value in this embodiment is within the range of the second heat value corresponding to the second storage address.

In this embodiment, by determining the first heat value of the first storage address according to the first write instruction, and writing data into the second storage address of the storage medium according to the first heat value, the heat value Close data is stored in the same physical block, which improves the efficiency of garbage collection by the system, and improves the service life of each physical block, thereby reducing the loss of the storage system.

On the basis of the foregoing embodiments, before S203, it may also include:

Delete the page unit address stored in the tail of the chain in the hot weight linked list;

Move the address of the page unit that has not been deleted in the hot weight linked list to the position of the end of the link by one node.

Further, on the basis of the above embodiments, after S205, it may also include

Send the information of the second storage address to the operating system.

Specifically, after writing data into the second storage address according to the first heat value, the data processing device needs to inform the operating system of the second storage address where the data is actually stored, so that the operating system can determine that the first write instruction actually writes data the address of.

FIG. 4 is a schematic structural diagram of a data processing device provided by an embodiment of the present invention. As shown in FIG. 4 , the data processing device includes: an acquisition module 41 , a processing module 42 and a writing module 43 . in,

The acquisition module 41 is configured to acquire a first write instruction sent by the operating system, the first write instruction is an instruction for preparing to write data into the storage medium, and the first write instruction carries data;

The processing module 42 is configured to determine the first temperature value of the first storage address according to the first write instruction, the first storage address is the address where the first write instruction prepares to write data into the storage medium, and the first temperature value is an identifier the frequency of operating on the first storage address;

The writing module 43 is configured to write data into the second storage address of the storage medium according to the first temperature value, and the first temperature value is within the range of the second temperature value corresponding to the second storage address.

The data processing method provided in this embodiment obtains the first write command sent by the operating system, and then determines the first heat value of the first storage address according to the first write command, and then, according to the first heat value, writes the The data is written to the second storage address, thereby realizing the storage of data with close heat values in the same physical block, thereby reducing the movement of valid data and improving the performance of the system, thereby reducing the additional loss of the system and improving the system performance. lifespan.

On the basis of the above embodiments, the processing module 42 is specifically configured to determine the first storage address according to the first write instruction; and determine the first heat value of the first storage address according to the first storage address.

Further, on the basis of the above embodiments, the first write instruction includes the start address of the data to be written and the length of the data;

The processing module 42 is specifically configured to determine, according to the start address of the data and the length of the read/write data, a page unit address to be written into a logical page unit in the storage medium; according to the The page unit address determines the first hotness value of the logical page unit.

On the basis of the above-mentioned embodiments, the processing module 42 is further configured to add the page unit address to the position of the link head in the thermal weight linked list, the thermal weight linked list includes multiple subsections, and each subsection corresponds to a thermal weight, Each subsection includes the information of the logical page unit operated by each operation instruction, wherein the heat weight linked list includes a chain head and a chain tail; add the heat weight corresponding to each page unit address in the heat weight linked list and, get the first heat value.

Further, on the basis of the above-mentioned embodiment, the processing module 42 is also used to delete the page unit address stored in the tail of the chain in the hot weight linked list; Move a node in position.

FIG. 5 is a schematic structural diagram of another data processing device provided by an embodiment of the present invention. As shown in FIG. 5 , the data processing device further includes: a sending module 44; wherein, the sending module 44 is configured to send the information of the second storage address to the operating system.

In this way, data with similar heat values can be stored in the same physical block, which improves the efficiency of garbage collection by the system, improves the reliability of each physical block, and reduces the loss of the storage system.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiments. The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

Those of ordinary skill in the art can understand that: the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description in the embodiment, and can also be changed and located in one or more devices different from the embodiment. The units in the above embodiments can be combined into one unit, and can also be further divided into multiple sub-modules. If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including instructions for making a computer device

(It may be a personal computer, a server, or a network device, etc.) Execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of an embodiment, and the modules or processes in the accompanying drawing are not necessarily necessary for implementing the present invention.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A data processing method, characterized in that, comprising: Obtaining a first write instruction sent by an operating system, the first write instruction is an instruction for preparing to write data into a storage medium, and the first write instruction carries the data; According to the first write instruction, determine the first temperature value of the first storage address, the first storage address is the address where the first write instruction prepares to write the data into the storage medium, the The first heat value identifies the frequency of operations on the first storage address; Writing the data into a second storage address of the storage medium according to the first heat value, where the first heat value is within a range of a second heat value corresponding to the second storage address; The first write instruction includes the start address of the data to be written and the length of the data; the determining the first heat value of the first storage address according to the first write instruction includes: determining, according to the start address of the data and the length of the data, a page unit address for writing the data into a logical page unit in the storage medium; determining a first thermal value of the logical page unit according to the page unit address; Wherein, the determining the first heat value of the logical page unit according to the page unit address includes: Add the address of the page unit to the position of the link head in the thermal weight linked list, the thermal weight linked list includes a plurality of subsections, and each subsection corresponds to a thermal weight, and each subsection includes each The information of the logical page unit operated by an operation instruction, wherein the hot weight linked list includes the link head and the link tail; Adding and summing the thermal weight corresponding to each page unit address in the thermal weight linked list to obtain the first thermal value. 2. The method according to claim 1, wherein the determining the first heat value of the storage address according to the first write instruction comprises: determining the first storage address according to the first write instruction; According to the first storage address, determine a first heat value of the first storage address. 3. The method according to claim 1 or 2, wherein adding the page unit address before the position of the link head in the heat weight linked list further includes: Delete the page unit address stored in the chain tail in the hot weight linked list; Moving the address of the page unit that is not deleted in the hot weight linked list to the position of the chain tail by one node. 4. The method according to claim 2, characterized in that, after writing the data into the second storage address according to the first heat value, further comprising: Send the information of the second storage address to the operating system. 5. The method according to claim 3, characterized in that, after writing the data into the second storage address according to the first heat value, further comprising: Send the information of the second storage address to the operating system. 6. A data processing device, characterized in that, comprising: An acquisition module, configured to acquire a first write instruction sent by an operating system, the first write instruction is an instruction to write data into a storage medium, and the first write instruction carries the data; A processing module, configured to determine a first heat value of a first storage address according to the first write instruction, and the first storage address is prepared for the first write instruction to write the data into the storage medium address, the first heat value identifies the frequency of operations on the first storage address; A writing module, configured to write the data into a second storage address of the storage medium according to the first heat value, where the first heat value is equal to the second heat value corresponding to the second storage address within the scope; The first write instruction includes the start address of the data to be written and the length of the data; the processing module is specifically configured to , determining a page unit address for writing the data into a logical page unit in the storage medium; determining a first temperature value of the logical page unit according to the page unit address; Wherein, the processing module is further configured to add the page unit address to the position of the link head in the thermal weight linked list, the thermal weight linked list includes a plurality of subsections, and each subsection corresponds to a thermal weight value, each of the subsections includes the information of the logical page unit operated by each operation instruction, wherein the hot weight linked list includes the linked head and the linked tail; The heat weights corresponding to the page unit addresses are added and summed to obtain the first heat value. 7. The device according to claim 6, wherein the processing module is specifically configured to determine the first storage address according to the first write instruction; determine the first storage address according to the first storage address The first temperature value of the first storage address. 8. The device according to claim 6 or 7, wherein the processing module is further configured to delete the page unit address stored in the chain tail in the hot weight linked list; The address of the page unit that has not been deleted in the weight linked list is moved to the position of the end of the link by one node. 9. The device according to claim 6 or 7, further comprising: a sending module; The sending module is configured to send the information of the second storage address to the operating system. 10. The device according to claim 8, further comprising: a sending module; The sending module is configured to send the information of the second storage address to the operating system.