WO2016095761A1 - Procédé et appareil de traitement de cache - Google Patents

Procédé et appareil de traitement de cache Download PDF

Info

Publication number
WO2016095761A1
WO2016095761A1 PCT/CN2015/097176 CN2015097176W WO2016095761A1 WO 2016095761 A1 WO2016095761 A1 WO 2016095761A1 CN 2015097176 W CN2015097176 W CN 2015097176W WO 2016095761 A1 WO2016095761 A1 WO 2016095761A1
Authority
WO
WIPO (PCT)
Prior art keywords
data block
large data
storage space
cache image
count value
Prior art date
Application number
PCT/CN2015/097176
Other languages
English (en)
Chinese (zh)
Inventor
李明君
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2016095761A1 publication Critical patent/WO2016095761A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/12—Replacement control

Definitions

  • the embodiments of the present invention relate to a storage technology, and in particular, to a cache processing method and apparatus.
  • the desktop cloud separates the personal computer desktop environment from the physical machine through the cloud computing mode, and becomes a service that can provide desktops externally.
  • FIG. 1 is a schematic diagram of an application scenario of a desktop cloud in the prior art.
  • the application scenario includes a parent image 1 , a cache image 2 , a sub mirror 3 , and a virtual machine 4 .
  • the cache image 2 link is cloned in the parent image 1 and the sub mirror 3 link is cloned in the cache image 2 .
  • FIG. 1 shows that as shown in FIG.
  • the write IO directly writes the data block into the sub-image 3; the read IO first reads the data block into the sub-image, if the data block If it exists in the submirror, it is read in the submirror; if the data block does not exist in the submirror, it is read in the cache image; if the data block does not exist in the cache image, it is read in the parent image. The read data block is written to the cache image so that the data block can be read from the cache image the next time the same data block is read.
  • the parent image, the cache image, and the sub-mirror are in the virtual hard disk format (VHD).
  • VHD virtual hard disk format
  • the maximum size of the VHD file does not exceed the size of the local storage file system, and the VHD cannot perform the data block replacement function.
  • the data blocks read from the parent image can only be written to the cache image in the same way. If the local storage file system space where the cache image resides is used up, the new data block cannot be written to the cache. In the image, some data in the cache image may not be read later or rarely, so the proportion of useful data blocks in the cache image is reduced, resulting in wasted cache space.
  • the embodiment of the invention provides a method and a device for processing a cache, which replaces the second largest data block that is aged with a new first large data block, and then writes the small data block that needs to be written into the cache image, thereby effectively improving The utilization of storage space.
  • a first aspect of the embodiments of the present invention provides a method for processing a cache, including:
  • the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to the preset ratio threshold, replacing the first large data block with the second largest data block in the cache image, And writing the small data block to a storage space corresponding to the first large data block in the cache image.
  • the method further includes:
  • the method further includes:
  • the identifier of the first large data block and the corresponding access count value are written into the management linked list.
  • the method further includes:
  • the replacing the first large data block with the second largest data block in the cache image, and The small data block is written into the storage space corresponding to the first large data block in the cache image, and specifically includes:
  • the starting offset position of the first large data block is the starting offset position of the second large data block;
  • the first large data block is added to the cache image, and the small data block is written
  • the storage space corresponding to the first large data block in the cache image includes:
  • a fifth possible implementation manner of the first aspect if the first large data block is in a cache image, the management is performed according to the access count value. Updating a location of the identifier of the first large data block and the access count value in a linked list, and writing the small data block to a storage space corresponding to the first large data block in the cache image.
  • a second aspect of the embodiments of the present invention provides a cache processing apparatus, including:
  • a reading module configured to receive a read input and output IO request, and read a small data block from the parent image according to the read IO request;
  • the checking module is configured to: if the first large data block that includes the small data block is not in the cache image, check whether the storage space of the cache image is full or whether the occupied ratio of the storage space is greater than or equal to a preset ratio Threshold value
  • a processing module if the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to the preset ratio threshold, replacing the first large data block with the first one of the cache image Two large data blocks, and writing the small data block to the first big data in the cache image The storage space corresponding to the block.
  • the reading module is further configured to increase an access count value of the first large data block that includes the small data block by one;
  • the processing module is further configured to write the identifier of the first large data block and the corresponding access count value into the management linked list.
  • the processing module is further configured to: if the storage space of the cache image is not full or the proportion of the storage space is occupied If the threshold is smaller than the preset ratio, the first large data block is added to the cache image, and the small data block is written into a storage space corresponding to the first large data block in the cache image.
  • the processing module includes:
  • a second large data block obtaining unit configured to sequentially search from the header position of the management linked list according to an access count value of the first large data block, acquire the second large data block, and use the second
  • the identification and access count value of the big data block is removed from the management linked list, and the identifier and the access count value of the first large data block are written before the identifier of the third large data block in the management linked list;
  • the second large data block is the first data block in the management linked list, and the access count value is less than or equal to the access count value of the first large data block; the third large data block is deleted.
  • the access data is greater than or equal to the first data block of the access count value of the first large data block;
  • a location marking unit configured to invalidate a location identifier corresponding to the second large data block in the BAT, and store a starting offset location of the first large data block to a location corresponding to the first large data block in the BAT;
  • the starting offset position of the first large data block is a starting offset position of the second large data block;
  • a small data block writing unit configured to write the small data block into a storage space corresponding to a starting offset position of the first large data block in the cache image.
  • the processing module includes:
  • An identifier writing unit configured to write an identifier of the first large data block and an access count value into the management linked list according to an access count value of the first large data block;
  • a free block obtaining unit configured to find a free block in the cache image, and write the small data block into the free block; wherein the free block is a storage corresponding to the first large data block Empty between;
  • a location writing unit configured to write a starting offset position of the free block to a location corresponding to the free block in the BAT.
  • the processing module includes:
  • an update unit configured to: when the first large data block is in the cache image, update the location of the identifier of the first large data block and the access count in the management link table according to the access count value And storing the small data block into a storage space corresponding to the first large data block in the cache image.
  • the processing method of the cache receives a read input and output IO request, and reads a small data block from the parent image according to the read IO request, and if the first large data block including the small data block is not in the cache image, Check whether the storage space of the cache mirror is full or the proportion of storage space is greater than or equal to the preset ratio threshold. If the storage space of the cache mirror is full or the storage ratio is greater than or equal to the preset ratio threshold, it will be the first.
  • the big data block replaces the second largest data block in the cache image, and writes the small data block into the storage space corresponding to the first largest data block in the cache image.
  • the new data block can no longer be written into the cache image.
  • the storage space in the cache image is full.
  • the occupation ratio of the storage space is greater than or equal to the preset ratio threshold, replace the second largest data block in the cache image with the first large data block, and write the small data block into the storage corresponding to the second largest data block in the cache image.
  • Space that is, replace the aging second largest data block with the new first large data block, and then write the small data block that needs to be written into the cache image, which effectively improves the utilization of the storage space.
  • FIG. 1 is a schematic diagram of an application scenario of a desktop cloud in the prior art
  • FIG. 2 is a flowchart of a method for processing a cache according to Embodiment 1 of the present invention
  • FIG. 3 is a flowchart of a method for processing a cache according to Embodiment 2 of the present invention.
  • FIG. 4 is a schematic diagram of a management chain table before the identification of the first big data block moves
  • FIG. 5 is a schematic diagram of a management chain table after the identification of the first large data block is moved
  • Figure 6 is a schematic diagram of bit chart management
  • FIG. 7 is a flowchart of a method for processing a cache according to Embodiment 3 of the present invention.
  • FIG. 8 is a schematic diagram of a management link table before the first large data block enters the cache image in the third embodiment
  • FIG. 9 is a schematic diagram of a management chain table after the first large data block enters the cache image in the third embodiment
  • FIG. 10 is a flowchart of a method for processing a cache according to Embodiment 4 of the present invention.
  • FIG. 11 is a schematic diagram of a management chain table before the first large data block enters the cache image in the fourth embodiment
  • FIG. 12 is a schematic diagram of a management chain table after the first large data block enters the cache image in the fourth embodiment
  • FIG. 13 is a schematic structural diagram of a cache processing apparatus according to Embodiment 5 of the present invention.
  • FIG. 14 is a schematic structural diagram of a cache processing apparatus according to Embodiment 6 of the present invention.
  • FIG. 15 is a schematic structural diagram of a cache processing apparatus according to Embodiment 7 of the present invention.
  • FIG. 16 is a schematic structural diagram of a cache processing apparatus according to Embodiment 8 of the present invention.
  • FIG. 2 is a flowchart of a method for processing a cache according to Embodiment 1 of the present invention.
  • the executor may be a central processing unit (CPU), a server, a physical host, a terminal device, etc., and is not limited thereto.
  • the method includes the following steps:
  • Step 101 Receive an Input and Output (IO) request, and read a small data block from the parent image according to the read IO request.
  • IO Input and Output
  • the small data block needs to be read from the parent image according to the read IO request.
  • the read IO request may include an identifier of the small data block, for example, a number of the small data block or a starting offset address of the small data block storage.
  • a large data block can be divided into multiple small data blocks. For example, 512B can be used as a small data block and 2MB as a large data block.
  • a 2MB data block can also be used as a small data block, and a 20 GB data block can be used as a big data block.
  • the block can be set according to actual conditions, and is not limited in the present invention.
  • Step 102 If the first large data block that includes the small data block is not in the cache image, check whether the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to a preset ratio threshold.
  • the first large data block including the small data block is not in the cache image, and it is indicated that any small data block in the first large data block is stored in the storage space of the cache image, if the first big data At least one small data block in the block is stored in the storage space of the cache image, and the first large data block is in the cache image.
  • the storage space of the cache image can be a local high-speed storage device.
  • the storage speed and the input and output per second (IOPS) are much larger than those of the ordinary hard disk. It can be either a memory or a solid state disk.
  • a storage device that can perform high-speed interoperability, such as Solid State Disk (SSD), can select a suitable storage device for caching from the perspective of cost, performance, and life cycle.
  • SSD Solid State Disk
  • Step 103 If the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to a preset ratio threshold, replace the first large data block with the second largest data block in the cache image, and write the small data block.
  • the storage space of the cached image is full, that is, the storage space cannot be stored in an additional manner, or if the storage ratio of the cached image is greater than or equal to a preset ratio threshold, for example,
  • a preset ratio threshold for example, When the occupation ratio of the storage space exceeds 80%, the first large data block is replaced with the second largest data block in the cache image, and the small data block is written into the storage space corresponding to the first large data block in the cache image.
  • the second largest data block is a large data block in the cache image that is accessed relatively small times, or a large data block that has not been accessed within a preset time period, or may be accessed less than or equal to the number of times.
  • a preset cache replacement policy may be used to replace the second largest data block in the cache image with the first large data block.
  • the cache replacement policy may be a Least Recently Used (LRU) policy, or a Least Frequently Used (LFU) policy, or other policies, and those skilled in the art may
  • LRU Least Recently Used
  • LFU Least Frequently Used
  • an appropriate cache replacement strategy is selected, which is not limited in this invention.
  • an aging time with a relatively small number of times of being accessed is found in the cache image.
  • the aging large data block is used as the second largest data block, and the storage space of the second largest data block in the cache image is reserved to the first large data block, and the small data in the first large data block is used.
  • the small data block is written to the storage space corresponding to the original second largest data block in the cache image.
  • the processing method of the cache receives a read input and output IO request, and reads a small data block from the parent image according to the read IO request, and if the first large data block including the small data block is not in the cache image, Check whether the storage space of the cache mirror is full or the proportion of storage space is greater than or equal to the preset ratio threshold. If the storage space of the cache mirror is full or the storage ratio is greater than or equal to the preset ratio threshold, it will be the first.
  • the big data block replaces the second largest data block in the cache image, and writes the small data block into the storage space corresponding to the first largest data block in the cache image.
  • the new data block can no longer be written into the cache image.
  • the storage space in the cache image is full.
  • the occupation ratio of the storage space is greater than or equal to the preset ratio threshold, replace the second largest data block in the cache image with the first large data block, and write the small data block into the storage corresponding to the second largest data block in the cache image.
  • Space that is, replace the aging second largest data block with the new first large data block, and then write the small data block that needs to be written into the cache image, which effectively improves the utilization of the storage space.
  • FIG. 3 is a flowchart of a method for processing a cache according to Embodiment 2 of the present invention. As shown in FIG. 3, the method includes the following steps:
  • Step 201 Receive a read input and output IO request, and read a small data block from the parent image according to the read IO request.
  • Step 202 Add an access count value of the first large data block including the small data block by one.
  • the access count value is used to record the number of times the first large data block is accessed, when there is a small data block in the first large data block, regardless of whether the first large data block is in the cache image, Add 1 to the access count value of the first large data block.
  • Step 203 Determine whether the first large data block that includes the small data block is in the cache image. If yes, that is, the first large data block that includes the small data block is in the cache image, step 208 is performed; if not, the small data is included. If the first large data block of the block is not in the cache image, step 204 is performed.
  • Step 204 Write the identifier of the first large data block and the corresponding access count value into the management linked list.
  • the management linked list is used to manage large data blocks in the cache image and store big data.
  • the identifier of the block and the access count value of the big data block, and the management chain table may specifically be an LFU linked list.
  • the head node points to the most recent big block with the smallest access count
  • the tail point points to the oldest block with the largest access count.
  • the first data block is in the cache image, that is, the small data block in the first large data block is stored in the storage space of the cache image, if there is no management list.
  • the identifier of the first large data block is not in the cache image, that is, no small data block in the first large data block is stored in the storage space of the cache image.
  • the identifier of the first large data block and the corresponding access count value are written into the management linked list, indicating that the first largest The data block is in the cache image.
  • the access count value of all large data blocks is set to 0, and these large data blocks are managed by the management linked list. If there are small data blocks in the large data block into the cache image, that is, the small data blocks included in the first large data block.
  • the first big block identifier and the access count value are added to the management list.
  • Step 205 Check whether the storage space of the cached image is full or the proportion of the storage space is greater than or equal to a preset ratio threshold. If yes, the storage space of the cached image is full or the occupied proportion of the storage space is greater than or equal to a preset ratio threshold. Then, step 206 is performed; if no, that is, the storage space of the cache image is not full or the occupation ratio of the storage space is less than the preset ratio threshold, step 207 is performed.
  • Step 206 Replace the first large data block with the second largest data block in the cache image, and write the small data block into the storage space corresponding to the first large data block in the cache image.
  • Step 207 Add the first large data block to the cache image, and write the small data block into the storage space corresponding to the first large data block in the cache image. End.
  • the first large data block can be added to the cache image by using two methods, one is preemption in advance, and the other is dynamic allocation.
  • the preemption method is to occupy the cache space in advance when the first large data block is cached and added to the cache, regardless of whether the small data block in the first large data block can be accessed and stored in the storage space of the cache image.
  • the space required for the first largest data block in the middle (such as 2MB); the dynamic allocation method allocates only the space required for small data blocks in the storage space (such as several 512B) when the first large data block is added to the cache image. This way you can make more useful small data blocks into the cache image.
  • Step 208 Update, according to the access count value, a location and an access count value of the identifier of the first large data block in the management linked list, and write the small data block into the storage space corresponding to the first large data block in the cache image. between.
  • the management link list when the first large data block is in the cache image, that is, the management link list has the identifier and the access count value of the first large data block, according to the access count value, in the management linked list, from the first big data.
  • the location of the identified identity of the block begins, the first large data block whose access count value is less than or equal to the access count value of the first large data block is found, and the identifier of the first large data block and the access count value are written to the first large Before the data block.
  • FIG. 4 is a schematic diagram of a management chain table before the identification of the first large data block moves
  • FIG. 5 is a schematic diagram of the management chain table after the identification of the first large data block moves.
  • BLOCK1 big data block 1
  • BLOCK6 big data block 6
  • BLOCK8 big data block 8
  • BLOCK8 In the management list, BLOCK8 is still in front of BLOCK6, then BLOCK8 needs to be moved in the management list. .
  • BLOCK8 At the current position of BLOCK8, look backwards to find the first data block whose access count value is greater than or equal to 3, in this case BLOCK1, as shown in Figure 5, move BLOCK8 to BLOCK1 in the management list.
  • a small data block in a large data block may also be managed by using a bitmap, for example, a 2 MB data block is used as a large data block, and a 512B data block is used as a small data block,
  • the 2MB big data block can be further divided into 4096 512B small data blocks.
  • These 512B data blocks can be managed by using a bit chart. Each 512B in the bit chart is managed with 1 bit.
  • 6 is a schematic diagram of bit chart management. As shown in FIG. 6, the bitmap shows the management diagram of two 2MB data blocks. If the corresponding bit value is 0, it means that the corresponding 512B is not in the storage space of the cache image.
  • the corresponding bit value is 1, it means that the corresponding 512B is in the storage space of the cache image. It should be noted that FIG. 6 only shows eight 512B data blocks, and all 512B data blocks in one 2MB data block are not used. The actual operation is subject to actual conditions.
  • the corresponding position in the bitmap is 0.
  • the position corresponding to the small data block in the bit chart is updated to 1.
  • the processing method of the cache receives the read input and output IO request, and reads the small data block from the parent image according to the read IO request, and adds the access count value of the first large data block to 1, if the first The big data block is in the cache image, and according to the access count value, the location and the access count value of the identifier of the first large data block are updated in the management linked list, and the small data block is written into the first large data block in the cache image.
  • the storage space if the first large data block is not in the cache image, write the identifier of the first large data block and the corresponding access count value into the management chain table, and check whether the storage space of the cache image is full or the storage space is occupied.
  • the ratio is greater than or equal to the preset ratio threshold. If the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to the preset ratio threshold, the first large data block is replaced with the second largest data block in the cache image. And the small data block is written into the storage space corresponding to the first large data block in the cache image. If the storage space of the cache image is not full or the storage space occupancy ratio is less than the preset ratio threshold, if not, the storage space of the cache image is not The occupancy ratio of full or storage space is less than the preset ratio threshold. Compared with the prior art, when the storage space of the cache image is used up, the new data block can no longer be written into the cache image.
  • not only the new first large data block can be implemented. Replace the aging second largest data block, and then write the small data block that needs to be written into the cache image, which effectively improves the utilization of the storage space.
  • the management chain table is used to manage the large data blocks in the cache image according to the big data. The access count value of the block can quickly and easily find the aging data block, and improve the efficiency of the replacement, join and data block identification of the big data block in the linked list.
  • FIG. 7 is a flowchart of a method for processing a cache according to Embodiment 3 of the present invention.
  • the step of “replace the first large data block with the second largest data block in the cache image and write the small data block into the storage space corresponding to the first large data block in the cache image” The implementation includes the following steps:
  • Step 301 According to the access count value of the first big data block, search from the header position of the management linked list, obtain the second largest data block, and remove the identifier and the access count value of the second large data block from the management linked list. And write the identity and access count value of the first large data block before the identity of the third big data block in the management linked list.
  • the second largest data block is the first data block in the management linked list, and the access count value is less than or equal to the access count value of the first large data block; the third largest data block is the identifier of the second largest data block deleted. In the subsequent management list, the access count value is greater than or equal to the first data block of the access count value of the first large data block.
  • the management linked list according to the access count value of the first large data block, starting from the position of the header, the data block whose first access count value is less than or equal to the access count value of the second data is found.
  • the data block is used as the second largest data block, and the identifier and the access count value of the second largest data block are removed from the management linked list, indicating that the second large data block does not exist in the cache image, and the first big data is The identity of the block and the access count value are written into the management list, indicating that the first large block of data exists in the cache image.
  • FIG. 8 is a schematic diagram of a management link table before the first large data block enters the cache image in the third embodiment
  • FIG. 9 is a schematic diagram of the management link table after the first large data block enters the cache image in the third embodiment.
  • the management list includes data block 3 (BLOCK3), data block 6 (BLOCK6), data block 1 (BLOCK1), and data block 8 (BLOCK8).
  • the access count value of BLOCK3 is 5, and the access of BLOCK6 is as shown in FIG.
  • the count value is 7, the access count value of BLOCK1 is 7, and the access count value of BLOCK8 is 8.
  • the first large data block is BLOCK5 and the access count value is 5.
  • BLOCK5 When BLOCK5 is accessed again, its access count value is increased to 1 and the access count value of BLOCK5 is already greater than BLOCK3. That is, BLOCK3 is the data block whose first access count value is less than BLOCK5, then BLOCK3 is the second largest. Data block, and in the management linked list, BLOCK5 is not in the linked list, that is, the first large data block is not in the cache image, then it needs to be replaced in the management linked list, as shown in Figure 9, BLOCK3 is removed from the linked list, and BLOCK5 joins the management list.
  • BLOCK6 that is, BLOCK6 is the fourth data block, then add BLOCK5 to the management list. Before BLOCK6.
  • the access count value of the identifier of more data blocks may be included in the management linked list, and is not limited to FIG. 8 and FIG. 9 .
  • Step 302 Disable the location tag corresponding to the second large data block in the BAT, and store the starting offset location of the first large data block to a location corresponding to the first large data block in the BAT.
  • the starting offset position of the first large data block is the starting offset position of the second large data block.
  • 2 MB data blocks are used as big data blocks
  • 512B data blocks are used as small data blocks
  • BAT is used to manage these 2 MB blocks
  • BAT stores the start of each 2 MB data block in the storage space. Offset position. For example, for a VHD virtual hard disk file with a capacity of 20 GB, the number of BAT entries is 10,240. If the 5th 2MB data block is stored at the location where the VHD virtual hard disk file offset address is 100M, the fifth in the BAT. The value at the table entry is 100M. If an item in the BAT is 0XFFFFFFFF, it means that the data block does not exist in the cache image.
  • Table 1 is the BAT before the first large data block is added to the cache image in the third embodiment.
  • Table 2 is the BAT after the first large data block is added to the cache image in the third embodiment.
  • the BAT stores the starting position offset addresses of the three data blocks, which are 6M+1.5K, 4M+1K, and 2M+512, respectively, and 0M indicates the storage space corresponding to the BAT. Relative starting offset position, not the actual starting point in the storage space Offset position.
  • the starting offset position of the second large data block is 6M+1.5K
  • the third entry in the BAT is marked as 0XFFFFFFFF, indicating that the third data is not in the cache image, and then The initial offset position is written into the fifth entry of the BAT written by 6M+1.5K.
  • the fifth entry in the BAT corresponds to the starting offset position of the first large data block. It is indicated that the first large data block exists in the cache image, and the storage space of the first large data block is the storage space of the original second largest data block.
  • Step 303 Write the small data block into the storage space corresponding to the starting offset position of the first large data block in the cache image.
  • each small data block has a corresponding position in the first large data block, and the first large data block occupies the storage space of the original second largest data block, if the small size needs to be written If the data block has a data block storage at a position corresponding to the original second largest data block, the data block in the original corresponding position is directly replaced by the small data block; if the small data block to be written is in the original second largest data block If there is no data block storage in the corresponding location, the small data block needs to be allocated a storage space in the corresponding storage location to store the small data block in the allocated storage space.
  • the third item in the entry corresponds to the second largest data block, and is marked as 0XFFFFFF, indicating that the second large data block is invalid, and may be This location stores the new data block, but in fact the small data block that was originally accessed in the second largest data block is still stored in the storage space of the cache image.
  • the data block 1 is the original second largest data block
  • the BAT entry corresponding to the second largest data block is marked as 0XFFFFFFFFFF
  • the third 512B small data block is still stored in the storage space.
  • the small data block of the first data block is written into the cache image
  • the small data block of the first data block is stored in the data block
  • the position of the third 512B data block of the bit chart directly replaces the third 512B data block of the original second largest data block with the small data block of the first data block; if the small data block of the first data block
  • the storage location is the location of the second 512B data block in the bitmap, but the location is 0, that is, there is no storage space at the location, and the storage space needs to be allocated in the location of the second 512B data block, and the small data is allocated.
  • the block is stored in and the value of the second position in the corresponding bit chart is set to 1.
  • the processing method of the cache searches in order from the header position of the management linked list according to the access count value of the first large data block, acquires the second largest data block, and counts the identifier and access of the second largest data block.
  • the value is removed from the management linked list, and the identifier of the first large data block and the access count value are written before the identifier of the third large data block in the management linked list, and the location mark corresponding to the second largest data block is invalidated in the BAT.
  • FIG. 10 is a flowchart of a method for processing a cache according to Embodiment 4 of the present invention.
  • the specific implementation manner of the step of adding the first large data block to the cache image and writing the small data block to the storage space corresponding to the first large data block in the cache image includes the following steps:
  • Step 401 Write the identifier of the first large data block and the access count value into the management linked list according to the access count value of the first large data block.
  • the access count value of the first large data block starts from the head node of the management linked list, finds the first large data block whose access count value is greater than or equal to the access count value of the first large data block, and the first large data block The identity and access count value are written before the first big data block.
  • FIG. 11 is a schematic diagram of a management link table before the first large data block enters the cache image in the fourth embodiment
  • FIG. 12 is a schematic diagram of the management link table after the first large data block enters the cache image in the fourth embodiment.
  • the cache image has two large data blocks, namely, data block 1 (BLOCK1) and data block 6 (BLOCK6), the access count value of BLOCK1 is 1, and the access count value of BLOCK6 is 3.
  • the access count value of the first large data block (BLOCK8) outside the cache image is changed from 1 to 2.
  • BLOCK8 is added to the management list, starting from the head node, the first access count value is greater than or equal to BLOCK8.
  • the data block of the access count value here BLOCK6, adds the BLOCK8 identification and access count value to the BLOCK6 in the management list.
  • Step 402 Find a free block in the cache image and write the small data block to the free block.
  • the free block is a storage space corresponding to the first large data block.
  • a free block is searched in the storage space of the cache image, and the free block is used as a storage space corresponding to the first large data block, and the small data block is written into the free block.
  • Step 403 Write a starting offset position of the free block to a position corresponding to the free block in the BAT.
  • the initial offset position of the free block is written to the entry position corresponding to the free block in the BAT, so that the storage space of the first large data block is found according to the initial offset position during the next access.
  • Table 3 is the BAT before the first large data block is added to the cache image in the fourth embodiment.
  • Table 4 is the BAT after the first large data block is added to the cache image in the fourth embodiment.
  • the starting offset position of BLOCK1 is 0M
  • the starting offset position of BLOCK6 is 2M+512.
  • the found offset position of the free block is 4M+1K
  • 4M+1K is written to the entry position corresponding to BLOCK8 in the BAT.
  • the processing method of the cache provided in this embodiment, when the first large data block including the small data block is not in the cache image, and the storage space of the cache image is not full or the occupation ratio of the storage space is less than a preset ratio threshold, according to the first
  • the access count value of a large data block, the identifier of the first large data block and the access count value are written into the management linked list, and a free block is found in the cache image, and the small data block is written into the free block, and the idle block is idle.
  • the starting offset position of the block is written to the position corresponding to the free block in the BAT, and the management chain table and the BAT are used to manage the data block in the cache image, so that the first large data block can be quickly and conveniently added to the cache image, and the read small
  • the data block is stored in the storage space of the cache image, which not only improves the utilization of the storage space, but also accesses the cache image directly when the next time the small data block is accessed, thereby reducing the pressure on the remote storage where the parent image is located.
  • FIG. 13 is a schematic structural diagram of a cache processing apparatus according to Embodiment 5 of the present invention.
  • the apparatus includes a reading module 11, an inspection module 12, and a processing module 13.
  • the reading module 11 is configured to receive the read input and output IO request and read the small data block from the parent image according to the read IO request.
  • the checking module 12 is configured to check whether the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to a preset ratio threshold if the first large data block that includes the small data block is not in the cache image.
  • the processing module 13 is configured to replace the first large data block with the second largest data block in the cache image, and replace the small data block, if the storage space of the cache image is full or the occupation ratio of the storage space is greater than or equal to a preset ratio threshold. Writes the storage space corresponding to the first large data block in the cache image.
  • the device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 2, and the implementation principle and technical effects are similar, and details are not described herein again.
  • FIG. 14 is a schematic structural diagram of a cache processing apparatus according to Embodiment 6 of the present invention.
  • the processing module includes an updating unit 21, configured to update the first large data block in the management linked list according to the access count value if the first large data block is in the cache image. The location of the identifier and the access count value, and write the small data block to the cache corresponding to the first largest data block in the cache image. Storage space.
  • the reading module 11 is further configured to increment the access count value of the first large data block containing the small data block by one.
  • the processing module 13 is further configured to write the identifier of the first large data block and the corresponding access count value into the management linked list.
  • the processing module 13 is further configured to add the first large data block to the cache image and write the small data block into the cache image if the storage space of the cache image is not full or the occupation ratio of the storage space is less than a preset ratio threshold.
  • the storage space corresponding to the data block is not full or the occupation ratio of the storage space is less than a preset ratio threshold.
  • the device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 3, and the implementation principle and technical effects are similar, and details are not described herein again.
  • FIG. 15 is a schematic structural diagram of a cache processing apparatus according to Embodiment 7 of the present invention.
  • the processing module 13 includes a second large data block acquiring unit 22, a position marking unit 23, and a small data block writing unit 24.
  • the second large data block obtaining unit 22 is configured to search, according to the access count value of the first large data block, from the header position of the management linked list, obtain the second large data block, and identify the second large data block.
  • the access count value is removed from the management linked list, and the identifier of the first large data block and the access count value are written before the identifier of the third large data block in the management linked list; wherein the second largest data block is in the management linked list The first data block whose access count value is less than or equal to the access count value of the first large data block; the third largest data block is the management link list after the identifier of the second largest data block is deleted, and the access count value is greater than or equal to The first data block of the access count value of the first large data block.
  • the location marking unit 23 is configured to invalidate the location tag corresponding to the second large data block in the BAT, and store the starting offset location of the first large data block to a location corresponding to the first large data block in the BAT;
  • the starting offset position of the first large data block is the starting offset position of the second largest data block.
  • the small data block writing unit 24 is configured to write the small data block into the storage space corresponding to the start offset position of the first large data block in the cache image.
  • the device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 7.
  • the implementation principle and technical effects are similar, and details are not described herein again.
  • FIG. 16 is a schematic structural diagram of a cache processing apparatus according to Embodiment 8 of the present invention.
  • the processing module 13 includes an identification writing unit 25, a free block acquiring unit 26, and a position writing unit 27.
  • the identifier writing unit 25 is configured to write the identifier of the first big data block and the access count value into the management linked list according to the access count value of the first large data block.
  • the free block obtaining unit 26 is configured to find a free block in the cache image and write the small data block into the free block; wherein the free block is the storage space corresponding to the first large data block.
  • the position writing unit 27 is for writing the start offset position of the free block to the position corresponding to the free block in the BAT.
  • the device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 10, and the implementation principle and technical effects are similar, and details are not described herein again.
  • the aforementioned program can be stored in a computer readable storage medium.
  • the program when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Memory System Of A Hierarchy Structure (AREA)
  • Image Input (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé et un appareil de traitement de cache, le procédé consistant : à recevoir une requête de lecture d'entrée/sortie (E/S), selon la requête de lecture E/S, à lire un petit bloc de données à partir d'un miroir parent (S101) ; si un premier grand bloc de données contenant le petit bloc de données n'est pas dans un miroir de cache, à vérifier si l'espace de stockage du miroir de cache a ou non un espace ou si un rapport d'occupation de l'espace de stockage est supérieur ou égal à un seuil de rapport préétabli (S102) ; si tel est le cas, à remplacer un second grand bloc de données du miroir de cache par le premier grand bloc de données, et à écrire le petit bloc de données dans l'espace de stockage correspondant au premier grand bloc de données dans le miroir de cache (S103), à remplacer le second grand bloc de données daté par un nouveau premier grand bloc de données, et à écrire le petit bloc de données ayant besoin d'être écrit dans le miroir de cache pour augmenter de manière efficace le taux d'utilisation de l'espace de stockage.
PCT/CN2015/097176 2014-12-16 2015-12-11 Procédé et appareil de traitement de cache WO2016095761A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410784834.6A CN104503703B (zh) 2014-12-16 2014-12-16 缓存的处理方法和装置
CN201410784834.6 2014-12-16

Publications (1)

Publication Number Publication Date
WO2016095761A1 true WO2016095761A1 (fr) 2016-06-23

Family

ID=52945104

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/097176 WO2016095761A1 (fr) 2014-12-16 2015-12-11 Procédé et appareil de traitement de cache

Country Status (2)

Country Link
CN (1) CN104503703B (fr)
WO (1) WO2016095761A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113655955A (zh) * 2021-07-16 2021-11-16 深圳大普微电子科技有限公司 缓存管理方法、固态硬盘控制器及固态硬盘

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104503703B (zh) * 2014-12-16 2018-06-05 华为技术有限公司 缓存的处理方法和装置
CN105227665B (zh) * 2015-07-20 2018-11-30 中国科学院计算技术研究所 一种用于缓存节点的缓存置换方法
CN106383666B (zh) * 2016-09-07 2020-05-01 东信和平科技股份有限公司 一种数据存储方法及装置
CN109074301B (zh) * 2017-03-13 2021-10-08 荣耀终端有限公司 一种数据处理方法及装置
CN109542348B (zh) * 2018-11-19 2022-05-10 郑州云海信息技术有限公司 一种数据下刷方法及装置
CN111324415A (zh) * 2019-10-28 2020-06-23 烽火通信科技股份有限公司 一种虚拟机镜像缓存创建方法、系统及计算机可读介质
CN111831655B (zh) * 2020-06-24 2024-04-09 北京字节跳动网络技术有限公司 一种数据处理的方法、装置、介质和电子设备
CN114371810B (zh) * 2020-10-15 2023-10-27 中国移动通信集团设计院有限公司 Hdfs的数据存储方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6418510B1 (en) * 2000-09-14 2002-07-09 International Business Machines Corporation Cooperative cache and rotational positioning optimization (RPO) scheme for a direct access storage device (DASD)
CN102999444A (zh) * 2012-11-13 2013-03-27 华为技术有限公司 一种用于替换缓存模块中数据的方法及装置
CN103136121A (zh) * 2013-03-25 2013-06-05 中国人民解放军国防科学技术大学 一种固态盘的缓存管理方法
CN103218248A (zh) * 2013-03-25 2013-07-24 华为技术有限公司 一种虚拟机镜像的更新方法、服务器和桌面云系统
CN104503703A (zh) * 2014-12-16 2015-04-08 华为技术有限公司 缓存的处理方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7058784B2 (en) * 2003-07-04 2006-06-06 Solid State System Co., Ltd. Method for managing access operation on nonvolatile memory and block structure thereof
CN103049397B (zh) * 2012-12-20 2015-09-16 中国科学院上海微系统与信息技术研究所 一种基于相变存储器的固态硬盘内部缓存管理方法及系统
CN103885728B (zh) * 2014-04-04 2016-08-17 华中科技大学 一种基于固态盘的磁盘缓存系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6418510B1 (en) * 2000-09-14 2002-07-09 International Business Machines Corporation Cooperative cache and rotational positioning optimization (RPO) scheme for a direct access storage device (DASD)
CN102999444A (zh) * 2012-11-13 2013-03-27 华为技术有限公司 一种用于替换缓存模块中数据的方法及装置
CN103136121A (zh) * 2013-03-25 2013-06-05 中国人民解放军国防科学技术大学 一种固态盘的缓存管理方法
CN103218248A (zh) * 2013-03-25 2013-07-24 华为技术有限公司 一种虚拟机镜像的更新方法、服务器和桌面云系统
CN104503703A (zh) * 2014-12-16 2015-04-08 华为技术有限公司 缓存的处理方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113655955A (zh) * 2021-07-16 2021-11-16 深圳大普微电子科技有限公司 缓存管理方法、固态硬盘控制器及固态硬盘
CN113655955B (zh) * 2021-07-16 2023-05-16 深圳大普微电子科技有限公司 缓存管理方法、固态硬盘控制器及固态硬盘

Also Published As

Publication number Publication date
CN104503703A (zh) 2015-04-08
CN104503703B (zh) 2018-06-05

Similar Documents

Publication Publication Date Title
WO2016095761A1 (fr) Procédé et appareil de traitement de cache
US10241919B2 (en) Data caching method and computer system
US10101930B2 (en) System and method for supporting atomic writes in a flash translation layer
CN105740164B (zh) 支持缓存一致性的多核处理器、读写方法、装置及设备
KR101620773B1 (ko) 복합식 비휘발성 저장 디바이스를 위한 데이터 이송
EP2735978B1 (fr) Système de stockage et procédé de gestion utilisés pour les métadonnées d'un système de fichiers en grappe
US8838936B1 (en) System and method for efficient flash translation layer
JP6613375B2 (ja) プロファイリングキャッシュ置換
US10409502B2 (en) Method and apparatus for writing metadata into cache
US20150113230A1 (en) Directory storage method and query method, and node controller
US9501419B2 (en) Apparatus, systems, and methods for providing a memory efficient cache
US20160041907A1 (en) Systems and methods to manage tiered cache data storage
CN110555001B (zh) 数据处理方法、装置、终端及介质
WO2017113213A1 (fr) Procédé et dispositif de traitement de demande d'accès, et système informatique
WO2020063355A1 (fr) Procédé et dispositif de mise en mémoire tampon de blocs de données, dispositif informatique et support d'enregistrement lisible par ordinateur
CN103677670A (zh) 读数据的方法及装置
WO2015180493A1 (fr) Procédé, appareil et système de traitement de mémoire de données
WO2023160358A1 (fr) Procédé et appareil de balayage de mémoire
WO2016131175A1 (fr) Procédé et dispositif permettant d'accéder à un répertoire de visiteurs de données dans un système multi-cœurs
CN109002400B (zh) 一种内容感知型计算机缓存管理系统及方法
WO2016206070A1 (fr) Procédé de mise à jour de fichier et dispositif de mémorisation
US10102116B2 (en) Multi-level page data structure
US11038797B2 (en) Forwarding table entry access
CN104252423A (zh) 基于多内核处理器的一致性处理方法和装置
US10162525B2 (en) Translating access requests for a multi-level page data structure

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15869268

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15869268

Country of ref document: EP

Kind code of ref document: A1