CN115268763A

CN115268763A - Cache management method, device and equipment

Info

Publication number: CN115268763A
Application number: CN202110481871.XA
Authority: CN
Inventors: 程卓; 陈俊伟; 项伟杰; 刘金虎
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-04-30
Filing date: 2021-04-30
Publication date: 2022-11-01

Abstract

The application provides a cache management method, a cache management device and cache management equipment. In the application, after receiving a read data request sent by a user and used for requesting to access first data, a storage device determines one or more second data to be prefetched according to the read data request, where prefetching may refer to moving data from a secondary storage unit to a primary storage unit, where the one or more second data include third data, determines whether associated data of the third data is in the primary storage unit, and if so, does not trigger a prefetching operation on the third data; or if not, a prefetch operation of the third data is triggered. This is because, since the associated data is already stored in the primary storage unit, and the third data can be obtained according to the associated data when a read data request for the third data is subsequently received, the data that may not generate a benefit is stored in the primary storage unit as much as possible, and the resource utilization rate is improved while the resource scheduling overhead is reduced.

Description

Cache management method, device and equipment

Technical Field

The present application relates to the field of storage technologies, and in particular, to a cache management method, apparatus, and device.

Background

With the progress of information technology and the coming of the internet of things era, data information shows explosive growth, and the demand of human beings on low-cost and high-performance storage is increasing day by day. How to make a storage system have both cost and performance becomes the focus of research on the storage system. At present, the mainstream scheme is to use a disk with low price and performance as a main memory to reduce the cost, and use high-end media such as DRAM as a cache layer to improve the read-write performance of the device.

Currently, the data management policy is a metadata (non-user data generated by managing user data on the device side) and data (user data) separation policy. The existing cache management strategy may cause waste of cache resources due to the fact that the relation between the two is not considered, and the utilization rate of the cache resources is low.

Disclosure of Invention

The application provides a cache management method, a cache management device and cache management equipment, which are used for improving the utilization rate of cache resources.

In a first aspect, an embodiment of the present application provides a cache management method, which may be implemented by a storage device, or may be implemented by components included in the storage device, such as components of a cache management apparatus, a circuit, a chip, a processor, and a controller of the storage device, where the storage device includes at least a first-level storage unit and a second-level storage unit; in the method, after receiving a read data request sent by a user for requesting access to first data, the storage device predicts one or more second data that the user is likely to access in the future according to the read data request, such as a pre-fetching algorithm; judging whether the associated data of the third data is in the primary storage unit, if so, not triggering the pre-fetching operation of the third data; the third data is any one of the one or more second data, or the third data is any one of the one or more second data that meets a preset condition, where the preset condition is that a heat value is lower than a preset threshold.

Optionally, when the third data is user data, its associated data refers to metadata of the user data, or other data used for indicating the user data. When the third data is metadata, its associated data refers to the user data to which the metadata points, or other data describing or indicating the user data.

According to the method provided by the first aspect, after receiving a read data request, the storage device predicts data which is possibly read in the future according to the request, and judges whether the associated data of the data is already stored in the primary storage unit. If the associated data is already stored in the primary storage unit, no prefetch operation is performed. This is because, since the associated data is already stored in the primary storage unit, a read data request for the third data (e.g., the user data) is subsequently received, the third data can also be acquired at a faster speed according to the associated data of the third data, or, a read data request for the associated data (e.g., the user data) is subsequently received, the associated data can also be directly acquired in the primary storage unit, and there is no need to acquire the third data; therefore, data which has repeated semantics or may not generate benefit is stored in the primary storage unit as much as possible, the resource scheduling cost is reduced, and the utilization rate of the storage resources is improved.

In a possible implementation manner, the third data is any one of the one or more second data whose heat value is lower than the first preset threshold.

By the method, the storage device can pre-fetch the second data with a higher heat value, for example, not lower than the first preset threshold value, to the primary storage unit, so as to guarantee that the reading performance of the storage device is prioritized. On the other hand, the second data with a lower heat value, such as the second data lower than the first preset threshold value, can be deduplicated, so that the consumption of scheduling resources is reduced, the waste of storage resources caused by storing repeated semantic data is reduced, and the utilization rate of the storage resources is improved.

In a possible implementation manner, the third data is user data, and the associated data of the third data is metadata of the user data; or the third data is metadata of the user data, and the associated data of the third data is the user data.

By the method, the storage device can perform combined management on the user data and the metadata when performing the pre-fetching operation, and compared with a cache management strategy of separating the metadata and the user data, the method can improve the utilization rate of memory resources and improve the reading performance of the device.

In a possible implementation manner, before the storage device determines whether the associated data of the third data is in the primary storage unit, the storage device further detects a management and control condition, where the management and control condition includes that a remaining available space of the primary storage unit is smaller than a second preset threshold, or that a read bandwidth is higher than a third preset threshold.

Through the method, when management and control are not needed, duplicate removal is not needed for the preset data with repeated semantics to ensure that the reading performance of the storage device is prior, and when management and control are needed, resource consumption can be reduced on the basis of ensuring the reading performance.

In one possible implementation, triggering a prefetch operation of third data includes: and acquiring third data from the secondary storage unit, and writing the third data into the primary storage unit.

In a second aspect, an embodiment of the present application provides another cache management method, where the method may be implemented by a storage device, or may be implemented by components included in the storage device, such as components of a cache management apparatus, a circuit, a chip, a processor, and a controller of the storage device, where the storage device includes at least a first-level storage unit and a second-level storage unit; in the method, a storage device determines first data to be eliminated in a primary storage unit, judges whether the associated data of the first data is in the primary storage unit, if so, eliminates the first data, otherwise, keeps the first data.

Optionally, when the first data is user data, its associated data refers to metadata of the user data, or other data used for indicating the user data. When the first data is metadata, its associated data refers to the user data pointed to by the metadata, or other data describing or indicating the user data.

According to the method provided by the second aspect, when the storage device triggers data elimination of the primary storage device, the storage device may preferentially eliminate the first data of the associated data in the primary storage unit, which is because since the associated data is already stored in the primary storage unit, a subsequent read data request for the first data (for example, user data) is received, the first data may also be obtained at a higher speed according to the associated data of the first data, or a subsequent read data request for the associated data (for example, user data) is received, the associated data may also be directly obtained in the primary storage unit, and there is no need to obtain the first data; therefore, data which has repeated semantics or may not generate profit is stored in the first-level storage unit as much as possible, and the hit rate is guaranteed while the utilization rate of storage resources is improved.

In one possible implementation, a retirement condition is detected, where the retirement condition includes that the remaining available space of the primary storage unit is smaller than a first preset threshold.

In a third aspect, an embodiment of the present application provides another cache management method, where the method may be executed by a storage device, and may also be executed by an element (e.g., a processor) included in the storage device, where the storage device includes at least a primary storage unit and a secondary storage unit; in the method, after the storage device acquires first pre-fetching information and second pre-fetching information, whether first data is associated with second data is judged; if so, the first prefetch information is deleted or the second prefetch information is deleted. The first prefetching information comprises information of first data to be prefetched; the second prefetch information includes information of second data to be prefetched.

Optionally, when the second data is user data, its associated data is metadata of the user data, or other data used for indicating the user data. When the second data is metadata, its associated data refers to the user data pointed to by the metadata, or other data describing or indicating the user data.

According to the method provided by the third aspect, when receiving the first prefetching information and the second prefetching information, the storage device judges whether the first prefetching information indicates that the first data to be prefetched is associated with the second data to be prefetched indicated by the second prefetching information; performing only one pre-fetch of the data if the first data is associated data of the second data, e.g., deleting the second pre-fetch information, performing a pre-fetch of the first data; alternatively, the first prefetch information is deleted and prefetching of the second data is performed. Therefore, if the second data (such as user data) is prefetched into the primary storage unit, then a subsequent read data request for the second data is received, the second data can be directly obtained in the primary storage unit, and the data associated with the second data does not need to be obtained; or, if the second data (e.g. metadata) is prefetched into the primary storage unit, then a read data request for the first data is received subsequently, and the first data can also be obtained at a higher speed according to the associated data (i.e. the second data) of the first data, so that data with repeated semantics or possibly without revenue is stored in the primary storage unit as much as possible, and the resource utilization rate is improved while the resource overhead of scheduling is reduced.

In one possible implementation, the first prefetch information includes a first prefetch address of the first data; the second prefetch information includes a second prefetch address of the second data; and if the first pre-fetching address and the second pre-fetching address have a preset functional relation, determining that the first data is associated with the second data.

In one possible implementation, if the first prefetch information is deleted, the second prefetch information is retained; the storage device may move second data from the secondary storage unit to the primary storage unit based on second prefetch information; or if the second prefetching information is deleted, keeping the first prefetching information; the storage device may move the first data from the secondary storage unit to the primary storage unit based on the first prefetch information.

In a possible implementation manner, before the storage device determines whether the metadata corresponding to the first data is the same as the first metadata, a control condition is detected, where the control condition includes that a remaining available space of the primary storage unit is smaller than a first preset threshold; or the read bandwidth is above a second preset threshold.

In a fourth aspect, an embodiment of the present application further provides a cache management device, where the cache management device has a function of implementing a behavior in the method instance of the first aspect, and beneficial effects may be referred to as descriptions of the first aspect and are not described herein again, or have a function of implementing a behavior in the method instance of the second aspect, and beneficial effects may be referred to as descriptions of the second aspect and are not described herein again, and have a function of implementing a behavior in the method instance of the third aspect, and beneficial effects may be referred to as descriptions of the third aspect and are not described herein again. The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.

In one possible design, the structure of the number cache management device includes a receiving module and a processing module; optionally, the method further includes a detection module and a prefetch module, and these modules may perform corresponding functions in the method example of the first aspect, for which specific reference is made to detailed description in the method example, and details are not repeated here.

In another possible design, the structure of the data cache management device includes a determining module and a processing module; optionally, the method further includes a detection module, and the modules may perform corresponding functions in the method example of the second aspect, for specific reference, detailed description in the method example is given, and details are not repeated here.

In yet another possible design, the structure of the number cache management device includes an obtaining module and a processing module; optionally, the method further includes a detection module, and the detection module may perform corresponding functions in the method example of the third aspect, for specific reference, detailed description in the method example is given, and details are not repeated here.

In a fifth aspect, the present application further provides a storage device, where the computing device includes a processor and a memory, and may further include a communication interface, and the processor executes program instructions in the memory to perform the method provided in the first aspect or any possible implementation manner of the first aspect, or to perform the method provided in the second aspect or any possible implementation manner of the second aspect, or to perform the method provided in any possible implementation manner of the third aspect or any possible implementation manner of the third aspect. The storage device may be a computing node, a server, or a controller in a storage system. The memory is coupled to the processor and stores program instructions and data necessary to determine the processing of data. The communication interface is used for communicating with other equipment, such as receiving a read data request or sending user data.

In a sixth aspect, the present application provides a computing device system comprising at least one computing device. Each computing device includes a memory and a processor. A processor of at least one computing device is configured to access code in the memory to perform the method provided by the first aspect or any one of the possible implementations of the first aspect, or to perform the method provided by the second aspect or any one of the possible implementations of the second aspect, or to perform the method provided by the third aspect or any one of the possible implementations of the third aspect.

In a seventh aspect, the present application provides a computer-readable storage medium, which, when executed by a computing device, performs the method provided in the foregoing first aspect or any possible implementation manner of the first aspect, performs the method provided in the foregoing second aspect or any possible implementation manner of the second aspect, or performs the method provided in any possible implementation manner of the foregoing third aspect or third aspect. The storage medium stores a program therein. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard Disk Drive (HDD), and Solid State Drive (SSD).

In an eighth aspect, the present application provides a computing device program product comprising computer instructions that, when executed by a computing device, performs the method provided in the foregoing first aspect or any possible implementation manner of the first aspect, performs the method provided in the foregoing second aspect or any possible implementation manner of the second aspect, or performs the method provided in the foregoing third aspect or any possible implementation manner of the third aspect. The computer program product may be a software installation package, and in case it is required to use the method provided in the foregoing first aspect or any possible implementation manner of the first aspect, or in case it is required to use the method provided in the foregoing second aspect or any possible implementation manner of the second aspect, and in case it is required to use the method provided in the foregoing third aspect or any possible implementation manner of the third aspect, the computer program product may be downloaded and executed on a computing device.

In a ninth aspect, the present application further provides a computer chip, where the chip is connected to a memory, and the chip is configured to read and execute a software program stored in the memory, to execute the method described in the first aspect and each possible implementation manner of the first aspect, to execute the method described in each possible implementation manner of the second aspect and the second aspect, and to execute the method described in each possible implementation manner of the third aspect and the third aspect.

Drawings

Fig. 1A to fig. 1E are schematic diagrams illustrating an architecture of a system according to the present application;

FIG. 2 is a schematic diagram of a memory structure;

FIG. 3 is a schematic diagram illustrating an implementation flow of a cache management policy;

fig. 4 is a schematic flowchart corresponding to a cache management method provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of an architecture of a storage device according to an embodiment of the present application;

fig. 6 is a schematic flowchart illustrating another cache management method according to an embodiment of the present application;

fig. 7 is a flowchart illustrating a third cache management method according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a cache management apparatus provided in the present application;

fig. 9 is a schematic structural diagram of another cache management apparatus provided in the present application;

fig. 10 is a schematic structural diagram of a third cache management apparatus provided in the present application.

Detailed Description

The data storage device is used for receiving a data writing request sent by a user and storing user data carried in the data writing request in a hard disk or a memory. When a data reading request sent by a user is received, the data requested to be accessed by the user can be obtained from the hard disk or the memory and sent to the user.

Compared with a hard disk, the read-write speed of the memory is higher, and if the data to be accessed can be hit in the memory, the data does not need to be read from the hard disk. In order to improve the reading performance of the storage device, the storage device may store the hot spot data, such as frequently-accessed data, in the memory, so that the storage device can acquire the hot spot data from the memory at a higher speed and feed the hot spot data back to the user when receiving a read data request of the hot spot data sent by the user.

Due to the limited capacity of the memory, all hot data are difficult to cache, the memory space in each block is precious, and the utilization rate of memory resources has important significance for improving the system performance. The existing cache management strategy used on the storage device is a user data and metadata separation strategy, namely the user data and the metadata are managed independently, however, a certain association exists between the user data and the metadata, and the existing cache management strategy can cause low utilization rate of cache resources.

In this embodiment of the present application, the storage device may use the cache management policy provided in this embodiment of the present application to perform joint management on the user data and the metadata stored in the memory, so as to improve the utilization rate of the memory resource as much as possible.

The following description is given taking an applicable system architecture in a data storage scenario as an example, and as shown in fig. 1A, the system architecture provided in the embodiment of the present application is a schematic diagram, where the system architecture includes an application server 100, a switch 110, and a storage system 120.

The user accesses the data through the application. The computer running these applications is called an "application server". The application server 100 may be a physical machine or a virtual machine. Physical application servers include, but are not limited to, desktop computers, servers, laptops, and mobile devices. The application servers access the storage system through the fabric switch 110 to access data. However, the switch 110 is only an optional device, and the application server 100 may also communicate directly with the storage system 120 through a network.

The storage system 120 shown in FIG. 1A is a centralized storage system. The centralized storage system is characterized by a unified entry through which all data from external devices pass, which is the engine 121 of the centralized storage system. The engine 121 is the most central component in a centralized storage system, in which the high-level functions of many storage systems are implemented.

As shown in FIG. 1A, there are one or more controllers in the engine 121, and FIG. 1A illustrates an engine including two controllers as an example. If a mirror channel is provided between the controller 0 and the controller 1, after the controller 0 writes a copy of data into its memory 124, the mirror channel may send a copy of the data to the controller 1, and the controller 1 stores the copy in its local memory 124. Therefore, the controller 0 and the controller 1 are backup to each other, when the controller 0 fails, the controller 1 can manage the service of the controller 0, and when the controller 1 fails, the controller 0 can manage the service of the controller 1, thereby avoiding the unavailability of the whole storage system 120 caused by hardware failure. When 4 controllers are deployed in the engine 121, any two controllers have mirror channels therebetween, so that any two controllers are backup to each other.

The engine 121 further comprises a front-end interface 125 and a back-end interface 126, wherein the front-end interface 125 is used for communicating with the application server 100, thereby providing a storage service for the application server 100. And a back-end interface 126 for communicating with a hard disk 134 to expand the capacity of the storage system. Through the backend interface 126, the engine 121 can connect more hard disks 134, thereby forming a very large pool of storage resources.

The controller 0 may execute the cache management method provided in the embodiment of the present application, and as shown in fig. 1A, the controller 0 at least includes a processor 123 and a memory 124.

The processor 123 is a Central Processing Unit (CPU) for processing data access requests from outside the storage system (server or other storage system) and also for processing requests generated inside the storage system. For example, when the processor 123 receives the write data requests sent by the application server 100 through the front-end port 125, the data in the write data requests is temporarily stored in the memory 124. When the total amount of data in the memory 124 reaches a certain threshold, the processor 123 sends the data stored in the memory 124 to the hard disk 105 through the back-end port for persistent storage.

The memory 124 is an internal memory directly exchanging data with the processor 123, and can read and write data at any time, and is fast, and serves as a temporary data storage for an operating system or other programs in operation. The memory includes at least two types of memory, for example, the memory may be a random access memory (ram) or a Read Only Memory (ROM). For example, the random access memory is a Dynamic Random Access Memory (DRAM), or a Storage Class Memory (SCM). DRAM is a semiconductor memory, and belongs to a volatile memory (volatile memory) device, like most Random Access Memory (RAM). SCM is a hybrid storage technology that combines the characteristics of both traditional storage devices and memory, memory-class memory providing faster read and write speeds than hard disks, but slower operation speed and lower cost than DRAM. However, the DRAM and the SCM are only exemplary in this embodiment, and the memory may also include other random access memories, such as Static Random Access Memory (SRAM), and the like. The rom may be, for example, a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), and the like. In addition, the memory 124 may also be a dual in-line memory module (DIMM), i.e., a module composed of Dynamic Random Access Memory (DRAM), or a Solid State Disk (SSD) having a memory access interface. In practical applications, the controller 0 may be configured with a plurality of memories 124 of the same type and different types. The number and type of the memories 124 are not limited in this embodiment. In addition, the memory 124 may be configured to have power conservation. The power conservation function means that when the system is powered down and powered up again, the data stored in the memory 124 will not be lost. A memory having a power retention function is called a nonvolatile memory.

In fact, rather than moving data or metadata from a hard disk to memory is referred to as prefetching, for any two memories with different performances, the operation of moving data from a relatively low-performance memory to a relatively high-performance memory to migrate data between the two memories in order to increase the read speed may be referred to as prefetching. For example, data is copied from the hard disk 105 to the SCM. For another example, data is copied from the hard disk 105 to the DRAM. For another example, the data is moved from the SCM to the DRAM, and for another example, the data is copied from the HDD to the SDD, etc., which is not limited in this embodiment. For convenience of description, prefetching will be used as an example for copying data from a hard disk to a memory.

The memory 124 stores software programs, and the processor 123 executes the software programs in the memory 124 to manage the hard disk. For example, the hard disk is abstracted into a storage resource pool, and then divided into LUNs for use by the server. The LUN here is in fact the hard disk seen on the server. Optionally, some centralized storage systems are file servers themselves, and may provide shared file services for the servers.

The hardware components and software structure of controller 1 (and other controllers not shown in FIG. 1A) are similar to controller 0 and will not be described again.

FIG. 1A illustrates a disk separated centralized storage system. In this system, the engine 121 may not have a hard disk slot, the hard disk 134 needs to be placed in the hard disk box 130, and the backend interface 126 communicates with the hard disk box 130. The backend interface 126 exists in the engine 121 in the form of an adapter card, and two or more backend interfaces 126 can be used on one engine 121 to connect a plurality of hard disk frames at the same time. Alternatively, the adapter card may be integrated on the motherboard, and the adapter card may communicate with the processor 123 through the PCIE bus.

Fig. 1B is a schematic diagram of another system architecture provided in the embodiment of the present application, where the system architecture includes an application server 100, a switch 110, and a storage system 120. The system architecture shown in fig. 1B is similar to the system architecture shown in fig. 1A, except that the hard disk frame is omitted from the system architecture shown in fig. 1B, and the controller is directly connected to each hard disk 134.

The cache management method provided in the embodiment of the present application is applicable to a centralized storage system as well as a distributed storage system, as shown in fig. 1C, and is a schematic diagram of a system architecture of the distributed storage system provided in the embodiment of the present application, where the distributed storage system includes a server cluster. The server cluster includes one or more servers 110 (three servers 110 are shown in fig. 1C, but the present application is not limited to two servers 110), and the servers 110 can communicate with each other. Server 110 is a device, such as a server, desktop computer, etc., that has both computing and storage capabilities. In software, each server 110 has an operating system thereon. The virtual machine 107 may be created on the server 110, and the computing resources required by the virtual machine 107 may be sourced from the processor 112 and the memory 113 local to the server 110, while the storage resources required by the virtual machine 107 may be sourced from the hard disk 105 local to the server 110, or may be sourced from the hard disk 105 in another server 110. In addition, various applications may be running in the virtual machine 107, and a user may trigger a read/write data request through the applications in the virtual machine 107.

The server 110 may execute the cache management method provided in the embodiment of the present application, and as shown in fig. 1C, the server 110 at least includes a processor 112, a memory 113, a network card 114, and a hard disk 105. The processor 112, the memory 113, the network card 114 and the hard disk 105 are connected through a bus. The processor 112 and the memory 113 are used to provide computing resources, among other things. Specifically, processor 112 is a Central Processing Unit (CPU) for processing data access requests from outside server 110 or requests generated internally within server 110. For example, when the processor 112 receives a read data request sent by a user, the cache management method provided in the embodiment of the present application may be executed to determine data to be prefetched and determine whether the data needs to be prefetched, and if so, copy the data determined to be prefetched from the hard disk 105 to the memory 113. The processor 112 may also discard the data stored in the memory 113 when the total amount of data in the memory 113 reaches a certain threshold.

In addition, the processor 112 is used for data calculation or processing, such as metadata management, data verification, virtualized storage space, address translation, and the like. Only one CPU 112 is shown in fig. 1C, and in practical applications, the number of CPUs 112 is often multiple, wherein one CPU 112 has one or more CPU cores. The number of CPUs and the number of CPU cores are not limited in this embodiment.

The function and type of the memory 113 are the same as those of the memory 124, and for details, reference may be made to the foregoing description and are not repeated herein. The hard disk 105 is used to provide storage resources, such as storing data. It may be a magnetic disk or other type of storage medium such as a solid state hard disk or a shingled magnetic recording hard disk. The network card 114 is used to communicate with other servers 110.

As shown in fig. 1D, which is a schematic system architecture of another distributed storage system provided in this embodiment of the present application, the system architecture shown in fig. 1C is similar to the system architecture shown in fig. 1D, except that no virtual machine is deployed on the server 110 in the system architecture shown in fig. 1D, and a user may initiate a read data request or a write data request to the server 110 through a device deployed outside the server 110.

As shown in fig. 1E, which is a schematic diagram of a system architecture of another distributed storage system provided in the embodiment of the present application, the system includes a compute node cluster and a storage node cluster. The computing node cluster includes one or more computing nodes 110 (three computing nodes 110 are shown in fig. 1E, but the application is not limited to three computing nodes 110), and the computing nodes 110 may communicate with each other. A compute node 110 is a computing device such as a server, a desktop computer, or a controller of a storage array, etc.

In hardware, as shown in fig. 1E, the computing node 110 includes at least a processor 112, a memory 113, and a network card 114. The role and function of the processor 112 are similar to those of the processor 112 in fig. 1C or fig. 1D, which may specifically refer to the foregoing contents, and are not described herein again, except that when the total amount of data in the memory 113 reaches a certain threshold, the processor 112 sends the data stored in the memory 113 to the storage node 100, and the hard disk 105 of the storage node 100 performs persistent storage. Accordingly, when receiving a read data request sent by a user, the processor 112 may obtain data to be read from the memory 113, or obtain the data to be read from the storage node 100, and send the data to the user.

The function of the memory 113 is similar to that of the memory 124, and the details thereof may be as described above, and are not described herein again.

The network card 114 is used to communicate with the storage node 100. For example, when the total amount of data in the memory 113 reaches a certain threshold, the computing node 110 may send a request to the storage node 100 through the network card 114 to perform persistent storage on the data. Additionally, the computing node 110 may also include a bus for communication between various components within the computing node 110. Functionally, since the primary function of the compute node 110 in FIG. 1E is compute traffic, persistent storage can be implemented using remote storage when storing data, and therefore it has less local storage than a conventional server, thereby achieving cost and space savings. This does not mean that the computing node 110 cannot have local storage, and in practical implementations, the computing node 110 may have a small number of hard disks built in or externally connected to it.

Any one of the computing nodes 110 may access any one of the storage nodes 100 in the storage node cluster over a network. The storage node cluster includes a plurality of storage nodes 100. A storage node 100 comprises one or more control units 101, a network card 104 and a plurality of hard disks 105. Network card 104 is used to communicate with computing node 110. The hard disk 105 is used for storing data, and may be a magnetic disk or other type of storage medium, such as a solid state disk or a shingled magnetic recording hard disk. The control unit 101 is configured to write data into the hard disk 105 or read data from the hard disk 105 according to a read/write data request sent by the computing node 110. In the process of reading and writing data, the control unit 101 needs to convert an address carried in the read/write data request into an address that can be recognized by the hard disk. It follows that the control unit 101 also has some simple calculation functions.

In the embodiment of the present application, the user data is data of a user, including but not limited to files, images, audio, video, documents, and the like, and the metadata is non-user data for describing the user data, and the metadata includes storage information of the user data. For example, the storage device may store first data (user data) sent by a user in a hard disk after receiving a write data request for the first data, and generate metadata for managing the first data, the metadata including storage information indicating a storage location of the first data in the hard disk, the metadata being also stored in the hard disk.

The first data and metadata may also be stored in memory if memory space is sufficient.

Fig. 2 is a schematic structural diagram of a memory. The memory in fig. 2 may be the memory 124 in the storage system 120, and may also be the memory 113 in the server 110 or the computing node.

Taking the memory shown in fig. 2 as the memory 124 in fig. 1A as an example, as shown in fig. 2, the memory 124 includes a data cache region 21 and a metadata cache region 22, where the data cache region 21 is used for caching user data, and the metadata cache region 22 is used for caching metadata. The data buffer 21 may be further divided into a write data buffer 210 and a read data buffer 211 according to the write user data and the read user data. It should be noted that the memory in this embodiment may be physically divided into a plurality of regions, or may be logically divided into a plurality of regions.

The write data process is described below in conjunction with fig. 1A and 2: when the processor 123 receives a write data request sent by the application server 100 through the front-end port 125, data in the write data request is temporarily stored in the write data buffer 210. When the total amount of data in the write data buffer 210 reaches a certain threshold, the processor 123 sends the data stored in the write data buffer 210 to the hard disk 105 through the back-end port for persistent storage, and generates metadata of the data. The metadata includes storage information indicating a storage location of the data in the hard disk 105. The metadata may be stored in the hard disk 105 and may be stored in the metadata cache 22 if the space in the metadata cache 22 is sufficient. It should be appreciated that the data in the write data buffer 210 may need to be deleted or evicted after being guaranteed to have been backed up in the hard disk 105, or that the data may be moved from the write data buffer 210 to the hard disk 105 at eviction.

There are various ways for the storage system 120 (or server) to obtain the data to be stored, for example, the data to be stored may be transmitted to the storage system 120 by the user through the application server 100, and for example, the data to be stored may be directly transmitted to the storage system 120 by the user. For another example, the data to be stored may be data recorded during the operation of the storage system 120, such as metadata of user data. The embodiment of the present application does not limit the manner of obtaining the data to be stored.

The read data process is described below in conjunction with FIGS. 1A and 2: when receiving a read data request of first data sent by the application server 100 through the front-end interface 125, the processor 123 firstly queries the first data from the write data cache 210, and if the first data is hit in the write data cache 210, acquires the first data from the write data cache 210, and sends the first data to the application server 100 through the front-end interface 125; if the first data is not hit in the write data cache 210, the read data cache 211 is continued to search for the first data. Similarly, if the first data is hit in the read data cache 211, the first data is obtained from the read data cache 211, and if the first data is not hit in the read data cache 211, the metadata of the data is continuously obtained, and the first data is read from the hard disk according to the metadata, and then the first data is sent to the application server 100 through the front-end interface 125. There are various ways to obtain the metadata, for example, when the metadata cache 22 hits the metadata, the metadata can be directly obtained from the metadata cache 22, or when the metadata cache 22 misses the metadata, the metadata can be obtained from the hard disk 105. The reason why the first data is first queried from the write data buffer 210 is that the data of the write data buffer 210 is the newest in consideration of the data update, and the data to be updated may not be synchronized into the read data buffer 211 or the hard disk 105, so the data can be first queried from the write data buffer 210 to ensure the consistency of the data. However, the above is merely an example, and the order of reading data in the memory is not limited in the embodiments of the present application.

Obviously, when user data or metadata hit in the memory, the reading speed can be effectively improved, and the higher the memory hit rate is, the stronger the reading performance is. In practical applications, the memory hit rate strongly depends on a prefetching algorithm, which may be used to predict data that may be accessed by a user in a future period of time based on characteristics of data currently accessed by the user and/or data accessed historically, so that the predicted data may be moved from a hard disk to a memory in advance, thereby improving the hit rate. Specifically, the prefetching algorithm may be divided into two categories according to the service object, including a data prefetching algorithm and a metadata prefetching algorithm, where the data prefetching algorithm is used to predict user data to be prefetched, and the metadata prefetching algorithm is used to predict metadata to be prefetched.

In one prior implementation, the data prefetching algorithm and the metadata prefetching algorithm work independently, and the data prefetching and the metadata prefetching are also independent of each other. Referring to fig. 3, a schematic flow chart of the method is shown, and the method is described below with reference to fig. 1A and 2.

As shown in fig. 3, when the processor 123 receives a read data request for reading the first data sent by the application server 100 through the front-end port 125 (step 1), on one hand, the processor 123 executes the aforementioned read data flow to obtain the first data and send the first data to the user (not shown in fig. 3). On the other hand, the processor 123 executes two processes in parallel, including a process one and a process two, where the process one is used to perform data prefetching; flow two is used to perform metadata prefetching.

Illustratively, the first process includes: (1) The processor 123 runs a data pre-fetching algorithm to predict data to be pre-fetched based on a read data request for the first data (step 2 a), e.g. data stored on the storage medium subsequent to the first data, it being understood that the data to be pre-fetched should be data not currently stored in the memory, if it is in the memory itself, no pre-fetching is required. For convenience of explanation, the data to be prefetched is referred to herein as second data. (2) The processor 123 triggers a pre-fetch scheduling of the second data, in particular, a pre-fetch operation of the second data is performed using a scheduling resource. (3) The processor 123 retrieves the metadata of the second data from the metadata cache 22 (step 3 a). If the metadata of the second data is not hit in the metadata cache 22, then the hard disk 105 is accessed (step 4 a), since step 4a is not a necessary step, it is indicated by a dashed line in fig. 3. (4) The processor 123 acquires the second data from the hard disk 105 based on the metadata of the second data and copies the second data to the read data buffer 211 (step 5 a). To this end, a prefetch of user data is completed.

The second process comprises the following steps: (1) The processor 123 runs a metadata prefetching algorithm to predict metadata to be prefetched based on the read data request for the first data (step 2 b), and similarly, the metadata to be prefetched should be metadata that is not currently stored in the memory, and if it exists, prefetching is not required. (2) The processor 123 triggers a metadata prefetch schedule, i.e., a scheduling resource is used to perform metadata prefetch operations. (3) The processor 123 retrieves the metadata from the hard disk 105 and copies the metadata to the metadata cache 22 (step 3 b). By this, a prefetch of metadata is completed.

However, in the second flow, the metadata to be prefetched determined by the metadata prefetching algorithm may be metadata of the second data, so that the tasks completed by the two prefetching schedules in the first flow and the second flow are the same, that is, the same metadata is read, which causes a waste of scheduling resources, and caching the user data and the metadata corresponding to the user data on the memory side does not generate a benefit, but wastes memory resources. In the prior art, scheduling resources are shared by prefetch scheduling and normal IO requests sent by users, and the scheduling resources wasted for executing the same task may cause that the normal IO requests cannot obtain the scheduling resources, and memory resources are wasted, resulting in low memory utilization rate.

In view of this, an embodiment of the present application provides a cache management method. In the application, the storage device, by using the cache management method provided by the embodiment of the application, performs joint management on the metadata and the user data, reduces the consumption of prefetching scheduling, and improves the utilization rate of storage resources.

Each cache management method provided in the embodiment of the present application may be implemented by a storage device, where the storage device may be the storage system 120 in fig. 1A and 1B, may also be the server 110 in fig. 1C and 1D, and may also be the computing node 110 in fig. 1E. The method may be implemented by a storage device, or may be implemented by a component in the storage device, such as a controller or a processor in the storage system 120, or a processor 112 in the server 110, or a processor 112 in the computing node 110, which is not described below repeatedly, and the cache management methods provided in the present application are described below by taking the storage device as an example.

With reference to fig. 4, the cache management method provided in the embodiment of the present application is applied to the system architecture shown in fig. 1A to 1E, and the cache management method provided in the embodiment of the present application is described below, and in the embodiment shown in fig. 4, a process of applying the cache management method to determining data or metadata to be prefetched is described. As shown in fig. 4, the method includes the steps of:

in step 401, a memory device receives a read data request, wherein the read data request requests access to first data.

Step 402, the storage device determines second data to be prefetched based on a data prefetching algorithm and the read data request; and determining metadata to be prefetched based on a metadata prefetch algorithm and the read data request.

It should be understood that the second data here is user data. In addition, in step 402, a plurality of second data may be determined by the data pre-fetching algorithm, and a plurality of metadata to be pre-fetched may be determined by the metadata pre-fetching algorithm, which are described by taking one user data and one metadata as an example.

In step 403, the storage device determines whether the second data and the metadata to be prefetched are related data, and if so, only triggers a prefetching schedule of the second data or only triggers a prefetching schedule of the metadata for the two objects (metadata or data) to be prefetched. This reduces the consumption of scheduling resources for prefetching. Otherwise, a prefetch schedule of the second data and a prefetch schedule of the metadata are triggered.

The method shown in fig. 4 is described in detail below in conjunction with fig. 5. Referring to fig. 5, a schematic architecture diagram of a storage device according to an embodiment of the present application is provided. The storage device 501 in fig. 5 may be the storage system 120 in fig. 1A and 1B, the server 110 in fig. 1C and 1D, or the computing node in fig. 1E. The storage device in the embodiment shown in fig. 4 may be, for example, the storage device 501 in fig. 5.

As shown in fig. 5, the memory device includes at least a prefetcher 5011, a data prefetcher 5012, and a metadata prefetcher 5013. Among them, the prefetcher 5011 is a unified portal for the data prefetcher 5012 and the metadata prefetcher 5013, and after the prefetcher 5011 receives a read data request transmitted from the application server 100 (step 1), the read data request is transmitted to the data prefetcher 5012 (step 2 a) and the metadata prefetcher 5013 synchronously (step 2 b). The data requested to be accessed in the read data request is not shown in fig. 5, and it does not mean that the memory device does not perform the operation.

The data prefetcher 5012 has a data prefetching algorithm stored thereon, and the metadata prefetching algorithm is present on the metadata prefetcher 5013. The data prefetcher 5012 determines data to be prefetched based on the read data request and transmits the determined data to be prefetched to the prefetcher 5011 (step 3 a). Similarly, the metadata area 5013 determines metadata to be prefetched based on the read data request and transmits the determined metadata to be prefetched to the prefetcher 5011 (step 3 b). The prefetcher 5011 receives data to be prefetched and metadata to be prefetched, and judges whether the data to be prefetched is the metadata of the data to be prefetched. If so, one of the entries is reserved for prefetching, e.g., the data to be prefetched is reserved, or the metadata to be prefetched is reserved. Subsequently, a pre-fetching schedule of the retained user data or metadata may be triggered, and in practical applications, the data to be pre-fetched is generally retained. The flow of the pre-fetching scheduling of the user data (step 4a to step 7 a) and the pre-fetching scheduling of the metadata (step 4b to step 5 b) may refer to the above related description of fig. 3, or may perform the pre-fetching scheduling by using the method shown in fig. 6, which will be described in detail below, and is not described herein again.

In an optional embodiment, if the scheduling resources are sufficient, the deduplication of the repeated prefetching may not be performed, and if the scheduling resources are insufficient, the management and control of the repeated prefetching may be started, that is, the data to be prefetched and the metadata to be prefetched are deduplicated. Illustratively, in the storage device, the scheduling resource utilization rate may be used to measure whether the scheduling resource is sufficient, specifically, the scheduling resource utilization rate usedRatio = usedrlota/totalquat; wherein, usedQuota is the number of used scheduling resources, and totalQuoat is the total number of scheduling resources. And when the utilization rate of the scheduling resources exceeds a first preset threshold value, the scheduling resources are considered to be insufficient, otherwise, if the utilization rate of the scheduling resources does not exceed the first preset threshold value, the scheduling resources are considered to be sufficient. As an example, in practical applications, the usedRatio is generally set to 70%, that is, when the usedRatio does not reach 70%, the management of the prefetched data may not be performed. The usedRatio reaches 70%, and management and control of repeated pre-fetching data are started. That is, when the usedRatio reaches 70%, the prefetcher 5011 deduplicates the received data to be prefetched and metadata to be prefetched. Of course, the usedquoteta is only an example, and other parameters may also be used to measure whether the scheduling resource is sufficient, for example, the utilization rate of the hardware resource, the read bandwidth, and the like, which is not limited in this embodiment of the present application.

Through the design, the flexibility of the system can be improved, the reading performance and the hit rate can be guaranteed preferentially when the scheduling resources are sufficient, and the memory utilization rate can be improved as much as possible on the basis of balancing the reading performance and the hit rate when the scheduling resources are insufficient.

It should be noted that, the prefetcher 5011 does not only perform deduplication on metadata and user data received simultaneously, and the prefetcher dimension has two alignments, including a first queue of user data to be scheduled and a second queue of metadata to be scheduled, where the first queue has prefetch address information of the user data to be scheduled, and the second queue has prefetch address information of the metadata to be scheduled.

The manner in which the data to be prefetched and the metadata to be prefetched are deduplicated is described below.

In an implementable manner, a preset functional relationship exists between an address of user data and an address of metadata of the user data, a prefetch address of each user data to be scheduled in a first queue is converted into an address of the metadata corresponding to the user data according to a preset address conversion relationship, and if the address of the metadata already exists in a second queue, the user data to be scheduled is determined to be repeated data.

For convenience of explanation, it is assumed that the prefetch address of the user data to be scheduled is denoted as DD, the prefetch address of the metadata to be scheduled is denoted as MD, one or more DDs of the user data to be scheduled exist in the first queue, and one or more MDs of the metadata to be scheduled exist in the second queue.

Illustratively, the address translation relationship is MD = f (DD). Each time a new DD is generated, the MD corresponding to the DD may be calculated according to the address translation relationship, and then, whether the MD exists in the second queue is queried, and if so, the DD is rejected from being inserted into the first queue, or the DD may be inserted into the first queue but the MD corresponding to the new DD is deleted from the second queue.

It should be noted that the generated new DD may be subjected to repeated prefetched deduplication, or may be subjected to repeated prefetched deduplication on the DD already existing in the first queue, for example, when the scheduling resources are sufficient, the new DD may be directly inserted into the first queue, when the scheduling resources are insufficient, the repeated prefetched deduplication may be performed on the new DD in the above manner, or may be performed on each DD already inserted into the first queue.

By the method, after receiving a read data request, the storage device predicts data which is possibly read in the future, such as first data and second data, according to the request, and judges whether the first data to be prefetched and the second data to be prefetched are related data; if the first data is associated data of the second data, a prefetch schedule of only one data is executed for the two prefetch data. Therefore, if the second data (such as user data) is prefetched into the memory, a subsequent read data request for the second data is received, and the second data can be directly obtained from the memory without obtaining the associated data of the second data; or, if the second data (e.g. metadata) is prefetched into the memory, then a data reading request for the first data is received subsequently, and the first data can also be obtained at a higher speed according to the associated data (i.e. the second data) of the first data in the memory, so that data with repeated semantics or possibly without revenue generation is stored in the memory element as much as possible, and the resource utilization rate of the memory is improved while the resource scheduling overhead is reduced.

Referring to fig. 6, an embodiment of the present application further provides another cache management method, and similarly, the cache management method is applied to the system architectures shown in fig. 1A to 1E to describe the cache management method provided in the embodiment of the present application, where the storage device may be the hardware architecture shown in fig. 3 or the hardware architecture shown in fig. 5, that is, the implementation of the method may be based on a prefetch schedule after prefetching data is deduplicated, or a prefetch schedule without prefetching data is deduplicated.

In the embodiment shown in fig. 6, a process in which the cache management method is applied to the prefetch operation will be described. As shown in fig. 6, the method includes the steps of:

in step 601, the memory device receives a read data request, wherein the read data request requests to access first data.

In step 602, the memory device determines second data to be prefetched according to the read data request.

The second data may be user data or metadata. If the second data is user data, performing step 603a or step; if the second data is metadata, step 603b is performed.

Step 603a, the storage device determines whether metadata of the second data (user data) is in the memory; if the metadata of the second data is in the memory, the pre-fetching of the second data is not triggered. Or if the metadata of the second data is not in the memory, the pre-fetching operation of the second data is triggered.

In step 602, the prefetch address of the second data is determined, and the prefetch address may be a Logical Block Address (LBA) of the second data, that is, the second data to be prefetched is identified by the LBA.

The metadata includes a Value (Value) information, an address field and the like are nested in the Value, the address field includes an address of user data corresponding to the metadata, such as an LBA, and if the LBA included in a certain metadata in the memory is a prefetch address of the second data to be prefetched, the metadata is determined to be the metadata of the second data.

In an implementation manner, the heat value of the second data is not limited, that is, whether the second data is heat data or not is not concerned, and if the metadata of the second data is in the memory, the prefetching of the second data is not performed. In another implementation manner, before performing step 603a, it may be further determined whether the second data is hot data, for example, whether the hot value of the second data exceeds a second preset threshold, if so, step 603a is performed, otherwise, the pre-fetching operation of the second data is triggered, that is, if the second data is hot data, the pre-fetching operation of the second data is triggered regardless of whether the metadata of the second data is in the memory. Therefore, the hot data can be acquired at a higher speed, the utilization rate of memory resources is improved, the reading performance of equipment is guaranteed, and the hit rate is improved.

As another optional implementation, if the memory resources are sufficient, the prefetch data may not be controlled, that is, the prefetch operation on the user data may be triggered regardless of whether the metadata of the user data is in the memory, and the prefetch operation on the metadata may be triggered regardless of whether the user data corresponding to the metadata is in the memory. Illustratively, when the remaining memory space capacity is smaller than a third preset threshold, the memory resource is considered to be insufficient, otherwise, the memory resource is considered to be sufficient.

In practical applications, when the memory resources are sufficient, although the prefetch data may not be controlled, the prefetch operation of the user data whose metadata is not in the memory and the prefetch operation of the metadata whose metadata is not in the memory may be preferentially performed. For example, the data to be prefetched includes metadata a and metadata B, and if the user data a corresponding to the metadata a is in the memory and the user data B corresponding to the metadata B is not in the memory, the prefetching scheduling of the metadata B may be preferentially performed. Therefore, when the user data B is accessed, the metadata B can be acquired at a higher speed, so that the reading performance of the equipment is improved, and the utilization rate of the memory resources is balanced as much as possible.

Step 603b, the storage device determines whether the user data corresponding to the second data (metadata) is in the memory; and if the user data corresponding to the second data is in the memory, not triggering the pre-fetching operation of the second data. Or if the user data corresponding to the second data is not in the memory, triggering the pre-fetching operation of the second data.

It should be noted that step 603a and step 603b are not steps executed at the same time, and are shown as dashed boxes in fig. 6.

By the method, after receiving the read data request, the storage device predicts the data which can be read in the future according to the request and judges whether the associated data of the data is stored in the memory. If the associated data is already stored in the memory, no prefetch operation is performed. Therefore, the data of the repeated semantics stored in the memory side can be reduced, wherein the repeated semantics comprise the same data, the user data and the metadata corresponding to the user data, and the metadata and the user data corresponding to the metadata, so that the data of the repeated semantics may not be accessed at all on the memory side.

In the foregoing, the memory hit rate is improved by the prefetch method, and the embodiment of the present application may also improve the memory hit rate by memory elimination, that is, when the memory space is insufficient, relatively cold data in the memory may be eliminated, so as to save more memory space for storing hot data.

The following describes the existing data elimination process with reference to fig. 1A and fig. 2:

for the metadata cache 22, when the total amount of data in the metadata cache 22 reaches a certain threshold, the processor 123 determines metadata to be eliminated, such as metadata that has not been accessed for a long time or metadata that is accessed less frequently, and the processor 123 eliminates or deletes the metadata from the metadata cache 22. In some scenarios, it may also be referred to as swapping out.

For the data buffer 21, when the total amount of data in the data swap-out area 21 reaches a certain threshold, the processor 123 determines user data to be eliminated, for example, user data that has not been accessed for a long time or user data that has been accessed less frequently, and the processor 123 eliminates the portion of user data from the data buffer 21. It should be noted that, if the part of data to be eliminated is located in the read data buffer area 211, the part of data may be eliminated directly, and if the part of data to be eliminated is located in the write data buffer area 210 and is not stored in the hard disk 105, the part of data may be moved to the hard disk 105; if the hard disk 105 stores the data, the data can be directly eliminated.

A third cache management method according to the embodiment of the present application is provided as follows, and in the embodiment shown in fig. 7, the application of the cache management method to a memory eviction process will be described.

Fig. 7 is a schematic flow chart of the method. Similarly, the cache elimination method provided in the embodiment of the present application will be described by applying the cache management method to the system architecture shown in fig. 1A to 1E. The storage device may be the hardware architecture shown in fig. 3, the hardware architecture shown in fig. 5, or other hardware architectures, which is not limited in this embodiment of the present application. As shown in fig. 7, the method includes the steps of:

in step 701, the storage device detects a condition of elimination.

The elimination condition may be that the total amount of data in the first storage area exceeds a fourth preset threshold, where the first storage area may be the metadata cache area 22 in fig. 2, or may also be the data cache area 21, or may also be the entire memory area.

It should be noted that step 701 is not a necessary step, and is therefore shown by a dashed box in fig. 7.

Step 702, the storage device determines target data to be eliminated in the memory.

For example, the data to be eliminated, that is, the target data, may be determined according to an elimination algorithm, for example, an Least Recently Used (LRU) algorithm, and the embodiment of the present application is not limited to a manner of determining the data to be eliminated.

Illustratively, the metadata to be evicted in the in-memory metadata cache 22 may be determined. As another example, it may also be determined that user data to be eliminated in the data cache area 21 in the memory is to be eliminated.

If the target data is user data, step 703a is performed. Alternatively, if the target data is metadata, step 703b is performed.

Step 703a, the storage device determines whether metadata corresponding to the to-be-eliminated user data is in the memory, and if so, eliminates the user data; otherwise, the data is retained, i.e., the user data is not obsolete.

Step 703b, the storage device determines whether the user data corresponding to the metadata to be eliminated is in the memory, and if so, eliminates the metadata; otherwise, the data is retained, i.e., the metadata is not obsolete.

By the mode, the storage equipment can eliminate data with repeated semantics at the memory side, improve the utilization rate of memory resources,

when the storage device triggers data elimination of the memory, the storage device may preferentially eliminate target data of the associated data in the memory, so that since the associated data is already stored in the memory, a subsequent read data request for the target data (such as user data) is received, the target data may be obtained at a higher speed according to the associated data of the target data, or a subsequent read data request for the associated data (such as user data) is received, the associated data may be directly obtained in the memory, and the target data does not need to be obtained; compared with the situation that both the user data and the metadata are not hit in the memory, the method can still acquire the data at a relatively high speed for feedback, and on the basis of balancing the utilization rate of memory resources, the reading performance and the hit rate of the storage device are guaranteed as far as possible.

Meanwhile, as the user data or the metadata corresponding to the user data is reserved on the memory side, the user data can be acquired at a higher speed according to the metadata of the user data after a data reading request aiming at the user data is received, or the user data can be acquired in the memory directly after the data reading request is received, and the metadata of the user data does not need to be acquired any more

In the above, the user data and the metadata are taken as an example of a pair of associated data, and the cache management policy provided by the present application is described in detail, the associated data involved in the cache management method provided by the present application may also be of other types, for example, the associated data of the user data may also be other data used for indicating the user data, and for example, the associated data of the metadata may also be other data used for describing or indicating the user data. The embodiment of the present application does not limit this.

Based on the same inventive concept as the method embodiment, the embodiment of the present application further provides a cache management apparatus, where the cache management apparatus is configured to execute the method executed by the storage device in the method embodiment shown in fig. 4. As shown in fig. 8, the cache management apparatus 800 includes an obtaining module 801 and a processing module 802; optionally, a detection module 803 may be further included, and since the detection module 803 and the prefetch module 804 are optional modules, they are shown in fig. 8 as dashed boxes. Specifically, in the cache management device 800, the modules are connected to each other through a communication path.

An obtaining module 800, configured to obtain first prefetch information and second prefetch information; the first prefetching information comprises information of first data to be prefetched; the second prefetch information includes information of second data to be prefetched;

a processing module 802, configured to determine whether the first data is associated with the second data; if so, the first prefetch information is deleted or the second prefetch information is deleted.

As a possible implementation manner, the second data is user data, and the associated data of the second data is metadata of the user data; or the second data is metadata, and the associated data of the second data is user data indicated by the metadata.

As a possible implementation, the first prefetch information includes a first prefetch address of the first data; the second prefetch information includes a second prefetch address of the second data; the processing module 802 is specifically configured to determine that the first data is associated with the second data if the first prefetch address and the second prefetch address have a preset functional relationship.

As a possible implementation, if the first prefetch information is deleted, the second prefetch information is retained; a prefetch module 804 for moving the second data from the secondary storage unit to the primary storage unit according to the second prefetch information; or deleting the second pre-fetching information and keeping the first pre-fetching information; the prefetch module 804 is configured to move the first data from the secondary storage unit to the primary storage unit according to the first prefetch information.

As a possible implementation manner, the detecting module 803 is further configured to detect a management condition, where the management condition includes that the remaining available space of the primary storage unit is smaller than a first preset threshold; or the read bandwidth is above a second preset threshold.

It should be noted that the above modules may also be executed by a processor in the storage device or a processor calling instructions in the memory.

Based on the same inventive concept as the method embodiment, an embodiment of the present application further provides a cache management apparatus, where the cache management apparatus is configured to execute the method executed by the storage device in the method embodiment shown in fig. 6. As shown in fig. 9, the cache management apparatus 900 includes a receiving module 901 and a processing module 902; optionally, a detection module 903 and a prefetch module 904 may be further included, and since the detection module 903 and the prefetch module 904 are optional modules, they are shown in fig. 9 by dashed boxes. Specifically, in the cache management apparatus 900, the modules are connected to each other through a communication path.

A receiving module 901, configured to receive a read data request, where the read data request is used to request to access first data;

a processing module 902, configured to determine one or more second data to be prefetched according to the read data request; judging whether the associated data of the third data is in the primary storage unit, if so, not triggering the pre-fetching operation of the third data; otherwise, triggering the pre-fetching operation of the third data; wherein the third data is part or all of the one or more second data.

As a possible implementation manner, the third data is any one of the one or more second data whose heat value is lower than the first preset threshold.

As a possible implementation manner, the third data is user data, and the associated data of the third data is metadata of the user data; or the third data is metadata of the user data, and the associated data of the third data is the user data.

As a possible implementation manner, the detecting module 903 is further configured to detect a management condition, where the management condition includes that the remaining available space of the primary storage unit is smaller than a second preset threshold, or the read bandwidth is higher than a third preset threshold.

As a possible implementation manner, the pre-fetching module 904 is configured to obtain the third data from a secondary storage unit, and write the third data into the primary storage unit; wherein the performance of the primary storage unit is higher than that of the secondary storage unit.

Based on the same inventive concept as the method embodiment, the embodiment of the present application further provides a cache management apparatus, where the cache management apparatus is configured to execute the method executed by the storage device in the method embodiment shown in fig. 7. As shown in fig. 10, the cache management apparatus 1000 includes a determination module 1001 and a processing module 1002; optionally, a detection module 1003 may be further included, and since the detection module 1003 is an optional module, it is shown by a dashed box in fig. 10. Specifically, in the cache management device 1000, the modules are connected to each other through a communication path.

A determining module 1001, configured to determine first data to be eliminated in a primary storage unit;

the processing module 1002 is configured to determine whether data associated with the first data is in the primary storage unit, if so, eliminate the first data, and otherwise, retain the first data.

As a possible implementation manner, the first data is user data, and the associated data of the first data is metadata of the user data; or the first data is metadata of the user data, and the associated data of the first data is the user data.

As a possible implementation manner, the detecting module 1003 is further configured to detect a removal condition, where the removal condition includes that the remaining available space of the primary storage unit is smaller than a first preset threshold.

The division of the modules in the embodiments of the present application is schematic, and is only a logic function division, and in actual implementation, there may be another division manner, and in addition, each function module in each embodiment of the present application may be integrated in one processor, or may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.

The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a terminal device (which may be a personal computer, a mobile phone, or a network device) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The descriptions of the flows corresponding to the above-mentioned figures have respective emphasis, and for parts not described in detail in a certain flow, reference may be made to the related descriptions of other flows.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes computer program instructions which, when loaded and executed on a computer, cause the processes or functions described in fig. 3 to be performed, in whole or in part, in accordance with embodiments of the invention.

The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, fiber optic, digital subscriber line, or wireless (e.g., infrared, wireless, microwave, etc.) manner.

It will be apparent to those skilled in the art that various changes and modifications can be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A cache management method is applied to a storage device, the storage device at least comprises a first-level storage unit and a second-level storage unit, the performance of the first-level storage unit is higher than that of the second-level storage unit, and the method comprises the following steps:

receiving a read data request, wherein the read data request is used for requesting to access first data;

determining one or more second data to be prefetched according to the read data request;

judging whether the associated data of the third data is in the primary storage unit;

if the associated data of the third data exists in the primary storage unit, not triggering the pre-fetching operation of the third data; alternatively, the first and second liquid crystal display panels may be,

if the associated data of the third data does not exist in the primary storage unit, triggering the pre-fetching operation of the third data;

wherein the third data is part or all of the one or more second data.

2. The method of claim 1, wherein the third data is any one of the one or more second data having a heat value lower than a first predetermined threshold.

3. The method according to claim 1 or 2, wherein the third data is user data, and the associated data of the third data is metadata of the user data; or the third data is metadata of user data, and the associated data of the third data is the user data.

4. The method according to any one of claims 1 to 3, wherein the determining whether the data associated with the third data is in the primary storage unit further comprises:

detecting a control condition, wherein the control condition comprises that the residual available space of the primary storage unit is smaller than a second preset threshold value, or the reading bandwidth is higher than a third preset threshold value.

5. The method of any of claims 1-4, wherein the triggering the prefetch operation of the third data comprises:

and acquiring the third data from the secondary storage unit, and writing the third data into the primary storage unit.

6. A cache management method is applied to a storage device, the storage device at least comprises a first-level storage unit and a second-level storage unit, the performance of the first-level storage unit is higher than that of the second-level storage unit, and the cache management method comprises the following steps:

determining first data to be eliminated in the primary storage unit;

judging whether the associated data of the first data is in the primary storage unit;

if the associated data of the first data exists in the primary storage unit, eliminating the first data; alternatively, the first and second electrodes may be,

and if the associated data of the first data does not exist in the primary storage unit, reserving the first data.

7. The method of claim 6, wherein the first data is user data, and the associated data of the first data is metadata of the user data; or the first data is metadata of user data, and the associated data of the first data is the user data.

8. The method of claim 6 or 7, further comprising:

detecting a removal condition, wherein the removal condition comprises that the remaining available space of the primary storage unit is smaller than a first preset threshold value.

9. A cache management method is applied to a storage device, the storage device at least comprises a first-level storage unit and a second-level storage unit, the performance of the first-level storage unit is higher than that of the second-level storage unit, and the cache management method comprises the following steps:

acquiring first pre-fetching information and second pre-fetching information; wherein the first prefetch information includes information of first data to be prefetched; the second prefetch information includes information of second data to be prefetched;

judging whether the first data is associated with the second data; and if so, deleting the first pre-fetch information or deleting the second pre-fetch information.

10. The method of claim 9, wherein the second data is user data, and the associated data of the second data is metadata of the user data; or the second data is user data metadata, and the associated data of the second data is the user data.

11. The method of claim 9 or 10, wherein the first prefetch information includes a first prefetch address of the first data; the second prefetch information includes a second prefetch address of the second data;

judging whether the first data is the associated data of the second data, including:

and if the first pre-fetching address and the second pre-fetching address have a preset functional relationship, determining that the first data is associated data of the second data.

12. The method of any of claims 9-11, wherein the first prefetch information is deleted, the second prefetch information is retained; the method further comprises the following steps:

moving the second data from the secondary storage unit to the primary storage unit according to the second pre-fetch information; alternatively, the first and second liquid crystal display panels may be,

deleting the second pre-fetching information and keeping the first pre-fetching information; the method further comprises the following steps:

and moving the first data from the secondary storage unit to the primary storage unit according to the first pre-fetching information.

13. The method of any of claims 9-12, wherein determining whether the first data is previous to the data associated with the second data further comprises:

detecting a control condition, wherein the control condition comprises that the residual available space of the primary storage unit is smaller than a first preset threshold value; or the read bandwidth is above a second preset threshold.

14. An apparatus for cache management, the apparatus comprising:

the device comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for receiving a read data request which is used for requesting to access first data;

the processing module is used for determining one or more second data to be prefetched according to the read data request; judging whether the associated data of the third data is in the primary storage unit; if the associated data of the third data exists in the primary storage unit, not triggering the pre-fetching operation of the third data; or if the associated data of the third data does not exist in the primary storage unit, triggering the pre-fetching operation of the third data; wherein the third data is part or all of the one or more second data.

15. The apparatus of claim 14, wherein the third data is any one of the one or more second data whose heat value is lower than a first preset threshold.

16. The apparatus according to claim 14 or 15, wherein the third data is user data, and the associated data of the third data is metadata of the user data; or the third data is metadata of the user data, and the associated data of the third data is the user data.

17. The apparatus of any one of claims 14-16, further comprising a detection module; the detection module is further configured to detect a management and control condition, where the management and control condition includes that a remaining available space of the primary storage unit is smaller than a second preset threshold, or a read bandwidth is higher than a third preset threshold.

18. The apparatus of any of claims 14-17, wherein the apparatus further comprises a pre-fetch module; the pre-fetching module is used for acquiring the third data from the secondary storage unit and writing the third data into the primary storage unit; wherein the performance of the primary storage unit is higher than that of the secondary storage unit.

19. An apparatus for cache management, the apparatus comprising:

the determining module is used for determining first data to be eliminated in the primary storage unit;

the processing module is used for judging whether the associated data of the first data is in the primary storage unit; if the associated data of the first data exists in the primary storage unit, eliminating the first data; or if the associated data of the first data does not exist in the primary storage unit, the first data is reserved.

20. The apparatus of claim 19, wherein the first data is user data, and the associated data of the first data is metadata of the user data; or the first data is metadata of the user data, and the associated data of the first data is the user data.

21. The apparatus of claim 19 or 20, wherein the apparatus further comprises a detection module; the detection module is further configured to detect an elimination condition, where the elimination condition includes that the remaining available space of the primary storage unit is smaller than a first preset threshold.

22. An apparatus for cache management, the apparatus comprising:

the acquisition module is used for acquiring first pre-fetching information and second pre-fetching information; wherein the first prefetch information includes information of first data to be prefetched; the second prefetch information includes information of second data to be prefetched;

the processing module is used for judging whether the first data is the associated data of the second data; and if so, deleting the first pre-fetch information or deleting the second pre-fetch information.

23. The apparatus of claim 22, wherein the second data is user data, and the associated data of the second data is metadata of the user data; or the second data is metadata, and the associated data of the second data is the user data indicated by the metadata.

24. The apparatus of claim 22 or 23, wherein the first prefetch information includes a first prefetch address for the first data; the second prefetch information includes a second prefetch address of the second data;

the processing module is specifically configured to determine that the first data is associated with the second data if a preset functional relationship exists between the first prefetch address and the second prefetch address.

25. The apparatus of any one of claims 22-24, wherein the apparatus further comprises a pre-fetch module;

the prefetching module is configured to, when the first prefetching information is deleted and the second prefetching information is retained, move the second data from the secondary storage unit to the primary storage unit according to the second prefetching information; or

And the second pre-fetching unit is used for moving the first data from the secondary storage unit to the primary storage unit according to the first pre-fetching information when the second pre-fetching information is deleted and the first pre-fetching information is reserved.

26. The apparatus of any one of claims 22-25, wherein the apparatus further comprises a detection module; the detection module is further configured to detect a management and control condition, where the management and control condition includes that a remaining available space of the primary storage unit is smaller than a first preset threshold; or the read bandwidth is above a second preset threshold.

27. A storage device, wherein the computing device comprises a processor and a memory;

the memory to store computer program instructions;

the processor executing instructions that invoke the computer program in the memory performs the method of any of claims 1 to 5, or performs the method of any of claims 6 to 8, or performs the method of any of claims 9 to 13.

28. A computer-readable storage medium, wherein when executed by a computing device, the computing device performs the method of any of claims 1 to 5, or performs the method of any of claims 6 to 8, or performs the method of any of claims 9 to 13.