CN114968076A

CN114968076A - Method, apparatus, medium, and program product for storage management

Info

Publication number: CN114968076A
Application number: CN202110615806.1A
Authority: CN
Inventors: 彭俊毅
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-02-25
Filing date: 2021-06-02
Publication date: 2022-08-30
Also published as: WO2022179032A1

Abstract

Embodiments of the present disclosure provide methods, apparatuses, media and program products for storage management. According to aspects of the present disclosure, a driver in an operating system may determine access information related to an access request in response to the operating system receiving the access request from an application layer. A manager in the application layer may obtain the access information. Thus, the manager may determine data information for potential access data of the application layer based on the access information and make the data information available to a driver in the operating system. Thus, the drive can load potential access data to the cache based on the data information, thereby improving storage performance, access efficiency, and user experience.

Description

Method, apparatus, medium, and program product for storage management

Technical Field

Embodiments of the present disclosure relate generally to the field of storage management and, more particularly, relate to a method, apparatus, medium, and program product for storage management.

Background

With the development of information technology, various storage devices have appeared. In general, the capacity of a storage device cannot be matched to input/output (I/O) performance or access performance. Specifically, the bandwidth of the storage device is inversely proportional to the delay and capacity. That is, the larger the bandwidth of the storage device, the lower the latency, and the smaller the capacity. For this reason, data acquired from a large-capacity low-speed storage device can be cached using a small-capacity high-speed storage device to improve the access performance of the entire system. However, current caching schemes are inefficient and complex and are not easily updated.

Disclosure of Invention

In general, embodiments of the present disclosure provide a solution for storage management to improve storage performance, access efficiency, and user experience.

In a first aspect of the disclosure, a storage management method is provided. The method comprises the following steps: a manager in the application layer acquires access information, wherein the access information is related to an access request received by an operating system from the application layer; the manager determines data information of potential access data of the application layer based on the access information; and the manager makes the data information available to a driver in the operating system to cause the driver to load potential access data into the cache based on the data information. For example, the potential access data may be associated with data to be accessed by the access request because the potential access data is predicted based on the data to be accessed by the access request. In this way, the manager may predict data information regarding potential access data that may be to be accessed based on the access information, and the drive may prefetch the potential access data based on the data information, thereby improving storage performance, access efficiency, and user experience.

In some embodiments, obtaining access information comprises: in response to receiving a notification from the drive regarding the access information, access information is retrieved from a first buffer in memory, the first buffer maintained by the drive for storing the access information. In this way, the manager can conveniently obtain access information from a buffer maintained by the drive based on the notification from the drive.

In some embodiments, determining the data information comprises: the access information is applied to a trained predictor to obtain data information, the predictor being trained based on training access information and training data information associated with the training access information. The predictor is a model trained based on training access information and training data information, which is used to predict access information of subsequent potential access data based on current access information. A manager in the application layer may be deployed with or retrieve the trained predictor and utilize the predictor to determine data information. In the training of the predictor, the historical access information may be used as training access information, and the historical data information associated with the historical access information may be used as training data information. As an example, the history data information may be information associated with a subsequently performed access operation after the execution of the access operation with respect to the history access information, such as an address, a size, and the like of data accessed by the subsequently performed access operation. The predictor may be implemented using any suitable prediction technique, such as random forest, least squares polynomial fitting, unary linear regression, multiple linear regression, non-linear methods, Markov chain prediction, data mining, neural networks, and the like. In this way, the trained predictor may be utilized to accurately and efficiently determine data information.

In some embodiments, determining the data information comprises: the data information is determined based on cache eviction information and access information retrieved from memory. For example, the manager may determine cache usage based on cache eviction information to determine whether to enable/disable prefetching or what prefetching policy to employ when determining that prefetching is enabled, and determine data information in conjunction with the access information based on the determined prefetching policy. In this way, the manager may accurately predict data information for potential access data in conjunction with cache eviction information and access information.

In some embodiments, the cache eviction information comprises at least one of: information associated with used portions in the cache when the cache is evicted, information associated with unused portions in the cache when the cache is evicted, and information associated with overwritten data in the cache when the cache is evicted. In this way, the manager may accurately predict data information for potentially accessed data taking into account various information related to eviction of the cache.

In some embodiments, the method further comprises: the manager, after determining the data information, sends a notification regarding the data information to the drive for the drive to retrieve the data information from a second buffer in the memory based on the notification, the second buffer maintained by the drive for storing the data information. In this way, the drive may conveniently retrieve data information from another buffer it maintains, loading potential access data into the cache based on the data information.

In some embodiments, the access information comprises at least one of: the time at which the access request was initiated; accessing an address of data to be accessed by the request; the size of the data to be accessed by the access request; the number of hit access requests, the data to be accessed by the hit access requests having been loaded into the cache; and the proportion of access requests that hit. In this way, the manager may accurately predict data information for potential access data taking into account various information related to the access request.

In some embodiments, the data information comprises at least one of: the address of the potential access data, and the size of the potential access data. In this way, the drive can conveniently retrieve the potential access data.

In some embodiments, the potential access data is stored in a storage device, and determining the data information comprises: determining the data information in response to determining that a number of access requests of an application layer is below a maximum throughput of a storage device. In this way, prefetching may be disabled or enabled according to different load conditions, thereby achieving better performance.

In a second aspect of the present disclosure, there is provided a storage management method including: a driver in the operating system determines access information related to the access request in response to the operating system receiving the access request from the application layer; the driver acquires data information of potential access data of the application layer, wherein the data information is determined by a manager in the application layer based on the access information; and the driver loads the potential access data into the cache based on the data information. In this way, the drive can prefetch potential access data that is likely to be accessed based on the data information determined by the manager, thereby improving storage performance, access efficiency, and user experience.

In some embodiments, the method further comprises: the driver, after determining the access information, sends a notification regarding the access information to the manager for the manager to retrieve the access information from a first buffer in the memory based on the notification, the first buffer maintained by the driver for storing the access information. In this way, the manager can conveniently obtain access information from a buffer maintained by the drive based on the notification from the drive.

In some embodiments, obtaining the data information comprises: in response to receiving a notification from the manager about the data information, the data information is retrieved from a second buffer in the memory, the second buffer maintained by the drive for storing the data information. In this way, the drive may conveniently retrieve data information from another buffer it maintains, loading potential access data into the cache based on the data information.

In some embodiments, the data information is determined by the manager based on cache eviction information and access information retrieved from memory. For example, the manager may determine cache usage based on cache eviction information to determine whether to enable/disable prefetching or what prefetching policy to employ when determining that prefetching is enabled, and determine data information in conjunction with the access information based on the determined prefetching policy. In this way, the manager may accurately predict data information for potential access data in conjunction with cache eviction information and access information.

In some embodiments, the data information comprises at least one of: the address of the potential access data, and the size of the potential access data. In this way, the drive can conveniently acquire the potential access data.

In some embodiments, the potential access data is stored in the storage device and the data information is determined by the manager in response to determining that the number of access requests of the application layer is below a maximum throughput of the storage device. In this way, prefetching may be disabled or enabled according to different load conditions, thereby achieving better performance.

In a third aspect of the disclosure, an electronic device is provided. The electronic device comprises at least one processing unit and at least one memory. At least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the apparatus to perform the method of the first aspect.

In a fourth aspect of the present disclosure, an electronic device is provided. The electronic device comprises at least one processing unit and at least one memory. At least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the apparatus to perform the method of the second aspect.

In a fifth aspect of the disclosure, a computer-readable storage medium is provided. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of the first aspect.

In a sixth aspect of the disclosure, a computer-readable storage medium is provided. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of the second aspect.

In a seventh aspect of the disclosure, a computer program product is provided. The computer program product comprises instructions, wherein the instructions, when executed by the processor, cause the electronic device to perform the method according to the first aspect.

In an eighth aspect of the disclosure, a computer program product is provided. The computer program product comprises instructions, wherein the instructions, when executed by the processor, cause the electronic device to perform the method according to the first aspect.

It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:

FIG. 1 illustrates a schematic diagram of an example three-level storage structure;

FIG. 2 illustrates a schematic diagram of an example conventional storage management environment;

FIG. 3 illustrates a schematic diagram of an example storage management environment in which embodiments of the present disclosure can be implemented;

FIG. 4 illustrates a schematic diagram of a storage management process, according to some embodiments of the present disclosure;

FIG. 5 illustrates a swim lane diagram of an example conventional caching process;

FIG. 6 illustrates a swim lane diagram of an example caching process with prefetching, according to some embodiments of the present disclosure;

FIG. 7 illustrates a flow diagram of a storage management process performed at a manager in accordance with some embodiments of the present disclosure;

FIG. 8 illustrates a flow diagram of a storage management process performed at a drive in accordance with some embodiments of the present disclosure; and

FIG. 9 illustrates a block diagram of an example device suitable for implementing embodiments of the present disclosure.

Detailed Description

The principles of the present disclosure will now be described with reference to a few exemplary embodiments. It is understood that these examples are described for illustrative purposes only and to aid those skilled in the art in understanding and enabling the present disclosure, and do not imply any limitation on the scope of the present disclosure. The disclosure described herein may be implemented in various ways other than those described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term "comprising" and its variants should be read as open-ended terms, meaning "including but not limited to". The term "based on" should be read as "based at least in part on. The terms "one embodiment" and "an embodiment" should be read as "at least one embodiment". The term "another embodiment" should be understood as "at least one other embodiment". The terms "first," "second," and the like may refer to different or the same object. Other definitions (explicit and implicit) may be included below.

As described above, with the development of information technology, various storage devices have appeared. In general, the capacity of a storage device cannot be matched to the I/O performance or access performance. Specifically, the bandwidth of the storage device is inversely proportional to the delay and capacity. That is, the larger the bandwidth of the storage device, the lower the latency, but the smaller the capacity. For this reason, data acquired from a large-capacity low-speed storage device can be cached using a small-capacity high-speed storage device to improve the access performance of the entire system.

The manner in which data retrieved from a large capacity low speed storage device is cached using a small capacity high speed storage device may be considered a hierarchical storage structure. Typically, the hierarchical storage structure may be a three-level storage structure. FIG. 1 illustrates a schematic diagram of an example three-level memory structure 100.

As shown in FIG. 1, registers 112 and one-to-three

level caches

114 and 118 within Central Processing Unit (CPU)110 are located at the first level of the three-level memory structure 100, which has large bandwidth, low latency, and small capacity.

Dynamic Random Access Memory (DRAM)120 is located in the second level of the three-level memory structure 100. The DRAM 120 has a smaller bandwidth, a larger latency and a larger capacity than the first level register 112 and the one-to-three

level cache

114 and 118. For example, DRAM 120 may have a delay of 50-100 ns. Further, the storage level memory (SCM)130 may be considered to be located between the second and third levels of the three-level storage architecture 100. Similar to DRAM 120, SCM 130 has a smaller bandwidth, a larger latency and a larger capacity than first level registers 112 and one-to-three

level caches

114 and 118. For example, the SCM 130 may have a delay of 120 ns.

Network storage 140 and network storage 150 are located at the third level of the three-level storage architecture 100. Network storage 140 and network storage 150 have smaller bandwidth, larger latency and larger capacity than second level DRAM 120 and SCM 130. For example, network storage 140 and network storage 150 may have a latency of 1-1000 μ s. In addition, a Solid State Drive (SSD)160 and a Hard Disk Drive (HDD)170 are also located at the third level of the three-level storage architecture 100. The SSD 160 and HDD 170 have smaller bandwidth, larger latency, and larger capacity than the second level DRAM 120 and SCM 130. For example, SSD 160 may have a 100 μ s delay, while HDD 170 may have a 1ms delay.

It can be seen that, from the register 112, the first-level to third-

level cache

114 and 118 in the CPU 110, to the DRAM 120, the SCM 130, and then to the network memory 140, the network storage device 150, the peripheral SSD, the HDD, the closer to the CPU in the above three-level storage structure, the larger the bandwidth, the lower the delay, and the smaller the capacity.

The latency of the external storage device varies from hundreds of clock cycles (e.g., DRAM 120) to millions of clock cycles (e.g., HDD 170) as compared to only a few clock cycles required by the CPU to internally operate the instructions. However, the storage capacity and access performance cannot be achieved simultaneously. Therefore, the access performance of the whole system can be remarkably improved by using a high-speed storage device with small capacity as a cache. Conventionally, a scheme has been proposed that can host a cache and a low-speed backend storage and provide a unified virtual block device access interface to upper-layer applications to accelerate access performance.

On the other hand, prefetch techniques may significantly reduce latency and improve access performance by loading data ahead of time. Currently, from operating systems to a variety of large applications, respective prefetching schemes are developed from their own perspective to improve access performance. However, these conventional prefetching schemes have various problems. For example, it does not solve well the mutual influence of prefetching between applications. Furthermore, it also fails to dynamically adjust the prefetch policy based on system load.

Furthermore, for a better understanding of embodiments of the present disclosure, a conventional storage management environment will first be described with reference to FIG. 2.

FIG. 2 illustrates a schematic diagram of an example conventional storage management environment 200. As shown in fig. 2, the application 210 located at the application layer may perform read and write operations by calling an access interface of the operating system. The read and write operations may follow the following example procedure. For example, a Virtual File System (VFS)220 within the operating system may resolve accesses by the access interface to accesses at a corresponding logical address of the storage block device 240 and call the driver 230 to read and write data at that logical address. Further, storage block device 240 may invoke storage driver 260 to read and write data at the physical address of storage 270 corresponding to the logical address.

In systems employing virtual block appliances (e.g., storage block appliance 240 and cache block appliance 250), driver 230 hosts back-end storage 270 and cache 280 and presents a unified virtual block appliance interface to upper layers, as opposed to conventional access operations. In particular, storage 270 maps to virtual storage block device 240 and cache 280 maps to virtual cache block device 250. Thus, the driver 230 may host the storage 270 and the cache 280 through the storage block device 240 and the cache block device 250, and call the storage driver 260 and the cache driver 265 through the storage block device 240 and the cache block device 250 to enable access to the storage 270 and the cache 280. In general, the storage device 270 is a large-capacity and low-cost storage disk (e.g., HDD or the like). All data is ultimately stored in storage 270. Cache 280 employs a high-speed and small-capacity storage disk (e.g., NVME SSD, etc.). Cache 280 is used to cache data read from storage 270 for use by applications.

When the application 210 accesses data, the driver 230 will obtain an access request from the application 210. In general, for a read access request from an application 210, the driver 230 may look up in its hosted cache 280 whether the data to be accessed by the read access request is already stored in the cache 280. If the data to be accessed by the read access request is already stored in cache 280 (also referred to as a "hit"), driver 230 reads the data directly from cache 280 and returns to application 210. If the data to be accessed by the read access request is not included in cache 280 (also referred to as a "miss"), driver 230 forwards the read access request to storage 270. After reading the data from storage 270, it is returned to application 210. In addition, the driver 230 may determine whether to store the read data in the cache 280 according to system or user configuration.

It can be seen that in the case where the application 210 repeatedly reads the same data and the data is loaded into the cache 280, the system can quickly read the data from the cache 280 by making full use of the high-speed access characteristics of the cache 280, thereby significantly improving the system performance.

However, this conventional scheme relies on the application 210 repeatedly reading the same data to improve system performance. Therefore, only the second and subsequent accesses to the same data can improve performance. However, in many cases, the application 210 only accesses the same data once. Alternatively, in the case where the amount of data is so large that the cache 280 cannot accommodate the data, even if there are multiple accesses to the same data, the drive 230 has to read the data from the low-speed storage device 270 because the cache 280 cannot store the data, and thus the system performance cannot be improved.

For this reason, some prefetching schemes have been conventionally proposed to load data ahead of time, thereby reducing latency and improving performance. However, conventional solutions have various drawbacks. For example, traditionally, various applications have implemented their respective prefetching schemes from their own perspective, but these prefetching schemes do not address well the interplay of prefetching between various applications. In addition, the conventional scheme cannot adjust the prefetching strategy in real time according to the system load. Additionally, in conventional approaches, the pre-fetch scheme is implemented in the operating system kernel. However, the operating system kernel is critical to stability and security, while also requiring a fast response to access requests of all applications 210. In this case, implementing a pre-fetch scheme in the operating system kernel will severely impact the stability, security, and response speed of the system. It can be seen that the conventional scheme lacks a sophisticated and comprehensive prefetching scheme for various applications, and cannot rely on data prefetching to significantly improve system performance in a large number of application scenarios.

With respect to the above-described conventional art, embodiments of the present disclosure provide a scheme for storage management. In general, according to various embodiments described herein, a driver in an operating system, in response to the operating system receiving an access request from an application layer, may determine access information related to the access request. A manager in the application layer may obtain the access information. Thus, the manager can determine data information of data that the application layer is likely to access in the future (hereinafter also referred to as "potential access data of the application layer") based on the access information, and make the data information available to the driver in the operating system. Thus, the driver may load data to the cache that the application layer may access in the future based on the data information.

In this way, the manager may determine data information of potential access data based on the access information, and the drive may prefetch the potential access data that may be to be accessed based on the data information determined by the manager, thereby improving storage performance, access efficiency, and user experience. In this way, a storage access acceleration scheme based on an intelligent manager with drivers in the operating system in conjunction with the application layer is implemented. Compared with the conventional scheme discussed above, the scheme can construct an intelligent prefetching strategy from the access characteristics of the access request of the upper-layer application to the underlying storage device, so as to utilize the high-speed access characteristics of the cache to realize intelligent prefetching with stronger system performance.

In particular, in response to the defect that the conventional scheme can only improve the performance of the second and later accesses to the same data, the scheme can predict the data which is likely to be accessed based on the pattern of the data which is already accessed in the manager, and prefetch the data which is likely to be accessed into the cache. For this reason, data that is likely to be accessed is already cached when it is first accessed, so that the conventional dependence on secondary access can be overcome.

Furthermore, for situations where the amount of data is so large that the cache cannot accommodate, in some embodiments of the present disclosure, cache utilization may be determined in the manager based on cache eviction information to determine whether to enable/disable prefetching, or what prefetching policy to employ when determining to enable prefetching, to determine data information in conjunction with access information based on the determined prefetching policy. For example, where the cache load is small, aggressive prefetching strategies may be employed to prefetch more of the potentially accessed data; in the case of a large cache load, a conservative prefetch policy may be employed to prefetch less of the potentially accessed data. In this way, the capacity of the cache provided by the system can be adapted to avoid situations where the cache cannot accommodate.

In addition, compared with the situation that each application respectively realizes the prefetching operation, the method realizes a universal prefetching scheme suitable for various applications by a special manager, so that mutual influence among the prefetching operations of the applications is avoided. In particular, in conventional approaches, individual applications, although independent of each other, are deployed on the same hardware environment. The applications each implement their own prefetching policy, but lack uniform coordinated management, and thus may conflict with the underlying shared hardware access characteristics. However, in the present solution, the collection of the access information of the application is implemented in a driver in the operating system, and thus the collected is the global access situation of all applications of the application layer. In this case, the manager may implement a better prefetching policy based on the global access situation.

Further, aiming at the condition that the traditional scheme can not adjust the prefetching strategy in real time according to the system load, the scheme can adopt different prefetching strategies for different global load conditions. For example, for a storage device, prefetching may be intelligently disabled if the number of total access requests for each application reaches the maximum throughput of the storage device. Conversely, if the total number of access requests does not reach the maximum throughput of the storage device, prefetching may be enabled or even further more aggressive prediction strategies may be employed to prefetch more potential access data for better performance, as the system may now be subject to the waste of storage resources caused by prefetch misses. Various example embodiments of the present disclosure will be described below with reference to the accompanying drawings.

FIG. 3 illustrates a schematic diagram of an example storage management environment 300 in which embodiments of the present disclosure can be implemented.

Some of the components of the storage management environment 300 may be referred to in implementations of the conventional storage management environment 200. For example, the access interface for the application 210 to perform read and write operations is consistent with the access interface of the conventional storage management environment 200. In the following, an embodiment according to the present disclosure will be described further with reference to fig. 3, in particular an improvement over the conventional storage management environment 200.

The storage management environment 300 includes a manager 310 implemented at an application layer and improves a driver 320 in an operating system (more specifically, an operating system in a kernel state). The application layer is a hierarchy that runs applications. The application layer may be in a user state. In the design of operating systems, user states refer to non-privileged execution states. The kernel of the operating system prohibits code in this state from potentially dangerous operations, such as writing system configuration files, killing processes of other users, restarting the system, and the like. The kernel mode is a mode run by the kernel of the operating system, and codes running in the mode can access system storage and external equipment without limitation. The driver 320 may be used to manage the storage 270 and cache 280, which may be, for example, a virtual block device driver in an operating system. The driver 320 does not have a one-to-one correspondence with the underlying block devices (e.g., storage block device 240, cache block device 250, etc.), but is a proxy for the underlying block devices. The driver 320 may virtualize a block device and apply access to the block device, with the access actually being forwarded by the driver 320 to the underlying block device. Manager 310, alternatively referred to as a prefetcher, may be used to predict potential access data to be accessed to improve system performance.

In particular, in some embodiments, manager 310 may be an application that executes a pre-fetch algorithm in the background. Manager 310 may make a prefetch decision by calling an access interface of driver 320 to identify access characteristics of an application. In particular, the driver 320 may store access information related to the access request in the memory 330. Thus, manager 310 may retrieve the access information from memory 330 and determine data information for the potential access data based on the access information. The manager 310 may then store the data information in the memory 330. Thus, the driver 320 may retrieve data information from the memory 330 and load potential access data to the cache 280 based on the data information.

In some embodiments, the manager 310 may also be enabled and disabled. For example, a user or administrator of the system may choose to enable or disable the manager 310. Alternatively, manager 310 may be enabled or disabled based on system load conditions. For example, in the case where the number of access requests is below a threshold number, the manager 310 is enabled, thereby determining the number information. And in the event that the number of access requests exceeds a threshold number, the manager 310 is shut down. In addition, the prefetching algorithm employed by manager 310 may also be dynamically adjusted based on system load conditions. For example, in the case of a small system load, the manager 310 may employ a more aggressive prediction strategy to prefetch more potential access data for better performance. Conversely, in the case of a heavy system load, manager 310 may employ a more conservative prediction strategy to prefetch less potential access data to conserve storage resources.

Since the manager 310 operates as a background service in the intelligent prefetching framework scheme, no interface exists between the application 210 and the manager 310, and therefore the application 210 and the manager 310 do not perceive the existence of each other. Thus, the intelligent pre-fetch framework scheme can be used in any application scenario context, and the application 210 does not need to make any adaptations. Furthermore, because the hypervisor 310 is implemented in the application layer rather than the operating system kernel, stability requirements for the hypervisor 310 are relatively low, and the hypervisor 310 can be updated easily and without requiring adaptive modifications to the kernel, thereby affecting the kernel's stability. In addition, even if the manager 310 fails to prefetch, the normal operation of the system is not affected because the normal cache function operates normally. Further, because manager 310 may dynamically adjust based on system load, manager 310 may also be able to ensure efficient operation under a variety of storage access loads without affecting system robustness.

FIG. 4 illustrates a schematic diagram of a storage management process 400, according to some embodiments of the present disclosure.

A driver 320 in the operating system may determine access information related to the access request in response to the operating system receiving the access request from the application layer. The principle is that all applications at the application layer above the operating system access the storage 270 by calling the interface provided by the driver 320 so that the driver 320 can gather global access information for the storage 270.

Specifically, an access request for data from the application 210, after being processed by the operating system kernel, is converted into an access request for the storage device 270 storing the data. Specifically, as described above with reference to fig. 2, the application 210 located at the application layer may perform read and write operations by calling an access interface of the operating system. For example, the virtual file system 220 within the operating system may resolve accesses by the access interface to accesses at corresponding logical addresses of the storage block device 240 and call the drivers 320 to read and write data at the logical addresses. Further, storage block device 240 may invoke storage driver 260 to read and write data at the physical address of storage 270 corresponding to the logical address.

In this case, when the application 210 accesses data stored in the storage block device 240, the driver 320 will acquire, through the operating system kernel, access information 410 associated with an access request from the application. In some embodiments, the access request may include a read access request and/or a write access request. For example, access information 410 may include the time at which the access request was initiated; accessing an address of data to be accessed by the request; the size of the data to be accessed by the access request; the number of hit access requests, the data to be accessed by the hit access requests having been loaded into the cache; and the proportion of access requests that hit, etc.

In addition, to further provide more comprehensive information to improve the accuracy of the prefetch decision of manager 310, driver 320 may obtain cache eviction information associated with cache 280 in addition to access information 410. In some embodiments, the manager may determine cache usage based on cache eviction information to determine whether to enable/disable prefetching or what prefetching policy to employ when determining that prefetching is enabled, to determine data information 420 in conjunction with access information 410 based on the determined prefetching policy. For example, if the cache eviction information indicates that the size of the used portion of cache 280 is below a threshold size, which means that the load of cache 280 is small, prefetching may be enabled or a more aggressive prefetching policy employed, and such prefetching policy used to determine data information 420 based on access information 410.

In some embodiments, cache eviction information may include information associated with used portions of cache 280 when evicting cache 280. E.g. the address, size, scale, etc. of the used part. In addition, cache eviction information may also include information associated with unused portions in cache 280 when evicting cache 280. Such as the address, size, proportion, etc. of the unused portion. Further, cache eviction information may also include information associated with overwritten data in cache 280 as cache 280 is evicted. For example, the address, size, proportion, etc. of the dirty data stored in cache 280.

Additionally, in some embodiments, the driver 320 may store the access information 410 in a first buffer in the memory 330 to cause the manager 310 to retrieve the access information 410 from the first buffer. In addition, the driver 320 may also store the cache eviction information in the first buffer in the memory 330. Alternatively, the driver 320 may also send the access information 410 and/or cache eviction information directly to the manager 310. Storage 330 may also be generally referred to as "memory" or "main memory". Since the access information 410 and/or cache eviction information is stored in the memory 330, it may be quickly accessed for efficiently determining the data information 420. It will be appreciated that although storage is described herein as "memory" or "main memory," storage may also be implemented using any other suitable high speed storage device.

In particular, the first buffer may be a storage area opened up in the memory 330, maintained by the driver 320, organized and managed in a dynamic data structure. In some embodiments, the dynamic data structure may be a circular buffer, but embodiments of the disclosure are not limited thereto, e.g., the dynamic data structure may also be any suitable structure such as a linear buffer. When the operating system kernel calls the interface of the driver 320, the interface writes the access information 410 and/or cache eviction information into the first buffer for retrieval by the manager 310. However, embodiments of the present disclosure are not limited thereto, e.g., the driver 320 may also send the access information 410 and/or cache eviction information directly to the manager 310.

Further, in some embodiments, the driver 320 may also send a notification to the manager 310 regarding the access information 410 and/or cache eviction information to enable the manager 310 to obtain the access information 410 and/or cache eviction information. For example, the driver 320 may send a notification to the manager 310 when the driver 320 stores the access information 410 and/or cache eviction information in the first buffer. More specifically, the driver 320 may synchronously send a notification to the manager 310 when storing the access information 410 and/or cache eviction information, which may be considered a synchronization notification. For example, if the driver 320 synchronously notifies the manager 310, the driver 320 needs to wait for the manager 310 to read the access information 410 before returning after writing the access information 410. In this case, the write operation of the driver 320 and the read operation of the manager 310 are coupled to each other and are not independent. Alternatively, the storage of access information 410 and/or cache eviction information and the sending of notifications may also be asynchronous, and such a manner of notification may be considered asynchronous notification. For example, in asynchronous notification, the manager 310 does not read the access information 410 when the drive 320 completes writing the access information 410. More specifically, the driver 320 may write the access information 410 to the first buffer and return after the write is complete. The manager 310's own thread may then call the system interface to read the access information 410 from the first buffer. In this case, the process of the drive 320 writing the access information 410 and the process of the manager 310 reading the access information 410 are independent of each other. Further, in some embodiments, the notification may also be made in the event that a predetermined condition is triggered. For example, the driver 320 may send a notification to the manager 310 when the driver 320 detects that the amount of data of the access information 410 and/or the cache eviction information in the first buffer is greater than a threshold amount of data or greater than a predetermined percentage of the size of the first buffer. It should be appreciated that the notification may be either a synchronous notification or an asynchronous notification, as embodiments of the disclosure are not limited in this respect.

Thus, the manager 310 can acquire the access information 410. As described above, in some embodiments, the driver 320 may store the access information 410 and/or cache eviction information in the first buffer. To this end, the driver 320 may provide the manager 310 with an access interface to the first buffer. The manager 310 may access the first buffer through the access interface such that the manager 310 may obtain the access information 410 and/or cache eviction information from the first buffer. However, embodiments of the present disclosure are not limited thereto, e.g., the manager 310 may also receive the access information 410 and/or cache eviction information directly from the drive 320.

Further, as described above, in certain embodiments, the driver 320 may also send notifications to the manager 310 regarding access information 410 and/or cache eviction information. In this case, if the manager 310 receives a notification from the driver 320, the manager 310 may obtain the access information 410 and/or cache eviction information. For example, the manager 310 may obtain the access information 410 from the first buffer in response to receiving the notification. Alternatively, the manager 310 may monitor the first buffer, such that upon monitoring that the access information 410 and/or the cache eviction information is placed in the first buffer, or that the amount of data of the access information 410 and/or the cache eviction information is greater than a threshold amount of data, or that the amount of data of the access information 410 and/or the cache eviction information is greater than a predetermined percentage of the size of the first buffer, the manager 310 may proactively obtain the access information 410 and/or the cache eviction information.

Manager 310 determines data information 420 for potential access data at the application layer based on access information 410 and/or cache eviction information. The potential access data is associated with data to be accessed by the access request, e.g., the potential access data is data that is likely to be accessed further after the data targeted by the access request is accessed. By way of example, data information 420 may include an address of the potential access data (such as an address in storage 270 or storage block device 240), and a size of the potential access data, and so on. However, embodiments of the present disclosure are not limited thereto, for example, the data information 420 may also include a starting address, an offset, etc. of the potential access data in the storage 270 or the storage block device 240. In this manner, the address of the potentially accessed data in storage 270 or storage block device 240 may be determined by the starting address and the offset. Thus, potential access data may be obtained using the data information 420.

In particular, manager 310 may analyze the regularity in access information 410, making pre-fetch decisions, and thereby derive data information 420 for use in obtaining potential access data. In some embodiments, to determine data information 420, manager 310 may apply access information 410 to the trained predictor to obtain data information 420. The predictor is trained based on training access information and training data information associated with the training access information. For example, historical access information may be used to train a predictor, in which case the historical access information may also be referred to as training access information. Furthermore, historical data information associated with historical access information may also be used to train the predictor, in which case the historical data information may also be referred to as training data information. For example, the history data information may be information associated with a subsequently performed access operation after the execution of the access operation with respect to the history access information, such as an address, a size, and the like of data accessed by the subsequently performed access operation. The predictor is a model trained based on training access information and training data information, which is used to predict access information of subsequent potential access data based on current access information. Manager 310 may be deployed with or retrieve a trained predictor and utilize the predictor to determine data information. The predictor may be implemented using any suitable prediction technique, such as random forest, least squares polynomial fitting, unary linear regression, multiple linear regression, non-linear methods, Markov chain prediction, data mining, neural networks, and the like.

As a specific example, manager 310 may predict the location and size of potential access data to be accessed based on the address and size of the data for which the access request is directed, etc., indicated in access information 410. For example, the manager 310 may know that the application read 64KB of data at address 0x10000 at time 0 and read 64KB of data at address 0x20000 at time +5, and the manager may make a prefetch decision before time + 10. The prefetch decision indicates that 64KB of data at 0x30000 addresses are prefetched. Thus, data information 420 may indicate that the address of the potential access data is 0x30000 and the size is 64 KB.

In some embodiments, the manager 310 may store the data information 420 in a second buffer in the memory 330 to enable the drive 320 to retrieve the data information 420 from the second buffer. In particular, similar to the first buffer, the second buffer may be a storage area opened up in the memory 330, maintained by the driver 320, organized and managed in a dynamic data structure. In some embodiments, the dynamic data structure may be a circular buffer, but embodiments of the disclosure are not limited thereto, and the dynamic data structure may be any suitable structure, such as a linear buffer. The manager 310 may store the data information 420 to the second buffer through an interface provided by the driver 320 and thus provide to the driver 320. However, embodiments of the present disclosure are not limited thereto, and for example, the manager 310 may also directly send the data information 420 to the driver 320. It should be understood that although the second buffer is described herein as being located in the same memory 330 as the first buffer, the second buffer may be located in a different memory than the first buffer.

Further, in some embodiments, the administrator 310 may also send a notification about the data information 420 to the drive 320 to cause the drive 320 to retrieve the data information 420. For example, when the manager 310 stores the data information 420 in the second buffer, the manager 310 may send a notification to the driver 320. Similar to the notification sent by the driver 320 to the manager 310, the notification may be either a synchronous notification or an asynchronous notification, and may also be notified if a predetermined condition is triggered, and embodiments of the present disclosure are not limited herein.

In some embodiments, asynchronous notification may be employed for access information 410, while synchronous notification may be employed for data information 420 to improve system robustness and performance. The principle is that the driver 320 is in the operating system of the bottom layer, and the manager 310 is in the application layer, so the stability and security of the driver 320 need to be guaranteed. Specifically, if the drive 320 writes the access information 410 asynchronously, the drive 320 is not affected to write the access information 410 even if the manager 310 crashes. Whereas in the case where the writing of the data information 420 by the manager 310 is a synchronous operation, if the manager 310 crashes, the synchronous operation does not occur, so that the stability of the drive 320 is not affected. Further, if the drive 320 crashes, the synchronous operation of the manager 310 will fail, which also corresponds to the control logic, because if the drive 320 fails to work normally, the manager 310 also cannot work normally.

Thus, the driver 320 can acquire the data information 420. As described above, in some embodiments, manager 310 may store data information 420 in a second buffer. To this end, the driver 320 may provide the manager 310 with an access interface to the second buffer. The manager 310 may access the second buffer through the access interface so that the driver 320 may retrieve the data information 420 from the second buffer. However, embodiments of the present disclosure are not limited thereto, and for example, the driver 320 may also receive access information directly from the manager 310.

Further, as described above, in some embodiments, the manager 310 may also send notifications regarding the data information 420 to the driver 320. In this case, if the driver 320 receives a notification from the manager 310, the driver 320 may acquire the data information 420. For example, the driver 320 may retrieve the data information 420 from the second buffer in response to receiving the notification. Alternatively, the driver 320 may monitor the second buffer, such that the driver 320 may actively retrieve the data information 420 when it is monitored that the data information 420 is placed in the second buffer, or the amount of data of the data information 420 is greater than a threshold amount of data, or the amount of data of the data information 420 is greater than a predetermined percentage of the size of the second buffer.

Thus, the driver 320 may load the potential access data to the cache 280 based on the data information 420. In certain embodiments, the driver 320 may retrieve the potential access data from the storage 270 based on the data information 420 and store the potential access data to the cache 280. Specifically, after the driver 320 acquires the data information 420, the data having the specified size at the address indicated by the data information 420 may be read from the back-end storage 270 and written into the cache 280, thereby completing the prefetch process. For example, the drive 320 may read the data in the storage block device 240 at the address indicated by the data information 420. To do so, the driver 320 may call the storage block device 240, which in turn calls the storage driver 260 from the storage block device 240 to read the data in the storage 270 at the physical address corresponding to the address indicated by the data information 420. The driver 320 may then write the read data to the cache block device 250 that logically caches the data, and then call the cache driver 265 by the cache block device 250 to write the cache 280 that actually caches the data.

In this manner, since the potential access data has already been prefetched into cache 280, the potential access data can be quickly read directly from cache 280 when an application actually accesses the potential access data, thereby significantly improving system performance.

Furthermore, because the first and second buffers are allocated and maintained by the driver 320 and implemented in the memory 330, access operations to the first and second buffers may each be asynchronous (i.e., non-blocking) memory operations. Specifically, the driver 320 adds and provides an interface to the manager 310 of the application layer for access to the two buffers. Manager 310 accesses the respective buffers through such an interface, thereby retrieving access information 410 associated with the access request of application 210 to storage 270 in a very short delay, and based on the access information, quickly making a prefetch decision and providing data information 420 for retrieving potential access data, eventually completing the prefetch. Because the latency is very short, prefetching may typically be completed before the potential access data has not actually been accessed by the application 210. In this manner, the channels for transferring both access information 410 and data information 420 may be implemented in non-blocking buffers, i.e., the transfer of access information 410 and data information 420 does not result in the driver 320 and manager 310 being suspended. Therefore, when the channel is abnormal, for example, when the read or write data in the channel is abnormal, the normal operation of the system is not affected.

In addition, the scheme realizes the separation of the logic by implementing the prefetch decision logic in the manager and the access information acquisition and prefetch execution logic in the driver, and avoids various potential defects caused by implementing the prefetch decision logic in the kernel. For example, complex neural network algorithms that consume large amounts of resources cannot be implemented in a resource-constrained kernel, but can be implemented in an application layer that provides more resources. As another example, complex prefetch decision logic may be more error prone due to its more powerful nature, in which case implementing it in the core would cause the core to be unstable. As yet another example, when the prefetch decision logic is implemented in the kernel, a malicious third party may intrude into the system via the prefetch decision logic, thereby affecting the security of the kernel. Therefore, compared with the prefetching decision logic implemented in the kernel, the prefetching decision logic needs to consider various aspects such as resources, safety, stability and the like, so that the prefetching decision logic can only be kept simple and stable under the condition of sacrificing accuracy and efficiency, and a manager implemented in an application layer can implement more complex, efficient and accurate prefetching decision logic, so that current and historically accumulated access information is effectively sensed, complex access characteristics of application are analyzed, and prefetching decisions are accurately made.

Further, by completely decoupling the manager 310 in the application layer from the driver 320 in the kernel, failure or malfunction of the manager 310 does not affect normal access operations at all, making the system robust significantly beyond traditional kernel native prefetching schemes. Furthermore, since application 210 is unaware of the operation of manager 310, dynamic upgrades of manager 310 may be implemented without requiring updates to application 210, resulting in a wider range of adaptability.

A storage management environment with a manager 310 according to some embodiments of the present disclosure is described in detail above in conjunction with fig. 3, and a storage management process with prefetching according to some embodiments of the present disclosure is described in detail above in conjunction with fig. 4. Hereinafter, a conventional caching process and a caching process with prefetching will be described with reference to fig. 5 to 6 to clearly present the differences and advantages of the present scheme from the conventional scheme.

FIG. 5 shows a swim lane diagram of an example conventional caching process 500. As shown in FIG. 5, application 210 reads 510 data A. After the 5ms delay, storage 270 returns 515 data A, and data A is cached in cache 280. In addition, application 210 reads 520 data B. After the 5ms delay, storage 270 returns 525 data B, and data B is cached in cache 280. In addition, application 210 reads 530 data C. After the 5ms delay, storage 270 returns 535 data C, and data C is cached in cache 280. Further, the application 210 reads 540 data D. After the 5ms delay, storage 270 returns 545 data D, and the data D is cached in cache 280.

During subsequent access reads, since data A-D is already cached in cache 280, when application 210 reads data A-D, the data will be read directly from high speed cache 280 without having to read the data from low speed storage 270. For example, application 210 reads 550 data A. After a delay of less than 100 mus, cache 280 returns 555 DATA A. In addition, application 210 reads 560 data B. After a delay of less than 100 mus, cache 280 returns 565 DATA B.

It can be seen that this conventional scheme relies on the application 210 repeatedly reading the same data to improve system performance. Therefore, only a second and subsequent accesses to the same data can improve performance. However, in many cases, the application 210 only accesses the same data once. Alternatively, in the case where the amount of data is so large that the cache 280 cannot accommodate the data, even if there are multiple accesses to the same data, the drive 230 still has to read the data from the low speed storage device 270 because the cache 280 is not sufficient to accommodate the data, and system performance cannot be improved.

FIG. 6 illustrates a swim lane diagram of an example caching process 600 with prefetching, according to some embodiments of the present disclosure.

As shown in FIG. 6, application 210 reads 610 data A. After the 5ms delay, storage 270 returns 615 data A, and data A is cached in cache 280. In addition, application 210 reads 620 data B. After the 5ms delay, storage 270 returns 630 data B, and data B is cached in cache 280.

The manager 310 may determine a prefetch decision to prefetch data C and D based on the data access rules of the application (e.g., sequential in this case). Thus, when subsequent accesses to data C and D are applied, these data can be read directly in cache 280, thereby significantly reducing the latency from 5ms to 100 μ s. Specifically, the hypervisor 310 determines to prefetch 625 data C and D, and the storage 270 returns 635 data C and returns 645 data D. In this case, application 210 reads 640 data C, and cache 280 returns 650 data C only after a delay of less than 100 μ s, since data C has already been prefetched into cache 280. In addition, the application 210 reads 655 data D, and since data D has already been prefetched into the cache 280, the cache 280 returns 660 data D only after a delay of less than 100 μ s.

Further, during subsequent access reads, since data A-D is already cached in cache 280, when application 210 reads data A-D, it will read the data directly from high speed cache 280 without reading the data from low speed storage 270. For example, application 210 reads 665 data a. After a delay of less than 100 mus, cache 280 returns 670 data A. In addition, application 210 reads 675 data B. After a delay of less than 100 mus, cache 280 returns 680 data B.

In this way, the intelligent prefetching scheme of the present disclosure may also make efficient use of the cache for application scenarios with large amounts of single-access data, thereby significantly improving system performance, relative to conventional caching processes.

The storage management process performed at the manager and the storage management process performed at the drive will be described in detail below with reference to fig. 7 and 8, respectively.

FIG. 7 illustrates a flow diagram of a storage management process 700 performed at a manager in accordance with some embodiments of the present disclosure. For example, method 700 may be performed by manager 310 as shown in FIG. 3. It should be understood that method 700 may also include additional steps not shown and/or may omit steps shown, as the scope of the present disclosure is not limited in this respect.

At 710, a manager in the application layer obtains access information relating to an access request received by the operating system from the application layer. At 720, the manager determines data information for potential access data of the application layer based on the access information. At 730, the manager makes the data information available to a driver in the operating system to cause the driver in the operating system to load potential access data into the cache based on the data information. For example, the manager may store the data information in a memory accessible to a drive in the operating system or in a predetermined storage area, so that the data information is accessible to the drive, i.e., in a usable state. In some implementations, the manager may notify the drive to retrieve the data information from the memory or from the predetermined storage area. In other implementations, the manager may not send a notification to the drive, but the drive may autonomously read data information from the memory or from the predetermined storage area when needed.

In this way, the manager may predict data information of potential access data that may be to be accessed based on the access information, and the driver may prefetch the potential access data based on the data information, thereby improving storage performance, access efficiency, and user experience.

In some embodiments, determining the data information comprises: the access information is applied to a trained predictor to obtain data information, the predictor being trained based on training access information and training data information associated with the training access information. In this way, the trained predictor may be utilized to accurately and efficiently determine data information.

In some embodiments, determining the data information comprises: the data information is determined based on cache eviction information and access information retrieved from memory. In this way, the manager may accurately predict data information for potential access data in conjunction with cache eviction information and access information.

FIG. 8 illustrates a flow diagram of a storage management process 800 performed at a drive 320 according to some embodiments of the present disclosure. For example, the method 800 may be performed by the driver 320 as shown in FIG. 3. It should be understood that method 800 may also include additional steps not shown and/or may omit steps shown, as the scope of the present disclosure is not limited in this respect.

At 810, a driver in the operating system determines access information related to the access request in response to the operating system receiving the access request from the application layer. At 820, the driver obtains data information for potential access data of the application layer, the data information being determined by a manager in the application layer based on the access information. At 830, the drive loads the potential access data into the cache based on the data information. In this way, the drive can prefetch potential access data that is likely to be accessed based on the data information determined by the manager, thereby improving storage performance, access efficiency, and user experience.

In some embodiments, the data information is determined by the manager based on cache eviction information and access information retrieved from memory. In this way, the manager may accurately predict data information for potential access data in conjunction with cache eviction information and access information.

In some embodiments, the cache eviction information comprises at least one of: information associated with used portions in the cache when the cache is reclaimed, information associated with unused portions in the cache when the cache is reclaimed, and information associated with overwritten data in the cache when the cache is reclaimed. In this way, the manager may accurately predict data information for potentially accessed data taking into account various information related to eviction of the cache.

Fig. 9 illustrates a block diagram of an example device 900 suitable for implementing embodiments of the present disclosure. For example, the manager 310 and the driver 320 as shown in FIG. 1 may be implemented by the device 900. As shown in fig. 9, the device 900 includes a Central Processing Unit (CPU)901 that can perform various appropriate actions and processes according to computer program instructions stored in a Read Only Memory (ROM)902 or loaded from a storage unit 905 into a Random Access Memory (RAM) 903. In the RAM903, various programs and data required for the operation of the device 900 can also be stored. The CPU 901, ROM 902, and RAM903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The various processes and processes described above, such as

methods

700 or 800, may be performed by processing unit 901. For example, in some embodiments, the

methods

700 or 800 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 905. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 900 via ROM 902 and/or communications unit 909. When loaded into RAM903 and executed by CPU 901, may perform one or more of the acts of

methods

700 or 800 described above.

The present disclosure may be methods, apparatus, systems, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for carrying out various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + +, Java, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments.

Claims

1. A storage management method, the method comprising:

a manager in an application layer acquires access information, wherein the access information is related to an access request received by an operating system from the application layer;

the manager determines data information of potential access data of the application layer based on the access information; and

the manager makes the data information available to a driver in the operating system to cause the driver to load the potential access data into a cache based on the data information.

2. The method of claim 1, wherein obtaining the access information comprises:

in response to receiving a notification from the drive regarding the access information, retrieve the access information from a first buffer in memory, the first buffer maintained by the drive for storing the access information.

3. The method of claim 1 or 2, wherein determining the data information comprises:

applying the access information to a trained predictor to obtain the data information, the predictor trained based on training access information and training data information associated with the training access information.

4. The method of claim 1 or 2, wherein determining the data information comprises:

the data information is determined based on cache eviction information retrieved from memory and the access information.

5. The method of claim 4, wherein the cache eviction information comprises at least one of:

information associated with used portions in the cache when the cache is reclaimed,

information associated with an unused portion of the cache when the cache is reclaimed, and

information associated with data in the cache that was overwritten when the cache was evicted.

6. The method according to claim 1 or 2, characterized in that the method further comprises:

the manager, after determining the data information, sends a notification regarding the data information to the drive for the drive to retrieve the data information from a second buffer in memory based on the notification, the second buffer maintained by the drive for storing the data information.

7. The method according to claim 1 or 2, wherein the access information comprises at least one of:

the time at which the access request was initiated,

the address of the data to be accessed by the access request,

the size of the data to be accessed by the access request,

the number of hit access requests, the data to be accessed by the hit access requests having been loaded into the cache, and

the proportion of access requests that hit.

8. The method according to claim 1 or 2, characterized in that the data information comprises at least one of the following:

the address of the potential access data, and

a size of the potential access data.

9. The method of claim 1 or 2, wherein the potential access data is stored in a storage device, and determining the data information comprises:

determining the data information in response to determining that the number of access requests of the application layer is below a maximum throughput of the storage device.

10. A storage management method, the method comprising:

a driver in an operating system determines access information related to an access request in response to the operating system receiving the access request from an application layer;

the driver acquires data information of potential access data of the application layer, wherein the data information is determined by a manager in the application layer based on the access information; and

the driver loads the potential access data into a cache based on the data information.

11. The method of claim 10, further comprising:

the driver, after determining the access information, sends a notification regarding the access information to the manager for the manager to retrieve the access information from a first buffer in memory based on the notification, the first buffer maintained by the driver for storing the access information.

12. The method of claim 10 or 11, wherein obtaining the data information comprises:

in response to receiving a notification from the manager about the data information, retrieving the data information from a second buffer in memory, the second buffer maintained by the driver for storing the data information.

13. The method of claim 10 or 11, wherein the data information is determined by the manager based on cache eviction information retrieved from memory and the access information.

14. The method of claim 13, wherein the cache eviction information comprises at least one of:

15. The method according to claim 10 or 11, wherein the access information comprises at least one of:

the time at which the access request was initiated,

the address of the data to be accessed by the access request,

the size of the data to be accessed by the access request,

the number of access requests that hit, the data to be accessed by the access requests that hit has been loaded into the cache, and

the proportion of access requests that hit.

16. The method according to claim 10 or 11, wherein the data information comprises at least one of:

the address of the potential access data, and

a size of the potential access data.

17. The method of claim 10 or 11, wherein the potential access data is stored in a storage device and the data information is determined by the manager in response to determining that a number of access requests of the application layer is below a maximum throughput of the storage device.

18. An electronic device, comprising:

at least one processing unit;

at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit, cause the apparatus to perform operations of the method of any of claims 1 to 9.

19. An electronic device, comprising:

at least one processing unit;

at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit, cause the apparatus to perform operations of the method of any of claims 10 to 17.

20. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the operations of the method according to any one of claims 1 to 9.

21. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs operations of the method according to any one of claims 10 to 17.

22. A computer program product, characterized in that the computer program product comprises instructions which, when executed by an electronic device, cause the electronic device to perform the method according to any of claims 1 to 9.

23. A computer program product comprising instructions which, when executed by an electronic device, cause the electronic device to perform the method according to any one of claims 10 to 17.