WO2022152086A1

WO2022152086A1 - Data caching method and apparatus, and device and computer-readable storage medium

Info

Publication number: WO2022152086A1
Application number: PCT/CN2022/071079
Authority: WO
Inventors: 郭畅
Original assignee: 华为云计算技术有限公司
Priority date: 2021-01-15
Filing date: 2022-01-10
Publication date: 2022-07-21
Also published as: CN114764416A

Abstract

A data caching method and apparatus, and a device and a computer-readable storage medium. The method comprises: acquiring a plurality of caching policies from a plurality of types of caching policy libraries (S101), wherein at least one caching policy comprised in each type of caching policy libraries is of the same type, the types of the caching policy libraries comprise at least one of a filtering type, a pre-fetching type, a replacement type and a victim cache type, a caching policy of the filtering type is used for filtering data, a caching policy of the pre-fetching type is used for pre-fetching data, a caching policy of the replacement type is used for eliminating data from a cache, and a caching policy of the victim cache type is used for processing the data eliminated from the cache; and applying a first caching policy group, which comprises the plurality of caching policies, to data generated by a first entity (S102). By using the above-mentioned method, a caching policy applied to data generated by a first entity can be flexibly selected, so as to improve the caching effect of a first caching policy group on the data generated by the first entity.

Description

Data caching method, apparatus, device, and computer-readable storage medium

This application claims the priority of the Chinese patent application filed on January 15, 2021 with the State Intellectual Property Office of China, the application number is 202110057914.1, and the invention name is "Data caching method, device, device and computer-readable storage medium", all of which are The contents are incorporated herein by reference.

technical field

The present application relates to the field of caching technologies, and in particular, to a data caching method, apparatus, device, and computer-readable storage medium.

Background technique

With the rapid development of caching technology, the speed at which users obtain data is getting faster and faster. Among them, a key factor that affects the speed at which users acquire data is the caching strategy. An appropriate caching strategy can ensure the effectiveness of the cache, thereby improving the speed at which users acquire data. However, in the actual application process, due to different business logic, the data access patterns of different users may also be different, so the caching strategies suitable for processing the data generated by these users will also be different. Therefore, in order to adapt to the data access patterns of different users, it is an urgent problem to be solved in the current field of caching technology to construct a flexible and adaptable caching strategy.

SUMMARY OF THE INVENTION

The present application discloses a data caching method, device, device and computer-readable storage medium, which can construct a caching strategy group with high flexibility and strong adaptability, and when using the caching strategy group for data caching, the cache performance can be improved. effectiveness, thereby increasing the speed of data reading.

In a first aspect, the present application provides a data caching method, which includes the following steps:

Obtain multiple cache policies from multiple types of cache policy libraries, each type of cache policy library includes at least one cache policy of the same type, wherein the types of the cache policy library include filter type, prefetch type, replacement type, At least one of the sacrificing cache types, the filtering type caching strategy is used to filter data, the prefetch type caching strategy is used to prefetch data, the replacement type caching strategy is used to evict data from the cache, and the sacrificing cache type caching strategy is used for processing data that has been eliminated from the cache;

A first set of caching policies comprising a plurality of caching policies is applied to the data generated by the first entity.

By implementing the method described in the first aspect, the user or device can choose the caching strategy applied to the data generated by the first entity. A caching strategy group has more flexibility, and can also improve the caching effect of the first caching strategy group on the data generated by the first entity.

In a possible implementation manner, applying the first cache policy group including multiple cache policies to the data generated by the first entity includes: optimizing the first cache policy group according to the access records of the data generated by the first entity , obtain the second cache strategy group; apply the second cache strategy group to the data generated by the first entity.

In the above implementation manner, since the second cache policy group is obtained by optimizing the first cache policy group according to the access records of the data generated by the first entity, the second cache policy group is more suitable than the first cache policy group. Processing the data generated by the first entity, that is, using the second cache policy group to cache the data generated by the first entity can achieve a better cache effect, thereby improving the speed at which the first entity acquires data.

In a possible implementation manner, the type of the cache policy library further includes an exclusive type, and the cache policy of the exclusive type is a cache policy set by a user.

In the above implementation manner, the user can set the required caching policy by himself, so the first caching policy group can have higher flexibility.

In a possible implementation manner, the above method further includes: optimizing the first cache policy group according to the access record of the data generated by the second entity to obtain a third cache policy group; applying the third cache policy group to the second cache policy group Data generated by the entity.

In the above implementation manner, the first cache policy group can also be used to process the data generated by the second entity, wherein the data generated by the first entity is different from the data generated by the second entity, therefore, the first cache policy group can be applied It has good adaptability to data generated by different entities.

In a possible implementation manner, the multiple cache policies in the first cache policy group are arranged in a preset order, and when the multiple cache policies include a proprietary type of cache policy, the exclusive type of cache policy The location is set by the user.

In the above implementation manner, the user can also set the location of the cache policy by himself. Then, the user can set the first cache policy group according to his own needs, so that the first cache policy group can have higher flexibility.

In a possible implementation manner, before applying the first cache policy group including multiple cache policies to the data generated by the first entity, the above method further includes: determining the validity of the first cache policy group.

Understandably, since the first cache policy group includes multiple cache policies, and there may be conflicts between multiple cache policies, in order to avoid this situation, before using the first cache policy group for data caching, it is necessary to A cache policy group is checked for validity. When the first caching strategy combination method is used, it can be applied to the data generated by the first entity. When the first caching strategy group is invalid, it needs to be further adjusted to a valid caching strategy group, and then applied to the data generated by the first entity. Data generated by the first entity.

In a possible implementation manner, each cache policy in the multiple cache policies corresponds to a policy attribute set, and determining the validity of the first cache policy group includes: according to the multiple policy attribute sets corresponding to the multiple cache policies, Determine the legitimacy of the first cache policy group; wherein, a policy attribute set corresponding to a cache policy includes at least one of a first attribute and a second attribute, and the first attribute is used to determine whether there is a combination of the above-mentioned one in the first cache policy group The cache policy of the cache policy conflict, and the second attribute is used to determine whether the first cache policy group includes a plurality of the one cache policy.

In the above implementation manner, by setting a policy attribute set for each cache policy, the validity of the first cache policy group can be more conveniently determined.

In a possible implementation manner, optimizing the first cache policy group according to the data access record generated by the first entity to obtain the second cache policy group includes: in the case of the first cache policy combination method, according to the first cache policy combination method. For the access records of data generated by an entity, each cache policy in the first cache policy group is iteratively optimized by using a heuristic algorithm or a machine learning algorithm, thereby obtaining a second cache policy group.

In the above implementation manner, using a heuristic algorithm or a machine learning algorithm to optimize each cache policy in the first cache policy group can improve the optimization speed, so that the second cache policy group can be obtained faster.

In a second aspect, the present application provides a data caching device, the device comprising:

The acquiring unit is configured to acquire multiple cache policies from multiple types of cache policy libraries, where each type of cache policy library includes at least one cache policy of the same type, wherein the types of the cache policy library include filtering types, prefetching At least one of Type, Replacement Type, and Sacrifice Cache Type, the filter-type cache policy is used to filter data, the prefetch-type cache policy is used to prefetch data, the replacement-type cache policy is used to evict data from the cache, and the sacrifice cache The type of caching strategy used to handle data evictions from the cache;

The cache unit is used for applying the first cache policy group including a plurality of cache policies to the data generated by the first entity.

In a possible implementation manner, the cache unit is specifically configured to: optimize the first cache policy group according to the access record of the data generated by the first entity to obtain the second cache policy group; apply the second cache policy group to the first cache policy group Data generated by an entity.

In a possible implementation manner, the cache unit is further configured to: optimize the first cache policy group according to the access record of the data generated by the second entity to obtain a third cache policy group; apply the third cache policy group to the third cache policy group Data generated by two entities.

In a possible implementation manner, the above-mentioned apparatus further includes a determination unit, where the determination unit is configured to: determine the validity of the first cache policy group.

In a possible implementation manner, each cache policy in the multiple cache policies corresponds to a policy attribute set, and the determining unit is specifically configured to: determine the first cache policy group according to the multiple policy attribute sets corresponding to the multiple cache policies The legality of the cache policy; wherein, a set of policy attributes corresponding to a cache policy includes at least one of a first attribute and a second attribute, and the first attribute is used to determine whether there is a cache policy conflicting with the above-mentioned one cache policy in the first cache policy group , and the second attribute is used to determine whether the first cache policy group includes a plurality of the above one cache policy.

In a possible implementation manner, the caching unit is specifically configured to: in the case of the first caching strategy combination method, use a heuristic algorithm or a machine learning algorithm for the first caching strategy according to the access records of the data generated by the first entity. Each cache policy in the group is iteratively optimized to obtain a second cache policy group.

In a third aspect, the present application provides a cache device. The cache device includes a processor and a memory. The processor executes code in the memory to implement some or all of the steps described in the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium storing computer instructions, where the computer instructions are used to implement some or all of the steps described in the first aspect.

In a fifth aspect, the present application provides a computer program product, including a computer program, which, when the computer program is read and executed by a computing device, implements some or all of the steps described in the first aspect.

Description of drawings

In order to explain the technical solutions involved in the present application more clearly, the following briefly introduces the accompanying drawings that are used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1A is a schematic diagram of an ARC strategy provided by the application;

1B is a schematic diagram of the principle of adaptive adjustment of an ARC strategy provided by the present application;

2A is a schematic diagram of a cache policy selection interface provided by the present application;

2B is a schematic diagram of another cache policy selection interface provided by the present application;

3 is a schematic flowchart of a data caching method provided by the present application;

4A is a schematic diagram of a cache management page provided by the present application;

4B is a schematic diagram of another cache management page provided by the present application;

5A is a schematic diagram of a first cache policy group provided by the present application;

5B is a schematic diagram of another first cache policy group provided by the present application;

6 is a schematic flowchart of a specific embodiment provided by the present application;

7 is a schematic structural diagram of a data cache device provided by the present application;

FIG. 8 is a schematic structural diagram of a cache device provided by the present application.

Detailed ways

The technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In order to facilitate the understanding of the technical solutions provided by the present application, related concepts involved in the present application are first introduced.

Data access entity (hereinafter referred to as "entity"): refers to data users who access data and have different data storage requirements (ie cache requirements), such as users (groups), applications (groups), processes (groups) ), thread (group), etc. The cache requirement in this application can be understood as the entity's requirement for the data stored in the cache, that is, when the data is stored in the cache, the cache is valid.

Cache validity: refers to whether the cache is valid. When the cache is valid, the entity can obtain more accessed data from the direct cache, thereby improving the speed at which the entity obtains data. Generally, indicators to measure cache effectiveness include: cache hit rate, data migration amount read, and magnification.

Cache hit rate: When an entity accesses data, if the data is stored in the cache, the entity can get the data from the cache, which is a hit, on the contrary, if the data is not stored in the cache, the entity needs to get it from memory That data, that's not a hit. Then, the cache hit rate = the number of hits/(the number of hits + the number of no hits), it can be seen that the higher the cache hit rate, the higher the cache usage rate, that is, most of the accessed data is from the cache. obtained from.

Data migration amount: refers to the migration amount of cached data. In order to speed up the performance of the storage system, data is usually stored in different storage methods on storage devices with different performances according to the importance, access frequency, retention time, capacity, performance and other indicators of the data, that is, through hierarchical storage. In this way, data that is not frequently accessed is automatically migrated to a lower level in the storage hierarchy, thereby releasing higher-cost storage space for frequently accessed data. Then, when the amount of data migration is larger, it means that a large amount of infrequently accessed data is stored in the cache, which will increase the cost of data migration and the burden on the storage system. It is worth noting that "data migration amount" can have different names, for example, different standards or different versions of the same standard, different manufacturers, and different application scenarios can have different names for "data migration amount", for example, cache overhead. Wait.

Read magnification: For a piece of data, if the cache policy determines that the data meets the cache requirements, the data will be read from the memory and copied to the cache. At this time, the data is equivalent to being read twice. Next, when the cache is full, if the cache policy determines that the data does not meet the cache, the data will be eliminated from the cache. When the data is accessed again, the cache policy will consider the data to meet the cache requirements, and then will The data is copied from memory to cache, at which point the data is read 2 more times. Then, when the read magnification of a data is larger, it means that the data is repeatedly eliminated from the cache, and the cache effect of the cache is poor. It is worth noting that "read magnification" can have different names, for example, different standards or different versions of the same standard, different manufacturers, and different application scenarios can have different names for "read magnification", for example, cache load. intake, etc.

Data Access Mode: Refers to the way an entity accesses data, such as recency mode and frequency mode. Recent access mode refers to the entity always accessing the most recently accessed data, and frequent access mode refers to the entity Always access data with high historical access frequency. Understandably, in order to improve the effectiveness of the cache, the data in the cache should meet the requirements of the data access mode. For example, if the entity adopts the most recent access mode, the cache should store the recently accessed data. If the entity adopts the frequent access mode, Then the cache should store data with high historical access frequency. In other words, the caching strategy used to process the data produced by the entity should be adapted to the data access pattern so that the caching needs can be satisfied and thus the effectiveness of the cache can be improved.

In order to facilitate the understanding of the technical solutions provided by this application, firstly, an application scenario applicable to this application: a caching scenario is introduced.

Cache refers to a structure located between two kinds of hardware with large speed differences (for example, processor and memory, memory and hard disk, hard disk and network, etc.) The effect of data read performance. It is not difficult to understand that when a user frequently accesses a piece of data, if the data is retrieved from the memory every time, the user needs to wait a long time to obtain the data each time it is accessed. The emergence of the cache effectively solves this problem. Specifically, the frequently accessed data is copied to the cache, so that when users access the data subsequently, they can directly read the data from the cache, thereby improving the speed of data reading. .

Simply put, a cache is a storage area for frequently accessed data. However, over time, frequently accessed data may change, that is, data that was frequently accessed in the previous period may not be accessed at this time, while data that was not frequently accessed in the previous period may be accessed at this time. Frequently accessed, how to determine which data is frequently accessed data? In addition, when the cache space is full, if there is new data that needs to be copied to the cache, which data should be kept in the cache and which data should be deleted? Therefore, a caching strategy needs to be designed to manage the cache to ensure the effectiveness of the cache.

It's worth noting that a "cache policy" can have a different name. For example, different standards or different versions of the same standard, different manufacturers, and different application scenarios may have different names for "cache strategy". For example, the term "cache strategy" may sometimes be called "cache method", "cache algorithm" Wait.

For an entity, an appropriate caching strategy is the key to improving the speed at which the entity obtains data. However, in practical applications, since the data access patterns corresponding to different entities may be different, the caching requirements corresponding to different entities may also be different. Then, Caching strategies that are suitable for handling data generated by different entities also vary. It is not difficult to understand that if a proper caching strategy is designed for each entity, it will consume a lot of resources. Therefore, in order to adapt to the data access patterns of different users, how to construct a flexible and adaptable caching strategy to improve the effectiveness of the cache is still an urgent problem to be solved in the current field of caching technology.

At present, common caching strategy schemes include a single adaptive strategy scheme and a hybrid caching strategy scheme, as follows:

(1) A single adaptive strategy scheme

Single adaptive strategy scheme: It refers to configuring the same caching strategy for different entities, and then improving the caching strategy according to the data access situation of each entity in the actual application process, so that it can adapt to different data access adopted by different entities. model. For example, the adaptive replacement cache (ARC) proposed by Megiddo and Modha is a typical single adaptive strategy, which combines the least recently used (LRU) algorithm. The idea of (that is, if a data has not been accessed in the recent period of time, it can be considered that the possibility of this data being accessed in the future is also very small) and the idea of the least frequently used (LFU) algorithm ( That is, if a piece of data is rarely accessed in the recent period, it can be considered that the possibility of this data being accessed in the future is also low). Therefore, the ARC strategy is suitable for processing data generated by entities with a recent access pattern, or data generated by an entity with a frequent access pattern.

As shown in FIG. 1A, FIG. 1A shows a schematic diagram of an ARC strategy. As can be seen from the figure, the ARC strategy specifically includes an LRU linked list, an LFU linked list, a linked list (Ghost LRU linked list) for storing information about data eliminated from the LRU linked list, and a linked list for storing information about data eliminated from the LFU linked list. Linked list (Ghost LFU linked list). Among them, the LRU linked list and the LFU linked list are used to store data. Specifically, the LRU linked list stores the most recently used data, and the LFU linked list stores the most recently used data; while the Ghost LRU linked list and Ghost LFU linked list does not store data , which stores information about the data (eg, offset).

In the actual application process, the ARC strategy will dynamically adjust the lengths of the LRU linked list and the LFU linked list according to the hits of the four linked lists, so that the ARC strategy has the ability to adapt. Specifically, if the hit rate of the LRU linked list and the Ghost-LRU linked list is high, the length of the LRU linked list is increased, and if the hit rate of the LFU linked list and the Ghost-LFU linked list is high, the length of the LFU linked list is increased. For example, as shown in Figure 1B, when the LRU linked list is full, if another data A needs to be written into the LRU linked list, the least recently accessed data B in the LRU linked list will be eliminated, and the data B will be eliminated. Will be put into the Ghost-LRU queue. Suppose that after a period of time, the data B is accessed again, and the Ghost-LRU queue is hit. In this case, the length of the LRU list will be increased by 1, and correspondingly, the length of the LFU list will be decreased by 1.

It can be seen that the adaptability of the ARC strategy is realized by switching between the LRU algorithm and the LFU algorithm based on the hit situation of the linked list. Therefore, the ARC strategy can only be adapted to the recent access mode and the frequent access mode, which leads to ARC The adaptability of the strategy is limited, and it is difficult to meet other cache requirements except the cache requirements corresponding to the above two data access modes. Similarly, other single adaptive strategies have limited flexibility and adaptability. In general, a single adaptive strategy usually suffers from poor flexibility and adaptability, making it difficult to meet many different caching needs.

(2) Hybrid cache strategy scheme

Hybrid caching strategy scheme: A candidate strategy set is provided. For different entities, the corresponding caching strategy can be selected from the candidate strategy set to process the data generated by the corresponding entity. At present, the hybrid caching strategy is the most commonly used caching strategy in cloud storage, content delivery network (CDN) and other fields.

In a possible implementation manner, the solution provides a cache policy selection interface, which displays a set of candidate policies to the user, so that the user can select an appropriate cache policy by himself. In the cache policy selection interface shown in FIG. 2A , a selection option of a cache policy is provided, and a user can select one or more corresponding cache policies according to the data access mode corresponding to the entity. In addition to the selection options of the cache policy, the cache policy selection interface can also provide the configuration options of the cache policy, so that the user can conveniently select the cache policy for the specified file. Taking the cache policy selection interface shown in FIG. 2B as an example, the user can Enter ".txt" in the configuration options, and select cache policy 1 for files with a file suffix of ".txt", so as to use cache policy 1 to process data in files with a file extension of ".txt".

In another possible implementation manner, the data access mode corresponding to the entity is analyzed, and then according to the data access mode of the entity, a caching policy suitable for the data access mode corresponding to the entity is selected from the candidate policy set.

It can be seen that the richer the caching strategies in the candidate strategy set in this scheme, the more suitable caching strategies can be selected from them. However, how to construct a set of candidate strategies including a large number of caching strategies is a difficult task in itself, which will lead to the inability of this solution to meet many different caching requirements.

In order to solve the above problems, the present application provides a data caching method, which can construct a caching strategy group with high flexibility and strong adaptability, thereby improving the effectiveness of caching. Hereinafter, the data caching method provided by the present application will be described in detail with reference to FIG. 3 to FIG. 6 .

First, please refer to FIG. 3, which shows a schematic flowchart of a data caching method provided by the present application. The method includes but is not limited to the following steps:

S101: The cache device acquires multiple cache policies from multiple types of cache policy libraries.

In a specific embodiment, each type of cache policy library includes at least one cache policy of the same type, wherein the type of the cache policy library includes at least one of a filter type, a prefetch type, a replacement type, and a sacrifice cache type. . Optionally, the type of the cache policy library also includes a proprietary type. The following is an introduction to each type of cache policy library and the cache policies it includes:

The filtering-type caching strategy library (hereinafter referred to as the filtering strategy library) includes one or more filtering-type caching strategies (hereinafter referred to as the filtering strategy), and the filtering strategy is used to filter data. For example, the processor is executing the program of face recognition. A large amount of data such as face images, face features, and face recognition results will be generated, and the filtering strategy can filter out data such as face images and face features. In this application, the filtering strategies included in the filtering-type caching strategy library may be classical filtering algorithms such as double filter, bloom filter, etc., or may be user-defined filtering rules, such as , the user filters files with the file name suffix ".jpg" in a self-defined manner, which is not specifically limited in this application.

The cache strategy library of the prefetch type (hereinafter referred to as the prefetch strategy library) includes one or more cache strategies of the prefetch type (hereinafter referred to as the prefetch strategy), and the prefetch strategy is used to prefetch data. The retrieval strategy is used to predict the data to be accessed by the entity, and store the predicted data in the cache in advance. Then, when the entity accesses the data, it can be obtained directly from the cache, thereby improving the access efficiency of the entity. For example, when the processor executes a loop instruction, the prefetch strategy can predict the data required for the next execution of the loop instruction according to the number of times the processor has executed the loop instruction, and calculate the data required for the next execution of the loop instruction by the processor. The data is stored in the cache in advance. The prefetching strategy included in the cache strategy library of the prefetching type in this application may be a readahead algorithm (readahead), an adaptive readahead algorithm (adaptive readahead), a smart prefetcher algorithm (smart prefetcher), etc. The defined prefetching rules are not specifically limited here.

The replacement-type caching strategy library (hereinafter referred to as the replacement strategy library) includes one or more replacement-type caching strategies (hereinafter referred to as the replacement strategy), and the replacement strategy is used to eliminate data from the cache. In this application, the replacement-type cache The replacement policies included in the policy library may be LRU, LFU, ARC, etc., or may be user-defined replacement rules, which are not specifically limited here.

The cache strategy library of sacrificial cache type (hereinafter referred to as sacrificial cache strategy library) includes one or more cache strategies of sacrificial cache type (hereinafter referred to as sacrificial cache strategy), and the sacrificial cache strategy is used to process data eliminated from the cache. Understandably, since the data eliminated from the cache may still be accessed again, in this case, the entity needs to re-obtain the eliminated data from the memory. In order to reduce the loss caused by this process, the cache strategy can be sacrificed. The eliminated data is temporarily stored in the victim cache, and then whether to eliminate the data from the victim cache is determined according to the probability of subsequent access, that is, the victim cache strategy can be eliminated from the cache but subsequently accessed. Data with high probability is stored in the sacrifice cache, so that when the entity accesses the eliminated data again, it can directly obtain the data from the sacrifice cache.

The proprietary type of caching policy library (hereinafter referred to as the proprietary policy library) includes one or more proprietary types of caching policies (hereinafter referred to as the proprietary policy), and the proprietary policy is the caching policy set by the user. For example, when an entity accesses a database, it extracts data according to the row number of the database every time. Therefore, the user can set a special purpose for extracting relevant data according to the row number of the database. There are strategies.

Optionally, multiple types of cache policy libraries may be configured in the cache device, or may be configured in other electronic devices or systems, and may also be partially configured in the cache device and partially configured in other electronic devices or systems. There is no specific limitation here.

Optionally, the types of multiple cache policies obtained from multiple types of cache policy libraries may be different, or may all be the same, or may be partially the same and partially different, which is not specifically limited here. Optionally, the multiple cache policies may include multiple identical cache policies, and the multiple cache policies may also be multiple different cache policies, which are not specifically limited here. In a specific implementation, the cache device can obtain multiple cache policies from multiple types of cache policy libraries in the following ways.

Manner 1: The cache device acquires multiple cache policies selected by the user from multiple types of cache policy libraries.

Specifically, multiple types of cache policy libraries can be displayed to users in the form of a cache management page. Taking FIG. 4A as an example, five types of cache policy libraries are displayed on the cache management page, including: filtering policy library, prefetching Strategy library, replacement strategy library, sacrifice cache strategy library, and proprietary strategy library. Among them, the filter strategy library includes 3 filtering strategies, the prefetch strategy library includes 4 prefetch strategies, and the replacement strategy library includes 5 replacement strategies. The strategy library includes 2 sacrificial caching strategies, and the proprietary strategy library includes 2 proprietary strategies. Then, the user can clearly know what types of cache policy libraries are available and which cache policies are available through the cache management page, and the user can also operate on the cache management page to select the above-mentioned multiple cache policies.

Optionally, configuration options can also be displayed on the cache management page, so that the user can define which data is processed by using the cache policy, where the processed data is cached, the cache time, and the cache priority. Taking FIG. 4B as an example, the user inputs directory A in the configuration options on the cache management page, and the cache device processes the data in the directory A by selecting multiple cache policies.

Optionally, the user may randomly select multiple cache policies from multiple types of cache policy libraries, or analyze the data access mode of the first entity (specifically, the access record of the data generated by the first entity), Thus, multiple cache policies are selected from multiple types of cache policy libraries, and multiple cache policies can also be selected from multiple types of cache policy libraries in other ways, which are not specifically limited in this application.

In a second manner, the cache device selects multiple cache policies from multiple types of cache policy libraries.

Optionally, the cache device may randomly select multiple cache policies from multiple types of cache policy libraries, or may analyze the data access mode of the first entity (specifically, the access record of the data generated by the first entity) , so that multiple cache policies can be selected from multiple types of cache policy libraries, and multiple cache policies can also be selected from multiple types of cache policy libraries according to the issued configuration file, which is not specifically limited in this application. . The configuration file includes one or more of the following: the total number of selected cache policies, which types of cache policies are selected, the number of selected cache policies of each type, and which cache policy is selected specifically.

In the third mode, the cache device can also obtain multiple cache policies by combining the first mode and the second mode, that is, a part of the cache policies are selected by the user, and the other part of the cache policies are selected by the cache device.

Through the above method, a user or a cache device can flexibly select a cache policy according to actual needs, which enables the first cache policy group to better meet the user's requirements. For example, when a sacrifice cache is not configured in the storage system, the sacrifice cache policy may not be selected. , for another example, when the data from the cache needs to be filtered first and then stored in the sacrifice cache, the user can set a proprietary strategy to achieve this purpose. Moreover, the types of cache policies in the first cache policy group, the number of each type of cache policies, etc. can be adjusted according to actual conditions, which makes the first cache policy group have multiple possibilities. It is not difficult to understand that compared with the single adaptive strategy and candidate strategy set mentioned in the foregoing content, more cache strategy groups can be easily expanded by using the above method, thereby providing more choices, that is, it can satisfy more cache requirements.

S102: The cache device applies a first cache policy group including multiple cache policies to the data generated by the first entity.

In a specific embodiment, the multiple cache policies in the first cache policy group are arranged in a preset order. When multiple cache policies include an exclusive policy, the location of the exclusive policy is set by the user.

More specifically, when the multiple caching strategies include filtering strategies, prefetching strategies, replacement strategies, sacrificing caching strategies and proprietary strategies, the preset order is: the filtering strategies are arranged before the prefetching strategies, and the prefetching strategies are arranged before the prefetching strategies. Before the replacement strategy, the replacement strategy is arranged before the sacrifice cache strategy, and the position of the exclusive strategy is set by the user, that is, the exclusive strategy can be arranged before or after any strategy. It can be understood that, by specifying the location of the dedicated policy, the user can make the first cache policy group more in line with the cache requirement, so that a better cache effect can be obtained.

In a specific embodiment, when the above-mentioned multiple cache policies include two or more cache policies of the same type, the preset order may also define the order of these cache policies of the same type. Taking the caching policy shown in FIG. 4A as an example, the preset sequence defines that filter policy 1 is arranged before filter policy 2 , and filter policy 3 is arranged before filter policy 1 . Then, when multiple cache policies include filter policy 1, filter policy 2, prefetch policy 4, replacement policy 3, sacrifice cache policy 1 and exclusive policy 2, the cache device can arrange these cache policies according to a preset order , so as to obtain the first cache policy group as shown in FIG. 5A . Optionally, the arrangement order between cache policies of the same type may not be defined in the preset order. Continuing to take the above example as an example, when the preset order does not define the arrangement between filter policy 1 and filter policy 2 In order, the cache device will obtain two first cache policy groups as shown in FIG. 5A and FIG. 5B .

In a specific embodiment, in order to facilitate the arrangement of multiple cache policies by the cache device, each cache policy in the multiple cache policies includes not only the algorithm description of the policy itself, but also the description of policy attributes. , is to use algorithm description and policy attribute description to jointly describe a cache policy. Specifically, each cache policy in the above-mentioned multiple cache policies corresponds to a policy attribute set, the policy attribute set includes the type of the cache policy, and the type of the cache policy may specifically be the filter type, prefetch type, replacement type mentioned in step S101 Any of type, sacrifice cache type, proprietary type. Then, the cache device may arrange the multiple cache policies in a preset order according to the type of each cache policy in the multiple cache policies, so as to obtain the first cache policy group. It should be noted here that one cache policy in this application corresponds to only one policy type, so that the cache device can arrange multiple cache policies according to the type of the cache policy.

Optionally, the first cache policy group may also be obtained in the following way: after the cache device acquires the above-mentioned multiple cache policies, it stores these cache policies in a preset file, and then names the file as the first cache policy Group.

In this application, considering that there may be incompatibility between different caching strategies, for example, filtering strategy 1 is used to filter data A, and prefetching strategy 4 is used to put data A into the cache, then, filtering strategy 1 and Prefetching strategy 4 is incompatible, which means that filtering strategy 1 and prefetching strategy 4 conflict with each other. If filtering strategy 1 and prefetching strategy 4 are used at the same time, filtering strategy 1 or prefetching strategy 4 will fail, resulting in the first Part of the function of a cache policy group is invalid. In order to avoid this situation, in a specific embodiment, before the cache device applies the first cache policy group including multiple cache policies to the data generated by the first entity, the cache device also needs to determine the legality of the first cache policy group , that is, checking for incompatibilities between multiple caching strategies.

In a specific embodiment, determining the validity of the first caching policy group by the caching device includes: determining the validity of the first caching policy group by the caching device according to multiple policy attribute sets corresponding to multiple caching policies; wherein, The policy attribute set of each cache policy further includes at least one of a first attribute and a second attribute. The first attribute is used to determine whether there is a cache policy conflicting with the cache policy in the first cache policy group, and the second attribute is used to determine whether there is a cache policy conflicting with the cache policy in the first cache policy group. It is determined whether a plurality of the cache policies can be included in the first cache policy group.

It should be noted that due to the incompatibility between different replacement strategies, if incompatible replacement strategies are combined, one of the replacement strategies will become invalid. For example, LRU is used to eliminate recently accessed strategies. The least data, LFU is used to eliminate the least frequently accessed data recently. If the first cache policy group includes both LRU and LFU, it is assumed that a certain data in the cache is the most accessed data in the last 20 minutes, but this data is also The least frequently accessed data in the last 2 hours, in this case, the data should not be eliminated according to the LRU, but the data should be eliminated according to the LFU. Therefore, the first attribute of the replacement policy may be set to be incompatible with other replacement policies, and the second attribute may be set to one, indicating that the first cache policy group can only include one replacement policy. The sacrifice cache strategy mainly acts on the data eliminated by the replacement strategy. Therefore, when the replacement strategy is one, the sacrifice cache strategy generally only needs one. Therefore, the first attribute of the sacrifice cache strategy can be set to be incompatible with other sacrifice cache strategies, and the second attribute can be set to one. It should also be noted that the first cache policy group may include multiple identical cache policies. For example, it is assumed that the entity first prefetches data according to the row number of the database, then prefetches the data according to the column number of the database, and then prefetches the data according to the row number of the database. To prefetch data, then, the above process can be implemented by executing prefetch strategy 1, prefetch strategy 2 and prefetch strategy 1 in sequence. In this case, the first cache strategy group needs two prefetch strategies 1, prefetch The second attribute of policy 1 can be set to multiple.

In a specific embodiment, when the multiple cache policies in the first cache policy group are arranged in a preset order, the policy attribute set corresponding to each cache policy may specifically include preconditions, postconditions, At least one of the first attribute and the second attribute. Wherein, the precondition of a cache policy refers to the condition that the cache policy arranged before the cache policy should satisfy, and/or the condition that the cache policy arranged before the cache policy should not satisfy, so that the cache device Whether the cache policy in the first cache policy group conflicts with the previous cache policy can be determined according to the precondition. Post-conditions refer to conditions that should be satisfied by the caching policy arranged after the caching policy, and/or conditions that should not be satisfied by the caching policies arranged behind the caching policy, so that the caching device can judge based on the post-conditions Whether the cache policy in the first cache policy group conflicts with the cache policy behind it. Taking the first cache policy group shown in FIG. 5A as an example, it is assumed that the post-condition of filtering policy 1 is to filter data A, or the pre-condition of prefetching policy 4 is to put data A into the cache. In this case, The cache device determines that the first cache policy group is invalid.

Optionally, the cache device may also select multiple cache policies from multiple types of cache policy libraries according to at least one of a precondition, a postcondition, a first attribute or a second attribute. Taking the first attribute of the replacement policy in the above example as an example, the cache device will only select one replacement policy according to the first attribute of the replacement policy. For another example, if the precondition of prefetching policy 4 is that prefetching policy 4 and filtering policy 2 should be used at the same time, then, after the caching device selects prefetching policy 4, filtering policy 2 will be selected.

In a specific embodiment, the caching device applies a first caching policy group including multiple caching policies to the data generated by the first entity, including: The group is optimized to obtain a second cache policy group; then, the second cache policy group is applied to the data generated by the first entity. It can be understood that, through the above steps, a second cache policy group that is more suitable for satisfying the data access mode of the first entity can be obtained, thereby improving the effectiveness of the cache.

In a specific embodiment, the caching device optimizes the first caching policy group according to the access records of the data generated by the first entity to obtain the second caching policy group, including: in the case of the first caching policy combination method, caching The device uses a heuristic algorithm or a machine learning algorithm to iteratively optimize each cache policy in the first cache policy group according to the data access record generated by the first entity, thereby obtaining the second cache policy group. The specific content of this step will be described in detail through steps S1021-S1025 below. Understandably, using a heuristic algorithm or a machine learning algorithm to optimize each cache policy in the first cache policy group can improve the optimization speed, so that the cache device can obtain the second cache policy group faster and apply it to the second cache policy group. Data generated by an entity.

In a specific embodiment, the caching device may also apply the first caching policy group including multiple caching policies to the data generated by the second entity, wherein the data generated by the first entity is different from the data generated by the second entity. The specific process of this step includes: the cache device optimizes the first cache policy group according to the access records of the data generated by the second entity to obtain a third cache policy group; and then applies the third cache policy group to the second entity generated data. It can be understood that the specific process of applying the first cache policy group to the data generated by the second entity by the cache device is similar to the specific process of applying the first cache policy group to the data generated by the first entity. Expand the details. It can be seen that the first caching strategy group provided by the present application can be applied to data generated by different entities, that is to say, the first caching strategy group has good adaptability.

The specific process of optimizing the first cache policy group by the cache device in the foregoing step S102 to obtain the second cache policy group will be further described below with reference to steps S1021-S1025.

S1021: Collect an access record of the data generated by the first entity.

S1022: Preprocess the access records of the data generated by the first entity, so as to remove abnormal data in the data generated by the first entity.

In a specific embodiment, the preprocessing methods include filtering, cleaning, etc., and abnormal data includes incomplete data (including truncated data, censored data, missing data, etc.), data with wrong time stamps, and data with addresses exceeding the address range. Wait.

S1023: Analyze and evaluate the caching effect of the first caching policy group according to the access records of the preprocessed data.

In a specific implementation, the preprocessed data is input into the first cache strategy group, and after being processed by the first cache strategy group, the data stored in the current cache is obtained, and then, according to the access record of the preprocessed data, the determination is made. The cache index corresponding to the current cache is further determined, so as to further determine the cache effect of the first cache policy group. The cache indicator refers to an indicator that measures the effectiveness of the current cache. It can be understood that the more effective the cache is, the better the cache effect of the corresponding cache policy is. Therefore, the cache effect of the first cache policy group can be determined according to the cache index corresponding to the current cache. Optionally, the cache indicator includes at least one of cache hit rate, cache migration amount, and read magnification, wherein, for definitions of cache hit rate, cache migration amount, and read magnification, refer to the introduction of related concepts in the foregoing content.

Taking the cache hit rate as an example, according to the access records of the preprocessed data, the specific process of determining the cache index corresponding to the current cache is as follows: First, determine which data in the preprocessed data is stored in the current cache, so as to obtain the cache The number of hits and the number of cache misses, and then, the cache hit rate of the current cache is calculated according to the number of cache hits and the number of cache misses.

S1024: Determine whether the first cache policy group needs to be optimized according to the cache effect of the first cache policy group.

In a specific embodiment, when the cache index does not meet the preset index, the cache device determines that the first cache policy group needs to be optimized. When the cache index meets the preset index, the cache device may not optimize the first cache policy group. The cache index satisfying the preset index includes one or more of the following: the cache hit rate is greater than the preset hit rate, the cache migration amount is smaller than the preset migration amount, and the read magnification is smaller than the preset multiple. The preset hit rate, the preset migration amount, and the preset multiple may be set by the user, or may be dynamically adjusted by the cache device according to the actual situation, which is not specifically limited here.

It can be understood that when the cache index meets the preset index, it means that the cache is effective, that is to say, a satisfactory cache effect can be obtained when the data generated by the first entity is cached by the first cache policy group. Therefore, in order to save resources, the cache device may not need to optimize the first cache policy group. On the contrary, when the cache index does not meet the preset index, it means that the cache is not effective, and at this time, the first cache policy group needs to be optimized to improve the effectiveness of the cache.

S1025. In the case that the first cache policy group needs to be optimized, optimize the first cache policy group according to the access record of the preprocessed data, thereby obtaining the second cache policy group.

In a more specific embodiment, considering that the first caching strategy group includes multiple caching strategies, the optimization process of the first caching strategy group belongs to a multi-objective optimization problem, and since the multi-objective optimization problem will involve each sub-objective. (here, each caching strategy) is mutually restricted, that is, the optimization of one sub-goal may cause the performance of other sub-goals to degrade. Therefore, this application uses a heuristic algorithm (for example, an evolutionary algorithm) ) or a machine learning algorithm (eg, a reinforcement learning algorithm) to optimize the first caching strategy group, so that each caching strategy in the first caching strategy group is as optimized as possible.

The following describes the optimization process of the first cache strategy group by taking the genetic algorithm as an example.

Genetic algorithm (GA) is an evolutionary algorithm with high robustness and wide adaptability. By simulating the phenomena of duplication, crossover, and mutation that occur in natural selection and genetics, the population can evolve into the search space. The better the area, so that a group of individuals that are most suitable for the application scenario can be generated.

Assuming that the first cache policy group includes K cache policies, where K is a positive integer, the specific process for the cache device to use the genetic algorithm to optimize the first cache policy group is as follows: first, the first cache policy group is regarded as a group, and the first Each cache policy in the cache policy group is regarded as an individual to be optimized, and each individual to be optimized is encoded (that is, the parameters of the cache policy are encoded), and then K data of string structure are randomly generated, The data of each string structure represents an individual to be optimized, so as to obtain the initial population data. Next, take the initial population data as the search point, calculate the fitness of each individual to be optimized, and calculate the fitness of the current population greater than the expected one. It is assumed that the individuals with fitness are inherited to the next generation group, and then new individuals are generated through the crossover operation and mutation algorithm to obtain a new generation group, and then the cache index of the cache strategy group corresponding to the new generation group is determined. When the cache index of the strategy group is greater than the preset threshold, the above optimization process is performed iteratively, and the iteration is stopped until the cache index of the optimized cache strategy group is less than or equal to the preset threshold, thereby obtaining the second cache strategy group.

Below, a specific example is used to further illustrate the first caching strategy group provided by the present application and the first data caching method, which specifically includes the following steps, as shown in FIG. 6 :

Step 1: Evaluate the caching effect of the caching strategy currently applied to the data generated by the entity.

Specifically, the cache index corresponding to the current cache is monitored. If the cache index corresponding to the current cache is greater than the preset threshold, it means that the current cache strategy has a better cache effect, and the current cache strategy can still be used. If the cache index corresponding to the current cache is less than or equal to the preset threshold, it means that a large amount of data that does not meet the cache requirements is stored in the current cache, which means that the cache effect of the current cache strategy is poor, and then steps ②- Step ⑥ to build a caching strategy that can meet the needs of entity caching.

Step ②: Collect the access log of the data generated by the entity.

Step 3: Build a corresponding cache policy group for the entity.

Step 4: The validity of the cache strategy group constructed in step ③ is checked, and in the case of the combination method of the cache strategy, the cache effect is evaluated.

Step ⑤: When the cache effect of the cache policy group does not reach the preset effect (that is, the cache index does not meet the preset index), the cache policy group is iteratively optimized according to the access log of the data generated by the entity, until a cache effect is obtained. New set of caching strategies for preset effects.

Step ⑥: Apply the new cache strategy group obtained in step ⑤ to the storage system to process the data generated by the entity, so as to achieve a better cache effect and further improve the speed at which the entity acquires data.

Optionally, step 1 may also be omitted, and steps 2 to 6 are performed periodically to optimize the caching policy corresponding to the entity, so that the caching policy can better adapt to the caching requirements of the entity.

It should be understood that, for the sake of simplicity, the above examples do not describe the specific implementation manner of each step in detail. For details, refer to the above method embodiments, which will not be repeated here.

The foregoing content describes the method of the present application in detail. In order to better implement the method provided by the present application, the relevant apparatus and equipment provided by the present application for implementing the above-mentioned method will be introduced next.

As shown in FIG. 7 , FIG. 7 shows a schematic structural diagram of a data caching apparatus provided by the present application. The apparatus 100 includes an obtaining unit 110 and a caching unit 120 .

The acquiring unit 110 is configured to acquire multiple cache policies from multiple types of cache policy libraries, where each type of cache policy library includes at least one cache policy of the same type, wherein the types of the cache policy library include filtering type, pre- At least one of the fetch type, the replacement type, and the sacrifice cache type, the filter type cache policy is used to filter data, the prefetch type cache policy is used to prefetch data, the replacement type cache policy is used to eliminate data from the cache, sacrifice Cache-type caching strategies are used to handle data that is evicted from the cache.

The cache unit 120 is configured to apply the first cache policy group including a plurality of cache policies to the data generated by the first entity.

In a specific embodiment, the cache unit 120 is specifically configured to: optimize the first cache policy group according to the access record of the data generated by the first entity to obtain the second cache policy group; apply the second cache policy group to the first cache policy group. Data generated by an entity.

In a specific embodiment, the type of the cache policy library further includes a proprietary type, and the cache policy of the proprietary type is a cache policy set by a user.

In a specific embodiment, the cache unit 120 is further configured to: optimize the first cache policy group according to the access records of the data generated by the second entity to obtain a third cache policy group; apply the third cache policy group to the third cache policy group. Data generated by two entities.

In a specific embodiment, the multiple cache policies in the first cache policy group are arranged in a preset order, and when the multiple cache policies include a proprietary type of cache policy, the location of the proprietary type of cache policy Set by the user.

In a specific embodiment, the data caching apparatus 100 further includes a determining unit 130, and the determining unit 130 is configured to: determine the validity of the first caching policy group.

In a specific embodiment, each cache policy in the multiple cache policies corresponds to a policy attribute set, and the determining unit 130 is specifically configured to: determine the first cache policy group according to the multiple policy attribute sets corresponding to the multiple cache policies The legality of the cache policy; wherein, a set of policy attributes corresponding to a cache policy includes at least one of a first attribute and a second attribute, and the first attribute is used to determine whether there is a cache policy conflicting with the above-mentioned one cache policy in the first cache policy group , and the second attribute is used to determine whether the first cache policy group includes a plurality of the above one cache policy.

In a specific embodiment, the caching unit 120 is specifically configured to: in the case of the first caching strategy combination method, use a heuristic algorithm or a machine learning algorithm for the first caching strategy according to the access records of the data generated by the first entity. Each cache policy in the group is iteratively optimized to obtain a second cache policy group.

The data caching apparatus 100 in this embodiment of the present application only uses the division of the above-mentioned functional modules as an example. In practical applications, the above-mentioned functions may be allocated by different functional modules as required, that is, the internal structure of the data caching apparatus 100 is divided into Different functional modules to complete all or part of the functions described above. In addition, the data caching apparatus 100 provided in the above embodiment belongs to the same concept as the caching device in the above method embodiment, and the specific implementation process is detailed in the above method embodiment. For example, the obtaining unit 110 obtains from multiple types of cache policy libraries For the process of multiple caching policies, please refer to the above step S101, and for the process of optimizing the first cache policy group by the caching unit 120, please refer to the above steps S1021-S1025, which will not be repeated here.

As shown in FIG. 8 , FIG. 8 shows a schematic structural diagram of a cache device provided by the present application. The cache device 200 includes a processor 210 , a communication interface 220 and a memory 230 . The processor 210 , the communication interface 220 and the memory 230 are coupled through the bus 240 .

The processor 210 may be a central processing unit (CPU), a general-purpose processor, a DSP, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other available processor. Programmable logic device (PLD), CPLD, transistor logic device, hardware component, or any combination thereof. The processor 210 may implement or execute various exemplary methods described in conjunction with the above method embodiments. Specifically, the processor 210 reads the program code stored in the memory 230, and cooperates with the communication interface 220 to execute part or all of steps S101-S102, S1021-S1025 and steps ①-⑥.

The communication interface 220 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a controller area network interface, a local interconnect network (LIN), and a FlexRay interface. The interface may be a cellular network interface or use a wireless local area network interface or the like. Specifically, the communication interface 220 can be connected to other devices. For example, the communication interface 220 can be connected to a storage system. After the processor 210 obtains the first cache policy group, the first cache policy can be sent to the storage system through the communication interface 220. for processing the data generated by the first entity.

The memory 230 may include volatile memory, such as random access memory (RAM); the memory 230 may also include non-volatile memory, such as read only memory (ROM), flash memory, hard disk (hard disk drive, HDD) or solid state drive (solid state drive, SSD), the memory 230 may also include a combination of the above-mentioned types of memory. The memory 230 may store program codes and program data. The program code is composed of codes of some or all of the units in the data caching apparatus 100 shown in FIG. The program data is the data generated by the data caching apparatus 100 shown in FIG. 7 in the process of running the program, for example, the data generated by the first entity, the cache policy, and the like.

Bus 240 may be a controller area network (CAN) or other implementation internal bus. The bus 240 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 8, but it does not mean that there is only one bus or one type of bus.

The cache device 200 in the embodiment of the present application is configured to execute the method executed by the cache device in the above method embodiments, which belongs to the same concept as the above method embodiments. For details of the specific implementation process, please refer to the above method embodiments, which will not be repeated here.

The present application also provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, when the computer instructions are stored in a computing device (for example, the data caching apparatus 100 shown in FIG. 7 or the caching apparatus 200 shown in FIG. ), causing the computing device to execute the method executed by the cache device in the foregoing method embodiments.

The present application also provides a computer program product, including a computer program, when the computer program is read and executed by a computing device (for example, the data caching device 100 shown in FIG. 7 or the caching device 200 shown in FIG. 8 ), using The method is used to implement the method executed by the cache device in the above method embodiment.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product described above includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The aforementioned computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The above-mentioned computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the above-mentioned computer instructions may be transmitted from a website site, computer, server or data center via wired communication. (eg, coaxial cable, optical fiber, digital subscriber line) or wireless (eg, infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, etc. that includes one or more available media integrated. The above-mentioned usable media may be magnetic media (eg, floppy disks, memory disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, SSD), and the like. In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus may also be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored or not implemented. On the other hand, the indirect coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions in the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art, or all or part of the technical solution, and the computer software product is stored in a storage medium, Several instructions are included to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium may include, for example, various media that can store program codes, such as a U disk, a removable hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

Claims

A data caching method, characterized in that the method comprises:

Acquire multiple cache policies from multiple types of cache policy libraries, each type of cache policy library includes at least one cache policy of the same type, wherein the types of the cache policy library include filter type, prefetch type, replacement type At least one of the type and the sacrificial cache type, the filter-type cache policy is used to filter data, the prefetch-type cache policy is used to prefetch data, and the replacement-type cache policy is used to eliminate data from the cache , the cache strategy of the sacrificial cache type is used to process the data eliminated from the cache;

A first set of caching policies including the plurality of caching policies is applied to the data generated by the first entity.
The method according to claim 1, wherein the applying the first cache policy group including the multiple cache policies to the data generated by the first entity comprises:

Optimizing the first cache policy group according to the access record of the data generated by the first entity to obtain a second cache policy group;

Applying the second set of caching policies to data generated by the first entity.
The method according to claim 1 or 2, wherein the type of the cache policy library further includes a proprietary type, and the cache policy of the proprietary type is a cache policy set by a user.
The method according to any one of claims 1-3, wherein the method further comprises:

Optimizing the first cache policy group according to the access record of the data generated by the second entity to obtain a third cache policy group;

Applying the third set of caching policies to data generated by the second entity.
The method according to claim 3, wherein the multiple cache policies in the first cache policy group are arranged in a preset order, and when the multiple cache policies include the proprietary type of cache The location of the proprietary type of cache policy is set by the user.
The method according to any one of claims 1-5, characterized in that before the applying the first cache policy group including the plurality of cache policies to the data generated by the first entity, the method further comprises: :

The validity of the first cache policy group is determined.
The method according to claim 6, wherein each cache policy in the plurality of cache policies corresponds to a policy attribute set, and the determining the validity of the first cache policy group comprises:

determining the validity of the first cache policy group according to multiple policy attribute sets corresponding to the multiple cache policies;

The set of policy attributes corresponding to one cache policy includes at least one of a first attribute and a second attribute, and the first attribute is used to determine whether there is a cache conflicting with the one cache policy in the first cache policy group policy, and the second attribute is used to determine whether the first cache policy group includes a plurality of the one cache policy.
The method according to claim 6 or 7, wherein the optimizing the first cache policy group according to the access record of the data generated by the first entity to obtain the second cache policy group, comprising:

In the case of the first caching strategy combination method, according to the access records of the data generated by the first entity, each caching strategy in the first caching strategy group is iterated by using a heuristic algorithm or a machine learning algorithm optimization, so as to obtain the second cache policy group.
A data cache device, characterized in that the device comprises:

The acquiring unit is configured to acquire multiple cache policies from multiple types of cache policy libraries, where each type of cache policy library includes at least one cache policy of the same type, wherein the types of the cache policy library include filter types, At least one of a prefetch type, a replacement type, and a sacrifice cache type, the filtering type caching policy is used for filtering data, the prefetching type caching policy is used for prefetching data, and the replacement type caching policy is used for Eliminate data from the cache, and the cache strategy of the sacrificial cache type is used to process the data eliminated from the cache;

The cache unit is configured to apply the first cache policy group including the plurality of cache policies to the data generated by the first entity.
The device according to claim 9, wherein the cache unit is specifically used for:

Optimizing the first cache policy group according to the access record of the data generated by the first entity to obtain a second cache policy group;

Applying the second set of caching policies to data generated by the first entity.
The apparatus according to claim 9 or 10, wherein the type of the cache policy library further includes a proprietary type, and the cache policy of the proprietary type is a cache policy set by a user.
The device according to any one of claims 9-11, wherein the cache unit is further configured to:

Optimizing the first cache policy group according to the access record of the data generated by the second entity to obtain a third cache policy group;

Applying the third set of caching policies to data generated by the second entity.
The apparatus according to claim 11, wherein the multiple cache policies in the first cache policy group are arranged in a preset order, and when the multiple cache policies include the proprietary type of cache The location of the proprietary type of cache policy is set by the user.
The apparatus according to any one of claims 9-13, characterized in that, the apparatus further comprises a determination unit, wherein the determination unit is configured to: determine the validity of the first cache policy group.
The device according to claim 14, wherein each cache policy in the plurality of cache policies corresponds to a policy attribute set, and the determining unit is specifically configured to:

determining the validity of the first cache policy group according to multiple policy attribute sets corresponding to the multiple cache policies;

The set of policy attributes corresponding to one cache policy includes at least one of a first attribute and a second attribute, and the first attribute is used to determine whether there is a cache conflicting with the one cache policy in the first cache policy group policy, and the second attribute is used to determine whether the first cache policy group includes a plurality of the one cache policy.
The device according to claim 14 or 15, wherein the cache unit is specifically used for:

In the case of the first caching strategy combination method, according to the access records of the data generated by the first entity, each caching strategy in the first caching strategy group is iterated by using a heuristic algorithm or a machine learning algorithm optimization, so as to obtain the second cache policy group.
A cache device, characterized in that the cache device includes a processor and a memory, and the processor executes codes in the memory to implement the method of any one of claims 1 to 8.
A computer-readable storage medium, characterized in that it stores computer instructions, and the computer instructions are used to implement the method of any one of claims 1 to 8.