CN114817173A - Data processing method, device, equipment and medium for multi-level cache - Google Patents
Data processing method, device, equipment and medium for multi-level cache Download PDFInfo
- Publication number
- CN114817173A CN114817173A CN202210223775.XA CN202210223775A CN114817173A CN 114817173 A CN114817173 A CN 114817173A CN 202210223775 A CN202210223775 A CN 202210223775A CN 114817173 A CN114817173 A CN 114817173A
- Authority
- CN
- China
- Prior art keywords
- data
- cache
- target data
- level cache
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/156—Query results presentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data processing, and discloses a data processing method, a device, equipment and a medium for multi-level cache. The data processing method of the multilevel cache is applied to a network disk sharing system of the multilevel cache, and comprises the following steps: acquiring a data query request; gradually inquiring in a multi-level cache according to the inquiry request to obtain target data corresponding to the inquiry request; and outputting the target data corresponding to the query request. The invention obtains the query request of the data of the user; inquiring in a multi-level cache step by step according to the inquiry request, and returning target data when the target data required by the user is inquired; outputting target data corresponding to the user query request; thereby further increasing the speed of data download.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a medium for processing data in a multi-level cache.
Background
With the development of internet and computer technology, data size is getting larger and larger. In order to deal with large-scale data writing or reading, the caching technology has become a main means for improving the concurrency performance and response speed of the server.
At present, a storage node of a network disk is usually only in a fixed region, a user needs to frequently return to a remote data center when downloading a file in a network disk system, and the file is easily influenced by network fluctuation when being downloaded, so that the problems of low downloading speed, unsmooth card and the like are caused.
Disclosure of Invention
The invention mainly aims to provide a data processing method, a device, equipment and a medium of a multi-level cache, aiming at improving the data downloading speed through the multi-level cache.
In order to achieve the above object, the present invention provides a data processing method for a multi-level cache, which comprises the following steps:
acquiring a query request of data;
gradually inquiring in a multi-level cache according to the inquiry request to obtain target data corresponding to the inquiry request;
and outputting the target data corresponding to the query request.
Preferably, the multi-level cache includes a local cache, a distributed cache, and a database, and before the step of obtaining the data query request, the data processing method of the multi-level cache further includes:
acquiring historical records of data query in the local cache and access frequency corresponding to each historical record;
calculating the historical records and the access frequency through a prediction model to obtain a data prediction list corresponding to the historical records and the access frequency;
and according to the data prediction list, obtaining cache data from the local cache, and encrypting and storing the cache data to a multi-level cache.
Preferably, before the step of calculating the history and the access frequency by using a prediction model to obtain a data prediction list corresponding to the history and the access frequency, the data processing method for a multi-level cache further includes:
acquiring sample history records inquired by sample data and sample access frequency corresponding to each sample history record;
and inputting the sample history record and the sample access frequency into a preset prediction model, and performing iterative training on the preset prediction model to obtain the prediction model.
Preferably, the multi-level cache comprises a local cache, a distributed cache and a database, and the step of obtaining the target data corresponding to the query request by querying in the multi-level cache step by step according to the query request comprises;
inquiring in the local cache according to the inquiry request;
if the target data is inquired in the local cache, returning the target data and quitting the inquiry, otherwise, entering the distributed cache for inquiry;
if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry;
and if the target data is inquired in the database, returning the target data and quitting the inquiry.
Preferably, the multi-level cache includes a distributed cache and a database, and the step of obtaining the target data corresponding to the query request by querying in the multi-level cache stage by stage according to the query request includes:
inquiring in the distributed cache according to the inquiry request;
if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry;
and if the target data is inquired in the database, returning the target data and quitting the inquiry.
Preferably, after the step of outputting the target data corresponding to the query request, the data processing method of the multi-level cache further includes:
and when a data updating request is received, updating the target data through a preset consensus protocol, and storing the updated target data in the multi-level cache.
Preferably, after the step of updating the target data by a preset consensus protocol and storing the updated target data in the multi-level cache when the data update request is received, the data processing method of the multi-level cache further includes:
and clearing expired data in the multi-level cache by a preset replacement protocol.
In addition, to achieve the above object, the present invention further provides a data processing apparatus of a multi-level cache, including:
the acquisition module is used for acquiring a data query request;
the query module is used for querying in a multi-level cache step by step according to the query request to obtain target data corresponding to the query request;
and the output module is used for outputting the target data corresponding to the query request.
In addition, to achieve the above object, the present invention further provides an apparatus, which is a data processing apparatus of a multi-level cache, and the data processing apparatus of the multi-level cache includes: the data processing system comprises a memory, a processor and a multi-level cache data processing program which is stored on the memory and can run on the processor, wherein when the multi-level cache data processing program is executed by the processor, the steps of the multi-level cache data processing method are realized.
In addition, to achieve the above object, the present invention further provides a medium, which is a computer readable storage medium, on which a data processing program of a multi-level cache is stored, and when the data processing program of the multi-level cache is executed by a processor, the steps of the data processing method of the multi-level cache are implemented.
The invention provides a data processing method, a device, equipment and a medium for multi-level cache; the data processing method of the multilevel cache is applied to a network disk sharing system of the multilevel cache, and comprises the following steps: acquiring a query request of data; gradually inquiring in a multi-level cache according to the inquiry request to obtain target data corresponding to the inquiry request; and outputting the target data corresponding to the query request. The invention obtains the query request of the data of the user; inquiring in a multi-level cache step by step according to the inquiry request, and returning target data when the target data required by the user is inquired; outputting target data corresponding to the user query request; thereby further increasing the speed of data download.
Drawings
FIG. 1 is a schematic diagram of an apparatus architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a data processing method for multi-level caches according to a first embodiment of the present invention;
FIG. 3 is a flowchart illustrating a data processing method for multi-level caches according to a second embodiment of the present invention;
FIG. 4 is a flow chart illustrating a data processing method for multi-level caches according to a third embodiment of the present invention;
FIG. 5 is a flowchart illustrating a fourth embodiment of a data processing method for multi-level caches according to the present invention;
FIG. 6 is a flowchart illustrating a fifth embodiment of a data processing method for multi-level caches according to the present invention;
FIG. 7 is a flowchart illustrating a sixth embodiment of a data processing method for multi-level caches according to the present invention;
fig. 8 is a functional block diagram illustrating a data processing method for a multi-level cache according to a first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
The device of the embodiment of the invention can be a mobile terminal or a server device.
As shown in fig. 1, the apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration of the apparatus shown in fig. 1 is not intended to be limiting of the apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a data processing program of a multi-level cache.
The operating system is a program for managing and controlling the data processing equipment and the software resources of the multi-level cache, and supports the operation of a network communication module, a user interface module, the data processing program of the multi-level cache and other programs or software; the network communication module is used for managing and controlling the network interface 1002; the user interface module is used to manage and control the user interface 1003.
In the data processing apparatus of the multi-level cache shown in fig. 1, the data processing apparatus of the multi-level cache calls a data processing program of the multi-level cache stored in the memory 1005 by the processor 1001 and performs operations in various embodiments of a data processing method of the multi-level cache described below.
Based on the hardware structure, the embodiment of the data processing method of the multi-level cache is provided.
Referring to fig. 2, fig. 2 is a schematic flow chart of a data processing method of a multi-level cache according to a first embodiment of the present invention, where the data processing method of the multi-level cache includes:
step S10, acquiring a query request of data;
step S20, according to the query request, querying in a multi-level cache step by step to obtain target data corresponding to the query request;
and step S30, outputting the target data corresponding to the query request.
The embodiment obtains the query request of the data of the user; inquiring in a multi-level cache step by step according to the inquiry request, and returning target data when the target data required by the user is inquired; outputting target data corresponding to the user query request; thereby further increasing the speed of data download.
The respective steps will be described in detail below:
step S10, a query request for data is obtained.
In this embodiment, a data processing method of a multi-level cache is provided, where the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; the multi-level cache comprises a first cache, a second cache and a third cache, and progressive query is carried out according to the query priority sequence of the first cache, the second cache and the third cache.
The query requests of the user are received from different channels, and the query requests can be passed through an edge node of a first cache in the multi-level caches or passed through an edge node of a second cache in the multi-level caches; or may be an edge node of a third cache in the multi-level cache. The present embodiment does not limit the channel of the query request for obtaining data.
And step S20, gradually inquiring in a multi-level cache according to the inquiry request to obtain target data corresponding to the inquiry request.
In this embodiment, after receiving a query request from a user, a progressive query is performed in a multi-level cache according to the query request from the user. The multi-level cache comprises a first cache, a second cache and a third cache, and the multi-level cache performs progressive query according to the query priority order of the first cache, the second cache and the third cache.
Firstly, inquiring in a first cache according to an inquiry request of a user; if the target data is inquired in the first cache, returning the target data and quitting the inquiry; if the target data is not inquired in the first cache, entering a second-level cache for inquiry;
if the target data is inquired in the second cache, returning the target data and quitting the inquiry; if the target data is not inquired in the second cache, the target data is inquired in a third cache;
if the target data is inquired in the third cache, returning the target data and quitting the inquiry; if the target data is not queried in the third cache, a query failure is returned.
And step S30, outputting the target data corresponding to the query request.
In one embodiment, in order to enable a user to quickly obtain a response to a query request, an EDNS Client Sub net feature is enabled on a DNS nameserver, and enables the user to respond by an edge node in a multi-level cache closest to the geographical location of the Client IP address when resolving a downloaded file address.
After receiving the query request of the user, the multi-level cache performs level-by-level query according to the query request of the user. And outputting the inquired target data through the edge node closest to the geographic position of the IP address of the client.
The embodiment obtains the query request of the data of the user; inquiring in a multi-level cache step by step according to the inquiry request, and returning target data when the target data required by the user is inquired; outputting target data corresponding to the user query request; thereby further increasing the speed of data download.
Further, based on the first embodiment of the data processing method of the multi-level cache of the present invention, a second embodiment of the data processing method of the multi-level cache of the present invention is provided.
The difference between the second embodiment of the data processing method of the multi-level cache and the first embodiment of the data processing method of the multi-level cache is that in this embodiment, before the step of acquiring the data query request in step S10, referring to fig. 3, the data processing method of the multi-level cache further includes:
step A10, obtaining the history records of data query in the local cache and the access frequency corresponding to each history record;
step A20, predicting the history records and the access frequency through a prediction model to obtain a data prediction list corresponding to the history records and the access frequency;
step A30, according to the data prediction list, obtaining cache data from the local cache, and encrypting and storing the cache data to a multi-level cache.
In this embodiment, a data processing method of a multi-level cache is provided, where the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; the multi-level cache comprises a local cache, a distributed cache and a database. Obtaining the history records of data query in a local cache and the access frequency corresponding to each history record; inputting the historical records and the access frequency into a prediction model for calculation so as to obtain a data prediction list; and obtaining cache data from the local cache through the data prediction list, and encrypting and storing the cache data into a multi-level cache, thereby improving the hit rate of the target data.
The respective steps will be described in detail below:
step a10, obtaining history records of data queries in the local cache, and access frequency corresponding to each history record.
In this embodiment, the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache, where the multi-level cache includes a local cache, a distributed cache, and a database. The network disk sharing system provides a local mounting function of a Web end and a Web DAV protocol.
When a user is used on a local system through a Web DAV protocol, a client caches the same type of files which are predicted to be downloaded in the future, such as files selected by the user, downloaded files and the like, through a prediction model, and the prediction model is only applied to the local cache.
The history records of data query in the local cache and the access frequency corresponding to each history record are obtained. Wherein the history record includes but is not limited to: a user clicks, opens, lists a directory and other system operation events; the access frequency is the number of accesses, downloads, opens, etc. of each history.
Step A20, predicting the history record and the access frequency through a prediction model to obtain a data prediction list corresponding to the history record and the access frequency.
In the embodiment, the historical records and the access frequency are input into a pre-constructed prediction model, and a threshold value is calculated by counting the access frequency of the access link, the media, the document type, the file preview time, the download frequency and other data; when the threshold value exceeds a preset value set by initialization, the prediction model can match, classify and predict all files of the user and all shared link files of the user, and then a data prediction list needing to be pre-cached is generated regularly.
Step A30, according to the data prediction list, obtaining cache data from the local cache, and encrypting and storing the cache data to a multi-level cache.
In this embodiment, according to the pre-cached data prediction list, cache data is obtained from the local cache, and the link of the data is encrypted and cached in the multi-level cache. The multi-level cache includes a local cache, a distributed cache, and a database. That is, when the calculated threshold exceeds the preset value set by initialization, the cache data is stored in the local cache, the distributed cache and each edge node of the database in the multi-level cache. When a user wants to download the cached data, the user can access and download from the nearest edge node.
In this embodiment, a data processing method of a multi-level cache is provided, where the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; the multi-level cache comprises a local cache, a distributed cache and a database. Obtaining the history records of data query in a local cache and the access frequency corresponding to each history record; inputting the historical records and the access frequency into a prediction model for calculation so as to obtain a data prediction list; and obtaining cache data from the local cache through the data prediction list, and encrypting and storing the cache data into a multi-level cache, thereby improving the hit rate of the target data.
Further, based on the first and second embodiments of the data processing method of the multi-level cache of the present invention, a third embodiment of the data processing method of the multi-level cache of the present invention is provided.
The third embodiment of the data processing method of the multi-level cache is different from the first and second embodiments of the data processing method of the multi-level cache in that, in this embodiment, before the step of predicting the history and the access frequency by using a prediction model to obtain the data prediction list corresponding to the history and the access frequency in step a20, referring to fig. 4, the data processing method of the multi-level cache further includes:
step B10, obtaining sample history records of sample data query and sample access frequency corresponding to each sample history record;
and step B20, inputting the sample history record and the sample access frequency into a preset prediction model, and performing iterative training on the preset prediction model to obtain the prediction model.
In the embodiment, a sample history record of sample data query and the access frequency of each sample record are obtained; inputting the historical records and the access frequency of the samples into a preset prediction model for iterative training to obtain the prediction model; thereby improving the analysis effect of the trained prediction model.
The respective steps will be described in detail below:
and step B10, acquiring sample history records of the sample data query and the sample access frequency corresponding to each sample history record.
In this embodiment, the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache, where the multi-level cache includes a local cache, a distributed cache, and a database. And acquiring sample history records of sample data query in a local cache and a distributed cache in the multi-level cache, and the sample access frequency corresponding to each sample history record.
And step B20, inputting the sample history records and the sample access frequency into a preset prediction model, and performing iterative training on the preset prediction model to obtain the prediction model.
In this embodiment, the sample history records and the sample access frequency corresponding to each sample history record are input into a preset prediction model, iterative training is performed on the preset prediction model, and a training model with the optimal model evaluation index is selected as the prediction model.
In the embodiment, a sample history record of sample data query and the access frequency of each sample record are obtained; inputting the historical records and the access frequency of the samples into a preset prediction model for iterative training to obtain the prediction model; therefore, the analysis effect of the trained prediction model is improved.
Further, based on the first, second, and third embodiments of the data processing method of the multi-level cache of the present invention, a fourth embodiment of the data processing method of the multi-level cache of the present invention is provided.
The difference between the fourth embodiment of the data processing method of the multi-level cache and the first, second, and third embodiments of the data processing method of the multi-level cache is that in this embodiment, for step a20, a step-by-step query is performed in the multi-level cache according to the query request, so as to obtain a refinement of the target data corresponding to the query request, and referring to fig. 5, the step specifically includes:
step C10, according to the query request, querying the local cache;
step C20, if the target data is queried in the local cache, returning the target data and quitting the query, otherwise, entering the distributed cache for query;
step C30, if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry;
and step C40, if the target data is inquired in the database, returning the target data and quitting the inquiry.
In this embodiment, a data processing method of a multi-level cache is provided, where the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; when a user is used by being mounted on a local system through a Web DAV protocol, wherein the multi-level cache comprises a local cache, a distributed cache and a database, and the multi-level cache is inquired step by step according to the inquiry priority sequence of the local cache, the distributed cache and the database; therefore, the efficiency of inquiring the target data when local mounting is adopted is improved.
The respective steps will be described in detail below:
and step C10, inquiring in the local cache according to the inquiry request.
In this embodiment, the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache, where the multi-level cache includes a local cache, a distributed cache, and a database. The network disk sharing system provides a local mounting function of a Web end and a Web DAV protocol.
When the user is used on a local system by the Web DAV protocol, the step-by-step query is carried out in the multi-level cache according to the query of the user, and the step-by-step query is carried out according to the query priority sequence of the local cache, the distributed cache and the database. Firstly, according to the query request of a user, a query is carried out in a local cache in the multi-level cache.
And step C20, if the target data is inquired in the local cache, returning the target data and quitting the inquiry, otherwise, entering the distributed cache for inquiry.
In this embodiment, firstly, according to a query request of a user, a query is performed in a local cache; if the target data is inquired in the local cache, returning the target data and quitting the inquiry; and if the target data is not queried in the local cache, entering the distributed level cache for querying.
And step C30, if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry.
In this embodiment, if the target data is queried in the distributed cache, returning the target data and quitting the query; and if the target data is not queried in the distributed cache, entering the database for querying.
C40, if the target data is inquired in the database, returning the target data and quitting the inquiry.
In this embodiment, if the target data is queried in the database, returning the target data and quitting the query; and if the target data is not queried in the database, returning the information of the failure of the query.
In this embodiment, a data processing method of a multi-level cache is provided, where the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; when a user is used by being mounted on a local system through a Web DAV protocol, wherein the multi-level cache comprises a local cache, a distributed cache and a database, and the multi-level cache is inquired step by step according to the inquiry priority sequence of the local cache, the distributed cache and the database; therefore, the efficiency of inquiring the target data when local mounting is adopted is improved.
Further, based on the first, second, third, and fourth embodiments of the data processing method of the multi-level cache of the present invention, a fifth embodiment of the data processing method of the multi-level cache of the present invention is provided.
The difference between the fifth embodiment of the data processing method of the multi-level cache and the first, second, third, and fourth embodiments of the data processing method of the multi-level cache is that in this embodiment, for step S20, the target data corresponding to the query request is obtained by querying in the multi-level cache step by step, and referring to fig. 6, the step specifically includes:
step D10, according to the query request, querying the distributed cache;
step D20, if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry;
and D30, if the target data is inquired in the database, returning the target data and quitting the inquiry.
The embodiment provides a data processing method of a multi-level cache, which is applied to a network disk sharing system of the multi-level cache; when no Web DAV protocol is mounted on a local system for use, wherein the multi-level cache comprises a distributed cache and a database, and the multi-level cache is queried step by step according to the query priority sequence of the distributed cache and the database; thereby improving the efficiency of querying the target data.
The respective steps will be described in detail below:
and D10, inquiring in the distributed cache according to the inquiry request.
In this embodiment, the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; when no Web DAV protocol is used on the local system, the multi-level cache comprises a distributed cache and a database.
When no Web DAV protocol is mounted on a local system for use, the step-by-step query is carried out in the multi-level cache according to the query of a user, and the step-by-step query is carried out according to the query priority sequence of the distributed cache and the database. Firstly, according to the query request of a user, querying is carried out in distributed caches in the multi-level caches.
And D20, if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry.
In this embodiment, if the target data is queried in the distributed cache, returning the target data and quitting the query; and if the target data is not queried in the distributed cache, the target data is queried in the database.
And D30, if the target data is inquired in the database, returning the target data and quitting the inquiry.
In this embodiment, if the target data is queried in the database, returning the target data and quitting the query; and if the target data is not queried in the database, returning the information of the failure of the query.
In this embodiment, a data processing method of a multi-level cache is provided, where the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache; when no Web DAV protocol is mounted on a local system for use, wherein the multi-level cache comprises a distributed cache and a database, and the multi-level cache is queried step by step according to the query priority sequence of the distributed cache and the database; thereby improving the efficiency of querying the target data.
Further, based on the first, second, third, fourth, and fifth embodiments of the data processing method of the multi-level cache of the present invention, a sixth embodiment of the data processing method of the multi-level cache of the present invention is provided.
The sixth embodiment of the data processing method of the multi-level cache is different from the first, second, third, fourth and fifth embodiments of the data processing method of the multi-level cache in that, in the present embodiment, after the step of outputting the target data corresponding to the query request in step S30, referring to fig. 7, the data processing method of the multi-level cache further includes:
step S40, when a data updating request is received, updating the target data through a preset consensus protocol, and storing the updated target data in the multi-level cache;
and step S50, clearing the expired data in the multi-level cache by a preset replacement protocol.
When a data updating request is received, the preset consensus protocol is improved to update the target data, and the updated target data is stored in the multi-level cache; then, removing the expired data in the multi-level cache through a preset replacement protocol; thereby achieving the hit rate of the target data.
The respective steps will be described in detail below:
step S40, when receiving a data update request, updating the target data by a preset consensus protocol, and storing the updated target data in the multi-level cache.
In this embodiment, after the queried target data is returned to the user, when a data update request is received, the target data is updated through a preset consensus protocol, and the updated target data is stored in the multi-level cache. In this embodiment, the preset consensus protocol is preferably a Paxos (distributed) consensus algorithm protocol, that is, the target data is updated by the Paxos consensus algorithm protocol, and the updated target data is stored in the local cache of the multi-level cache, the distributed cache, and each edge node of the database. When a user wants to download the cached data, the user can access and download from the nearest edge node.
And step S50, clearing the expired data in the multi-level cache by a preset replacement protocol.
In this embodiment, the expired data in the multi-level cache is cleared by a preset replacement protocol by acquiring the data of the expired cache in the multi-level cache. In the present embodiment, the predetermined replacement protocol is preferably an LRU (Least recent utilized) protocol. And clearing the expired data in the multi-level cache through an LRU protocol.
When a data updating request is received, the preset consensus protocol is improved to update the target data, and the updated target data is stored in the multi-level cache; then, removing the expired data in the multi-level cache through a preset replacement protocol; thereby achieving the hit rate of the target data.
The invention also provides a data processing device of the multi-level cache. Referring to fig. 8, the data processing apparatus of the present invention includes:
an obtaining module 10, configured to obtain a query request of data;
the query module 20 is configured to query the multi-level cache step by step according to the query request to obtain target data corresponding to the query request;
and the output module 30 is configured to output target data corresponding to the query request.
Furthermore, the present invention also provides a medium, which is a computer readable storage medium, on which a multi-level cached data processing program is stored, and when being executed by a processor, the multi-level cached data processing program implements the steps of the multi-level cached data processing method as described above.
The method implemented when the data processing program of the multi-level cache running on the processor is executed may refer to each embodiment of the data processing method of the multi-level cache of the present invention, and details are not described here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A data processing method of a multi-level cache is characterized in that the data processing method of the multi-level cache is applied to a network disk sharing system of the multi-level cache, and comprises the following steps:
acquiring a query request of data;
gradually inquiring in a multi-level cache according to the inquiry request to obtain target data corresponding to the inquiry request;
and outputting the target data corresponding to the query request.
2. The method for processing data of a multi-level cache according to claim 1, wherein the multi-level cache includes a local cache, a distributed cache, and a database, and before the step of obtaining the data query request, the method for processing data of a multi-level cache further includes:
acquiring historical records of data query in the local cache and access frequency corresponding to each historical record;
predicting the historical records and the access frequency through a prediction model to obtain a data prediction list corresponding to the historical records and the access frequency;
and according to the data prediction list, obtaining cache data from the local cache, and encrypting and storing the cache data to a multi-level cache.
3. The method for processing data in a multi-level cache according to claim 2, wherein before the step of calculating the history and the access frequency by using a prediction model to obtain the data prediction list corresponding to the history and the access frequency, the method for processing data in a multi-level cache further comprises:
acquiring sample history records inquired by sample data and sample access frequency corresponding to each sample history record;
and inputting the sample history record and the sample access frequency into a preset prediction model, and performing iterative training on the preset prediction model to obtain the prediction model.
4. The data processing method of multi-level cache according to claim 1, wherein the multi-level cache includes a local cache, a distributed cache, and a database, and the step of obtaining the target data corresponding to the query request by querying in the multi-level cache stage by stage according to the query request includes:
inquiring in the local cache according to the inquiry request;
if the target data is inquired in the local cache, returning the target data and quitting the inquiry, otherwise, entering the distributed cache for inquiry;
if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry;
and if the target data is inquired in the database, returning the target data and quitting the inquiry.
5. The data processing method of multi-level cache according to claim 1, wherein the multi-level cache includes a distributed cache and a database, and the step of obtaining the target data corresponding to the query request by querying in the multi-level cache stage by stage according to the query request includes:
inquiring in the distributed cache according to the inquiry request;
if the target data is inquired in the distributed cache, returning the target data and quitting the inquiry, otherwise, entering the database for inquiry;
and if the target data is inquired in the database, returning the target data and quitting the inquiry.
6. The method as claimed in claim 1, wherein said step of outputting the target data corresponding to the query request is followed by:
and when a data updating request is received, updating the target data through a preset consensus protocol, and storing the updated target data in the multi-level cache.
7. The method as claimed in claim 6, wherein after the step of updating the target data by a predetermined consensus protocol and storing the updated target data in the multi-level cache when a data update request is received, the method further comprises:
and clearing expired data in the multi-level cache by a preset replacement protocol.
8. A multi-level buffered data processing apparatus, comprising:
the acquisition module is used for acquiring a data query request;
the query module is used for querying in a multi-level cache step by step according to the query request to obtain target data corresponding to the query request;
and the output module is used for outputting the target data corresponding to the query request.
9. An apparatus, the apparatus being a multi-level cached data processing apparatus, the multi-level cached data processing apparatus comprising: memory, processor and a multi-level cached data processing program stored on the memory and executable on the processor, the multi-level cached data processing program, when executed by the processor, implementing the steps of the multi-level cached data processing method as claimed in any one of claims 1 to 7.
10. A medium, which is a computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a multi-level cached data processing program, which when executed by a processor implements the steps of the multi-level cached data processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210223775.XA CN114817173A (en) | 2022-03-07 | 2022-03-07 | Data processing method, device, equipment and medium for multi-level cache |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210223775.XA CN114817173A (en) | 2022-03-07 | 2022-03-07 | Data processing method, device, equipment and medium for multi-level cache |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114817173A true CN114817173A (en) | 2022-07-29 |
Family
ID=82529516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210223775.XA Pending CN114817173A (en) | 2022-03-07 | 2022-03-07 | Data processing method, device, equipment and medium for multi-level cache |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114817173A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116010472A (en) * | 2023-03-28 | 2023-04-25 | 山东浪潮科学研究院有限公司 | Data query method, device, equipment and storage medium |
| CN117539915A (en) * | 2024-01-09 | 2024-02-09 | 腾讯科技（深圳）有限公司 | Data processing method and related device |
-
2022
- 2022-03-07 CN CN202210223775.XA patent/CN114817173A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116010472A (en) * | 2023-03-28 | 2023-04-25 | 山东浪潮科学研究院有限公司 | Data query method, device, equipment and storage medium |
| CN117539915A (en) * | 2024-01-09 | 2024-02-09 | 腾讯科技（深圳）有限公司 | Data processing method and related device |
| CN117539915B (en) * | 2024-01-09 | 2024-04-23 | 腾讯科技（深圳）有限公司 | Data processing method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107426302B (en) | Access scheduling method, device, system, terminal, server and storage medium | |
| JP5828760B2 (en) | Method and system for cache optimization | |
| US20100306339A1 (en) | P2p content caching system and method | |
| CN114817173A (en) | Data processing method, device, equipment and medium for multi-level cache | |
| EP2347561B1 (en) | Determining a network interface to access a network | |
| JP5389153B2 (en) | Method and system for requesting routing based on class | |
| EP3022708B1 (en) | Content source discovery | |
| US20080320003A1 (en) | Scaling network services using dns | |
| US20080183721A1 (en) | Method of and system for providing performance information in a uddi system | |
| CN109167840B (en) | Task pushing method, node autonomous server and edge cache server | |
| CN110430188B (en) | Rapid URL filtering method and device | |
| US8706856B2 (en) | Service directory | |
| WO2015168024A1 (en) | Smart caching based on reputation information | |
| CN102771080A (en) | System and methods for efficient media delivery using cache | |
| US20120331085A1 (en) | Load balancing based upon data usage | |
| CN109634753B (en) | Data processing method, device, terminal and storage medium for switching browser kernels | |
| CN113127420B (en) | Metadata request processing method, device, equipment and medium | |
| WO2015047968A1 (en) | Data caching policy in multiple tenant enterprise resource planning system | |
| CN111464521A (en) | Method, device, computer equipment and storage medium for preventing domain name from being hijacked | |
| CN103269353A (en) | Web cache and return optimization method and Web cache system | |
| CN107451252A (en) | Method for quickly querying and its system based on API | |
| CN113420241A (en) | Page access method and device, electronic equipment and storage medium | |
| CN112968980B (en) | Probability determination method and device, storage medium and server | |
| CN113472914B (en) | DNS directional prefetching caching method and system | |
| CN113014683B (en) | Request processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |