CN112749198A - Multi-level data caching method and device based on version number - Google Patents
Multi-level data caching method and device based on version number Download PDFInfo
- Publication number
- CN112749198A CN112749198A CN202110084049.XA CN202110084049A CN112749198A CN 112749198 A CN112749198 A CN 112749198A CN 202110084049 A CN202110084049 A CN 202110084049A CN 112749198 A CN112749198 A CN 112749198A
- Authority
- CN
- China
- Prior art keywords
- version number
- query result
- data
- information
- request information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The invention discloses a multi-level data caching method and device based on version numbers, wherein the method comprises the following steps: configuring first version number management table information according to a preset rule to obtain first transaction request information, and loading all version number information in the first version number management table according to the first transaction request information to obtain first access request information; and determining a first version number from all the version number information in the loaded first version number management table according to the first access request information. When data is queried, the latest version number is used for querying from the cache, if the data is hit, the data is directly returned, if the data is not hit, the data is penetrated into a database for querying, the result is returned, and then the data corresponding to the latest version number is synchronized into each layer of cache. The method solves the technical problems that the real-time consistency and integrity of the multi-level cache data of the distributed system cannot be ensured and the system availability is poor in the prior art.
Description
Technical Field
The invention relates to the field of distributed system data caching, in particular to a multi-level data caching method and device based on version numbers.
Background
In a distributed system architecture with a large flow rate, a multi-level cache design is often adopted, that is, data caching is performed at different levels of the whole system architecture to improve access efficiency, which is one of the most widely applied schemes. When a data query request is received, the application firstly queries in a local process cache, if the data query request is hit, the data is directly returned, and if the data query request is not hit, the data query request continues to go to a distributed cache layer for query; if the distributed cache layer is hit, directly returning, if the distributed cache layer is not hit, directly requesting the database layer, returning the result, and then synchronizing the data to the multi-level cache layer. After the data of the relational database layer is synchronized to the distributed cache layer and the local cache layer, if the synchronized data of the database layer is changed, such as newly added, deleted or modified, the data of the cache layer at each level is inconsistent with the data of the database layer. When an application program accesses data, if the data in the cache layer is inconsistent with the data in the database layer, a distributed system requiring real-time consistency of the data is a serious security event. The data consistency problem is mainly embodied in how to update the cache when the cache data is updated, and the data consistency between the database and the cache and between each layer of cache layers is ensured.
The existing methods for ensuring data consistency by multi-level cache of a distributed system mainly comprise several methods: deleting the cache, then executing database transaction to write the database, and setting the cache when reading the data; a message queue modification scheme; a Timer modification scheme; performing asynchronous serialization on the database, the cache update and the reading operation; scheme based on timestamp alignment.
In the process of implementing the technical scheme of the invention in the embodiment of the present application, the inventor of the present application finds that the above-mentioned technology has at least the following technical problems:
the system delay and the system availability are poor, the consistency and the integrity of real-time data cannot be guaranteed, the data synchronism is poor, and the design of an application layer is complex.
Disclosure of Invention
The embodiment of the application provides a multi-level data caching method and device based on version numbers, solves the technical problems that in the prior art, system delay and system availability are poor, real-time data consistency and integrity cannot be guaranteed, data synchronism is poor, and application layer design is complex, realizes multi-level cache data real-time consistency based on version number identification cache data, combines preloading and lazy loading modes to efficiently synchronize multi-level cache data, and effectively supports the technical purposes of local same-city double-activity or two-place three-center deployment implementation of a distributed system multi-level cache architecture.
The embodiment of the present application provides a multi-level data caching method based on a version number, which is applied to a multi-level cache system, wherein the multi-level cache system includes a first memory cache library, a second memory cache library, and a first database, and the method includes: configuring first version number management table information according to a preset rule; obtaining first transaction request information; loading all version number information in the first version number management table according to the first transaction request information; obtaining first access request information, wherein the first access request information comprises a first target data table; determining a first version number from all version number information in the loaded first version number management table according to the first access request information, wherein the first version number corresponds to the first target data table; according to the first version number, inquiring first data from the first memory cache library and obtaining a first inquiry result, wherein the first memory cache library is a local memory cache; when the first query result is a no-value hit, querying the first data from the second memory cache library and obtaining a second query result, wherein the second memory cache library is a distributed cache; when the second query result is no value hit or abnormal, second access request information is obtained; according to the second access request information, after the first database is accessed, a third query result is obtained; and writing the third query result into the second memory cache library and the first memory cache library in sequence, and returning the third query result to the application node in the multilayer cache system.
On the other hand, the present application further provides a multi-level data caching apparatus based on version number, wherein the apparatus includes: the first configuration unit is used for configuring the first version number management table information according to a preset rule; a first obtaining unit for obtaining first transaction request information; the first loading unit is used for loading all version number information in the first version number management table according to the first transaction request information; a second obtaining unit, configured to obtain first access request information, where the first access request information includes a first target data table; a third obtaining unit, configured to determine, according to the first access request information, a first version number from all version number information in the loaded first version number management table, where the first version number corresponds to the first target data table; a fourth obtaining unit, configured to query, according to the first version number, first data from the first memory cache library and obtain a first query result, where the first memory cache library is a local memory cache; a fifth obtaining unit, configured to, when the first query result is a no-value hit, query the first data from the second memory cache library, and obtain a second query result, where the second memory cache library is a distributed cache; a sixth obtaining unit, configured to obtain second access request information when the second query result is a no-value hit or an exception; a seventh obtaining unit, configured to obtain a third query result after accessing the first database according to the second access request information; and the first reading unit is used for sequentially writing the third query result into the second memory cache library and the first memory cache library, and returning the third query result to the application node in the multi-layer cache system.
On the other hand, an embodiment of the present application further provides a multi-level data caching apparatus based on a version number, including a memory, a processor, and a computer program that is stored in the memory and can be run on the processor, where the processor implements the steps of the method according to the first aspect when executing the program.
One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
because the data is cached by using the version number identification, the direct data consistency of each layer of cache and the direct data consistency of cache and a database are ensured, and the version number in the configuration table needs to be synchronously changed each time when a certain data table has data change. When data is queried, the latest version number is used for querying from the cache, if the data is hit, the data is directly returned, if the data is not hit, the data is penetrated into a database for querying, the result is returned, and then the data corresponding to the latest version number is synchronized into each layer of cache. Meanwhile, the multi-level cache data synchronization method combining the preloading mode and the lazy loading mode is adopted, the synchronization efficiency is high, the use is simple, the real-time consistency of the multi-level cache data is realized based on the version number identification cache data, the multi-level cache data is efficiently synchronized by combining the preloading mode and the lazy loading mode, and the technical purpose of local same-city double-activity or two-place three-center deployment implementation of the multi-level cache architecture of the distributed system is effectively supported.
The foregoing is a summary of the present disclosure, and embodiments of the present disclosure are described below to make the technical means of the present disclosure more clearly understood.
Drawings
Fig. 1 is a schematic flowchart of a multi-level data caching method based on version numbers according to an embodiment of the present application;
fig. 2 is another schematic flow chart of a multi-level data caching method based on version numbers according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a multi-level cache system architecture;
fig. 4 is a schematic structural diagram of a multi-level data caching apparatus based on version numbers according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an exemplary electronic device according to an embodiment of the present application.
Description of reference numerals: a first configuration unit 11, a first obtaining unit 12, a first loading unit 13, a second obtaining unit 14, a third obtaining unit 15, a fourth obtaining unit 16, a fifth obtaining unit 17, a sixth obtaining unit 18, a seventh obtaining unit 19, a first reading unit 20, a bus 300, a receiver 301, a processor 302, a transmitter 303, a memory 304, and a bus interface 305.
Detailed Description
The embodiment of the application provides a multi-level data caching method and device based on version numbers, solves the technical problems that in the prior art, system delay and system availability are poor, real-time data consistency and integrity cannot be guaranteed, data synchronism is poor, and application layer design is complex, realizes multi-level cache data real-time consistency based on version number identification cache data, combines preloading and lazy loading modes to efficiently synchronize multi-level cache data, and effectively supports the technical purposes of local same-city double-activity or two-place three-center deployment implementation of a distributed system multi-level cache architecture. Hereinafter, example embodiments of the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are merely some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.
Summary of the application
In a distributed system, particularly a distributed transaction and payment system, a scenario of high-concurrency reading of some hot data (low-frequency modification and high-frequency access) often occurs, in order to improve access efficiency, a multi-level cache architecture design of a local memory cache and a distributed cache (such as a redis cluster) is generally adopted, and such a system requires that cache data and database data must meet real-time consistency. The prior art also has the technical problems of system delay, poor system availability, incapability of ensuring the consistency and integrity of real-time data, poor data synchronism and more complex application layer design.
In view of the above technical problems, the technical solution provided by the present application has the following general idea:
the embodiment of the present application provides a multi-level data caching method based on a version number, which is applied to a multi-level cache system, wherein the multi-level cache system includes a first memory cache library, a second memory cache library, and a first database, and the method includes: configuring first version number management table information according to a preset rule; obtaining first transaction request information; loading all version number information in the first version number management table according to the first transaction request information; obtaining first access request information, wherein the first access request information comprises a first target data table; determining a first version number from all version number information in the loaded first version number management table according to the first access request information, wherein the first version number corresponds to the first target data table; according to the first version number, inquiring first data from the first memory cache library and obtaining a first inquiry result, wherein the first memory cache library is a local memory cache; when the first query result is a no-value hit, querying the first data from the second memory cache library and obtaining a second query result, wherein the second memory cache library is a distributed cache; when the second query result is no value hit or abnormal, second access request information is obtained; according to the second access request information, after the first database is accessed, a third query result is obtained; and writing the third query result into the second memory cache library and the first memory cache library in sequence, and returning the third query result to the application node in the multilayer cache system.
Having thus described the general principles of the present application, various non-limiting embodiments thereof will now be described in detail with reference to the accompanying drawings.
Example one
As shown in fig. 1, an embodiment of the present application provides a multi-level data caching method based on version numbers, which is applied to a multi-level cache system, where the multi-level cache system includes a first memory cache library, a second memory cache library, and a first database, where the method includes:
step S100: configuring first version number management table information according to a preset rule;
further, the embodiment S100 of the present application further includes:
step S101 a: obtaining a first header name, a second header name and a third header name, wherein a first association relationship exists between the first header name and the second header name;
step S102 a: respectively obtaining a first table content according to the first table head name, a second table content according to the second table head name, and a third table content according to the third table head name;
step S103 a: and configuring the first version number management table information according to the first header name, the first table content, the second header name, the second table content, the third header name and the third table content.
Specifically, the first table content, the second table content, and the third table content are determined by determining the first header name, the second header name, and the third header name, thereby completing the configuration of the first version number management table.
The VRSNNUM table structure is as follows:
TABLE 1 structural information of VRSNNUM tables
| Name of field | Meaning of a field | Whether or not it can be empty |
| BANK_NUM | Bank number | Can not be empty |
| VRSN_TYP | Type of parameter | Can not be empty |
| LAST_VRSN_NUM | Last version number | Can not be empty |
| CRNT_VRSN_NUM | Version number of the day | Can not be empty |
| USER_NAM | Maintenance person | |
| UPD_TIME | Maintenance time |
Specifically, the version number is a common version design scheme, that is, a version number field is added on a table where historical data is to be preserved, a new version is created every time data operation is performed, and the version is only increased or decreased, so that the latest business data can be obtained only by taking the maximum version number. Besides the ability to retain historical data, one function of version numbers is to avoid concurrent editing operations. When the data table needs to use the cache function, the VRSNNUM table needs to be configured.
The VRSNNUM is used for managing the version numbers of all the data tables, the version numbers are used for identifying the cache data in the scheme, the direct data consistency of each layer of cache and the direct data consistency of the cache and the database are ensured, and the version number +1 configured in the VRSNNUM needs to be synchronized each time when data change exists in a certain data table.
Step S200: obtaining first transaction request information;
step S300: loading all version number information in the first version number management table according to the first transaction request information;
specifically, as shown in fig. 2, after the application receives the transaction request, all version number information in the VRSNNUM table is read into the local memory Map structure during initialization, and the version numbers form a rule: 8-bit accounting date + 3-bit bank number + 5-bit sequence number, the first version number of each day, the sequence number starting at 00000. The Map structure is an associative container of STL that provides one-to-one (where the first may be referred to as a key, each key can only appear once in the Map, and the second may be referred to as the value of the key) data processing capability, and due to this property, it accomplishes the potential to programmatically provide a fast channel while we are processing one-to-one data.
Step S400: obtaining first access request information, wherein the first access request information comprises a first target data table;
step S500: determining a first version number from all version number information in the loaded first version number management table according to the first access request information, wherein the first version number corresponds to the first target data table;
specifically, the multi-layer cache system obtains first access request information of the application, where the first access request includes access request information of the application to the first target data table, and the first access request information finds version number information corresponding to the data table from the first version number management table in a local memory version number Map structure, that is, obtains the first version number information. And the data read in one transaction processing period is ensured to be consistent through the first version number.
Step S600: according to the first version number, inquiring first data from the first memory cache library and obtaining a first inquiry result, wherein the first memory cache library is a local memory cache;
step S700: when the first query result is a no-value hit, querying the first data from the second memory cache library and obtaining a second query result, wherein the second memory cache library is a distributed cache;
specifically, the architecture of the multi-level cache system in this embodiment is shown in fig. 3, where the local cache is a memory space where data is stored in application code, and the local cache has the advantages of providing fast data access and having the disadvantages of no distributed sharing and no fault-tolerant processing. The distributed cache is used for storing data in a distributed mode among a fixed number of cluster nodes, and has the advantages that cache capacity is expandable, and the defects that a large amount of configuration is needed in the expansion process and a fault-tolerant mechanism is not available. The mode of the application accessing the data table is single-stroke or multi-stroke access by pressing a primary key, and the data access is carried out by using the primary key value or index value and version number. When the application accesses the first target data table, the first data is searched in the first memory cache library, namely a local memory cache according to the first version number information, and the first query result is obtained and is divided into a result of cache data with a value hit and a result of cache data without a value hit. And when the first query result is no-value hit, using a primary key value or an index value and a version number to continuously search the first data in the second memory cache library, namely the distributed cache library, and obtaining a second query result, wherein the second query result also comprises the results of value hit and no-value hit of the first data in the distributed cache.
Step S800: when the second query result is no value hit or abnormal, second access request information is obtained;
specifically, the second access request information is request information for directly accessing the database by the application, and when the distributed cache has no value hit or access abnormality, the second access request information is obtained, the application directly accesses the data, and the first data information is searched in the database.
Step S900: according to the second access request information, after the first database is accessed, a third query result is obtained;
specifically, according to the second access request information, the application directly accesses the database by using the primary key value or the index value, and obtains the third query result information, where the third query result information includes information on whether the first data has a numerical hit in the database.
Step S1000: and writing the third query result into the second memory cache library and the first memory cache library in sequence, and returning the third query result to the application node in the multilayer cache system.
Specifically, after the application directly accesses the database to obtain the third query result information, the system writes the third query result into a distributed cache (Redis) and a local memory cache, and finally sends the third query result to the application. Redis is free of source, complies with the BSD protocol, and is a high-performance key-value non-relational database. The application node is a level below the application virtualization node in a scope pane of an application virtualization client management console, and the data processing system realizes transmission and acquisition of information between the application node and the application.
Further, step S100 in the embodiment of the present application further includes:
step S101 b: obtaining the current time;
step S102 b: when daily cutting processing is carried out within the current time, first use information is obtained;
step S103 b: according to the first use information, using a first initial version number of the current time, preloading the whole data in the first version number management table into the second memory cache library, wherein the cache data is stored in a key-value mode;
step S104 b: assigning the first initial version number to a second version number in a full preloading mode, and assigning a third version number to the first initial version number so as to access a cache by using the third version number in a time period from the completion of full preloading to the time before the daily cutting processing, wherein the second version number is a version number corresponding to the last time of the current time, and the third version number is a version number corresponding to the next time of the current time;
step S105 b: and when the daily cutting processing is finished, using the third version number.
Specifically, switching the dates in a daily cutting mode, changing the time of system billing, performing centralized processing on system services on the same day, switching the system from the current working day to the next working day after the processing is finished, and submitting transactions as usual and processing the transactions correctly in the daily cutting process. After the VRSNNUM table is maintained, the first initial version number is used before every day, the total data of the configured data table is preloaded into the distributed cache (redis cache), the cache data is stored in a key-value mode, and the key can be spliced with the current version number of the data table. The Key-value database is a non-relational database model, and data of the Key-value database is organized according to a Key-value pair form and is organized, indexed and stored in the Key-value pair form. And the total preloading assigns the version number of the current day to the version number of the previous day, assigns the version number of the next day to the version number of the current day, and initializes the version number of the current day so as to use the third version number to access the cache in the time period from the completion of the total preloading to the time before the daily cutting, ensure that the transaction can use the version number of the previous day to normally access the cache after the completion of the preloading and use the third version number after the daily cutting.
Further, step S200 in the embodiment of the present application further includes:
step S201 a: and reading all version number information into a Map structure of the first memory cache library in the process of loading all version number information in the first version number management table, wherein the generation rule of the version number comprises a first-digit accounting date, a second-digit bank number and a third-digit sequence number.
Specifically, after receiving the transaction request, the application reads all version number information in the VRSNNUM table to the Map structure of the local first memory cache library during initialization, and the generation rule of the version number is as follows: 8-bit accounting date + 3-bit bank number + 5-bit sequence number, the first version number of each day, the sequence number starting at 00000.
Further, step S200 in the embodiment of the present application further includes:
step S201 b: when the first target data table is changed, a second target data table and a first calling instruction are obtained;
step S202 b: and according to the first calling instruction, after a calling interface changes all version number information in the first version number management table, obtaining all changed version number information so as to enable the second target data table to correspond to all changed version number information.
Specifically, when the first target data table has data change, a uniformly packaged interface is called to update the version number information in the VRSNNUM table, and the data change of the first target data table and the version number change of the VRSNNUM table are in the same transaction, so that the second target data table after being changed and the updated new version number are ensured to keep a mutual corresponding relation.
Further, step S202b in the embodiment of the present application further includes:
step S202b 1: obtaining second access request information, wherein the second access request information comprises a second target data table;
step S202b 2: determining a second target version number from all the change version number information according to the second access request information, wherein the second target version number corresponds to the second target data table;
step S202b 3: according to the second target version number, after the first database is accessed, a fourth query result is obtained;
step S202b 4: and writing the fourth query result into the second memory cache library and the first memory cache library in sequence, and returning the fourth query result to the application node in the multilayer cache system.
Specifically, after the data in the first target data table is changed, when the application accesses the data table, a new version number corresponding to the table, that is, the information of the second target version number, is obtained. When the application searches for data in the first memory cache and the second memory cache respectively by using a primary key value or an index value and a new version number, the application directly accesses the database, sequentially writes the fourth query result into the second memory cache and the first memory cache, namely the distributed cache and the local memory cache, and sends the fourth query result to the application through an application node in the multi-layer cache system. This manner of passively synchronizing cached data may be referred to as lazy loading or reactive loading.
Further, step S700 in the embodiment of the present application further includes:
step S701: when the first query result is a hit with a value, obtaining a fifth query result, and returning the fifth query result to the application node in the multi-layer cache system;
step S702: and when the second query result is a hit with a value, obtaining a sixth query result, writing the sixth query result into the first memory cache library, and returning the sixth query result to the application node in the multi-layer cache system.
Specifically, when the application searches for data in the first memory cache library, that is, the local memory cache, according to the version number, if a value of cache data is hit, the fifth query result is obtained, and the fifth query result is sent to the application through an application node in the multi-layer cache system; when the application searches data in the second memory cache library, namely the distributed memory cache, according to the version number, if the cache data has a hit value, the sixth query result is obtained, and the sixth query result is sent to the application through the application node in the multi-layer cache system.
In practical use, the multi-level data caching method based on the version number in this embodiment may also be applied to a bank system, that is, a core business system of a bank, as shown in fig. 3, where the system employs a distributed system architecture, a distributed database, and a multi-level caching mechanism (local memory, redis cluster, database). In the system, data read after one transaction comes in must be consistent, the data reading in one transaction processing period is ensured to be consistent through the version number, and the consistency of data of a database, a redis cache and a local memory is ensured while the timeliness is ensured. When data are updated, the cached data are lazily loaded to the redis cluster and the local memory cache by synchronously modifying the version number. By adopting a multi-level cache mechanism, the redis load is effectively reduced, the node pressure of the distributed database is relieved, and the response efficiency of the system is improved.
To sum up, the multi-level data caching method based on the version number provided by the embodiment of the application has the following technical effects:
because the data is cached by using the version number identification, the direct data consistency of each layer of cache and the direct data consistency of cache and a database are ensured, and the version number in the configuration table needs to be synchronously changed each time when a certain data table has data change. When data is queried, the latest version number is used for querying from the cache, if the data is hit, the data is directly returned, if the data is not hit, the data is penetrated into a database for querying, the result is returned, and then the data corresponding to the latest version number is synchronized into each layer of cache. Meanwhile, the multi-level cache data synchronization method combining the preloading mode and the lazy loading mode is adopted, the synchronization efficiency is high, the use is simple, the real-time consistency of the multi-level cache data is realized based on the version number identification cache data, the multi-level cache data is efficiently synchronized by combining the preloading mode and the lazy loading mode, and the technical purpose of local same-city double-activity or two-place three-center deployment implementation of the multi-level cache architecture of the distributed system is effectively supported.
Example two
Based on the same inventive concept as the multi-level data caching method based on the version number in the foregoing embodiment, the present invention further provides a multi-level data caching apparatus based on the version number, as shown in fig. 4, the apparatus includes:
a first configuration unit 11, where the first configuration unit 11 is configured to configure first version number management table information according to a preset rule;
a first obtaining unit 12, wherein the first obtaining unit 12 is used for obtaining first transaction request information;
a first loading unit 13, where the first loading unit 13 is configured to load all version number information in the first version number management table according to the first transaction request information;
a second obtaining unit 14, where the second determining unit 14 is configured to obtain first access request information, where the first access request information includes a first target data table;
a third obtaining unit 15, where the third obtaining unit 15 is configured to determine, according to the first access request information, a first version number from all version number information in the loaded first version number management table, where the first version number corresponds to the first target data table;
a fourth obtaining unit 16, where the fourth obtaining unit 16 is configured to query the first data from the first memory cache library according to the first version number, and obtain a first query result, where the first memory cache library is a local memory cache;
a fifth obtaining unit 17, where the fifth obtaining unit 17 is configured to, when the first query result is a no-value hit, query the first data from the second memory cache library, and obtain a second query result, where the second memory cache library is a distributed cache;
a sixth obtaining unit 18, where the sixth obtaining unit 18 is configured to obtain second access request information when the second query result is a no-value hit or an exception;
a seventh obtaining unit 19, where the seventh obtaining unit 19 is configured to obtain a third query result after accessing the first database according to the second access request information;
the first reading unit 20, where the first reading unit 20 is configured to sequentially write the third query result into the second memory cache library and the first memory cache library, and return the third query result to an application node in the multi-level cache system.
Further, the apparatus further comprises:
an eighth obtaining unit, configured to obtain a first header name, a second header name, and a third header name, where the first header name and the second header name have a first association relationship therebetween;
a ninth obtaining unit, configured to obtain a first table content according to the first header name, a second table content according to the second header name, and a third table content according to the third header name, respectively;
and the second configuration unit is used for configuring the first version number management table information according to the first header name, the first table content, the second header name, the second table content, the third header name and the third table content.
Further, the apparatus further comprises:
a tenth obtaining unit configured to obtain a current time;
an eleventh obtaining unit configured to obtain first usage information when the daily cutting process is performed within the current time;
a second loading unit, configured to preload, according to the first usage information and using a first initial version number of a current time, full data in the first version number management table into the second memory cache library, where the cache data is stored in a key-value manner;
a twelfth obtaining unit, configured to assign the first initial version number to a second version number in a full preloading manner, and assign a third version number to the first initial version number, so that the third version number is used to access the cache in a time period from completion of full preloading to before the japanese-cut processing, where the second version number is a version number corresponding to a previous time of the current time, and the third version number is a version number corresponding to a next time of the current time;
a thirteenth obtaining unit configured to use the third version number after the end of the japanese cutting process.
Further, the apparatus further comprises:
and the second reading unit is used for reading all version number information into a Map structure of the first memory cache library in the process of loading all version number information in the first version number management table, wherein the generation rule of the version number comprises a first-digit accounting date, a second-digit bank number and a third-digit sequence number.
Further, the apparatus further comprises:
a fourteenth obtaining unit, configured to obtain a second target data table and a first call instruction when the first target data table is changed;
a fifteenth obtaining unit, configured to obtain all version number information after a call interface changes all version number information in the first version number management table according to the first call instruction, so that the second target data table corresponds to all version number information.
Further, the apparatus further comprises:
a sixteenth obtaining unit, configured to obtain second access request information, where the second access request information includes a second target data table;
a seventeenth obtaining unit, configured to determine a second target version number from the all change version number information according to the second access request information, where the second target version number corresponds to the second target data table;
an eighteenth obtaining unit, configured to obtain a fourth query result after accessing the first database according to the second target version number;
and the third reading unit is used for sequentially writing the fourth query result into the second memory cache library and the first memory cache library and returning the fourth query result to the application node in the multilayer cache system.
Further, the apparatus further comprises:
a nineteenth obtaining unit, configured to, when the first query result is a hit with a value, obtain a fifth query result, and return the fifth query result to an application node in the multilayer cache system;
a twentieth obtaining unit, configured to obtain a sixth query result when the second query result is a value hit, write the sixth query result into the first memory cache library, and return the sixth query result to the application node in the multi-tiered cache system.
Various changes and specific examples of the version number-based multi-level data caching method in the first embodiment of fig. 1 are also applicable to the version number-based multi-level data caching device in this embodiment, and through the foregoing detailed description of the version number-based multi-level data caching method, those skilled in the art can clearly know the version number-based multi-level data caching device in this embodiment, so for the sake of brevity of the description, detailed descriptions thereof are omitted here.
Exemplary electronic device
The electronic device of the embodiment of the present application is described below with reference to fig. 5.
Fig. 5 illustrates a schematic structural diagram of an electronic device according to an embodiment of the present application.
Based on the inventive concept of the multi-level data caching method based on the version number in the foregoing embodiments, the present invention further provides a multi-level data caching apparatus based on the version number, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any one of the above-mentioned multi-level data caching methods based on the version number.
Where in fig. 5 a bus architecture (represented by bus 300), bus 300 may include any number of interconnected buses and bridges, bus 300 linking together various circuits including one or more processors, represented by processor 302, and memory, represented by memory 304. The bus 300 may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface 305 provides an interface between the bus 300 and the receiver 301 and transmitter 303. The receiver 301 and the transmitter 303 may be the same element, i.e., a transceiver, providing a means for communicating with various other apparatus over a transmission medium.
The processor 302 is responsible for managing the bus 300 and general processing, and the memory 304 may be used for storing data used by the processor 302 in performing operations.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (9)
1. A multi-level data caching method based on version numbers is applied to a multi-level caching system, wherein the multi-level caching system comprises a first memory caching library, a second memory caching library and a first database, and the method comprises the following steps:
configuring first version number management table information according to a preset rule;
obtaining first transaction request information;
loading all version number information in the first version number management table according to the first transaction request information;
obtaining first access request information, wherein the first access request information comprises a first target data table;
determining a first version number from all version number information in the loaded first version number management table according to the first access request information, wherein the first version number corresponds to the first target data table;
according to the first version number, inquiring first data from the first memory cache library and obtaining a first inquiry result, wherein the first memory cache library is a local memory cache;
when the first query result is a no-value hit, querying the first data from the second memory cache library and obtaining a second query result, wherein the second memory cache library is a distributed cache;
when the second query result is no value hit or abnormal, second access request information is obtained;
according to the second access request information, after the first database is accessed, a third query result is obtained;
and writing the third query result into the second memory cache library and the first memory cache library in sequence, and returning the third query result to the application node in the multilayer cache system.
2. The method of claim 1, wherein the configuring the first version number management table information according to the preset rule comprises:
obtaining a first header name, a second header name and a third header name, wherein a first association relationship exists between the first header name and the second header name;
respectively obtaining a first table content according to the first table head name, a second table content according to the second table head name, and a third table content according to the third table head name;
and configuring the first version number management table information according to the first header name, the first table content, the second header name, the second table content, the third header name and the third table content.
3. The method as claimed in claim 1, wherein the configuring the first version number management table information according to the preset rule comprises:
obtaining the current time;
when daily cutting processing is carried out within the current time, first use information is obtained;
according to the first use information, using a first initial version number of the current time, preloading the total data in the first version number management table into the second memory cache library, wherein the cache data is stored in a key-value mode;
assigning the first initial version number to a second version number in a full preloading mode, and assigning a third version number to the first initial version number so as to access a cache by using the third version number in a time period from the completion of full preloading to the time before the daily cutting processing, wherein the second version number is a version number corresponding to the last time of the current time, and the third version number is a version number corresponding to the next time of the current time;
and when the daily cutting processing is finished, using the third version number.
4. The method of claim 1, wherein after obtaining the first transaction request information, the method further comprises:
and reading all version number information into a Map structure of the first memory cache library in the process of loading all version number information in the first version number management table, wherein the generation rule of the version number comprises a first-digit accounting date, a second-digit bank number and a third-digit sequence number.
5. The method of claim 1, wherein after obtaining the first transaction request information, the method further comprises:
when the first target data table is changed, a second target data table and a first calling instruction are obtained;
and according to the first calling instruction, after a calling interface changes all version number information in the first version number management table, obtaining all changed version number information so as to enable the second target data table to correspond to all changed version number information.
6. The method of claim 5, wherein the method further comprises:
obtaining second access request information, wherein the second access request information comprises a second target data table;
determining a second target version number from all the change version number information according to the second access request information, wherein the second target version number corresponds to the second target data table;
according to the second target version number, after the first database is accessed, a fourth query result is obtained;
and writing the fourth query result into the second memory cache library and the first memory cache library in sequence, and returning the fourth query result to the application node in the multilayer cache system.
7. The method of claim 1, wherein the method further comprises:
when the first query result is a hit with a value, obtaining a fifth query result, and returning the fifth query result to the application node in the multi-layer cache system;
and when the second query result is a hit with a value, obtaining a sixth query result, writing the sixth query result into the first memory cache library, and returning the sixth query result to the application node in the multi-layer cache system.
8. A multi-level data caching apparatus based on a version number, wherein the apparatus comprises:
the first configuration unit is used for configuring the first version number management table information according to a preset rule;
a first obtaining unit for obtaining first transaction request information;
the first loading unit is used for loading all version number information in the first version number management table according to the first transaction request information;
a second obtaining unit, configured to obtain first access request information, where the first access request information includes a first target data table;
a third obtaining unit, configured to determine, according to the first access request information, a first version number from all version number information in the loaded first version number management table, where the first version number corresponds to the first target data table;
a fourth obtaining unit, configured to query, according to the first version number, first data from the first memory cache library and obtain a first query result, where the first memory cache library is a local memory cache;
a fifth obtaining unit, configured to, when the first query result is a no-value hit, query the first data from the second memory cache library, and obtain a second query result, where the second memory cache library is a distributed cache;
a sixth obtaining unit, configured to obtain second access request information when the second query result is a no-value hit or an exception;
a seventh obtaining unit, configured to obtain a third query result after accessing the first database according to the second access request information;
and the first reading unit is used for sequentially writing the third query result into the second memory cache library and the first memory cache library, and returning the third query result to the application node in the multi-layer cache system.
9. A multi-level data caching device based on a version number, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110084049.XA CN112749198A (en) | 2021-01-21 | 2021-01-21 | Multi-level data caching method and device based on version number |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110084049.XA CN112749198A (en) | 2021-01-21 | 2021-01-21 | Multi-level data caching method and device based on version number |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112749198A true CN112749198A (en) | 2021-05-04 |
Family
ID=75652795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110084049.XA Pending CN112749198A (en) | 2021-01-21 | 2021-01-21 | Multi-level data caching method and device based on version number |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112749198A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115913646A (en) * | 2022-10-21 | 2023-04-04 | 网易（杭州）网络有限公司 | Method and device for intercepting blacklist object, electronic equipment and storage medium |
| CN115905246A (en) * | 2023-03-14 | 2023-04-04 | 智者四海（北京）技术有限公司 | KV cache method and device based on dynamic compression prefix tree |
| CN116226203A (en) * | 2023-05-05 | 2023-06-06 | 北京零壹视界科技有限公司 | Single object data caching method and device, electronic equipment and storage medium |
| CN117914944A (en) * | 2024-03-20 | 2024-04-19 | 暗物智能科技(广州)有限公司 | Distributed three-level caching method and device based on Internet of things |
-
2021
- 2021-01-21 CN CN202110084049.XA patent/CN112749198A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115913646A (en) * | 2022-10-21 | 2023-04-04 | 网易（杭州）网络有限公司 | Method and device for intercepting blacklist object, electronic equipment and storage medium |
| CN115905246A (en) * | 2023-03-14 | 2023-04-04 | 智者四海（北京）技术有限公司 | KV cache method and device based on dynamic compression prefix tree |
| CN116226203A (en) * | 2023-05-05 | 2023-06-06 | 北京零壹视界科技有限公司 | Single object data caching method and device, electronic equipment and storage medium |
| CN117914944A (en) * | 2024-03-20 | 2024-04-19 | 暗物智能科技(广州)有限公司 | Distributed three-level caching method and device based on Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112749198A (en) | Multi-level data caching method and device based on version number | |
| CN111338766B (en) | Transaction processing method and device, computer equipment and storage medium | |
| US9767131B2 (en) | Hierarchical tablespace space management | |
| CN109906448B (en) | Method, apparatus, and medium for facilitating operations on pluggable databases | |
| CN108509462B (en) | Method and device for synchronizing activity transaction table | |
| US20090063489A1 (en) | Accessing Data Entities | |
| EP3401807B1 (en) | Synopsis based advanced partition elimination | |
| CN109240946A (en) | The multi-level buffer method and terminal device of data | |
| CN111797121A (en) | Strong consistency query method, device and system for read-write separation architecture service system | |
| CN112162846B (en) | Transaction processing method, device and computer readable storage medium | |
| CN112035528A (en) | Data query method and device | |
| CN106934048A (en) | Online data moving method, agent node | |
| US6687798B1 (en) | Methods for intra-partition parallelism for inserts | |
| EP2380090A2 (en) | Data integrity in a database environment through background synchronization | |
| CN1829974B (en) | Parallel recovery by non-failed nodes | |
| CN115129618A (en) | Method and apparatus for optimizing data caching | |
| CN108804571B (en) | Data storage method, device and equipment | |
| CN114785662B (en) | Storage management method, device, equipment and machine-readable storage medium | |
| CN115964444A (en) | Cloud native distributed multi-tenant database implementation method and system | |
| CN115934583A (en) | Hierarchical caching method, device and system | |
| CN113127717A (en) | Key retrieval method and system | |
| CN115292394A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| KR102214697B1 (en) | A computer program for providing space managrment for data storage in a database management system | |
| US7185029B1 (en) | Method and apparatus for maintaining, and updating in-memory copies of the first and second pointers to reference the new versions of the first and second control structures that indicate available and allocated portions of usable space in the data file | |
| EP3844635A1 (en) | Method and apparatus for data writing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |