CN111221828A - Method and terminal for improving consistency of database data and cache data - Google Patents
Method and terminal for improving consistency of database data and cache data Download PDFInfo
- Publication number
- CN111221828A CN111221828A CN201811417566.9A CN201811417566A CN111221828A CN 111221828 A CN111221828 A CN 111221828A CN 201811417566 A CN201811417566 A CN 201811417566A CN 111221828 A CN111221828 A CN 111221828A
- Authority
- CN
- China
- Prior art keywords
- data
- cache
- updating
- message
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a terminal for improving the consistency of database data and cache data, wherein the method comprises the steps of receiving an updating operation on a database, deleting corresponding first data in a cache according to the updating operation, presetting the expiration time of an updating message corresponding to the updating operation, and deleting second data corresponding to the first data in the cache when the expiration time of the updating message is reached; the cache corresponding to the update message is deleted for the second time by reaching the preset expiration time of the update message, so that the problem that data in the cache can be written into dirty data by other processes after the current process is deleted under the condition of concurrent update and query is avoided, and the condition that the database data and the cache data are inconsistent is reduced.
Description
Technical Field
The invention relates to the field of data caching, in particular to a method and a terminal for improving the consistency of database data and cached data.
Background
When the access amount of the internet WEB application system is greatly increased, the query pressure of the database system is increased, and the response speed of the system is reduced. One common solution is to cache database data, because the cached data in the cache system is usually stored in the memory of the computer, and the performance is much higher than that of the traditional database data which needs to be read from the disk. The data requested by the user is preferentially acquired from the cache, so that the request amount of the database is reduced. However, in the process of updating the database data to the cache, there may be a case that the cache data is inconsistent with the database data due to a concurrent request or the like, so that the user queries dirty data from the cache.
For example, in a book management system, in order to improve the query efficiency, a program uses a cache in the process of acquiring the borrowable number of books in a database, and the specific process is as follows: when the user inquires, the program firstly inquires whether the record of the book exists in the cache, and if so, the program acquires data from the cache and returns an inquiry result. If the data does not exist, inquiring the record of the book from the database, writing the data into a cache, and simultaneously returning an inquiry result; when the user borrows, the records of the book in the database are updated, the borrowable number is reduced by 1, and the data of the book in the cache are deleted;
when a concurrency condition is met, such as stock inquiry and borrowing of the same book, the condition that the cache data is inconsistent with the database data may exist, and the user can inquire the dirty data. Supposing that the borrowable number of a book (computer principle) in a database is 8, a user A needs to inquire the borrowable number of the book, and a user B performs borrowing operation on the book; when a user A initiates a query request, a corresponding process A 'acquires a book record from a database and stores the book record in an inventory variable of a computer memory, the inventory variable value is 8, the user B performs a borrowing operation, the corresponding process B' changes the borrowable number of the book record in the database to 7, and meanwhile, the cache is updated; process A' then writes the in-memory inventory variable values 8 to the cache, thereby resulting in an inconsistency between the cached data and the database data. If the expiration time of the cache is long and the book is not borrowed any more later, other users will query the wrong inventory data for the book.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: a method and a terminal for improving the consistency of database data and cache data are provided, which can reduce the inconsistency of the database data and the cache data.
In order to solve the technical problems, the invention adopts a technical scheme that:
a method for improving the consistency of database data and cache data comprises the following steps:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
and S2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
a terminal for improving consistency of database data and cached data, comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor executes the computer program to implement the following steps:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
and S2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again.
The invention has the beneficial effects that: deleting corresponding first data in a cache according to the updating operation by receiving the updating operation of the database, presetting the expiration time of an updating message corresponding to the updating operation, and deleting second data corresponding to the first data in the cache again when the expiration time of the updating message is reached; the cache corresponding to the update message is deleted for the second time by reaching the preset expiration time of the update message, so that the problem that data in the cache can be written into dirty data by other processes after the current process is deleted under the condition of concurrent update and query is avoided, and the condition that the database data and the cache data are inconsistent is reduced.
Drawings
FIG. 1 is a flowchart of a method for improving consistency of database data and cached data according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a terminal for improving the consistency between database data and cached data according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating details of a method for improving the consistency of database data and cached data according to an embodiment of the present invention;
FIG. 4 is a flowchart of a data interaction of a method for improving consistency of database data and cached data according to an embodiment of the present invention;
fig. 5 is a schematic view of RabbitMQ message processing in the method for improving the consistency between database data and cache data according to the embodiment of the present invention;
description of reference numerals:
1. the terminal improves the consistency of database data and cache data; 2. a memory; 3. a processor.
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
The most key concept of the invention is as follows: by presetting the expiration time, when the expiration time is reached, the cache is deleted for the second time, so that the problem of inconsistency between database data and cache data under the conditions of concurrent updating and query is avoided.
Referring to fig. 1, a method for improving the consistency between database data and cache data includes the steps of:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
and S2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again.
From the above description, the beneficial effects of the present invention are: deleting corresponding first data in a cache according to the updating operation by receiving the updating operation of the database, presetting the expiration time of an updating message corresponding to the updating operation, and deleting second data corresponding to the first data in the cache again when the expiration time of the updating message is reached; the cache corresponding to the update message is deleted for the second time by reaching the preset expiration time of the update message, so that the problem that data in the cache can be written into dirty data by other processes after the current process is deleted under the condition of concurrent update and query is avoided, and the condition that the database data and the cache data are inconsistent is reduced.
Further, the step S1 further includes:
and storing the updating message corresponding to the updating operation.
As can be seen from the above description, by storing the update message corresponding to the update operation, the cache corresponding to the update message is conveniently deleted for the second time in the following process, so as to reduce the inconsistency between the database data and the cache data.
Further, the storing the update message corresponding to the update operation includes:
and storing the update message corresponding to the update operation in a delay queue, and setting the delay time as the expiration time of the preset update message corresponding to the update operation.
As can be seen from the above description, by storing the update message corresponding to the update operation in the delay queue, and setting the delay time as the expiration time, effective and accurate monitoring of the message expiration time is achieved through the delay queue.
Further, the S2 includes:
when the expiration time of the updating message is reached, sending the updating message to a deception queue from the delay queue;
and when the update message is monitored to exist in the deadlock queue, deleting the second data corresponding to the update message in the cache again.
As can be seen from the above description, by monitoring the update message in the deadlock queue, and deleting the second data corresponding to the update message in the cache again, dirty data in the cache is avoided.
Further, deleting the second data corresponding to the update message in the cache includes:
and deleting the second data corresponding to the key of the update message in the cache.
As can be seen from the above description, according to the key of the update message, the second data corresponding to the key in the cache is deleted, so that the accuracy of deletion is improved.
Referring to fig. 2, a terminal for improving consistency between database data and cached data includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the following steps:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
and S2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again.
From the above description, the beneficial effects of the present invention are: deleting corresponding first data in a cache according to the updating operation by receiving the updating operation of the database, presetting the expiration time of an updating message corresponding to the updating operation, and deleting second data corresponding to the first data in the cache again when the expiration time of the updating message is reached; the cache corresponding to the update message is deleted for the second time by reaching the preset expiration time of the update message, so that the problem that data in the cache can be written into dirty data by other processes after the current process is deleted under the condition of concurrent update and query is avoided, and the condition that the database data and the cache data are inconsistent is reduced.
Further, the step S1 further includes:
and storing the updating message corresponding to the updating operation.
As can be seen from the above description, by storing the update message corresponding to the update operation, the cache corresponding to the update message is conveniently deleted for the second time in the following process, so as to reduce the inconsistency between the database data and the cache data.
Further, the storing the update message corresponding to the update operation includes:
and storing the update message corresponding to the update operation in a delay queue, and setting the delay time as the expiration time of the preset update message corresponding to the update operation.
As can be seen from the above description, by storing the update message corresponding to the update operation in the delay queue, and setting the delay time as the expiration time, effective and accurate monitoring of the message expiration time is achieved through the delay queue.
Further, the S2 includes:
when the expiration time of the updating message is reached, sending the updating message to a deception queue from the delay queue;
and when the update message is monitored to exist in the deadlock queue, deleting the second data corresponding to the update message in the cache again.
As can be seen from the above description, by monitoring the update message in the deadlock queue, and deleting the second data corresponding to the update message in the cache again, dirty data in the cache is avoided.
Further, deleting the second data corresponding to the update message in the cache includes:
and deleting the second data corresponding to the key of the update message in the cache.
As can be seen from the above description, according to the key of the update message, the second data corresponding to the key in the cache is deleted, so that the accuracy of deletion is improved.
Example one
Referring to fig. 1, fig. 3, fig. 4 and fig. 5, a method for improving consistency of database data and cache data includes the steps of:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
wherein the step S1 further includes:
storing an updating message corresponding to the updating operation;
wherein the storing of the update message corresponding to the update operation includes:
storing the update message corresponding to the update operation in a delay queue, and setting the delay time as the expiration time of the update message corresponding to the preset update operation;
the method specifically comprises the following steps: receiving an updating operation on a database, storing an updating message corresponding to the updating operation into a delay queue through a delay exchanger of a RabbitMQ according to the updating operation, and setting the delay time as the expiration time of the updating message corresponding to the preset updating operation;
s2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again;
wherein the S2 includes:
when the expiration time of the updating message is reached, sending the updating message to a deception queue from the delay queue;
when the update message is monitored to exist in the deadlock queue, deleting second data corresponding to the update message in the cache again;
wherein deleting the second data corresponding to the update message in the cache comprises:
and deleting the second data corresponding to the key of the update message in the cache.
The method specifically comprises the following steps: as shown in fig. 5, when the expiration time of the update message is reached, the update message is sent from the delay queue to a deadlock queue via a deadlock switch; and when the update message is monitored to exist in the deadlock queue, deleting the second data corresponding to the key of the update message in the cache again.
Example two
Referring to fig. 2, a terminal 1 for improving consistency between database data and cached data includes a memory 2, a processor 3, and a computer program stored on the memory 2 and capable of running on the processor 3, where the processor 3 implements the steps in the first embodiment when executing the computer program.
EXAMPLE III
Referring to fig. 1, fig. 3, fig. 4, and fig. 5, the method for improving the consistency between the database data and the cached data is applied to a specific application scenario, for example, there is a book data table in the Mysql database, the name of the book data table is book, the caching system adopts memcached, and the data structure of the book data table book is shown in table 1 below:
TABLE 1
| Name of field | Data type |
| Book id | Numerical value |
| Name of book | Character string |
| Borrowable amount of money | Integer number of |
One of them is a book record: book id is 100, the name of the book is 'computer principle', borrowable quantity is 8, and the following steps are implemented:
step 1: as shown in fig. 4, a delay queue is configured in the RabbitMQ, and when a message in the delay queue expires, the RabbitMQ will automatically reissue the message to the set "DLX", and then route the message to the "DLQ", and the program performs corresponding processing by monitoring the message in the queue;
parameters are set for RabbitMQ as follows:
x-dead-letter-exchange DLX (defines a deceased switch);
x-dead-letter-routing-key DLQ (defines a deadlock queue);
x-max-length:1000000 (message length ═ 1 million);
x-message-ttl:3000 (message expiry time ═ 3 s);
step 2: the method comprises the steps that a user A performs one-time borrowing operation on a book with id of 100, after a process A 'corresponding to the borrowing operation of the user A executes a deduction statement, the value of a borrowable number field of a record in a database system is 7, meanwhile, a program deletes a cache record with a cache key of' book _100 'in a memached cache system, finally, the process A' sends an update message of 'book _ 100' to a delay queue of a RabbitMQ through a delay exchanger, and the delay time of the update message is set to be the expiration time of the update message corresponding to the preset update operation;
and step 3: when the expiration time of the update message is 3s, the update message is sent from the delay queue to a dead-letter queue DLQ through a dead-letter switch DLX, the dead-letter queue DLQ of the RabbitMQ receives the message of 'book _ 100', and the program deletes the cache record with the cache key of 'book _ 100' in the memached once again.
In summary, according to the method and the terminal for improving the consistency between the database data and the cache data provided by the present invention, by receiving the update operation on the database, the corresponding first data in the cache is deleted according to the update operation, the expiration time of the update message corresponding to the update operation is preset, and when the expiration time of the update message is reached, the second data corresponding to the first data in the cache is deleted again; the cache corresponding to the update message is deleted for the second time by reaching the expiration time of the preset update message, so that the problem that data in the cache can be written into dirty data by other processes after the current process is deleted under the condition of concurrent update and query is avoided, and the condition that the database data and the cache data are inconsistent is reduced; by self-defining configuration of the RabbitMQ, the data in the cache can be flexibly deleted for the second time according to the requirements of users, so that the use flexibility is improved; by utilizing the characteristics of the deadlock queue of the RabbitMQ, the condition that the database data and the cache data are inconsistent can be reduced under the condition of high concurrency or program exception.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for improving the consistency of database data and cache data is characterized by comprising the following steps:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
and S2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again.
2. The method for improving the consistency of database data and cached data according to claim 1, wherein said step S1 further comprises:
and storing the updating message corresponding to the updating operation.
3. The method of claim 2, wherein the storing the update message corresponding to the update operation comprises:
and storing the update message corresponding to the update operation in a delay queue, and setting the delay time as the expiration time of the preset update message corresponding to the update operation.
4. The method for improving the consistency of database data and cached data as recited in claim 3, wherein the step S2 comprises:
when the expiration time of the updating message is reached, sending the updating message to a deception queue from the delay queue;
and when the update message is monitored to exist in the deadlock queue, deleting the second data corresponding to the update message in the cache again.
5. The method of claim 4, wherein deleting the second data in the cache corresponding to the update message comprises:
and deleting the second data corresponding to the key of the update message in the cache.
6. A terminal for improving the consistency of database data and cached data, comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor executes the computer program to implement the following steps:
s1, receiving an updating operation on the database, deleting corresponding first data in a cache according to the updating operation, and presetting the expiration time of an updating message corresponding to the updating operation;
and S2, when the expiration time of the updating message is reached, deleting the second data corresponding to the first data in the cache again.
7. The terminal for improving the consistency of database data and cached data according to claim 6, wherein said step S1 further comprises:
and storing the updating message corresponding to the updating operation.
8. The terminal for improving the consistency of database data and cached data according to claim 7, wherein the storing the update message corresponding to the update operation comprises:
and storing the update message corresponding to the update operation in a delay queue, and setting the delay time as the expiration time of the preset update message corresponding to the update operation.
9. The terminal for improving the consistency of database data and cached data according to claim 8, wherein said S2 comprises:
when the expiration time of the updating message is reached, sending the updating message to a deception queue from the delay queue;
and when the update message is monitored to exist in the deadlock queue, deleting the second data corresponding to the update message in the cache again.
10. The terminal for improving the consistency between the database data and the cached data according to claim 9, wherein the deleting the second data in the cache corresponding to the update message comprises:
and deleting the second data corresponding to the key of the update message in the cache.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811417566.9A CN111221828A (en) | 2018-11-26 | 2018-11-26 | Method and terminal for improving consistency of database data and cache data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811417566.9A CN111221828A (en) | 2018-11-26 | 2018-11-26 | Method and terminal for improving consistency of database data and cache data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111221828A true CN111221828A (en) | 2020-06-02 |
Family
ID=70827763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811417566.9A Pending CN111221828A (en) | 2018-11-26 | 2018-11-26 | Method and terminal for improving consistency of database data and cache data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111221828A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112416955A (en) * | 2020-11-18 | 2021-02-26 | 四川长虹电器股份有限公司 | Cache consistency problem solving method based on high concurrency environment |
| CN112487037A (en) * | 2020-12-08 | 2021-03-12 | 平安国际智慧城市科技股份有限公司 | Cache data processing method and device, computer equipment and storage medium |
| CN113486037A (en) * | 2021-07-27 | 2021-10-08 | 北京京东乾石科技有限公司 | Cache data updating method, manager and cache server |
| CN113542113A (en) * | 2021-07-13 | 2021-10-22 | 浪潮云信息技术股份公司 | Method and tool for processing delay task based on RabbitMQ |
| CN113626457A (en) * | 2021-08-19 | 2021-11-09 | 福建天晴数码有限公司 | Method and system for realizing database and cache consistency by cache deletion retry mechanism |
| CN114138807A (en) * | 2021-12-06 | 2022-03-04 | 四三九九网络股份有限公司 | Method for updating hot spot data cache of database in real time |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020053668A (en) * | 2000-12-27 | 2002-07-05 | 오길록 | Method for providing data consistency between web server and web cache server in active network environment |
| CN102023973A (en) * | 2009-09-14 | 2011-04-20 | 中国移动通信集团公司 | Method, device and system for synchronizing database with application cache server |
| CN104156361A (en) * | 2013-05-13 | 2014-11-19 | 阿里巴巴集团控股有限公司 | Method and system for achieving data synchronization |
| CN106294607A (en) * | 2016-07-29 | 2017-01-04 | 北京奇虎科技有限公司 | Data cached update method and updating device |
-
2018
- 2018-11-26 CN CN201811417566.9A patent/CN111221828A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020053668A (en) * | 2000-12-27 | 2002-07-05 | 오길록 | Method for providing data consistency between web server and web cache server in active network environment |
| CN102023973A (en) * | 2009-09-14 | 2011-04-20 | 中国移动通信集团公司 | Method, device and system for synchronizing database with application cache server |
| CN104156361A (en) * | 2013-05-13 | 2014-11-19 | 阿里巴巴集团控股有限公司 | Method and system for achieving data synchronization |
| CN106294607A (en) * | 2016-07-29 | 2017-01-04 | 北京奇虎科技有限公司 | Data cached update method and updating device |
Non-Patent Citations (1)
| Title |
|---|
| WXD_1024: "RabbitMQ死信机制实现延迟队列", 《HTTPS://BLOG.CSDN.NET/WXD_1024/ARTICLE/DETAILS/83549967》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112416955A (en) * | 2020-11-18 | 2021-02-26 | 四川长虹电器股份有限公司 | Cache consistency problem solving method based on high concurrency environment |
| CN112416955B (en) * | 2020-11-18 | 2022-02-22 | 四川长虹电器股份有限公司 | Cache consistency problem solving method based on high concurrency environment |
| CN112487037A (en) * | 2020-12-08 | 2021-03-12 | 平安国际智慧城市科技股份有限公司 | Cache data processing method and device, computer equipment and storage medium |
| CN113542113A (en) * | 2021-07-13 | 2021-10-22 | 浪潮云信息技术股份公司 | Method and tool for processing delay task based on RabbitMQ |
| CN113486037A (en) * | 2021-07-27 | 2021-10-08 | 北京京东乾石科技有限公司 | Cache data updating method, manager and cache server |
| CN113626457A (en) * | 2021-08-19 | 2021-11-09 | 福建天晴数码有限公司 | Method and system for realizing database and cache consistency by cache deletion retry mechanism |
| CN114138807A (en) * | 2021-12-06 | 2022-03-04 | 四三九九网络股份有限公司 | Method for updating hot spot data cache of database in real time |
| CN114138807B (en) * | 2021-12-06 | 2024-04-12 | 四三九九网络股份有限公司 | Method for updating database hot spot data cache in real time |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111221828A (en) | Method and terminal for improving consistency of database data and cache data | |
| US9058351B2 (en) | Apparatus and method for read optimized bulk data storage | |
| CN102902730B (en) | Based on data reading method and the device of data buffer storage | |
| CN103617131B (en) | Data caching achieving method | |
| US8819074B2 (en) | Replacement policy for resource container | |
| CN109241072A (en) | Buffering updating method and system based on Canal | |
| JP2003006036A (en) | Clustered application server and web system having database structure | |
| US8793288B2 (en) | Online access to database snapshots | |
| CN102867070A (en) | Method for updating cache of key-value distributed memory system | |
| US11216412B2 (en) | Intelligent merging for efficient updates in columnar databases | |
| CN112307119A (en) | Data synchronization method, device, equipment and storage medium | |
| CN103365987B (en) | Clustered database system and data processing method based on shared-disk framework | |
| US10747773B2 (en) | Database management system, computer, and database management method | |
| CN113836162A (en) | Method and device for service decoupling and automatic updating of multi-level cache | |
| KR101806394B1 (en) | A data processing method having a structure of the cache index specified to the transaction in a mobile environment dbms | |
| CN109871338B (en) | Data storage method and device and computer equipment | |
| US7890456B2 (en) | Sharing of database objects | |
| US10417215B2 (en) | Data storage over immutable and mutable data stages | |
| CN108038121B (en) | Caching method and terminal | |
| CN103353891A (en) | Database management system and data processing method thereof | |
| US8843708B2 (en) | Control block linkage for database converter handling | |
| CN110581779A (en) | Configuration issuing platform system and configuration issuing method based on local cache | |
| US20120323874A1 (en) | Resource-specific control blocks for database cache | |
| US11822485B1 (en) | Systems and methods for coupled cache management | |
| WO2024108640A1 (en) | Pure column-based updating method and apparatus supporting row-level concurrency control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |