CN114896344A - Data synchronization method and system for distributed memory cache - Google Patents
Data synchronization method and system for distributed memory cache Download PDFInfo
- Publication number
- CN114896344A CN114896344A CN202210586332.7A CN202210586332A CN114896344A CN 114896344 A CN114896344 A CN 114896344A CN 202210586332 A CN202210586332 A CN 202210586332A CN 114896344 A CN114896344 A CN 114896344A
- Authority
- CN
- China
- Prior art keywords
- file
- version number
- local
- data
- updating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to a data synchronization method and a system of distributed memory cache, which relate to the technical field of computer data processing, and the method comprises the following steps: acquiring the version number of a remote file and the version number of a local file; the remote file is a file located in the distributed cache; the local file is a file located in a local cache; judging whether the version number of the remote file is consistent with the version number of the local file; when the version number of the remote file is inconsistent with the version number of the local file, updating the version number of the local file according to the version number of the target file, and updating the data of the local file according to the data of the target file; the target file is a file located in the database. The data synchronization method and the device update the data stored in the local memory based on the distributed cache, so that the data synchronization is more convenient and efficient.
Description
Technical Field
The present invention relates to the field of computer data processing technologies, and in particular, to a data synchronization method and system for a distributed memory cache.
Background
In the server-side programming, caching refers to loading the data of the database into the memory, and then, the access to the data is completed in the memory, so that the access to the database is reduced. The memory may be divided into a local cache and a distributed cache according to whether the cache and the application process belong to the same process.
The local cache is used for caching data in a memory space in the same process, and data reading and writing are completed in the same process; the distributed cache is an independently deployed process and is generally deployed on a different machine from the application process, so that data transmission of data read-write operation of the distributed cache needs to be completed through a network. However, in the prior art, there is no technical scheme for loading data in a database by combining a local cache and a distributed cache.
Disclosure of Invention
The invention aims to provide a data synchronization method and a data synchronization system for a distributed memory cache, which are used for updating data stored in a local memory based on the distributed cache, so that the data synchronization is more convenient and efficient.
In order to achieve the purpose, the invention provides the following scheme:
the invention provides a data synchronization method of a distributed memory cache, which comprises the following steps:
acquiring the version number of a remote file and the version number of a local file; the remote file is a file located in a distributed cache; the local file is a file located in a local cache;
judging whether the version number of the remote file is consistent with the version number of the local file;
when the version number of the remote file is inconsistent with the version number of the local file, updating the version number of the local file according to the version number of a target file, and updating the data of the local file according to the data of the target file; the target file is a file located in a database.
Optionally, the obtaining of the version number of the remote file specifically includes:
inquiring the version number of the target file;
judging whether the version number of the target file is consistent with the version number of the remote file or not;
and when the version number of the target file is inconsistent with the version number of the remote file, updating the version number of the remote file to be the version number of the target file.
Optionally, the data synchronization method for the distributed memory cache further includes:
acquiring a data updating instruction;
and updating the data of the target file according to the data updating instruction, and updating the version number of the target file.
Optionally, the obtaining of the version number of the remote file and the version number of the local file specifically includes:
and acquiring the version number of the remote file and the version number of the local file in a synchronous or asynchronous mode.
The invention provides a data synchronization method of a distributed memory cache, which comprises the following steps:
acquiring the version number of a target file and the version number of a remote file; the target file is a file located in a database; the remote file is a file located in a distributed cache;
judging whether the version number of the target file is consistent with the version number of the remote file or not;
when the version number of the target file is inconsistent with the version number of the remote file, updating the version number of the remote file according to the version number of the target file;
updating the version number of the local file according to the updated version number of the remote file; the local file is a file located in a local cache;
and updating the data of the local file according to the version number of the updated local file and the data of the target file.
In order to achieve the purpose, the invention also provides the following technical scheme:
the invention provides a data synchronization system based on the data synchronization method of the distributed memory cache, which comprises the following steps:
the version number acquisition module is used for acquiring the version number of the remote file and the version number of the local file; the remote file is a file located in a distributed cache; the local file is a file located in a local cache;
the version number judging module is used for judging whether the version number of the remote file is consistent with the version number of the local file or not;
the first updating module is used for updating the version number of the local file according to the version number of the target file and updating the data of the local file according to the data of the target file when the version number of the remote file is inconsistent with the version number of the local file; the target file is a file located in a database.
Optionally, in terms of obtaining the version number of the remote file, the version number obtaining module specifically includes:
the version number query submodule is used for querying the version number of the target file;
the judging submodule is used for judging whether the version number of the target file is consistent with the version number of the remote file or not;
and the updating submodule is used for updating the version number of the remote file into the version number of the target file when the version number of the target file is inconsistent with the version number of the remote file.
Optionally, the data synchronization system of the distributed memory cache further includes:
the instruction acquisition module is used for acquiring a data updating instruction;
and the second updating module is used for updating the data of the target file according to the data updating instruction and updating the version number of the target file.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
when the version number of a file in the distributed cache is inconsistent with the version number of a file in the local cache, the data of the file in the database is indicated to be modified, and the version number of the file in the local cache and the data of the file in the local cache need to be further updated correspondingly. The scheme provided by the invention combines the distributed cache with the local cache, integrates the advantage of the access speed of the local cache and the advantage of large data storage amount of the distributed cache, and realizes the quick update and real-time synchronization of a large amount of data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a data synchronization method for a distributed memory cache according to a first embodiment of the present invention;
FIG. 2 is a schematic structural diagram illustrating an initialization step according to a first embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a synchronous mode data query according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an asynchronous mode data query according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating a data synchronization method for a distributed memory cache according to a second embodiment of the present invention;
FIG. 6 is a schematic structural diagram illustrating a synchronous update mode according to a second embodiment of the present invention;
FIG. 7 is a block diagram illustrating an asynchronous update mode according to a second embodiment of the present invention;
FIG. 8 is a schematic structural diagram illustrating a timing update mode according to a second embodiment of the present invention;
fig. 9 is a schematic structural diagram of a data synchronization system of a distributed memory cache according to a third embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present invention will be described in further detail with reference to the accompanying drawings and detailed description, in order to make the objects, features and advantages thereof more comprehensible.
Example one
As shown in fig. 1, the present embodiment provides a data synchronization method for a distributed memory cache, including:
Specifically, the obtaining of the version number of the remote file specifically includes:
1) and querying the version number of the target file.
2) And judging whether the version number of the target file is consistent with the version number of the remote file.
3) And when the version number of the target file is inconsistent with the version number of the remote file, updating the version number of the remote file to be the version number of the target file.
And if the version number of the remote file is consistent with the version number of the local file, controlling the local cache not to perform data synchronization.
In a specific embodiment, before synchronizing data, all file data are initialized, as shown in fig. 2, the initializing step specifically includes:
1) and performing initial caching on the local cache.
2) And filtering repeated files of the file data in the database, and correspondingly deleting the version number of the deleted target file in the database.
3) In the distributed cache, based on the connection between the distributed cache and the database, the file version with the timestamp as the latest timestamp is determined, and the corresponding version number is determined as the version number of the remote file.
4) And recording the version of the timestamp in the local cache as the latest timestamp, and determining the corresponding version number as the version number of the local file.
5) And loading the data in the local file to a cache, and constructing a file index.
Preferably, the version number of the remote file and the version number of the local file are acquired in a synchronous or asynchronous manner, and further query of corresponding data is performed.
As shown in fig. 3, for the synchronous mode data query, the specific steps of the data query include:
1) and after the synchronous data query command is acquired, acquiring the version number of the remote file in the distributed cache and acquiring the version number of the local file in the local cache.
2) And comparing and judging whether the version number of the remote file is consistent with the version number of the local file.
3) And when the version number of the remote file is consistent with that of the local file, directly calling the data to be inquired from the local cache.
4) And when the version number of the remote file is inconsistent with the version number of the local file, inquiring the target file corresponding to the local file in the database, and determining the latest version number of the target file (which is higher than the version number of the local file).
5) In the distributed cache, the version number of the remote file is updated according to the latest version number of the target file, and the version number of the local file in the local cache is further updated.
6) And correspondingly calling corresponding file data in the database according to the version number of the updated local file, and outputting the file data as a query result.
As shown in fig. 4, for the asynchronous mode data query, the asynchronous mode data query is different from the synchronous mode data query in that: after the front-end application inputs the first asynchronous data query command, the local cache and the distributed cache can continue to input the second asynchronous data query command while performing data query. The input of different query commands and the response process to different query commands do not influence each other.
Further, the data synchronization method for the distributed memory cache further includes: acquiring a data updating instruction; and updating the data of the target file according to the data updating instruction, and updating the version number of the target file.
Example two
As shown in fig. 5, the present embodiment provides a data synchronization system for a distributed memory cache, including:
Specifically, in this embodiment, when updating data, a synchronous update mode and/or an asynchronous update module may be adopted, as shown in fig. 6, in the synchronous update module, the data synchronization step specifically includes:
1) after a data synchronization updating instruction input by the front-end application is acquired, file data in the database is updated according to the updating instruction, and version numbers corresponding to files are updated at the same time.
2) And in the distributed cache, acquiring and updating the version number of the remote file.
3) And acquiring the current version number of the local file in the local cache, and updating the version number of the local file according to the updated version number of the remote file.
4) And according to the version number of the updated local file, corresponding file data is called from the database and is updated into data stored in the local file.
As shown in fig. 7, the asynchronous update mode is different from the synchronous update mode in that: after the front-end application inputs an asynchronous data synchronization command, file data in the database are updated on one hand, and the version number of a remote file is acquired and updated in the distributed cache on the other hand.
As shown in fig. 8, the timing update mode is different from the synchronous update mode in that: after the step 1), judging whether the version number of the target file of the database is consistent with the version number of the remote file of the distributed cache, and if not, acquiring and updating the remote version number.
EXAMPLE III
As shown in fig. 9, the present embodiment provides a data synchronization system based on the data synchronization method of the distributed memory cache of the first embodiment, including:
a version number obtaining module 101, configured to obtain a version number of a remote file and a version number of a local file; the remote file is a file located in a distributed cache; the local file is a file located in a local cache.
A version number determining module 201, configured to determine whether the version number of the remote file is consistent with the version number of the local file.
A first updating module 301, configured to update the version number of the local file according to the version number of the target file and update the data of the local file according to the data of the target file when the version number of the remote file is inconsistent with the version number of the local file; the target file is a file located in a database.
Specifically, in the aspect of acquiring the version number of the remote file, the version number acquiring module specifically includes:
and the version number query submodule is used for querying the version number of the target file.
And the judging submodule is used for judging whether the version number of the target file is consistent with the version number of the remote file.
And the updating submodule is used for updating the version number of the remote file into the version number of the target file when the version number of the target file is inconsistent with the version number of the remote file.
Further, the data synchronization system of the distributed memory cache further includes:
and the instruction acquisition module is used for acquiring a data updating instruction.
And the second updating module is used for updating the data of the target file according to the data updating instruction and updating the version number of the target file.
Preferably, the data synchronization system of the distributed memory cache provided by this embodiment can be applied to a plurality of servers or computer instances, so as to update data stored in the local memory based on the distributed cache, and achieve the purpose of data synchronization more conveniently and efficiently.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. A data synchronization method of a distributed memory cache is characterized in that the data synchronization method of the distributed memory cache comprises the following steps:
acquiring the version number of a remote file and the version number of a local file; the remote file is a file located in a distributed cache; the local file is a file located in a local cache;
judging whether the version number of the remote file is consistent with the version number of the local file;
when the version number of the remote file is inconsistent with the version number of the local file, updating the version number of the local file according to the version number of a target file, and updating the data of the local file according to the data of the target file; the target file is a file located in a database.
2. The data synchronization method for the distributed memory cache according to claim 1, wherein the obtaining of the version number of the remote file specifically includes:
inquiring the version number of the target file;
judging whether the version number of the target file is consistent with the version number of the remote file or not;
and when the version number of the target file is inconsistent with the version number of the remote file, updating the version number of the remote file to be the version number of the target file.
3. The data synchronization method for the distributed memory caches according to claim 1, further comprising:
acquiring a data updating instruction;
and updating the data of the target file according to the data updating instruction, and updating the version number of the target file.
4. The data synchronization method for the distributed memory cache according to claim 1, wherein the obtaining of the version number of the remote file and the version number of the local file specifically includes:
and acquiring the version number of the remote file and the version number of the local file in a synchronous or asynchronous mode.
5. A data synchronization method of a distributed memory cache is characterized in that the data synchronization method of the distributed memory cache comprises the following steps:
acquiring the version number of a target file and the version number of a remote file; the target file is a file located in a database; the remote file is a file located in a distributed cache;
judging whether the version number of the target file is consistent with the version number of the remote file or not;
when the version number of the target file is inconsistent with the version number of the remote file, updating the version number of the remote file according to the version number of the target file;
updating the version number of the local file according to the updated version number of the remote file; the local file is a file located in a local cache;
and updating the data of the local file according to the version number of the updated local file and the data of the target file.
6. A data synchronization system based on the data synchronization method of the distributed memory cache according to any one of claims 1 to 5, wherein the data synchronization system comprises:
the version number acquisition module is used for acquiring the version number of the remote file and the version number of the local file; the remote file is a file located in a distributed cache; the local file is a file located in a local cache;
the version number judging module is used for judging whether the version number of the remote file is consistent with the version number of the local file or not;
the first updating module is used for updating the version number of the local file according to the version number of the target file and updating the data of the local file according to the data of the target file when the version number of the remote file is inconsistent with the version number of the local file; the target file is a file located in a database.
7. The data synchronization system of the distributed memory cache according to claim 6, wherein in acquiring the version number of the remote file, the version number acquiring module specifically includes:
the version number query submodule is used for querying the version number of the target file;
the judging submodule is used for judging whether the version number of the target file is consistent with the version number of the remote file or not;
and the updating submodule is used for updating the version number of the remote file into the version number of the target file when the version number of the target file is inconsistent with the version number of the remote file.
8. The data synchronization system for distributed memory caches as claimed in claim 6, further comprising:
the instruction acquisition module is used for acquiring a data updating instruction;
and the second updating module is used for updating the data of the target file according to the data updating instruction and updating the version number of the target file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210586332.7A CN114896344A (en) | 2022-05-26 | 2022-05-26 | Data synchronization method and system for distributed memory cache |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210586332.7A CN114896344A (en) | 2022-05-26 | 2022-05-26 | Data synchronization method and system for distributed memory cache |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114896344A true CN114896344A (en) | 2022-08-12 |
Family
ID=82726608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210586332.7A Pending CN114896344A (en) | 2022-05-26 | 2022-05-26 | Data synchronization method and system for distributed memory cache |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114896344A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102833352A (en) * | 2012-09-17 | 2012-12-19 | 深圳中兴网信科技有限公司 | Distributed cache management system and method for implementing distributed cache management |
| CN104537013A (en) * | 2014-12-18 | 2015-04-22 | 北京京东尚科信息技术有限公司 | Data processing method and device |
| US20150169497A1 (en) * | 2013-12-12 | 2015-06-18 | International Business Machines Corporation | Efficient Distributed Cache Consistency |
| CN108363764A (en) * | 2018-02-05 | 2018-08-03 | 山东地主网络科技创新有限公司 | A kind of distributed caching management system and method |
-
2022
- 2022-05-26 CN CN202210586332.7A patent/CN114896344A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102833352A (en) * | 2012-09-17 | 2012-12-19 | 深圳中兴网信科技有限公司 | Distributed cache management system and method for implementing distributed cache management |
| US20150169497A1 (en) * | 2013-12-12 | 2015-06-18 | International Business Machines Corporation | Efficient Distributed Cache Consistency |
| CN104537013A (en) * | 2014-12-18 | 2015-04-22 | 北京京东尚科信息技术有限公司 | Data processing method and device |
| CN108363764A (en) * | 2018-02-05 | 2018-08-03 | 山东地主网络科技创新有限公司 | A kind of distributed caching management system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107391628B (en) | Data synchronization method and device | |
| CN106874281B (en) | Method and device for realizing database read-write separation | |
| CN109471843B (en) | Metadata caching method, system and related device | |
| CN104092726A (en) | Automatic replacing method and apparatus of same-name file | |
| CN113064919B (en) | Data processing method, data storage system, computer device and storage medium | |
| CN108304144B (en) | Data writing-in and reading method and system, and data reading-writing system | |
| CN112000850B (en) | Method, device, system and equipment for processing data | |
| CN111858581B (en) | Paging query method and device, storage medium and electronic equipment | |
| CN112948025A (en) | Data loading method and device, storage medium, computing equipment and computing system | |
| CN114896344A (en) | Data synchronization method and system for distributed memory cache | |
| CN111767282A (en) | MongoDB-based storage system, data insertion method and storage medium | |
| JP2022550380A (en) | Date-switching data offload method, device, equipment and medium | |
| CN111949725A (en) | Data query method and device, electronic equipment and computer readable storage medium | |
| CN116049306A (en) | Data synchronization method, device, electronic equipment and readable storage medium | |
| CN114328738A (en) | Data processing method and device, electronic equipment and readable storage medium | |
| CN113312412A (en) | Statistical data acquisition method and device, electronic equipment and storage medium | |
| CN111399753B (en) | Method and device for writing pictures | |
| CN108376104B (en) | Node scheduling method and device and computer readable storage medium | |
| CN111966460A (en) | Method, system, device and medium for optimizing display process information in container | |
| JP2005234945A (en) | Concurrent execution control method and device | |
| CN112988708A (en) | Version updating method and device, computer readable storage medium and processor | |
| CN113420088B (en) | Method, device and terminal for synchronizing cache database and database | |
| CN110798222A (en) | Data compression method and device | |
| CN111881085B (en) | Method and system for optimizing read-write bandwidth performance | |
| CN112783927B (en) | Database query method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |