CN116401317B - Cache data synchronization method and device, electronic equipment and medium - Google Patents
Cache data synchronization method and device, electronic equipment and medium Download PDFInfo
- Publication number
- CN116401317B CN116401317B CN202310424213.6A CN202310424213A CN116401317B CN 116401317 B CN116401317 B CN 116401317B CN 202310424213 A CN202310424213 A CN 202310424213A CN 116401317 B CN116401317 B CN 116401317B
- Authority
- CN
- China
- Prior art keywords
- cache
- information
- synchronized
- synchronization
- updating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 295
- 230000008569 process Effects 0.000 claims abstract description 213
- 230000001360 synchronised effect Effects 0.000 claims abstract description 121
- 230000004048 modification Effects 0.000 claims description 19
- 238000012986 modification Methods 0.000 claims description 19
- 238000012544 monitoring process Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 4
- 230000003139 buffering effect Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 3
- 230000008859 change Effects 0.000 description 7
- 239000000178 monomer Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 230000009466 transformation Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a cache data synchronization method, a device, electronic equipment and a medium, wherein the method comprises the following steps: starting a plurality of processes of a target application; wherein, each process runs a buffer synchronization tool; determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process; generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation; the process to be synchronized receives the cache synchronization instruction, and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction, so that the synchronization of the caches among a plurality of processes is realized, the cache of a processing process level is not required to be stripped, and the code of query logic is not required to be modified.
Description
Technical Field
The present application relates to the field of computer data processing technologies, and in particular, to a method, an apparatus, an electronic device, and a medium for synchronizing cache data.
Background
With the expansion of market scale, the original monomer application cannot bear increasing user access, so that the original monomer application needs to be modified with high availability, and the horizontal expansion of the project is realized in a technical mode, wherein the main difficulty is how to process-level cache of the monomer project is modified. If the caches of the multiple processes of the application are not synchronized, the caches among the different processes are inconsistent after horizontal expansion, which can cause serious production problems. Therefore, the caches among the processes need to be synchronized to achieve the cache consistency among the nodes of the horizontal extension. At present, the method for solving the synchronization of the inter-process caches in the market generally comprises the steps of independently separating the caches from a plurality of processes, wherein the processes share the same cache, but the method has higher transformation cost and longer transformation time and needs to change a plurality of codes to realize.
Disclosure of Invention
Accordingly, the present application is directed to a method, an apparatus, an electronic device, and a medium for synchronizing cache data, which can reconstruct a single application at low cost and low code, and realize synchronization of caches among multiple processes of the reconstructed application.
The embodiment of the application provides a cache data synchronization method, which comprises the following steps:
starting a plurality of processes of a target application; wherein, each process runs a buffer synchronization tool;
determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process;
generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation;
and the process to be synchronized receives the cache synchronization instruction and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
In some embodiments, in the method for synchronizing cache data, the determining a cache update process for updating the cache information includes:
monitoring the cache of each process by a cache synchronization tool running in each process, and judging whether the cache information of the process is updated or not;
if yes, determining the process as a cache updating process.
In some embodiments, in the method for synchronizing cache data, the cache synchronization tool of the cache update process generates a cache synchronization instruction carrying the information to be synchronized, including:
the cache synchronization tool of the cache update process acquires the information to be synchronized of the cache update process;
and converting the information to be synchronized into the cache synchronization instruction.
In some embodiments, in the method for synchronizing cache data, the determining information to be synchronized updated by the cache update process includes:
determining a cache ID updated by the cache updating process, corresponding update information of the cache ID and an update type;
and determining the information to be synchronized based on the cache ID, the update information corresponding to the cache ID and the update type.
In some embodiments, in the method for synchronizing cache data, the synchronizing the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction includes:
the cache synchronization tool of each process to be synchronized analyzes the information to be synchronized based on the received cache synchronization instruction;
determining target cache entries in the cache according to the cache ID in the information to be synchronized;
and updating the information of the target cache entry according to the update information and the update type corresponding to the cache ID.
In some embodiments, in the cache data synchronization method, updating information of a target cache entry according to update information and update type corresponding to a cache ID includes:
determining a target updating mode of a target cache entry according to the updating type corresponding to the cache ID; the target updating mode comprises writing, deleting and covering;
and updating the information of the target cache entry based on the target updating mode and the corresponding updating information of the cache ID.
In some embodiments, in the method for synchronizing cached data, before starting the plurality of processes of the target application, the method further includes:
responding to a modification instruction aiming at the code of the target application, and inserting a cache synchronization tool into the code of the target application to obtain the modified code of the target application;
based on the modified code of the target application, a plurality of processes of the target application are started.
In some embodiments, there is also provided an apparatus for inter-process cache synchronization, the apparatus comprising:
the starting module is used for starting a plurality of processes of the target application; wherein, each process runs a buffer synchronization tool;
the determining module is used for determining a cache updating process of cache information updating and determining information to be synchronized updated by the cache updating process;
the generation module is used for generating a cache synchronization instruction carrying the information to be synchronized through a cache synchronization tool of a cache updating process, and sending the cache synchronization instruction to a pre-deployed message queue so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation;
and the synchronization module is used for enabling the process to be synchronized to receive the cache synchronization instruction and synchronizing the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
In some embodiments, there is also provided an electronic device comprising: the system comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory are communicated through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the cache data synchronization method.
In some embodiments, a computer readable storage medium is also provided, on which a computer program is stored, which computer program, when being executed by a processor, performs the steps of the cache data synchronization method.
The application provides a cache data synchronization method, a device, electronic equipment and a medium, which can run a cache synchronization tool in each process after a plurality of processes of a target application are started; determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process; generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation; the process to be synchronized receives the cache synchronization instruction and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction; therefore, the buffer synchronization tool and the message queue in the process realize buffer synchronization among the processes, so that buffer synchronization among the processes can be realized, too many codes are not required to be modified for target application, the modification cost is low, the modification time is short, and the method is suitable for more scenes.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for synchronizing cache data according to an embodiment of the present application;
FIG. 2 is a flowchart of a method for determining a cache update procedure for updating cache information according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating a method for generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process according to an embodiment of the present application;
fig. 4 is a flowchart of a method for synchronizing a buffer of each process to be synchronized based on information to be synchronized carried in the buffer synchronization instruction according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of an apparatus for inter-process cache synchronization according to an embodiment of the present application;
fig. 6 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for the purpose of illustration and description only and are not intended to limit the scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.
In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.
It should be noted that the term "comprising" will be used in embodiments of the application to indicate the presence of the features stated hereafter, but not to exclude the addition of other features.
With the expansion of market scale, the original monomer application cannot bear increasing user access, so that the original monomer application needs to be modified with high availability, and the horizontal expansion of the project is realized in a technical mode, wherein the main difficulty is how to process-level cache of the monomer project is modified. If the caches of the multiple processes of the application are not synchronized, the caches among the different processes are inconsistent after horizontal expansion, which can cause serious production problems. Therefore, the caches among the processes need to be synchronized to achieve the cache consistency among the nodes of the horizontal extension. At present, the method for solving the synchronization of the inter-process caches in the market generally comprises the steps of independently separating the caches from a plurality of processes, wherein the processes share the same cache, but the method has higher transformation cost and longer transformation time and needs to change a plurality of codes to realize.
Based on the above, the application provides a cache data synchronization method, a device, an electronic device and a medium, which can run a cache synchronization tool in each process after starting a plurality of processes of a target application; determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process; generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation; the process to be synchronized receives the cache synchronization instruction and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction; therefore, the buffer synchronization tool and the message queue in the process realize buffer synchronization among the processes, so that buffer synchronization among the processes can be realized, too many codes are not required to be modified for target application, the modification cost is low, the modification time is short, and the method is suitable for more scenes.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for synchronizing cache data according to an embodiment of the application; the method comprises the following steps S101-S104;
s101, starting a plurality of processes of a target application; wherein, each process runs a buffer synchronization tool;
s102, determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process;
s103, a cache synchronization tool of a cache updating process generates a cache synchronization instruction carrying the information to be synchronized, and sends the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation;
and S104, the process to be synchronized receives the cache synchronization instruction, and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
The cache data synchronization method can run a cache synchronization tool in each process after starting a plurality of processes of the target application; determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process; generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation; the process to be synchronized receives the cache synchronization instruction and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction; therefore, the buffer synchronization tool and the message queue in the process realize buffer synchronization among the processes, so that buffer synchronization among the processes can be realized, too many codes are not required to be modified for target application, the modification cost is low, the modification time is short, and the method is suitable for more scenes.
In the embodiment of the present application, the method for synchronizing cache data may be operated in a terminal device or a server; the terminal device may be a local terminal device.
Specifically, taking a server as an example, when the cache data synchronization method is run on the server, the cache data synchronization method is used for realizing synchronization of inter-process caches.
In the step S101, a plurality of processes of a target application are started; wherein, a buffer synchronization tool is operated in each process.
Specifically, before starting the multiple processes of the target application, the method further includes:
responding to a modification instruction aiming at the code of the target application, and inserting a cache synchronization tool into the code of the target application to obtain the modified code of the target application;
based on the modified code of the target application, a plurality of processes of the target application are started.
In the embodiment of the application, the cache synchronization tool comprises a monitoring code, an updating code and an interaction code, and the change of the cache is monitored by running the monitoring code; updating the changed cache by running an update code; interaction with the message queue is achieved by running interaction code.
That is, the code of the target application does not need to be modified in a large amount, but the code of the cache synchronization tool is inserted into the code of the target application, so that a cache synchronization tool is operated in each process, and the inter-process cache synchronization is realized through the cache synchronization tool.
The starting of multiple instances is to horizontally expand a single application to be modified, which contains multiple inter-process caches, namely, a target application simultaneously starts multiple processes, and each process runs independently. For example, in a single user management system, a process level cache exists in a user query service, so that user information can be conveniently and quickly queried, but when a plurality of processes are started, inconsistency exists in each process cache. Here, a plurality of user inquiry service processes are started.
In the embodiment of the present application, the described target application is only illustrative, and may be a service node, a micro-service, a web-monolithic application, a common monolithic application, a logically partitioned module, or the like.
In the step S102, a cache update process for updating the cache information is determined, and information to be synchronized updated by the cache update process is determined.
Specifically, referring to fig. 2, the process of determining the update of the cache information includes the following steps S201 to S202;
s201, monitoring the cache of each process by a cache synchronization tool running in each process, and judging whether the cache information of the process is updated or not;
s202, if yes, determining the process as a cache updating process.
That is, each process monitors its own inter-process cache changes. In the user management system according to the embodiment of the present application, each user needs to monitor its own process level cache in order to query the process of the service, and specifically, may use design modes such as an observer mode to implement the monitoring.
The determining the information to be synchronized updated by the cache updating process comprises the following steps:
determining a cache ID updated by the cache updating process, corresponding update information of the cache ID and an update type;
and determining the information to be synchronized based on the cache ID, the update information corresponding to the cache ID and the update type.
The process corresponds to a plurality of cache entries, such as Zhang three, lifour, wang five, etc.; the cache ID is used to determine a cache entry. For example, it is determined that the buffer information of Zhang three is modified,
and the cache ID corresponds to update information, namely the change content of the cache entry corresponding to the cache ID. For example, zhang San, age 19; the modification is as follows: zhang three, age 20.
The update type, that is, the update mode, includes: adding, deleting and modifying; specifically, in order to add cache information, delete cache information, and modify original information.
In step S103, the cache synchronization tool of the cache update process generates a cache synchronization instruction carrying the information to be synchronized, and sends the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relationship.
Here, at the time of starting up the plurality of processes of the target application, a subscription relationship between the plurality of processes and the message queue is established. The message queue may be disposed on the same server or on another server.
In the embodiment of the present application, the types of the message queues are not limited, and the message queues such as ActiveMQ, rabbitMQ, kafka, rocketMQ can support the method described in the embodiment of the present application, and the function of the message queues is that of a core synchronizer for implementing the method of the present application. In an example subscriber management system, rabbitMQ may be used as a core synchronizer because the response time of RabbitMQ is relatively faster than other centralized message queues.
After the message queue is started, a plurality of processes started before are subscribed to the message queue, and illustratively, an ActiveMQ can use a topic mode, a RabbitMQ can use a fanout mode, a Kafka can use a topic mode, and a RocketMQ can use a topic mode. In an exemplary subscriber management system, each subscriber query process may be subscribed to a different message queue of the same switch of RabbitMQ (the switch uses fanout mode to send messages so that each message sent to the switch is sent to all message queues under the switch at the same time.
In the method according to the embodiment of the present application, the mentioned several common message queues are only used for illustration, and other message queues may also meet the requirements of the method.
Referring to fig. 3, the cache synchronization tool of the cache update process generates a cache synchronization instruction carrying the information to be synchronized, which includes steps S301-S302;
s301, a cache synchronization tool of the cache update process acquires the information to be synchronized of the cache update process;
s302, converting the information to be synchronized into the cache synchronization instruction.
Here, the information to be synchronized includes a cache ID, update information corresponding to the cache ID, and an update type; the update information includes attributes and attribute values.
That is, the cache synchronization instruction carries the cache ID, the update information corresponding to the cache ID, the update type, and other important information.
After receiving the buffer synchronization instruction, the message queue sends the buffer synchronization instruction to each process subscribing the message queue, so that the buffer synchronization instruction is sent to each process to be synchronized.
In step S104, the process to be synchronized receives the cache synchronization instruction, and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
Specifically, referring to fig. 4, the step of synchronizing the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction includes the following steps S401 to S403;
s401, analyzing the information to be synchronized by a cache synchronizing tool of each process to be synchronized based on the received cache synchronizing instruction;
s402, determining a target cache entry in the cache according to the cache ID in the information to be synchronized;
s403, updating the information of the target cache entry according to the update information and the update type corresponding to the cache ID.
That is, the buffer synchronization instruction sent from the message queue is received to be synchronized, and the buffer ID, the update information, the update type and other important information to be updated are acquired by analyzing the instruction. In this example user management system, each user inquiry process subscribes to the message queue, receives the cache synchronization instruction sent by the message queue, and parses important information such as ID corresponding to user information to be updated, update information, and the like from the instruction, for example, parses the cache ID to be Zhang three, updates the type to be modified, and updates the update information to be age 18 to be age 19.
After the process determines the target cache entry in the cache according to the cache ID, the process realizes the update of the information of the target cache entry according to the update information and the update type corresponding to the cache ID, and each process achieves the cache synchronization. In the user management system of this example, each user inquiry service updates its own process level cache according to the cache change content analyzed from the cache synchronization instruction, so as to achieve the final objective of cache synchronization.
Updating the information of the target cache entry according to the update information and the update type corresponding to the cache ID, wherein the update information comprises the following steps:
determining a target updating mode of a target cache entry according to the updating type corresponding to the cache ID; the target updating mode comprises writing, deleting and covering;
and updating the information of the target cache entry based on the target updating mode and the corresponding updating information of the cache ID.
Differentiating the update types of addition, deletion and modification, and carrying out different update treatments aiming at different update types when the process-level cache data are synchronous: when the update type is increased, writing the update mode, specifically writing the KEY and the VALUE of the update information into a cache of the process; when the update type is modified, the update mode is coverage, and specifically, the KEY and the VALUE of the update information are covered on the existing data in the process cache; when the update type is delete, the update mode is delete, specifically delete the KEY existing in the cache of the process.
In the prior art, other cache synchronization modes mostly need to strip off the cache of a process level, and relate to code modification of query logic, code modification of storage logic, code modification of cache logic, call logic code addition of cache middleware and the like, and the code modification is larger. The method only needs to add codes of a cache synchronization tool, specifically, the cache synchronization tool comprises monitoring codes, and the change of the cache is monitored by running the monitoring codes; the cache synchronization tool comprises an update code, and the changed cache is updated by running the update code; the cache synchronization tool comprises interaction codes, interaction with the message queue is achieved, and the code change is small.
After the process-level caches are stripped off in the prior art, new cache middleware such as Redis is introduced, the original process-level cache query is changed into remote call middleware for query, and the query efficiency is reduced.
Finally, the method is suitable for a plurality of scenes, such as scenes with requirements on software license, the cache middleware has only a few types of flexible index, and the message queue is richer than the cache middleware.
Based on the same inventive concept, the embodiment of the present application further provides an inter-process cache synchronization device corresponding to the cache data synchronization method, and since the principle of solving the problem of the device in the embodiment of the present application is similar to that of the cache data synchronization method in the embodiment of the present application, the implementation of the device may refer to the implementation of the method, and the repetition is omitted.
Referring to fig. 5, an embodiment of the present application provides an apparatus for inter-process cache synchronization, where the apparatus includes:
a starting module 501, configured to start a plurality of processes of a target application; wherein, each process runs a buffer synchronization tool;
a determining module 502, configured to determine a cache update process for updating cache information, and determine information to be synchronized updated by the cache update process;
a generating module 503, configured to generate a cache synchronization instruction carrying the information to be synchronized by using a cache synchronization tool of a cache update process, and send the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relationship;
and the synchronization module 504 is configured to enable the process to be synchronized to receive the cache synchronization instruction, and synchronize the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
The application provides an inter-process cache synchronization device, which can run a cache synchronization tool in each process after a plurality of processes of a target application are started; determining a cache updating process of cache information updating, and determining information to be synchronized updated by the cache updating process; generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation; the process to be synchronized receives the cache synchronization instruction and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction; therefore, the buffer synchronization tool and the message queue in the process realize buffer synchronization among the processes, so that buffer synchronization among the processes can be realized, too many codes are not required to be modified for target application, the modification cost is low, the modification time is short, and the method is suitable for more scenes.
In some embodiments, the device for synchronization of inter-process caches, when determining a cache update process for updating cache information, is specifically configured to:
monitoring the cache of each process by a cache synchronization tool running in each process, and judging whether the cache information of the process is updated or not;
if yes, determining the process as a cache updating process.
In some embodiments, the generating module is specifically configured to, when generating, by the cache synchronization tool of the cache update process, a cache synchronization instruction carrying the information to be synchronized:
the cache synchronization tool of the cache update process acquires the information to be synchronized of the cache update process;
and converting the information to be synchronized into the cache synchronization instruction.
In some embodiments, the determining module is configured to, when determining that the information to be synchronized updated by the cache update process is specifically:
determining a cache ID updated by the cache updating process, corresponding update information of the cache ID and an update type;
and determining the information to be synchronized based on the cache ID, the update information corresponding to the cache ID and the update type.
In some embodiments, the device for synchronization of inter-process caches is specifically configured to:
the cache synchronization tool of each process to be synchronized analyzes the information to be synchronized based on the received cache synchronization instruction;
determining target cache entries in the cache according to the cache ID in the information to be synchronized;
and updating the information of the target cache entry according to the update information and the update type corresponding to the cache ID.
In some embodiments, the synchronization module updates information of the target cache entry according to the update information and the update type corresponding to the cache ID, and is specifically configured to:
determining a target updating mode of a target cache entry according to the updating type corresponding to the cache ID; the target updating mode comprises writing, deleting and covering;
and updating the information of the target cache entry based on the target updating mode and the corresponding updating information of the cache ID.
In some embodiments, the device for inter-process cache synchronization further includes an inserting module, configured to, before starting a plurality of processes of the target application, insert a cache synchronization tool into a code of the target application in response to a modification instruction for the code of the target application, to obtain a modified code of the target application;
based on the modified code of the target application, a plurality of processes of the target application are started.
Based on the same inventive concept, the embodiment of the present application further provides an electronic device corresponding to the method for synchronizing cached data, and since the principle of solving the problem of the electronic device in the embodiment of the present application is similar to that of the method for synchronizing cached data described in the embodiment of the present application, implementation of the electronic device may refer to implementation of the method, and repeated parts are not repeated.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the application; specifically, the electronic device 600 includes: a processor 602, a memory 601 and a bus, said memory 601 storing machine readable instructions executable by said processor 602, said processor 602 and said memory 601 communicating via the bus when the electronic device 600 is running, said machine readable instructions when executed by said processor 602 performing the steps of said cache data synchronization method.
Based on the same inventive concept, the embodiment of the present application further provides an electronic device corresponding to the method for synchronizing cached data, and since the principle of solving the problem of the electronic device in the embodiment of the present application is similar to that of the method for synchronizing cached data described in the embodiment of the present application, implementation of the electronic device may refer to implementation of the method, and repeated parts are not repeated.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the cache data synchronization method.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the method embodiments, and are not repeated in the present disclosure. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, and for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, indirect coupling or communication connection of devices or modules, electrical, mechanical, or other form.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily appreciate variations or alternatives within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (8)
1. A method for synchronizing cached data, the method comprising:
responding to a modification instruction aiming at the code of the target application, and inserting a cache synchronization tool into the code of the target application to obtain the modified code of the target application;
starting a plurality of processes of the target application based on the modified code of the target application; wherein, each process runs a buffer synchronization tool;
monitoring the cache of each process by a cache synchronization tool running in each process, and judging whether the cache information of the process is updated or not;
if yes, determining the process as a cache updating process, and determining information to be synchronized updated by the cache updating process;
generating a cache synchronization instruction carrying the information to be synchronized by a cache synchronization tool of a cache update process, and sending the cache synchronization instruction to a pre-deployed message queue, so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation;
and the process to be synchronized receives the cache synchronization instruction and synchronizes the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
2. The method for synchronizing cache data according to claim 1, wherein the cache synchronizing means of the cache update process generates a cache synchronizing instruction carrying the information to be synchronized, comprising:
the cache synchronization tool of the cache update process acquires the information to be synchronized of the cache update process;
and converting the information to be synchronized into the cache synchronization instruction.
3. The method for synchronizing cache data according to claim 1, wherein the determining the information to be synchronized updated by the cache update process includes:
determining a cache ID updated by the cache updating process, corresponding update information of the cache ID and an update type;
and determining the information to be synchronized based on the cache ID, the update information corresponding to the cache ID and the update type.
4. The method for synchronizing buffered data according to claim 3, wherein said synchronizing the buffering of each process to be synchronized based on the information to be synchronized carried in the buffered synchronization instruction comprises:
the cache synchronization tool of each process to be synchronized analyzes the information to be synchronized based on the received cache synchronization instruction;
determining target cache entries in the cache according to the cache ID in the information to be synchronized;
and updating the information of the target cache entry according to the update information and the update type corresponding to the cache ID.
5. The cache data synchronization method according to claim 4, wherein updating the information of the target cache entry according to the update information and the update type corresponding to the cache ID comprises:
determining a target updating mode of a target cache entry according to the updating type corresponding to the cache ID; the target updating mode comprises writing, deleting and covering;
and updating the information of the target cache entry based on the target updating mode and the corresponding updating information of the cache ID.
6. An apparatus for interprocess cache synchronization, the apparatus comprising:
the starting module is used for responding to a modification instruction of the code of the target application, inserting a cache synchronization tool into the code of the target application, and obtaining the modified code of the target application; starting a plurality of processes of the target application based on the modified code of the target application; wherein, each process runs a buffer synchronization tool;
the determining module is used for monitoring the cache of each process through a cache synchronizing tool running in each process and judging whether the cache information of the process is updated or not; if yes, determining the process as a cache updating process, and determining information to be synchronized updated by the cache updating process;
the generation module is used for generating a cache synchronization instruction carrying the information to be synchronized through a cache synchronization tool of a cache updating process, and sending the cache synchronization instruction to a pre-deployed message queue so that the message queue sends the cache synchronization instruction to each process to be synchronized based on a pre-established subscription relation;
and the synchronization module is used for enabling the process to be synchronized to receive the cache synchronization instruction and synchronizing the cache of each process to be synchronized based on the information to be synchronized carried in the cache synchronization instruction.
7. An electronic device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating over the bus when the electronic device is running, said machine readable instructions when executed by said processor performing the steps of the cache data synchronization method according to any one of claims 1 to 5.
8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the cache data synchronization method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310424213.6A CN116401317B (en) | 2023-04-19 | 2023-04-19 | Cache data synchronization method and device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310424213.6A CN116401317B (en) | 2023-04-19 | 2023-04-19 | Cache data synchronization method and device, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116401317A CN116401317A (en) | 2023-07-07 |
| CN116401317B true CN116401317B (en) | 2023-11-17 |
Family
ID=87019709
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310424213.6A Active CN116401317B (en) | 2023-04-19 | 2023-04-19 | Cache data synchronization method and device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116401317B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105700962A (en) * | 2014-11-26 | 2016-06-22 | 中兴通讯股份有限公司 | Data update processing method and apparatus |
| CN106230978A (en) * | 2016-08-31 | 2016-12-14 | 广州市乐商软件科技有限公司 | Distributed caching synchronous method, Apparatus and system |
| CN108200219A (en) * | 2018-03-13 | 2018-06-22 | 广东欧珀移动通信有限公司 | Method of data synchronization, device, server and storage medium |
| CN111400408A (en) * | 2020-04-13 | 2020-07-10 | 上海东普信息科技有限公司 | Data synchronization method, device, equipment and storage medium |
| CN111563102A (en) * | 2020-04-10 | 2020-08-21 | 中国联合网络通信集团有限公司 | Cache updating method, server, system and storage medium |
| CN111782419A (en) * | 2020-06-23 | 2020-10-16 | 北京青云科技股份有限公司 | Cache updating method, device, equipment and storage medium |
| CN113031864A (en) * | 2021-03-19 | 2021-06-25 | 上海众源网络有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113051091A (en) * | 2021-04-30 | 2021-06-29 | 中国银行股份有限公司 | Process-level cache data synchronization method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6340977B1 (en) * | 1999-05-07 | 2002-01-22 | Philip Lui | System and method for dynamic assistance in software applications using behavior and host application models |
-
2023
- 2023-04-19 CN CN202310424213.6A patent/CN116401317B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105700962A (en) * | 2014-11-26 | 2016-06-22 | 中兴通讯股份有限公司 | Data update processing method and apparatus |
| CN106230978A (en) * | 2016-08-31 | 2016-12-14 | 广州市乐商软件科技有限公司 | Distributed caching synchronous method, Apparatus and system |
| CN108200219A (en) * | 2018-03-13 | 2018-06-22 | 广东欧珀移动通信有限公司 | Method of data synchronization, device, server and storage medium |
| CN111563102A (en) * | 2020-04-10 | 2020-08-21 | 中国联合网络通信集团有限公司 | Cache updating method, server, system and storage medium |
| CN111400408A (en) * | 2020-04-13 | 2020-07-10 | 上海东普信息科技有限公司 | Data synchronization method, device, equipment and storage medium |
| CN111782419A (en) * | 2020-06-23 | 2020-10-16 | 北京青云科技股份有限公司 | Cache updating method, device, equipment and storage medium |
| CN113031864A (en) * | 2021-03-19 | 2021-06-25 | 上海众源网络有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113051091A (en) * | 2021-04-30 | 2021-06-29 | 中国银行股份有限公司 | Process-level cache data synchronization method and device |
Non-Patent Citations (2)
| Title |
|---|
| A fast and resource-conscious MPI message queue mechanism for large-scale jobs;Judicael A. Zounmevo et al.;《Future Generation Computer Systems》;265-290 * |
| 基于RapidIO总线的进程间通信系统的研究与实现;潘灵;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》;C031-36 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116401317A (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2948875B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN109032796B (en) | Data processing method and device | |
| CN107040576A (en) | Information-pushing method and device, communication system | |
| CN113885780A (en) | Data synchronization method, device, electronic equipment, system and storage medium | |
| US10938633B2 (en) | Facilitating resilient and fault tolerant asynchronous messaging | |
| CN115934414A (en) | Data backup method, data recovery method, device, equipment and storage medium | |
| CN116401317B (en) | Cache data synchronization method and device, electronic equipment and medium | |
| CN112363815A (en) | Redis cluster processing method and device, electronic equipment and computer readable storage medium | |
| CN112000850A (en) | Method, device, system and equipment for data processing | |
| CN109165259B (en) | Index table updating method based on network attached storage, processor and storage device | |
| CN117331576A (en) | Application pre-download method, device, computer and storage medium | |
| CN111177109A (en) | Method and device for deleting overdue key | |
| US11386043B2 (en) | Method, device, and computer program product for managing snapshot in application environment | |
| CN115238006A (en) | Retrieval data synchronization method, device, equipment and computer storage medium | |
| CN110661857B (en) | Data synchronization method and device | |
| CN114945026A (en) | Data processing method, device and system | |
| CN114500416A (en) | Delivery method and delivery system for at most one message delivery | |
| CN113742376A (en) | Data synchronization method, first server and data synchronization system | |
| CN108984431B (en) | Method and apparatus for flushing stale caches | |
| CN111061719A (en) | Data collection method, device, equipment and storage medium | |
| CN111459981A (en) | Query task processing method, device, server and system | |
| CN113364864B (en) | Server data synchronization method, system and storage medium | |
| CN113297002B (en) | Database working mode switching method and device | |
| CN117950850A (en) | Data transmission method, device, electronic equipment and computer readable medium | |
| CN115396285A (en) | Performance monitoring method and system for message middleware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |