CN110413418B - Cache synchronization device and method, cache synchronization system and electronic equipment - Google Patents
Cache synchronization device and method, cache synchronization system and electronic equipment Download PDFInfo
- Publication number
- CN110413418B CN110413418B CN201910557298.9A CN201910557298A CN110413418B CN 110413418 B CN110413418 B CN 110413418B CN 201910557298 A CN201910557298 A CN 201910557298A CN 110413418 B CN110413418 B CN 110413418B
- Authority
- CN
- China
- Prior art keywords
- cache
- local
- local cache
- target application
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a cache synchronization device, which belongs to the technical field of data processing, and the cache synchronization device disclosed by the embodiment of the application is used for packaging various open source caches by arranging the synchronization cache device in a server where an application is located and calling the open source caches based on a generic type; when the local cache is operated by the synchronous cache device to cause the change of the local cache, the local cache change information is synchronized to other nodes which are registered in advance by the application through a built-in cache updating management module so as to carry out data synchronization; and the synchronous cache device receives the message that the cache data of other nodes are changed through the cache updating management module and updates the local cache. The cache synchronization device disclosed by the embodiment of the application does not need to carry out additional development and complex configuration on the server side, and can realize synchronization of local caches among different cache nodes only by installing the local cache synchronization device on the cache nodes, so that the development cost is saved, and the development efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a cache synchronization apparatus and method, a cache synchronization system, an electronic device, and a computer-readable storage medium.
Background
In the development process of the server-side application, the local cache of the server side is generally used. The local cache has the advantages of high reading speed, safety and stability. Compared with external distributed caches such as redis/memcache (an open source cache), the local cache directly reads a local memory, so that the processes of serialization and network input and output interfaces are avoided, and great help is brought to the improvement of system performance. Because the number of clients of the application is usually large, in order to improve the processing efficiency of the application, the access of the clients is usually mapped to different server sides, and therefore, the synchronization of the application data cached between the server sides is a key technology for ensuring the stability of the application program. In the prior art, an external message queue is usually introduced to solve the problem of local cache synchronization of different server ends of an application, and when a data source or a single server end node (i.e. a server) cache data is changed, a data change message is broadcasted to each server end node of a cluster through the message queue in a publish/subscribe mode, so as to inform each server of performing data synchronization updating.
When the data change message is broadcasted to each server of the cluster through the message queue in the publish/subscribe mode, the message queue configuration is needed at first, and the server end node needs to apply for the message queue theme. On the other hand, each server-end node needs to be configured as an independent consumer, and relevant consumer registration and consumption resolution logic is developed.
It can be seen that the local cache synchronization method in the prior art has at least the defect of high development and configuration cost.
Disclosure of Invention
The application provides a cache synchronization device, which is beneficial to reducing development and configuration cost of local cache synchronization of a server end, so that a business system can realize cache synchronization of the server end with zero development cost.
In order to solve the foregoing problem, in a first aspect, an embodiment of the present application provides a cache synchronization apparatus, including: a cache operation interface module, a cache registration management module, a cache operation management module and a cache updating module, wherein,
the cache operation interface module is used for responding to interface call of updating a local cache by a target application and sending the interface call of updating the local cache to the cache operation management module;
the cache operation management module is used for calling and mapping the interface for updating the local cache into a cache operation interface packaged in advance and calling the cache operation interface to update the local cache;
the cache operation interface module is also used for responding to interface call of updating the local cache by the target application and constructing local cache updating information;
the cache registration management module is configured to obtain a cache node list corresponding to the target application through a preset node registration center, where the cache node list includes an IP Address (Internet Protocol Address ) and a port of a node where a local cache is located, where the node is pre-registered by different service terminals of the target application;
and the cache updating module is used for acquiring the IP address and the port through the cache registration management module and broadcasting the local cache updating information to each node of the target application through the IP address and the port so as to enable each node of the target application to execute cache updating operation.
In a second aspect, an embodiment of the present application provides a cache synchronization method, including:
responding to interface call of a target application for updating a local cache, mapping the interface call for updating the local cache into a cache operation interface packaged in advance, and calling the cache operation interface to update the local cache;
responding to the interface call of the target application for updating the local cache, and constructing local cache updating information;
obtaining a cache node list corresponding to the target application through a preset node registration center, wherein the cache node list comprises IP addresses and ports of nodes where local caches are located and registered in advance by different service terminals of the target application;
broadcasting the local cache updating information to each node of the target application through the IP address and the port so as to enable each node of the target application to execute cache updating operation.
In a third aspect, an embodiment of the present application further discloses an electronic device, which includes a memory, a processor, and a computer program that is stored in the memory and is executable on the processor, and when the processor executes the computer program, the cache synchronization method according to the embodiment of the present application is implemented.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor, and the computer program includes steps of the cache synchronization method disclosed in the present application.
The cache synchronization device disclosed in the embodiment of the present application, by setting the synchronization cache device at the application server, the synchronization cache device further includes: the cache operation interface module is used for responding to interface calling of a target application for updating a local cache and sending the interface calling for updating the local cache to the cache operation management module; the cache operation management module is used for calling and mapping the interface for updating the local cache into a cache operation interface packaged in advance and calling the cache operation interface to update the local cache; the cache operation interface module is also used for responding to interface call of updating the local cache by the target application and constructing local cache updating information; the cache registration management module is used for acquiring a cache node list corresponding to the target application through a preset node registration center, wherein the cache node list comprises IP addresses and ports of nodes where local caches are located and registered in advance by different service terminals of the target application; the cache updating module is used for acquiring the IP address and the port through the cache registration management module, and broadcasting the local cache updating information to each node of the target application through the IP address and the port so as to enable each node of the target application to execute cache updating operation, and under the condition that additional development and complex configuration are not needed to be carried out on a server side, synchronization of local caches among different cache nodes can be realized only by installing a local cache synchronizing device on the cache nodes, development cost is saved, and development efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic structural diagram of a cache synchronization apparatus according to a first embodiment of the present application;
FIG. 2 is a schematic structural diagram of a cache synchronization system applied in an embodiment of the present application;
FIG. 3 is a diagram of an application system architecture to which embodiments of the present application are applied;
fig. 4 is a second schematic structural diagram of a cache synchronization apparatus according to a first embodiment of the present application;
fig. 5 is a schematic diagram of a specific structure of an application system of a cache synchronization apparatus according to a first embodiment of the present application;
fig. 6 is a flowchart of a cache synchronization method according to a second embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Example one
As shown in fig. 1, a cache synchronization apparatus disclosed in an embodiment of the present application includes: a cache operation interface module 110, a cache registration management module 120, a cache operation management module 130, and a cache update module 140.
Generally, the target application will include a plurality of servers, and the cache synchronization apparatus in the embodiment of the present application may be provided in the form of a component on each server of the target application. The server processes the request of the target application and locally caches part or all data of the target application. The cache synchronization device provides a packaged local cache operation interface for the target application to call. And the cache synchronization device further executes the operations of registration, query, update, deletion and the like of the local cache according to the encapsulated local cache operation interface called by the target application. In the process of executing the above operation, when the local cache changes, the cache synchronization device sends information that the local cache changes to the cache synchronization device on the other server of the target application through network communication, so that the other server of the target application realizes synchronous update of the local cache of the other server through the cache synchronization device arranged on the other server.
As shown in fig. 2, in some embodiments of the present application, the cache synchronization apparatus is applied in a cache synchronization system as shown in fig. 2, and the cache synchronization system includes: a plurality of nodes 210 of a target application, a node registry 220, and a cache synchronization apparatus 230 arranged on each node 210 in the embodiment of the present application, wherein the node 210 includes a local cache 2101, and the node 210 where the target application is located performs local cache update by the cache synchronization apparatus 230 arranged on the node; the node registry 220 is configured to send node change information of the target application to the cache synchronization apparatus according to the registration information of the node.
The following describes embodiments of the cache synchronization system and the cache synchronization device in detail with a specific application architecture of the cache synchronization device and the cache synchronization system. As shown in the application architecture of fig. 3, the target application includes 3 servers (i.e., nodes), which are respectively shown as server 310, server 320, and server 330 as an example, and the specific implementation of each module is described separately in connection with the application system architecture. As shown in fig. 3, the cache synchronization apparatuses are provided in the server 310, the server 320, and the server 330, respectively, and the cache synchronization apparatus provided in the server 310 is denoted as a cache synchronization apparatus 3101, the cache synchronization apparatus provided in the server 320 is denoted as a cache synchronization apparatus 3201, and the cache synchronization apparatus provided in the server 330 is denoted as a cache synchronization apparatus 3301. The servers 310, 320, and 330 have different servers (i.e., server-side software of the target application) 3102, 3202, and 3302 installed thereon at the same time. When processing data of a target application, a server of the target application typically reads the data from the database 340 and caches the data in the local caches of the servers 310 to 330. The node registry 350 is configured to complete node registration according to the registration information of the nodes (e.g., servers 310 to 330 in fig. 3) sent by the cache synchronizing apparatuses 3101, 3201 and 3301, and send change information of the nodes (i.e., servers 310 to 330) of the target application to the cache synchronizing apparatuses 3101, 3201 and 3301.
In some embodiments of the present application, ZooKeeper (an open source distributed coordination service) may be used as a node registry, that is, a registry used as an IP and a port of a server (that is, a node) in a cache synchronization system, and when there is an up/down line change in the server in the cache synchronization system, the cache synchronization apparatus of all other nodes may be notified.
In a specific application process, for example, a server of a target application on the server 310, the target application applies for a local cache from the server 310, and then registers the local cache in the cache synchronization device 3101 on the server 310. When an application starts using the local cache, the cache synchronization apparatus 3101 first registers the IP address and port of the server 310 with the node registry 350. Similarly, other servers 320 and 330 of the target application may also register with the node registry 350. When the node registry 350 detects that a server of the target application changes (for example, the server 330 cancels registration or adds registration information of other servers of the target application) during the operation of the application system, the server information of the target application is sent to the cache synchronizing apparatuses 3101 to 3103.
Further, the cache operation interface module 110 is configured to respond to a cache registration operation of registering a local cache by a target application, generate cache registration information according to the cache registration operation, and forward the cache registration information to the cache registration management module 120, where the cache registration information includes: and the cache registration operation registers the IP address and the port of the node where the local cache is located.
The cache registration management module 120 is configured to register the IP address and the port to the preset node registry (e.g., 350 in fig. 3) according to the received cache registration information.
The cache operation interface module 110 is configured to respond to an interface call for updating a local cache by a target application, and send the interface call for updating the local cache to the cache operation management module 130.
When a target application needs to modify data cached on the server 210, the target application may call an encapsulation interface provided by the cache synchronizer 2101 on the server 210, for example, call an interface to write a local cache. The cache operation interface module 110 responds to an interface call of writing to the local cache by the target application, and sends the interface call of writing to the local cache to the cache operation management module 130 to further execute the write operation of the local cache.
The cache operation management module 130 is configured to map the interface call for updating the local cache to a cache operation interface packaged in advance, and call the cache operation interface to update the local cache.
In specific implementation, the cache operation management module 130 encapsulates various caches, and provides an encapsulation interface for implementing the cache operation, where the encapsulation interface provided by the cache operation management module 130 includes: registration, query, update, deletion, and the like. The packaging interfaces are compatible with various local caches (some open source caches) such as HashMap, Guava cache, Caffeine, EhCache and the like through generic and reflective technologies.
In some embodiments of the present application, the step of mapping the interface call for updating the local cache to a cache operation interface packaged in advance and calling the cache operation interface to update the local cache includes: determining the cache type according to the received interface call for updating the local cache; mapping the interface call for updating the local cache to a corresponding cache operation interface call of the cache category by a reflection technology; and calling the cache operation interface to update the local cache.
Generally, before using the local cache, the target application needs to call the encapsulation interface provided by the cache operation management module 130 to register the local cache with the cache operation management module 130. When registering a local cache with the cache operation management module 130, information such as a local cache type (e.g., guava cache), a size, and an effective time needs to be provided. The cache operation management module 130 creates a cache name for the local cache according to the above registration information of the local cache, and manages the corresponding relationship between the created cache name and cache information such as cache type, size, valid time, and the like. For example, the target application registers the created guava cache instance in the local cache device by calling a cache registration interface of the local cache device, at this time, the cache operation management module 130 assigns a cache name to the registered local cache, and adds the guava cache instance to the management list of the cache operation management module 130.
When the target application calls the interfaces such as query and update provided by the cache operation management module 130, the cache name needs to be input, and the cache operation management module 130 determines the cache type (such as guava cache) of the target application operation according to the cache name. The cache operation management module 130 may determine the cache type of the interface call operation for updating the local cache according to the cache name transmitted when the interface is called and the correspondence between the cache name and the cache type created in advance. And then, mapping the interface call for updating the local cache to a corresponding cache operation interface of the cache type.
Further, the cache operation management module 130 sets a mapping method for each type of cache based on the reflection mechanism. The cache operation management module 130 maps calls of the package interfaces such as query, update, and delete to corresponding cache operation interfaces (such as a cache query interface, a cache update interface, and a cache delete interface) calls of the cache type (such as guava cache) through a reflection technology.
Then, the cache operation management module 130 calls a cache operation interface of the local cache through a reflection technique to implement a local cache operation (e.g., write to the local cache).
Because the data cached on server 310 by the target application may change after a write operation is performed on the local cache, a synchronous update of the data cached on server 320 and server 330 is required. The cache operation interface module 110 is further configured to respond to an interface call for updating the local cache by the target application, construct local cache update information, and send the local cache update information to the cache update module 140. Generally, the interface call for updating the local cache by the target application includes a cache name, a key (i.e. data information or address information) of the cache, and a change type (update or delete), and the cache operation interface module 110 constructs the local cache update information according to the information included in the interface call for updating the local cache. The local cache update information follows a TCP (Transmission Control Protocol) data Transmission Protocol, and includes information such as a cache name, a cache key (i.e., data information or address information), and a change type (update or delete). The interface call of the local cache usually includes cache name, operation type, data identification or cache location information. In some embodiments of the present application, a local cache update message may be constructed according to the above information carried in the interface call of the local cache, and the cache name, the location information, or the data information that needs to be updated is sent to the server 320 and the server 330, so as to update the cache data at the corresponding location.
The cache update module 140 is configured to obtain the IP address and the port through the cache registration management module 120, and broadcast local cache update information to each node of the target application through the IP address and the port, so that each node of the target application performs a cache update operation.
The cache registration management module 120 is configured to obtain a cache node list (i.e., a server list included in the target application) corresponding to the target application through a preset node registry, where the cache node list includes IP addresses and ports of nodes where local caches are located and registered in advance by different servers of the target application. Specifically, in this embodiment, the cache node list acquired by the cache registration management module 120 includes IP addresses and ports of the server 310, the server 320, and the server 330.
Thereafter, the cache update module 140 will broadcast local cache update information to the server 320 and the server 330 successively through the IP addresses and ports of the server 320 and the server 330, so that the server 320 and the server 330 perform cache update operations. The local cache update information includes a cache name, updated location information, or data information.
In some embodiments of the present application, as shown in fig. 4, the cache update module 140 further includes: a message broadcasting sub-module 1401 and a message transmitting sub-module 1402.
The message broadcasting sub-module 1401, configured to obtain, through the cache registration management module 120, an IP address and a port of a node of a local cache that is registered in advance by the target application; and the local cache update information received from the cache operation interface module 110.
The message sending sub-module 1402 is configured to send the local cache update information one by one to a node of a local cache that is registered in advance to an opposite-end target application through TCP connection with the IP address and the port. The peer target application is the same application as the target application on a different node, and may also be considered as a different server of the target application. In specific implementation, the message sending sub-module 1402 may be implemented by a sending module of the open source software Netty, and sends data such as local cache update information to a specified IP address and port through TCP connection.
In some embodiments of the present application, as shown in fig. 4, the cache update module 140 further includes: a message reception sub-module 1403 and a message processing sub-module 1404.
The message receiving sub-module 1403 is configured to receive, through a TCP connection, the local cache update information sent by a node where a local cache of any one of the peer target applications is located. For example, the message receiving sub-module 1403 is implemented by a receiving module of the open source software Netty, and the message receiving sub-module 1403 receives local cache update information sent by an external server (e.g., the server 320 and the server 330) through a TCP connection on a port registered by the server.
The message processing sub-module 1404 is configured to parse the local cache update information, and send an interface call for updating the local cache obtained through parsing to the cache operation management module 130. Specifically, the message processing sub-module 1404 parses the local cache update information sent by the external server, and obtains information such as a change type of the cache operation (e.g., update cache, delete cache), a cache name, a key (i.e., data information or address information) of the key cache, and converts the local cache update information into an interface call for updating the local cache corresponding to the local cache synchronization device (e.g., the cache synchronization device 3101 on the server 310) according to the information. Then, the message processing sub-module 1404 sends the interface call for updating the local cache obtained by the parsing to the cache operation management module 130, and the cache operation management module 130 finally realizes the cache update.
In some embodiments of the present application, as shown in fig. 4, the cache update module 140 further includes: a connection management submodule 1405, the connection management submodule 1405 configured to manage network connections between a node at which a local cache of a target application is located and a correspondent node of the target application. In a specific application, the node where the local cache of the target application is located is connected with the message sending submodule of the node where the opposite-end target application is located (such as the server 320 and the server 320) through the message receiving submodule of the node (such as the server 310) of the target application, and the node where the local cache of the target application is located (such as the server 310) is connected with the message receiving submodule of the node where the opposite-end target application is located (such as the server 320 and the server 320) through the message sending submodule of the node of the target application. For example, a long connection between a message receiving submodule (e.g., NettyServer) in the cache synchronizer 3101 on the server 310 and a message sending submodule (e.g., NettyClient) in the cache synchronizer 3201 on the server 320 enables the server 310 to receive the local cache update information sent by the server 320 in time through the cache synchronizer 3101.
In other embodiments of the present application, the cache operation interface module 110 is further configured to respond to an interface call of a target application querying a local cache, and send the interface call querying the local cache to the cache operation management module 130; the cache operation management module 130 is configured to determine a cache type according to the received interface call for querying the local cache, map the interface call for querying the local cache to a corresponding cache query interface call of the cache type through a reflection technology, and call the cache query interface to query the local cache. Namely, the cache synchronization device completely realizes cache operation in the prior art, and realizes effective cache synchronization between different nodes without changing the prior system and increasing development cost.
For the convenience of the reader to understand the present application, the following describes the working process of the synchronous Cache apparatus in the application system described in fig. 5 by taking the local Cache as the Guava Cache for example, and referring to the synchronous Cache apparatus described in fig. 4. The cache synchronization device is installed in each server of the application program in the form of a local cache component. The local cache component comprises: the cache updating method comprises a cache operation interface module (such as a Client), a cache registration management module (such as a node watch), a cache operation management module (such as a CacheManager), and a cache updating module, wherein the cache updating module further comprises: a message broadcasting sub-module (e.g., BroadCaster), a message sending sub-module (e.g., nettypeclient), a message receiving sub-module (e.g., nettypeserver), a message processing sub-module (e.g., RequestHandler), and a connection management sub-module (e.g., ChannelManager).
Firstly, when an application system is started, each server of the application registers a self-defined local cache in the synchronous cache device.
When the application program is started, the application program registers the created guava cache instance in the local cache component at the server side of each server (namely, cache node). For example, a customized local cache is registered to the node registration center (e.g., Zookeeper) by a cache syncclient-addlocalcache (e.g., Client) method of a cache operation interface module (e.g., Client), where the cache name is a cache name returned by a cache registration management module (e.g., node watch) of a local cache component, and the cache is a cache instance created by a server of an application program, and includes: cache type, size, etc.
Then, the application system starts the cache synchronization function of the local cache component. The cache synchronization function of the local cache component is started, for example, by calling the interface cachesyncclient. The local cache component registers the IP address and the port of the service (i.e. the node where the local cache is located) where the application program is located in a preset node registry (such as zookeeper) through a cache registration management module (for example, denoted as node watch). Meanwhile, NodeWatcher acquires the IP addresses and ports registered by other servers under the application program from the zooKeeper.
During the running process of the application program, the application program can inquire the local cache through the cache interface of the local cache component. For example: query the local cache through the interface cachesyncclient. In the process, a pre-registered guava cache is found in a cache operation management module (such as a cache manager) through the cache name, then a cache query method of the guava cache is executed through a generic type (namely a mapping relation between a pre-established packaging interface and an interface of the guava cache), and a corresponding Key value is transmitted to perform query operation.
In the running process of the application program, the application program may also modify or delete the local cache through the cache interface of the local cache component, at this time, the data of the local cache may change, and the data change of the local cache needs to be synchronized to other local cache nodes (i.e., other registered servers) of the application program. For example, the application refreshes the local cache through the interface cachesyncclient. When the local cache component of the application program senses the data change, a refreshing method provided by the cache operation interface module is called, then, a message broadcasting submodule of the cache updating module constructs a broadcast message (namely constructs local cache updating information), and a broadcasting interface of the message broadcasting submodule (such as BroadCaster) is called; the BroadCaster's broadcast interface will go to NodeWatcher to obtain the IP addresses and ports of other servers, and then send the local cache update information to each server one by one through the messaging submodule (e.g. nettypeclient). When nettypeclient interacts with other servers, it maintains network connections with other servers through a connection management submodule (e.g., ChannelManager).
Further, when a certain server of the application receives local cache update information sent by other servers through a message receiving submodule (e.g., NettyServer), the message is forwarded to a message processing submodule (e.g., RequestHandler), the message processing submodule identifies information such as specific cache names, categories, keys and the like corresponding to data changes, and then finds a corresponding cache from a cache operation management module CacheManager to execute local refresh operation.
The cache synchronization device disclosed in the embodiment of the present application, by setting the synchronization cache device at the application server, the synchronization cache device further includes: the cache operation interface module is used for responding to interface calling of a target application for updating a local cache and sending the interface calling for updating the local cache to the cache operation management module; the cache operation management module is used for calling and mapping the interface for updating the local cache into a cache operation interface packaged in advance and calling the cache operation interface to update the local cache; the cache operation interface module is also used for responding to interface call of updating the local cache by the target application and constructing local cache updating information; the cache registration management module is used for acquiring a cache node list corresponding to the target application through a preset node registration center, wherein the cache node list comprises IP addresses and ports of nodes where local caches are located and registered in advance by different service terminals of the target application; the cache updating module is used for acquiring the IP address and the port through the cache registration management module, and broadcasting the local cache updating information to each node of the target application through the IP address and the port so as to enable each node of the target application to execute cache updating operation, and under the condition that additional development and complex configuration are not needed to be carried out on a server side, synchronization of local caches among different cache nodes can be realized only by installing a local cache synchronizing device on the cache nodes, development cost is saved, and development efficiency is improved.
On the other hand, through technologies such as cluster automatic registration and long connection communication among servers, an automatic synchronization function among a plurality of servers in a single application is provided, and the effects of one-point updating and multi-point real-time synchronization are achieved. By packaging the uniform cache operation interface for various cache applications, different servers can use different cache applications, and the application system is more flexibly built.
Example two
As shown in fig. 6, a cache synchronization method disclosed in this embodiment includes: step 610 to step 650. The cache synchronization method of the present application is applied to the cache synchronization apparatus and the cache synchronization system shown in fig. 4 and fig. 3. In this embodiment, the cache synchronization apparatus is installed on the server with the target application in the form of a local cache component.
In some embodiments of the present application, before the step of responding to the interface call of the target application updating the local cache, the method further includes: responding to cache registration operation of registering local cache by a target application, and generating cache registration information according to the cache registration operation, wherein the cache registration information comprises: the IP address and the port of the node where the local cache is located are registered by the cache registration operation; and registering the IP address and the port to the preset node registration center according to the received cache registration information.
In specific implementation, when the application system is started, each server of the target application registers the customized local cache in the synchronous cache device. For example, the IP address and the port of the server where the customized local cache is located are registered to the node registry through the cache operation interface module. Then, the application system starts the cache synchronization function of the cache synchronization device.
After the registration of the node where the local cache is located is completed, the cache synchronization device may obtain the registered IP addresses and ports of other servers under the target application, so as to implement network communication between different servers.
During the running process of the application program, the application program can modify or delete the local cache through the cache interface of the local cache component. The step of mapping the interface call for updating the local cache to a cache operation interface packaged in advance and calling the cache operation interface to update the local cache comprises the following steps: determining the cache type according to the received interface call for updating the local cache; mapping the interface call for updating the local cache to a corresponding cache operation interface call of the cache category by a reflection technology; and calling the cache operation interface to update the local cache.
During specific implementation, the cache synchronization component obtains cache update-related operations such as cache modification and deletion of target reference through the built-in cache operation interface module, and then completes the operations such as modification and deletion of cache application through the built-in cache operation management module.
For a specific implementation manner of responding to an interface call of a target application for updating a local cache, mapping the interface call for updating the local cache to a cache operation interface packaged in advance, and calling the cache operation interface to update the local cache, reference is made to relevant descriptions in embodiment one, and details are not described here again.
After the application program modifies or deletes the local cache through the cache interface of the local cache component, the locally cached data may change, and the locally cached data needs to be synchronized to other registration servers of the target application. And the local cache component constructs a local cache updating message through a built-in cache operation interface module. The specific implementation of the cache operation interface module for constructing the local cache update message refers to the first embodiment, and details are not described in this embodiment.
And then, the cache operation interface module sends the constructed local cache updating message to the cache updating module.
After receiving the local cache update message, the cache update module first obtains a cache node list corresponding to the target application through a preset node registration center, where the cache node list includes IP addresses and ports of nodes where local caches are located, and the nodes are registered in advance by different servers of the target application.
Next, the cache update module broadcasts the local cache update information to each node of the target application through the IP address and port. In some embodiments of the present application, the step of broadcasting the local cache update information to each node of the target application through the IP address and the port includes: and sending the local cache updating information to the nodes of the local cache which are registered in advance to the opposite end target application one by one through the TCP connection with the IP address and the port.
Controlling the cache update module to broadcast the local cache update information to each node of the target application through the IP address and the port, so that each node of the target application executes a specific implementation of the cache update operation, see the related description in the first embodiment, which is not described again in this embodiment.
In other embodiments of the present application, as shown in fig. 6, after the step of registering the IP address and the port to the preset node registry according to the received cache registration information, the method further includes: step 660 and step 670.
And 660, receiving local cache updating information sent by a node where the local cache of any one opposite-end target application is located through the TCP connection.
When the method is implemented specifically, a target application in a cache synchronization system applied to the cache synchronization method comprises a plurality of servers, after a local cache on each server is updated, a local cache update message is sent to other servers, after a certain server receives the local cache update message, the message content is analyzed, and the local cache is updated according to the analysis result. Therefore, the local cache component on each server receives the local cache updating information sent by the opposite server in real time through the built-in cache updating module.
The specific implementation manner of the interface call for updating the local cache by the target application is obtained by receiving, through TCP connection, local cache update information sent by a node where the local cache of any one of the peer target applications is located, and analyzing the local cache update information sent by the node where the local cache of the peer target application is located, refer to the specific description of the cache update module in embodiment one, and this embodiment is not described again.
After the interface call for updating the local cache by the target application is obtained through analysis, the interface call for updating the local cache is mapped into a cache operation interface packaged in advance, and the cache operation interface is called to update the local cache.
In still other embodiments of the present application, the method further comprises: responding to the interface call of the target application for inquiring the local cache, and determining the cache type according to the received interface call for inquiring the local cache; and mapping the interface call for inquiring the local cache to the corresponding cache inquiry interface call of the cache type by a reflection technology, and calling the cache inquiry interface to inquire the local cache.
In the running process of the target application, the target application can inquire a local cache through a cache interface of the local cache component. In the process of inquiring the local cache, the local cache component finds the local cache which is registered in advance in the cache operation management module, then the interface call of inquiring the local cache is mapped into the self interface of the cache application through a reflection technology, the cache inquiring method of the corresponding cache application is executed, and the corresponding parameters are transmitted to carry out the inquiring operation.
The cache synchronization device disclosed in the embodiment of the present application is used to implement each step of the cache synchronization method described in the second embodiment of the present application, and for specific implementation of each step of the method, reference is made to corresponding modules, which are not described herein again.
According to the cache synchronization method disclosed by the embodiment of the application, after the target application is started, the local cache node applied by the target application is registered to a preset node registration center for management, then, in the running process of the target application, the interface call for updating the local cache is responded to the target application, the interface call for updating the local cache is mapped into a cache operation interface packaged in advance, and the cache operation interface is called to update the local cache; responding to the interface call of the target application for updating the local cache, and constructing local cache updating information; obtaining a cache node list corresponding to the target application through a preset node registration center, wherein the cache node list comprises IP addresses and ports of nodes where local caches are located and registered in advance by different service terminals of the target application; the local cache updating information is broadcasted to each node of the target application through the IP address and the port, so that each node of the target application executes cache updating operation, and under the condition that additional development and complex configuration are not needed to be carried out on a server side, synchronization of local caches among different cache nodes can be realized only by installing a local cache synchronizing device on the cache nodes, development cost is saved, and development efficiency is improved.
On the other hand, through technologies such as cluster automatic registration and long connection communication among servers, an automatic synchronization function among a plurality of servers in a single application is provided, and the effects of one-point updating and multi-point real-time synchronization are achieved. By packaging the uniform cache operation interface for various cache applications, different servers can use different cache applications, and the application system is more flexibly built.
Correspondingly, the present application also discloses an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the cache synchronization method according to the second embodiment of the present application is implemented. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer and the like.
The present application also discloses a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the cache synchronization method according to the second embodiment of the present application.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The cache synchronization method and apparatus provided by the present application are introduced in detail above, and a specific example is applied in the text to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understanding the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Claims (12)
1. A cache synchronization apparatus, comprising: a cache operation interface module, a cache registration management module, a cache operation management module and a cache updating module, wherein,
the cache operation interface module is used for responding to interface call of updating a local cache by a target application and sending the interface call of updating the local cache to the cache operation management module;
the cache operation management module is used for calling and mapping the interface for updating the local cache into a cache operation interface packaged in advance and calling the cache operation interface to update the local cache;
the cache operation interface module is also used for responding to interface call of updating the local cache by the target application and constructing local cache updating information;
the cache registration management module is used for acquiring a cache node list corresponding to the target application through a preset node registration center, wherein the cache node list comprises IP addresses and ports of nodes where local caches are located and registered in advance by different service terminals of the target application;
the cache updating module is configured to acquire the IP address and the port through the cache registration management module, and broadcast the local cache updating information to each node of the target application through the IP address and the port, so that each node of the target application executes a cache updating operation;
the cache operation interface module is further configured to respond to a cache registration operation of registering a local cache by a target application, generate cache registration information according to the cache registration operation, and forward the cache registration information to the cache registration management module, where the cache registration information includes: the IP address and the port of the node where the local cache is located are registered by the cache registration operation;
the cache registration management module is used for registering the IP address and the port to the preset node registration center according to the received cache registration information;
wherein the cache synchronization device is arranged on each node of the target application in a component form.
2. The apparatus of claim 1, wherein the cache update module further comprises:
the message broadcasting sub-module is used for acquiring the IP address and the port of the node of the local cache registered in advance by the target application through the cache registration management module; receiving the local cache updating information from the cache operation interface module;
and the message sending submodule is used for sending the local cache updating information to the nodes applying the pre-registered local cache to the opposite terminal target one by one through the TCP connection with the IP address and the port.
3. The apparatus of claim 2, wherein the cache update module further comprises:
the message receiving submodule is used for receiving the local cache updating information sent by the node where the local cache of any one opposite-end target application is located through TCP connection;
and the message processing submodule is used for analyzing the local cache updating information and sending the interface call for updating the local cache obtained by analysis to the cache operation management module.
4. The apparatus of claim 3, wherein the node where the local cache of the target application is located is connected to the message sending submodule of the peer target application by its own message sending submodule, and the node where the local cache of the target application is located is connected to the message receiving submodules of the plurality of peer target applications by its own message sending submodule.
5. The apparatus according to any one of claims 1 to 4, wherein the step of mapping the interface call for updating the local cache to a cache operation interface packaged in advance and calling the cache operation interface to update the local cache comprises:
determining the cache type according to the received interface call for updating the local cache;
mapping the interface call for updating the local cache to a corresponding cache operation interface call of the cache category by a reflection technology;
and calling the cache operation interface to update the local cache.
6. A cache synchronization method, comprising:
responding to interface call of a target application for updating a local cache, mapping the interface call for updating the local cache into a cache operation interface packaged in advance, and calling the cache operation interface to update the local cache;
responding to the interface call of the target application for updating the local cache, and constructing local cache updating information;
obtaining a cache node list corresponding to the target application through a preset node registration center, wherein the cache node list comprises IP addresses and ports of nodes where local caches are located and registered in advance by different service terminals of the target application;
broadcasting the local cache updating information to each node of the target application through the IP address and the port so as to enable each node of the target application to execute cache updating operation;
responding to cache registration operation of registering local cache by a target application, and generating cache registration information according to the cache registration operation, wherein the cache registration information comprises: the IP address and the port of the node where the local cache is located are registered by the cache registration operation;
registering the IP address and the port to the preset node registration center according to the received cache registration information;
wherein the cache synchronization device is arranged on each node of the target application in a component form.
7. The method of claim 6, wherein the step of broadcasting the local cache update information to the nodes of the target application via the IP address and port comprises:
and sending the local cache updating information to the nodes of the local cache which are registered in advance to the opposite end target application one by one through the TCP connection with the IP address and the port.
8. The method according to claim 7, wherein after the step of registering the IP address and the port with the predetermined node registry according to the received cache registration information, the method further comprises:
receiving local cache updating information sent by a node where a local cache of any one opposite-end target application is located through TCP connection;
analyzing the local cache updating information sent by the node where the local cache of the opposite-end target application is located to obtain the interface call of the target application for updating the local cache, skipping to the interface call for responding the target application for updating the local cache, mapping the interface call for updating the local cache to a cache operation interface packaged in advance, and calling the cache operation interface to update the local cache.
9. The method according to any one of claims 6 to 8, wherein the step of mapping the interface call for updating the local cache to a pre-encapsulated cache operation interface and calling the cache operation interface to update the local cache comprises:
determining the cache type according to the received interface call for updating the local cache;
mapping the interface call for updating the local cache to a corresponding cache operation interface call of the cache category by a reflection technology;
and calling the cache operation interface to update the local cache.
10. A cache synchronization system, comprising: a plurality of nodes of a target application, a node registry, and a cache synchronization apparatus according to any one of claims 1 to 5 provided on each of the nodes,
the node comprises a local cache, and the node where the target application is located executes local cache updating through the cache synchronization device arranged on the node;
the node registration center is used for sending the node change information of the target application to the cache synchronization device according to the registration information of the node;
wherein the cache synchronization device is arranged on each node of the target application in a component form.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the cache synchronization method of any one of claims 6 to 9 when executing the computer program.
12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the cache synchronization method according to any one of claims 6 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910557298.9A CN110413418B (en) | 2019-06-25 | 2019-06-25 | Cache synchronization device and method, cache synchronization system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910557298.9A CN110413418B (en) | 2019-06-25 | 2019-06-25 | Cache synchronization device and method, cache synchronization system and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110413418A CN110413418A (en) | 2019-11-05 |
| CN110413418B true CN110413418B (en) | 2022-04-05 |
Family
ID=68358225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910557298.9A Active CN110413418B (en) | 2019-06-25 | 2019-06-25 | Cache synchronization device and method, cache synchronization system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110413418B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111159233B (en) * | 2019-12-18 | 2024-03-08 | 金蝶软件（中国）有限公司 | Distributed caching method, system, computer equipment and storage medium |
| CN111522605B (en) * | 2020-04-16 | 2022-07-29 | 艾普阳科技（深圳）有限公司 | Local procedure calling method, device, equipment and medium |
| CN111797119B (en) * | 2020-05-19 | 2024-01-30 | 武汉乐程软工科技有限公司 | Caching device, system and caching method |
| CN111866170B (en) * | 2020-07-30 | 2023-08-08 | 小视科技(江苏)股份有限公司 | Method for transmitting synchronous message in IOT cluster |
| CN111988427A (en) * | 2020-08-31 | 2020-11-24 | 湖南快乐阳光互动娱乐传媒有限公司 | Distributed local cache processing method and system |
| CN112559632A (en) * | 2020-12-15 | 2021-03-26 | 北京百度网讯科技有限公司 | Method, device, electronic equipment and medium for synchronizing state of distributed graph database |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6477543B1 (en) * | 1998-10-23 | 2002-11-05 | International Business Machines Corporation | Method, apparatus and program storage device for a client and adaptive synchronization and transformation server |
| CN106790705A (en) * | 2017-02-27 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of Distributed Application local cache realizes system and implementation method |
| CN107291750A (en) * | 2016-03-31 | 2017-10-24 | 阿里巴巴集团控股有限公司 | A kind of data migration method and device |
| CN107948318A (en) * | 2017-12-27 | 2018-04-20 | 世纪龙信息网络有限责任公司 | Cache synchronization method and system between multinode |
| CN108897868A (en) * | 2018-06-29 | 2018-11-27 | 北京奇虎科技有限公司 | Cache synchronization method and device, calculating equipment and storage medium based on trigger |
| CN108920670A (en) * | 2018-07-06 | 2018-11-30 | 深圳市小牛在线互联网信息咨询有限公司 | Cache synchronization method, device, system and storage medium |
| CN109582730A (en) * | 2018-10-11 | 2019-04-05 | 阿里巴巴集团控股有限公司 | Cache synchronization method, device, electronic equipment and computer readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7716182B2 (en) * | 2005-05-25 | 2010-05-11 | Dassault Systemes Enovia Corp. | Version-controlled cached data store |
| US8161244B2 (en) * | 2009-05-13 | 2012-04-17 | Microsoft Corporation | Multiple cache directories |
| CN105515872B (en) * | 2015-12-23 | 2019-05-28 | 北京奇虎科技有限公司 | The update method of configuration information, apparatus and system |
| CN109213792B (en) * | 2018-07-06 | 2021-11-09 | 武汉斗鱼网络科技有限公司 | Data processing method, server, client, device and readable storage medium |
-
2019
- 2019-06-25 CN CN201910557298.9A patent/CN110413418B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6477543B1 (en) * | 1998-10-23 | 2002-11-05 | International Business Machines Corporation | Method, apparatus and program storage device for a client and adaptive synchronization and transformation server |
| CN107291750A (en) * | 2016-03-31 | 2017-10-24 | 阿里巴巴集团控股有限公司 | A kind of data migration method and device |
| CN106790705A (en) * | 2017-02-27 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of Distributed Application local cache realizes system and implementation method |
| CN107948318A (en) * | 2017-12-27 | 2018-04-20 | 世纪龙信息网络有限责任公司 | Cache synchronization method and system between multinode |
| CN108897868A (en) * | 2018-06-29 | 2018-11-27 | 北京奇虎科技有限公司 | Cache synchronization method and device, calculating equipment and storage medium based on trigger |
| CN108920670A (en) * | 2018-07-06 | 2018-11-30 | 深圳市小牛在线互联网信息咨询有限公司 | Cache synchronization method, device, system and storage medium |
| CN109582730A (en) * | 2018-10-11 | 2019-04-05 | 阿里巴巴集团控股有限公司 | Cache synchronization method, device, electronic equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110413418A (en) | 2019-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110413418B (en) | Cache synchronization device and method, cache synchronization system and electronic equipment | |
| CN110351283B (en) | Data transmission method, device, equipment and storage medium | |
| CN112527528A (en) | Data transmission method, device and storage medium based on message queue | |
| US9401885B2 (en) | Data push service method and system using data pull model | |
| CN101808051B (en) | Application integration gateway and control method thereof | |
| CN112328413B (en) | Application service calling method, device and system | |
| CN115562757A (en) | Data processing method, configuration center system, electronic device, and storage medium | |
| CN106357654B (en) | Remote procedure calling method, device and communication system | |
| CN109981546B (en) | Method and device for acquiring remote call relation between application modules | |
| CN110865973B (en) | Data processing method and equipment and related device | |
| CN112073488A (en) | Method and device for processing request | |
| CN111581576A (en) | Development processing method and device based on micro-service and storage medium | |
| CN113127335A (en) | System testing method and device | |
| CN112187916B (en) | Cross-system data synchronization method and device | |
| CN113746851B (en) | Proxy system and method supporting real-time analysis of GRPC request | |
| CN115629909A (en) | Service data processing method and device, electronic equipment and storage medium | |
| CN110659299B (en) | Data updating method and device and front-end development client | |
| CN109669793B (en) | Object calling method in middleware process | |
| CN114064165A (en) | Application page processing method and device | |
| CN112231405A (en) | Data storage device | |
| CN111078736A (en) | Data aggregation processing method and device, terminal and storage medium | |
| CN112306984A (en) | Data source routing method and device | |
| CN109582481A (en) | Transmission method, device, equipment and the storage medium of call result | |
| CN115297066B (en) | Resource dynamic cooperation method, device, server and medium | |
| US11799982B2 (en) | Message push method, apparatus, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |