CN113364822A

CN113364822A - Cache data updating method and device

Info

Publication number: CN113364822A
Application number: CN202010148244.XA
Authority: CN
Inventors: 丁鹏程; 刘成诚
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2020-03-05
Filing date: 2020-03-05
Publication date: 2021-09-07

Abstract

The invention discloses a cache data updating method and device, and relates to the technical field of computers. One embodiment of the method comprises: responding to a registration monitoring request of a first client to a node, and establishing a temporary node under the node to store an IP address of the first client; receiving a cache updating notification transmitted by a second client, transmitting the cache updating notification to a first client through an IP address, and then receiving a message fed back by at least one first client; and if the number of the first clients of the feedback message is less than the number of the temporary nodes, the cache updating notification is sent to the first clients again so as to perform local cache updating on the first clients. The embodiment adds a feedback mechanism and a compensation mechanism, when the first client side does not receive the cache updating notification, the cache updating notification can be retransmitted to all the first client sides or only the third client sides which do not receive the notification, and the method has real-time notification reliability.

Description

Cache data updating method and device

Technical Field

The invention relates to the technical field of computers, in particular to a cache data updating method and device.

Background

The application local cache is usually stored in a JVM (Java Virtual Machine), and has the characteristics of high access efficiency compared with a unified cache center because no network and serialization overhead is involved,

the existing local cache management needs to set cache expiration time, and when the expiration time is reached, the cache is updated again. However, in some cases (for example, the change of the city message), the cache expiration time is unknown, and if the setting is too long, the local cache is not updated in time, which may cause a problem that the data in the database is not consistent with the data in the local cache, so that it is necessary to monitor whether the data in the database is changed in real time, so as to update the local cache in time.

For the above defects, the data update notification received from the first client is mainly pushed to the second client in real time by means of a Zookeeper Watch mechanism in the prior art. However, in the process of implementing the invention, the inventor finds that the prior art has at least the following problems:

the Zookeeper Watch mechanism is not suitable for the situation of high data concurrency, the mechanism is single monitoring, monitoring events can be cleared after the monitoring is finished, re-registration is needed if the monitoring is continued, but during the registration period, if the data in the first client is changed, the second client cannot be notified to update the cache in time.

Disclosure of Invention

In view of this, embodiments of the present invention provide a cache data updating method and apparatus, which can at least solve the problem in the prior art that a local cache is not updated in real time.

To achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a cache data updating method, including:

responding to a registration monitoring request of a first client to a node, and establishing a temporary node under the node to store an IP address of the first client;

receiving a cache updating notification transmitted by a second client, transmitting the cache updating notification to the first client through the IP address, and then receiving a message fed back by at least one first client; wherein the message indicates that a cache update notification has been received;

and if the number of the first clients of the feedback message is less than the number of the temporary nodes, the cache updating notification is sent to the first client again so as to perform local cache updating on the first client.

Optionally, the receiving the cache update notification transmitted by the second client includes:

receiving an update notification of a second client to the cache data, and storing a key name of the cache data into the node so as to generate the cache update notification based on the key name through the node; wherein the node represents a node in which cache data is changed;

the transmitting the cache update notification to the first client via the IP address includes:

and transmitting the cache updating notification to the first client through the IP address so as to trigger the first client to load a key value from the second client according to the key name.

Optionally, the resending the cache update notification to the first client to perform local cache update on the first client includes:

and resending the cache updating notification to all the first clients under the temporary node so as to perform local cache updating on all the first clients.

Optionally, the message includes the key name and an IP address;

the resending the cache update notification to the first client to perform local cache update on the first client includes:

determining an IP address of a third client which does not receive the cache update notification based on the IP address in the temporary node and the IP address in the feedback message; wherein the third client is a subset of the first client;

and retransmitting the cache updating notification to the third client through the IP address of the third client so as to update the local cache of the third client.

Optionally, after receiving the message fed back by the at least one first client, the method further includes:

storing the message in a feedback record corresponding to the key name; and

and storing the feedback record into the second client.

To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a cache updating apparatus, including:

the device comprises a request receiving module, a node processing module and a monitoring module, wherein the request receiving module is used for responding to a registration monitoring request of a first client to the node and establishing a temporary node under the node so as to store an IP address of the first client;

the message feedback module is used for receiving a cache updating notification transmitted by a second client, transmitting the cache updating notification to the first client through the IP address, and then receiving a message fed back by at least one first client; wherein the message indicates that a cache update notification has been received;

and the notification compensation module is used for resending the cache updating notification to the first client side if the number of the first client sides of the feedback message is less than the number of the temporary nodes so as to update the local cache of the first client side.

Optionally, the message feedback module is configured to:

Optionally, the notification compensation module is configured to: and resending the cache updating notification to all the first clients under the temporary node so as to perform local cache updating on all the first clients.

Optionally, the message includes the key name and an IP address;

the notification compensation module is configured to:

Optionally, the system further includes a message recording module, configured to:

storing the message in a feedback record corresponding to the key name; and

and storing the feedback record into the second client.

To achieve the above object, according to still another aspect of the embodiments of the present invention, an electronic device for cache update is provided.

The electronic device of the embodiment of the invention comprises: one or more processors; a storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, enable the one or more processors to implement any of the above cache data updating methods.

To achieve the above object, according to a further aspect of the embodiments of the present invention, there is provided a computer readable medium having a computer program stored thereon, the computer program, when executed by a processor, implementing any one of the above-mentioned cache data updating methods.

According to the scheme provided by the invention, one embodiment of the invention has the following advantages or beneficial effects: and a feedback mechanism and a compensation mechanism are added, when the number of the clients which feedback the received notifications is less than the total number of the clients, the cache updating notifications can be retransmitted to all the first clients or only the third clients which do not receive the notifications, so that the cache data key is compensated and updated, and the reliability of the real-time notifications is realized.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

fig. 1 is a schematic main flow chart of a cache data updating method according to an embodiment of the present invention;

FIG. 2 is a flow chart illustrating an alternative cache data update method according to an embodiment of the present invention;

FIG. 3 is a block diagram of a method for updating cache data according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of major blocks of a cache update apparatus according to an embodiment of the present invention;

FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;

FIG. 6 is a schematic block diagram of a computer system suitable for use with a mobile device or server implementing an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that the embodiment of the present invention relates to a local cache update scheme, which is applicable to multiple computer languages, such as java. The nodes are mainly in zookeeper; wherein zookeeper is a distributed, open source distributed application coordination service.

Referring to fig. 1, a main flowchart of a cache data updating method according to an embodiment of the present invention is shown, including the following steps:

s101: responding to a registration monitoring request of a first client to a node, and establishing a temporary node under the node to store an IP address of the first client;

s102: receiving a cache updating notification transmitted by a second client, transmitting the cache updating notification to the first client through the IP address, and then receiving a message fed back by at least one first client; wherein the message indicates that a cache update notification has been received;

s103: and if the number of the first clients of the feedback message is less than the number of the temporary nodes, the cache updating notification is sent to the first client again so as to perform local cache updating on the first client.

In the above embodiment, for step S101, the first client requests to register one node in the snooper for monitoring Zookeeper, and invokes the Watch interface of the Zookeeper to complete the registration operation. And after the registration is finished, a temporary node is established under the node for storing the IP address of the first client.

It should be noted that one node may correspond to a plurality of temporary nodes, one client may correspond to one temporary node, and the number of the first clients corresponding to the node and which first clients are specific may be counted by storing the client IP address.

In addition, here the snooping is still a one-way snoop. The temporary node is a node type under the Zookeeper, and when a first client creating the temporary node crashes or closes a connection with the Zookeeper, the node is deleted.

For step S102, when the cache data in the second client (e.g., the database) is changed, a cache update notification is transmitted to the Zookeeper, and the Zookeeper forwards the cache update notification to the first client.

The cache data in the present invention is in a key-value pair form, the key of which is stored in the node, and the value of which is stored in the second client, which is described in detail with reference to fig. 2 later. After receiving the notification, the first client loads a corresponding value from the second client according to the key in the cache data, and then updates the local cache of the first client.

In addition, after receiving the notification, the first client invokes a feedback interface (the interface is associated with the message feedback module) provided by the Proxy (Proxy), feeds back the message of receiving the cache update notification to the Proxy, and uploads the updated cache data key, the IP address and other information.

It should be noted that the whole notification mechanism of the present invention is implemented on the proxy. After receiving the feedback message of the first client, the agent may store the feedback message into a feedback record corresponding to the cache data key, where the feedback record may be stored in a second client or a preset file/module.

For step S103, when there is a network anomaly or the notification is inconvenient to send or receive, some first clients may not receive the cache update notification, and thus cannot update the local cache.

Based on the solution of the above problem, in addition to the message feedback module, the present invention further provides a notification compensation module in the proxy, so as to perform a compensation operation when detecting that the first client does not receive the cache update notification. Specifically, the method comprises the following steps:

1) counting the number of IP addresses in the temporary node under the Zookeeper node, and taking the number as the total number of the first clients; wherein the temporary node may be screened based on the cached data key;

2) counting the number of the first clients which have received the cache updating notice from the feedback record;

3) comparing the total number of first clients in 1) with the amount of feedback in 2):

if the feedback quantity is equal to the total quantity of the first clients, setting the cache key state in the node as the updating success;

if the feedback quantity is less than the total quantity of the first clients, the fact that partial clients (namely third clients) do not receive the cache updating notice is indicated, and the feedback record is set to be in an expired (or failed) state;

4) triggering a compensation mechanism, and performing buffer updating notification transmission again:

firstly, the first client sends the cache updating notice once again no matter whether the first client receives the cache updating notice or not;

determining a third client which does not receive the cache updating notification according to the first client IP address in the temporary node and the first client IP address in the feedback record, and then only sending the cache updating notification to the third client;

5) whether all the first clients are notified again or only the third client is notified, the feedback quantity and the total number of the clients still need to be compared repeatedly subsequently until the repetition number reaches the upper limit or all the first clients receive the cache updating notification.

It should be noted that, compared with the method of 4, the method of the present invention needs to determine the third client that has not received the notification, and then sends the cache update notification, and the operation steps are fewer, and only the first client loads the cache data again, which is efficient.

In addition, a background task may be initiated to periodically (e.g., 5s) scan the cached data key to check if all first clients have received the cache update notification. It should be noted that, for the same cached data key in the same node, multiple first clients may be associated, and when the cached data in the second client is changed, the caches of all the first clients need to be updated.

The method provided by the embodiment adds the feedback mechanism and the compensation mechanism, and when the number of the clients which feedback the received notifications is smaller than the total number of the clients, the cache update notification can be resent to all the first clients or only the third clients which do not receive the notifications, so that the compensation update of the cache data key is realized, and the method has real-time notification reliability.

Referring to fig. 2, a schematic flow chart of an optional cache data updating method according to an embodiment of the present invention is shown, including the following steps:

s201: responding to a registration monitoring request of a first client to a node, and establishing a temporary node under the node to store an IP address of the first client;

s202: receiving an update notification of a second client to the cache data, and storing a key name of the cache data into the node so as to generate the cache update notification based on the key name through the node; wherein the node represents a node in which cache data is changed;

s203: transmitting the cache updating notification to the first client through the IP address so as to trigger the first client to load a key value from the second client according to the key name;

s204: receiving a message fed back by at least one first client; wherein the message indicates that a cache update notification has been received;

s205: and if the number of the first clients of the feedback message is less than the number of the temporary nodes, the cache updating notification is sent to the first client again so as to perform local cache updating on the first client.

In the above embodiment, for steps S201, S204, and S205, reference may be made to the descriptions of steps S101 to S103 shown in fig. 1, and details are not repeated here.

In the above embodiment, for steps S202 and S203, the operator may add, modify, and delete the cache data through a CMS (Content Management System), and store the changed data (value thereof) in the second client.

The cache data in the invention is in a key value pair form, the key of the cache data is stored in the Zookeeper node by CMS through an agent, and the node content of the node is the cache data key at this time. It should be noted that, the node here only indicates that the value corresponding to the key is updated, and after determining that all the first clients receive the cache update notification, the node performs a clearing operation on the cache data key.

The cache update notification transmitted to the first client may be generated by the node after receiving the cache data key. And then, calling a real-time notification module of the proxy, and transmitting the cache updating notification to the first client. After receiving the notification, the first client loads a corresponding value from the second client according to the key of the cache data, and then updates the local cache of the first client.

In the method provided by the embodiment, only the cache data key in the second client is stored in the node, and the node generates the cache update notification, and transmits the cache update notification to the first client through the real-time notification module, so as to trigger the first client to perform message feedback and key value loading operations.

Referring to fig. 3, a schematic diagram of a cache data updating method according to an embodiment of the present invention is shown, including a real-time updating module, a message feedback module, and a notification compensation module, where:

1) and the real-time notification module is used for registering and monitoring a node to the Zookeeper when a first Client (Client) starts a program, and establishing a temporary node under the node to store the IP address of the first Client.

The CMS updates the cached data in the second client, stores the key of the cached data in the node through a real-time updating module of the Proxy (Proxy), and generates a cache updating notice to be transmitted to the first client through the node.

2) A message feedback module: after receiving the cache updating notification, the first client calls a feedback interface provided by a proxy (proxy), feeds back the received cache updating notification to the proxy, and uploads fields such as an IP address and an updated cache key of the first client. And after receiving the feedback, the message feedback module reports the client information in a feedback record.

3) And the notification compensation module periodically scans the cache data key, checks whether all the first clients receive the cache updating notification, sets the cache key state as the updating success state if the update messages are not received by the clients, sets the feedback record as the expiration state if the update messages are not received by the clients, and calls the real-time notification module again to initiate the notification.

Referring to fig. 4, a schematic diagram of main modules of a cache data updating apparatus 400 according to an embodiment of the present invention is shown, including:

a request receiving module 401, configured to, in response to a registration monitoring request of a first client to a node, establish a temporary node under the node to store an IP address of the first client;

a message feedback module 402, configured to receive a cache update notification transmitted by a second client, transmit the cache update notification to the first client through the IP address, and then receive a message fed back by at least one first client; wherein the message indicates that a cache update notification has been received;

a notification compensation module 403, configured to resend the cache update notification to the first client if the number of the first clients of the feedback message is smaller than the number of the temporary nodes, so as to perform local cache update on the first client.

In the device for implementing the present invention, the message feedback module 402 is configured to:

In the implementation apparatus of the present invention, the notification compensation module 403 is configured to: and resending the cache updating notification to all the first clients under the temporary node so as to perform local cache updating on all the first clients.

In the device for implementing the invention, the message comprises the key name and the IP address;

the notification compensation module 403 is configured to:

The device for implementing the present invention further includes a message recording module 404 (not shown in the figure), configured to:

storing the message in a feedback record corresponding to the key name; and

and storing the feedback record into the second client.

In addition, the detailed implementation of the device in the embodiment of the present invention has been described in detail in the above method, so that the repeated description is not repeated here.

Fig. 5 illustrates an exemplary system architecture 500 to which embodiments of the invention may be applied.

As shown in fig. 5, the system architecture 500 may include

terminal devices

501, 502, 503, a network 504, and a server 505 (by way of example only). The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. Various communication client applications may be installed on the

terminal devices

501, 502, 503.

The

terminal devices

501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the

terminal devices

501, 502, 503.

It should be noted that the method provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the apparatus is generally disposed in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a request receiving module, a message feedback module, and a notification compensation module. Where the names of these modules do not in some cases constitute a limitation on the module itself, for example, the notification compensation module may also be described as a "module that compensates for cache update notifications".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:

According to the technical scheme of the embodiment of the invention, the feedback mechanism and the compensation mechanism are added, when the number of the clients which feed back the received notifications is less than the total number of the clients, the cache updating notification can be resent to all the first clients or only the third clients which do not receive the notifications, the compensation updating of the cache data key is realized, and the reliability of the real-time notification is realized.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cache data updating method is characterized by comprising the following steps:

2. The method of claim 1, wherein receiving the cache update notification transmitted by the second client comprises:

3. The method of claim 1, wherein the resending the cache update notification to the first client for local cache update to the first client comprises:

4. The method of claim 1, wherein the message comprises the key name and an IP address;

5. The method according to any of claims 1-4, further comprising, after receiving the at least one first client feedback message:

storing the message in a feedback record corresponding to the key name; and

and storing the feedback record into the second client.

6. A cache data update apparatus, comprising:

7. The apparatus of claim 6, wherein the message feedback module is configured to:

8. The apparatus of claim 6, wherein the notification compensation module is configured to: and resending the cache updating notification to all the first clients under the temporary node so as to perform local cache updating on all the first clients.

9. The apparatus of claim 6, wherein the message comprises the key name and an IP address;

the notification compensation module is configured to:

10. The apparatus according to any one of claims 6-9, further comprising a message recording module configured to:

storing the message in a feedback record corresponding to the key name; and

and storing the feedback record into the second client.

11. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.

12. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-5.