Disclosure of Invention
Based on this, the embodiments of the present specification provide a data caching method, a data caching system, a cache disaster recovery method, and a cache system.
According to a first aspect of embodiments herein, there is provided a data caching method, the method including: searching service data in a cache system when receiving a service data request sent by a service system; if the service data is found, acquiring the logic validity period of the service data; wherein the logical validity period is a validity period of the service data corresponding to a service; if the logic validity period is over, the service data is cached to the cache system from a remote end, and the physical validity period of the service data in the cache system is set to be permanently valid.
Optionally, the method further comprises: and if the service data is not found in the cache system, caching the service data from a remote end to the cache system.
Optionally, the method further comprises: and if the service data is not searched in the cache system, searching the service data in the cache system again after the cache data lock is obtained, and if the service data is not searched yet, caching the service data from a remote end to the cache system.
Optionally, the method further comprises: if the acquired cache data lock is overtime, the process is ended.
Optionally, the step of caching the service data from a remote end to the cache system includes: and caching the first service data with high importance degree and the second service data with low importance degree into different areas which are isolated from each other in the cache system respectively.
Optionally, before caching the service data from the remote end to the cache system, the method further includes: and if the cache unit for storing the second business data is full, the second business data with the longest cache time is evicted from the cache.
Optionally, the method further comprises: and returning the service data in the cache system to the service system.
Optionally, the method further comprises: if the logic validity period is over, judging whether the service data request carries a cache data lock; if yes, returning to the step of caching the service data from the remote end to the cache system.
Optionally, the method further comprises: and if the logic validity period is not over, returning the service data searched in the cache system to the service system.
Optionally, the physical validity period of the service data in the cache system is permanently valid.
According to a second aspect of embodiments herein, there is provided a cache disaster recovery method, including: if the remote service data acquisition is overtime, searching the service data in a cache system; sending the searched service data to a service system; wherein, the service data is cached to the cache system according to the data caching method of any embodiment.
According to a third aspect of embodiments herein, there is provided a data caching apparatus, the apparatus including: the first searching module is used for searching the service data in the cache system when receiving a service data request sent by the service system; the first acquisition module is used for acquiring the logic validity period of the service data if the service data is found; wherein the logical validity period is a validity period of the service data corresponding to a service; and the first cache module is used for caching the service data from a remote end to the cache system if the logic validity period is over, and setting the physical validity period of the service data in the cache system as permanent validity.
Optionally, the apparatus further comprises: and the second cache module is used for caching the service data from a remote end to the cache system if the service data is not found in the cache system.
Optionally, the apparatus further comprises: the third searching module is used for searching the service data in the cache system again after the cache data lock is obtained; and the third cache module is used for caching the service data from a remote end to the cache system when the third searching module does not search the service data.
Optionally, the apparatus further comprises: and the overtime waiting module is used for ending the process if the acquired cache data lock is overtime.
Optionally, the first cache module includes: and the cache unit is used for caching the first service data with high importance degree and the second service data with low importance degree into different areas which are isolated from each other in the cache system respectively.
Optionally, the apparatus further comprises: and the data eviction module is used for evicting the second business data with the longest cache time from the cache if the cache unit for storing the second business data is full.
Optionally, the apparatus further comprises: and the first sending module is used for returning the service data in the cache system to the service system.
Optionally, the apparatus further comprises: the judging module is used for judging whether the service data request carries a cache data lock or not if the logic validity period is over; and if so, returning to execute the function of the first cache module.
Optionally, the apparatus further comprises: and the second sending module is used for returning the service data searched in the cache system to the service system if the logic validity period does not expire.
Optionally, the physical validity period of the service data in the cache system is permanently valid.
According to a fourth aspect of embodiments herein, there is provided a cache disaster recovery apparatus, including: the second searching module is used for searching the service data in the cache system if the remote service data is overtime; the sending module is used for sending the searched service data to the service system; wherein, the service data is cached to the cache system according to the data caching method of any embodiment.
According to a fifth aspect of embodiments herein, there is provided a computer readable storage medium having a computer program stored thereon, wherein the program when executed by a processor implements the method of any of the embodiments.
According to a sixth aspect of the embodiments of the present specification, there is provided a cache system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of the embodiments when executing the program.
Optionally, the cache system includes a general cache unit and a white list cache unit; the white list cache unit is used for storing first service data with high importance degree, and the general cache unit is used for storing second service data with low importance degree; and when the general cache unit is full, the first cached second business data is evicted from the cache.
By applying the scheme of the embodiment of the description, on one hand, the physical validity period of the business data in the cache system is set to be permanently valid, so that the business data for realizing cache bottom holding always exist in the cache system, and the cache disaster tolerance is realized; on the other hand, when the service data logic in the cache system is out of date, the latest service data is requested from the remote end, so as to ensure that the service data acquired by the service system is the service data within the logic validity period under the condition that the remote end data can be acquired.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the invention.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the examples of this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the embodiments of the specification, as detailed in the appended claims.
The terminology used in the embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of the present specification. As used in the specification examples and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the embodiments herein. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Fig. 1 is a schematic diagram illustrating the interaction between the distal end and the proximal end of one embodiment of the present disclosure. Therein, the proximal end 102 may communicate with the distal end 104 to obtain desired data from the distal end 104. The near end 102 may include a service system 102a and a cache system 102b for executing a service, where the service system 102a may operate a corresponding service according to remote data acquired from the far end 104, and in order to improve efficiency of the entire system and reduce data interaction with the far end 104, the near end 102 may further store data in the cache system 102b after acquiring the remote data. The cache system 102b may be a same-process memory, a cross-process memory, a memory of a virtual machine different from the physical machine, or a storage medium such as a hard disk. In some practical scenarios, it is often necessary to ensure high availability of services. For example, for a pay gateway scenario, the gateway has a strong dependency on the configuration data in the cache system 102b, and if the cache data in the cache system 102b fails, the back-end service cannot acquire the configuration data, which will result in a failure of the entire service link.
Based on this, the embodiments of the present specification provide a data caching method. As shown in fig. 2, the method may include:
step 202: searching service data in a cache system when receiving a service data request sent by a service system;
step 204: if the service data is found, acquiring the logic validity period of the service data; wherein the logical validity period is a validity period of the service data corresponding to a service;
step 206: if the logic validity period is over, the service data is cached to the cache system from a remote end, and the physical validity period of the service data in the cache system is set to be permanently valid.
In step 204, the logical validity period of the service data refers to the validity period of the service data required for executing a certain service relative to the service. After the logic validity period, the service data is considered as out-of-date data relative to the service, but in order to realize cache disaster tolerance, the out-of-date service data is still stored in the cache system, and the out-of-date data is not deleted, so that a cache bottom is realized. Therefore, the logical validity period may be understood as the validity period of the service data. In contrast, the physical validity period is the period of time during which the service data is stored in the cache system. The business data that exceeds the physical validity period will be evicted from the cache.
The logical validity period of the service data may be set in advance, for example, to 1 hour. The time mark can be set for the cache system, the time mark carries the logic validity information, the time can be started to be counted from the service data stored in the cache system, and when the counted time reaches the logic validity, the logic validity of the service data is considered to be passed.
In step 206, if the logic validity period of the service data is over, an attempt may be made to acquire the latest service data from the remote end, and if the latest service data is acquired, the latest service data is cached in the cache system, and the service data with the logic expiration is covered, so as to ensure the timeliness of the service data. The remote end can be an upstream application system, a database and the like.
On one hand, in the embodiment of the present description, the physical validity period of the business data in the cache system is set as permanent validity, so that the business data for implementing cache bottom entry always exists in the cache system, so as to implement cache disaster tolerance; on the other hand, when the service data logic in the cache system is out of date, the latest service data is requested from the remote end, so as to ensure that the service data acquired by the service system is the service data within the logic validity period under the condition that the remote end data can be acquired.
In an embodiment, if the service data is not found in the cache system, the service data may be directly cached from a remote end to the cache system. If the business data is not found in the cache system, the business data is not cached in the cache system, or even if the business data is cached, the business data is evicted from the cache because the space of the cache system is full and the like. Therefore, the business data can be requested from a remote end and cached in the cache system, so as to realize the cache bottom.
In one embodiment, if the service data is not found in the cache system, after the cache data lock is obtained, the service data is found in the cache system again; and if the service data is not found, caching the service data from a remote end to the cache system.
In this embodiment, each cache data has a cache data lock, the service data request may carry a "key" value, and the key may be regarded as a key value of the cache system and is equivalent to a unique identifier of the service data in the cache. And when the service data request is received, judging whether the service data request acquires a cache data lock. And only the service data request of the cache data lock is acquired, the remote data is acquired for the service data request, so that the data interaction pressure of the remote end is reduced in a high concurrency scene. The reason why the service data is searched in the cache system again is that when a service data request gets to the cache data lock, it is possible that other threads have already acquired the service data from the remote end and stored the service data in the cache system. Therefore, unnecessary data interaction with the remote end can be avoided through the secondary judgment.
Further, in this embodiment, a timeout mechanism for acquiring the cache data lock may be provided. If the cache data lock is acquired overtime, the process is directly ended. The time-out mechanism is to prevent unnecessary waiting and thus wasting resources, and to prevent the extreme case that the request queue is waiting all the time and overload occurs to cause system avalanche.
In an embodiment, when the service data is cached from a remote end to the cache system, the first service data with a high importance level and the second service data with a low importance level may be cached in different areas isolated from each other in the cache system, respectively. The first service data may be service data corresponding to a service requiring high availability, such as payment service data. The tags may be set for the service data in advance, and the service data with different tags may be cached in different cache regions, respectively.
Further, the cache region for caching the first service data may be set to never evict data, and the cache region for caching the second service data may be set to evict the service data with the longest cache time. The method comprises the steps of isolating the key data from the near-end cache of the general data in a key data isolation mode, performing full cache on the key data, and performing eviction on the general data according to a cache strategy, wherein if a cache system is not enough to support the full data, the method can ensure that the key data (such as business data of key merchants) can be disaster-tolerant to a certain extent, and the long tail data can be disaster-tolerant to a certain extent.
In one embodiment, if the logic validity period of the service data is over, it may be determined whether the service data request carries a cache data lock; if yes, returning to the step of caching the service data from the remote end to the cache system. In this embodiment, for the service data with expired logic, the remote data may also be acquired through the cache data lock. Unlike the previous embodiment in which the cache data lock is set, the present embodiment may not set the timeout. Because the business data is stored in the cache system, the business data with the logic expiration can still be returned to the business although the logic expiration is finished, so that the high availability of the business is ensured. The purpose of obtaining the cache data lock without setting the timeout is to reduce the response time of the service data request and improve the performance of the system under the condition of physical storage hit.
If the logic validity period is not over, the service data searched in the cache system can be directly returned to the service system.
The physical validity period of the service data in the cache system is permanent. It should be noted that "permanently effective" herein means that the buffering time exceeds a certain time threshold, which may be a very long time (e.g. 1 year) as long as the implementation of the buffering disaster tolerance can be guaranteed.
The near-end cache logic of this embodiment may be encapsulated in one cache component, and when the service needs to acquire data, the service directly acquires the data through the cache component, and as for whether the data is acquired from the cache or acquired from the remote server side, the service does not need to sense any cache and disaster recovery logic. The program flow diagram of fig. 3 shows how the near-end caching component caches data and disaster recovery in case of an exception. The method specifically comprises the following steps:
step 302: and the service system acquires the cache data through the cache component.
Step 304: the cache component searches the service data from the cache system, and if the cache is broken down, step 320 is executed; if the cache hits (i.e., the business data is found in the cache), step 306 is performed.
Step 306: the service data request carries key values (keys) of the service data, each key value corresponds to a unique cache data lock, and only one key value is used for obtaining the cache data lock at each time point to obtain the data at a far end. After the cache is broken in step 304, the cache data lock corresponding to the key value is obtained, and then step 308 is performed.
Step 308: if the cache data lock is acquired, go to step 310; otherwise, step 318 is performed.
Step 310: since there may be other threads that have already go to the remote end to acquire the service data and store the service data in the cache system, after the cache data lock is acquired, it is determined again whether the cache is hit, if not, step 312 is performed, if hit, the data is returned to the service, and the process ends.
Step 312: the remote data is acquired and then step 314 is performed.
Step 314: the acquired far-end data is written into a near-end cache system, wherein the data written into the near-end cache is never outdated on physical storage, the data is triggered to be physically deleted only when the cache is full and data eviction is needed, and the logic expiration time of business is set. Step 316 is then performed.
Step 316: and returning the service data in the cache system to the service system. In this step, if the remote end has problems such as network abnormality and database crash, the mechanism is used to store the data in the cache system to ensure that the service can always acquire the data, thereby ensuring high availability of the service.
Step 318: if the acquired cache data lock is overtime, the process is directly ended.
Step 320: when the business data exists in the near-end cache system, firstly judging whether the logic of the cache data is expired, if so, performing step 322, otherwise, directly returning the cache data to the business system, and ending the process.
Step 322: similar to step 306, the cache data lock corresponding to the key value is obtained, and then step 324 is performed.
Step 324: if a cache data lock is acquired, go to step 326; otherwise, returning the service data and ending the process.
Step 326: the remote service data is acquired and step 328 is performed.
Step 328: if the acquisition is successful, go to step 314; if the acquisition fails (when the remote server is abnormal), step 316 is performed.
The scheme of the embodiment of the specification has at least the following advantages:
(1) under the condition of disaster, the data hit physically is used as disaster recovery data to carry out bottom reception, so that the high availability of the business is ensured.
(2) By setting the cache data lock, the remote end can be protected in a high concurrency scene.
(3) By setting a timeout mechanism, unnecessary waiting is prevented, thereby wasting resources, and in an extreme case, a request queue is prevented from waiting all the time, and overload occurs, thereby causing system avalanche.
(4) The first business data with higher importance degree and the second business data with lower importance degree are separately cached, the first business data is never evicted from the cache, and the first cached second business data is evicted only when the corresponding cache region is full, so that disaster tolerance of key businesses is ensured.
As shown in fig. 4, an embodiment of the present specification further provides a cache disaster recovery method, where the method includes:
step 402: if the remote service data is overtime, searching the service data in the cache system;
step 404: sending the searched service data to a service system;
wherein, the service data is cached to the cache system according to the data caching method of any embodiment.
Corresponding to the embodiment of the method, the embodiment of the specification also provides an embodiment of a device, a computing storage medium and a cache system.
As shown in fig. 5, fig. 5 is a block diagram of a data caching apparatus according to an exemplary embodiment shown in this specification, where the apparatus includes:
a first searching module 502, configured to search service data in a cache system when receiving a service data request sent by a service system;
a first obtaining module 504, configured to obtain a logic validity period of the service data if the service data is found; wherein the logical validity period is a validity period of the service data corresponding to a service;
the first caching module 506 is configured to cache the service data from a remote location to the caching system if the logical validity period has passed, and set a physical validity period of the service data in the caching system as a permanent validity period.
As shown in fig. 6, fig. 6 is a block diagram of a cache disaster recovery apparatus according to an exemplary embodiment shown in this specification, where the apparatus includes:
a second searching module 602, configured to search for the service data in the cache system if the remote service data is overtime;
a sending module 604, configured to send the found service data to a service system;
wherein, the service data is cached to the cache system according to the data caching method of any embodiment.
The specific details of the implementation process of the functions and actions of each module in the device are referred to the implementation process of the corresponding step in the method, and are not described herein again.
The embodiments of the apparatus of the present specification can be applied to a computer device, such as a server or a terminal device. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory for operation through the processor in which the file processing is located. From a hardware aspect, as shown in fig. 7, it is a hardware structure diagram of a computer device in which the apparatus of this specification is located, except for the processor 702, the memory 704, the network interface 706, and the nonvolatile memory 708 shown in fig. 7, a server or an electronic device in which the apparatus is located in an embodiment may also include other hardware according to an actual function of the computer device, which is not described again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the present specification. One of ordinary skill in the art can understand and implement it without inventive effort.
Accordingly, the embodiments of the present specification also provide a computer storage medium, in which a program is stored, and the program, when executed by a processor, implements the method in any of the above embodiments.
Accordingly, an embodiment of the present specification further provides a cache system, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any of the foregoing embodiments when executing the program.
Further, as shown in fig. 8, the cache system includes a general cache unit 802 and a white list cache unit 804; the white list caching unit 804 is configured to store first service data with a high importance degree, and the general caching unit 802 is configured to store second service data with a low importance degree; when the general cache unit 802 is full, the second service data cached first is evicted from the cache.
This application may take the form of a computer program product embodied on one or more storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of the storage medium of the computer include, but are not limited to: phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by a computing device.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
The above description is meant to be illustrative of the preferred embodiments of the present disclosure and not to be taken as limiting the disclosure, and any modifications, equivalents, improvements and the like that are within the spirit and scope of the present disclosure are intended to be included therein.