CN111444039A - Cache data rollback method and cache data rollback device - Google Patents
Cache data rollback method and cache data rollback device Download PDFInfo
- Publication number
- CN111444039A CN111444039A CN201911394738.XA CN201911394738A CN111444039A CN 111444039 A CN111444039 A CN 111444039A CN 201911394738 A CN201911394738 A CN 201911394738A CN 111444039 A CN111444039 A CN 111444039A
- Authority
- CN
- China
- Prior art keywords
- data
- cache
- rollback
- point
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
Abstract
The invention relates to a cache data rollback method and a cache data rollback device. The method comprises the following steps: a buffer loading step for loading data into a buffer, wherein the buffer loading step comprises: calculating the latest backup time point when the data is loaded into the cache; judging whether the latest data restoring point of the data exists in the cache, if not, continuing the following creating step, and if so, jumping to the following loading step; judging whether the data exists in the cache or not, if so, creating a data restoration point based on the currently existing data, and if not, identifying the non-existing data and creating the data restoration point; and a loading step of loading the data into the cache. According to the present invention, it is possible to provide a cache data rollback method and a cache data rollback apparatus that can quickly resume production when data is abnormal.
Description
Technical Field
The present invention relates to computer technologies, and in particular, to a cache data rollback method and a cache data rollback apparatus implemented based on computer technologies.
Background
The internal system merchant online query service system is mainly a system which is responsible for loading merchant data synchronized at the upstream into Redis and providing an interface for querying merchant information for an external system. Since wrong service data are input due to the problems of misoperation of a service party or unreliable external systems and the like in the previous production for many times, the production is influenced, in this case, the positioning problem and the production recovery time are long, so that the online inquiry service of the merchant is used as a data service type system, the problem of high availability of data in the scene needs to be considered, and the synchronization of the information data of the merchant has the characteristics of batch updating of the data and the like, and the system belongs to the scene where similar problems are easy to occur.
In the scenario of using such cache optimization query, data needs to be backed off once an exception occurs, and in the prior art, the method mainly focuses on backing off data in a database preferentially and then refreshing the data into a cache. Another idea is to directly correct the abnormal data and overlay it into the cache, and during the correction, the production system chooses to use the alternative service or directly interrupt the external service.
The existing technology has the disadvantages that the time overhead of data rollback and recovery is too long, the complexity is high, the operations such as database rollback and cache data refreshing are involved, even if the database rollback is not performed but direct repair is selected, the range of the affected data is determined quickly in a short time with certain difficulty, and finally the emergency goal of short-time recovery production cannot be realized. In addition, in the prior art, a service interruption influence is caused under a service scene with higher query requirement and frequency.
Disclosure of Invention
In view of the above problems, the present invention is directed to a cache data rollback method and a cache data rollback apparatus capable of quickly resuming production when data is abnormal.
The cache data rollback method is characterized by comprising the following steps: a buffer loading step for loading data into the buffer,
wherein the cache loading step comprises:
a calculation step of calculating a latest backup time point when data is loaded to a cache;
judging whether the latest data restoring point of the data exists in the cache or not, if not, continuing the following creating step, and if so, jumping to the following loading step;
a creation step of judging whether the data exists in the cache, if so, creating a data restoration point based on the currently existing data, and if not, identifying the non-existing data and creating the data restoration point; and
a loading step of loading the data into a cache,
wherein, the backup time point is calculated from a specified time, and is a backup time point when the time is increased by one backup period,
the data restore point refers to a value representing a copy of data at a backup point in time, created when the value is to be modified.
Optionally, in the creating step, it is determined whether the data exists in the cache, if the data exists in the cache, the currently existing data is acquired, a data restoring point is created based on the currently existing data, and if the data does not exist in the cache, the data restoring point is created by using data with a special fixed value, where the data with the special fixed value is used to identify that the data does not exist at the latest backup time point.
Optionally, any one of the data restore points is set with a backup failure time, and when the backup failure time is reached, the data restore point is automatically cleared from the cache.
Optionally, the naming rule of the data restore point in the cache is as follows: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, rather than the creation time.
Optionally, after the loading step, the method further comprises:
and a cache query step, which is used for querying data from the cache.
Optionally, the cache querying step includes:
setting a rollback flag, wherein the rollback flag is used for identifying whether to rollback or not and identifying a rollback time point;
and an inquiry step, namely judging whether the backspacing mark is set up or not, and inquiring data based on the backspacing mark.
Optionally, the querying step includes:
if the backspacing mark is marked as not backspacing, inquiring current data in normally accessed cache;
if the rollback flag is identified as rollback, inquiring whether the data exists at the rollback time point identified by the rollback flag, if not, indicating that the data to be inquired is not modified at the rollback time point, and therefore, trying to inquire the data version of the latest backup time point after the rollback time point again until the data version is inquired; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
Optionally, the querying of the rollback time point identified by the rollback flag refers to a backup time point calculated according to the rollback flag.
Optionally, the querying step further includes a number-of-rollback querying step, where multiple queries in rollback are implemented by L UA scripts.
Optionally, after the loading step, the method further comprises:
and an exception rollback repairing step, namely when data exception is found, incrementally refreshing the data in the database into a cache after the data in the database is repaired.
Optionally, the exception rollback repairing step includes:
when data are found to be abnormal, setting rollback marks of all query nodes, and assigning values according to backup time points to which rollback is required;
positioning data abnormity problems, and repairing data in a database;
incrementally refreshing repaired data in the database into a cache; and
the rollback flags of all querying nodes are reset.
The cache data rollback device of the present invention is characterized by comprising:
a buffer load module for loading data into the buffer,
wherein the cache load module comprises:
the calculation module is used for calculating the latest backup time point when the data are loaded to the cache;
the judging module is used for judging whether the latest data restoring point of the data exists in the cache, if the latest data restoring point does not exist, the following creating module is continued, and if the latest data restoring point exists, the loading module is jumped to;
a creating module, configured to determine whether the data exists in the cache if the latest data restoring point does not exist, create a data restoring point based on the currently existing data if the data exists in the cache, and identify the non-existing data and create a data restoring point if the data does not exist in the cache; and
a loading module for loading data into the cache,
wherein, the backup time point is calculated from a specified time, and is a backup time point when the time is increased by one backup period,
the data restore point refers to a value representing a copy of data at a backup point in time, created when the value is to be modified.
Optionally, in the creating module, it is determined whether the data exists in the cache, if the data exists in the cache, the currently existing data is obtained, a data restoring point is created based on the currently existing data, and if the data does not exist in the cache, the data restoring point is created by using data with a special fixed value, where the data with the special fixed value is used to identify that the data does not exist at the latest backup time point.
Optionally, any one of the data restore points is set with a backup failure time, and when the backup failure time is reached, the data restore point is automatically cleared from the cache.
Optionally, the naming rule of the data restore point in the cache is as follows: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, rather than the creation time.
Optionally, further comprising:
and the cache query module is used for querying data from the cache.
Optionally, the cache query module includes:
the device comprises a setting module, a back-off module and a control module, wherein the back-off module is used for setting a back-off mark, and the back-off mark is used for identifying whether to back off and identifying a back-off time point; and
and the query module is used for judging whether the backspacing mark is set up or not and carrying out data query based on the backspacing mark.
Optionally, the query module performs the following actions:
if the backspacing mark is marked as not backspacing, inquiring current data in normally accessed cache;
if the rollback flag is identified as rollback, inquiring whether the data exists at the rollback time point identified by the rollback flag, if not, indicating that the data to be inquired is not modified at the rollback time point, and therefore, trying to inquire the data version of the latest backup time point after the rollback time point again until the data version is inquired; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
Optionally, the querying of the rollback time point identified by the rollback flag refers to a backup time point calculated according to the rollback flag.
Optionally, the query module further comprises:
and a rollback number query module for implementing multiple queries in rollback through L UA scripts.
Optionally, further comprising:
and the exception rollback repairing module is used for repairing the data in the database and then incrementally refreshing the data into the cache when the data are found to be abnormal.
Optionally, the exception rollback repairing module includes:
the assignment module is used for setting the backspacing marks of all the query nodes under the condition of finding out data abnormality and assigning the backup time points to which the data need to be backspaced;
the repairing module is used for positioning the data abnormity problem and repairing the data in the database;
the refreshing module is used for refreshing the repaired data increment in the database into a cache; and
and the resetting module is used for resetting the rollback marks of all the query nodes.
The computer-readable medium of the present invention, on which a computer program is stored, is characterized in that,
the computer program, when executed by a processor, implements the cache data rollback method described above.
The computer device of the present invention includes a storage module, a processor, and a computer program stored on the storage module and executable on the processor, and is characterized in that the processor implements the above cache data rollback method when executing the computer program.
As described above, according to the cache data rollback method and the cache data rollback device of the present invention, the problem that fast rollback cannot be performed when cache data is abnormal can be solved, second-level rollback and fast service recovery can be achieved, the method implementation cost is low, rollback operation does not involve data modification, consistency between a database and a cache is not broken, and reloading after data cache needs to be emptied due to modification operation of cache data for rollback in subsequent data recovery is also avoided.
Drawings
Fig. 1 is a flowchart illustrating a cache loading step in a cache data rollback method according to an embodiment of the present invention.
Fig. 2 is a flowchart illustrating a cache query step in a cache data rollback method according to an embodiment of the present invention.
Fig. 3 is a flowchart illustrating an exception rollback repair procedure in the cache data rollback method according to an embodiment of the present invention.
Fig. 4 is a schematic diagram illustrating a cache loading step and a cache query step in the cache data rollback method according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram illustrating a cache data rollback apparatus according to an embodiment of the present invention.
FIG. 6 shows a schematic diagram of the relationship between an external system, a query node, a Redis cache, a load module, and a database.
Detailed Description
The following description is of some of the several embodiments of the invention and is intended to provide a basic understanding of the invention. It is not intended to identify key or critical elements of the invention or to delineate the scope of the invention.
For the purposes of brevity and explanation, the principles of the present invention are described herein with reference primarily to exemplary embodiments thereof. However, those skilled in the art will readily recognize that the same principles are equally applicable to all types of cache data rollback methods and cache data rollback apparatuses, and that these same principles, as well as any such variations, may be implemented therein without departing from the true spirit and scope of the present patent application.
Moreover, in the following description, reference is made to the accompanying drawings that illustrate certain exemplary embodiments. Electrical, mechanical, logical, and structural changes may be made to these embodiments without departing from the spirit and scope of the invention. In addition, while a feature of the invention may have been disclosed with respect to only one of several implementations/embodiments, such feature may be combined with one or more other features of the other implementations/embodiments as may be desired and/or advantageous for any given or identified function. The following description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims and their equivalents.
Terms such as "comprising" and "comprises" mean that, in addition to having elements (modules) and steps that are directly and explicitly stated in the description and claims, the solution of the invention does not exclude the presence of other elements (modules) and steps that are not directly or explicitly stated.
The invention provides a cache data rollback method and a cache data rollback device, wherein cache data can be quickly rolled back. The concept of the present invention is divided into a cache loading mechanism, a cache access mechanism and an exception rollback repair mechanism, and specific step contents and apparatuses will be described below one by one, for convenience of description, the following description will use Redis as an example of a cache database, but the present invention is not limited to the Redis cache database, and can also be applied to type cache databases.
Several technical terms to be presented in the present invention will be explained herein.
And (3) backup period: minimum time interval for data to be backed up twice consecutively.
Backup time point: the time is counted from a specified time, and each time the backup period is increased, the backup time point is defined. The start time is the backup time point.
Data reduction point: representing the value of a copy of data at a backup point in time, created when the value is to be modified. Key naming rule of restore points in Redis: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, instead of the creation time; the time stamp is connected with the key by ". -". If the restoring point of the backup time point exists, the backup time point is not repeatedly created.
Backup failure time: the time interval from creation to deletion of the data restore point. Each data restore point must set a backup expiration time.
A rollback flag: and a global flag, wherein the value of the global flag is 0 to indicate that the data is not rolled back, the value of the global flag is 1 to indicate that the data is rolled back to the previous version, namely the data of the latest data restoring point is used, and the value of the global flag is 2 to indicate that the data is rolled back by one version. The restoration point data of the backup time point corresponding to the rollback flag value must not be invalidated.
Here, an environment in which the cache data rollback method and the cache data rollback apparatus according to the present invention are applied will be described.
FIG. 6 is a diagram illustrating the relationship between external systems, query nodes, Redis clusters, load modules, and databases.
As shown in fig. 6, the loading module acquires data from the database ("1 acquisition data" in fig. 6), loads the acquired data into the Redis cache ("2 load Redis" in fig. 6), queries the query node 1 to the query node 4 from the external system for data ("3 query data" in fig. 6), and then the query node accesses the data in the Redis cluster ("4 access data" in fig. 6). The cache data rollback method and the cache data rollback device are applied to an environment represented by a broken line frame in fig. 6.
First, the concept of the present invention will be explained.
(1) Cache loading mechanism
In order to ensure fast cache rollback, when data is loaded into the cache, the data in the historical state needs to be saved. Therefore, for the modified data in the cache, the old value is backed up first, the current data is accessed during normal query, and only when the backspacing is needed, the historical state data is preferentially accessed during query.
When data is backed up once, a data restoring point is not created every time the data is modified, but is created in a time dimension, namely, a data restoring point is created every too long time, because the rollback degree is determined according to which time the data runs smoothly in the invention, but not the version of the data.
When a data restore point is created, not all data is created, but a data restore point is created if a data is first modified after a backup time point is passed. If no modification occurs during two consecutive backup time point periods, no backup is performed.
The method can effectively backup data according to the time dimension, and can avoid unnecessary data backup and reduce the expenditure of computing resources and memory resources.
(2) Cache access mechanism
When rollback is needed, the behavior of the query side is changed by modifying the mark, so that the aim of preferentially querying the data of the backup version is fulfilled;
when not backing, inquiring the current data in the normal access cache; when the backspacing is found to be needed, the query firstly accesses the data version of the backspacing time point, if the data version of the backspacing time point is not queried, the data version of the data to be queried is not modified at the backspacing time point, and therefore the query is tried again to query the data version of the latest backup time point after the backspacing time point until the data version is queried; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
(3) Abnormal rollback repair mechanism
In the invention, the rollback operation does not modify the data in the cache, so that the data consistency between the cache and the database is always maintained, and the data content actually accessed by the query side during the rollback period is not influenced even if the modification operation of the database data is synchronously updated into the cache during the rollback period.
Based on the above, the back-end problem locating and repairing process is isolated for the front-end service query. If the synchronization strategy from the database to the cache is not interrupted in the repair process, the cache is synchronously repaired when the database data is repaired; if the synchronization strategy is closed, the problem is repaired, and the cache data repair can be completed only by updating the increment modification into the cache.
If the repair process can be completed before the next point in time of backup comes, the abnormal data will not be backed up; otherwise, the abnormal data can be backed up, but the abnormal data can be automatically cleared as long as the backup failure time is up.
After the cache is repaired, the latest data can be accessed immediately only by resetting the rollback flag.
Next, a cache data rollback method according to the present invention will be described.
First, a cache loading step is explained.
The cache data rollback method of one aspect of the invention comprises the following steps: a cache loading step for loading data into the cache.
The cache loading step comprises:
a calculation step of calculating a latest backup time point when data is loaded to a cache;
judging whether the latest data restoring point of the data exists in the cache or not, if not, continuing the following creating step, and if so, jumping to the following loading step;
a creation step of judging whether the data exists in the cache, if so, creating a data restoration point based on the currently existing data, and if not, identifying the non-existing data and creating the data restoration point; and
a loading step of loading the data into a cache,
wherein, the backup time point is calculated from a specified time, and is a backup time point when the time is increased by one backup period,
the data restore point refers to a value representing a copy of data at a backup point in time, created when the value is to be modified.
In the creating step, whether the data exists in the cache is judged, if the data exists in the cache, the currently existing data is acquired, a data restoring point is created based on the currently existing data, and if the data does not exist in the cache, the data restoring point is created by adopting data with a special fixed value, wherein the data with the special fixed value is used for identifying that the data does not exist at the latest backup time point. The special fixed value can be customized, and as long as the value is found in query, the backup time point does not exist.
In the invention, any data restore point is set with backup failure time, and when the backup failure time is reached, the data restore point is automatically cleared from the cache.
The naming rule of the data recovery point in the cache is as follows: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, rather than the creation time.
Next, a cache access mechanism in the cache data rollback method of the present invention is described.
Further comprising after the loading step:
setting a rollback flag, wherein the rollback flag is used for identifying whether to rollback or not and identifying a rollback time point; and
and an inquiry step, namely judging whether the backspacing mark is set up or not, and inquiring data based on the backspacing mark.
Wherein the querying step comprises:
if the backspacing mark is marked as not backspacing, inquiring current data in normally accessed cache;
if the rollback flag is identified as rollback, inquiring whether the data exists at the rollback time point identified by the rollback flag, if not, indicating that the data to be inquired is not modified at the rollback time point, and therefore, trying to inquire the data version of the latest backup time point after the rollback time point again until the data version is inquired; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
And querying the rollback time point identified by the rollback flag refers to calculating a backup time point according to the rollback flag.
Wherein the querying step further comprises:
and a number-of-rollback query substep, wherein a plurality of queries in rollback are realized through L UA scripts.
Again, the exception rollback repair mechanism of the present invention is explained.
The cache data rollback method further comprises the following steps:
and an exception rollback repairing step, namely when data exception is found, incrementally refreshing the data in the database into a cache after the data in the database is repaired.
The abnormal rollback repairing step comprises the following steps:
when data are found to be abnormal, setting rollback marks of all query nodes, and assigning values according to backup time points to which rollback is required;
positioning data abnormity problems, and repairing data in a database;
incrementally refreshing repaired data in the database into a cache; and
the rollback flags of all querying nodes are reset.
Next, a cache data rollback method according to an embodiment of the present invention will be described.
First, a buffer loading step is explained.
Fig. 1 is a flowchart illustrating a cache loading step in a cache data rollback method according to an embodiment of the present invention.
As shown in fig. 1, the cache loading step in the cache data rollback method according to an embodiment of the present invention includes the following steps:
step S1: when loading data to Redis, firstly calculating the latest backup time point < timestamp >;
step S2: judging whether the latest data reduction point < timestamp >. key exists, if so, jumping to the step S7, and if not, continuing to the step S3;
step S3: judging whether the piece of data exists in the Redis, if so, continuing to the step S4, and if not, jumping to the step S6;
step S4: acquiring the current existing data, namely directly copying one current data;
step S5: setting a data key value generated by copying to be < timestamp >. key, namely, establishing a data restoring point based on the current data;
step S6: if the piece of data does not exist in Redis, a value is used for creating a restore point for the data with a special fixed value, and the restore point is used for identifying that the data does not exist at the latest backup time point;
step S7: load the data into Redis.
In the invention, any data restore point is set with backup failure time, and when the restore point fails, the restore point is automatically cleared from Redis.
As described above, with the above-described cache loading step, it is possible to realize:
(1) if any piece of data is modified (added, deleted, changed or checked), backup is attempted;
(2) if one data is modified for many times in two backup time points, only the old value before the first modification is reserved, and the subsequent backup is not carried out, so that unnecessary resource overhead can be effectively saved under the conditions of no modification and no backup;
(3) because the data restoring point is set with backup failure time, the backup data is automatically deleted after being expired, and the resources are not occupied for a long time.
Next, the cache lookup procedure is described.
Fig. 2 is a flowchart illustrating a cache query step in a cache data rollback method according to an embodiment of the present invention.
As shown in fig. 2, the cache query step in the cache data rollback method according to an embodiment of the present invention includes:
step S11: when data is inquired for Redis, judging whether a backspacing mark is 0, if so, indicating that the backspacing mark is not set, jumping to step S17, otherwise, continuing to step S12;
step S12: when the rollback flag is set, the backup time point needs to be rolled back to (for example, the rollback flag is 1, the backup time point is a latest backup time point, and so on), wherein the rollback flag may be manually set by an operation and maintenance worker, for example, the system management page initiation or the login server host initiation, and each data type may have its own independent rollback flag, and the data types are divided according to whether the data types are independent or not during updating;
step S13: querying the Redis for the value of the corresponding data reduction point, namely the value corresponding to the < timestamp >. key, if the result is found, jumping to the step S16, and if the result is not found, continuing to the step S14;
step S14: if the query is not found, calculating a backup time point after the backup time point, if the backup time point does not exist, the query is not found, jumping to the step S17, otherwise, continuing to the step S15;
step S15: inquiring restoring point data of the next backup time point, and returning to the step S13;
step S16: directly inquiring current data in a Redis cache;
step S17: directly inquiring current data in a Redis cache;
step S18: and returning a data result.
As described above, with the above-described cache loading step, it is possible to realize:
(1) when the backspacing mark is not set, normal data query is carried out, and no extra overhead exists;
(2) after the rollback flag is set, the query mode is equivalent to that the data in Redis immediately backs to the version of the specified backup time point;
(3) after the rollback flag is reset to 0, the query is immediately recovered to normal;
(4) data rollback does not modify data in the cache and does not break data consistency between the database and the cache.
In the above cache loading step, if the rollback flag is set, the number of times of querying data that has not been modified between the backup time point and the current time may be increased, increasing the time consumed for querying, and using some ways to reduce the influence of the problem as much as possible, taking a Redis scenario as an example, multiple queries during rollback may be implemented by using L UA scripts, so that the additional overhead may be greatly reduced.
Next, the procedure of recovering the abnormal rollback will be described.
Fig. 3 is a flowchart illustrating an exception rollback repair procedure in the cache data rollback method according to an embodiment of the present invention.
Step S31: when data are found to be abnormal, setting rollback marks of all query nodes at the first time, and assigning values according to backup time points needing to be backed up;
step S32: positioning data abnormity problems, and repairing data in a database;
step S33: after the data in the database is repaired, refreshing the repaired data increment into a cache;
step S34: resetting the rollback marks of all the query nodes and recovering normal query.
Here, due to the characteristics of the loading mechanism, the abnormal data may be used as a data restoring point, but may be automatically deleted after the expiration time elapses. In the abnormal rollback repairing step, the data can be rolled back to the historical version in a second level after the data has problems, so that external query cannot be influenced by the problem data. Meanwhile, the background data problem positioning and repairing process is isolated from the front-end service, service interruption is avoided, and no data modification operation and extra overhead are required during backspacing. For example, the system of the present invention provides a data query service for the external system a, and if problem data occurs, a will find the problem data, and thus be affected, for example, a transaction cannot be made. The front-end service here refers to the query service of the external system a. However, the problem data must be modified recently, and then after the rollback flag is set, the access to the problem data is directed to the corresponding correct historical data, so that the problem data cannot be affected. Fig. 4 is a schematic diagram illustrating specific contents of a cache loading step and a cache querying step in the cache data rollback method according to an embodiment of the present invention.
Regarding the cache loading step, please refer to the flow portion of the solid line in fig. 4, as shown in fig. 4, when data e is loaded, it is found that the latest data reduction point (i.e., the data reduction point of 20181231) does not have data of e, but data e already exists in the Redis cache, so that the data reduction point 20190101.e is created with old data e, then the data e in the Redis cache is overwritten with the inserted new data, and then the cache loading step ends.
For the cache query step, please refer to the flow portion of the dashed and dotted lines in fig. 4, wherein the dashed flow is the process of the normal query, and the dotted flow is the query process when the rollback flag value is 2.
As shown by the flow of the dotted line, in the case that the rollback flag is 0, that is, in the normal case, the data b is directly queried in the Redis cache.
When the rollback flag is 2, as shown in the dash-dot flow, a backup time point closest to the current time, namely 20181231, is calculated, whether 20181231.b exists or not is queried, and if so, a backup time point after the backup time point is calculated, namely 20181231, whether 20190101.b exists or not is queried, and a query result is returned. The query result returned at this time is 20181231 of the real data of b at this time point.
The above description is directed to a cache data rollback method according to the present invention, and the following description is directed to a cache data rollback apparatus according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram illustrating a cache data rollback apparatus according to an embodiment of the present invention.
As shown in fig. 5, a cache data rollback apparatus according to an embodiment of the present invention includes:
a buffer loading module 100 for loading data into a buffer;
a cache query module 200, configured to query the cache for data; and
an exception rollback repairing module 300, configured to repair the data in the database when the data exception is found, and then perform incremental refresh into the cache.
In addition, the exception rollback repairing module 300 may use an incremental data refreshing interface, not shown, to implement incremental refreshing through the incremental data refreshing interface.
Wherein the cache load module 100 includes:
a calculating module 110, configured to calculate a latest backup time point when the data is loaded into the cache;
a judging module 120, configured to judge whether a latest data restoring point of the data exists in the cache;
a creating module 130, configured to determine whether the data exists in the cache if the determining module determines that the latest data restoring point does not exist, if the data exists in the cache, create a data restoring point based on the currently existing data, and if the data does not exist in the cache, identify the non-existing data and create a data restoring point; and
a loading module 140 for loading the data into the cache if the latest data restore point already exists or is created,
wherein, the backup time point is calculated from a specified time, and is a backup time point when the time is increased by one backup period,
the data restore point refers to a value representing a copy of data at a backup point in time, created when the value is to be modified.
In the creating module 130, it is determined whether the data exists in the cache, if the data exists in the cache, the currently existing data is obtained, a data restoring point is created based on the currently existing data, and if the data does not exist in the cache, the data restoring point is created by using data with a special fixed value, where the data with the special fixed value is used to identify that the data does not exist at the latest backup time point.
In the invention, any data restoring point is set with backup failure time, and when the backup failure time is reached, the data restoring point is automatically cleared from the cache.
Moreover, the naming rule of the data restore point in the cache is as follows: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, rather than the creation time.
Further, the cache query module 200 includes:
a setting module 210, configured to set a rollback flag, where the rollback flag is used to identify whether to rollback and to identify a rollback time point; and
the query module 220 determines whether the rollback flag is set, and performs data query based on the rollback flag.
Wherein the query module 220 performs the following actions:
if the backspacing mark is marked as not backspacing, inquiring current data in normally accessed cache;
if the rollback flag is identified as rollback, inquiring whether the data exists at the rollback time point identified by the rollback flag, if not, indicating that the data to be inquired is not modified at the rollback time point, and therefore, trying to inquire the data version of the latest backup time point after the rollback time point again until the data version is inquired; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
And querying the rollback time point identified by the rollback flag refers to calculating a backup time point according to the rollback flag.
The query module 220 may further include a rollback number query module that implements multiple queries during rollback through L UA scripts, where the multiple queries during rollback through L UA scripts are implemented because the backup node data meeting the conditions is sequentially queried (twice queries are needed with reference to the dashed query route in fig. 4), and it may be seen that multiple Redis data may be accessed by one data query, which may cause additional network overhead.
In addition, L UA needs to be used to consider the distributed situation, because in the distributed Redis cluster scenario, data may be stored in different Redis slices, and L UA needs to ensure that data accessed in one script is all in the same slice, so it needs to ensure that one piece of data and all backup point data thereof are stored in the same slice, which can be implemented by the client itself, or implemented by using the proxy of Redis to perform slice rule setting.
The exception rollback recovery module 300 includes:
the assignment module 310 is configured to set rollback flags of all query nodes when data abnormality is found, and assign a value according to backup time points to which rollback is required;
the repairing module 320 is used for positioning data abnormity problems and repairing data in the database;
a refresh module 330 for incrementally refreshing the repaired data in the database into the cache; and
a resetting module 340, configured to reset the rollback flags of all querying nodes.
The invention also provides a cache data rollback method implemented by the computer program when the computer program is executed by the processor.
The invention also provides a computer device, which comprises a storage module, a processor and a computer program stored on the storage module and capable of running on the processor, and is characterized in that the processor implements the cache data rollback method when executing the computer program.
In the invention, the reds data can be quickly recovered only by increment and a small amount of coverage without refreshing the full data into the reds, and most systems still remain in the mode of refreshing the reds data after the database is repaired in the existing scene using the reds. Moreover, an incremental data refresh interface is also provided in the present invention, i.e. as long as the incremental data content is provided, incremental refresh can be realized through the interface.
Although the above description has been given by taking a Redis database as an example, the cache data rollback method and the cache data rollback apparatus of the present invention are also applicable to other databases in the key-value format, such as memcache.
As described above, according to the cache data rollback method and the cache data rollback device of the present invention, the problem that fast rollback cannot be performed when cache data is abnormal can be solved, second-level rollback and fast service recovery can be achieved, the method implementation cost is low, rollback operation does not involve data modification, consistency between a database and a cache is not broken, and reloading after data cache needs to be emptied due to modification operation of cache data for rollback in subsequent data recovery is also avoided. Moreover, the subsequent data repair process can be isolated from the service, the data operation is not carried out in the rollback process, extra overhead is not needed in the recovery process, and powerful guarantee is provided for high availability of the cache data.
The above examples mainly illustrate the cache data rollback method and the cache data rollback apparatus of the present invention. Although only a few embodiments of the present invention have been described in detail, those skilled in the art will appreciate that the present invention may be embodied in many other forms without departing from the spirit or scope thereof. Accordingly, the present examples and embodiments are to be considered as illustrative and not restrictive, and various modifications and substitutions may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims.
Claims (24)
1. A cache data rollback method, comprising: a buffer loading step of loading data into a buffer,
wherein the cache loading step comprises:
a calculation step of calculating a latest backup time point when data is loaded to a cache;
judging whether the latest data restoring point of the data exists in the cache or not, if not, continuing the following creating step, and if so, jumping to the following loading step;
a creation step of judging whether the data exists in the cache, if so, creating a data restoration point based on the currently existing data, and if not, identifying the non-existing data and creating the data restoration point; and
a loading step of loading the data into a cache,
wherein, the backup time point is calculated from a specified time, and is a backup time point when the time is increased by one backup period,
the data restore point refers to a value representing a copy of data at a backup point in time, created when the value is to be modified.
2. The cache data rollback method of claim 1,
in the creating step, whether the data exists in the cache is judged, if the data exists in the cache, the currently existing data is obtained, a data restoring point is created based on the currently existing data, and if the data does not exist in the cache, the data restoring point is created by adopting data with a special fixed value, wherein the data with the special fixed value is used for identifying that the data does not exist at the latest backup time point.
3. The cache data rollback method of claim 1,
and any one data restoring point is set with backup failure time, and when the backup failure time is reached, the data restoring point is automatically cleared from the cache.
4. The cache data rollback method of claim 1,
the naming rule of the data recovery point in the cache is as follows: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, rather than the creation time.
5. The cache data rollback method according to any one of claims 1-4, further comprising, after the loading step:
and a cache query step, which is used for querying data from the cache.
6. The cache data rollback method of claim 5, wherein the cache lookup step comprises:
setting a rollback flag, wherein the rollback flag is used for identifying whether to rollback or not and identifying a rollback time point;
and an inquiry step, namely judging whether the backspacing mark is set up or not, and inquiring data based on the backspacing mark.
7. The cache data rollback method of claim 6, wherein the querying step comprises:
if the backspacing mark is marked as not backspacing, inquiring current data in normally accessed cache;
if the rollback flag is identified as rollback, inquiring whether the data exists at the rollback time point identified by the rollback flag, if not, indicating that the data to be inquired is not modified at the rollback time point, and therefore, trying to inquire the data version of the latest backup time point after the rollback time point again until the data version is inquired; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
8. The cache data rollback method of claim 7,
and the step of inquiring the back-off time point identified by the back-off mark refers to a backup time point calculated according to the back-off mark.
9. The cache data rollback method of claim 7, wherein the querying step further comprises:
and a step of inquiring the number of backspacing, wherein a plurality of inquiries in backspacing are realized through L UA scripts.
10. The cache data rollback method according to any one of claims 1-4, further comprising, after the loading step:
and an exception rollback repairing step, namely when data exception is found, incrementally refreshing the data in the database into a cache after the data in the database is repaired.
11. The cache data rollback method of claim 10, wherein the exception rollback repair step comprises:
when data are found to be abnormal, setting rollback marks of all query nodes, and assigning values according to backup time points to which rollback is required;
positioning data abnormity problems, and repairing data in a database;
incrementally refreshing repaired data in the database into a cache; and
the rollback flags of all querying nodes are reset.
12. A cache data rollback apparatus, comprising:
a buffer load module for loading data into the buffer,
wherein the cache load module comprises:
the calculation module is used for calculating the latest backup time point when the data are loaded to the cache;
the judging module is used for judging whether the latest data restoring point of the data exists in the cache, if the latest data restoring point does not exist, the following creating module is continued, and if the latest data restoring point exists, the loading module is jumped to;
a creating module, configured to determine whether the data exists in the cache if the latest data restoring point does not exist, create a data restoring point based on the currently existing data if the data exists in the cache, and identify the non-existing data and create a data restoring point if the data does not exist in the cache; and
a loading module for loading data into the cache,
wherein, the backup time point is calculated from a specified time, and is a backup time point when the time is increased by one backup period,
the data restore point refers to a value representing a copy of data at a backup point in time, created when the value is to be modified.
13. The cache data rollback apparatus of claim 12,
and in the creating module, judging whether the data exists in the cache or not, if so, acquiring the currently existing data, creating a data restoring point based on the currently existing data, and if not, creating the data restoring point by adopting data with a special fixed value, wherein the data with the special fixed value is used for identifying that the data does not exist at the latest backup time point.
14. The cache data rollback apparatus of claim 12,
and any one data restoring point is set with backup failure time, and when the backup failure time is reached, the data restoring point is automatically cleared from the cache.
15. The cache data rollback apparatus of claim 12,
the naming rule of the data recovery point in the cache is as follows: key, where < timestamp > is a timestamp of a backup time point prior to and closest to the creation time, rather than the creation time.
16. The cache data rollback apparatus of any one of claims 12-15, further comprising:
and the cache query module is used for querying data from the cache.
17. The cache data rollback apparatus of claim 16, wherein the cache query module comprises:
the device comprises a setting module, a back-off module and a control module, wherein the back-off module is used for setting a back-off mark, and the back-off mark is used for identifying whether to back off and identifying a back-off time point; and
and the query module is used for judging whether the backspacing mark is set up or not and carrying out data query based on the backspacing mark.
18. The cache data rollback apparatus of claim 17, wherein the query module performs the following:
if the backspacing mark is marked as not backspacing, inquiring current data in normally accessed cache;
if the rollback flag is identified as rollback, inquiring whether the data exists at the rollback time point identified by the rollback flag, if not, indicating that the data to be inquired is not modified at the rollback time point, and therefore, trying to inquire the data version of the latest backup time point after the rollback time point again until the data version is inquired; if the current latest data version is searched up to the last time but is not searched, the data to be searched does not exist in the cache.
19. The cache data rollback apparatus of claim 18,
and the step of inquiring the back-off time point identified by the back-off mark refers to a backup time point calculated according to the back-off mark.
20. The cache data rollback apparatus of claim 19, wherein the query module further comprises:
and a rollback number query module for implementing multiple queries in rollback through L UA scripts.
21. The cache data rollback apparatus of any one of claims 12-15, further comprising:
and the exception rollback repairing module is used for repairing the data in the database and then incrementally refreshing the data into the cache when the data are found to be abnormal.
22. The cache data rollback apparatus of claim 21, wherein the exception rollback repair module comprises:
the assignment module is used for setting the backspacing marks of all the query nodes under the condition of finding out data abnormality and assigning the backup time points to which the data need to be backspaced;
the repairing module is used for positioning the data abnormity problem and repairing the data in the database;
the refreshing module is used for refreshing the repaired data increment in the database into a cache; and
and the resetting module is used for resetting the rollback marks of all the query nodes.
23. A computer-readable medium, having stored thereon a computer program,
the computer program, when executed by a processor, implements the cache data rollback method of any of claims 1-11.
24. A computer device comprising a storage module, a processor and a computer program stored on the storage module and executable on the processor, wherein the processor implements the cache data rollback method according to any one of claims 1-11 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911394738.XA CN111444039B (en) | 2019-12-30 | 2019-12-30 | Cache data rollback method and cache data rollback device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911394738.XA CN111444039B (en) | 2019-12-30 | 2019-12-30 | Cache data rollback method and cache data rollback device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111444039A true CN111444039A (en) | 2020-07-24 |
| CN111444039B CN111444039B (en) | 2023-03-21 |
Family
ID=71648662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911394738.XA Active CN111444039B (en) | 2019-12-30 | 2019-12-30 | Cache data rollback method and cache data rollback device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111444039B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022048495A1 (en) * | 2020-09-01 | 2022-03-10 | 阿里云计算有限公司 | Data backup method and apparatus, data recovery method and apparatus, and electronic device |
| CN114218013A (en) * | 2021-12-13 | 2022-03-22 | 北京字节跳动网络技术有限公司 | Searching method, searching device and electronic equipment storage medium |
| CN114816845A (en) * | 2022-04-06 | 2022-07-29 | 在线途游（北京）科技有限公司 | MongoDB-based rapid data rollback method and device |
| CN114816845B (en) * | 2022-04-06 | 2024-05-10 | 在线途游（北京）科技有限公司 | MongoDB-based rapid data rollback method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5437026A (en) * | 1989-06-13 | 1995-07-25 | International Business Machines Corporation | Removing uncommitted changes made to stored data by a database management system |
| US6185577B1 (en) * | 1998-06-23 | 2001-02-06 | Oracle Corporation | Method and apparatus for incremental undo |
| US20070288526A1 (en) * | 2006-06-08 | 2007-12-13 | Emc Corporation | Method and apparatus for processing a database replica |
| CN101179437A (en) * | 2007-11-29 | 2008-05-14 | 上海华为技术有限公司 | Method and apparatus for implementing preactivation of batch configuration |
| CN102387204A (en) * | 2011-10-21 | 2012-03-21 | 中国科学院计算技术研究所 | Method and system for maintaining consistency of cluster caching |
| CN103500204A (en) * | 2013-09-27 | 2014-01-08 | 北京京东尚科信息技术有限公司 | Web surfing backspacing method, web server and client side |
| CN104133783A (en) * | 2014-07-11 | 2014-11-05 | 北京京东尚科信息技术有限公司 | Method and device for processing distributed cache data |
| CN106599199A (en) * | 2016-12-14 | 2017-04-26 | 国云科技股份有限公司 | Data caching and synchronization method |
| US20170249375A1 (en) * | 2016-02-29 | 2017-08-31 | International Business Machines Corporation | Continuous Automatic Update Statistics Evaluation Using Change Data Capture Techniques |
| CN107193947A (en) * | 2017-05-22 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of file system cache incremental refreshment method and system |
| US20190394138A1 (en) * | 2018-06-26 | 2019-12-26 | International Business Machines Corporation | Guaranteed delivery in receiver side overcommitted communication adapters |
-
2019
- 2019-12-30 CN CN201911394738.XA patent/CN111444039B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5437026A (en) * | 1989-06-13 | 1995-07-25 | International Business Machines Corporation | Removing uncommitted changes made to stored data by a database management system |
| US6185577B1 (en) * | 1998-06-23 | 2001-02-06 | Oracle Corporation | Method and apparatus for incremental undo |
| US20070288526A1 (en) * | 2006-06-08 | 2007-12-13 | Emc Corporation | Method and apparatus for processing a database replica |
| CN101179437A (en) * | 2007-11-29 | 2008-05-14 | 上海华为技术有限公司 | Method and apparatus for implementing preactivation of batch configuration |
| CN102387204A (en) * | 2011-10-21 | 2012-03-21 | 中国科学院计算技术研究所 | Method and system for maintaining consistency of cluster caching |
| CN103500204A (en) * | 2013-09-27 | 2014-01-08 | 北京京东尚科信息技术有限公司 | Web surfing backspacing method, web server and client side |
| CN104133783A (en) * | 2014-07-11 | 2014-11-05 | 北京京东尚科信息技术有限公司 | Method and device for processing distributed cache data |
| US20170249375A1 (en) * | 2016-02-29 | 2017-08-31 | International Business Machines Corporation | Continuous Automatic Update Statistics Evaluation Using Change Data Capture Techniques |
| CN106599199A (en) * | 2016-12-14 | 2017-04-26 | 国云科技股份有限公司 | Data caching and synchronization method |
| CN107193947A (en) * | 2017-05-22 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of file system cache incremental refreshment method and system |
| US20190394138A1 (en) * | 2018-06-26 | 2019-12-26 | International Business Machines Corporation | Guaranteed delivery in receiver side overcommitted communication adapters |
Non-Patent Citations (3)
| Title |
|---|
| QUISLANT R.ET AL: "Lasy Irrevocability for Best-Effort Transactional Memory Systems", 《IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS》 * |
| 侯鸿林等: "OLAP系统中回退功能的设计与实现", 《计算机工程与科学》 * |
| 王苑瑾: "硬件事物存储器体系结构", 《中国知网硕士学位论文全文数据库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022048495A1 (en) * | 2020-09-01 | 2022-03-10 | 阿里云计算有限公司 | Data backup method and apparatus, data recovery method and apparatus, and electronic device |
| CN114218013A (en) * | 2021-12-13 | 2022-03-22 | 北京字节跳动网络技术有限公司 | Searching method, searching device and electronic equipment storage medium |
| CN114816845A (en) * | 2022-04-06 | 2022-07-29 | 在线途游（北京）科技有限公司 | MongoDB-based rapid data rollback method and device |
| CN114816845B (en) * | 2022-04-06 | 2024-05-10 | 在线途游（北京）科技有限公司 | MongoDB-based rapid data rollback method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111444039B (en) | 2023-03-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8127174B1 (en) | Method and apparatus for performing transparent in-memory checkpointing | |
| US8301600B1 (en) | Failover recovery in a distributed data store | |
| JP5075736B2 (en) | System failure recovery method and system for virtual server | |
| US6654771B1 (en) | Method and system for network data replication | |
| CN108319617B (en) | Method and device for determining master-slave difference of database and switching control method and device | |
| JP2009015476A (en) | Journal management method in cdp remote configuration | |
| CN111444039B (en) | Cache data rollback method and cache data rollback device | |
| CN111901176B (en) | Fault determination method, device, equipment and storage medium | |
| CN111176900A (en) | Distributed storage system and data recovery method, device and medium thereof | |
| US10078558B2 (en) | Database system control method and database system | |
| US10671482B2 (en) | Providing consistency in a distributed data store | |
| WO2021082925A1 (en) | Transaction processing method and apparatus | |
| CN112015595B (en) | Master-slave database switching method, computing device and storage medium | |
| US20190303233A1 (en) | Automatically Detecting Time-Of-Fault Bugs in Cloud Systems | |
| CN111625264B (en) | Method, device, equipment and medium for upgrading multi-level management system | |
| JPH11259326A (en) | Hot standby system, automatic re-execution method for the same and storage medium therefor | |
| CN115314361B (en) | Server cluster management method and related components thereof | |
| JP2019020798A (en) | Information processing device and program | |
| CN110688252A (en) | Data recovery method, data backup device and storage medium | |
| CN112231150B (en) | Method and device for recovering fault database in database cluster | |
| CN111273933B (en) | BIOS FW refreshing method and device | |
| CN109218044B (en) | Fault processing method and device | |
| CN113900952A (en) | Test case execution method and device, terminal equipment and readable storage medium | |
| CN116931814A (en) | Cloud hard disk capacity expansion method and device, electronic equipment and storage medium | |
| CN114968656A (en) | Data rollback method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |