CN106682193A - Device and method for data persistent storage on basis of cache - Google Patents

Device and method for data persistent storage on basis of cache Download PDF

Info

Publication number
CN106682193A
CN106682193A CN201611249423.2A CN201611249423A CN106682193A CN 106682193 A CN106682193 A CN 106682193A CN 201611249423 A CN201611249423 A CN 201611249423A CN 106682193 A CN106682193 A CN 106682193A
Authority
CN
China
Prior art keywords
data
caching
timestamp
base
write
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201611249423.2A
Other languages
Chinese (zh)
Other versions
CN106682193B (en
Inventor
王文铎
陈宗志
彭信东
王康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Qihoo Technology Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201611249423.2A priority Critical patent/CN106682193B/en
Publication of CN106682193A publication Critical patent/CN106682193A/en
Application granted granted Critical
Publication of CN106682193B publication Critical patent/CN106682193B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/08Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10Providing a specific technical effect
    • G06F2212/1032Reliability improvement, data loss prevention, degraded operation etc
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/16General purpose computing application
    • G06F2212/163Server or database system

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a device and a method for data persistent storage on the basis of cache and at least solves the technical problem of failure in prevention of data errors in a database in cache failure due to poor disaster tolerance of an existing cache mechanism. The method includes: when data are written into cache, respectively setting corresponding timestamps for data in cache according to data writing time; when the data in cache are persistently stored into the database, respectively acquiring the timestamp of each datum in cache and a timestamp, of a data record corresponding to the datum, in the database; if the timestamps of the data in cache are smaller than those of the data records corresponding to the data, deleting the data in cache; otherwise, persistently storing the data in cache and the corresponding timestamps into the database.

Description

Data persistence storage method and device based on caching
Technical field
The present invention relates to communication technical field, and in particular to a kind of data persistence storage method and dress based on caching Put.
Background technology
Caching is exactly the relief area of data exchange, due to the read or write speed for caching it is very fast, therefore, can be lifted using caching Digital independent or the efficiency of write.For example, when reading the data, first desired data is searched from caching, it is direct if finding It is read out from caching;When the data is written, the data write caching being first written into, then again holds the data in caching Longization is stored in data base.As can be seen here, can be offered convenience for read-write process by caching mechanism, therefore, caching mechanism Obtain increasingly being widely applied.
But, inventor has found that at least there are the following problems for existing caching mechanism during the present invention is realized:When When caching normal, the data of cache storage can be stored in sorted order in data base according to preset rules.But, once caching is sent out Failure is given birth to, after fault recovery, the data of cache storage then out of order or even may be stored mistakenly in data base.Example Such as, it is assumed that a part of data in caching temporarily break down and cannot read, therefore, the data persistence in caching is being deposited When storing up data base, the partial data is missed;After a period of time, the partial data in caching recovers normal, then exist During next time the data persistence in caching is stored to data base, the partial data will be stored in data base. However, in a practical situation, the partial data is likely to have become expired wrong data, thus causes the number in data base According to there is mistake.As can be seen here, existing caching mechanism disaster tolerance is poor, it is impossible to the number in preventing data base when failure is cached According to error.
The content of the invention
In view of the above problems, it is proposed that the present invention so as to provide one kind overcome the problems referred to above or at least in part solve on State the data persistence storage method based on caching and device of problem.
According to an aspect of the invention, there is provided a kind of data persistence storage method based on caching, including:Xiang Huan Deposit it is middle write data when, according to Data writing time be caching in each data be respectively provided with corresponding timestamp;Will be described When each data persistence storage in caching is to data base, for each data in the caching, obtain described slow respectively Deposit the timestamp of the data and the timestamp of data record corresponding with the data in the data base;When it is determined that described slow The timestamp for depositing the data is described slow less than during the timestamp of data record corresponding with the data, deleting in the data base The data in depositing;Otherwise, by the data in the caching and its corresponding timestamp persistent storage to the data base In.
According to another aspect of the present invention, there is provided it is a kind of based on caching data persistence storage device, including:If Put module, be suitable to in caching write data when, according to Data writing time be caching in each data be respectively provided with correspondence Timestamp;Acquisition module, when being suitable to for each data persistence storage in the caching to arrive data base, for the caching In each data, number corresponding with the data in the timestamp of the data and the data base is obtained in the caching respectively According to the timestamp of record;Processing module, in being suitable to the timestamp of the data in determine the caching less than the data base with During the timestamp of the corresponding data record of the data, the data in the caching are deleted;Otherwise, by the number in the caching According to and its corresponding timestamp persistent storage in the data base.
In a kind of data persistence storage method and device based on caching that the present invention is provided, write in caching first When entering data, according to Data writing time be caching in each data be respectively provided with corresponding timestamp;Then in caching The storage of each data persistence to data base when, for each data in caching, obtain respectively the data in caching when Between stamp and data base in data record corresponding with the data timestamp;Finally, when it is determined that caching in the data time Stamp is less than the data in data base during the timestamp of data record corresponding with the data, in deletion caching;Otherwise, will cache In the data and its corresponding timestamp persistent storage in data base.As can be seen here, the present invention is each in caching Data are respectively provided with corresponding timestamp, correspondingly, when by certain the data persistence storage in caching to data base, pass through The mode that the timestamp of the data in caching is compared with the timestamp of data record corresponding with the data in data base, Judge cache in data whether be invalid data out of date, so as to improve data persistence during reliability, have Effect is prevented because caching the caused error in data phenomenon of failure institute.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention, And can be practiced according to the content of description, and in order to allow the above and other objects of the present invention, feature and advantage can Become apparent, below especially exemplified by the specific embodiment of the present invention.
Description of the drawings
By the detailed description for reading hereafter preferred implementation, various other advantages and benefit is common for this area Technical staff will be clear from understanding.Accompanying drawing is only used for illustrating the purpose of preferred implementation, and is not considered as to the present invention Restriction.And in whole accompanying drawing, it is denoted by the same reference numerals identical part.In the accompanying drawings:
Fig. 1 shows that a kind of flow process of data persistence storage method based on caching that the embodiment of the present invention one is provided is shown It is intended to;
Fig. 2 shows that a kind of flow process of data persistence storage method based on caching that the embodiment of the present invention two is provided is shown It is intended to;
Fig. 3 shows a kind of structural frames of data persistence storage device based on caching that the embodiment of the present invention three is provided Figure.
Specific embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing the disclosure in accompanying drawing Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure and should not be by embodiments set forth here Limited.On the contrary, there is provided these embodiments are able to be best understood from the disclosure, and can be by the scope of the present disclosure Complete conveys to those skilled in the art.
A kind of data persistence storage method based on caching and device are embodiments provided, at least be can solve the problem that Existing caching mechanism disaster tolerance is poor, it is impossible to the technical problem of the corrupt data in data base is prevented when failure is cached.
Embodiment one
Fig. 1 shows that a kind of flow process of data persistence storage method based on caching that the embodiment of the present invention one is provided is shown It is intended to, the method includes:
Step S110:To in caching write data when, according to Data writing time be caching in each data set respectively Put corresponding timestamp.
Wherein, timestamp is used for time when representing that data write is cached.The concrete form of timestamp and precision can be by these Art personnel flexibly arranged, and the present invention is not limited this.For example, it is assumed that a data are in the morning 10 on December 13rd, 2016 01 minute write caching of point, then the corresponding timestamp of the data is 201612131001.In addition, in the buffer data storage when, can So that data and its corresponding timestamp are associated into storage as a data set, it is also possible to using timestamp as data itself An attribute stored, the present invention concrete storage form is not limited.
Step S120:During by each data persistence storage in caching to data base, for each data in caching, The timestamp of the timestamp of the data and data record corresponding with the data in data base in caching is obtained respectively.
Wherein, each data persistence storage to the opportunity in data base can flexibly be arranged by those skilled in the art, The present invention is not limited this.In persistent storage, first, the timestamp of the data in caching is obtained;Then, counting According to lookup data record corresponding with the data in storehouse, and obtain the timestamp of data record corresponding with the data.Wherein, with The corresponding data record of the data can be stored by various forms.
Step S130:When it is determined that the timestamp of the data is less than data record corresponding with the data in data base in caching Timestamp when, delete caching in the data;Otherwise, by the data in caching and its corresponding timestamp persistent storage To in data base.
Specifically, timestamp is used to represent write time of data, can be indicated by numeric form.In the present invention In, the priority of time is represented by the size of timestamp:The timestamp of time more forward data is less, and the time is more rearward Data timestamp it is bigger.As can be seen here, the size for comparing timestamp is actually the priority of determination time.For example, it is false If the write time of data one is 01 minute morning 10 point of on December 13rd, 2016, timestamp is 201612131001;Data two Write time be 01 minute morning 10 point of on December 14th, 2016, timestamp is 201612141001, due to data one write when Between earlier than data two write time, then the timestamp of data one less than data two timestamp.
Correspondingly, when the timestamp of the data in caching is less than the time of data record corresponding with the data in data base During stamp, illustrate cache in the data write time earlier than corresponding data record in data base write time, i.e.,:Caching In the data be invalid data out of date, thus delete caching in the data, to prevent error in data phenomenon.When During the timestamp of the data in data base not less than during the timestamp of data record corresponding with the data, illustrating to cache in caching Write time of the data be later than write time of corresponding data record in data base, i.e.,:The data in caching are not for Expired valid data, thus by the data in caching and its corresponding timestamp persistent storage in data base.
As can be seen here, in a kind of data persistence storage method based on caching that the present invention is provided, first to caching It is middle write data when, according to Data writing time be caching in each data be respectively provided with corresponding timestamp;Then will be slow When each data persistence storage in depositing is to data base, for each data in caching, the data in caching are obtained respectively Timestamp and data base in data record corresponding with the data timestamp;Finally, when it is determined that the data in caching Timestamp is less than the data in data base during the timestamp of data record corresponding with the data, in deletion caching;Otherwise, will Data and its corresponding timestamp persistent storage in caching are in data base.Therefore, the side in the present invention in embodiment Method is that each data in caching are respectively provided with corresponding timestamp, correspondingly, certain data persistence in caching is being deposited When storing up data base, by by the time of the timestamp of the data in caching and data record corresponding with the data in data base The mode that stamp is compared, judges whether the data in caching are invalid data out of date, so as to improve data persistence During reliability, error in data phenomenon caused by effectively prevent because caching failure institute.
Embodiment two
Fig. 2 shows that a kind of flow process of data persistence storage method based on caching that the embodiment of the present invention two is provided is shown It is intended to, as shown in Fig. 2 the method includes:
Step S210:To in caching write data when, according to Data writing time be caching in each data set respectively Put corresponding timestamp.
Wherein, timestamp is used for time when representing that data write is cached.The concrete form of timestamp and precision can be by these Art personnel flexibly arranged, and the present invention is not limited this.For example, it is assumed that a data are in the morning 10 on December 13rd, 2016 01 minute write caching of point, then the corresponding timestamp of the data is 201612131001.
In addition, when to data storage in caching, various storage modes can be adopted.In the first storage mode, can So that data and its corresponding timestamp are associated into storage as a data set.In addition, in associated storage, actually relating to And two generic operations, the first generic operation is write data, and the second generic operation is the corresponding timestamp of write data, in order to prevent because of it In a generic operation run succeeded, another generic operation performs a mistake caused by failure institute, can be by the first generic operation and Equations of The Second Kind Operation is performed collectively as an affairs, and only when the first generic operation and the second generic operation all run succeeded, just submitting to should Affairs, otherwise, then rollback affairs, to re-execute data write operation, so as to ensure that caching in each data and at that time Between stab accuracy.In second storage mode, it is also possible to stored timestamp as an attribute of data itself, Can also be stored timestamp as the suffix or prefix of data.The present invention is not limited specific storage form.
In addition, in the present embodiment, various types of write operations such as the write of data includes writing direct and change.
Step S220:During by each data persistence storage in caching to data base, for each data in caching, The timestamp of the timestamp of the data and data record corresponding with the data in data base in caching is obtained respectively.
In the present embodiment, can carry out when default persistence condition meets above-mentioned by each data in caching To the operation of data base, specific persistence condition can flexibly set according to demand persistent storage by those skilled in the art. For example, in a kind of implementation of the present embodiment, the persistence condition is:In caching written into data reach default ratio Example.And for example, in another kind of implementation, the persistence condition is:When reaching the predetermined persistence time, or distance is last When the time of persistence reaches prefixed time interval.Specific persistence time and prefixed time interval can be by art technologies Personnel flexibly set.
Specifically, in persistent storage, first, the timestamp of the data that persistence is treated in caching is obtained;Then, exist Data record corresponding with the data is searched in data base, and obtains the timestamp of data record corresponding with the data.In addition, Because same data often successively perform in the buffer multiple write operations, therefore, in this step, can respectively obtain slow Corresponding timestamp during each write of the data in depositing, by the maximum (timestamp in corresponding timestamp during each write The maximum explanation write time is the latest) it is defined as the timestamp of the data in caching;Obtain respectively corresponding with the data in data base Pieces of data record timestamp, by pieces of data record timestamp in maximum be defined as in data base and the data The timestamp of corresponding data record.As can be seen here, can determine the data slow by determining the maximum data of timestamp Time when being ultimately written in depositing, by determining that the maximum data record of timestamp can determine corresponding data Ji Lu data Time when being ultimately written in storehouse, and then can rapidly and accurately judge whether the data in caching are effective.Further, since data Generally unique mark is carried out by key assignments, therefore, key assignments identical data are considered as same data, correspondingly, above-mentioned During, corresponding timestamp when the key assignments of the data in caching determines each write of the data;According in caching Pieces of data record corresponding with the data in the key assignments searching data storehouse of the data, and determine the time that pieces of data is recorded Stamp.
Step S230:Whether the timestamp of the data is less than data note corresponding with the data in data base in judging to cache The timestamp of record, if the determination result is YES, execution step S240;If judged result is no, execution step S250.
Specifically, timestamp is used to represent write time of data, can be indicated by numeric form.In the present invention In, the priority of time is represented by the size of timestamp:The timestamp of time more forward data is less, and the time is more rearward Data timestamp it is bigger.As can be seen here, the size for comparing timestamp is actually the priority of determination time.For example, it is false If the write time of data one is 01 minute morning 10 point of on December 13rd, 2016, timestamp is 201612131001;Data two Write time be 01 minute morning 10 point of on December 14th, 2016, timestamp is 201612141001, due to data one write when Between earlier than data two write time, then the timestamp of data one less than data two timestamp.
Step S240:Delete the data in caching.
Specifically, when the timestamp of the data in caching is less than the time of data record corresponding with the data in data base During stamp, illustrate cache in the data write time earlier than corresponding data record in data base write time, i.e.,:Caching In the data be invalid data out of date, thus delete caching in the data, to prevent error in data phenomenon.
In addition, in this step, when it is determined that the timestamp of the data is corresponding with the data less than in data base in caching During the timestamp of data record, following operation is can further include:By the data in data base and its corresponding timestamp Preserve into caching.Many advantages are waited soon because caching possesses access speed, therefore, the data in judging to cache are for During expired invalid data, further the data in data base and its corresponding timestamp are preserved into caching, to realize carrying Rise the effect of access speed.In addition, can be with by the way that the data in data base and its corresponding timestamp are preserved into caching The data lost during caching failure are enable to recover.Specifically, the data in data base actually refer in data base with The corresponding data record of the data.In a practical situation, in the storage form of the data in caching and data base with the data pair The storage form of the data record answered both can be the same or different.For example, the data in caching can pass through Word document Stored, data record corresponding with the data can be stored by Excel forms in data base.
Step S250:By the data in caching and its corresponding timestamp persistent storage in data base.
Specifically, when the data in caching timestamp not less than in data base data record corresponding with the data when Between when stabbing, the write time for illustrating the data in caching is later than the write time of corresponding data record in data base, i.e.,:It is slow The data in depositing are not out of date valid data, thus by the data in caching and its corresponding timestamp persistent storage To in data base.By the judgement of timestamp, the correctness of the data stored in data base can be lifted.
In addition, the disaster tolerance performance in order to further lift caching system, in the execution of above-mentioned steps S210 to step S250 During, can further include following steps:When detect caching failure when, by caching failure during produce data and Its corresponding timestamp persistent storage is in data base.The present invention was not limited the concrete execution opportunity of the step.For example, This step can be put into after step S240 and be performed, so as to trigger this when there is wrong data out of date in finding caching Step;Whether failure can also be once cached every default time interval automatic detection, and be triggered when testing result is to be This step.By this step, can be by the data produced during caching failure and its direct persistent storage of corresponding timestamp To in data base, so as to prevent the loss of data, the reliability service for whole caching system provides guarantee.In addition, here On the basis of, due to when performing persistent storage after caching fault recovery and operating, can further compare each it is data cached when Between stab, it is thus possible to judge in data base through caching and direct write time of the partial data of persistence, and When the timestamp of the data in determining caching is less than the timestamp of data record corresponding with the data in data base, will The data and its corresponding timestamp are preserved into caching in data base so that the data during caching failure be able to it is completely extensive It is multiple, improve the disaster tolerance performance of caching system.
In addition, those skilled in the art can also carry out various changes and deformation to above-described embodiment.For example, can will be upper State each step in embodiment and split into more steps, or merge into less step, can also adjust each step it Between execution sequence, it might even be possible to delete part steps or increase newly some steps, in a word, the present invention to concrete implementation details not Limit.
In addition, inventor has found during the present invention is realized, common caching is mostly by the internal memory of non-persistentization Or other storage devices are realized, the characteristic that non-persistentization caching falls with power-off, i.e.,:After power-off or failure, store thereon Data will be emptied therewith, so as to cause data cached loss.And, non-persistentization caching often has the offices such as capacity is less than normal It is sex-limited.In order to solve the above problems, in the present embodiment, it is possible to use persistence caching is realized.For example, persistence caching includes But it is not limited to:SSD, solid state hard disc and/or mechanical hard disk etc..These persistences caching be often provided simultaneously with read or write speed it is fast and The big advantage of memory capacity.And, it is that each data are provided with timestamp in the present embodiment, therefore, even if persistence caching There are the abnormal conditions such as power-off or failure, the data for storing thereon still retain, and can sentence according to the timestamp of each data Whether each data of breaking are still effective, so as to improve the performance of caching system.
Embodiment three
Fig. 3 shows a kind of structural frames of data persistence storage device based on caching that the embodiment of the present invention three is provided Figure.As shown in figure 3, the device includes:Setup module 31, acquisition module 32, processing module 33, time memory module 34 and deposit Storage processing module 35.
It is each data point in caching according to Data writing time when setup module 31 is suitable to write data in caching Corresponding timestamp is not set.
Wherein, timestamp is used for time when representing that data write is cached.The concrete form of timestamp and precision can be by these Art personnel flexibly arranged, and the present invention is not limited this.For example, it is assumed that a data are in the morning 10 on December 13rd, 2016 01 minute write caching of point, then the corresponding timestamp of the data is 201612131001.In addition, setup module 31 writes according to data When time is that each data in caching are respectively provided with corresponding timestamp, various storage modes can be set.Deposit at the first In storage mode, data and its corresponding timestamp can be associated storage by setup module 31 as a data set.In addition, In associated storage, two generic operations are actually related to, the first generic operation is write data, and the second generic operation is write data correspondence Timestamp, in order to prevent because a generic operation therein runs succeeded, another generic operation performs a mistake caused by failure institute, can be with First generic operation and the second generic operation are performed collectively as an affairs, only when the first generic operation and the second generic operation whole When running succeeded, the affairs are just submitted to, otherwise, then the rollback affairs, slow so as to ensure that to re-execute data write operation The accuracy of each data and its timestamp in depositing.In second storage mode, setup module 31 can also make timestamp An attribute for data itself is stored, and can also be stored timestamp as the suffix or prefix of data.This It is bright that specific storage form is not limited.In addition, in the present embodiment, the write of data includes writing direct and changing Various types of write operations.
When acquisition module 32 is suitable to for each data persistence storage in caching to arrive data base, for each in caching Data, obtain respectively the timestamp of the timestamp of the data and data record corresponding with the data in data base in caching.
In the present embodiment, acquisition module 32 can carry out when default persistence condition meets it is above-mentioned will be in caching The storage of each data persistence to data base operation, specific persistence condition can be by those skilled in the art according to demand Flexibly setting.For example, in a kind of implementation of the present embodiment, the persistence condition is:In caching written into data reach To preset ratio.And for example, in another kind of implementation, the persistence condition is:When reaching the predetermined persistence time, or away from When reaching prefixed time interval from the time of last persistence.Specific persistence time and prefixed time interval can be by these Art personnel flexibly set.
Specifically, in persistent storage, first, acquisition module 32 will obtain the time of the data that persistence is treated in caching Stamp;Then, search corresponding with data data record in data base, and acquisition data record corresponding with the data when Between stab.Further, since same data often successively perform in the buffer multiple write operations, therefore, acquisition module 32 can be with Corresponding timestamp when obtaining each write of the data in caching respectively, by the maximum in corresponding timestamp during each write Value (the timestamp maximum explanation write time is the latest) is defined as the timestamp of the data in caching;And, data base is obtained respectively In corresponding with data pieces of data record timestamp, the maximum in the timestamp that pieces of data is recorded is defined as number According to the timestamp of data record corresponding with the data in storehouse.As can be seen here, timestamp maximum is determined by acquisition module 32 Time when data can determine that the data is ultimately written in the buffer, the maximum number of timestamp is determined by acquisition module 32 Time when being ultimately written in corresponding data Ji Lu data base is can determine according to record, and then can rapidly and accurately be judged Whether the data in caching are effective.Further, since data generally carry out unique mark by key assignments, therefore, key assignments identical number According to being considered as same data, correspondingly, the correspondence when key assignments of the data in caching determines each write of the data Timestamp;Pieces of data record corresponding with the data in the key assignments searching data storehouse of the data in caching, and really Determine the timestamp of pieces of data record.
Processing module 33 is suitable to the timestamp of the data in caching is determined less than number corresponding with the data in data base During according to the timestamp for recording, the data in caching are deleted;Otherwise, it is the data and its corresponding timestamp in caching are lasting Change and store in data base.
Specifically, timestamp is used to represent write time of data, can be indicated by numeric form.In the present invention In, the priority of time is represented by the size of timestamp:The timestamp of time more forward data is less, and the time is more rearward Data timestamp it is bigger.Therefore, the size that processing module 33 compares timestamp is actually the priority of determination time.Example Such as, it is assumed that the write time of data one is 01 minute morning 10 point of on December 13rd, 2016, and timestamp is 201612131001;Data Two write time is 01 minute morning 10 point of on December 14th, 2016, and timestamp is 201612141001, due to writing for data one The angle of incidence earlier than data two write time, then the timestamp of data one less than data two timestamp.When processing module 33 it is true During the timestamp of the data in data base less than during the timestamp of data record corresponding with the data, illustrating to cache in fixed caching The data write time earlier than corresponding data record in data base write time, i.e.,:The data in caching are for Expired invalid data, thus the data in caching are deleted, to prevent error in data phenomenon.When the data in caching Timestamp is not less than the write that the data in caching during the timestamp of data record corresponding with the data, are illustrated in data base Time is later than the write time of corresponding data record in data base, i.e.,:The data in caching are not out of date valid data, Thus processing module 33 by the data in caching and its corresponding timestamp persistent storage in data base.By timestamp Judgement, the correctness of the data stored in data base can be lifted.
Time memory module 34 is suitable to the data in data base and its corresponding timestamp be preserved into caching.
When processing module 33 determines the timestamp of the data in caching less than data note corresponding with the data in data base During the timestamp of record, further the data in data base and its corresponding timestamp can also be protected by time memory module 34 Deposit into caching.Many advantages are waited soon because caching possesses access speed, therefore, being somebody's turn to do in judging to cache of processing module 33 When data are invalid data out of date, time memory module 34 is further by the data in data base and its corresponding timestamp Preserve into caching, to realize lifting the effect of access speed.In addition, time memory module 34 is by by the data in data base And its corresponding timestamp preserves into caching and the data lost during caching failure can also be enable to recover.Specifically, number Actually refer to data record corresponding with the data in data base according to the data in storehouse.In a practical situation, in caching The storage form of data both can be the same or different with the storage form of data record corresponding with the data in data base. For example, the data in caching can be stored by Word document, and data record corresponding with the data can be with data base Stored by Excel forms.
Storage processing module 35 is suitable to when caching failure is detected, by the data produced during caching failure and its correspondence Timestamp persistent storage in data base.
Specifically, storing processing module 35 is used for when caching failure is detected, the data that will be produced during caching failure And its corresponding timestamp persistent storage is in data base, and then the purpose of the disaster tolerance performance for realizing lifting caching system. Here, the opportunity that performs for storing processing module 35 can exist after the data in above-mentioned caching is deleted in caching is found Triggering storage processing module 35 performs corresponding persistent storage operation during wrong data out of date;Or can also be every Default time interval automatic detection once caches whether failure, and triggering storage processing module 35 is held when testing result is to be The corresponding persistent storage operation of row, the present invention was not limited the concrete execution opportunity for storing processing module 35.Therefore, pass through Storage processing module 35 can be by the data produced during caching failure and its direct persistent storage of corresponding timestamp to number According to storehouse, so as to prevent the loss of data, the reliability service for whole caching system provides guarantee.
The concrete structure and operation principle of above-mentioned modules can refer to the description of corresponding steps in embodiment of the method, herein Repeat no more.
As can be seen here, in a kind of data persistence storage device based on caching that the present invention is provided, first by setting When putting module 31 and writing data in caching, when according to Data writing time being that each data in caching are respectively provided with corresponding Between stab;Then by acquisition module 32 by each data persistence storage in caching to data base when, for caching in it is every Individual data, obtain respectively the time of the timestamp of the data and data record corresponding with the data in data base in caching Stamp;And by the timestamp of the data in caching is determined of processing module 33 less than data note corresponding with the data in data base During the timestamp of record, the data in caching are deleted;Otherwise, the data in caching and its corresponding timestamp persistence are deposited In storing up data base.Meanwhile, further the data in data base and its corresponding timestamp are protected by time memory module 34 Deposit into caching, by store processing module 35 when detect caching failure when, by caching failure during produce data and its Corresponding timestamp persistent storage is in data base.Therefore, the method in the present invention in embodiment is each number in caching According to corresponding timestamp is respectively provided with, correspondingly, when by certain data persistence storage in caching to data base, by inciting somebody to action The mode that the timestamp of the data is compared with the timestamp of data record corresponding with the data in data base in caching, sentences Whether the data in disconnected caching are invalid data out of date, so as to improve data persistence during reliability, effectively Error in data phenomenon caused by preventing because caching failure institute, even and if the exception feelings such as power-off or failure occurs in persistence caching Condition, the data for storing thereon still retain, and whether still effectively can judge each data according to the timestamp of each data, So as to improve the performance of caching system.
Provided herein algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment. Various general-purpose systems can also be used together based on teaching in this.As described above, construct required by this kind of system Structure be obvious.Additionally, the present invention is also not for any certain programmed language.It is understood that, it is possible to use it is various Programming language realizes the content of invention described herein, and the description done to language-specific above is to disclose this Bright preferred forms.
In description mentioned herein, a large amount of details are illustrated.It is to be appreciated, however, that the enforcement of the present invention Example can be put into practice in the case of without these details.In some instances, known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and help understand one or more in each inventive aspect, exist Above in the description of the exemplary embodiment of the present invention, each feature of the present invention is grouped together into single enforcement sometimes In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention:I.e. required guarantor The more features of feature that the application claims ratio of shield is expressly recited in each claim.More precisely, such as following Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore, Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim itself All as the separate embodiments of the present invention.
Those skilled in the art are appreciated that can be carried out adaptively to the module in the equipment in embodiment Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment Unit or component are combined into a module or unit or component, and can be divided in addition multiple submodule or subelement or Sub-component.In addition at least some in such feature and/or process or unit is excluded each other, can adopt any Combine to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification is (including adjoint power Profit is required, summary and accompanying drawing) disclosed in each feature can it is identical by offers, be equal to or the alternative features of similar purpose carry out generation Replace.
Although additionally, it will be appreciated by those of skill in the art that some embodiments described herein include other embodiments In included some features rather than further feature, but the combination of the feature of different embodiments means in of the invention Within the scope of and form different embodiments.For example, in the following claims, embodiment required for protection appoint One of meaning can in any combination mode using.
The present invention all parts embodiment can be realized with hardware, or with one or more processor operation Software module realize, or with combinations thereof realization.It will be understood by those of skill in the art that can use in practice Microprocessor or digital signal processor (DSP) are come in the consumption device for realizing message queue according to embodiments of the present invention The some or all functions of some or all parts.The present invention is also implemented as performing method as described herein Some or all equipment or program of device (for example, computer program and computer program).Such reality The program of the existing present invention can be stored on a computer-readable medium, or can have the form of one or more signal. Such signal can be downloaded from internet website and obtained, or be provided on carrier signal, or in any other form There is provided.
It should be noted that above-described embodiment the present invention will be described rather than limits the invention, and ability Field technique personnel can design without departing from the scope of the appended claims alternative embodiment.In the claims, Any reference markss between bracket should not be configured to limitations on claims.Word "comprising" is not excluded the presence of not Element listed in the claims or step.Word "a" or "an" before element does not exclude the presence of multiple such Element.The present invention can come real by means of the hardware for including some different elements and by means of properly programmed computer It is existing.If in the unit claim for listing equipment for drying, several in these devices can be by same hardware branch To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and be run after fame Claim.
The invention discloses:A1, a kind of data persistence storage method based on caching, including:
To in caching write data when, according to Data writing time be caching in each data be respectively provided with corresponding when Between stab;
During by each data persistence storage in the caching to data base, for each data in the caching, The time of data record corresponding with the data in the timestamp of the data in the caching and the data base is obtained respectively Stamp;
When it is determined that the timestamp of the data is less than data record corresponding with the data in the data base in the caching Timestamp when, delete the data in the caching;Otherwise, the data and its corresponding timestamp in the caching are held Longization is stored in the data base.
A2, the method according to A1, wherein, when it is determined that the timestamp of the data is less than the data base in the caching In data record corresponding with the data timestamp when, methods described is further included:By data in data base and its right The timestamp answered is preserved into the caching.
A3, the method according to A1 or A2, wherein, further include:When the caching failure is detected, will be described The data produced during caching failure and its corresponding timestamp persistent storage are in the data base.
A4, according to the arbitrary described methods of A1-A3, wherein, the timestamp that the data in the caching are obtained respectively And specifically include the step of the timestamp of data record corresponding with the data in the data base:
Corresponding timestamp when obtaining each write of the data in the caching respectively, when will be corresponding during each write Between stab in maximum be defined as the timestamp of the data in the caching;
The timestamp of pieces of data record corresponding with the data in the data base is obtained respectively, and pieces of data is recorded Timestamp in maximum be defined as the timestamp of data record corresponding with the data in the data base.
A5, the method according to A4, wherein, correspondence during each write of described data obtained respectively in the caching Timestamp the step of specifically include:The key assignments of the data in the caching determines corresponding during each write of the data Timestamp;
It is described concrete the step of obtain the timestamp of corresponding with data pieces of data record in the data base respectively Including:The key assignments of the data in the caching searches pieces of data record corresponding with the data in the data base, And determine the timestamp of pieces of data record.
A6, the method according to A1-A5 is arbitrary, wherein, the caching is persistence caching, wherein,
The persistence caching includes:Solid state hard disc and/or mechanical hard disk.
The invention also discloses:B7, a kind of data persistence storage device based on caching, including:
Setup module, is each data point in caching according to Data writing time when being suitable to write data in caching Corresponding timestamp is not set;
Acquisition module, when being suitable to for each data persistence storage in the caching to arrive data base, for the caching In each data, number corresponding with the data in the timestamp of the data and the data base is obtained in the caching respectively According to the timestamp of record;
Processing module, is suitable in the timestamp of the data in the determination caching is less than the data base and the data pair During the timestamp of the data record answered, the data in the caching are deleted;Otherwise, by data in the caching and its right The timestamp persistent storage answered is in the data base.
B8, the device according to B7, wherein, described device is further included:Time memory module, is suitable to data base In the data and its corresponding timestamp preserve into the caching.
B9, the device according to B7 or B8, wherein, described device is further included:Storage processing module, is suitable to when inspection When measuring the caching failure, by the data produced during the caching failure and its corresponding timestamp persistent storage to institute In stating data base.
B10, according to the arbitrary described devices of B7-B9, wherein, the acquisition module is further used for:
Corresponding timestamp when obtaining each write of the data in the caching respectively, when will be corresponding during each write Between stab in maximum be defined as the timestamp of the data in the caching;
The timestamp of pieces of data record corresponding with the data in the data base is obtained respectively, and pieces of data is recorded Timestamp in maximum be defined as the timestamp of data record corresponding with the data in the data base.
B11, the device according to B10, wherein, the acquisition module specifically for:The number in the caching According to key assignments determine the write of each time of the data when corresponding timestamp;
The acquisition module specifically for:The key assignments of the data in the caching is searched in the data base and is somebody's turn to do The corresponding pieces of data record of data, and determine the timestamp of the pieces of data record.
B12, the device according to B7-B11 is arbitrary, wherein, the caching is persistence caching, wherein, it is described lasting Changing caching includes:Solid state hard disc and/or mechanical hard disk.

Claims (10)

1. it is a kind of based on caching data persistence storage method, including:
To in caching write data when, according to Data writing time be caching in each data be respectively provided with the corresponding time Stamp;
During by each data persistence storage in the caching to data base, for each data in the caching, difference Obtain the timestamp of data record corresponding with the data in the timestamp of the data in the caching and the data base;
When it is determined that in the caching data timestamp less than in the data base data record corresponding with the data when Between when stabbing, delete the data in the caching;Otherwise, by the data in the caching and its corresponding timestamp persistence In storing the data base.
2. method according to claim 1, wherein, when it is determined that the timestamp of the data is less than the data in the caching In storehouse during the timestamp of data record corresponding with the data, methods described is further included:By the data in data base and its Corresponding timestamp is preserved into the caching.
3. method according to claim 1 and 2, wherein, further include:When the caching failure is detected, by institute The data and its corresponding timestamp persistent storage of generation are in the data base during stating caching failure.
4. according to the arbitrary described method of claim 1-3, wherein, the timestamp that the data in the caching are obtained respectively And specifically include the step of the timestamp of data record corresponding with the data in the data base:
Corresponding timestamp when obtaining each write of the data in the caching respectively, by corresponding timestamp during each write In maximum be defined as the timestamp of the data in the caching;
Obtain the timestamp of corresponding with data pieces of data record in the data base respectively, by pieces of data record when Between stab in maximum be defined as the timestamp of data record corresponding with the data in the data base.
5. method according to claim 4, wherein, during each write of described data obtained respectively in the caching pair The step of timestamp answered, specifically includes:The correspondence when key assignments of the data in the caching determines each write of the data Timestamp;
It is described to specifically include the step of obtain the timestamp of corresponding with data pieces of data record in the data base respectively: The key assignments of the data in the caching searches pieces of data record corresponding with the data in the data base, and determines The timestamp of the pieces of data record.
6. according to the arbitrary described method of claim 1-5, wherein, the caching is persistence caching,
Wherein, the persistence caching includes:Solid state hard disc and/or mechanical hard disk.
7. it is a kind of based on caching data persistence storage device, including:
Setup module, be suitable to in caching write data when, according to Data writing time be caching in each data set respectively Put corresponding timestamp;
Acquisition module, when being suitable to for each data persistence storage in the caching to arrive data base, in the caching Each data, obtains respectively in the caching data note corresponding with the data in the timestamp of the data and the data base The timestamp of record;
Processing module, the timestamp for being suitable to the data in the caching is determined is corresponding with the data less than in the data base During the timestamp of data record, the data in the caching are deleted;Otherwise, by data in the caching and its corresponding Timestamp persistent storage is in the data base.
8. device according to claim 7, wherein, described device is further included:Time memory module, is suitable to data The data and its corresponding timestamp are preserved into the caching in storehouse.
9. the device according to claim 7 or 8, wherein, described device is further included:Storage processing module, is suitable to work as When detecting the caching failure, the data produced during the caching failure and its corresponding timestamp persistent storage are arrived In the data base.
10. according to the arbitrary described device of claim 7-9, wherein, the acquisition module is further used for:
Corresponding timestamp when obtaining each write of the data in the caching respectively, by corresponding timestamp during each write In maximum be defined as the timestamp of the data in the caching;
Obtain the timestamp of corresponding with data pieces of data record in the data base respectively, by pieces of data record when Between stab in maximum be defined as the timestamp of data record corresponding with the data in the data base.
CN201611249423.2A 2016-12-29 2016-12-29 Data persistence storage method and device based on cache Active CN106682193B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611249423.2A CN106682193B (en) 2016-12-29 2016-12-29 Data persistence storage method and device based on cache

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611249423.2A CN106682193B (en) 2016-12-29 2016-12-29 Data persistence storage method and device based on cache

Publications (2)

Publication Number Publication Date
CN106682193A true CN106682193A (en) 2017-05-17
CN106682193B CN106682193B (en) 2021-01-08

Family

ID=58873390

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611249423.2A Active CN106682193B (en) 2016-12-29 2016-12-29 Data persistence storage method and device based on cache

Country Status (1)

Country Link
CN (1) CN106682193B (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107589907A (en) * 2017-08-10 2018-01-16 上海壹账通金融科技有限公司 Data processing method, electronic equipment and computer-readable recording medium
CN109032505A (en) * 2018-06-26 2018-12-18 深圳忆联信息系统有限公司 Data read-write method, device, computer equipment and storage medium with timeliness
CN109408496A (en) * 2018-09-18 2019-03-01 沈文策 A kind of method and device reducing data redundancy
CN109815287A (en) * 2019-01-30 2019-05-28 网易(杭州)网络有限公司 A kind of data base access system
CN109902130A (en) * 2019-01-31 2019-06-18 北京明略软件系统有限公司 A kind of date storage method, data query method and apparatus, storage medium
CN110347545A (en) * 2019-05-21 2019-10-18 深圳壹账通智能科技有限公司 A kind of test method and device of business platform cache policy
CN111708783A (en) * 2020-06-18 2020-09-25 北京金山云网络技术有限公司 Data storage and data recovery method and device and electronic equipment
CN112434066A (en) * 2020-11-24 2021-03-02 深圳市前海迅智系统科技有限公司 Multi-channel high-frequency real-time data aggregation method and terminal
CN113778723A (en) * 2021-11-11 2021-12-10 中汽数据(天津)有限公司 Data playback method, electronic device and readable storage medium
CN114490775A (en) * 2022-02-11 2022-05-13 广东美房智高机器人有限公司 Robot real-time operation data processing method, device, server and medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075864A (en) * 2007-07-16 2007-11-21 腾讯科技(深圳)有限公司 Method for synchronizing and processing data, customer terminal equipment and servo
CN101257485A (en) * 2007-03-02 2008-09-03 华为技术有限公司 Web applied system and method
CN102024032A (en) * 2010-11-29 2011-04-20 广州明朝网络科技有限公司 Distributed data caching and persisting method and system based on Erlang
US20140115245A1 (en) * 2012-10-19 2014-04-24 Oracle International Corporation Apparatus system and method for providing raw data in a level-two cache
CN104599032A (en) * 2014-11-28 2015-05-06 国家电网公司 Distributed memory power grid construction method and system for resource management
CN105741070A (en) * 2016-04-18 2016-07-06 深圳马可孛罗科技有限公司 Rapid differentiation update and storage method for mass real-time shipping space and freight rate result data

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101257485A (en) * 2007-03-02 2008-09-03 华为技术有限公司 Web applied system and method
CN101075864A (en) * 2007-07-16 2007-11-21 腾讯科技(深圳)有限公司 Method for synchronizing and processing data, customer terminal equipment and servo
CN102024032A (en) * 2010-11-29 2011-04-20 广州明朝网络科技有限公司 Distributed data caching and persisting method and system based on Erlang
US20140115245A1 (en) * 2012-10-19 2014-04-24 Oracle International Corporation Apparatus system and method for providing raw data in a level-two cache
CN104599032A (en) * 2014-11-28 2015-05-06 国家电网公司 Distributed memory power grid construction method and system for resource management
CN105741070A (en) * 2016-04-18 2016-07-06 深圳马可孛罗科技有限公司 Rapid differentiation update and storage method for mass real-time shipping space and freight rate result data

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107589907A (en) * 2017-08-10 2018-01-16 上海壹账通金融科技有限公司 Data processing method, electronic equipment and computer-readable recording medium
CN107589907B (en) * 2017-08-10 2019-12-13 深圳壹账通智能科技有限公司 Data processing method, electronic device and computer readable storage medium
CN109032505A (en) * 2018-06-26 2018-12-18 深圳忆联信息系统有限公司 Data read-write method, device, computer equipment and storage medium with timeliness
CN109408496A (en) * 2018-09-18 2019-03-01 沈文策 A kind of method and device reducing data redundancy
CN109815287A (en) * 2019-01-30 2019-05-28 网易(杭州)网络有限公司 A kind of data base access system
CN109902130A (en) * 2019-01-31 2019-06-18 北京明略软件系统有限公司 A kind of date storage method, data query method and apparatus, storage medium
CN110347545A (en) * 2019-05-21 2019-10-18 深圳壹账通智能科技有限公司 A kind of test method and device of business platform cache policy
CN111708783A (en) * 2020-06-18 2020-09-25 北京金山云网络技术有限公司 Data storage and data recovery method and device and electronic equipment
CN112434066A (en) * 2020-11-24 2021-03-02 深圳市前海迅智系统科技有限公司 Multi-channel high-frequency real-time data aggregation method and terminal
CN113778723A (en) * 2021-11-11 2021-12-10 中汽数据(天津)有限公司 Data playback method, electronic device and readable storage medium
CN114490775A (en) * 2022-02-11 2022-05-13 广东美房智高机器人有限公司 Robot real-time operation data processing method, device, server and medium

Also Published As

Publication number Publication date
CN106682193B (en) 2021-01-08

Similar Documents

Publication Publication Date Title
CN106682193A (en) Device and method for data persistent storage on basis of cache
US8429134B2 (en) Distributed database recovery
CN102929750B (en) Nonvolatile media dirty region tracking
US8396840B1 (en) System and method for targeted consistency improvement in a distributed storage system
CN109086388B (en) Block chain data storage method, device, equipment and medium
US8560500B2 (en) Method and system for removing rows from directory tables
KR101870521B1 (en) Methods and systems for improving storage journaling
US8468134B1 (en) System and method for measuring consistency within a distributed storage system
US6675180B2 (en) Data updating apparatus that performs quick restoration processing
US11176110B2 (en) Data updating method and device for a distributed database system
US10261705B2 (en) Efficient data consistency verification for flash storage
CN103475716B (en) The method and system of data sharing is realized by shared storage
US10169391B2 (en) Index management
CN113111129A (en) Data synchronization method, device, equipment and storage medium
CN112579327B (en) Fault detection method, device and equipment
CN106557278A (en) A kind of method of data cached persistence
CN1696906A (en) Method, system and program product for autonomous error recovery for memory devices
CN111367926A (en) Data processing method and device for distributed system
US20140279943A1 (en) File system verification method and information processing apparatus
CN108021562B (en) Disk storage method and device applied to distributed file system and distributed file system
US10783042B2 (en) System and method of assessing and managing storage device degradation
WO2021027271A1 (en) Bad block information protection method and apparatus, computer device and storage medium
US7409519B2 (en) Synchronizing logical systems
US20100169572A1 (en) Data storage method, apparatus and system for interrupted write recovery
CN111949479B (en) Interactive system and index creation condition determining method and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant