CN103631676A - Snapshot data generating method and device for read-only snapshot - Google Patents
Snapshot data generating method and device for read-only snapshot Download PDFInfo
- Publication number
- CN103631676A CN103631676A CN201310548175.1A CN201310548175A CN103631676A CN 103631676 A CN103631676 A CN 103631676A CN 201310548175 A CN201310548175 A CN 201310548175A CN 103631676 A CN103631676 A CN 103631676A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- read
- snapshot
- materialized view
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a snapshot data generating method and device for a read-only snapshot. The method includes the steps that after a data modification instruction is received, it is judged that snapshot data of the read-only snapshot needs to be generated, and current version data of all nodes are obtained; when the number of the nodes having latest version data meets a redundancy rate, the snapshot data of the read-only snapshot are generated according to the current version data of all the nodes respectively. By the adoption of the technical scheme, the number of the nodes having the latest version data is compared with the redundancy rate, so that the snapshot data of the read-only snapshot are generated based on the current version data of all the nodes meeting the redundancy rate, the accuracy of the snapshot data of the read-only snapshot is ensured, and the problem that due to error superposition caused by errors in the snapshot data of the read-only snapshot, reliability of a distributed storage file system is affected finally is solved.
Description
Technical field
The present invention relates to computer application field, relate in particular to a kind of snapshot data generation method and device of read-only materialized view.
Background technology
At present, at NAS(Network Attached Storage; Network connection storage) in application, for the needs of file data protection, can apply COW(Copy On Write; The snapshot data of read-only materialized view) technology creates read-only materialized view in the different moment to file system, and the file system of preserving not in the same time by the snapshot data of this read-only materialized view of generating is videoed; Wherein, be illustrated in the mark that this creates a read-only materialized view sometime creating sometime read-only materialized view, and the snapshot data that generates this read-only materialized view after creating above-mentioned read-only materialized view represents that the data of the file system sometime based on above-mentioned mark generate the reflection of a file system, the data that comprise above-mentioned file system sometime in the snapshot data of this read-only materialized view.When file system breaks down or based on business needs itself, need to be using read-only materialized view sometime as target read-only materialized view, and when the data in file system are returned to the snapshot data of target read-only materialized view, only need to carry out rolling back action to read-only materialized view can return to the data in file system the snapshot data of target read-only materialized view.
Consult shown in Fig. 1, in distributed storage file system, conventionally adopting the device of redundant slices is a plurality of fixed length bursts by whole file cutting, and the form with file is stored to respectively in different memory nodes by this each fixed length burst.And, in order to obtain more quickly each fixed length burst, the distributing position information of each fixed length burst is stored in the metadata of distributed file system.
Consult shown in Fig. 2, in above-mentioned distributed storage file system, all data of same fixed length burst all belong to same data itemize and form an edition data, and when there is the data itemize of redaction data, the data itemize of redaction data covers the data itemize of legacy data.When guaranteeing the data in any node corresponding to fixed length burst to modify, the consistance that belongs to other node datas that this fixed length burst is corresponding, the modification of data in fixed length burst all be take to this data itemize as the base unit operation of modifying, in this data itemize, comprise data and corresponding check code in fixed length burst.Be specially: when receiving data modification indication, in the time of need to modifying to the data on any node, in existing distributed storage file system, the renewal of the data of all nodes in the whole data itemize that under this any node, fixed length burst is corresponding need to being modified, wherein, for the check code in this data itemize, need to recalculate.For example, consult shown in Fig. 2, data itemize 1 is the data itemize that fixed length burst (Chunk0) is corresponding, and this data itemize 1 comprises two data (D is stored in respectively in node one and node two) and a check code, and (P, also referred to as redundancy, be stored in node three), in the time need to modifying to any one data in data itemize 1, need to recalculate above-mentioned check code, and guarantee that three data in this data itemize 1 are refreshed simultaneously.
Based on above-mentioned distributed storage file system, in the data modification process to any node, may there is revising failed situation.For example, consult shown in Fig. 3, at the modification process node two of the versions of data three (Ver=3) to data itemize 1, synchronously do not revised, thereby cause in versions of data four node two to have the problem of shortage of data.Now, after file system being created to above-mentioned read-only materialized view one, before this shortage of data is repaired, if when the data of this read-only materialized view protection are modified, the data of the original protection in this read-only materialized view one (i.e. the current version data to each node before data modification) will be as legacy data, need to generate read-only materialized view one snapshot data according to this legacy data, missing data in this read-only materialized view one snapshot data in node two will be registered as the data of some old version data (as the data of version three), consult shown in Fig. 4, thereby cause the legacy data in read-only materialized view one to have versions of data number wrong problem.Consult shown in Fig. 5; after file system is created to read-only materialized view; if there is again the data modification failure of another node when the data of this read-only materialized view protection are modified before this shortage of data is repaired; by the stack leading to errors; cause snapshot data unavailable, thereby reduced the accuracy of data in distributed storage file system.
In sum, the low problem of distributed storage file system accuracy causing due to error in data in the snapshot data of read-only materialized view at present.
Summary of the invention
The embodiment of the present invention provides a kind of snapshot data generation method and device of read-only materialized view, in order to solve the low problem of distributed storage file system accuracy causing due to error in data in the snapshot data of read-only materialized view at present.
The concrete technical scheme that the embodiment of the present invention provides is as follows:
First aspect, provides a kind of snapshot data generation method of read-only materialized view, comprising:
Receive after data modification indication, when judgement need to generate the snapshot data of read-only materialized view, obtain current version data and the current data version number of each node; Wherein, the current version data of described each node are the data of each node before the data of each node not being modified according to the indication of described data modification, the versions of data number that described current data version number is each node before the data of each node not being modified according to described data modification indication;
When having the interstitial content of latest edition data and meet redundancy ratio, according to described current version data and the current data version number that has each node of latest edition data, generate the snapshot data of read-only materialized view.
In conjunction with first aspect, in the possible implementation of the first, judgement need to generate the snapshot data method of read-only materialized view, comprise: obtain the read-only materialized view creating for the last time before current time, and detect after the read-only materialized view of described last establishment, whether there is the snapshot data of the read-only materialized view of described last establishment; If current time exists the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view do not need to generate described last establishment; If current time does not exist the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view that need to generate described last establishment.
In conjunction with the possible implementation of the first of first aspect, in the possible implementation of the second, if having the current version data of any node is not latest edition data,, according to current version data and current version data number with other nodes that have latest edition data of described any node in same data itemize, the current version data of described any node and current data version number are repaired; Or the log of preserving according to this locality, repairs the current version data of described any node and current data version number.
In conjunction with the possible implementation of the second of first aspect, in the third possible implementation, after generating the snapshot data of described read-only materialized view, receive the data modification success response message that each node sends, when the successful interstitial content of data modification meets redundancy ratio, the Update Table based on carrying in described data modification indication generates redaction data; Wherein, described redaction packet is containing described Update Table and the versions of data number of each node; By the snapshot data of described read-only materialized view, and described redaction data are carried out affairs submission.
In conjunction with the third possible implementation of first aspect, in the 4th kind of possible implementation, the data modification failure response message sending from each node receiving, obtain the node identification of the node of data modification failure; And according to the Update Table carrying in described data modification indication and the data modification success response message that each node receiving sends, obtain the successfully amended data of each node of data modification; Respectively according to check code corresponding to the node identification of each node of described data modification failure and with the amended data of other nodes of described each node in same data itemize, adopt redundancy check algorithm to repair the data of each node of described data modification failure; Data according to the data of the successful all nodes of described data modification and after repairing, generate redaction data respectively.
Second aspect, provides a kind of snapshot data generating apparatus of read-only materialized view, comprising:
Judging unit, for receiving after data modification indication, judges whether to generate the snapshot data of read-only materialized view, and judged result is sent to acquiring unit;
Acquiring unit, the judged result sending for receiving judging unit, and when judgment result is that the snapshot data that need to generate read-only materialized view, obtain current version data and the current data version number of each node, and the current version data of each node and current data version number are sent to generation unit; Wherein, the current version data of described each node are the data of each node before the data of each node not being modified according to the indication of described data modification, the versions of data number that described current data version number is each node before the data of each node not being modified according to described data modification indication;
Generation unit, for receiving current version data and the current data version number of each node of acquiring unit transmission, when having the interstitial content of latest edition data and meet redundancy ratio, according to described current version data and the current data version number that has each node of latest edition data, generate the snapshot data of read-only materialized view.
In conjunction with second aspect, in the possible implementation of the first, described judging unit, specifically for: obtain the read-only materialized view creating for the last time before current time, and detect after the read-only materialized view of described last establishment, whether there is the snapshot data of the read-only materialized view of described last establishment; If current time exists the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view do not need to generate described last establishment; If current time does not exist the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view that need to generate described last establishment.
In conjunction with the possible implementation of the first of second aspect, in the possible implementation of the second, also comprise reparation unit, be used for: if there are the current version data of any node, be not latest edition data,, according to current version data and current version data number with other nodes that have latest edition data of described any node in same data itemize, the current version data of described any node and current data version number are repaired; Or the log of preserving according to this locality, repairs the current version data of described any node and current data version number.
In conjunction with the possible implementation of the second of second aspect, in the third possible implementation, also comprise commit unit, be used for: after generating the snapshot data of described read-only materialized view, receive the data modification success response message that each node sends, when the successful interstitial content of data modification meets redundancy ratio, the Update Table based on carrying in described data modification indication generates redaction data; Wherein, described redaction packet is containing described Update Table and the versions of data number of each node; By the snapshot data of described read-only materialized view, and described redaction data are carried out affairs submission.
The third possible implementation in conjunction with second aspect, in the 4th kind of possible implementation, described commit unit, specifically for: the data modification failure response message sending from each node receiving, obtain the node identification of the node of data modification failure; And according to the Update Table carrying in described data modification indication and the data modification success response message that each node receiving sends, obtain the successfully amended data of each node of data modification; Respectively according to check code corresponding to the node identification of each node of described data modification failure and with the amended data of other nodes of described each node in same data itemize, adopt redundancy check algorithm to repair the data of each node of described data modification failure; Data according to the data of the successful all nodes of described data modification and after repairing, generate redaction data respectively.
In the embodiment of the present invention, after receiving data modification indication, when judgement need to generate the snapshot data of read-only materialized view, obtain the current version data of each node; When having the interstitial content of latest edition data and meet redundancy ratio, respectively according to the current version data of above-mentioned each node, generate the snapshot data of read-only materialized view.Adopt technical solution of the present invention, by current time being had to interstitial content and the redundancy ratio of latest edition data, compare, thereby the current version data based on meeting each node of redundancy ratio generate the snapshot data of read-only materialized view, guarantee the accuracy of the snapshot data of read-only materialized view, avoided the mistake stack that the snapshot data mistake due to read-only materialized view causes finally to affect the problem of distributed storage file system reliability.
Accompanying drawing explanation
Fig. 1 is distributed storage file system files storage schematic diagram in prior art;
Fig. 2 is data modification schematic diagram one in distributed storage file system in prior art;
Fig. 3 is data modification schematic diagram two in distributed storage file system in prior art;
Fig. 4 is data modification schematic diagram three in distributed storage file system in prior art;
Fig. 5 is data modification schematic diagram four in distributed storage file system in prior art;
Fig. 6 is system architecture schematic diagram in the embodiment of the present invention;
Fig. 7 is the snapshot data product process figure of read-only materialized view in the embodiment of the present invention;
Fig. 8 is the snapshot data product process figure of read-only materialized view under concrete application scenarios in the embodiment of the present invention;
Fig. 9 is the snapshot data generating apparatus structural representation of read-only materialized view in the embodiment of the present invention;
Figure 10 is that the snapshot data of read-only materialized view in the embodiment of the present invention generates device structure schematic diagram.
Embodiment
In order to solve the low problem of distributed storage file system accuracy causing due to error in data in read-only materialized view data at present.In the embodiment of the present invention, after receiving data modification indication, when judgement need to generate the snapshot data of read-only materialized view, obtain the current version data of each node; When having the interstitial content of latest edition data and meet redundancy ratio, respectively according to the current version data of above-mentioned each node, generate the snapshot data of read-only materialized view.Adopt technical solution of the present invention, by current time being had to interstitial content and the redundancy ratio of latest edition data, compare, thereby the current version data based on meeting each node of redundancy ratio generate the snapshot data of read-only materialized view, guarantee the accuracy of the snapshot data of read-only materialized view, avoided the mistake stack that the snapshot data mistake due to read-only materialized view causes finally to affect the problem of distributed storage file system reliability.
Consult and Figure 6 shows that system architecture schematic diagram in the embodiment of the present invention, this system comprises client, and distributed storage file system server.
Based on above-mentioned file system framework, to consult shown in Fig. 7, in the embodiment of the present invention, the snapshot data detailed process that generates read-only materialized view is:
Step 700: receive after data modification indication, when judgement need to generate the snapshot data of read-only materialized view, obtain current version data and the current data version number of each node.
In the embodiment of the present invention, in above-mentioned data modification indication, comprise Update Table, according to this Update Table, the data in distributed storage file system are modified.
Optionally, judgement need to generate the method for the snapshot data of read-only materialized view, be specially: obtain the read-only materialized view (hereinafter to be referred as read-only materialized view) creating for the last time before current time, and detect after this read-only materialized view, whether have the snapshot data of above-mentioned read-only materialized view; If current time exists the snapshot data of above-mentioned read-only materialized view, judge the snapshot data that does not need to generate this read-only materialized view; If current time does not exist the snapshot data of above-mentioned read-only materialized view, judge the snapshot data that need to generate this read-only materialized view.For example, consult shown in Fig. 4, after the versions of data four of data itemize 1, create read-only materialized view, if the versions of data five of generated data itemize 1 after the current version data of any node in data itemize 1 being modified after creating read-only materialized view, owing to there not being the versions of data of data itemize 1 after creating read-only materialized view, need to create the snapshot data of read-only materialized view, for the current version data of preserving above-mentioned each node, the i.e. data of versions of data four correspondences of protected data itemize 1; If after creating read-only materialized view to the modify versions of data six of generated data itemize 1 of the current version data of any node in data itemize 1, owing to there being snapshot data in distributed storage file system, therefore, for above-mentioned current version data modification, (i.e. the modification to versions of data five) do not generate the snapshot data of read-only materialized view.
In the embodiment of the present invention; be illustrated in the mark that this creates a read-only materialized view sometime creating sometime read-only materialized view; and the snapshot data that generates this read-only materialized view after creating above-mentioned read-only materialized view represents that the data based on above-mentioned file system sometime generate the reflection of a file system, in the snapshot data of this read-only materialized view, protect current version data and the current data version number of above-mentioned file system sometime.Wherein, current data version number can be numeral, also can represent for other other marks of versions of data numbering.
In addition, in distributed storage file system, in the data modification indication receiving, carry Update Table, according to this Update Table, upgrade current version data and the current data version number of corresponding data itemize.
In said process, the current version data of each node are the data of each node before the data of each node not being modified according to described data modification indication, and current data version number is the versions of data number of each node before the data of each node not being modified according to data modification indication.For example, consult shown in Fig. 4, before creating read-only materialized view, the current version data of node one are the data of versions of data four correspondences, the current version data of node two are empty, with old version data, supplement, the current version data of node two are the data of version three correspondences, and the current version data of node three are the data of version four correspondences; The current data version number of node one is versions of data four, and the current data version number of node two is version three, and the current data version number of node three is version four.
Step 710: when having the interstitial content of latest edition data and meet redundancy ratio, have respectively current version data and the current data version number of each node of latest edition data according to this, generate the snapshot data of read-only materialized view.
In the embodiment of the present invention, if distributed storage file system is the system that is configured to N piece of data and M part redundancy, meets redundancy ratio and be in any data itemize, do not exist the interstitial content of shortage of data to be more than or equal to N.
When the interstitial content that has latest edition data in any data itemize meets redundancy ratio, according to the current version data of each node and current data version number, generate the snapshot data of read-only materialized view, be specially: when any data itemize, when having the interstitial content of latest edition data and be more than or equal to N, if having the current version data of any node is not edition data, according to current version data and current data version number with other nodes that have latest edition data of this any node in same data itemize, adopt redundancy check method to repair the current version data of this any node and current data version number, or, the log of preserving according to this locality, the current version data of above-mentioned any node and current version data number are repaired, according to current version data and the current version data number of each node after repairing, generate the snapshot data of read-only materialized view.Wherein, above-mentioned N represents in this distributed storage file system the umber of data in any data itemize; In log, store the amendment record to any node in distributed system.
In said process, any node has current data version number that latest edition data represent this any node for correct, and whether the current data version number of this any node correctly can be according to judging with the current data version number of other nodes of this any node in same data itemize.Because the characteristic of file system is that new versions of data covers old versions of data, after the data in file system are modified, the current data version number of file system (being version number) is and increases progressively state.Therefore, obtain current data version number maximum in the current data version number with other nodes of above-mentioned any node in same data itemize, if the current data version number of this any node equals the current data version number of above-mentioned maximum, the current data version number of this any node is correct.For example, consult shown in Fig. 4, data itemize in version four correspondences (comprises node one, node two, node three) in, the current data version number of node one is version four, node two is owing to revising unsuccessfully, and the version number that saves historical data, the current data version number that is node two is version three, the current data version number of node three is version four, as can be seen here, in this data itemize, maximum current data version number is version four, the current data version number of node one and the current data version number of node three are correctly, and the current data version number of node two is mistake.
Situation for the current data version number mistake of any node of above-mentioned existence, can be referring to following example: consult shown in Fig. 5, after after creating read-only materialized view one, the data of data itemize 1 modify in to distributed storage file system, generating snapshot data, the data of protecting due to this snapshot data are the data of versions of data four correspondences of data itemize 1, and there is shortage of data problem in the node two of data itemize 1, versions of data number in the node two of now snapshot data protection is versions of data three, thereby the problem that causes the current data version number mistake of the node two protected in read-only materialized view one.Based on this technical matters, in the embodiment of the present invention, according to current version data and the current data version number with the node one of this node two in same data itemize, and the current version data of node three and current data version number, adopt redundancy check method to repair the current version data of this node two and current data version number; Or the log of preserving according to this locality, repairs the current version data of node two; According to the current version data of each node after repairing, generate the snapshot data of read-only materialized view.In Fig. 5, after generating the snapshot data of read-only materialized view one, because node one is revised unsuccessfully again, the snapshot data failed regeneration that causes node one, but due to the time update of data of node two, the snapshot data of read-only materialized view one still meets redundancy ratio, it is available versions of data, thereby guaranteed the accuracy of data in the snapshot data of the read-only materialized view that generates, and then effectively improved the reliability of system.
Further, after generating the snapshot data of read-only materialized view, the data modification success response message sending according to each node receiving, when the successfully modified interstitial content of decision data meets redundancy ratio, the Update Table based on carrying in above-mentioned data modification indication generates redaction data; The snapshot data of read-only materialized view and redaction data are carried out to affairs submission.
Further, said process only meets the situation of redundancy ratio for the successful interstitial content of data modification, and for any data itemize, the successful interstitial content of data modification can equal the number of all nodes, also can be less than the number of all nodes.When the successful interstitial content of data modification equals the number of all nodes, represent that the data of all nodes are all successfully modified, now directly according to revising the Update Table carrying in indication, generate redaction data; When the successful interstitial content of data modification is less than the number of all nodes, represent to exist the node of data modification failure.
Wherein, for the treatment scheme of the node of this data modification failure, be: when the data modification failure response message that receives each node and send, obtain the node identification of the node of data modification failure; According to the amended data of check code corresponding to the node identification of each node of data modification failure and other nodes in same data itemize corresponding to each node, adopt redundancy check algorithm to repair the data of the node of described data modification failure respectively; Data according to the data of the successful all nodes of data modification and after repairing, generate redaction data respectively.Optionally, above-mentioned data repair complete after, in log recording, record is this time revised, so that when wrong, the snapshot data of this read-only materialized view is repaired appears in the snapshot data of the read-only materialized view creating afterwards.Special, when existing any node not return to any response message, this any node is also the node of data modification failure.
In said process, when the successful interstitial content of data modification meets redundancy ratio, and while there is the node of data modification failure, can, according to the amended data of other nodes in same data itemize, adopt redundancy check method repair data to revise the data of failed node.For example, consult shown in Fig. 4, for data itemize 1, comprise three nodes, and there is shortage of data on the node two of the versions of data four of data itemize 1, and data on node one and node three are all correct, now, in data itemize 1, the successful interstitial content of data modification meets redundancy ratio, can adopt redundancy check algorithm, by the data on node one and node three, the data on node two be repaired.
Special, when the successfully modified interstitial content of decision data does not meet redundancy ratio, to client returning remind message, there is mistake in prompting user current data version, needs user to re-enter.
Consult shown in Fig. 8, below in conjunction with concrete application scenarios, describe in detail according to the snapshot data of read-only materialized view and carry out the process of submitting in kind:
Step 800: receive data modification indication.
Step 810: judge whether to generate the snapshot data of read-only materialized view, if so, perform step 820; Otherwise, finish current flow process.
Step 820: the current version data and the current data version number that obtain each node.
Step 830: whether the correct interstitial content of current data version number that judges above-mentioned each node meets redundancy ratio, if so, performs step 840; Otherwise, perform step 850.
Step 840: when not there is not the node of current data version number mistake, respectively according to the current version data of each node, generate the snapshot data of read-only materialized view; When there being the node of current data version number mistake, the log of preserving according to this locality, repairs the current version data of each node and current data version number, and according to the data of the read-only materialized view after repairing, generates the snapshot data of read-only materialized view.
In the embodiment of the present invention, node for current data version number mistake, can also be by the current version data with other nodes of number wrong node of this current versions of data in same data itemize, current version data and the current data version number of number wrong node of this current versions of data are repaired.
Step 850: returning remind message, prompting user data is revised unsuccessfully.
Step 860: the data modification success response message sending according to each node receiving, judges that whether the successful interstitial content of data modification meets redundancy ratio, if so, performs step 870; Otherwise, perform step 880.
Step 870: the Update Table based on carrying in above-mentioned data modification indication generates redaction data, and the snapshot data of above-mentioned read-only materialized view and redaction data are carried out to affairs submission.
In the embodiment of the present invention, when the successful interstitial content of data modification equals the number of all nodes, represent that the data of all nodes are all successfully modified, now directly according to revising the Update Table carrying in indication, generate redaction data; When the successful interstitial content of data modification is less than the number of all nodes, adopt redundancy check algorithm to repair the data of the node of described data modification failure; Data according to the data of the successful all nodes of data modification and after repairing, generate redaction data respectively.
Step 880: returning remind message, prompting user current data version exists mistake, needs user to re-enter.
In the embodiment of the present invention, the situation that does not meet redundancy ratio for the successful interstitial content of data modification has only provided the mode repair data version that user initiatively revises.
Based on technique scheme, consult shown in Fig. 9, the invention provides a kind of snapshot data generating apparatus of read-only materialized view, comprise judging unit 90, acquiring unit 91, and generation unit 92, wherein:
Judging unit 90, for receiving after data modification indication, judges whether to generate the snapshot data of read-only materialized view, and judged result is sent to acquiring unit 91;
Acquiring unit 91, the judged result sending for receiving judging unit 90, and when judgment result is that the snapshot data that need to generate read-only materialized view, obtain current version data and the current data version number of each node, and the current version data of each node and current data version number are sent to generation unit 92; Wherein, the current version data of described each node are the data of each node before the data of each node not being modified according to the indication of described data modification, the versions of data number that described current data version number is each node before the data of each node not being modified according to described data modification indication;
Wherein, judging unit 90 specifically for: obtain the last read-only materialized view creating before current time, and detect after the read-only materialized view of described last establishment, whether have the snapshot data of the read-only materialized view of described last establishment; If current time exists the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view do not need to generate described last establishment; If current time does not exist the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view that need to generate described last establishment.
Further, said apparatus also comprises repairs unit 93, be used for: if there are the current version data of any node, be not latest edition data,, according to current version data and current version data number with other nodes that have latest edition data of described any node in same data itemize, the current version data of described any node and current data version number are repaired; Or the log of preserving according to this locality, repairs the current version data of described any node and current data version number.
Further, said apparatus also comprises commit unit 94, be used for: after generating the snapshot data of described read-only materialized view, receive the data modification success response message that each node sends, when the successful interstitial content of data modification meets redundancy ratio, the Update Table based on carrying in described data modification indication generates redaction data; Wherein, described redaction packet is containing described Update Table and the versions of data number of each node; By the snapshot data of described read-only materialized view, and described redaction data are carried out affairs submission.
Commit unit 94 specifically for: the data modification failure response message sending from each node receiving, obtain the node identification of the node of data modification failure; And according to the Update Table carrying in described data modification indication and the data modification success response message that each node receiving sends, obtain the successfully amended data of each node of data modification; Respectively according to check code corresponding to the node identification of each node of described data modification failure and with the amended data of other nodes of described each node in same data itemize, adopt redundancy check algorithm to repair the data of each node of described data modification failure; Data according to the data of the successful all nodes of described data modification and after repairing, generate redaction data respectively.
Based on technique scheme, consult shown in Figure 10, the present invention also provides a kind of snapshot data of read-only materialized view to generate equipment, comprises transceiver 100, storer 101, and processor 102, wherein:
Storer 101, for storing application program;
Receive after the data modification indication of transceiver 100 transmissions, when judgement need to generate the snapshot data of read-only materialized view, obtain current version data and the current data version number of each node; When having the interstitial content of latest edition data and meet redundancy ratio, according to described current version data and the current data version number that has each node of latest edition data, generate the snapshot data of read-only materialized view; Wherein, the current version data of described each node are the data of each node before the data of each node not being modified according to the indication of described data modification, the versions of data number that described current data version number is each node before the data of each node not being modified according to described data modification indication.
Wherein, above-mentioned processor 102, specifically for: obtain the read-only materialized view creating for the last time before current time, and detect after the read-only materialized view of described last establishment, whether have the snapshot data of the read-only materialized view of described last establishment; If current time exists the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view do not need to generate described last establishment; If current time does not exist the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view that need to generate described last establishment.
Further, above-mentioned processor 102, also for: if there are the current version data of any node, be not latest edition data,, according to current version data and current version data number with other nodes that have latest edition data of described any node in same data itemize, the current version data of described any node and current data version number are repaired; Or the log of preserving according to this locality, repairs the current version data of described any node and current data version number.
Further, above-mentioned processor 102, also for: after generating the snapshot data of described read-only materialized view, receive the data modification success response message that each node sends, when the successful interstitial content of data modification meets redundancy ratio, the Update Table based on carrying in described data modification indication generates redaction data; Wherein, described redaction packet is containing described Update Table and the versions of data number of each node; By the snapshot data of described read-only materialized view, and described redaction data are sent to transceiver 100, make transceiver 100 by the snapshot data of described read-only materialized view, and described redaction data are carried out affairs submission.
Further, transceiver 100, the snapshot data of the read-only materialized view also sending for: receiving processor 102, and after redaction data, by the snapshot data of read-only materialized view, and redaction data are carried out affairs submission.
Further, processor 102, also for the data modification failure response message that sends from each node receiving, obtains the node identification of the node of data modification failure; And according to the Update Table carrying in described data modification indication and the data modification success response message that each node receiving sends, obtain the successfully amended data of each node of data modification; Respectively according to check code corresponding to the node identification of each node of described data modification failure and with the amended data of other nodes of described each node in same data itemize, adopt redundancy check algorithm to repair the data of each node of described data modification failure; Data according to the data of the successful all nodes of described data modification and after repairing, generate redaction data respectively.
In sum, in the embodiment of the present invention, receive after data modification indication, when judgement need to generate the snapshot data of read-only materialized view, obtain current version data and the current data version number of each node; When having the interstitial content of latest edition data and meet redundancy ratio, according to this, have respectively current version data and the current data version number of each node of latest edition data, generate the snapshot data of read-only materialized view.Adopt technical solution of the present invention, by current time being had to interstitial content and the redundancy ratio of latest edition data, compare, thereby the current version data based on meeting each node of redundancy ratio generate the snapshot data of read-only materialized view, guarantee the accuracy of the snapshot data of read-only materialized view, avoided the mistake stack that the snapshot data mistake due to read-only materialized view causes finally to affect the problem of distributed storage file system reliability.
Those skilled in the art should understand, embodiments of the invention can be provided as device, system or computer program.Therefore, the present invention can adopt complete hardware implementation example, implement software example or in conjunction with the form of the embodiment of software and hardware aspect completely.And the present invention can adopt the form that wherein includes the upper computer program of implementing of computer-usable storage medium (including but not limited to magnetic disk memory, CD-ROM, optical memory etc.) of computer usable program code one or more.
The present invention is with reference to describing according to process flow diagram and/or the block scheme of the device of the embodiment of the present invention, equipment (system) and computer program.Should understand can be in computer program instructions realization flow figure and/or block scheme each flow process and/or the flow process in square frame and process flow diagram and/or block scheme and/or the combination of square frame.Can provide these computer program instructions to the processor of multi-purpose computer, special purpose computer, Embedded Processor or other programmable data processing device to produce a machine, the instruction of carrying out by the processor of computing machine or other programmable data processing device is produced for realizing the device in the function of flow process of process flow diagram or a plurality of flow process and/or square frame of block scheme or a plurality of square frame appointments.
These computer program instructions also can be stored in energy vectoring computer or the computer-readable memory of other programmable data processing device with ad hoc fashion work, the instruction that makes to be stored in this computer-readable memory produces the manufacture that comprises command device, and this command device is realized the function of appointment in flow process of process flow diagram or a plurality of flow process and/or square frame of block scheme or a plurality of square frame.
These computer program instructions also can be loaded in computing machine or other programmable data processing device, make to carry out sequence of operations step to produce computer implemented processing on computing machine or other programmable devices, thereby the instruction of carrying out is provided for realizing the step of the function of appointment in flow process of process flow diagram or a plurality of flow process and/or square frame of block scheme or a plurality of square frame on computing machine or other programmable devices.
Although described the preferred embodiments of the present invention, once those skilled in the art obtain the basic creative concept of cicada, can make other change and modification to these embodiment.So claims are intended to all changes and the modification that are interpreted as comprising preferred embodiment and fall into the scope of the invention.
Obviously, those skilled in the art can carry out various changes and modification and not depart from the spirit and scope of the embodiment of the present invention the embodiment of the present invention.Like this, if within these of the embodiment of the present invention are revised and modification belongs to the scope of the claims in the present invention and equivalent technologies thereof, the present invention is also intended to comprise these changes and modification interior.
Claims (10)
1. the snapshot data generation method of a read-only materialized view, is characterized in that, comprising:
Receive after data modification indication, when judgement need to generate the snapshot data of read-only materialized view, obtain current version data and the current data version number of each node; Wherein, the current version data of described each node are the data of each node before the data of each node not being modified according to the indication of described data modification, the versions of data number that described current data version number is each node before the data of each node not being modified according to described data modification indication;
When having the interstitial content of latest edition data and meet redundancy ratio, according to described current version data and the current data version number that has each node of latest edition data, generate the snapshot data of read-only materialized view.
2. the method for claim 1, is characterized in that, judgement need to generate the snapshot data of read-only materialized view, comprising:
Obtain the read-only materialized view creating for the last time before current time, and detect after the read-only materialized view of described last establishment, whether have the snapshot data of the read-only materialized view of described last establishment;
If current time exists the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view do not need to generate described last establishment;
If current time does not exist the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view that need to generate described last establishment.
3. method as claimed in claim 1 or 2, is characterized in that, before generating the snapshot data of read-only materialized view, further comprises:
If having the current version data of any node is not latest edition data,, according to current version data and current version data number with other nodes that have latest edition data of described any node in same data itemize, the current version data of described any node and current data version number are repaired; Or the log of preserving according to this locality, repairs the current version data of described any node and current data version number.
4. method as claimed in claim 3, is characterized in that, after generating the snapshot data of described read-only materialized view, further comprises:
Receive the data modification success response message that each node sends, when the successful interstitial content of data modification meets redundancy ratio, the Update Table based on carrying in described data modification indication generates redaction data; Wherein, described redaction packet is containing described Update Table and the versions of data number of each node;
By the snapshot data of described read-only materialized view, and described redaction data are carried out affairs submission.
5. method as claimed in claim 4, is characterized in that, the Update Table based on carrying in described data modification indication generates redaction data, specifically comprises:
The data modification failure response message sending from each node receiving, obtain the node identification of the node of data modification failure; And
According to the Update Table carrying in described data modification indication and the data modification success response message that each node receiving sends, obtain the successfully amended data of each node of data modification;
Respectively according to check code corresponding to the node identification of each node of described data modification failure and with the amended data of other nodes of described each node in same data itemize, adopt redundancy check algorithm to repair the data of each node of described data modification failure;
Data according to the data of the successful all nodes of described data modification and after repairing, generate redaction data respectively.
6. a snapshot data generating apparatus for read-only materialized view, is characterized in that, comprising:
Judging unit, for receiving after data modification indication, judges whether to generate the snapshot data of read-only materialized view, and judged result is sent to acquiring unit;
Acquiring unit, the judged result sending for receiving judging unit, and when judgment result is that the snapshot data that need to generate read-only materialized view, obtain current version data and the current data version number of each node, and the current version data of each node and current data version number are sent to generation unit; Wherein, the current version data of described each node are the data of each node before the data of each node not being modified according to the indication of described data modification, the versions of data number that described current data version number is each node before the data of each node not being modified according to described data modification indication;
Generation unit, for receiving current version data and the current data version number of each node of acquiring unit transmission, when having the interstitial content of latest edition data and meet redundancy ratio, according to described current version data and the current data version number that has each node of latest edition data, generate the snapshot data of read-only materialized view.
7. device as claimed in claim 6, is characterized in that, described judging unit, specifically for:
Obtain the read-only materialized view creating for the last time before current time, and detect after the read-only materialized view of described last establishment, whether have the snapshot data of the read-only materialized view of described last establishment; If current time exists the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view do not need to generate described last establishment; If current time does not exist the snapshot data of the read-only materialized view of described last establishment, judge the snapshot data of the read-only materialized view that need to generate described last establishment.
8. the device as described in claim 6 or 7, is characterized in that, also comprises reparation unit, for:
If having the current version data of any node is not latest edition data,, according to current version data and current version data number with other nodes that have latest edition data of described any node in same data itemize, the current version data of described any node and current data version number are repaired; Or the log of preserving according to this locality, repairs the current version data of described any node and current data version number.
9. device as claimed in claim 8, is characterized in that, also comprises commit unit, for:
After generating the snapshot data of described read-only materialized view, receive the data modification success response message that each node sends, when the successful interstitial content of data modification meets redundancy ratio, the Update Table based on carrying in described data modification indication generates redaction data; Wherein, described redaction packet is containing described Update Table and the versions of data number of each node; By the snapshot data of described read-only materialized view, and described redaction data are carried out affairs submission.
10. device as claimed in claim 9, is characterized in that, described commit unit, specifically for:
The data modification failure response message sending from each node receiving, obtain the node identification of the node of data modification failure; And according to the Update Table carrying in described data modification indication and the data modification success response message that each node receiving sends, obtain the successfully amended data of each node of data modification; Respectively according to check code corresponding to the node identification of each node of described data modification failure and with the amended data of other nodes of described each node in same data itemize, adopt redundancy check algorithm to repair the data of each node of described data modification failure; Data according to the data of the successful all nodes of described data modification and after repairing, generate redaction data respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310548175.1A CN103631676B (en) | 2013-11-06 | 2013-11-06 | The snapshot data of a kind of read-only materialized view generates method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310548175.1A CN103631676B (en) | 2013-11-06 | 2013-11-06 | The snapshot data of a kind of read-only materialized view generates method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103631676A true CN103631676A (en) | 2014-03-12 |
| CN103631676B CN103631676B (en) | 2016-08-24 |
Family
ID=50212763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310548175.1A Active CN103631676B (en) | 2013-11-06 | 2013-11-06 | The snapshot data of a kind of read-only materialized view generates method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103631676B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109857593A (en) * | 2019-01-21 | 2019-06-07 | 北京工业大学 | A kind of data center's log missing data restoration methods |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101504625A (en) * | 2009-03-04 | 2009-08-12 | 成都市华为赛门铁克科技有限公司 | Method for implementing independent disk redundancy array, solid state disk and electronic equipment |
| US7987156B1 (en) * | 2005-06-10 | 2011-07-26 | American Megatrends, Inc. | Method, system, apparatus, and computer-readable medium for taking and managing snapshots of a storage volume |
| CN102549554A (en) * | 2009-09-29 | 2012-07-04 | 美光科技公司 | Stripe-based memory operation |
| CN102968381A (en) * | 2012-11-19 | 2013-03-13 | 浪潮电子信息产业股份有限公司 | Method for improving snapshot performance by using solid state disk |
| CN103336727A (en) * | 2013-06-20 | 2013-10-02 | 酷丰实业（上海）有限公司 | NAND flash memory storage device and data operation method for same |
-
2013
- 2013-11-06 CN CN201310548175.1A patent/CN103631676B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7987156B1 (en) * | 2005-06-10 | 2011-07-26 | American Megatrends, Inc. | Method, system, apparatus, and computer-readable medium for taking and managing snapshots of a storage volume |
| CN101504625A (en) * | 2009-03-04 | 2009-08-12 | 成都市华为赛门铁克科技有限公司 | Method for implementing independent disk redundancy array, solid state disk and electronic equipment |
| CN102549554A (en) * | 2009-09-29 | 2012-07-04 | 美光科技公司 | Stripe-based memory operation |
| CN102968381A (en) * | 2012-11-19 | 2013-03-13 | 浪潮电子信息产业股份有限公司 | Method for improving snapshot performance by using solid state disk |
| CN103336727A (en) * | 2013-06-20 | 2013-10-02 | 酷丰实业（上海）有限公司 | NAND flash memory storage device and data operation method for same |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109857593A (en) * | 2019-01-21 | 2019-06-07 | 北京工业大学 | A kind of data center's log missing data restoration methods |
| CN109857593B (en) * | 2019-01-21 | 2020-08-28 | 北京工业大学 | Data center log missing data recovery method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103631676B (en) | 2016-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102145136B1 (en) | Data processing method and device | |
| CN111078662B (en) | Block chain data storage method and device | |
| CN106802892B (en) | Method and equipment for checking consistency of main and standby data | |
| CN103034564B (en) | Data disaster tolerance drilling method, data disaster tolerance practice device and system | |
| US10212229B2 (en) | Reliable data storage for decentralized computer systems | |
| CN101198937A (en) | Secure storage management system and method | |
| CN108932249B (en) | Method and device for managing file system | |
| KR20130048595A (en) | Apparatus and method for filtering duplication data in restricted resource environment | |
| WO2017028375A1 (en) | Version upgrading method and system | |
| CN102217230A (en) | Provisioning method and system | |
| CN105335143A (en) | Business processing method and apparatus | |
| CN105516230A (en) | Data processing method and device | |
| KR20140063138A (en) | Rfid tag and method of updating rfid tag's key | |
| CN110147273A (en) | A kind of task executing method and device | |
| CN111930489A (en) | Task scheduling method, device, equipment and storage medium | |
| CN106682017A (en) | Database update method and device | |
| US8838545B2 (en) | Incremental and prioritized restoration of blocks | |
| CN109614268A (en) | A kind of restoration methods of cloud Backup Data, apparatus and system | |
| CN103631676A (en) | Snapshot data generating method and device for read-only snapshot | |
| CN110968456B (en) | Method and device for processing fault disk in distributed storage system | |
| CN105790975A (en) | Service processing operation execution method and device | |
| CN103488549A (en) | Roll-back processing system and roll-back processing method for multi-mirror-image data | |
| CN106371919A (en) | Shuffle data caching method based on mapping-reduction calculation model | |
| CN112910697B (en) | Fault processing method and device | |
| EP3119043A1 (en) | Flow table entry management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |