Consistent putting inserted and continuous data protection system of volume level and the method recovering to support
Technical field
The invention belongs to technical field of computer data storage, the especially consistent continuous data protection system its implementation of volume level inserting and recover to support of putting is for the continuous data protection system consistance of volume level recovers to provide auxiliary support means.
Technical background
Continuous data protection is a kind of continuous time point resist technology of data, and its effect is to finish the fault recovery of point any time at instant of failure, fundamentally solves congenital vulnerability issue such as low recovery capability and non-meticulous time strategy in traditional redundancy technique.Continuous data protection system can be caught all data change of protected system, and the data variation that is writing down all historical data states is provided.For example, if the data variation of same address physically 10 times, continuous data protection system changes this 10 secondary data all to be noted, and sends to long-range backup server.When breaking down in protected system like this, continuously protection system can guarantee that protected system puts recovery at any time by the teledata alternate device.
From operation, the implementation method of continuous data protection system can be divided into based on piece, based on file with based on three types of the System and method fors of using.Block-based implementation method is positioned on physical storage or the logical volume management layer.When data block was written into primary memory, the data trnascription that writes was just caught and is stored into by continuous data protection system in the independent place.Similarly, just in time be positioned on the file system based on the implementation method of the continuous data protection system of file, it can catch file system data and metadata incident, as document creation, modification or deletion etc.Then be located immediately among the shielded application-specific based on the implementation method of the continuous data protection system of using, with concrete application close-coupled.
Data consistency is the key index that is related to continuous data protection restore data availability, and only the continuous data protection scheme of guaranteed data consistency could guarantee that data are available fast.Continuous data protection recovers for the user provides a large amount of arbitrfary points, but can not guarantee that the data in magnetic disk that each time point recovers is consistent.Certainly to guarantee data consistent, be file system also Database Systems good checking mechanism all is provided, but these checking mechanisms all are postmortem.For this reason, delivered the paper of a time " a kind of effective disaster recovery point authentication scheme " on the 6th file that the Acker Saudi Arabia Fu Erma people such as (Akshat Verma) of American Business Machine Corp (IBM) holds at California, USA Joseph of Arimathea, Saint in 2008 and the memory technology nd Annual Meeting collection, this article has proposed a kind of effective recovery point authentication scheme SWEEPER, SWEEPER is by setting up system event, start event databases such as item as software upgrading, fire wall setting and renewal, and set up the probabilistic relation between the system event and the system failure.When recovering, the quick scanning algorithm that SWEEPER provides can provide the consistent fast auxiliary support function of putting by the probability evaluation method according to the RPO target component of user's selection.But SWEEPER need define the probabilistic relation of many system events and the incident and the system failure, and in actual applications, is difficult to set up the probabilistic relation between system event and the fault, and the accuracy of recovering is not high.2008 in the 2008 credible distribution system scientific seminars that Italy takes that the Pori holds, Lu Maohua (Lu Maohua), fault will gram (Tzi-cker Chiueh) and Lin Shibiao (Lin Shibiao) have delivered the paper of a piece " a kind of increment type file system consistency checking method by Study document system metadata write operation " by name, this article proposed a kind of by the Study document system metadata write operation and insert the method for consistent recovery point, this method is based on file system ext2 (the second extended file system of open form, second extended file system) realizes, versatility is not strong, depend on the metadata structure of file system by force, the new technology file system form as MS then is not suitable for.
Summary of the invention
For overcoming the deficiencies in the prior art, the object of the present invention is to provide a kind of consistent putting to insert and continuous data protection system of volume level and the method recovering to support.
For achieving the above object, the overall technological scheme that the present invention takes is inserted and the continuous data protection system of volume level that recovers to support for consistent putting, and comprises with lower module:
The application layer module is used for the data that are stored on the disk are carried out read-write operation;
Consistent putting inserted driver module, is used for that designated directory folder and file are carried out consistency operation and catches, and consistent dot information is packaged into common inputoutput data bag;
File system module is used to realize the conversion of file-level logical address to volume level logical address;
The mirror image driver module, the data that are used for writing protected disk send to the continuous data protection server;
The logical volume that local logical volume module, continuous data protection system can be this module provides the data of arbitrfary point to recover;
The continuous data protection system server; be used to distinguish common inputoutput data and the input and output that comprise consistent dot information; if the general data input and output then with data storage to the continuous data storage pool; if comprise the inputoutput data of consistent dot information, then extract consistent dot information and write in the consistent dot information file.
Described consistent dot information comprises at least to be caught, creates or modification information.
The described consistent form that is packaged into common inputoutput data bag of putting is an input and output request package form in Microsoft's file system.
Described mirror image driver module is equipped with network small computer system interface originating end module, is the module that is used for starting according to the Data Transport Protocol of client and server end the inputoutput data transmission.
Described continuous data protection system server is equipped with network small computer system interface receiving terminal module, is the module that is used for accepting according to the Data Transport Protocol of client and server end the inputoutput data of mirror image driver module network small computer system interface originating end module transmission.
Described mirror image driver module has a formation of writing the input and output request package, will write the input and output request package by asynchronous system and be sent to the continuous data protection system server.
The consistent continuous data guard method of volume level that inserts and recover to support of putting may further comprise the steps:
Step 1, consistent putting are caught driver module and are caught the consistent point operation information of file-level;
Step 2, consistent dot information is packaged at least one inputoutput data bag;
Step 3, will comprise the consistent inputoutput data bag of putting inputoutput data and be inserted in the normal data stream;
Step 4, continuous data protection server are distinguished common data input and output and the input and output that comprise consistent dot information; if the general data input and output then with data storage to the continuous data storage pool, then extract consistent dot information and write consistent dot information file if comprise the inputoutput data of consistent dot information.
Step 5, judge whether need to recover consistent point, recover then to search the consistent index file of put if desired, generate and tabulate release time, with the user expectation time point before data block merge and generate data image and send to client, otherwise go to step 1.
Described step 2 is consistent to be put and catches driver module and catch the consistent point operation information of file-level and specifically may further comprise the steps:
Step 1, wait upper strata write the input and output request package;
Step 2, analysis input and output request package action type, if the consistent input and output request package type of being concerned about of putting, then create the new input and output request package of writing, and write the input and output request package with what consistent dot information was packaged into new establishment, and will newly create comprise the writing in the normal input and output request package data stream that the input and output request package is inserted into the mirror image driver module of consistent dot information;
Step 3, go back to and continue to wait for new input and output request package.
Consistent dot information is packaged at least one inputoutput data bag in the described step 2 further may further comprise the steps:
Step 1, create at least one new type input and output request package of writing;
Step 2, be provided with that to write logical volume in the input and output request package be mirrored volume, be provided with to write the logical volume offset address;
Step 3, consistent dot information write the data field of the input and output request package of new establishment.
This has tangible advantage and good effect, specifically comprise: 1, the consistent aid decision-making method of putting of the present invention's proposition need not to define many system events, only just can insert consistent time point by the input and output (IO, input and output) of catching particular type.The present invention can catch the consistent point operation of file layers, and consistent dot information can be inserted in the normal data I stream.The present invention by on file system, insert consistent put to catch to drive catch file operational semantics information, then consistent dot information is packaged into special IRP and sends to the mirror image driving, that is in common IO flows, inserted the IO blocks of data of consistent dot information.2, backstage of the present invention storage system can be distinguished consistent IO of putting and common IO, and consistent dot information is write in the specific file.In the storage system of continuous data protection backstage, distinguish common data I and consistent dot information IO; if consistent dot information IO then writes consistent dot information independently in the file, this method provides convenient and effective means for the recovery of continuous data protection system.3, the present invention can recover to provide the aid decision making support for the continuous data protection consistance, inserts strategy by User Defined, and native system can weaken and be continuous snapshot guard method.4, satisfy the different grain size coherence request, can provide varigrained consistent putting to recover to support.The customizable different capturing events of the present invention can provide 3 kinds of means such as weak unanimity, strongly consistent and pressure be consistent, can satisfy the needs that monofile, catalogue and logical volume different grain size consistance are recovered respectively.5, highly versatile.This method for designing realizes on the file system upper strata; by catching the critical events such as closing, refresh of file, in normal IO stream, insert auxiliary consistent dot information, for the continuous data protection consistance recovers to provide supplementary means; irrelevant with upper layer application, have very strong versatility.
Description of drawings
Fig. 1 is the consistent continuous data protection system structural drawing of volume level that inserts and recover to support of putting of the present invention;
Fig. 2 puts the main flow chart of the continuous data guard method of volume level that inserts and recover to support for the present invention is consistent;
Fig. 3 puts for the present invention is consistent and catches synoptic diagram constantly;
Fig. 4 puts the encapsulation process flow diagram that inserts for the present invention is consistent;
Fig. 5 puts the Stored Procedure figure that inserts for the present invention is consistent;
Fig. 6 is the consistent input and output request package processing flow chart of putting of the present invention;
Fig. 7 recovers process flow diagram for the present invention rolls up a grade continuous data protection.
Embodiment
Below in conjunction with Figure of description the specific embodiment of the present invention is described.
With reference to figure 1, the consistent continuous data protection system of volume level that inserts and recover to support of putting of the present invention comprises with lower module:
Application layer module 1, this module are the program of actual motion on the operating system, and it can carry out read-write operation to the data that are stored on the disk.
Consistent putting inserted driver module 2, this module can be according to user's policy definition, the file type of designated directory folder and appointment is carried out consistency operation catch, and just consistent dot information is packaged into specific input and output request package (IRP, IO Request Package).
File system module 3 is the file system module on the operating system.Adopt the New Technology File System (NTFS, New Technology FileSystem) in form (Windows) operating system among the present invention.File system module has mainly been realized the conversion of file-level logical address to volume level logical address.The 8K byte content need be write as going into file A from the file skew as 1 write request in the file system.If the file system block size is the 4K byte, after then handling, can produce 2 and write IRP and send to mirror image and drive through file system, specified the 4K byte to write the offset address of logical volume among each IRP.
Mirror image driver module 4, volume level data protection have continuously added the mirror image driving in order to catch all read-write operations to the protection disk, and it mainly acts on is that the data that will write protected disk send to the continuous data protection server.A network small computer system interface originating end module (iSCSI initiator is arranged on mirror image driver module 4, internet Small Computer SystemInterface initiator) 7, the present invention adopts the Data Transport Protocol of iSCSI agreement as the client and server end, and client has been installed iSCSI originating end module 7 iSCSI initiator.All data that write protected disk are sent to long-range continuous data protection server by the iSCSI agreement.Mirror image driver module 5 is safeguarded a formation of writing IRP, will write on the continuous data protection server that IRP is sent to the continuous data protection background module by asynchronous system.The consistent IRP that puts only inserts the IRP formation of writing toward (please provide Chinese implication) continuous data protection server, to the IRP that writes local logical volume without any influence.
Local logical volume module 5, this module is protected logical volume module, continuous data protection system can recover for this volume provides the data of arbitrfary point.
Continuous data protection system server 6, iSCSI target module on this server.This module is distinguished common IO data and the IO that comprises consistent dot information.If general data IO, then with data storage to the continuous data storage pool, if comprise the IO data of consistent dot information, then extract consistent dot information and write in the consistent dot information file.Network small computer system interface receiving terminal module (iSCSI target has been installed on the continuous data protection system server 6; internet SmallComputer System Interface target) 8, be used to accept the write IRP of the iSCSIinitiator module 7 of mirror image driver module 5 by the asynchronous system transmission.
See also Fig. 2, the present invention is implemented in the consistent dot information of insertion in the normal piece level IO stream by insert " consistent putting inserted driving " respectively in the file system the upper and lower with " mirror image driving ".Of the present inventionly may further comprise the steps: at first based on the consistent continuous data guard method of volume level that inserts and recover of putting; start application layer; application layer module 1 begins the data that are stored on the file system module 3 on the disk are carried out read-write operation; consistent putting caught driver module 2 and according to instruction the file type of designated directory folder and appointment carried out consistency operation and catch; then, consistent dot information is packaged into specific input and output request IO packet.The IO packet that mirror image driver module 4 will comprise the consistent IO of putting data is inserted in the normal data stream, and is sent to continuous data protection server 6.Continuous data protection server 6 is distinguished common data I and the IO that comprises consistent dot information; if general data IO then with data storage to the continuous data storage pool, then extract consistent dot information and write consistent dot information file if comprise the IO data of consistent dot information.Afterwards, judge whether to need to recover consistent point, then recover to select to search the consistent index file of putting if desired according to recovery of user's strongly consistent point or weak consistent putting, generate tabulation release time, return to the user and wait for that the user selects the time point of its expectation, the time point of selecting according to the user then, data block before this time point is merged, generate the data image of this time point, and data are sent to client carry out data and recover, otherwise application layer module 1 continues the data that are stored on the file system module 3 on the disk are carried out read-write operation.
Consistent putting of the present invention can define according to subscriber policy the opportunity of catching.Consistent put to catch to drive catch the IRP of All Files layer, decide consistent insertion opportunity and the consistent content of putting put by action type among the IRP.IRP is a kind of very important data structure in the Windows kernel.When upper level applications was communicated by letter with bsp driver, application program can be sent the I/O request, and operating system converts corresponding I/O request to corresponding IRP, and different IRP can be dispatched to different sending in the routine according to type and handle.The related function of file I/O, for example create file (CreateFile), read file (ReadFile), written document (WriteFile), (close file handle CloseHandle) etc. can cause operating system respectively and produce and create file type IRP (IRP_MJ_CREATE), read file type IRP (IRP_MJ_READ), written document type IRP (IRP_MJ_WRITE), close file type IRP dissimilar IRP such as (IRP_MJ_CLOSE), these IRP can be sent to corresponding the sending in the routine of driver and handle.
See also Fig. 3, the consistent IRP that puts that the present invention handles is divided into 3 classes, catches explanation signal on opportunity and sees Fig. 3:
1, weak consistent checkpoint IRP (Checkpoint IRP).Write when operations such as single closing of a file, deletion and rename, conforming support granularity is a file.Weak consistent Checkpoint IRP produces with the IRP that is made as two types of fileinfo type IRP (IRP_SET_INFORMATION) by analyzing IRP_MJ_CLOSE.Upper layer application is invoke system call CloseHandle method when the close file handle, and CloseHandle sends the IRP of IRP_MJ_CLOSE type to file system.Consistent putting insert to drive by analyzing fileinfo related among the IRP_MJ_CLOSE type i RP to judge it is user's interest file type or file protection catalogue, and the IRP with upper layer IRP_MJ_CLOSE type mails to file system behind the weak consistent IRP of an encapsulation if then insert.IRP to the IRP_SET_INFORMATION class, consistent putting catches that only to drive the subfunction type be that the IRP of deletion, rename and modification attribute produces weak consistent Checkpoint IRP, and to the subfunction type for shear and the operation of revised file pointer do not produce a little less than consistent Checkpoint IRP.The IRP type that the present invention handles sees Table 1.
The consistent Checkpoint IRP type of putting of table 1
Operation |
The source |
Call on the upper strata |
Catch |
IRP_MJ_CREATE |
Create, open file |
CreateFile |
Consistent putting caught the driving processing |
IRP_MJ_CLEANU P |
The IRP that cancellation is hung when closing handle |
CloseHandle |
Do not handle |
IRP_MJ_CLOSE |
The close file handle |
CloseHandle |
Consistent putting caught the driving processing |
IRP_SET_INFOR MATION |
Deletion, rename, modification attribute, shearing, revised file pointer. |
SetFileLength |
Consistent putting caught the drive part processing |
IRP_MJ_WRITE |
Write data |
WriteFile |
Mirror image drives to be handled |
IRP_MJ_FLUSH_ BUFFERS |
Write output buffer or abandon the input block |
FlushFileBuff ers |
Do not handle |
2, strongly consistent Checkpoint IRP.Write when all files of opening of protected catalogue are all closed, consistance supports that granularity is a catalogue.Weak consistent Checkpoint IRP has indicated the consistent point of file, and strongly consistent Checkpoint IRP has then indicated the consistent point of catalogue.Strongly consistent CheckpointIRP realizes by counter.To the protected catalogue of user's appointment, when opening file under the catalogue, upper layer application can send the IRP of IRP_MJ_CREATE type toward file system.The counter related with this catalogue adds 1, subtracts 1 closing certain file hour counter.If counter is zero, then show all file Close Alls of opening under this catalogue.At this moment, consistent putting caught the Checkpoint IRP that drives an encapsulation of insertion strongly consistent information
3, force consistent Checkpoint IRP.Write by user-defined strategy is mandatory, consistance supports that granularity is a logical volume.Force consistent Checkpoint IRP then to generate according to the predefined strategy of user.For example, customer requirements generated one in 1 hour and forces uniform view, and then system inserted one in per 1 hour and forces consistent Checkpoint IRP.The Flush command forces of call operation system was with data cached the flushing on the disk in the file system when system insert to force consistent Checkpoint IRP.Force consistent Checkpoint IRP by inserting, system can only provide other data consistency of collapse consistance (Crash-consistency) level to guarantee, and the snap view that is similar to the application layer consistance (Application-level consistency) that Windows volume shadow copy services provides can not be provided.
Consistent putting caught driving and caught consistent dot information according to the IRP type and specifically may further comprise the steps.Wait for that at first the upper strata writes IRP, analyze the IRP action type then, if the consistent IRP type of being concerned about of putting, then create the new IRP that writes, and consistent dot information is packaged into the IRP that writes of new establishment, and in the normal IRP data stream of writing the driving of IRP insertion mirror image that comprises consistent dot information that will newly create, otherwise original I RP is sent to file system, go back at last and continue to wait for new IRP.
The present invention design consistent put the IRP that IRP is designed to write type, and write address is the 0x1CE place of logical volume, promptly writes the 2nd disk partition table place of Master boot sector (MBR).A MBR record comprises 4 disk partition information.Because the continuous data protection system server carries out mirror image by a logical unit number (LUN, Logical Unit Number) and local protected logical volume.In actual use, a LUN of continuous data protection system has only used first subregion, so the information of 2-4 the subregion of the MBR of continuous data protection system server mirroring LUN does not have practical significance.1 partitioned record comprises 16 bytes, so 3 Free Partition information of available MBR in the native system, amounts to 48 bytes and transmits and express the consistent information of putting
See also Fig. 4.Consistent putting caught driver module 2 the unanimity point is packaged among the IRP that comprises consistent dot information, specifically may further comprise the steps: at first, create a new type i RP that writes, then, being provided with and writing logical volume among the IRP is mirrored volume, and next, being provided with and writing the logical volume offset address is 0x1CE, and consistent dot information write the data field of the IRP of new establishment, finish at last.
See also Fig. 5 and Fig. 6, continuous data protection server module 6 need be distinguished normal data I and consistent point data IO when receiving the IRP of front end 7 transmissions, specifically may further comprise the steps.At first, analysis is the common IRP of writing, and still consistently puts IRP, if ordinary write operation IRP then extracts write address and is offset and writes content, is packaged into minimum log unit form in conjunction with the server current time, and is transferred to buffer memory and waits processing; If the consistent IRP that puts, then earlier remove and repeat to write processing waiting the minimum log unit that writes in the buffer memory, promptly only keep last portion two consistent data that repeat in the time interval to write of putting toward same address, start afterwards and write daily record, consistent dot information is write the specific consistent index file of putting after writing successfully data cached.
Consistent to put that IRP arrives be the data cached trigger condition that writes daily record, but be not adequate condition.The present invention has introduced 3 kinds and has write trigger condition: time-based triggering, trigger with the consistent IRP of putting based on the triggering of spatial cache.For safety, when data cached in internal memory the hold-up time, or the buffer memory utilization rate arrives also to trigger after the preset threshold value and writes log event.
See also Fig. 7, when the user recovered, the present invention recovered to support for the user provides consistent point, specifically may further comprise the steps.At first, the user selects to recover kind, is the recovery or the weak consistent recovery of putting of strongly consistent point.Then,, search the consistent index file of putting, generate the time list that recovers and return to the user, wait for that the user selects the time point of its expectation according to user's selection.Afterwards, the time point according to the user selects merges the data block before this time point, generates the data image of this time point.At last, data are sent to client and carry out the data recovery.
The above is the preferred embodiments of the present invention only, is not limited to the present invention, and for a person skilled in the art, the present invention can have various changes and variation.Within the spirit and principles in the present invention all, any modification of being done, be equal to replacement, improvement etc., all should be included within the claim scope of the present invention.