CN102929998A - File increment intercepting method and device - Google Patents
File increment intercepting method and device Download PDFInfo
- Publication number
- CN102929998A CN102929998A CN2012104129845A CN201210412984A CN102929998A CN 102929998 A CN102929998 A CN 102929998A CN 2012104129845 A CN2012104129845 A CN 2012104129845A CN 201210412984 A CN201210412984 A CN 201210412984A CN 102929998 A CN102929998 A CN 102929998A
- Authority
- CN
- China
- Prior art keywords
- file
- increment
- current generation
- writing events
- intercepting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a file increment intercepting method and a device. The method comprises the following steps of: configuring a file directory required to be intercepted; obtaining a filename of a file in which a write-in event occurs according to a file structure body once the write-in event occurs; comparing the filename of the file in which the write-in event occurs with the file directory required to be intercepted, determining whether the current file in which the write-in event occurs is required to be subjected to increment intercepting, if so, intercepting the current file in which the write-in event occurs, copying the intercepted file for caching, performing increment acquisition on the cached file, judging whether the increment is required to be intercepted once the write-in event occurs, and intercepting the increment in real time. Compared with the mode of updating the file, blocking the updated file and performing HASH comparison on the version before update so as to position a changed block in the prior art, the method has the advantages of timely extracting the incremental data, along with high accuracy and rapidness.
Description
Technical field
The present invention relates to increment and intercept and capture technical field, relate in particular to a kind of based on file increment method for interception and device.
Background technology
Correlation technique is abstracted into computational resource with computer system usually, these three kinds of resources of storage resources and Internet resources, and the essential target of computer system combines this three large resource exactly, for the user provides various services.If further will carry out for the various service behaviors that the user provides abstractly, can be summarized as the storage data, deal with data and present the behavior of data.For the processing of data, because the service that data processing software self provides is different, the mode that data are processed is also varied, and wherein a part is the extraction that concentrates on incremental data.Orient incremental data in data centralization, and then only need to process incremental data, again result and result are before merged, finally obtain the result of current data state.Just being based on this principle such as the remote copy technology of using in database replication realizes: source database is divided into data and daily record two parts, wherein log recording a bit begins change to database from certain, when data-base remote copies, only need basic data is copied in the object library, subsequently the changing unit of daily record sent the remote copy that gets final product fulfillment database.
For this structural data storing software of database, to support mainly with the mode of Incremental Log greatly, this a part of technology has developed very ripely.But for ordinary file, i.e. the incremental data of unstructured data mostly adopts at present and carries out the mode that HASH compares with history file and obtain.The mode of the at present data of this part extraction mainly is, file is carried out piecemeal, and then carry out HASH with version before and compare to locate the variation piece, this mode can satisfy the requirement of extracting incremental data, but calculate and compare owing to need to carry out HASH, whole leaching process is too slow, and also because it is the piecemeal location, often workload was larger when increment extracted simultaneously.
Summary of the invention
Await this, the technical problem to be solved in the present invention provides a kind of file increment method for interception and device, this document increment method for interception and device can Dynamic Interception file increments, compare with present increment extracting method that to extract incremental data more timely, accurately and fast.
In order to address the above problem, the embodiment of the invention provides a kind of file increment acquisition mechanism, it is characterized in that, this device comprises configuration module, monitoring module, determination module, intercepting and capturing module and increment acquisition module, wherein:
Described configuration module is arranged to: configuration needs the file directory of intercepting and capturing;
Described monitoring module is arranged to: monitor writing events, obtain the filename of the file of current generation writing events according to the file structure body;
Described determination module is arranged to: filename and the file directory that the needs that configure are intercepted and captured of the file of current generation writing events are compared, determine whether to carry out the file of current generation writing events increment and intercept and capture;
Described intercepting and capturing module is arranged to: when described determination module judges that need to carry out to the file of current generation writing events increment intercepts and captures, intercept and capture the file of current generation writing events, and the file copy portion of intercepting and capturing is carried out buffer memory;
Described increment acquisition module is arranged to: the file of buffer memory is carried out increment obtain.
Alternatively, described determination module is arranged in such a way the filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, and determines whether to carry out the file of current generation writing events increment and intercepts and captures:
If the filename of the file of current generation writing events in the file directory that needs are intercepted and captured or son file or the sub-folder of the file or folder in the file of the file of the current generation writing events file directory that needs to intercept and capture by name, determines that then need to carry out increment intercepts and captures; Otherwise, do not need to carry out increment and intercept and capture.
Alternatively, described intercepting and capturing module also is arranged to:
Intercept and capture the file of current generation writing events, and when the file copy portion of intercepting and capturing carried out buffer memory, clearance this document;
Described determination module judges that the file that does not need current generation writing events carries out increment when intercepting and capturing, this document of directly letting pass.
Alternatively, this device also comprises the incremental processing module, wherein:
Described incremental processing module is arranged to: the file increment that obtains is processed;
Described processing comprises carries out write operation to described file increment, writes corresponding file, also comprises this document increment is analyzed.
Alternatively, the file or folder in the described file directory that needs to intercept and capture is the unstructured data file or folder.
The embodiment of the invention also provides a kind of file increment method for interception, it is characterized in that, the method comprises:
Configuration needs the file directory of intercepting and capturing;
In case the generation writing events then obtains the filename of the file of current generation writing events according to the file structure body;
The filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, determining whether to carry out the file of current generation writing events increment intercepts and captures, if need, then intercept and capture the file of current generation writing events, and the file copy portion of intercepting and capturing carried out buffer memory, afterwards the file of buffer memory is carried out increment and obtain.
Alternatively, filename and the file directory that the needs that configure are intercepted and captured of the file of current generation writing events are compared, determine whether to carry out the file of current generation writing events the step that increment intercepts and captures and comprise:
The filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, if the filename of the file of current generation writing events in the file directory that needs are intercepted and captured or son file or the sub-folder of the file or folder in the file of the file of the current generation writing events file directory that needs to intercept and capture by name, determines that then need to carry out increment intercepts and captures; Otherwise, do not need to carry out increment and intercept and capture.
Alternatively, the method also comprises:
The file increment that obtains is processed;
Described processing comprises carries out write operation to described file increment, writes corresponding file, also comprises this document increment is analyzed.
Alternatively, the method also comprises:
Intercept and capture the file of current generation writing events, and when the file copy portion of intercepting and capturing carried out buffer memory, clearance this document;
Do not need file to current generation writing events to carry out increment when intercepting and capturing, this document of directly letting pass.
Alternatively, the file or folder in the described file directory that needs to intercept and capture is the unstructured data file or folder.
File increment method for interception and the device of the embodiment of the invention, in case writing events occurs, then judging whether needs to intercept and capture and carries out corresponding increment and intercept and capture, and has realized carrying out in real time increment and has intercepted and captured.After finishing renewal with the prior art File, the file after upgrading is carried out piecemeal, and then carry out HASH and compare to locate the mode that changes piece and compare with version before upgrading, extract incremental data more timely, accurately and fast.
Description of drawings
Fig. 1 is the increment method for interception of the embodiment of the invention;
Fig. 2 is that the WRITE method of writing of the embodiment of the invention is replaced schematic diagram;
Fig. 3 is the file increment acquisition mechanism structural drawing of the embodiment of the invention.
Embodiment
Below in conjunction with drawings and Examples technical scheme of the present invention is described in detail.
Need to prove, if do not conflict, each feature among the embodiment of the invention and the embodiment can mutually combine, all within protection scope of the present invention.In addition, although there is shown logical order in flow process, in some cases, can carry out step shown or that describe with the order that is different from herein.
With reference to the accompanying drawings, describe the operational process of content of the present invention in detail.
The embodiment of the invention provides a kind of increment method for interception, and the method is based on Virtual File System (VFS, Virtual File System), and as shown in Figure 1, the method comprises that mainly step 101 is to step 105:
In the embodiments of the present invention, in case writing events occurs, then judging whether needs to intercept and capture and carries out corresponding increment and intercept and capture, and has realized carrying out in real time increment and has intercepted and captured.After finishing renewal with the prior art File, the file after upgrading is carried out piecemeal, and then carry out HASH and compare to locate the mode that changes piece and compare with version before upgrading, extract incremental data more timely, accurately and fast.
Alternatively, described step 103 comprises: filename and the file directory that the needs that configure are intercepted and captured of the file of current generation writing events are compared, if the filename of the file of current generation writing events in the file directory that needs are intercepted and captured or son file or the sub-folder of the file or folder in the file of the file of the current generation writing events file directory that needs to intercept and capture by name, determines that then need to carry out increment intercepts and captures; Otherwise, do not need to carry out increment and intercept and capture.
Alternatively, the method also comprises: the file increment that obtains is processed.
Processing herein comprises carries out write operation to described file increment, writes corresponding file, also comprises this document increment such as is analyzed at the operation.
Alternatively, the file or folder that the needs in the embodiment of the invention are intercepted and captured increment refers to ordinary file or file, i.e. the unstructured data file or folder.
Said method realizes by putting inner function (hook_write), when specific implementation, needs intercepted and captured (write) method of writing of file operation (file_operations) of the file or folder of increment and replaced; When replacing former write method old_write, hook_write will keep the function pointer of old_write, in order to call old_write after hook_write finishes dealing with the file of letting pass is carried out normal write operation, and the method for replacement as shown in Figure 2.Through the file of replacing in a single day write operation occuring, then carries out described step 102-106 by described hook_write.
The embodiment of the invention also provides a kind of file increment acquisition mechanism, this device is based on Virtual File System (VFS, Virtual File System), as described in Figure 3, this device comprises configuration module 301, monitoring module 302, determination module 303, intercepts and captures module 304 and increment acquisition module 305, wherein:
Described configuration module 301 is arranged to: configuration needs the file directory of intercepting and capturing.
Described monitoring module 302 is arranged to: monitor writing events, obtain the filename of the file of current generation writing events according to the file structure body;
Described determination module 303 is arranged to: filename and the file directory that the needs that configure are intercepted and captured of the file of current generation writing events are compared, determine whether to carry out the file of current generation writing events increment and intercept and capture.
Described determination module 303 is arranged in such a way the filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, determining whether to carry out the file of current generation writing events increment intercepts and captures: if the filename of the file of current generation writing events in the file directory that needs are intercepted and captured or son file or the sub-folder of the file or folder in the file of the file of the current generation writing events file directory that needs to intercept and capture by name, determines that then need to carry out increment intercepts and captures; Otherwise, do not need to carry out increment and intercept and capture.
Described intercepting and capturing module 304 is arranged to: when described determination module 303 judges that need to carry out to the file of current generation writing events increment intercepts and captures, intercept and capture the file of current generation writing events, and the file copy portion of intercepting and capturing is carried out buffer memory;
Described increment acquisition module 305 is arranged to: the file of buffer memory is carried out increment obtain.
Alternatively, described intercepting and capturing module 304 also is arranged to: intercept and capture the file of current generation writing events, and when the file copy portion of intercepting and capturing carried out buffer memory, clearance this document; Described determination module 303 judges that the file that does not need current generation writing events carries out increment when intercepting and capturing, this document of directly letting pass.
Alternatively, this device also comprises incremental processing module 306, and it is arranged to: the file increment that obtains is processed.
Alternatively, processing herein comprises carries out write operation to described file increment, writes corresponding file, also comprises this document increment such as is analyzed at the operation.
Alternatively, the file or folder that the needs in the embodiment of the invention are intercepted and captured increment refers to ordinary file or file, i.e. the unstructured data file or folder.
One of ordinary skill in the art will appreciate that all or part of step in the said method can come the instruction related hardware to finish by program, described program can be stored in the computer-readable recording medium, such as ROM (read-only memory), disk or CD etc.Alternatively, all or part of step of above-described embodiment also can realize with one or more integrated circuit.Correspondingly, each the module/unit in above-described embodiment can adopt the form of hardware to realize, also can adopt the form of software function module to realize.The present invention is not restricted to the combination of the hardware and software of any particular form.
Certainly; the present invention also can have other various embodiments; in the situation that do not deviate from spirit of the present invention and essence thereof; those of ordinary skill in the art work as can make according to the present invention various corresponding changes and distortion, but these corresponding changes and distortion all should belong to the protection domain of claim of the present invention.
Claims (10)
1. a file increment acquisition mechanism is characterized in that, this device comprises configuration module, monitoring module, determination module, intercepting and capturing module and increment acquisition module, wherein:
Described configuration module is arranged to: configuration needs the file directory of intercepting and capturing;
Described monitoring module is arranged to: monitor writing events, obtain the filename of the file of current generation writing events according to the file structure body;
Described determination module is arranged to: filename and the file directory that the needs that configure are intercepted and captured of the file of current generation writing events are compared, determine whether to carry out the file of current generation writing events increment and intercept and capture;
Described intercepting and capturing module is arranged to: when described determination module judges that need to carry out to the file of current generation writing events increment intercepts and captures, intercept and capture the file of current generation writing events, and the file copy portion of intercepting and capturing is carried out buffer memory;
Described increment acquisition module is arranged to: the file of buffer memory is carried out increment obtain.
2. file increment acquisition mechanism as claimed in claim 1, it is characterized in that, described determination module is arranged in such a way the filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, and determines whether to carry out the file of current generation writing events increment and intercepts and captures:
If the filename of the file of current generation writing events in the file directory that needs are intercepted and captured or son file or the sub-folder of the file or folder in the file of the file of the current generation writing events file directory that needs to intercept and capture by name, determines that then need to carry out increment intercepts and captures; Otherwise, do not need to carry out increment and intercept and capture.
3. file increment acquisition mechanism as claimed in claim 1 or 2 is characterized in that, described intercepting and capturing module also is arranged to:
Intercept and capture the file of current generation writing events, and when the file copy portion of intercepting and capturing carried out buffer memory, clearance this document;
Described determination module judges that the file that does not need current generation writing events carries out increment when intercepting and capturing, this document of directly letting pass.
4. file increment acquisition mechanism as claimed in claim 3 is characterized in that, this device also comprises the incremental processing module, wherein:
Described incremental processing module is arranged to: the file increment that obtains is processed;
Described processing comprises carries out write operation to described file increment, writes corresponding file, also comprises this document increment is analyzed.
5. file increment acquisition mechanism as claimed in claim 1 is characterized in that, the file or folder in the described file directory that needs to intercept and capture is the unstructured data file or folder.
6. a file increment method for interception is characterized in that, the method comprises:
Configuration needs the file directory of intercepting and capturing;
In case the generation writing events then obtains the filename of the file of current generation writing events according to the file structure body;
The filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, determining whether to carry out the file of current generation writing events increment intercepts and captures, if need, then intercept and capture the file of current generation writing events, and the file copy portion of intercepting and capturing carried out buffer memory, afterwards the file of buffer memory is carried out increment and obtain.
7. file increment method for interception as claimed in claim 6, it is characterized in that, the filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, determines whether to carry out the file of current generation writing events the step that increment intercepts and captures and comprise:
The filename of the file of current generation writing events is compared with the file directory that the needs that configure are intercepted and captured, if the filename of the file of current generation writing events in the file directory that needs are intercepted and captured or son file or the sub-folder of the file or folder in the file of the file of the current generation writing events file directory that needs to intercept and capture by name, determines that then need to carry out increment intercepts and captures; Otherwise, do not need to carry out increment and intercept and capture.
8. such as claim 6 or 7 described file increment method for interception, it is characterized in that, the method also comprises:
The file increment that obtains is processed;
Described processing comprises carries out write operation to described file increment, writes corresponding file, also comprises this document increment is analyzed.
9. file increment method for interception as claimed in claim 8 is characterized in that, the method also comprises:
Intercept and capture the file of current generation writing events, and when the file copy portion of intercepting and capturing carried out buffer memory, clearance this document;
Do not need file to current generation writing events to carry out increment when intercepting and capturing, this document of directly letting pass.
10. file increment method for interception as claimed in claim 6 is characterized in that, the file or folder in the described file directory that needs to intercept and capture is the unstructured data file or folder.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012104129845A CN102929998A (en) | 2012-10-25 | 2012-10-25 | File increment intercepting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012104129845A CN102929998A (en) | 2012-10-25 | 2012-10-25 | File increment intercepting method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102929998A true CN102929998A (en) | 2013-02-13 |
Family
ID=47644796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012104129845A Pending CN102929998A (en) | 2012-10-25 | 2012-10-25 | File increment intercepting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102929998A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104714878A (en) * | 2013-12-11 | 2015-06-17 | 阿里巴巴集团控股有限公司 | Method and device for collecting log data |
| CN106155750A (en) * | 2016-07-29 | 2016-11-23 | 腾讯科技（深圳）有限公司 | The loading method of a kind of resource file and device |
| CN108509172A (en) * | 2018-04-13 | 2018-09-07 | 平安普惠企业管理有限公司 | Data transmission method, device, system and the computer storage media of simultaneous display |
| CN110099117A (en) * | 2019-05-05 | 2019-08-06 | 中国互联网络信息中心 | A kind of method and apparatus that multi version dns zone file full dose issues |
| CN114356245A (en) * | 2022-01-12 | 2022-04-15 | 济南点量软件有限公司 | Method and system for rapidly comparing and updating mass files |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101604268A (en) * | 2009-07-13 | 2009-12-16 | 浪潮电子信息产业股份有限公司 | A kind of method for filtering monitored directory change events |
| CN102236589A (en) * | 2010-05-04 | 2011-11-09 | 南京壹进制信息技术有限公司 | Data protection method for managing increment file based on digital identifiers |
| CN102479118A (en) * | 2010-11-30 | 2012-05-30 | 英业达集团(天津)电子技术有限公司 | Incremental backup method for data blocks |
-
2012
- 2012-10-25 CN CN2012104129845A patent/CN102929998A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101604268A (en) * | 2009-07-13 | 2009-12-16 | 浪潮电子信息产业股份有限公司 | A kind of method for filtering monitored directory change events |
| CN102236589A (en) * | 2010-05-04 | 2011-11-09 | 南京壹进制信息技术有限公司 | Data protection method for managing increment file based on digital identifiers |
| CN102479118A (en) * | 2010-11-30 | 2012-05-30 | 英业达集团(天津)电子技术有限公司 | Incremental backup method for data blocks |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104714878A (en) * | 2013-12-11 | 2015-06-17 | 阿里巴巴集团控股有限公司 | Method and device for collecting log data |
| CN106155750A (en) * | 2016-07-29 | 2016-11-23 | 腾讯科技（深圳）有限公司 | The loading method of a kind of resource file and device |
| CN106155750B (en) * | 2016-07-29 | 2018-09-07 | 腾讯科技（深圳）有限公司 | A kind of loading method and device of resource file |
| CN108509172A (en) * | 2018-04-13 | 2018-09-07 | 平安普惠企业管理有限公司 | Data transmission method, device, system and the computer storage media of simultaneous display |
| CN108509172B (en) * | 2018-04-13 | 2024-05-24 | 北京神州数码云科信息技术有限公司 | Synchronous display data transmission method, device, system and computer storage medium |
| CN110099117A (en) * | 2019-05-05 | 2019-08-06 | 中国互联网络信息中心 | A kind of method and apparatus that multi version dns zone file full dose issues |
| CN110099117B (en) * | 2019-05-05 | 2021-10-22 | 中国互联网络信息中心 | Method and device for issuing full amount of multi-version DNS zone files |
| CN114356245A (en) * | 2022-01-12 | 2022-04-15 | 济南点量软件有限公司 | Method and system for rapidly comparing and updating mass files |
| CN114356245B (en) * | 2022-01-12 | 2023-09-22 | 济南点量软件有限公司 | Method and system for fast comparing and updating mass files |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9251011B2 (en) | Backup of in-memory databases | |
| KR102311032B1 (en) | Database Synchronization | |
| US11347681B2 (en) | Enhanced reading or recalling of archived files | |
| US20170147469A1 (en) | Correlation of source code with system dump information | |
| US20210182160A1 (en) | System and method for generating file system and block-based incremental backups using enhanced dependencies and file system information of data blocks | |
| US9223679B1 (en) | Lightweight, non-intrusive, and flexible apparatus to obtain system and process state | |
| US20170344433A1 (en) | Apparatus and method for data migration | |
| CN102929998A (en) | File increment intercepting method and device | |
| US20220318095A1 (en) | Using a storage log to generate an incremental backup | |
| US11307922B2 (en) | Application crash analysis techniques when memory dump and debug symbols are not co-located | |
| WO2009123342A1 (en) | Database system, database update method, database, and database update program | |
| US9037539B2 (en) | Data synchronization | |
| JP2020530155A (en) | Systems and methods for redundant array data alignment | |
| US10915403B2 (en) | Versioned records management using restart era | |
| US20220236999A1 (en) | Unified way to track user configuration on a live system | |
| WO2015015502A1 (en) | Writing to files and file meta-data | |
| US11630742B2 (en) | System and method of performing recovery using a backup image | |
| CN113849119A (en) | Storage method, storage device, and computer-readable storage medium | |
| US8819657B1 (en) | Method and apparatus for maintaining data consistency in a virtualized application during software update installation | |
| CN114978963B (en) | Network system monitoring analysis method and device, electronic equipment and storage medium | |
| CN116360931A (en) | Link tracking method, device, system and storage medium | |
| CN115113989A (en) | Transaction execution method and device, computing equipment and storage medium | |
| CN108959604B (en) | Method, apparatus and computer readable storage medium for maintaining database cluster | |
| KR20150087990A (en) | System and Method for Caching Disk Image File of Full-Cloned Virtual Machine | |
| US11755538B2 (en) | Distributed management of file modification-time field |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130213 |