CN102467569A - Method for supporting concurrent read-write of file in data de-duplication - Google Patents
Method for supporting concurrent read-write of file in data de-duplication Download PDFInfo
- Publication number
- CN102467569A CN102467569A CN2010105558247A CN201010555824A CN102467569A CN 102467569 A CN102467569 A CN 102467569A CN 2010105558247 A CN2010105558247 A CN 2010105558247A CN 201010555824 A CN201010555824 A CN 201010555824A CN 102467569 A CN102467569 A CN 102467569A
- Authority
- CN
- China
- Prior art keywords
- file
- program
- write
- data
- duplication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention relates to a method for supporting concurrent read-write of a file in data de-duplication, which is used for solving the delay errors during the file access of file backup program and realizing the concurrent read-write operation of the file when a data de-duplication program is carried out by utilizing a file backup program. The method comprises the steps of: running an interrupt driving program and the file backup program in a client side; sending an operation request to the file backup program through the client side so as to carry out corresponding file operations on an object file; and intercepting the operation request and ordering a service program to carry out the operation of the object file according to the operation request by utilizing the interrupt driving program.
Description
Technical field
The present invention relates to a kind of file backup disposal route, particularly a kind of method of in data de-duplication, supporting the file cocurrent read-write.
Background technology
Data de-duplication is a kind of data reduction technology, is generally used for the standby system based on disk, and fundamental purpose is to reduce the memory capacity of using in the storage system.Its working method is the repetition variable-size data block of in certain time cycle, searching diverse location in the different files.The data block that repeats replaces with designator.Owing to always be flooded with a large amount of redundant datas in the storage system.In order to address this problem, save more spaces, the focus that " repeating deletion " technology has become people to pay close attention to just naturally.Adopting " repeating deletion " technology can be original 1/20 with the data reduction of storage; Thereby abdicate more backup space; Not only can make the Backup Data on the storage system preserve the longer time, but also required a large amount of bandwidth can practice thrift offline storage the time.
After the drives interrupts program was taken data and passed to handling procedure, the drives interrupts program was just thought and is accomplished the action that writes, and returns to the upper strata and writes information of successful (WRITE_OK).The data that drives interrupts program copy is desired to write are used the confession client process to shared drive.The drives interrupts program just thinks that the processing of this operation requests just finishes.But because the existence of processing delay, maybe be when service end also not to be handled this segment data fully, archival backup sends new this partial data (IRP_READ) order of reading immediately again.So, new data do not write as yet, so service end can only be sent to client with old data, certainly will cause client to receive wrong data.
Summary of the invention
In view of above problem; The object of the present invention is to provide a kind of method of in data de-duplication, supporting the file cocurrent read-write; When the solution archival backup carries out the data de-duplication program, the delay mistake of archival backup when accessing file, and support the file cocurrent read-write.
The method of in data de-duplication, supporting the file cocurrent read-write that the present invention disclosed may further comprise the steps: outage driver and archival backup in client; The timing of executable operations time is in order to confirm among the running time, whether having operation requests to send; When client during the running time to archival backup transmit operation request, in order to file destination is carried out corresponding file operation; By drives interrupts program interception operation requests, the drives interrupts program is carried out the operation of file destination according to operation requests command service program; The drives interrupts program is sent to client with object information.
The present invention provides a kind of kernel that does not need the revised file storage system, also can be so that document storage system provides the function of data de-duplication.Thus, except reducing the file storing space, also can reduce software developer's cost of development.
Describe the present invention below in conjunction with accompanying drawing and specific embodiment, but not as to qualification of the present invention.
Description of drawings
Fig. 1 is a configuration diagram of the present invention;
Fig. 2 is an operation workflow synoptic diagram of the present invention;
Fig. 3 A is the schematic flow sheet of operations request of the present invention;
Fig. 3 B is the schematic flow sheet of operations request of the present invention;
Fig. 4 is the schematic flow sheet of the read-write of file destination of the present invention.
Wherein, Reference numeral
Client 100
Drives interrupts program 110
Archival backup 120
Data de-duplication program 130
Embodiment
Below in conjunction with accompanying drawing structural principle of the present invention and principle of work are done concrete description:
The present invention is applied to have the computing machine of handling the data de-duplication program, for example: PC, notebook computer or server.Be convenient explanation, thus be with a client as explanation, but its quantity is not limited thereto.Please refer to shown in Figure 1ly, it is a configuration diagram of the present invention.
Step S210: outage driver and archival backup in client;
Step S220: the timing of executable operations time, in order to confirm among the running time, whether having operation requests to send;
Step S230: when user side service during the running time to archival backup transmit operation request, in order to file destination is carried out corresponding file operation;
Step S240: by drives interrupts program interception operation requests, the drives interrupts program is carried out the operation of file destination according to operation requests command service program; And
Step S250: the drives interrupts program is sent to client with object information.
At first, drives interrupts program 110 is responsible for monitoring in the client 100 whether send operation requests.Client 100 operate during, if client 100 to archival backup 120 transmit operation requests, then operation requests will be directed to the user side service by drives interrupts program 110 and carry out corresponding file operation.The kind of operation requests in the present invention is for file reads processing (File Read), file writes processing (File Write) or residual capacity query processing (Query).And file writes processing and also comprises: begin to write processing (Start Write) and stop to write processing (Stop Write).
Read when handling if operation requests is a file, then file destination 140 is write in the shared drive, read file destination 140 to archival backup 120 in order to offer drives interrupts program 110 by service routine.If when service routine was the capacity query processing, then drives interrupts program 110 was returned the capacity of shared drive current residual.
Write when handling if operation requests is a file, then file destination 140 is write in the ring-type buffer memory, carry out data de-duplication program 130 in order to offer service routine by drives interrupts program 110.And when beginning to write processing, can set and write flag, use making other client 100 these files of forbidding.In like manner, stopping to write when handling beginning, also can set and stop flag, making other client 100 can use this file destination 140.After accomplishing the relevant treatment of above-mentioned writing/stop to write, will remove and write flag (or stopping flag).
For clearly describing the operation of operation requests and equipment bottom, therefore with the windows workspace system that Microsoft was released be example as explanation, but its operational architecture can be applied in other operating system.
The driving framework of windows program (Window Driver Model, WDM Driver) is after receiving operation requests, just begins to carry out the processing of I/O demand.Operations such as this has comprised wherein that data read, write, the beginning of the control of hardware device, power management, Interrupt Process and WDM Driver life cycle and end, these all be WDM Driver the operation of the I/O that must support.
Ring-type buffer memory (Ring Buffer) is the shared drive that is arranged between drives interrupts program and the user side service.The ring-type buffer memory is used to handle the IRP_MJ_WRITE order.IRP_MJ_WRITE order is in order to will be by the hardware device (like hard disk, the carry-on dish of USB) of bottom when writing data when application program, can see IRP_MJ_WRITE off and orders being responsible for WDM Driver that this device moves.Be responsible for data are write the ring-type buffer memory by drives interrupts program 110, the user side service then is responsible for from the ring-type buffer memory, data being read.Fig. 3 A is user side service and the drives interrupts program 110 access synoptic diagram for the ring-type buffer memory.Between drives interrupts program 110 and user side service, be provided with one and share internal memory, shared drive is used to handle the IRP_MJ_READ order.Be responsible for writing fashionable when the user side service, drives interrupts program 110 is responsible for reading.IRP_MJ_READ order is in order to when application program will be by hardware device (like hard disk, the USB flash disk) reading of data of bottom, can see IRP_MJ_READ off and orders being responsible for WDM Driver that this device moves.Fig. 3 B is user side service and the drives interrupts program access synoptic diagram for shared drive.
When handling the IRP_MJ_WRITE order, for the use of ring-type buffer memory the producer/consumer's pattern (producer/consumer).Wherein, with drives interrupts program 110 as the producer (producer), in order to be responsible for to wherein writing data; The ring-type buffer memory is consumer (consumer).Whether drives interrupts program 110 constantly inquiry ring-type buffer memory detection has enough remaining spaces to carry out write operation.
When user side service during as the consumer, the user side service is responsible for from the ring-type buffer memory sense data and is handled.When detecting the ring-type data in buffer and begin to change, the user side service is promptly known has data to handle, and the user side service is upgraded in the back of finishing dealing with, so that drives interrupts program 110 can continue to write data.Therefore in the processing of the read-write of carrying out file destination 140 just as shown in Figure 4, it may further comprise the steps:
Step S410: the displacement of the file destination that reads and length;
Step S420: judge whether the displacement of file destination and length are present in the ring-type buffer memory;
Step S430: if when existing, then displacement and the length with file destination copies to shared drive; If when not existing, then skip to step S440;
Step S440: judge whether the remaining length that reads is 0;
Step S450: read length at non-0 o'clock as if remaining, then continue to obtain the corresponding data of file destination from client;
Step S460: the data that obtained are copied in the shared drive; And
Step S470: if remaining when reading length 0, then the target end file reads.
After drives interrupts program 110 is accomplished above-mentioned operations request, will return an object information to client 100, and reset the running time, use the running situation of the monitoring client 100 of repetition.
The present invention provides a kind of kernel that does not need the revised file storage system, also can be so that document storage system provides the function of data de-duplication.Thus, except reducing the file storing space, also can reduce software developer's cost of development.
Certainly; The present invention also can have other various embodiments; Under the situation that does not deviate from spirit of the present invention and essence thereof; Those of ordinary skill in the art work as can make various corresponding changes and distortion according to the present invention, but these corresponding changes and distortion all should belong to the protection domain of the appended claim of the present invention.
Claims (5)
1. in data de-duplication, support the method that file cocurrent is read and write for one kind; When solving an archival backup and carrying out a data de-duplication program; The delay mistake of this document stand-by program when accessing file; It is characterized in that, should support in data de-duplication that the method for file cocurrent read-write may further comprise the steps:
Operation one drives interrupts program and this document stand-by program in a client;
This client is sent an operation requests to this document stand-by program, in order to a file destination is carried out corresponding file operation; And
By this this operation requests of drives interrupts program interception, this drives interrupts program is carried out a service routine according to this operation requests, in order to this file destination is operated accordingly;
Read when handling if this service routine is a file, then this file destination is write to one and share in the internal memory, read this file destination to this document stand-by program in order to offer this drives interrupts program by this service routine; And
Write when handling if this service routine is a file, then this file destination is write in the ring-type buffer memory, carry out this data de-duplication program in order to offer this service routine by this drives interrupts program.
2. the method for in data de-duplication, supporting the file cocurrent read-write according to claim 1; It is characterized in that; Before sending this operation requests, also comprise: carry out the timing of a running time, in order to confirm among this running time, whether having this operation requests to send.
3. the method for in data de-duplication, supporting the file cocurrent read-write according to claim 1 is characterized in that this document writes processing and also comprises: write processing at the beginning and stop to write processing with one.
4. the method for in data de-duplication, supporting the file cocurrent read-write according to claim 1 is characterized in that when being a capacity query processing as if this service routine, then this drives interrupts program is returned the capacity of this shared drive current residual.
5. the method for in data de-duplication, supporting the file cocurrent read-write according to claim 1 is characterized in that after carrying out this service routine, also comprise: this drives interrupts program is sent to this client with an object information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105558247A CN102467569A (en) | 2010-11-16 | 2010-11-16 | Method for supporting concurrent read-write of file in data de-duplication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105558247A CN102467569A (en) | 2010-11-16 | 2010-11-16 | Method for supporting concurrent read-write of file in data de-duplication |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102467569A true CN102467569A (en) | 2012-05-23 |
Family
ID=46071212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105558247A Pending CN102467569A (en) | 2010-11-16 | 2010-11-16 | Method for supporting concurrent read-write of file in data de-duplication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102467569A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115098454A (en) * | 2022-07-26 | 2022-09-23 | 北京天融信网络安全技术有限公司 | Data processing method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101295276A (en) * | 2008-06-20 | 2008-10-29 | 中国科学院计算技术研究所 | Magnetic disk data backup system and method |
| US20090182789A1 (en) * | 2003-08-05 | 2009-07-16 | Sepaton, Inc. | Scalable de-duplication mechanism |
| CN101535965A (en) * | 2005-10-14 | 2009-09-16 | 塞门铁克操作公司 | Technique for improving scalability and portability of a storage management system |
| CN101631328A (en) * | 2009-08-14 | 2010-01-20 | 北京星网锐捷网络技术有限公司 | Synchronous method performing mutual exclusion access on shared resource, device and network equipment |
-
2010
- 2010-11-16 CN CN2010105558247A patent/CN102467569A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090182789A1 (en) * | 2003-08-05 | 2009-07-16 | Sepaton, Inc. | Scalable de-duplication mechanism |
| CN101535965A (en) * | 2005-10-14 | 2009-09-16 | 塞门铁克操作公司 | Technique for improving scalability and portability of a storage management system |
| CN101295276A (en) * | 2008-06-20 | 2008-10-29 | 中国科学院计算技术研究所 | Magnetic disk data backup system and method |
| CN101631328A (en) * | 2009-08-14 | 2010-01-20 | 北京星网锐捷网络技术有限公司 | Synchronous method performing mutual exclusion access on shared resource, device and network equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115098454A (en) * | 2022-07-26 | 2022-09-23 | 北京天融信网络安全技术有限公司 | Data processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11025717B2 (en) | Method and system for backing up and restoring data | |
| US8694764B2 (en) | Multi-phase resume from hibernate | |
| JP6024296B2 (en) | Information processing apparatus, copy control program, and copy control method | |
| KR102114109B1 (en) | Data storage device | |
| CN104685476A (en) | Techniques for recovering a virtual machine | |
| CN101819543A (en) | Snapshot data processing method, system and storage device | |
| CN101359321A (en) | Method and apparatus for implementing intercommunication of processors | |
| JP4717922B2 (en) | Information processing system, control method in information processing system, and management apparatus | |
| CN108874574B (en) | Garbage recycling method and related device | |
| CN106104515A (en) | Utilize File system design and the fault recovery method of nonvolatile memory | |
| KR101996641B1 (en) | Apparatus and method for memory overlay | |
| CN103559119B (en) | File operation requests disposal route and device | |
| CN105389268A (en) | Data storage system and operating method thereof | |
| CN102467557B (en) | Data de-duplication processing method | |
| WO2014147786A1 (en) | Storage system and data management method | |
| CN102467569A (en) | Method for supporting concurrent read-write of file in data de-duplication | |
| WO2016132572A1 (en) | Data placement device and data placement method | |
| CN109753222A (en) | Data-erasure method, server and computer-readable medium | |
| JP2017174205A (en) | Print control device and program | |
| WO2019196157A1 (en) | File reading method and application entity | |
| JP2010026626A (en) | Method, system and program for recording data | |
| US11755223B2 (en) | Systems for modular hybrid storage devices | |
| US11550506B2 (en) | Systems and methods for accessing hybrid storage devices | |
| US20220100614A1 (en) | Systems and methods for backing up volatile storage devices | |
| JP2011113625A (en) | Recording/reproducing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120523 |