CN102508740A - Remote replication method of file system - Google Patents
Remote replication method of file system Download PDFInfo
- Publication number
- CN102508740A CN102508740A CN2011103303036A CN201110330303A CN102508740A CN 102508740 A CN102508740 A CN 102508740A CN 2011103303036 A CN2011103303036 A CN 2011103303036A CN 201110330303 A CN201110330303 A CN 201110330303A CN 102508740 A CN102508740 A CN 102508740A
- Authority
- CN
- China
- Prior art keywords
- file system
- data
- remote copy
- reformed
- copy method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a remote replication method of a file system. The remote replication method comprises the following steps: A) identifying a file which is changed in the current period in a source file system; B) getting an expansion region which is changed in the source file system and generating incremental data to be synchronized; and C) synchronizing the incremental data to a target file system. As only data blocks which are changed are transmitted during a replication process according to the method provided by the invention, the quantity of the data to be transmitted is greatly reduced and the replication efficiency is improved.
Description
Technical field
The present invention relates to computer memory technical field, particularly a kind of file system remote copy method.
Background technology
At present, some significant datas during computer memory system is widely used in storing people's daily life and working, and efficient, safe data backup receives computer user's care and attention.When operating system or reservoir break down and damage, can replace damaged data through storing Backup Data rapidly, to guarantee that critical data is not lost and the continuity of regular traffic.
Data are duplicated and can be regarded a kind of backup policy as, and can be divided into local replica and remote copy.Local replica need not pass through Network Transmission, so backup rate is than very fast.But thisly duplicate down raw data and Backup Data all is in same system, therefore can only system can with prerequisite under solve the problem that raw data is destroyed.Remote copy is that business continuance and disaster recovery provide more strong assurance, and it sets up data image at remote site, even therefore local system damages, and also can very fast restore data.
In the remote copy process; Guarantee the important method that efficiently provides Network Transmission and protection that valid data duplicate; But existing remote copy technology all is that the data on the source disk are all copied on the remote disk generally, must just cause having a lot of invalid data transmission like this.For example have only the variation of 1Byte data in the data of 8K, but existing remote copy technology often need be transmitted the data of whole 8K, rather than only transmit the 1Byte data that change.
Therefore how a kind of method being provided, as much as possible to transmit the purpose that minimum data reach remote copy, effectively utilizing network bandwidth conserve memory resource, is that present data sharply increase the challenge that faces.
Summary of the invention
The purpose of this invention is to provide a kind of file system remote copy method, existing remote copy technical data transfer efficiency is low to solve, the problem of waste network bandwidth resources.
The present invention proposes a kind of file system remote copy method, may further comprise the steps:
A, in source file system identification current period in reformed file;
B, obtain reformed expansion area in the source file system, and generate and treat synchronous incremental data;
C, incremental data is synchronized to target file system.
Further, in the steps A, in source file system, discern reformed file in the current period through document change daily record technology.
Further, step B comprises:
B1, through storage checkpoint technology, the reformed file of comparison of aforementioned one by one between the storage checkpoint of current period and the storage checkpoint of last one-period;
B2, discern reformed expansion area according to comparative result;
B3, extract the data in the expansion area, and generate and treat synchronous incremental data.
Further, among the step B1, utilize metadata to be compared in the storage checkpoint of current period and the storage checkpoint of last one-period.
Further, among the step C, utilize rsync data sync instrument that incremental data is synchronized to target file system.
Further, among the step C, utilize rsync data sync instrument, and, incremental data is synchronized to target file system through the SSH mode.
With respect to prior art, the invention has the beneficial effects as follows:
1, method of the present invention is only transmitted the data block that changes to remote object file system copy data the time, and greatly reducing needs data quantity transmitted, has promoted duplicating efficiency.
2, the present invention utilizes document change daily record technology and storage checkpoint technology, and combines metadata comparison technology when the file that identification changes, and has promoted the incremental data identification efficiency greatly, has further optimized the data duplicating efficiency.
Description of drawings
Fig. 1 is a kind of embodiment process flow diagram of file system remote copy method of the present invention.
Embodiment
Below in conjunction with accompanying drawing, specify the present invention.
See also Fig. 1, it is for a kind of embodiment process flow diagram of file system remote copy method of the present invention, and it may further comprise the steps:
S11, reformed file in the identification current period in source file system.
Along with the read-write operation of source file system, its data therefore in order to protect the data in the source file system better, are necessary the schedule backup source file system also in continuous variation, and in the time interval of a source file system of every backup, the present invention is referred to as replicative cycle.
If each replicative cycle all duplicates data all in the whole source file system; Then certainly will to spend the long time and waste more Internet resources; And except first replicative cycle (first replicative cycle needs all data of copy source file system), those unaltered data there is no need in each all copy source file system.Therefore, the present invention can at first identify the file that those change at current period.
The present invention preferably adopts document change daily record technology (FCL), identifies in the source file system reformed file in current period rapidly, and generates the tabulation of these files of record.Document change daily record technology can TFS in time of data change; And record change type; Can write down the change of multiple file system, the for example establishment of file, additional data, cover data, brachymemma data, extended attribute are revised, punch, link, are cancelled link, rename etc.Remote copy utilizes this technology can discern rapidly in the period demand time range, and the data change has taken place in which file.
In this step, system need not whole file system is scanned, and with traditional document level remote copy compared with techniques, has significantly promoted handling property.
S12 obtains reformed expansion area in the source file system, and synchronous incremental data is treated in generation.
The present invention preferably adopts storage checkpoint (Storage Checkpoint) technology to confirm described expansion area (extent), and storage checkpoint technology can be put quick establishment file system at any time and video lastingly.Reflection includes only the one group of pointer that points to file system when generating, after the data of file system changed, legacy data was copied in the reflection, and new data just can be write in the host file system then, and promptly writing is copy mode.Each storage checkpoint reflection all can independently carry out read-write operation; Therefore; The whole process of remote copy can be stored the checkpoint based on file system and provided, even like this in reproduction process, has lasting write operation to exist; Reproduction process also can be carried out smoothly, and customer service can not be affected.
In this step; Can be earlier through storage checkpoint technology; More reformed one by one file between the storage checkpoint of the storage checkpoint of current period and last one-period; Just can identify reformed expansion area then, and extract the data in the expansion area, generate and treat synchronous incremental data according to comparison result.Therefore, owing to only need the reformed file of contrast, even have mass file in the whole file system, also need not the concern of data comparison process can expend the long period.And after the reproduction process of current period accomplished, in source file system, older storage checkpoint will be deleted, and the storage checkpoint of up-to-date establishment can be retained, so that next cycle carries out the data contrast when duplicating.
Wherein, the entire comparison process can all be carried out through metadata.Metadata (Metadata) is the data of objects such as descriptor resource or data, its application target is recognition resource, estimate resource, follow the trail of in use variation of resource, realize simply managing a large amount of networked data efficiently, realize effective discovery of information resources and search, integration organization and to using the resources effective management.Here utilize metadata to be compared in the storage checkpoint of current period and the storage checkpoint of last one-period, just need not the access file data, efficient is high.
S13 is synchronized to target file system with incremental data.
Transmission data file of the present invention can utilize rsync data sync instrument; Rsync data sync instrument is a teledata synchronization means; Can be through the file between the fast synchronous multiple host of LAN/WAN, it has a plurality of characteristics: can preserve whole directory tree and file system by mirror image 1.; 2. can be easy to accomplish to keep authority, time, soft or hard link of original file etc.; 3. need not special authority can install; 4. the flow process of optimizing, file transfer efficient is high; 5. support anonymous transmission.
In addition; Rsync data sync instrument can use modes such as rcp, ssh to come transfer files; The present invention preferably adopts SSH (Secure Shell) mode to come transfer files, and it is that network work group (Network Working Group) by IETF (Internet Engineering Task Force) formulates.Through using SSH, can encrypt the data of all transmission, and can prevent DNS (Domain Name System, domain name system) deception and IP spoofing.Use SSH, also having the data of an extra benefit transmission is through overcompression, so can accelerate transmitting speed.
When source file system breaks down, can switch to target file system rapidly, provide professional through target file system.Because therefore target file system in fact can, can change to target file system in extremely short time inscribe separately for the user provides the read-only data visit, and read-only business is provided.If need target file system that read-write business is provided, only need carry out a spot of setting and can accomplish switching.
What deserves to be mentioned is that the target file system of remote copy can provide read-only business to the user at any time.User capture this moment in fact be not real file system, but video immediately in the storage checkpoint that target file system was duplicated the back establishment of finishing in lasted one-period.So both guaranteed that the user can have access to complete available file at any time, can not have influence on carrying out smoothly of remote copy process because of user's visit yet.
More than disclosedly be merely several specific embodiment of the present invention, but the present invention is not limited thereto, any those skilled in the art can think variation, only otherwise exceed the said scope of appended claims, all should drop in protection scope of the present invention.
Claims (6)
1. a file system remote copy method is characterized in that, may further comprise the steps:
A, in source file system identification current period in reformed file;
B, obtain reformed expansion area in the source file system, and generate and treat synchronous incremental data;
C, incremental data is synchronized to target file system.
2. file system remote copy method as claimed in claim 1 is characterized in that, in the steps A, in source file system, discerns reformed file in the current period through document change daily record technology.
3. file system remote copy method as claimed in claim 1 is characterized in that step B comprises:
B1, through storage checkpoint technology, the reformed file of comparison of aforementioned one by one between the storage checkpoint of current period and the storage checkpoint of last one-period;
B2, discern reformed expansion area according to comparative result;
B3, extract the data in the expansion area, and generate and treat synchronous incremental data.
4. file system remote copy method as claimed in claim 3 is characterized in that, among the step B1, utilizes metadata to be compared in the storage checkpoint of current period and the storage checkpoint of last one-period.
5. file system remote copy method as claimed in claim 1 is characterized in that, among the step C, utilizes rsync data sync instrument that incremental data is synchronized to target file system.
6. file system remote copy method as claimed in claim 5 is characterized in that, among the step C, utilizes rsync data sync instrument, and through the SSH mode, incremental data is synchronized to target file system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103303036A CN102508740A (en) | 2011-10-26 | 2011-10-26 | Remote replication method of file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103303036A CN102508740A (en) | 2011-10-26 | 2011-10-26 | Remote replication method of file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102508740A true CN102508740A (en) | 2012-06-20 |
Family
ID=46220832
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011103303036A Pending CN102508740A (en) | 2011-10-26 | 2011-10-26 | Remote replication method of file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102508740A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104662521A (en) * | 2012-07-16 | 2015-05-27 | 康佩伦特科技公司 | System and method for replication of data utilizing delta volumes |
| CN105791357A (en) * | 2014-12-24 | 2016-07-20 | 金蝶软件（中国）有限公司 | File synchronization method and system for use in ERP (Enterprise Resource Planning) multi-machine cluster system |
| CN106202262A (en) * | 2016-06-29 | 2016-12-07 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN108228226A (en) * | 2017-12-29 | 2018-06-29 | 北京元心科技有限公司 | Difference method, device and the corresponding terminal of hard link |
| CN108647317A (en) * | 2018-05-10 | 2018-10-12 | 东软集团股份有限公司 | Generation method, device and the storage medium and electronic equipment of delta file |
| CN108733541A (en) * | 2017-04-17 | 2018-11-02 | 伊姆西Ip控股有限责任公司 | The method and apparatus for replicating progress for determining data in real time |
| CN110109871A (en) * | 2018-04-26 | 2019-08-09 | 中国科学院高能物理研究所 | A kind of cross-site high-energy physics data access method and system |
| CN110618642A (en) * | 2019-10-25 | 2019-12-27 | 蚌埠凯盛工程技术有限公司 | Intelligent edge roller system control method and device |
| WO2021129151A1 (en) * | 2019-12-28 | 2021-07-01 | 华为技术有限公司 | File backup method and apparatus, and terminal device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1455348A (en) * | 2002-04-03 | 2003-11-12 | 微软公司 | Incremental duplication of source document and its bag on networked resources |
| CN101051285A (en) * | 2006-09-21 | 2007-10-10 | 上海交通大学 | File matching method in computer network data backup |
| US20080126442A1 (en) * | 2006-08-04 | 2008-05-29 | Pavel Cisler | Architecture for back up and/or recovery of electronic data |
| CN102096615A (en) * | 2011-01-25 | 2011-06-15 | 上海爱数软件有限公司 | Remote incremental backup method and system |
-
2011
- 2011-10-26 CN CN2011103303036A patent/CN102508740A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1455348A (en) * | 2002-04-03 | 2003-11-12 | 微软公司 | Incremental duplication of source document and its bag on networked resources |
| US20080126442A1 (en) * | 2006-08-04 | 2008-05-29 | Pavel Cisler | Architecture for back up and/or recovery of electronic data |
| CN101051285A (en) * | 2006-09-21 | 2007-10-10 | 上海交通大学 | File matching method in computer network data backup |
| CN102096615A (en) * | 2011-01-25 | 2011-06-15 | 上海爱数软件有限公司 | Remote incremental backup method and system |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104662521B (en) * | 2012-07-16 | 2018-11-20 | 康佩伦特科技公司 | Utilize the system and method for delta volumes of replicate data |
| CN104662521A (en) * | 2012-07-16 | 2015-05-27 | 康佩伦特科技公司 | System and method for replication of data utilizing delta volumes |
| CN105791357A (en) * | 2014-12-24 | 2016-07-20 | 金蝶软件（中国）有限公司 | File synchronization method and system for use in ERP (Enterprise Resource Planning) multi-machine cluster system |
| CN106202262B (en) * | 2016-06-29 | 2020-09-25 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106202262A (en) * | 2016-06-29 | 2016-12-07 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN108733541A (en) * | 2017-04-17 | 2018-11-02 | 伊姆西Ip控股有限责任公司 | The method and apparatus for replicating progress for determining data in real time |
| CN108228226A (en) * | 2017-12-29 | 2018-06-29 | 北京元心科技有限公司 | Difference method, device and the corresponding terminal of hard link |
| CN110109871A (en) * | 2018-04-26 | 2019-08-09 | 中国科学院高能物理研究所 | A kind of cross-site high-energy physics data access method and system |
| CN110109871B (en) * | 2018-04-26 | 2021-04-06 | 中国科学院高能物理研究所 | Cross-site high-energy physical data access method and system |
| CN108647317A (en) * | 2018-05-10 | 2018-10-12 | 东软集团股份有限公司 | Generation method, device and the storage medium and electronic equipment of delta file |
| CN110618642A (en) * | 2019-10-25 | 2019-12-27 | 蚌埠凯盛工程技术有限公司 | Intelligent edge roller system control method and device |
| CN110618642B (en) * | 2019-10-25 | 2023-02-28 | 蚌埠凯盛工程技术有限公司 | Intelligent edge roller system control method and device |
| WO2021129151A1 (en) * | 2019-12-28 | 2021-07-01 | 华为技术有限公司 | File backup method and apparatus, and terminal device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102508740A (en) | Remote replication method of file system | |
| KR100985169B1 (en) | Apparatus and method for file deduplication in distributed storage system | |
| US8521691B1 (en) | Seamless migration between replication technologies | |
| US8600945B1 (en) | Continuous data replication | |
| US9471579B1 (en) | Replicating selected snapshots from one storage array to another, with minimal data transmission | |
| US8712962B1 (en) | Snapshots in de-duplication | |
| US9940205B2 (en) | Virtual point in time access between snapshots | |
| US8924668B1 (en) | Method and apparatus for an application- and object-level I/O splitter | |
| US9740572B1 (en) | Replication of xcopy command | |
| US8914595B1 (en) | Snapshots in deduplication | |
| US9389800B1 (en) | Synthesizing virtual machine disk backups | |
| US9069479B1 (en) | Snapshots in deduplication | |
| US8862546B1 (en) | Virtual access roll | |
| US8521694B1 (en) | Leveraging array snapshots for immediate continuous data protection | |
| US9672117B1 (en) | Method and system for star replication using multiple replication technologies | |
| US8725691B1 (en) | Dynamic LUN resizing in a replication environment | |
| US8850143B1 (en) | Point in time access in a replication environment with LUN resizing | |
| JP4923016B2 (en) | Virtual data long-term storage processing technology | |
| CN103875229B (en) | asynchronous replication method, device and system | |
| US11080148B2 (en) | Method and system for star replication using multiple replication technologies | |
| KR101662212B1 (en) | Database Management System providing partial synchronization and method for partial synchronization thereof | |
| CN102968486B (en) | A kind of highly reliable file synchronisation method based on change journal | |
| CN102096615A (en) | Remote incremental backup method and system | |
| CN111221678B (en) | Hbase data backup/recovery system, method and device and electronic equipment | |
| US11386070B2 (en) | Method and system for secure data replication data integrity verification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120620 |
|
| WD01 | Invention patent application deemed withdrawn after publication |