CN101256524A - Method for maintaining file system caching compatibility - Google Patents
Method for maintaining file system caching compatibility Download PDFInfo
- Publication number
- CN101256524A CN101256524A CNA2007101156125A CN200710115612A CN101256524A CN 101256524 A CN101256524 A CN 101256524A CN A2007101156125 A CNA2007101156125 A CN A2007101156125A CN 200710115612 A CN200710115612 A CN 200710115612A CN 101256524 A CN101256524 A CN 101256524A
- Authority
- CN
- China
- Prior art keywords
- data
- backup
- server
- disk
- file system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000008569 process Effects 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 11
- 230000005540 biological transmission Effects 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 238000012217 deletion Methods 0.000 claims description 2
- 230000037430 deletion Effects 0.000 claims description 2
- 238000012795 verification Methods 0.000 claims description 2
- 238000004883 computer application Methods 0.000 abstract 1
- 238000013500 data storage Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Abstract
The invention relates to a computer application technology, concretely is a method for maintaining file system buffer coherence. A data is real-timely backed up in a backup server when the data is written into a working server file system layer, and the process data is informed that the writing is finished after the backup is completed. After the I/O operation of data magnetic disk is finished, the backup is real-timely relieved so that the using backup can not increase limitlessly. The useful function of the invention reside in: the backup data block is stored in tree structure, global information can be stored in root nodes, root nodes of the sub-tree stand for magnetic disk which need be backed up in service terminal, is arranged in equipment numbers, data blocks form sub-trees of magnetic disk nodes according to initial sector numbers, and the sub-trees are arranged in binary tree type so that all data can be searched in order. Therefore, the data structure optimized in operation process of the system layer can greatly improve the system performance.
Description
1, technical field
The present invention relates to computer system and communication technical field, be specifically related to a kind of method that keeps file system caching compatibility.
2, technical background
In order to improve the access efficiency of file system, the generic-document system all can use caching mechanism.This mechanism adopts software approach, with the data storage of nearest up-to-date visit in internal memory.When process needs once more visit data, do not need to visit disk, but directly in internal memory, read and write.
For write operation, when process write data in disk, operating system confirmed at first whether the data block of being correlated with exists in the buffer memory, if do not have, then reads in buffer memory from disk, then new data is write buffer memory.The magnetic disc i/o operation can carried out after a while certain suitable opportunity.The process of so just having given is revised the chance of these data once more, has reduced the magnetic disc i/o number of times, thereby has improved throughput of system greatly.
The problem that this mechanism causes is that when system's accident was delayed machine, " dirty " data in the buffer memory may not lost because of meeting in time writes back disk.In 7*24 hour running environment, the consequence that this class situation causes is serious.Generally speaking, the not responsible data storage of server does not naturally need to consider this type of risk, and therefore, present most of HA (high available) scheme is not taken this risk into account.But if memory device, server directly is responsible for the storage of data in other words, and so, we just must have a kind of scheme head it off.
3, summary of the invention
To be buffered in unusually the machine of delaying in the file system and to lose problem in order to solve, the invention provides a method that keeps file system caching compatibility.
Method of the present invention wants the technical solution scheme to realize in the following manner, when workspace server file system layer writing data blocks, the backup server of backing up in realtime (or miscellaneous equipment), after backup is finished just the notification process data write and finish.After data disk I/O operation is finished, remove backup in real time, use backup can infinitely not increase, concrete steps are as follows:
1) in the processing procedure of workspace server operating system write data, adds call back function block_commit_write (), by an application protocol data transmission is arrived backup machine, and be kept at an io_sync () in the internal memory, obtain the data block information that is written to disk, be transferred to backup machine by application protocol, and according to this information deletion backed up data piece, if the machine of delaying appears in workspace server, backup machine at first writes disk with the buffer memory in the tree-like chained list, takes over service then;
2) call back function obtains all writing data blocks and the relevant information in the designated disk, is transferred to backup machine, and after obtaining the response that data backup finishes, notice user disk write operation is finished, and call back function does not rely on concrete disk on Drive Layer.
The invention has the beneficial effects as follows: do not rely on concrete disk type, effectively reduce and dispose and use cost.In order to improve the efficient of traversal and modification Backup Data piece, the backed up data piece adopts the tree structure storage.Root node can be stored some overall information, and the root node of its subtree is represented the disk that service end need back up, and arranges in order by device number.Data block is arranged with the binary tree form by the subtree of start sector number size formation disk node, and all like this data can find in order.Therefore, the data structure of optimizing in the system layer operating process can obviously improve system performance.
4, description of drawings
Accompanying drawing 1 is buffer consistency realization flow figure;
Accompanying drawing 2 is buffer memory back-up storage structural representations.
5, embodiment
With reference to accompanying drawing method of the present invention is done following detailed explanation.
Method of the present invention wants the technical solution scheme to realize in the following manner, when workspace server file system layer writing data blocks, the backup server of backing up in realtime (or miscellaneous equipment), after backup is finished just the notification process data write and finish.After data disk I/O operation is finished, remove backup in real time, use backup can infinitely not increase.
Concrete steps are as follows:
1) obtain file system write operation and magnetic disc i/o and obtain system's write operation, magnetic disc i/o can be according to the difference of operating system platform and difference.For example, in linux system, can add call back function at VFS (Virtual File System) layer and realize.Write fashionablely when data, can in function block_commit_write (), add readjustment, same, when data are submitted to disk, can in Function e nd_bio_io_sync (), add readjustment.
2) workspace server is communicated by letter with the backup end for convenience of service end with communicating by letter of backup server, and what the invention provides a whole set of can believe agreement, and this communication protocol may operate in the multiple network environment, is not limited to communication modes such as Ethernet, PCI-E.This agreement has two kinds of packets, is respectively operation bag and respond packet.The operation bag is used for workspace server to backup server requests backup or releasing backup operation.Respond packet is a backup server when handling the operation of appointment, to the response of workspace server, is used for server decision further work.
If this agreement is moved in non-reliable network environment, this agreement provides verification and retransmission mechanism, guarantees reliability of data transmission with this.
● operation is surrounded by four necessary fields and three Optional Fields constitute:
I. operation: the operation of specifying backup server to carry out
Ii. device identification number: disk (subregion) unique identification
Iii. initial sector: pairing first sector number of data block
Iv. big or small: the length of data block (byte)
Data item (option): the data that comprise in the data block, if operation is the Backup Data piece, this must fill out.
V. check code (option): the structure sign indicating number of whole operation bag.This is made as 0 when the calculation check sign indicating number
Vi. transmitting time is stabbed (option): the time that packet generates
● response is surrounded by a necessary field and two Optional Fields constitute:
I. status word: backup server is to the result of operational processes
Ii. transmitting time is stabbed (option): the time that packet generates
Iii. check code (option): the structure sign indicating number of whole operation bag.This is made as 0 when the calculation check sign indicating number
● concrete transmission course is as follows:
I. when process writes data or file system submit data block in disk, obtain backup information
Ii. the information in (i) is inserted the operation bag and sent to backup server
Iii. backup server is handled packet according to the operation of operation bag appointment
Iv. backup server is inserted result respond packet and is sent to workspace server
V. workspace server is handled accordingly
Claims (4)
1, a kind of method that keeps file system caching compatibility, it is characterized in that, when workspace server file system layer writing data blocks, the backup server of backing up in realtime, after backup is finished just the notification process data write and finish, after data disk I/O operation is finished, remove backup in real time, concrete steps are as follows:
1) in the processing procedure of workspace server operating system write data, add call back function block_commit_write (), by an application protocol with data transmission to backup machine, and be kept in the internal memory in the tree structure; Simultaneously, be submitted in data and add call back function end_bio_io_sync () in the processing procedure of disk, obtain the data block information that is written to disk, be transferred to backup machine by application protocol, and according to this information deletion backed up data piece, if the machine of delaying appears in workspace server, backup machine at first writes disk with the buffer memory in the tree-like chained list, takes over service then;
2) call back function obtains all writing data blocks and the relevant information in the designated disk, is transferred to backup machine, and after obtaining the response that data backup finishes, notice user disk write operation is finished, and call back function does not rely on concrete disk on Drive Layer.
2, method according to claim 1, it is characterized in that, application layer protocol and concrete network environment are irrelevant, be not limited to Ethernet, communication modes such as PCI-E, and expand to reliable protocol with verifying function and retransmission mechanism according to the specific requirement of communication protocol, this agreement has two kinds of packets, be respectively operation bag and respond packet, the operation bag is used for workspace server to backup server requests backup or releasing backup operation, respond packet is that backup server is when handling the operation of appointment, to the response of workspace server, be used for server decision further work.
3, method according to claim 2 is characterized in that application protocol moves in non-reliable network environment, agreement provides verification and retransmission mechanism to guarantee reliability of data transmission, and concrete steps are as follows:
● operation is surrounded by four necessary fields and three Optional Fields constitute:
I. operation: the operation of specifying backup server to carry out;
Ii. device identification number: disk unique identification;
Iii. initial sector: pairing first sector number of data block;
Iv. big or small: the length of data block;
V. data item: the data that comprise in the data block, if operation is the Backup Data piece, this must fill out;
Vi. check code: the structure sign indicating number of whole operation bag.This is made as 0 when the calculation check sign indicating number;
Vii. transmitting time is stabbed: the time that packet generates;
● response is surrounded by a necessary field and two Optional Fields constitute:
I. status word: backup server is to the result of operational processes;
Ii. transmitting time is stabbed: the time that packet generates;
Iii. check code: the structure sign indicating number of whole operation bag.This is made as 0 when the calculation check sign indicating number;
● concrete transmission course is as follows:
I. when process writes data or file system submit data block in disk, obtain backup information;
Ii. the information among the i is inserted the operation bag and sent to backup server;
Iii. backup server is handled packet according to the operation of operation bag appointment;
Iv. backup server is inserted result respond packet and is sent to workspace server;
V. workspace server is handled accordingly.
4, method according to claim 3, it is characterized in that, the employing tree structure storage of Backup Data piece, root node is stored some overall information, the root node of its subtree is represented the disk that service end need back up, and arrange in order by device number, data block constitutes the subtree of disk node by the start sector number size, arranges to make things convenient for data searching in order all with the binary tree form.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2007101156125A CN101256524A (en) | 2007-12-17 | 2007-12-17 | Method for maintaining file system caching compatibility |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2007101156125A CN101256524A (en) | 2007-12-17 | 2007-12-17 | Method for maintaining file system caching compatibility |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101256524A true CN101256524A (en) | 2008-09-03 |
Family
ID=39891366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2007101156125A Pending CN101256524A (en) | 2007-12-17 | 2007-12-17 | Method for maintaining file system caching compatibility |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101256524A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101866307A (en) * | 2010-06-24 | 2010-10-20 | 杭州华三通信技术有限公司 | Data storage method and device based on mirror image technology |
| CN102340489A (en) * | 2010-07-20 | 2012-02-01 | 阿里巴巴集团控股有限公司 | Data transmission method between servers and servers |
| CN102955720A (en) * | 2011-08-25 | 2013-03-06 | 北京中科智网科技有限公司 | Method for improving stability of EXT (extended) file system |
| CN103365656A (en) * | 2013-06-28 | 2013-10-23 | 华为技术有限公司 | Method and terminal for guaranteeing cache consistency |
| CN103853827A (en) * | 2014-03-07 | 2014-06-11 | 浪潮电子信息产业股份有限公司 | Database backup method capable of guaranteeing data consistency |
| CN106055281A (en) * | 2016-06-29 | 2016-10-26 | 广州华多网络科技有限公司 | Data writing method and device |
-
2007
- 2007-12-17 CN CNA2007101156125A patent/CN101256524A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101866307A (en) * | 2010-06-24 | 2010-10-20 | 杭州华三通信技术有限公司 | Data storage method and device based on mirror image technology |
| CN102340489A (en) * | 2010-07-20 | 2012-02-01 | 阿里巴巴集团控股有限公司 | Data transmission method between servers and servers |
| CN102340489B (en) * | 2010-07-20 | 2014-10-29 | 阿里巴巴集团控股有限公司 | Data transmission method between servers and servers |
| CN102955720A (en) * | 2011-08-25 | 2013-03-06 | 北京中科智网科技有限公司 | Method for improving stability of EXT (extended) file system |
| CN102955720B (en) * | 2011-08-25 | 2015-09-09 | 北京中科智网科技有限公司 | A kind of method improving ext file system stability |
| CN103365656A (en) * | 2013-06-28 | 2013-10-23 | 华为技术有限公司 | Method and terminal for guaranteeing cache consistency |
| CN103365656B (en) * | 2013-06-28 | 2016-10-19 | 华为技术有限公司 | Ensure method and the terminal of buffer consistency |
| CN103853827A (en) * | 2014-03-07 | 2014-06-11 | 浪潮电子信息产业股份有限公司 | Database backup method capable of guaranteeing data consistency |
| CN106055281A (en) * | 2016-06-29 | 2016-10-26 | 广州华多网络科技有限公司 | Data writing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10437721B2 (en) | Efficient garbage collection for a log-structured data store | |
| US7849282B2 (en) | Filesystem building method | |
| JP5235899B2 (en) | Method, system, and program for transparent backup to tiered storage system | |
| JP5967673B2 (en) | Data maintenance methods | |
| US6785693B2 (en) | Management of multiple links to a file in a file system | |
| US20030229764A1 (en) | Data storage subsystem | |
| JP4615344B2 (en) | Data processing system and database management method | |
| US20080263259A1 (en) | Hints model for optimization of storage devices connected to host and write optimization schema for storage devices | |
| CN106021016A (en) | Virtual point in time access between snapshots | |
| JP2005276192A (en) | Method and apparatus for increasing data storage capacity | |
| CN101854388A (en) | Method and system concurrently accessing a large amount of small documents in cluster storage | |
| JP2009505255A (en) | Method, system, and program for maintaining an aggregate containing active files in a storage pool in a random access medium (maintaining an aggregate containing active files in a storage pool in a random access medium) | |
| CN101256524A (en) | Method for maintaining file system caching compatibility | |
| US20020161982A1 (en) | System and method for implementing a storage area network system protocol | |
| US8190834B2 (en) | Process for contiguously streaming data from a content addressed storage system | |
| US20070174360A1 (en) | Storage system embedding database | |
| CN108897822A (en) | A kind of data-updating method, device, equipment and readable storage medium storing program for executing | |
| CN111435286B (en) | Data storage method, device and system | |
| CN103150268A (en) | Block-level data capture method in CDP (Continuous Data Protection) | |
| CN112464044B (en) | File data block change information monitoring and management system and method thereof | |
| US8577890B2 (en) | Modifying data set name qualifiers | |
| US6029229A (en) | Digital data storage subsystem including directory for efficiently providing formatting information for stored records | |
| CN109960460B (en) | Distributed storage system | |
| CN107615259A (en) | A kind of data processing method and system | |
| CN104317530A (en) | Design of data capture method in remote disaster recovery technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20080903 |