CN105094711A - Method and device for achieving copy-on-write file system - Google Patents
Method and device for achieving copy-on-write file system Download PDFInfo
- Publication number
- CN105094711A CN105094711A CN201510607910.0A CN201510607910A CN105094711A CN 105094711 A CN105094711 A CN 105094711A CN 201510607910 A CN201510607910 A CN 201510607910A CN 105094711 A CN105094711 A CN 105094711A
- Authority
- CN
- China
- Prior art keywords
- metadata
- file
- data
- write
- dirty data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method and device for achieving a copy-on-write file system. The method includes the steps of searching an internal storage for first meta-data with dirt data identification, setting the lock identification of the first meta-data, distributing a new disk space for dirt data corresponding to the first meta-data, brushing the dirt data downwards to the new disk space, writing the first meta-data in a meta-data file adjacent to the position of the new disk space, writing second meta-data, corresponding to the dirt data, in a block map file, and brushing the dirt data downwards to the new disk space. By means of the method of not directly conducting modification on existing disk data, it is ensured that a file system is in the consistent state all the time, the meta-data are written in the meta-data file, and the unnecessary disk seeking time is shortened.
Description
Technical field
The present invention relates to storage, field of filesystems, more particularly, relate to a kind of method and the device that realize copy-on-write file system.
Background technology
Common linux file system, when Update Table, can be modified to legacy data on disk, if system is delayed machine in the process of amendment, such as, during power fail, will cause corrupted data, thus cause the inconsistent of file system.Inux file system write data time magnetic head can be stored between the inode data of disk fixed area and data data come and go repeatedly operate, add the unnecessary disk seek time that this " read-modify-write " process is paid.
Therefore, how to ensure that the file system moment is in coherency state, and reduce the problem that unnecessary disk seek time is needs solution now.
Summary of the invention
The object of the present invention is to provide a kind of method and the device that realize copy-on-write file system, to ensure that the file system moment is in coherency state, and reduce unnecessary disk seek time.
For achieving the above object, following technical scheme is embodiments provided:
Realize a method for copy-on-write file system, comprising:
The first metadata with dirty data mark is found in internal memory;
The lock of described first metadata is identified set;
The dirty data corresponding for described first metadata distributes new disk space, and by brush under described dirty data to described new disk space;
Described first metadata is write in the meta data file adjacent with described new disk space position; And second metadata corresponding with described dirty data in blockmap file is write in described meta data file.
Preferably, described the first metadata found in internal memory with dirty data mark, comprising:
The metadata chained list having described dirty data is found in internal memory; And/or, there is no described dirty data but metadata itself amendment chained list on metadata.
Preferably, the dirty data corresponding for described first metadata distributes new disk space, comprising:
In advance initial metadata in blockmap file is write in described meta data file, and distribute new disk space for described dirty data.
Preferably, described second metadata corresponding with described dirty data in blockmap file is write in described meta data file after, comprising:
The fs place value of data block corresponding for described dirty data is copied to the update position in described blockmap file.
Preferably, if described second metadata does not successfully write in described meta data file, then described second metadata is left in writewait queue, and re-execute the step described second metadata write in described meta data file.
Realize a device for copy-on-write file system, comprising:
Find module, for finding the first metadata with dirty data mark in internal memory;
Set module, for identifying set by the lock of described first metadata;
Distribution module, distributes new disk space for the dirty data corresponding for described first metadata;
Lower brush module, for brushing to described new disk space under described dirty data;
First metadata writing module, for writing in the meta data file adjacent with described new disk space position by described first metadata;
Second metadata writing module, for writing second metadata corresponding with described dirty data in blockmap file in described meta data file.
Preferably, described searching module, comprising:
The metadata chained list having described dirty data is found in internal memory; And/or, there is no described dirty data but metadata itself amendment chained list on metadata.
Preferably, described distribution module, comprising:
In advance initial metadata in blockmap file is write in described meta data file, and distribute new disk space for described dirty data.
Preferably, described second metadata writing module, comprising:
The fs place value of data block corresponding for described dirty data is copied to the update position in described blockmap file.
Preferably, if described second metadata does not successfully write in described meta data file by described second metadata writing module, then described second metadata is left in writewait queue, and again described second metadata is write in described meta data file.
Known by above scheme, a kind of method and device realizing copy-on-write file system that the embodiment of the present invention provides, comprising: in internal memory, find the first metadata with dirty data mark; The lock of described first metadata is identified set; The dirty data corresponding for described first metadata distributes new disk space, and by brush under described dirty data to described new disk space; Described first metadata is write in the meta data file adjacent with described new disk space position; And second metadata corresponding with described dirty data in blockmap file is write in described meta data file, thisly will brush new disk space under dirty data, the method of directly not modifying on former data in magnetic disk, ensure that the file system moment is in coherency state, and metadata is write in meta data file, decrease unnecessary disk seek time.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is a kind of method flow schematic diagram realizing copy-on-write file system disclosed in the embodiment of the present invention;
Fig. 2 is the organizational form schematic diagram of file data in internal memory disclosed in the embodiment of the present invention;
Fig. 3 is a kind of apparatus structure schematic diagram realizing copy-on-write file system disclosed in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
The embodiment of the invention discloses a kind of method and the device that realize copy-on-write file system, to ensure that the file system moment is in coherency state, and reduce unnecessary disk seek time.
See Fig. 1, a kind of method realizing copy-on-write file system that the embodiment of the present invention provides, comprising:
S101, find in internal memory with dirty data mark the first metadata;
Preferably, described the first metadata found in internal memory with dirty data mark, comprising:
The metadata chained list having described dirty data is found in internal memory; And/or, there is no described dirty data but metadata itself amendment chained list on metadata.
S102, by the lock of described first metadata identify set;
Concrete, in the present embodiment, by identifying set to the lock of the first metadata, can ensure in lower swiped through journey, the content of these the first metadata is not by other modifications of program.
S103, distribute new disk space for the dirty data that described first metadata is corresponding, and will under described dirty data brush to described new disk space;
Preferably, the dirty data corresponding for described first metadata distributes new disk space, comprising:
In advance initial metadata in blockmap file is write in described meta data file, and distribute new disk space for described dirty data.
Concrete, in advance the inode (metadata) of blockmap file is write in inodefile, be in order to the relevant block at the blockmapinode place by inodefile is made dirty, next can distribute disk space to it.
Concrete, for dirty data block distributes disk space, comprise the relevant block of blockmap file and inodefile.
S104, described first metadata is write in the meta data file adjacent with described new disk space position; And second metadata corresponding with described dirty data in blockmap file is write in described meta data file.
Concrete, re-write in inodefile by the inode of blockmap file, for re-writing in the position write in advance, current inode comprises the necessary informations such as the actual magnetic disk address that blockmap file distributed.
Preferably, described second metadata corresponding with described dirty data in blockmap file is write in described meta data file after, comprising:
The fs place value of data block corresponding for described dirty data is copied to the update position in described blockmap file.
Concrete, upgrade blockmap file, by the fs place value of promising dirty block copy to update position, comprise blockmap file and inodefile dirty piece equally.
Concrete, this " copy-on-write " file system metadata comprises: inode file, the file inode that include file system is all; Block map file, the block of include file system describes, in order to identify free block.
Concrete, the inode of this " copy-on-write " file system is divided into the inode (inode_incore) in disk inode (inode_disk) and internal memory, and both structures have certain difference.
1、inode_disk:
The Inode_disk that the present embodiment provides is made up of two parts, and Part I comprises the information such as the owner, authority, atime.Part II is 16 pointers pointing to data block or indirect block, and each pointer length is 4Byte, and as file data curtailment 64Byte, data are placed directly in the Part II of inode_disk; As file data curtailment 64KB, 16 pointers directly point to data block; As file data curtailment 64M, by one-level indirect block organising data; When being more than or equal to 64M, need two-stage indirect block organising data.
2、inode_incore:
The inode_incore that the present embodiment provides is made up of three parts.Part I comprises dirty mark, dirty mark, and the pointer of residing internal memory inode chained list.Part II is similarly the pointer of 16 data structures, similar with inode_disk, the long 4Byte of each pointer, points to data block or indirect block.Part III is inode_disk structure, and this part is loaded into by the inode in the inodefile of disk.
Concrete, see Fig. 2, for the organizational form schematic diagram of file data in internal memory that the present embodiment provides, the buffer pointed indirect data buffer of inode_incore, comprise 1024 pointed, 2014 4k file data buffer in indirect buffer, and record the disk address of 1024 file data blocks.
Concrete, in the present embodiment, each blockmap length is that 32bit, bit0 indicate block whether in current active file system; Whether bit1-bit20 is snapshot position, indicate in certain snapshot; Bit31 is update position, indicate current block the need of under brush disk.Only have when all positions of blockmap are all 0, this block is only idle available.
A kind of method realizing copy-on-write file system that the embodiment of the present invention provides, comprising: in internal memory, find the first metadata with dirty data mark; The lock of described first metadata is identified set; The dirty data corresponding for described first metadata distributes new disk space, and by brush under described dirty data to described new disk space; Described first metadata is write in the meta data file adjacent with described new disk space position; And second metadata corresponding with described dirty data in blockmap file is write in described meta data file, thisly will brush new disk space under dirty data, the method of directly not modifying on former data in magnetic disk, ensure that the file system moment is in coherency state, and metadata is write in meta data file, decrease unnecessary disk seek time.
Preferably, in another embodiment of the invention, if described second metadata does not successfully write in described meta data file, then described second metadata is left in writewait queue, and re-execute the step described second metadata write in described meta data file.
See Fig. 3, a kind of device realizing copy-on-write file system that the embodiment of the present invention provides, comprising:
Find module 100, for finding the first metadata with dirty data mark in internal memory;
Set module 200, for identifying set by the lock of described first metadata;
Distribution module 300, distributes new disk space for the dirty data corresponding for described first metadata;
Lower brush module 400, for brushing to described new disk space under described dirty data;
First metadata writing module 500, for writing in the meta data file adjacent with described new disk space position by described first metadata;
Second metadata writing module 600, for writing second metadata corresponding with described dirty data in blockmap file in described meta data file.
Preferably, in another embodiment of the invention, described searching module 100, comprising:
The metadata chained list having described dirty data is found in internal memory; And/or, there is no described dirty data but metadata itself amendment chained list on metadata.
Preferably, in another embodiment of the invention, described distribution module 300, comprising:
In advance initial metadata in blockmap file is write in described meta data file, and distribute new disk space for described dirty data.
Preferably, in another embodiment of the invention, described second metadata writing module 600, comprising:
The fs place value of data block corresponding for described dirty data is copied to the update position in described blockmap file.
Preferably, in another embodiment of the invention, if described second metadata does not successfully write in described meta data file by described second metadata writing module 600, then described second metadata is left in writewait queue, and again described second metadata is write in described meta data file.
In this instructions, each embodiment adopts the mode of going forward one by one to describe, and what each embodiment stressed is the difference with other embodiments, between each embodiment identical similar portion mutually see.
To the above-mentioned explanation of the disclosed embodiments, professional and technical personnel in the field are realized or uses the present invention.To be apparent for those skilled in the art to the multiple amendment of these embodiments, General Principle as defined herein can without departing from the spirit or scope of the present invention, realize in other embodiments.Therefore, the present invention can not be restricted to these embodiments shown in this article, but will meet the widest scope consistent with principle disclosed herein and features of novelty.
Claims (10)
1. realize a method for copy-on-write file system, it is characterized in that, comprising:
The first metadata with dirty data mark is found in internal memory;
The lock of described first metadata is identified set;
The dirty data corresponding for described first metadata distributes new disk space, and by brush under described dirty data to described new disk space;
Described first metadata is write in the meta data file adjacent with described new disk space position; And second metadata corresponding with described dirty data in blockmap file is write in described meta data file.
2. method according to claim 1, is characterized in that, described the first metadata found in internal memory with dirty data mark, comprising:
The metadata chained list having described dirty data is found in internal memory; And/or, there is no described dirty data but metadata itself amendment chained list on metadata.
3. method according to claim 2, is characterized in that, the dirty data corresponding for described first metadata distributes new disk space, comprising:
In advance initial metadata in blockmap file is write in described meta data file, and distribute new disk space for described dirty data.
4. method according to claim 3, is characterized in that, described second metadata corresponding with described dirty data in blockmap file is write in described meta data file after, comprising:
The fs place value of data block corresponding for described dirty data is copied to the update position in described blockmap file.
5. method according to claim 4, it is characterized in that, if described second metadata does not successfully write in described meta data file, then described second metadata is left in writewait queue, and re-execute the step described second metadata write in described meta data file.
6. realize a device for copy-on-write file system, it is characterized in that, comprising:
Find module, for finding the first metadata with dirty data mark in internal memory;
Set module, for identifying set by the lock of described first metadata;
Distribution module, distributes new disk space for the dirty data corresponding for described first metadata;
Lower brush module, for brushing to described new disk space under described dirty data;
First metadata writing module, for writing in the meta data file adjacent with described new disk space position by described first metadata;
Second metadata writing module, for writing second metadata corresponding with described dirty data in blockmap file in described meta data file.
7. device according to claim 6, is characterized in that, described searching module, comprising:
The metadata chained list having described dirty data is found in internal memory; And/or, there is no described dirty data but metadata itself amendment chained list on metadata.
8. device according to claim 7, is characterized in that, described distribution module, comprising:
In advance initial metadata in blockmap file is write in described meta data file, and distribute new disk space for described dirty data.
9. device according to claim 8, is characterized in that, described second metadata writing module, comprising:
The fs place value of data block corresponding for described dirty data is copied to the update position in described blockmap file.
10. device according to claim 9, it is characterized in that, if described second metadata does not successfully write in described meta data file by described second metadata writing module, then described second metadata is left in writewait queue, and again described second metadata is write in described meta data file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510607910.0A CN105094711B (en) | 2015-09-22 | 2015-09-22 | A kind of method and device for realizing copy-on-write file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510607910.0A CN105094711B (en) | 2015-09-22 | 2015-09-22 | A kind of method and device for realizing copy-on-write file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105094711A true CN105094711A (en) | 2015-11-25 |
| CN105094711B CN105094711B (en) | 2018-05-18 |
Family
ID=54575271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510607910.0A Active CN105094711B (en) | 2015-09-22 | 2015-09-22 | A kind of method and device for realizing copy-on-write file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105094711B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106959819A (en) * | 2016-01-08 | 2017-07-18 | 广州市动景计算机科技有限公司 | Date storage method, device and system and portable electric appts |
| CN109240993A (en) * | 2018-07-24 | 2019-01-18 | 郑州云海信息技术有限公司 | Metadata management method and storage server |
| CN109388346A (en) * | 2018-10-15 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of data rule method and relevant apparatus |
| CN109783023A (en) * | 2019-01-04 | 2019-05-21 | 平安科技（深圳）有限公司 | The method and relevant apparatus brushed under a kind of data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7406484B1 (en) * | 2000-09-12 | 2008-07-29 | Tbrix, Inc. | Storage allocation in a distributed segmented file system |
| CN101706756A (en) * | 2009-12-10 | 2010-05-12 | 金蝶软件（中国）有限公司 | Method and device for caching metadata |
| CN102024016A (en) * | 2010-11-04 | 2011-04-20 | 天津曙光计算机产业有限公司 | Rapid data restoration method for distributed file system (DFS) |
| CN102521147A (en) * | 2011-11-17 | 2012-06-27 | 曙光信息产业(北京)有限公司 | Management method by using rapid non-volatile medium as cache |
| CN103885728A (en) * | 2014-04-04 | 2014-06-25 | 华中科技大学 | Magnetic disk cache system based on solid-state disk |
-
2015
- 2015-09-22 CN CN201510607910.0A patent/CN105094711B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7406484B1 (en) * | 2000-09-12 | 2008-07-29 | Tbrix, Inc. | Storage allocation in a distributed segmented file system |
| CN101706756A (en) * | 2009-12-10 | 2010-05-12 | 金蝶软件（中国）有限公司 | Method and device for caching metadata |
| CN102024016A (en) * | 2010-11-04 | 2011-04-20 | 天津曙光计算机产业有限公司 | Rapid data restoration method for distributed file system (DFS) |
| CN102521147A (en) * | 2011-11-17 | 2012-06-27 | 曙光信息产业(北京)有限公司 | Management method by using rapid non-volatile medium as cache |
| CN103885728A (en) * | 2014-04-04 | 2014-06-25 | 华中科技大学 | Magnetic disk cache system based on solid-state disk |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106959819A (en) * | 2016-01-08 | 2017-07-18 | 广州市动景计算机科技有限公司 | Date storage method, device and system and portable electric appts |
| CN106959819B (en) * | 2016-01-08 | 2019-12-20 | 广州市动景计算机科技有限公司 | Data storage method, device and system and portable electronic equipment |
| CN109240993A (en) * | 2018-07-24 | 2019-01-18 | 郑州云海信息技术有限公司 | Metadata management method and storage server |
| CN109388346A (en) * | 2018-10-15 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of data rule method and relevant apparatus |
| CN109388346B (en) * | 2018-10-15 | 2022-02-18 | 郑州云海信息技术有限公司 | Data dropping method and related device |
| CN109783023A (en) * | 2019-01-04 | 2019-05-21 | 平安科技（深圳）有限公司 | The method and relevant apparatus brushed under a kind of data |
| CN109783023B (en) * | 2019-01-04 | 2024-06-07 | 平安科技（深圳）有限公司 | Method and related device for data scrubbing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105094711B (en) | 2018-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9563636B2 (en) | Allowing writes to complete without obtaining a write lock to a file | |
| US8880488B1 (en) | Efficient extent-based B-tree insertion | |
| WO2016086819A1 (en) | Method and apparatus for writing data into shingled magnetic record smr hard disk | |
| CN103744936A (en) | Multi-version concurrency control method in database and database system | |
| CN102831069B (en) | Internal memory processing method, memory management equipment | |
| CN104881371A (en) | Persistent internal memory transaction processing cache management method and device | |
| CN103544045A (en) | HDFS-based virtual machine image storage system and construction method thereof | |
| CN101819543A (en) | Snapshot data processing method, system and storage device | |
| CN105094711A (en) | Method and device for achieving copy-on-write file system | |
| CN104021145A (en) | Mixed service concurrent access method and device | |
| CN107665219B (en) | Log management method and device | |
| CN101122924A (en) | Logic log generation method, database backup/ restoration method and system | |
| US20120066182A1 (en) | Inode management in redirect-on-write file system | |
| US9684664B2 (en) | Elimination of fragmentation of files in storage medium by utilizing head movement time | |
| CN103996412A (en) | Power-fail protection method applied to intelligent-card nonvolatile memories | |
| CN106104515A (en) | Utilize File system design and the fault recovery method of nonvolatile memory | |
| JP2016504700A (en) | Method and system for object-based transactions in a storage system | |
| CN105138284A (en) | Virtual machine disk mirror image synchronization operation optimization system and method | |
| Son et al. | SSD-assisted backup and recovery for database systems | |
| CN104461384A (en) | Data write-in method and storage device | |
| CN104408041A (en) | A method for storing GPS data | |
| CN102024051B (en) | Data updating method for distributed memory database | |
| US20120311021A1 (en) | Processing method of transaction-based system | |
| CN108431784A (en) | Access request processing method, device and computer system | |
| US9176675B1 (en) | Fast-zeroing in a file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |