CN100498781C - Method for storing metadata of logic document system by adhesion property - Google Patents
Method for storing metadata of logic document system by adhesion property Download PDFInfo
- Publication number
- CN100498781C CN100498781C CNB200610089249XA CN200610089249A CN100498781C CN 100498781 C CN100498781 C CN 100498781C CN B200610089249X A CNB200610089249X A CN B200610089249XA CN 200610089249 A CN200610089249 A CN 200610089249A CN 100498781 C CN100498781 C CN 100498781C
- Authority
- CN
- China
- Prior art keywords
- file
- file system
- catalogue
- local
- logical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method to use adhering attribute storing logic file system metadata. It stores the metadata of logic file system on local file system as local file system file attribute. The method includes the following steps: creating file or catalog of the logic file system; deleting a file or catalog; modifying logic file system attribute; renaming the file or catalog on logic file system; acquiring the metadata of file or catalog on logic file system. By using the method, the logic file system could directly use the metadata process function by using the local file system to inhere the stability and performance of the local file system. It also has the advantages of easy to realize and strong adaptability.
Description
Technical field
The present invention relates to the storage of metadata, particularly the storage means of metadata in the logical file system.
Background technology
Along with the development of computing application, growing data storage requirement and diversified data storage access module have brought great challenge to Design of File Systems.File server is as a kind of centralized stores heap file data computing machine, and its file system has the specific (special) requirements that is different from general computer system, and file server generally has high performance processor, a large amount of internal memories and a large amount of disks.In order to adapt to the demand of file server, those skilled in the art has designed a kind of based on the logical file system that makes up on a plurality of local file systems.Accompanying drawing 1 is the structural drawing of a typical logical file system, in this logical file system, data storage is in memory device, logical file system is the file system that is based upon on the local file system, logical file system can not DASD, need finish the storage and the visit of data and metadata by local file system.
Metadata is " about the data of data ", common metadata comprises: filename (dentry-〉d_namc), the I node number (inode-〉i_ino), file mode (inode-〉i_mode), the file linking number (inode-〉i_nlink), the user ID of file (inode-〉i_uid), user's group id of file (inode-〉i_gid), the access time of file (inode-〉i_atime), the file modifying time (inode-〉i_mtime), the creation-time of file (inode-〉i_ctime), the figure place of file data block size (inode-〉i_blkbits), the size of the data block of file (i_blksize), FileVersion (inode-〉i_version), the generation of file (inode-〉i_generation).The researchist finds that in great majority were used at present, the quantity of metadata operation was far longer than common file read-write quantity.An outstanding logical file system must have quite outstanding metadata processing power.
In some distributed file systems, special meta data server is arranged, metadata is taken as data block in these server inside and handles, and promptly the metadata information of file is direct coding and is stored in meta data server inside with certain form.The major defect of this type of way is the operation more complicated, needs to handle all relevant with metadata operation and storage, is not easy to realize, also is difficult to reach reasonable performance.
In existing local file system, there are some file system to use the storage of file under the mode organization directory of balanced binary tree or B+ tree specially, its metadata processing power is quite outstanding.If can make full use of the metadata processing power of local file system, then storage and the processing to metadata in the file system has great significance.
Summary of the invention
The objective of the invention is to overcome the operation more complicated of existing meta data server to metadata, need to handle all relevant operation and storage with metadata, be not easy to realize, also be difficult to reach relatively good performance defect, thereby improve a kind of method of efficient storage logical file system metadata.
To achieve these goals, the present invention has improved a kind of method with storing metadata of logic document system by adhesion property, the mode of the metadata of the logical file system file system interface by a Virtual File System layer with the local file system file attribute is stored on the local file system, comprises following operation:
When 1), creating the file of logical file system or catalogue, the parent directory information at the file that will create or catalogue place is taken out, a new file or catalogue are created with same filename or directory name in parent directory the inside at the local file system of correspondence, in the index node of the file of the new local file system of establishment or catalogue and index node and directory entry that the data item in the directory entry directly copies logical file system to;
When 2), deleting a file or catalogue, the index node of the parent directory by the file that will delete in the logical file system or catalogue place finds the index node of the parent directory of corresponding file in the local file system or catalogue, then in the parent directory the inside deletion corresponding file or the catalogue of the local file system of correspondence, after deleting successfully, the information of the parent directory of deleted file or catalogue in the renewal logical file system;
When 3), revising the logical file system attribute, mask according to the attribute that will revise, revise the attribute of local file system file inode correspondence, call the function of local file system enactment document attribute then, the index node of amended local file system file is stored on the disk of local file system;
4), when the file on the logical file system or catalogue are renamed, the file that will rename by logical file system or the directory entry of catalogue obtain the directory entry of local file system corresponding document or catalogue, obtain the index node of the parent directory of local file system respective file or catalogue by will the rename index node of parent directory of file or catalogue of logical file system, call the function that renames of local file system then, change the filename or the directory name of local file system, at last the directory entry of the logical file system of rename back file or catalogue and the directory entry of local file system are associated;
5), when the metadata information of file on the query logic file system or catalogue, at first find the parent directory of corresponding local file system file or catalogue according to the parent directory at the file of the logical file system that will inquire about or catalogue place, and then according to the filename that will inquire about inquiry file or catalogue in the local file system parent directory, after finding corresponding file or catalogue, the index node of local file system file or catalogue and directory entry information are directly copied on the index node and directory entry data structure of logical file system respective file.
In the technique scheme, in described index node, the metadata that is comprised comprises filename, the I node number, file mode, file linking number, the user ID of file, user's group id of file, the access time of file, the file modifying time, the creation-time of file, the figure place of file data block size, the size of the data block of file, FileVersion, the generation of file.
In the technique scheme, in described directory entry, the metadata that is comprised comprises filename and file path.
In the technique scheme, in described step 3), the function of described local file system enactment document attribute is realized by the interface setattr of Virtual File System layer.
In the technique scheme, in described step 4), the described function that renames is the rename function of local file system.
Utilize method of the present invention, logical file system can directly utilize the metadata processing capacity of local file system, inherits the stability and the performance characteristic of local file system, and this method have realize easy, adaptable advantage.
Description of drawings
Fig. 1 is the structural drawing of logical file system;
Fig. 2 is index node in the logical file system and the relation between directory entry and local file system index node and the directory entry;
Fig. 3 is that the attribute of logical file system I node is attached to the synoptic diagram on the attribute of local file system I node.
Embodiment
Below in conjunction with the drawings and specific embodiments method of the present invention is described.
In the present embodiment, described adhesion property is meant that the metadata of utilizing local file system comes the metadata of stored logic file system, and the metadata of logical file system is not to exist as the data block of a file the inside of local file system.Therefore, the metadata of file in the logical file system or catalogue is operated, need be forwarded to such operation to carrying out on local file system corresponding file or the catalogue with same parameter.Such forwarding operation is to realize by the file system interface of realizing a Virtual File System (VFS) layer.Typical metadata operation comprises establishment, deletion, revises attribute, renames and inquire about.Respectively the aforesaid operations of metadata is described below.
1, creates: when the file of a logical file system of establishment or catalogue, the parent directory information at the file that will create or catalogue place is taken out, a new file or catalogue are created with same filename or directory name in parent directory the inside at the local file system of correspondence, newly the file of the local file system of establishment or the index node (inode) of catalogue and the partial data item in the directory entry (dentry) directly copy among the inode and dentry of logical file system.Wherein, the information that comprises in the index node comprises filename (dentry-〉d_name), the I node number (inode-〉i_ino), file mode (inode-〉i_mode), the file linking number (inode-〉i_nlink), the user ID of file (inode-〉i_uid), user's group id of file (inode-〉i_gid), the access time of file (inode-〉i_atime), the file modifying time (inode-〉i_mtime), the creation-time of file (inode-〉i_ctime), the figure place of file data block size (inode-〉i_blkbits), the size of the data block of file (i_blksize), FileVersion (inode-〉i_version), the generation of file (inode-〉i_generation), comprised filename in the directory entry, file path information and some pointers of safeguarding the kernel directory tree.During information in duplicating index node and directory entry, the data item that needs in the index node (inode) to duplicate is i_ino, i_mode, i_nlink, i_uid, i_gid, i_atime, i_mtime, i_ctime, i_blkbits, i_blksize, i_versioni_generation, an xcopy name d_name in the directory entry (dentry).The inode of logical file system and local file system and dentry relation are as shown in Figure 2.
For example, under the root directory of logical file system, create the sub-directory of abc by name, then in local file system, find root directory earlier, then the same sub-directory of creating abc by name under the root directory of local file system.Local file system is when creating sub-directory, can create directory index node and directory entry (dentry) about sub-directory abc, data item in directory index node (inode) and the directory entry (dentry) is directly copied among the inode and dentry of logical file system, make the file of logical file system and the file of local file system have identical attribute.Wherein, index node the inside comprises I node number (inode-〉i_ino), file mode (inode-〉i_mode), the file linking number (inode-〉i_nlink), the user ID of file (inode-〉i_uid), user's group id of file (inode-〉i_gid), the access time of file (inode-〉i_atime), the file modifying time (inode-〉i_mtime), the creation-time of file (inode-〉i_ctime), the figure place of file data block size (inode-〉i_blkbits), the size of the data block of file (i_blksize), FileVersion (inode-〉i_version), the generation of file (inode-〉i_generation) all to fill in according to the attribute of local file correspondence, promptly directly copy the value of the attribute of local file correspondence.
2, deletion: when deleting a file or catalogue, the index node of the parent directory by the file that will delete in the logical file system or catalogue place finds the index node of the parent directory of corresponding file in the local file system or catalogue, then in the parent directory the inside deletion corresponding file or the catalogue of the local file system of correspondence, after deleting successfully, upgrade the information of logical file system parent directory.
For example, deletion sub-directory abc under root directory, its operation is as follows:
(1) inode of the parent directory (being the root directory of logical file system) by wanting deleted sub-directory abc in the logical file system finds the inode of the parent directory (being the root directory of local file system) of the sub-directory abc that wants deleted in the local file system;
(2) the directory name abc that will deltree by logical file system finds the directory name abc that local file system will deltree;
(3) call the rmdir system call of local file system, the dentry that deltree local file system parent directory inode and local file system passes to the rmdir interface of local file system, the file on the deletion local file system as parameter.
(4) if delete successfully, then need to upgrade some information of logical file system parent directory (parent directory of deleted catalogue), i_size for example, i_nlink, i_ctime etc.When upgrading, copy corresponding data from the attribute of local file system correspondence.For example, the variable of the inode of logic of propositions file system is called lfs_inode, and the inode variable of local file system is called meta_inode, and so Dui Ying operation is exactly lfs_inode-〉i_size=meta_inode-〉i_size; Lfs_inode-〉i_nlink=meta_inode-〉i_nlink; Lfs_inode-〉i_ctime=meta_inode-〉i_ctime.
3, revise attribute: when revising the logical file system attribute, mask according to the attribute that will revise, revise the attribute of local file system file inode correspondence, call the function of local file system enactment document attribute then, the inode of amended local file system file is stored on the disk of local file system.
In this step, described modification attribute is that the metadata that is kept in the index node (inode) is made amendment, and filename and the file path that is kept in the directory entry (dentry) then do not belonged in the scope of the described modification attribute of this step.The modification of attribute among the inode is realized that by the interface setattr of Virtual File System (VFS) layer the specific implementation step is as follows:
1), obtains the dentry pointer of local file system by the dentry of logical file system;
2), call the setattr interface function of local file system, the metadata information that revise directly transmitted be modification the inode of local file system.
If among the sub-directory abc under root directory, a file A is arranged, its original access rights are readable, its access rights are revised as can write, to being operating as of its:
1), the dentry of the file A by logical file system finds the dentry of file A in the file file system;
2), call the setattr function of local file system, can write the access rights among the inode of file A in the local file system by readable being revised as.
4, rename: when the directory name of the filename of a file on the logical file system or a catalogue is renamed as another one filename or directory name, at first the dentry of file that will rename by logical file system or catalogue obtains the dentry of local file system corresponding document or catalogue, by will the rename inode of parent directory of file or catalogue of logical file system, obtain the inode of the parent directory of local file system respective file or catalogue, call the rename function of local file system then, change the filename or the directory name of local file system, again the dentry of the logical file system of rename back file or catalogue and the dentry of local file system are associated.
For example, the directory name of sub-directory abc under the logical file system root directory is revised as def, obtain the directory name (dentry) of sub-directory abc corresponding in the local file system by the directory name (dentry) of logical file system sub-directory abc, the inode of the root directory (parent directory of sub-directory abc) by logical file system, obtain the inode of local file system root directory, the rename function that calls local file system then is def with the name modifications of sub-directory abc, at last the dentry of the dentry of def and local file system is associated.
Because filename or directory name are not kept in the logical file system, thus as long as in local file system, filename or directory name are made amendment, need be in logical file system revised file name or directory name.
5, inquiry: in the time of the metadata information of file on the query logic file system or catalogue, at first find the parent directory of corresponding local file system file or catalogue according to the parent directory at the logical file system file that will inquire about or catalogue place, and then according to the filename that will inquire about inquiry file or catalogue in the local file system parent directory, after finding corresponding file or catalogue, the inode of local file system file or catalogue and dentry information are directly copied on the inode and dentry data structure of logical file system respective file.
For example, in logical file system, sub-directory abc is positioned under the root directory, in the time will inquiring about sub-directory abc, at first inquires about root directory in local file system, then inquiry sub-directory abc under the root directory of local file system.After finding sub-directory abc, on the inode and dentry data structure that directly copy to the logical file system respective file about the inode and the dentry information of this sub-directory in the local file system.
The size of the file of logical file system is can not be attached directly on the metadata of local file system, because the file of logical file system will be stored actual user data, and the file of local file system only is a ghost of adhering to metadata, and both file sizes are also inequality.So must be the size of logical file system file as the file content data of local file system and store.
Claims (5)
1, a kind of method with storing metadata of logic document system by adhesion property, the mode of the metadata of the logical file system file system interface by a Virtual File System layer with the local file system file attribute is stored on the metadata of local file system, may further comprise the steps:
1), creates the metadata of logical file system: creating file or catalogue simultaneously, create the metadata of described logical file system, the parent directory information at the file that will create or catalogue place is taken out, in the parent directory the inside of the local file system of correspondence with same filename, perhaps directory name is created a new file or catalogue, the file of the new local file system of creating, perhaps the index node of catalogue and the data item in the directory entry directly copy in the index node and directory entry of logical file system;
2), the metadata of deletion logical file system: the metadata of deletion logical file system be deletion in the logical file system file or carry out during catalogue, when deleted file or catalogue, by the file that will delete in the logical file system, the perhaps index node of the parent directory at catalogue place, find the index node of the parent directory of corresponding file in the local file system or catalogue, then in the parent directory the inside deletion corresponding file or the catalogue of the local file system of correspondence, after deleting successfully, the information of the parent directory of deleted file or catalogue in the renewal logical file system;
3), revise the metadata of logical file system: the metadata of revising logical file system is exactly to revise the attribute of logical file system, when revising attribute, mask according to the attribute that will revise, revise the attribute of local file system file inode correspondence, call the function of local file system enactment document attribute then, the index node of amended local file system file is stored on the disk of local file system;
4), file on the logical file system or catalogue are renamed: the file that will rename by logical file system, or the directory entry of catalogue obtains the directory entry of local file system corresponding document or catalogue, by will the rename index node of parent directory of file or catalogue of logical file system, obtain the index node of the parent directory of local file system respective file or catalogue, call the function that renames of local file system then, change the filename or the directory name of local file system, at last the directory entry of the logical file system of rename back file or catalogue and the directory entry of local file system are associated;
5), the file on the query logic file system or the metadata information of catalogue: at first according to the file of the logical file system that will inquire about, perhaps the parent directory at catalogue place finds the parent directory of corresponding local file system file or catalogue, and then according to the filename that will inquire about, inquiry file or catalogue in the local file system parent directory, after finding corresponding file or catalogue, the index node of local file system file or catalogue and directory entry information are directly copied on the index node and directory entry data structure of logical file system respective file.
2, the method with storing metadata of logic document system by adhesion property according to claim 1 is characterized in that, in described index node, the metadata that is comprised comprises filename, I node number, file mode, the file linking number, the user ID of file, user's group id of file, the access time of file, file modifying time, the creation-time of file, the figure place of file data block size, the size of the data block of file, FileVersion, the generation of file.
3, the method with storing metadata of logic document system by adhesion property according to claim 1 is characterized in that in described directory entry, the metadata that is comprised comprises filename and file path.
4, the method with storing metadata of logic document system by adhesion property according to claim 1, it is characterized in that, in described step 3), the function of described local file system enactment document attribute is realized by the interface setattr of Virtual File System layer.
5, the method with storing metadata of logic document system by adhesion property according to claim 1 is characterized in that in described step 4), the described function that renames is the rename function of local file system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB200610089249XA CN100498781C (en) | 2006-08-11 | 2006-08-11 | Method for storing metadata of logic document system by adhesion property |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB200610089249XA CN100498781C (en) | 2006-08-11 | 2006-08-11 | Method for storing metadata of logic document system by adhesion property |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1904889A CN1904889A (en) | 2007-01-31 |
| CN100498781C true CN100498781C (en) | 2009-06-10 |
Family
ID=37674154
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB200610089249XA Expired - Fee Related CN100498781C (en) | 2006-08-11 | 2006-08-11 | Method for storing metadata of logic document system by adhesion property |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100498781C (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100595761C (en) * | 2007-12-29 | 2010-03-24 | 中国科学院计算技术研究所 | Metadata management method for splitting name space |
| US9552363B2 (en) * | 2013-04-30 | 2017-01-24 | Microsoft Technology Licensing, Llc | File management with placeholders |
| US9405767B2 (en) | 2013-05-01 | 2016-08-02 | Microsoft Technology Licensing, Llc | Streaming content and placeholders |
| CN104572762B (en) * | 2013-10-24 | 2019-04-12 | 杭州海康威视数字技术股份有限公司 | The method and apparatus for deleting and restoring video file |
| CN104572059B (en) * | 2013-10-29 | 2020-04-10 | 中兴通讯股份有限公司 | Processing method and device for modifying application name |
| CN104391952A (en) * | 2014-11-28 | 2015-03-04 | 公安部第三研究所 | File system index establishing method and file system query implementing method |
| CN106682003B (en) * | 2015-11-06 | 2019-09-20 | 中国电信股份有限公司 | The path segmentation mapping method and device of distributed storage NameSpace |
| CN105893633A (en) * | 2016-06-06 | 2016-08-24 | 浪潮电子信息产业股份有限公司 | File system real-time synchronization method based on rsync |
| CN106503198A (en) * | 2016-11-02 | 2017-03-15 | 北京集奥聚合科技有限公司 | A kind of cold data recognition methodss and system based on hadoop metadata |
| CN106919347A (en) * | 2017-02-28 | 2017-07-04 | 北京华云网际科技有限公司 | The creation method and device of the catalogue of distributed block storage system |
| CN109918355B (en) * | 2019-03-05 | 2020-12-15 | 杭州前云数据技术有限公司 | Virtual metadata mapping system and method for implementing NAS based on object storage service |
-
2006
- 2006-08-11 CN CNB200610089249XA patent/CN100498781C/en not_active Expired - Fee Related
Non-Patent Citations (4)
| Title |
|---|
| PVFS寄生式元数据管理的设计与实现. 庞丽萍，何飞跃，徐婕，岳建辉.计算机工程，第30卷第20期. 2004 |
| PVFS寄生式元数据管理的设计与实现. 庞丽萍，何飞跃，徐婕，岳建辉.计算机工程，第30卷第20期. 2004 * |
| 高性能集群文件系统的研究. 马俊，杨树军.计算机工程与设计，第27卷第13期. 2006 |
| 高性能集群文件系统的研究. 马俊，杨树军.计算机工程与设计，第27卷第13期. 2006 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1904889A (en) | 2007-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100498781C (en) | Method for storing metadata of logic document system by adhesion property | |
| US8868624B2 (en) | Blob manipulation in an integrated structured storage system | |
| US8620884B2 (en) | Scalable blob storage integrated with scalable structured storage | |
| US20170123931A1 (en) | Object Storage System with a Distributed Namespace and Snapshot and Cloning Features | |
| US7672981B1 (en) | Object classification and indexing of very large name spaces using grid technology | |
| CN100541441C (en) | A kind of snapshot creation of computer file system, deletion and amending method | |
| US7395389B2 (en) | Extending non-volatile storage at a computer system | |
| US8650228B2 (en) | Methods and systems for space management in data de-duplication | |
| AU739236B2 (en) | File system interface to a database | |
| KR101224680B1 (en) | File system represented inside a database | |
| CN101743546B (en) | Hierarchical storage management for a file system providing snapshots | |
| US9069792B1 (en) | Method and system for persistently cached, copy-on-write view of revision control trees | |
| US20080040388A1 (en) | Methods and systems for tracking document lineage | |
| US20060059204A1 (en) | System and method for selectively indexing file system content | |
| US20100138394A1 (en) | Multiple concurrent active file systems | |
| US20100114977A1 (en) | Method, system, and computer program product for enabling file system tagging by applications | |
| JPH07219832A (en) | Outline catalog | |
| CN104081391A (en) | Single instantiation method using file clone and file storage system utilizing the same | |
| CN102012933A (en) | Distributed file system and method for storing data and providing services by utilizing same | |
| US11847028B2 (en) | Efficient export of snapshot changes in a storage system | |
| US20090254585A1 (en) | Method for Associating Administrative Policies with User-Definable Groups of Files | |
| EP4100845A2 (en) | Data management system and method of controlling | |
| Lovelace et al. | VSAM demystified | |
| US7844596B2 (en) | System and method for aiding file searching and file serving by indexing historical filenames and locations | |
| GB2439577A (en) | Storing data in streams of varying size |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090610 Termination date: 20190811 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |