CN103927315A - Method and device for linking plurality of unit files - Google Patents
Method and device for linking plurality of unit files Download PDFInfo
- Publication number
- CN103927315A CN103927315A CN201310021255.1A CN201310021255A CN103927315A CN 103927315 A CN103927315 A CN 103927315A CN 201310021255 A CN201310021255 A CN 201310021255A CN 103927315 A CN103927315 A CN 103927315A
- Authority
- CN
- China
- Prior art keywords
- files
- file
- symbolic link
- unit
- many symbolic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for linking a plurality of unit files in a file system under a Linux operating system kernel. The method comprises the steps of: firstly creating a multiple-symbol link file, and then writing pointers of the plurality of unit files into the multiple-symbol link file sequentially. According to the method and the device, each unit file can be linked quickly, each unit file and combined file can exist and be used simultaneously, the situation that the size of combined file exceeds limit of the file system is avoided, and each unit file can belong to different file systems. An MSL file can be accessed by a process using a standard file system programming interface.
Description
Technical field
The present invention relates to areas of information technology, more specifically, the present invention relates to improve the file system of (SuSE) Linux OS kernel for example for connecting the method and apparatus of a plurality of unit files.
Background technology
In telecommunication apparatus real-time parallel computing environment, connect rapidly a plurality of files that produced by concurrent process, be requirement and the difficulty that will often face.For example, telecommunication apparatus often needs very rapidly the journal file that has been produced or produced by different processes is connected into Single document, no matter will connect how many files or file size that each will connect will be much, conventionally requires to be shorter than several seconds.The file connecting is called unit file.The file that connection unit file obtains is called merged file.The big or small sum of unit file even may exceed the upper limit of a file system (for example Linux file system).And unit file may belong to dissimilar file system.
Current, the unique method that connects unit file under Linux operation is that the content of unit file is read in user's space, then unit file is written in a merged file according to priority.If unit file scale is too large, for example there is the byte of several G, due to a large amount of I/O of needs (I/O) operation, the performance that unit file is merged to merged file is non-constant, for example, conventionally need to spend the time of several hours so.In addition, if the big or small sum of unit file surpasses the upper limit of Linux file system, can not merge so.
After merging, process may continue to upgrade the content of corresponding unit file.This means, merged file and unit file must be present in file system simultaneously, and can use independently.Use existing method after completing merging, if there is a unit file to upgrade, need so all unit files to re-start once and merge, need to repeat identical process slowly.
Summary of the invention
According to a first aspect of the invention, propose a kind of method that connects a plurality of unit files, having comprised: created symbolic link file more than; And the pointer of a plurality of unit files is written in described many symbolic link files according to priority to the path of a unit file of one of them pointed.
According to a second aspect of the invention, proposed a kind of device that connects a plurality of unit files, having comprised: creating unit, for creating symbolic link file more than; And writing unit, for the pointer of a plurality of unit files being written to according to priority to described many symbolic link files, the path of a unit file of one of them pointed.
Once it is complete that symbolic link file more than creates, application program can be used the standard file system DLL (dynamic link library) of operating system to visit the content of many symbolic link files as ordinary file.The operating system nucleus file system that finally all can be modified the read-write of many symbolic link files is mapped on the unit file being linked.
According to the present invention, can connect rapidly unit file, unit file and merged file can exist simultaneously and can use simultaneously, avoid occurring that the size of merged file exceeds the appearance of this situation of restriction of file system, and unit file can belong to different file system.
For example, 10 unit files that have 50GB size in order to connect each, according to the present invention, only needed just can complete for 1 second, and according to prior art recited above, need to spend several hours.Application program is opened after these many symbolic link files, can read and write 10 unit files that are linked by the DLL (dynamic link library) of standard.
Accompanying drawing explanation
By below in conjunction with the description of the drawings, and along with understanding more comprehensively of the present invention, other objects of the present invention and effect will become and more know and easy to understand, wherein:
Fig. 1 shows according to the method, device of a plurality of unit files of connection of an embodiment of the invention;
Fig. 2 schematically shows the MSL file of establishment;
Fig. 3 schematically shows the unit file sequential list that a MSL file has.
In all above-mentioned accompanying drawings, identical label represents to have identical, similar or corresponding feature or function.
Embodiment
Central idea of the present invention is not in the content of access phase reading unit file, thereby avoids a large amount of I/O operations.
As everyone knows, a symbolic link file only includes a text-string (pointer), and it is interpreted as a path of pointing to another file destination or catalogue by operating system.It is a unique file, and its existence does not also rely on file destination.If delete a symbolic link file, the file destination that it points to is unaffected.If file destination is moved, rename or deletion, the symbolic link file of any this file destination of sensing still exists, but they will point to a file not existed.
In the present invention, introduce a kind of new file type, be called many Symbolic Links (MSL) file.It is similar to existing symbolic link file, and difference is, MSL file is supported a plurality of tactic pointers, and existing symbolic link file is only supported single pointer.
The same with the file of other types, each MSL file has corresponding index node (inode), and the inside has comprised some information relevant with this MSL file, for example, can comprise:
* the byte number of this MSL file;
* this MSL file owner's user ID;
* the group ID of this MSL file;
* the reading and writing of this MSL file or carry out authority;
* the timestamp of this MSL file, has three: ctime refers to the time of the last change of corresponding inode, and mtime refers to the time of the last change of content of this MSL file, and atime refers to the time that this MSL file last time opens;
* link number, have how many filenames to point to this inode;
* this MSL file position in data block.
As everyone knows, symbolic link file exists with the file that it links simultaneously, and therefore, many symbolic link files are also simultaneous with its unit file linking.
More particularly, in many symbolic link files, the unit file linking can belong to different file system, because in many symbolic link files, and the just path of unit file of storage.
Fig. 1 shows according to the method, device of a plurality of unit files of connection of an embodiment of the invention.
As shown in Figure 1, the method/device 100 comprises foundation step/creating unit 110, for creating symbolic link file more than; And write step/writing unit 120, for the pointer of a plurality of unit files being written to according to priority to described many symbolic link files, the path of a unit file of one of them pointed.
When creating a MSL file, can specify the path of corresponding units file, and generate the inode that represents this MSL file in file system.
Fig. 2 schematically shows the MSL file of establishment.As shown in Figure 2, the MSL file of this establishment comprises three paths (pointer) "/d11/f11 ", "/d22/f22 ", "/d33/f33 ", points to respectively unit file 1, unit file 2, unit file 3.
The pointer of corresponding units file is written in the index node inode or data block of many symbolic link files (for example, in the situation that inode does not have enough spaces) according to priority.Than the particular content of unit file, the size of the pointer in the path of indicating member file is almost negligible, therefore, according to the present invention, can avoid occurring that the size of merged file exceeds the appearance of this situation of restriction of file system completely.In addition, due to this feature of pointer, be written in symbolic link file and also substantially do not need spended time, that is to say, can connect rapidly unit file.
File with existing other types, for example regular file (text, video file, image file etc.), block device file, character device file, first-in first-out FIFO file, external member word file, symbolic link file etc. are the same, and MSL file can support to open, close, read, write, lock, locate etc. system call.These system calls will be mapped to the operation of corresponding unit file.To specifically describe these system calls below.
MSL file has two types, is respectively file type MSL file and Directory Type MSL file.The in the situation that of file type MSL file, the unit file linking can be regular file (text, video file, image file etc.), block device file, character device file, first-in first-out FIFO file, external member word file, symbolic link file or many symbolic link files.But the in the situation that of file type MSL file, catalogue can not be directly or indirectly as unit file.The in the situation that of Directory Type MSL file, the unit file linking can be regular file (text, video file, image file etc.), conventional catalogue, symbolic link file, many symbolic link files or the many symbolic link files of other catalogues.
In addition, the unit file of many symbolic link files is not linked to described many symbolic link files itself directly or indirectly.
For many symbolic link files of file type, can carry out one or more in following operation:
*the pointer of extra one or more unit files is added in described many symbolic link files.After creating a MSL file, can dynamically add to the MSL file of this establishment one or more extra unit files.This operation can be specified the path of the unit file of the end of the path list in the MSL file to creating or middle part interpolation.
*from described many symbolic link files, delete the pointer of one or more unit files.After creating a MSL file, can from the MSL file of this establishment, delete one or more unit files.The path of deleting corresponding unit file in the path list of this operation from the MSL file creating, but delete cells file itself not.
*the order of the pointer that changes described unit file in described many symbolic link files.This operation can complete by combining the operation of the extra unit file of foregoing delete cells file and interpolation.
*delete described many symbolic link files.If do not need this many symbolic link files, can delete this many symbolic link files.Delete this many symbolic link files, mean the inode that deletes these many symbolic link files.It should be understood, however, that and delete these many symbolic link files, and do not mean that and also delete the unit file that it links.
*change the title of described many symbolic link files.The same with the file of any other type, change the filename of a MSL file, only can revise the information in the inode of this MSL file.
*change the attribute of described many symbolic link files.The same with the file of any other type, change the attribute of a MSL file, timestamp for example, only can revise the information in the inode of this MSL file.
*open described many symbolic link files.After creating a MSL file, process can be opened this MSL file.When carrying out opening operation, operating system will be called the deployment method of all unit files.If opening operation success, so will be to user's space backspace file descriptor.
*the position of described many symbolic link files that open location.After a MSL File Open, process can be adjusted the position of the file pointer of this MSL file.Operating system, by according to the order of unit file, is carried out calculating location.For example, if process wishes MSL file pointer to move forward from beginning 12345 bytes, suppose that first module file has the length of 8000 bytes, and second unit file has the length of 5000 bytes, pass through so positioning action, actual file pointer will be positioned at 4345 byte places of second unit file.
*described many symbolic link files of opening are carried out to read operation.After a MSL File Open, corresponding process can read the data that start from current file pointer.Read operation can be synchronous or asynchronous.When file pointer arrives the end of a unit file, will resume studies and fetch data from next unit file relaying.
*described many symbolic link files of opening are carried out to write operation.After a MSL File Open, corresponding process can be from current file pointer, data writing.Write operation can be synchronous or asynchronous.When file pointer arrives the end of active cell file, can continue to write data into next unit file, or the end of data being added to this active cell file, depend on the option that is delivered to write operation.
*many symbolic link files and its unit file described in locking or release.Because the unit file of MSL file and its link can be used simultaneously, therefore need to support the Concurrency Access to MSL file and its unit file.This can realize by file locking.Wherein file locking has two features.A feature is lock-in range, and it can be file-level or region (that is to say a part for file) level, and another feature is lockType, it can be shared _ and reads _ locks or exclusive _ write _ lock.When a unit file is locked in file/region class, the MSL file that is linked to so this unit file is considered to be locked in region class.When a MSL file is locked to file-level, its all unit files that link are locked to file-level so.When a MSL file is locked to region class, a part for its unit file linking is locked to region class so.In addition, lockType mapping pellucidly between MSL file and its unit file.
*close described many symbolic link files of opening.Close the operation of a MSL file of opening by being mapped to the shutoff operation of all unit files, close all unit files.
*refresh the data in disk buffering.A MSL file does not directly have any file content, so disk buffering always belongs to unit file.When a MSL file of opening receives flush buffers instruction, operating system will correspondingly be called the flush buffers operation of all unit files.
For many symbolic link files of Directory Type, can carry out one or more in following operation:
*the pointer of extra one or more unit files is joined to described many symbolic link files;
*from described many symbolic link files, delete the pointer of one or more unit files;
*the order of the pointer that changes described a plurality of unit files in described many symbolic link files;
*delete described many symbolic link files;
*change the title of described many symbolic link files;
*change the attribute of described many symbolic link files;
(all these operations are similar with the respective operations of the above-described MSL file for file type, and difference is, in the situation that the MSL file of Directory Type, catalogue can be used as unit file)
*obtain file and the directory listing of described many symbolic link files; And
*accessed path.
The file and the directory listing operation that wherein obtain described many symbolic link files will be returned to the list of unit file, and wherein the order of unit file is identical with the order of the unit file defining in MSL file.The conventional catalogue comprising in catalog unit file, the symbolic link file that is linked to catalogue and other catalogue MSL file, will be considered to the sub-directory of this catalogue MSL file.
In file system, the most important effect of catalogue is accessed path.If comprise in path and comprise catalogue MSL file, some rules need to change so.When in route searching process, while obtaining the inode of catalogue MSL file, operating system will sequentially check each pointer comprising in catalogue MSL file.
For locating file name, operating system sequentially checks the unit file in MSL catalogue.For the unit file of file type, check whether the filename of this unit file mates; For the unit file of Directory Type, check the All Files name under the unit file of this Directory Type, until find.If searched filename has a plurality of files of the same name under MSL catalogue, what can indicate needs searching is which is individual.If do not specified, acquiescence is to search first coupling name.
In order to search catalogue, operating system sequentially checks the unit file of Directory Type.
Operating system will, by using depth-first algorithm, be carried out searching route tree, until find coupling.If all branch roads all can not find coupling, operating system will be indicated " can not find " message so.
Illustrate below.Suppose and find path "/d1/d2/f1 ", wherein d1 is conventional catalogue, and d2 is MSL catalogue, and the filename that f1 is searched.First operating system obtain the inode of root directory, then checks the inode that whether has d1 and obtain d1.Because d1 is conventional catalogue, operating system checks its content and finds that d2 exists, and therefore obtains the inode of d2.Because d2 is Directory Type MSL file, operating system will sequentially be searched for the pointer of its unit file.Suppose that d2 has unit file sequential list as shown in Figure 3, f7 and f8 are that regular file and d4 and d6 are conventional catalogue.First operating system check that whether the title of the f7 being linked by f2 is identical with f1.If identical, search finishes so.If not identical, search for so the file under the catalogue d4 being linked by d3.If find file f 1 under catalogue d4, search finishes so.If do not find file f 1, check that so whether the title of the f8 being linked by f3 is identical with f1.If identical, search finishes so.If not identical, search for so the file under the catalogue d6 being linked by d5.If find file f 1 under catalogue d6, search finishes so.If do not find file f 1, operating system is returned to " file does not find " message so.
It should be noted that for the present invention is easier to understand, description has above been omitted to be known for a person skilled in the art and may to be essential some ins and outs more specifically for realization of the present invention.
Those skilled in the art it is also understood that and the invention is not restricted to step described above, and the present invention also comprises combination that step described above is carried out, order conversion etc.Final scope of the present invention is defined by the appended claims.
Therefore; selecting and describing embodiment is in order to explain better principle of the present invention and practical application thereof; and those of ordinary skills are understood, do not departing under the prerequisite of essence of the present invention, within all modifications and change all fall into protection scope of the present invention defined by the claims.
Claims (14)
1. connect a method for a plurality of unit files, comprising:
Create symbolic link file more than; And
The pointer of a plurality of unit files is written in described many symbolic link files according to priority to the path of a unit file of one of them pointed.
2. method according to claim 1, is wherein written to the pointer of described a plurality of unit files in the index node inode or data block of described many symbolic link files according to priority.
3. method according to claim 1, wherein said many symbolic link files are the many symbolic link files of file type, described a plurality of unit files are file type unit files and are not linked to directly or indirectly catalogue.
4. method according to claim 3, for described many symbolic link files, can carry out one or more in following operation:
The pointer of extra one or more unit files is written to described many symbolic link files;
From described many symbolic link files, delete the pointer of the one or more unit files in described a plurality of unit file;
The order of the pointer that changes described a plurality of unit files in described many symbolic link files;
Delete described many symbolic link files;
Change the title of described many symbolic link files;
Change the attribute of described many symbolic link files;
Open described many symbolic link files;
The position of described many symbolic link files that open location;
Described many symbolic link files of opening are carried out to read operation;
Described many symbolic link files of opening are carried out to write operation;
Many symbolic link files and its unit file described in locking or release;
Close described many symbolic link files of opening; And
Refresh the data in disk buffering.
5. method according to claim 1, wherein said many symbolic link files are the many symbolic link files of Directory Type, any one unit file in described a plurality of unit files can be file type unit file or Directory Type unit file.
6. method according to claim 5, for described many symbolic link files, can carry out one or more in following operation:
The pointer of extra one or more unit files is written to described many symbolic link files;
From described many symbolic link files, delete the pointer of the one or more unit files in described a plurality of unit file;
The order of the pointer that changes described a plurality of unit files in described many symbolic link files;
Delete described many symbolic link files;
Change the title of described many symbolic link files;
Change the attribute of described many symbolic link files;
Obtain file and the directory listing of described many symbolic link files; And
Accessed path.
7. according to the method described in any in claim 1-6, the unit file of wherein said many symbolic link files is not linked to described many symbolic link files itself directly or indirectly.
8. connect a device for a plurality of unit files, comprising:
Creating unit, for creating symbolic link file more than; And
Writing unit, for being written to the pointer of a plurality of unit files according to priority described many symbolic link files, the path of a unit file of one of them pointed.
9. device according to claim 7, wherein said writing unit is written to the pointer of described a plurality of unit files in the index node inode or data block of described many symbolic link files according to priority.
10. device according to claim 7, wherein said many symbolic link files are the many symbolic link files of file type, described a plurality of unit files are file type unit files and are not linked to directly or indirectly catalogue.
11. devices according to claim 10, for described many symbolic link files, can carry out one or more in following operation:
The pointer of extra one or more unit files is written to described many symbolic link files;
From described many symbolic link files, delete the pointer of the one or more unit files in described a plurality of unit file;
The order of the pointer that changes described a plurality of unit files in described many symbolic link files;
Delete described many symbolic link files;
Change the title of described many symbolic link files;
Change the attribute of described many symbolic link files;
Open described many symbolic link files;
The position of described many symbolic link files that open location;
Described many symbolic link files of opening are carried out to read operation;
Described many symbolic link files of opening are carried out to write operation;
Many symbolic link files and its unit file described in locking or release;
Close described many symbolic link files of opening; And
Refresh the data in disk buffering.
12. devices according to claim 7, wherein said many symbolic link files are the many symbolic link files of Directory Type, any one unit file in described a plurality of unit files can be file type unit file or Directory Type unit file.
13. devices according to claim 12, for described many symbolic link files, can carry out one or more in following operation:
The pointer of extra one or more unit files is written to described many symbolic link files;
From described many symbolic link files, delete the pointer of the one or more unit files in described a plurality of unit file;
The order of the pointer that changes described a plurality of unit files in described many symbolic link files;
Delete described many symbolic link files;
Change the title of described many symbolic link files;
Change the attribute of described many symbolic link files;
Obtain file and the directory listing of described many symbolic link files; And
Accessed path.
Device in 14. according to Claim 8-13 described in any, the unit file of wherein said many symbolic link files is not linked to described many symbolic link files itself directly or indirectly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310021255.1A CN103927315A (en) | 2013-01-11 | 2013-01-11 | Method and device for linking plurality of unit files |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310021255.1A CN103927315A (en) | 2013-01-11 | 2013-01-11 | Method and device for linking plurality of unit files |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103927315A true CN103927315A (en) | 2014-07-16 |
Family
ID=51145537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310021255.1A Pending CN103927315A (en) | 2013-01-11 | 2013-01-11 | Method and device for linking plurality of unit files |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103927315A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6711630B2 (en) * | 2001-05-22 | 2004-03-23 | Intel Corporation | Method and apparatus for communicating with plug and play devices |
| KR20050015157A (en) * | 2003-08-04 | 2005-02-21 | 엘지전자 주식회사 | Optical disc and recording method of File Information of optical disc |
| CN101000616A (en) * | 2006-01-11 | 2007-07-18 | 三星电子株式会社 | Method of creating symbolic link, and method and apparatus for accessing file or directory by using symbolic link |
| CN102158546A (en) * | 2011-02-28 | 2011-08-17 | 中国科学院计算技术研究所 | Cluster file system and file service method thereof |
-
2013
- 2013-01-11 CN CN201310021255.1A patent/CN103927315A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6711630B2 (en) * | 2001-05-22 | 2004-03-23 | Intel Corporation | Method and apparatus for communicating with plug and play devices |
| KR20050015157A (en) * | 2003-08-04 | 2005-02-21 | 엘지전자 주식회사 | Optical disc and recording method of File Information of optical disc |
| CN101000616A (en) * | 2006-01-11 | 2007-07-18 | 三星电子株式会社 | Method of creating symbolic link, and method and apparatus for accessing file or directory by using symbolic link |
| CN102158546A (en) * | 2011-02-28 | 2011-08-17 | 中国科学院计算技术研究所 | Cluster file system and file service method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 李清: "基于数据库技术的文件系统XDBFS的设计与实现", 《中国优秀硕士学位论文全文数据库》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109117425B (en) | Method, system, and medium for digital asset synchronization | |
| EP4270209A2 (en) | Deduplicated merged indexed object storage file system | |
| US5261088A (en) | Managing locality in space reuse in a shadow written B-tree via interior node free space list | |
| EP2754027B1 (en) | Method for creating clone file, and file system adopting the same | |
| CN104714755B (en) | Snapshot management method and device | |
| US8548957B2 (en) | Method and system for recovering missing information at a computing device using a distributed virtual file system | |
| US8762344B2 (en) | Method for managing information processing system and data management computer system | |
| CN101446984B (en) | Method and device for storing files and method and device for deleting files | |
| US20170124345A1 (en) | Reducing Resource Consumption Associated with Storage and Operation of Containers | |
| US8290996B2 (en) | Data sharing for file clones using reverse ditto references | |
| US8694497B2 (en) | Method, system, and computer program product for enabling file system tagging by applications | |
| US8495022B1 (en) | Systems and methods for synthetic backups | |
| US20170068469A1 (en) | Remote Shared Virtual Disk Snapshot Creation | |
| US20150106335A1 (en) | Hierarchical data archiving | |
| CN103020315A (en) | Method for storing mass of small files on basis of master-slave distributed file system | |
| KR20060095448A (en) | File system represented inside a database | |
| US20090254585A1 (en) | Method for Associating Administrative Policies with User-Definable Groups of Files | |
| CN103282899A (en) | File system data storage method and access method and device therefor | |
| CN112800019A (en) | Data backup method and system based on Hadoop distributed file system | |
| US11687533B2 (en) | Centralized storage for search servers | |
| CN105653731A (en) | Method for restoring deleted data of journaling file system | |
| EP2669806B1 (en) | Storage system | |
| CN110622147A (en) | Metadata storage of placeholders in a storage virtualization system | |
| US11010408B2 (en) | Hydration of a hierarchy of dehydrated files | |
| US8612717B2 (en) | Storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140716 |
|
| WD01 | Invention patent application deemed withdrawn after publication |