EP3084638A1

EP3084638A1 - Posix-compatible file system, method for producing a file list and memory apparatus

Info

Publication number: EP3084638A1
Application number: EP14783855.1A
Authority: EP
Inventors: Alexander KÖNIG; Christoph König
Original assignee: Fujitsu Technology Solutions Intellectual Property GmbH
Current assignee: Fujitsu Ltd
Priority date: 2013-12-17
Filing date: 2014-10-13
Publication date: 2016-10-26
Also published as: US20160283501A1; DE102013114214A1; JP2016526737A; JP6430499B2; WO2015090668A1

Abstract

The invention relates to a POSIX-compatible file system (2) comprising at least one directory (A) and a plurality of files (a1, a2) stored in the directory (A), wherein each file (a1, a2) has an associated inode (7) with metadata about the respective file (a1, a2), and the directory (A) comprises an association between file names (14) of the plurality of files (a1, a2) and the inode (7) respectively associated with a file (a1, a2). The metadata about the respective file (a1, a2) comprises at least the file names (14) of the associated file (a1, a2) and/or information about the at least one directory (A). The invention additionally relates to a method for producing a file list (10), to the use of extended attributes of a POSIX-compatible file system (2) and to a memory apparatus (50).

Description

description

POSIX compatible file system, method for creating a file list and storage device

The invention relates to a POSIX compatible file system comprising at least one directory and a plurality of files stored in the directory. Moreover, the invention relates to a method for generating a file list with files of a POSIX-compatible file system, a use of extended attributes and a

Storage device.

POSIX compatible file systems are known in the art. In particular, most known Linux distributors are based on different, POSIX-compatible ones

File systems, such as ext2 and ext3.

Basically, such file systems have one

relatively high flexibility and are suitable for storing a large number of files.

POSIX-compatible file systems manage physical

Disk using so-called inodes, wherein the inodes metadata about stored on the disk

Information included. Examples of such metadata are

Access rights, as well as the creation, modification, and reading dates of files stored in the file system.

Such information is used inter alia by backup and archiving solutions for managing the

File system stored files used.

However, the evaluation and updating of such and similar metadata can be particularly extensive File systems, such as those found in so-called "big data" applications, also lead to problems.

It is an object of the present invention to describe a POSIX compliant file system as well as methods and apparatus for its creation and use that has improved over known implementations

Have performance. In particular, a number of I / O accesses to a mass storage are to be reduced when browsing files stored on the mass storage. The devices and methods described are intended especially for use in security and

Archiving systems that are hundreds of thousands up

Manage billions of files.

According to a first aspect of the invention, there is provided a POSIX compatible file system comprising at least one directory and a plurality of ones stored in the directory

Files revealed. Each file is associated with an inode with metadata about the particular file, and the directory comprises an association between filenames of the plurality of files and the inode associated with each file. In this case, the metadata about the respective file at least the file name of the associated file and / or information about the at least one directory.

Such a file system allows the determination of

Filenames and / or paths based solely on metadata stored in the inodes. For example, such a file system allows the metadata to be based on a combination of name and directory information

Inodes to create a file list without having to resort to other data blocks of the file system, in particular directories, must be accessed. The storage of either only directory or name information allows at least pre-filtering of associated file objects, such that fewer fewer data blocks need to be accessed. By locating name and / or directory information in the inodes, I / O access to other parts of a mass storage can be avoided or reduced. Thus, administrative tasks based on corresponding information and metadata of the inodes can be accelerated.

According to a first embodiment, the information about the at least one directory comprises a reference to an inode associated with the directory. By providence of a

Referring to an inode of a parent directory, the path to the file can be built backwards from an inode of a file without a

Directory structure must be established by reading directory objects. According to an alternative embodiment, the file system is embodied as a WORM file system and the information about the at least one directory comprises a path specification of the assigned file from a predetermined reference point, in particular a mountpoint of the WORM file system. When the file system is called a WORM file system (English:

"Write once, read multiple"), so only once

writable file system is formed, after the first storage of a file no further changes in its path, for example, by renaming or moving the file or parent directories arise. In this case, the full path can be stored in the inode associated with the file. In such a system it is through a linear scan through the list of Inodes possible to build a file list with path information to files and associated inodes.

For storing the file name and the information about the at least one directory are in accordance with another

Aspect uses extended attributes of the file system.

According to another aspect of the invention, a method for creating a file list with files of a POSIX compatible file system is described. The method comprises the steps:

Scanning a predetermined group of inodes for

Acquiring metadata of a plurality of files respectively associated with the inodes, the metadata including at least the file name of the respective associated file and / or

Include information about a directory in which the respective file is stored;

- Determining file names and / or paths from

Files based solely on the metadata stored in the inodes; and

- Create a file list based on the specific file names or paths.

The above method allows the creation of a file list exclusively for information contained in inodes of a POSIX-compatible file system. In this way, in particular the multiple back and forth

Jump back of read / write heads of a physical storage device between a first storage area with information of the inodes and a second storage area with other data blocks are avoided, so that the structure of the file list is accelerated as a whole. According to an advantageous embodiment, the

Metadata at least one further attribute, in particular a date of a last backup of the respective associated file, wherein the at least one further attribute is transferred to the file list and / or a transfer of a file in the file list based on the at least one further

Attribute is decided. By combining the

Filenames with additional attributes of the associated file, in particular a date of a last backup, can backup and archiving systems and similar

Software programs based solely on the metadata contained in the inodes decide which files need to be further processed, for example, backed up. According to another aspect of the invention is a

A storage device comprising at least one interface for accessing files stored by the storage device and at least one mass storage system for

nonvolatile storage of the files revealed. In this case, the storage device is set up to receive a

Write command for writing or changing a file via the at least one interface to store metadata about the at least one file in the mass storage system or

to change already stored metadata, wherein the metadata stored on the file at least the file name of the file and / or information about a directory in which the file is stored.

Said storage device stores the necessary information for carrying out the above-mentioned method and for providing a POSIX-compatible file system. According to further embodiments of the invention, the memory device further comprises a software component, wherein the at least one software component thereto

is set up to create a file list based solely on the stored metadata. For example, it may be a file management software component that selects files for further processing based solely on the stored metadata. In particular, a backup of files based on at least one of the metadata stored time, in particular a comparison of an additional stored date of a last backup with a default stored date of a last change of the metadata associated file, be performed.

Further advantageous embodiments and details of the invention are set forth in the appended claims and the following detailed description of

Embodiments disclosed.

The invention will now be described in detail by means of different embodiments with reference to the attached figures. Different instances of identical or equivalent components are marked with alphabetical suffixes. Should be on everyone

Instances are referenced together, that becomes

appropriate suffix omitted. In the figures show:

Figures 1A to IC, a tree view, a

Memory allocation and a list representation of a POSIX compatible file system in general, Figures 2A and 2B, a conventional data structure and a conventional method for creating a file list,

FIGS. 3A and 3B, a data structure and a method according to a first embodiment of the invention,

Figures 4A and 4B, a data structure and a method according to a second embodiment of the invention and Figure 5, a schematic representation of a

Storage device according to an embodiment of the invention.

For a better understanding of the invention, a POSIX-compatible file system will generally be described below with reference to FIGS. 1A to 1C and a conventional file system

Data structure for its implementation and a method for creating a file list with reference to Figures 2A and 2B described. FIG. 1A shows a directory tree 1 of a POSIX-compatible file system 2. For better distinction, capital letters for identifying directories and lowercase letters as well as numbers for characterizing regular files of the file system 2 are used in the further course. Of course, POSIX compliant file systems 2 usually allow the use of uppercase and lowercase letters as well as numbers and other special characters for both directory and file names. Furthermore, clear longer names and the production of clearly more complicated directory trees than the directory tree 1 shown in FIG. 1A are possible. As far as files are referenced in the following description and appended claims, there are regular files in the sense of a POSIX compatible one

File system, that is, in the file system 2

stored user data to understand. Other objects, in particular directory objects, so-called hard and

Softlinks and other file system 2 metadata are explicitly named as such in the following to better distinguish them from regular files.

The directory tree 1 includes a so-called

The root directory 3 is the operating system Unix and related operating systems such as Linux

The directory tree 1 first branches from the root directory 3 into three directories A, B and C of a first hierarchical level The first directory A contains two regular files al in the directory tree 1 according to FIG The second directory B comprises a single file bl The third directory C comprises a further subdirectory D of a second hierarchical level and regular files cl and c2 In the subdirectory D of second order, a further file d1 is stored in the exemplary embodiment according to FIG.

It should be noted at this point that the directory tree 1 shown in FIG. 1A represents a simplified special case involving only a single root file system without so-called mount points and no links between different file and directory objects of the

File system 2 contains. Basically, it is common for POSIX compatible file systems, the content of others

Mass storage with directory trees stored on it predetermined locations, called mountpoints, of a parent file system, usually the

Root file system with the root directory 3, to integrate. Also, through the use of soft or hard links, network-like structures can be created so that multiple paths from the root directory 3 to one and the same regular file can result. Such structures are not shown here for reasons of clarity. Their importance for the implementation of the following

described devices and methods are on

corresponding place listed.

FIG. 1B schematically shows a data structure for mapping the directory tree 1 onto a physical data carrier 4. For example, the physical data carrier 4 is a mass storage device in the form of a data storage device

Hard disk drive or a flash drive.

In principle, however, other storage media are possible. The physical data carrier 4 is in the embodiment in a first memory area 5 and a second

Memory area 6 divided. For example, it may be a division according to block addresses or other

Address information of the physical disk 4 act. Such a division is usually at a first

Initialization of the disk 4, for example, when formatting with the file system 2 made.

In the first memory area 5 so-called inodes 7 are stored in the embodiment. The inodes are POSIX-compatible data blocks with metadata about the data stored in the file system 2. The exact

Data structure of each Inodes 7 is discussed below Referring to Figure 2A described. In the illustration according to FIG. 1B, a predetermined inode 7r is the one

Root directory 3 assigned. For example, it is a first addressable block of the first

Memory area 5. Further inodes 7 are the respective

Directories A, B, C and D and the files al, a2, bl, cl, c2 and dl assigned. The respective assignment between the inodes 7 and the associated directories

or files is from the implementation of the

File system 2 dependent. Only for the sake of

Clarity are the inodes 7 of the first

Memory area 5 in the above order

shown. In practice, the mapping of inodes to related directories or files is more dependent on the order of their creation.

In the second memory area 6, the actual data of the file system 2 are stored. In particular, for each of the directories A, B, C and D and the root directory "/", a directory object 8 is stored in the second memory area 6. The corresponding directory objects 8 essentially comprise a list of list entries, each list entry containing the names and associated inodes of the list entries For example, directory object 8 for directory A includes two list entries with filenames a1 and a2 and an associated numeric address of inodes 7 associated with files a1 and al2. In second memory area 6, further, file objects 9 are more regular Files are stored associated with corresponding inodes 7 of the first memory area 5. Depending on the size of the stored files, the actual data may be one or more File objects 9 extend. However, this is not shown in FIG. 9 for reasons of clarity.

For very large files, a single inode 7 is not always sufficient to store references to all the file objects 9 of the second memory area 6 in a single inode 7. In this case, the inode 7 refers to further second or third order inodes 7, so that a large number of references to corresponding file objects 9

can be stored. For further details, for example, the article "Inodes" of

English-language online encyclopedia referenced Wikipedia (http://en.wikipedia.org/wiki/Inode, version of 27.10.2013). In the figure IC, a file list 10 is shown. The

File list 10 includes list entries 11 for regular files. Each list entry 11 comprises at least one complete path 12, consisting of a simple path 13 to the directory in which the file is stored, as well as the actual file name 14, a numeric address 15 of the inode 7 associated with the file, and other attributes 16 of the file, such as For example, the date of creation or last modification of the file. The information of the file list 10 may, for example, of software components of a

Archiving system can be used to decide which file objects 9 of the second storage area 6 on another storage medium, such as a magnetic security tape, must be stored. Such file lists 10 are required, inter alia, in hierarchical storage system (HSM) but for other systems and programs for managing large data.

Problematic with known POSIX compatible file systems is that the creation of such a file list 10 takes, inter alia, a considerable time. Especially in so-called "big data" systems with millions of distributed over different nodes of a cluster system

Files is the creation or ongoing maintenance of one

corresponding file list 10 in the time available with known file systems impossible.

The following will be described with reference to Figures 2A and 2B, as based on the on the physical disk. 4

stored information in conventional file systems a file list 10 can be created according to Figure IC. FIG. 2A schematically shows the one used for this purpose

Data structures and Figure 2B is a flowchart of a

Conventional method for processing a

Directory tree 1 for creating the file list 10.

In a first step 20, the inode 7r of the

Root directory 3 determined and loaded. For example, an inode entry with the address "0" can be read in from the first memory area 5. At the same time, a current variable p is used to record the current path, which in this case is represented by a simple "/". Subsequently, in a step 21, an associated

Directory object 8r loaded from the second memory area 6. In the exemplary embodiment illustrated in FIG. 2A, the directory object 8r comprises three directory entries 17 for the directories A, B and C of the first hierarchical level.

Accordingly, it is determined in a subsequent step 22 that directory entries 17 still to be processed are present in the directory object 8r. In the following step 23, first, the first

Directory entry 17A with respect to the still unknown at this point memory object A read. From the

Directory entry 17A shows that the inode 7A with the address "1" is assigned to the memory object A. Furthermore, the directory entry 17A results in the name "A" of the memory object A. In the subsequent step 24, the associated inode 7A of the memory object A is created with the address "1" to determine the metadata associated with the storage object A.

In step 25, it is checked what type of object the memory object A is. From the loaded metadata of the inode 7A can be determined, for example, based on the so-called mode information, whether it is, as in the present case, another directory or a regular file or other object of a POSIX-compatible file system 2. In FIG. 2B, for the sake of simplicity, only the

Processing of files and directories indicated.

Is it as in the present case another

Directory object 8, in a subsequent step 26, the working variable for the path p to be examined is supplemented by the name determined in step 23. Furthermore, the directory that is currently valid is buffered in another variable q. Thereafter, the process is continued recursively in step 21 with the loading of the associated directory object 8A. In the example described, the directory A contains only regular file objects 9 for the files al and a2.

Accordingly, in the subsequent check in step 25, it is determined that the directory entries 17 of the directory object 8A refer to regular files. Accordingly, in a step 27, a list entry 11 for the file list 10 is generated based on the current path p and the file name "al" extracted from the directory entry 17 in step 23. Thereafter, the method is described in

Step 22 for the next directory entry 17

continued. Accordingly, a further list entry 11 for the file a2 is generated in a subsequent circulation in step 27. In a renewed call of step 22, it is determined that no further directory entries 17 are to be processed in directory A. Accordingly, in one

following step 28 is the path to the parent

Directory of the directory A reset, ie the root directory "/", and the process is continued at the next higher level in step 22.

Through the described method, the directory tree 1 shown in FIG. 1A is progressively visited by the recursive algorithm, directory entries 17 for file objects 9 being generated in the file list 10.

As can be seen in particular from the illustration according to FIG. 2A, the method according to FIG. 2B causes a

regular jumping back and forth between the first

Memory area 5 and the second memory area 6. This is due in particular to the fact that without reading out the information of the inodes 7 from the first memory area 5, the associated directory objects 8 and file objects 9 in the second memory area 6 can not be located. Conversely, in the inodes 7 of the first

Memory area 5 does not contain all the information about

To determine, for example, names or paths of individual files or to determine the hierarchical structure of the directory tree 1. In practice, this approach therefore has the disadvantage that a frequent

Repositioning of read / write heads or other read devices of a physical disk 4 are required to create the file list 10.

Particularly in the case of very extensive data collections, as occur, for example, in archiving systems, this leads in practice to long runtimes and therefore to

Problems with frequently occurring processes.

FIG. 3A shows an improved data structure for mapping the directory tree 1 described at the beginning. In particular, in the individual inodes 7, in addition to the above

the information given in each case the name of the associated information

File object 9 or directory object 8 stored. In the inodes 7, a back reference 18 is likewise stored on the inode 7 of a higher-level directory object 8, that is, for example, in the inode 7A of the directory A

Reference to the inode 7r. Only in the inode 7r of the

Root directory 3 itself there is no entry.

Optionally, in all inodes 7, another reference to a predetermined reference point of the file system 2

are stored (not shown in Figure 3A). in the

In the case of the simple directory tree 1 according to FIG. 1A, the entry for the reference point refers directly to the

Root directory 3. For more complex directory trees, the distributed over several disks are correspondingly have multiple mount points, the reference points to the

Mountpoint of the file system containing the file. In this way, the metadata stored in the inodes 7 remain valid even if the relevant file system is integrated elsewhere in a higher-level file system, in particular the root file system.

The name of the associated file object 9 respectively

Directory object 8, the back reference 18 and, if appropriate, the reference to the reference point can be stored, for example, as extended attributes of known file systems, for example by using the so-called xattr function.

Figure 3B shows an improved method of creating the file list 10 based on the data structure of Figure 3A.

In a first step 30, it is checked whether further inodes 7 of a list of inodes are to be processed. For example, the inode list may contain all inodes 7 of the first

Memory area 5 include.

If so, the next one is in step 31

processing inode 7 read. In a subsequent step 32 it is checked whether the inode 7 is assigned to a regular file object 9. If this is not the case, the processing in step 30 can be continued immediately with the next inode 7.

If, on the other hand, it is determined in step 32 that the last-read inode 7 is assigned to a regular file object 9, for example the file al, in step 33 the Address 15 of the inode 7 and the associated name "al" of the file object 9 are cached in a variable n.

In a step 34, subsequently the inode 7A is loaded, which the return link 18 of the previously loaded inode 7 references as inode 7A of the higher-level directory object 8A. Subsequently, in a step 35, the variable n is supplemented by the name "A" of the higher-level directory A. In a step 36, it is checked whether the higher-level directory is already the root directory 3. If this is the case, then a Step 37 now stores the complete path 12 contained in the variable n, including the path 13 and the file name 14 in the file list 10, as well as the address 15 of the inode 7 associated with the regular file "al". Thereafter, the method is continued in step 30 with the processing of possibly further inodes 7 of the inode list. If, however, it is determined in step 36 that it is not yet the root directory 3, the method is continued in step 34 with the loading of the inode 7 of the directory which is the parent directory, so that the path in step 35 is the next higher one

Directory level is supplemented until the process finally arrives at the root directory 3. In this way, for each inode 7 by following the return references 15

parent inodes 7 in a very short time a full path 12 can be determined. In conventional

File system 2, however, the structure or search would be required over a complete directory tree 1. The method according to FIG. 3B has a number of advantages over the method according to FIG. 2B. In particular, when creating the file list 10, a jumping back and forth between the first memory area 5 and the second

Memory area 6 are avoided on the physical disk 4. All data required to create the file list 10 is completely in the inodes 7

saved . At the same time, a complete recursive processing of the directory tree 1 can be dispensed with. In particular, it is possible to perform a linear scan over all inodes 7 that are in the first memory area 5

are stored. Only for the inodes 7, which are assigned to a regular file object 9 and the others

For example, metadata about a particular search profile

must be supplemented by a recursive algorithm by a full path 12. In particular, if already based on a pre-filtering a plurality of inodes 7 of this processing

can be excluded, for example at a

Archive system that captures only the filenames of files modified since a last backup, however, this recursive portion of the algorithm plays only one

subordinate importance for its runtime behavior.

In a non-illustrated modification of

In the embodiment according to FIG. 3A, the inodes 7 contain only the reference back to the higher-level directory object 8, but no attribute with the name of the inode 7

associated file object 9 or directory object 8. In this embodiment, to create a file list 10 still on the directory objects 8 of the second

Memory area 6 are accessed. However, the number of I / O accesses can already be significantly reduced in this embodiment, if only paths for a

relatively small proportion of inodes 7 must be determined. For example, as explained later, due to the other metadata contained in the inode 7, only certain files for further processing, such as a backup, can be selected. If, for example, only 1% of the stored file objects 9 are backed up, the structure of paths for the selected inodes 7 by reading a few directory objects 8 is still associated with significantly fewer I / O accesses than a complete one

Search the directory tree 1.

Figures 4A and 4B show a data structure

or a working method according to another embodiment of the invention. As can be seen in particular from FIG. 4A, in the inodes 7 according to the other

Embodiment instead of the name of the memory object and the back reference 18 to the inode 7 of a parent

Directory object 8 a path 12, stored at least from the parent mountpoint. Furthermore, in each inode 7, a reference 19 to the inode 7 of a

stored in higher reference point. In the

described embodiment serves as a reference point for objects of the root file system, the root directory 3, here a

Reference to the inode 7r. For objects of others in the

The root file system of mounted file systems, such as other partitions of a hard disk, network volumes, or removable storage media, is stored as the reference point of the inode of the mount point of the corresponding mounted file system at which path 12 begins. After all in each inode 7 additionally a backup date 20 is stored, which is the time of a last backup of the associated object on another storage medium, such as a tape storage medium by a

Specifies backup component.

In a modification of the embodiment according to FIGS. 4A and 4B, a fixed reference point, for example the

Root directory 3 or another specified node of file system 2, fixed or elsewhere

saved. In this embodiment, the storage of the reference 19 in the inodes 7 can be dispensed with. Preferably, in this embodiment, only a single additional attribute, namely a complete path 12 including the simple path 13 from the root directory 3 and the file name 14 itself, stored as a contiguous string.

The data structure according to FIG. 4A and its modification described above are suitable for even faster creation of file lists 10. In the exemplary embodiment, the file system 2 is a so-called WORM file system, in which after the first-time writing of a file or the creation of a file Directory name changes or shifts are no longer possible. Thus effectively the path to a once stored file from the given reference point, either a mountpoint or the

Root directory 3, can not change, the

complete path 12 as shown in Figure 4A are stored contiguously in an extended attribute of an inode 7. Creating a file list is particularly easy in this case. In a first step 40, a check is made as to whether further inodes 7 of a list of inodes are to be processed. If this is the case, the first to be processed inode 7 is loaded in step 41. Subsequently, it is checked in step 42 whether the loaded inode 7 a regular

File object 9 is assigned. If this is not the case, the method can be continued in step 40 with the processing of possibly further inodes 7. Otherwise, an entry for the file list 10 can be created immediately in step 43. Is it the stored one

Reference point directly to the root directory 3, the information stored in the current inode 7 for

full path 12 of the data object 9 can be used directly to create the entry. For more complex file systems with multiple mountpoints, a path from the root directory 3 to the respective stored mountpoint may have to be determined. Again, this is usually possible without further I / O access to a mass storage, since even in complex file system usually only a few mountpoints are available, which in a central location, in particular a so-called filesystem table fstab (English: filesystem table) , are stored. The contents of the file system table fstab will be for a variety

different purposes and is therefore usually stored in a cache memory or other memory resident data structure. Thus, the entry through

Combination of a cached path to the specified mountpoint and the path data stored in the inode 7 are formed. After that it will be

Procedure in step 40 continued with the possibly existing next inode 7. The method according to FIG. 4B also has the advantage that only the information stored in the inodes 7 for creating the file list 10 is sufficient. In addition, the method according to FIG. 4B has the advantage that recursive creation of path information is no longer necessary at all. Instead, the file list 10 may be replaced by a

simple, linear algorithm that creates the usually consecutively stored inodes 7

processed sequentially. By doing so, file system with several million inodes 7 is one

Acceleration of about a factor of 100 over known directory scans possible.

If a decision regarding a possible further processing of the data objects 9 can be made solely on the basis of the metadata stored in the inodes 7, the data of the second memory area 6 need not be read at all, which leads to another

substantially accelerating the process.

In the example shown, the additional save date 20 serves this purpose

Exemplary embodiment, all objects of the file system are regularly secured on a further storage medium, such as a magnetic tape. This is a

so-called incremental backup method, where only new since a previous backup

added or changed objects are to be saved. The decision of which objects at a

pending backup run can be based solely on the stored in the inodes 7

Information is taken. In particular, it can be seen in FIG. 4A that the root directory 3 has been since the last one Backup on 01.07.2012 has not been changed and therefore does not need to be secured. By contrast, the directory A was last changed on 03.07.2012 and thus after its last backup on 01.07.2012 and must be secured accordingly. The regular file al has not been backed up at all and therefore needs to be backed up as well.

FIG. 5 shows a memory device 50 according to FIG

Embodiment of the invention. In the

Storage device 50 is for example a so-called storage appliance, which is the archiving

different versions of files or other

Data objects. In this case, at least the respective current version of a file on a powerful, in the embodiment internal mass storage 51 is maintained. In addition, copies of the current version as well

possibly existing previous versions held on a further storage medium. For example, FIG. 5 indicates that such copies are made on a tape drive 52 which is external in the exemplary embodiment or in a so-called cloud memory 53, that is to say one via

Data network, especially the Internet, connected

Storage system to be stored. The memory device 50 according to FIG. 5 has a first interface 54 for accessing those in FIG

Storage device 50 stored data.

For example, this is an interface according to the NFS protocol for the Unix operating system family or an interface according to the CIFS protocol for the

Windows family of operating systems. About the first interface 54 known from these protocols requests for writing and reading and, where appropriate, for deleting and Rename individual files to the storage device 50 are transmitted.

In the exemplary embodiment, the commands received via the interface 54 are analyzed via a processing component 55 of the memory device 50 and, if permissible, implemented. In the exemplary embodiment serves a

Software component running on a non-volatile

Storage medium of the storage device 50 is stored. The processing component 55 sets the corresponding ones

Requirements in a conventional manner for a file system 2 of the mass storage 51, for example, the

Clustering systems particularly suitable GPFS file system

(English: "General Parallel File System") of IBM, um.

When configuring all or part of the

Storage device 50 as WORM systems, requests for changing and renaming files and directories are acknowledged with a corresponding error message and not executed. A deletion of a file is either not allowed or only after expiry of a predetermined retention period. Invalid delete commands are also acknowledged with an error message. In the case of permissible deletion commands, in particular deletion commands for regular files in an ordinary, rewriteable file system, the deletion of a file is entered in an additional protocol file which is used in the later creation of file lists 10 and similar information based on the meta information

Information is taken into account accordingly.

For example, a corresponding inode 7 by

Storing an additional attribute or by

Storing a predetermined value in an existing one Attribute as invalid and thus the associated object marked as deleted.

In addition to the implementation of the actual request are still certain in the described embodiment

Perform previous and subsequent actions to save the additional metadata. In particular, associated with the writing of new files or directories metadata in the associated inode 7 of the file system 2 of the mass storage 51. This includes the

Software component a so-called daemon process that responds to events according to the so-called Data Management API (DMAPI) interface and stores, among other things, the additional information required for the above-described procedures in extended attributes of the GPFS file system.

For example, via the DMAPI interface the

Events "create" and "postcreate" are caught for creating files. Alternatively, it is also possible to get the required

To obtain additional information in a one-time and / or regular search across all directories. For example, in a first step, all

Directory objects 8 of a file system 2 are checked for changes since a last search. In a second step, the inodes 7 of the directory entries 17 of the changed directory objects 8 are determined. For these inodes 7 then in a third step corresponding

Additional information stored. Optionally, before re-saving, it can first be checked whether the inodes 7 already contain current additional information. The storage device 50 further includes a second interface 56 for administration. About the second

Interface 56 has a system administrator or other authorized user access to a configuration dialog 57, with which the behavior of the memory device 50 in

Detail can be configured. For example, it can be selected via the configuration dialog 57 whether the file system provided by the interface 57 of the

Mass memory 51 or individual areas thereof as a WORM storage medium or how a normal, multiple (re) write-capable storage medium should behave.

The memory device 50 further comprises a scanning component 58 which is used inter alia for processing the

additionally stored in the inodes 7 stored metadata. In addition, the memory device 50 comprises a so-called object mover 59, which is responsible for the need-based backup of files on another storage medium. A fuse can be made for various reasons. For example, it may be a regular one

Backup of the stored in the storage device 50 objects act. Alternatively, it can also be a swapping out of a file or file version in a hierarchical storage system or archive system

in which, for example, unused files or outdated versions are moved from the mass storage 51 to the tape drive 52 and / or the cloud storage 53. At the same time, the object mover 59 creates a new object at the destination, which contains at least the data of the secured object and possibly further metadata for the backup. In addition, the metadata stored in the inodes 7 of the file system of the mass storage 51 is updated accordingly. For example, during a backup in an extended attribute the current date as the last backup date 20 notes. Alternatively, at a

Outage, the object associated with the inode 7 may be identified as being deleted from the mass storage 51 or replaced by a so-called stub that is based on the new one

Location references.

The storage device 50 further comprises a so-called backup manager 60 for self-service

Data backup stored on the mass storage 51

Data according to the default settings of the configuration dialog 57. In order to perform this task as efficiently as possible, the backup manager 60 accesses, among other things, the scanning device 58 to objects

to be selected by the object mover 59 from the internal mass storage 51 to another external storage medium.

Subsequently, with reference to FIG 5, a so-called

"Incremental forever" backup system described

The term infinite, incremental fuse is intended to express that after an initial

Basic backup always only the last saved since the last backup on the mass storage 51 or changed

Objects, especially regular files and

Directory objects to be backed up. Of course, other backup strategies, such as a differential backup, in which the difference is always backed up since a last basic backup, or a full backup, in which always the entire content or

predetermined portions of the mass memory 51 is secured can be used. Various system parameters are specified via the configuration interface 57. For example, a so-called mountpoint of a file system 2, for which the incremental backup is to be performed, is determined. In addition, a time interval between

various file versions to be backed up. For example, it can be specified that the

Storage device 50 every hour, daily or weekly backup of the stored in the mass storage 51 files makes. Also other criteria, such as the minimum or maximum size of files to be backed up can be specified via the configuration dialog 57.

If a predetermined backup time or any other event triggering a backup has been reached, all files of the mass memory 51 that currently have to be backed up must first be determined. For this purpose uses the

Storage device 50 scans the scanner 58 for all

Specify files whose creation or modification date is later than the date of the last backup. It should be noted that this information is already included in the exemplary embodiment according to FIG. 4A as backup date 20 in the inodes 7 of the file system 2. Thus, based solely on a scan of all inodes 7, it can be ascertained which file objects 9 are in principle for a

Consider backup without loading the individual file objects 9 of the second memory area 6.

Alternatively, the date of the last backup may also be stored elsewhere in the storage device. For example, a global for all objects of the

Mass storage system 51 valid backup time

stored or determined. In case of a differential backup, use the date of the last full backup instead of the date of the last backup.

Only for the objects since the time of the last one

Backups are subsequently changed based on the metadata contained in the inodes 7, in particular the file name contained therein 14 and / or further

Information a path statement 12 or 13 created a corresponding file list 10 with objects to be backed up. The file list 10 to be created can contain, in addition to the path information 12 or 13 and / or the file name 14, further information,

in particular a direct reference to the associated inode 7, the size of the file and other information of the

Contain metadata. Such a list 10 can

For example, to pass to the object mover 59 to new backup objects on the

Tape drive 52 or the cloud storage 53 store.

In addition to the above-mentioned security system, the data structures according to FIGS. 3A and 4A as well as the

However, algorithms according to FIGS. 3B and 4B also apply to a large number of other applications. In particular, the mechanisms described are suitable for a large number of

Data processing tasks where data is at least partially processed with associated metadata.

As another example, a

Image management software outlines individual images from a large number of existing image data

Based on metadata to be filtered out, such as a creation period, a location or information about a subject contained in the image. Conventionally, such data processing tasks are solved by one of two possible approaches. Either the meta-information is stored together with the actual data, in this case the image data. This has the disadvantage that when searching through the metadata all image files must be opened and at least partially read. Be in the case of the here described

For example, metadata associated with image data, for example as so-called EXIF data in an image format, such as a JPEG image format, must first be read in at least the header information of the respective image file before processing is possible. This leads to the frequent repositioning of read heads of a mass memory already described above and thus to a slower processing speed.

Another approach is to get the required

Metadata in a separate database, in particular a relational database or object database too

to save. In this case, all the information needed to answer a query is available in a common database for expedited processing. However, the second approach mentioned has the disadvantage that changes of other software components to the stored data, such as a revision of the image by a

Image processing program, if necessary, not recognized by the database. Accordingly, there is basically the problem of keeping the metadata stored in the database consistent with the actual data.

According to an embodiment of the invention, such application-specific metadata are additionally stored in the inodes 7 of the file system 2. For example, a Location of a photo in the advanced attributes of a GPFS file system. In this case, the methods described above are applicable for scanning a large list of files. In particular, it can then be determined on the basis of a list of inodes 7 alone which further files have to be taken into account for processing, for example the compilation of a slide show. Since the application-specific attributes are stored directly in the extended file system 2, this can not lead to a discrepancy between the stored metadata on the one hand and possibly changed image data on the other.

In particular, the method already described above using the DMAPI interface is particularly suitable for this purpose. Are appropriate mechanisms for creating and maintaining the stored in the inodes 7 advanced metadata

File system level integrated into a storage device 50, these are always kept independent of a used application running.

Reference sign list

1 directory system

2 file system

3 root directory

4 physical disk

5 first storage area

6 second memory area

7 inodes

8 directory object

9 file object

10 file list

11 List entry

12 (complete) Path 13 (simple) Path

14 file name

15 address (of the inode)

16 more attribute

17 directory entry

18 remittance

19 reference (to reference point)

20 backup date

50 storage device

51 mass storage

52 tape drive

53 cloud storage

54 first interface

55 processing component

56 second interface 57 Configuration dialog

58 scanning device

59 Object Movers

60 backup managers

Claims

claims

A POSIX-compatible file system (2) comprising at least one directory (A) and a plurality of files (a1, a2) stored in the directory (A), wherein

each file (al, a2) is assigned an inode (7) with metadata about the respective file (al, a2);

the directory (A) comprises an association between file names (14) of the plurality of files (al, a2) and the respective inode (7) associated with a file; and

the metadata about the file (al, a2)

at least the file name (14) of the associated file (al, a2) and information about the at least one directory (A).

2. POSIX-compatible file system (2) according to claim 1, in which the information about the at least one directory (A) has a reference (18) to the directory (A)

associated inode (7A).

3. POSIX-compatible file system (2) according to claim 1, wherein the file system (2) is designed as a WORM file system and the information on the at least one directory (A) a path (12, 13) of the associated file (al, a2) from a predetermined reference point, in particular a

Mountpoint of the WORM file system.

4. POSIX-compatible file system (2) according to claim 3, wherein the metadata about the respective file (al, a2) a complete path (12) with a path (13) to the directory in which the file (al, a2) is stored, and the file name (14) itself includes.

5. POSIX compatible file system (2) after one of the

Claims 1 to 4, wherein the metadata on the file (al, a2) additionally a reference (19) to a

predetermined reference point, in particular a mount point, of the file system (2) associated inode (7r).

6. POSIX compatible file system (2) after one of the

Claims 1 to 5, wherein the file name (14) and the

Information about the at least one directory (A) is stored in an extended attribute of the file system (2).

7. A method for generating a file list (10) with files of a POSIX-compatible file system (2) comprising the steps of: - scanning a predetermined group of inodes (7) for acquiring metadata of a plurality of files respectively associated with the inodes (7) al, a2), wherein the metadata comprise at least the file name (14) of the respectively assigned file (al, a2) and information about a directory (A) in which the respective file (al, a2) is stored;

Determining file names (14) and paths (12, 13) of files (al, a2) based solely on the metadata stored in the inodes (7); and

Generating a file list (10) based on

certain file names (14) and path information (12, 13).

8. The method of claim 7, wherein the metadata

at least one further attribute (16), in particular a date of a last backup (20) of the respectively assigned file (al, a2), wherein the at least one further attribute (16) is taken over in the file list (10) and / or a takeover a file in the file list (10) based on the at least one further attribute (16) is decided.

9. The method of claim 8, wherein

in the step of scanning the predetermined list of inodes (7) it is checked whether the metadata of an inode (7) fulfills at least one predetermined condition; and

in the step of generating the file list (10), an entry for a file (a1, a2) associated with an inode (7) is only generated in the file list (10) if the predetermined condition for the metadata of the respective inode (7) is met is.

10. Using extended attributes of a POSIX-compatible file system (2) for storing

Metadata of the file (al, a2) in an inode (7) associated with the file, the metadata including at least the file name (14) of the respective associated file (al, a2) and information about a directory (A) in which the respective File (al, a2) is stored.

11. A memory device (50) comprising:

at least one interface (54) for accessing files (al, a2) stored by the storage device (50); and

at least one mass storage system (51) for

nonvolatile storage of the files (al, a2);

in which

the memory device (50) is adapted, upon receipt of a write command for writing a file (al, a2) via the at least one interface (54), metadata about the at least one file to be written (al, a2) in the mass storage system (51) store, wherein the metadata stored on the file (al, a2) at least the

Filenames (14) of the file (al, a2) and information about a Include directory (A) in which the file (al, a2)

is stored.

The storage device (50) of claim 11, further comprising at least one software component executed by the storage device, wherein the at least one software component is adapted to maintain a file list (10) based solely on the stored metadata, in particular according to a method of any one of claims 7 to 9, to produce.

The storage device (50) of claim 12, wherein the software component is arranged for file management and selects files for further processing based solely on the stored metadata.

14. The storage device (50) according to claim 13, wherein the software component for saving files (al, a2) based on at least one time information stored in the metadata, in particular a modification, creation, and / or backup date (20) of the Metadata associated file (al, a2) is set up.

A storage device (50) according to any one of claims 11 to 14, arranged to store the metadata in the files

(al, a2) associated with inodes (7) of a POSIX-compatible file system (2) of the at least one mass storage device (51).

16. A storage device according to claim 15, wherein the

Storage device (50) is arranged to store at least part of the stored metadata of a selected file (al, a2) comprising the file name (14) of the file (al, a2) and to provide the information about the directory (A) in which the file (al, a2) is stored as extended attributes via the at least one interface (54).