WO2013091244A1

WO2013091244A1 - File system data storage method and access method and device therefor

Info

Publication number: WO2013091244A1
Application number: PCT/CN2011/084551
Authority: WO
Inventors: 王凤; 张敬亮
Original assignee: 华为技术有限公司
Priority date: 2011-12-23
Filing date: 2011-12-23
Publication date: 2013-06-27
Also published as: CN103282899A; CN103282899B

Abstract

The present invention provides a file system data storage method and an access method and device therefor. A magnetic disk space corresponding to a file system is divided into a first storage portion and a second storage portion, the first storage portion is used for storing a metadata file, and all path information required for accessing the magnetic disk space is stored in the metadata file; the second storage portion is used for storing file data; when the system is started, the metadata file is read into an internal memory from the magnetic disk space, and then the file system performs storage and access operations on the magnetic disk space based on the metadata file in the internal memory. All the path information is stored in the metadata file, so the storage location of a target node or file may be found from the metadata file just once, thereby reducing the number of times of interaction between the internal memory and the magnetic disk space, and greatly improving the access efficiency.

Description

The present invention relates to the field of storage technologies, and in particular, to a method, an access method and a device for storing data in a file system. Background technique

A file system is a method and data structure used by an operating system to clarify files on a disk or partition, that is, a method of organizing files on a disk. The Namespace is the organization of file directories in the file system, or the logical organization of files in the file system. It is an important part of the file system. From the user's point of view, the namespace provides users with a logical, clear, easy-to-use, visual, and understandable form of logical files. Users perform various operations on files in the organization provided by the namespace, regardless of how the files are placed on the storage device, thereby solving or reducing the semantic separation between the human and the computer on the data store.

At present, the file directory organization of the tree structure is most similar to the real world organization structure, and is widely accepted. Therefore, most file systems logically organize file directories in a tree manner, including various disk file systems, such as the X extended file system (EXTx) and the log file system ( Journal file system, JFS), Reiserfs (Reiser file system), ZFS (Zettabyte File System), B+ tree file system (Btrfs), NTFS (New Technology File System), and file partition (File Allocation) Table, FAT32), etc.; network file system, such as Network File System (NFS), AFS (Old File System), Common Internet File Systems (CIFS), etc.; cluster file system, such as parallel network Parallel network file system (pNFS), parallel virtual file system (PVFS), General Parallel File System (GPFS), etc.; distributed file system, such as Google file system (google file system) System, GoogleFS ), distributed file system ( Hadoop Distr Ibuted File System; hadoo, HDFS), MFS (Moose File System), KFS (Kosmos distributed file system), Taobao file system (taobao file system, TaobaoFS). Although the file system logically adopts a tree structure to organize the file directory, in the specific implementation, it is implemented in multiple layers and multiple levels in the disk and the memory, and the specific implementation is related to the physical organization of the file system. The physical organization of the file system represents the location, connection, and cataloging of files on the file storage device (physical) in the file system. At present, most file systems have a similar physical organization. The central concept of the physical organization is superblock, inode, Uata block, Uentry. The super block is used to store the overall information of the file system, such as the size of the file system (the size depends on the file system); the directory entry is used to record the name and inode number of the ordinary file in the current directory, or record the current directory. The name of the subdirectory and the index node number; the index node is used to record the metadata information of a common file or directory in the file system, and the metadata information includes the data block where the ordinary file is located or the next level directory entry of the directory. Data block; a data block is used to place the actual contents of a normal file or to store all directory entries in a directory.

FIG. 1 is a schematic diagram of a disk distribution manner of the file system ext3. As shown in FIG. 1, the physical storage space (or partition) of the file system ext3 is divided into a plurality of block groups, each of which is composed of a super block, a group descriptor, a data block bitmap, and an inode. It consists of a bitmap, an inode table, a data block, and so on. Among them, the data block bitmap can quickly find out which data blocks have content and which data blocks are empty blocks. The data block bitmap is an indispensable part of the file system disk distribution. The index node bitmap can also be used to quickly find out which index nodes are free and which index nodes have content. The inode table records the inode number and the metadata information of the file or directory corresponding to the inode number, including the attributes of the file or directory and the data block where the file or directory is located. The actual block file content is recorded in the data block, or the directory entry is the name of the next-level subdirectory of the current directory and the corresponding index node. The file system ext3 can search and access the file directory through the index table and the directory entries recorded in the data block.

As a whole, the existing file system namespace organization adopts a tree structure, and the physical space organization mode of the file system is implemented by hierarchical distribution. The file system performs file directory lookup and access, that is, the resource mapping from the namespace to the file storage device needs to be divided into two layers: from the path (path) to the inode (inode) and from the inode (inode) to the data block (block) ). Take the home/picture (picture)/flower.jpg (flower.jpg) file as an example. The existing file system needs to start from the root directory, where the index node of the root directory is recorded in the file system. Then, the file system needs to access the disk space, and the root node is obtained according to the physical organization of the disk space. The data block pointed to by the index node, and the next level of content of the root directory, that is, the directory entry, is obtained from the data block. Then, the file system matches the home with the obtained directory entry, and finds the index node corresponding to the home directory. Then, the file system accesses the disk space again, and obtains the data block pointed to by the home index node according to the physical organization of the disk space, and obtains the next level content of the home directory, that is, the directory entry from the data block. Then, the file system searches and matches the picture with the obtained directory entry, and finds the index node corresponding to the picture directory. Then, the file system accesses the disk space again, and obtains the data block pointed to by the index node of the picture according to the physical organization of the disk space, and obtains the next level content of the picture directory, that is, the directory item, from the data block. Next, the file system matches the flower.jpg with the obtained directory entry and finds the index node of the flower.jpg file. Then, the file system accesses the disk space again, and obtains the data block pointed to by the index node of the flower.jpg file according to the physical organization of the disk space, and then goes to the data block to obtain the flower.jpg file.

It can be seen from the above that when an existing file system accesses data such as a file or a directory through a namespace, the operation is complicated, and multiple interactions between the memory and the disk space are required to complete, and the access or read efficiency is low. Summary of the invention

The embodiment of the invention provides a data storage method, an access method and a device in a file system, which are used to reduce the number of interactions between the memory and the disk space when the file system accesses data such as files or directories, and improve the access efficiency.

An embodiment of the present invention provides a method for storing data in a file system. The disk space corresponding to the file system is divided into a first storage portion and a second storage portion, where the first storage portion is configured to store a metadata file. The metadata file stores all path information required to access the disk space; the second storage portion is configured to store file data; and the file system converts the metadata when the data storage device in the file system is started The file is read from the disk space into the memory, and the storage method includes:

If the file system establishes a next-level directory node under the first directory node, the file system stores the directory metadata information of the next-level directory node into a metadata file in the memory, and in the memory Adding first path information indicating the next-level directory node to the directory metadata information of the first directory node in the metadata file; the number of directory elements of the next-level directory node The information includes: attribute information of the next-level directory node;

If the file system stores a file under the first directory node, the file system acquires a free data block in the disk space, stores the file into the acquired data block, and files the file Metadata information is stored in the metadata file in the memory, and second path information pointing to the file is added to the directory metadata information of the first directory node in the metadata file in the memory; The file metadata information of the file includes: attribute information of the file and a label of a data block storing the file;

The file system writes the metadata file in the memory into the disk space according to a preset period.

An embodiment of the present invention provides a storage device for data in a file system, including: a file system and a disk space corresponding to the file system;

The disk space is divided into a first storage portion and a second storage portion, the first storage portion is configured to store a metadata file, where the metadata file stores all path information required to access the disk space, The second storage portion is configured to store file data;

The file system includes:

a first initial reading module, configured to read the metadata file from the disk space into a memory when the storage device of the data in the file system is started;

a first storage module, configured to: when the file system establishes a next-level directory node under the first directory node, store the directory metadata information of the next-level directory node into a metadata file in the memory, and Adding, to the directory metadata information of the first directory node in the metadata file in the memory, first path information that points to the next-level directory node; and the directory metadata information of the next-level directory node includes: Attribute information of the next-level directory node;

a second storage module, configured to: when the file system stores a file under the first directory node, acquire a data block that is free in the disk space, and store the file into the acquired data block, where The file metadata information of the file is stored in the metadata file in the memory, and the second path information pointing to the file is added to the metadata information of the first directory node in the metadata file in the memory. The file metadata information of the file includes: attribute information of the file and a label of a data block storing the file;

a synchronization module, configured to write the metadata file in the memory into the disk space according to a preset period. An embodiment of the present invention provides a method for accessing data in a file system. The disk space corresponding to the file system is divided into a first storage portion and a second storage portion, where the first storage portion is configured to store a metadata file. The metadata file stores all path information required to access the disk space; the second storage portion is configured to store file data; and the file system activates the element when the data access device of the file system is started The data file is read from the disk space into the memory, and the access method includes:

If the data to be accessed by the file system is a target directory node, the file system starts from a root node in the metadata file in the memory, and points to a next-level directory according to directory metadata information of each directory node. The first path information of the node is searched and matched in the directory metadata information of the directory nodes of each level until the target directory node is found or the search fails; if the target directory node is found, the file system is at the target Obtaining attribute information of the target directory node in the directory metadata information of the directory node;

If the data to be accessed by the file system is a target file, the file system starts from a root node in the metadata file in the memory, and points to a next-level directory node according to directory metadata information of each directory node. The first path information is searched and matched in the directory metadata information of the directory nodes of each level, and the last level directory node where the target file is located is obtained, and then pointed to according to the directory metadata information of the last level directory node. The second path information of the next file is searched and matched in the file metadata information of the file under the last level directory node until the target file is found or the search fails; if the target file is found, the file The system acquires, from the file metadata information of the target file, a label of a data block in the disk space in which the target file is stored, and then reads from a data block of the disk space identified by a label of the acquired data block. Take the target file.

An embodiment of the present invention provides an apparatus for accessing data in a file system, including: a file system and a disk space corresponding to the file system;

The disk space is divided into a first storage portion and a second storage portion, wherein the first storage portion is configured to store a metadata file, where the metadata file stores all path information required to access the disk space; The second storage portion is configured to store file data;

The file system includes:

a second initial reading module, configured to read the metadata file from the disk space into a memory when the access device of the data in the file system is started; a first access module, configured to start from a root node in the metadata file in the memory when the file system accesses the target directory node, and point to the next according to the directory metadata information of each directory node The first path information of the level directory node performs lookup matching in the directory metadata information of the directory nodes of each level until the target directory node is found or the search fails, and if the target directory node is found, the target directory node Obtaining attribute information of the target directory node in the directory metadata information;

a second access module, configured to: when the file system accesses the target file, start from a root node in the metadata file in the memory, and point to a next level according to directory metadata information of each directory node The first path information of the directory node is searched and matched in the directory metadata information of each directory node, and the last-level directory node where the target file is located is obtained, and then according to the directory metadata information of the last-level directory node. The second path information pointing to the file below is searched and matched in the file metadata information of the file under the last level directory node until the target file is found or the search fails, and if the target file is found, Obtaining, in the file metadata information of the target file, a label of a data block storing the target file in the disk space, and then reading the data block of the disk space identified by a label of the acquired data block The target document.

The data storage method and device in the file system provided by the embodiment of the present invention stores all path information of the disk space by using the metadata file, and reads the metadata file from the disk space into the memory when the system starts, in the When creating a next-level directory node or storing a file under a directory node, directly store the metadata information of the next-level directory node or the file to be stored into the metadata file in the memory, and at the previous level. Add the path information of the next-level directory node or the file to be stored in the metadata information of the directory node, organize the management of the namespace by using the metadata file, and divide only the space and storage file for storing the metadata file in the disk space. The data block can be used, and the spatial division is not scattered as in the prior art, which improves the efficiency of the file system in storing data in the disk space, and lays a foundation for improving the efficiency of the file system accessing the data in the disk space.

The data access method and device in the file system provided by the embodiment of the present invention cooperate with the data storage method and device in the file system provided by the embodiment of the present invention, and the file system only needs the metadata in the disk space when the system is started. The file is read into the memory space, according to the metadata information of the directory nodes and the files under the various levels of the directory stored in the metadata file (ie accessing the entire magnetic The path information required for the disk space is searched. The metadata file in the memory can be used to find the target file or the target directory at one time, which reduces the number of interactions between the memory and the disk space, and greatly improves the access efficiency. DRAWINGS

The drawings used in the embodiments or the description of the prior art are briefly described. It is obvious that the drawings in the following description are some embodiments of the present invention, and are not creative to those skilled in the art. Other drawings can also be obtained from these drawings on the premise of labor.

1 is a schematic diagram of a disk distribution manner of a file system ext3;

2A is a schematic diagram of a tree structure of a metadata file according to an embodiment of the present invention; FIG. 2B is another schematic diagram of a tree structure of a metadata file according to an embodiment of the present invention;

3 is a schematic structural diagram of a data storage device in a file system according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an apparatus for accessing data in a file system according to an embodiment of the present invention. The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. The embodiments are a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The present invention provides a metadata organization manner that can be directly mapped from a namespace to a disk space, so as to improve the file, in order to solve the problem that the file system in the prior art has a low efficiency in accessing the data in the disk space through the namespace. The efficiency with which the system stores and accesses data. In the embodiment of the present invention, the namespace of the file system is implemented by a metadata file in the disk space in a specific implementation. Wherein, embodiments of the present invention employ a data link of a variant trie The organization manages the metadata file (that is, the directory nodes at all levels in the metadata file form a tree structure), and the target file or the target directory can be found at one time through the metadata file, thereby greatly improving the search efficiency.

Based on the foregoing metadata file, an embodiment of the present invention provides a method for storing data in a file system. In this storage method, the file system manages the file directories in the namespace through metadata files. The execution body of the embodiment of the present invention is a storage device for data in a file system, the device includes a file system and a disk space corresponding to the file system; the disk space is divided into a first storage portion and a second storage portion, and the first storage portion is used For storing metadata files, the metadata file stores all the path information needed to access the entire disk space; the second storage portion is used to store the file data. When the storage device for data in the file system is booted (or the system is booted), the file system reads the metadata file from disk space into memory. Subsequently created directories and files in each directory are managed by metadata files in memory. The storage method provided by the embodiment of the present invention includes: a file system creates a new directory (ie, a next-level directory) in a certain directory, and stores a storage operation of the information about the newly created directory and a file system that creates and stores the file in a certain directory. operating.

First, the file system creates a root node and stores the attribute information of the root node into a metadata file in memory. The file system then creates and stores new directory nodes and/or files based on the root node.

If the file system establishes the next-level directory node under the first directory node, the file system stores the directory metadata information of the next-level directory node into the metadata file in the memory, and the directory element in the first directory node The first path information pointing to the next-level directory node is added to the data information. The directory metadata information of the next-level directory node includes: attribute information of the next-level directory node.

The first directory node may be any directory node that already exists in the file system namespace, and may be, for example, a root directory node, a first-level sub-directory node that may be a root directory node, or a first-level subdirectory. The next-level subdirectory node of the node, and so on. This embodiment takes a new directory node (i.e., the next-level directory node) under any of the directory nodes as an example.

The file system can determine in advance whether the directory or file is to be created. When it is determined that the next-level directory node is to be created under the first directory node, the file system stores the directory metadata information of the next-level directory node to be created into the metadata file in the memory to implement the metadata file. The pieces are managed for each directory node. At the same time, the file system stores the path information between the first directory node and the next-level directory node in the metadata file in the memory, that is, the storage in the directory metadata information of the first directory node points to the next The path information of the level directory node, so that the next directory node can be found through the first directory node. The attribute information of the directory node may include: one of a directory node name, a type, a size, and a creation time, or any combination thereof. The size of the directory node is generally 4096 bytes.

That is, if the directory node in the namespace of the file system has a lower-level directory node, the directory metadata information of the directory node stored in the metadata file in the memory includes not only the attribute information of the directory node. Also includes path information for the directory node to point to its next-level directory node. The structure of the metadata file can be regarded as a variant of trie. Each directory node can correspond to a trie node, and the directory metadata information of each directory node can be used as the content of the corresponding trie node. The variant trie of this embodiment differs from the standard trie in that each node of the standard trie stores one character, and each node of the variant trie of the embodiment stores a character string. The string here mainly refers to the attribute information of each directory node and the path information to the next-level directory node. For the directory node that does not have the next-level directory node, the corresponding string mainly refers to the attribute information of the directory node.

If the file system is to store the file under the first directory node, the file system obtains the free data block in the disk space, stores the file in the acquired data block, and stores the file metadata information of the file into the metadata in the memory. In the file, a second path information pointing to the file is added to the directory metadata information of the first directory node in the metadata file in the memory. The file metadata information of the file includes attribute information of the file and a label of a data block in the disk space in which the file is stored.

In this embodiment, the disk space corresponding to the file system is mainly divided into two parts, one part is used to store the metadata file, and the other part is used to store the data of each file. Wherein, the portion of the storage file is divided into a plurality of data blocks. In order to keep the metadata files in the memory synchronized with the metadata files in the disk space, the file system also stores the file metadata information in the memory into the disk space according to a preset period. The embodiments of the present invention do not limit the specific value of the preset period. For example, the period may be 10 minutes, and the cycle may be cycled once (that is, the synchronization is performed before the shutdown, so as to facilitate the data in the file system. When the storage device is restarted, the file system can read the latest metadata file from disk space). When the file system creates a file in a directory, it needs to store the created file in disk space. Therefore, the file system queries the disk space for a block of data that is free in disk space and can store the file, and then stores the file in the fetched data block. The file system then stores the file metadata information of the file into a metadata file in memory to manage the files in the file system's namespace through the metadata file. The file metadata information mainly includes attribute information of the file and a label of a data block in the disk space in which the file is stored. The attribute information of the file may include: one of the file name, the type, the size, and the creation time of the file, or any combination thereof. For example, the file name of the file can be picture.jpg, thesis.doc, etc.; the file type can be ordinary file, symbolic link, socket, etc. The file information can be learned through the attribute information of the file, and stored in disk space. The label of the data block of the file can be used to know where the file is stored in disk space in order to read or access the file from disk space.

In this embodiment, each file in the metadata file may also correspond to a trie node, and the file metadata information of each file is the content of the corresponding trie node. The trie node here stores a string compared to the standard trie node. The string here mainly refers to the attribute information of each file and the label of the data block of the file stored in the disk space.

Further, the file system further needs to store the information under the directory node directly in the metadata file, that is, add the directory path to the second path of the file in the directory metadata information of the directory node directly adjacent to the file. Information, so that the file is found by traversing the directory node.

That is, if a file exists under a directory node, the directory metadata information of the directory node includes not only the attribute information of the directory node but also the second path information of the directory node to which the directory node points.

Further, if a directory node has both a file and a next-level directory node, the directory metadata information of the directory node includes attribute information of the directory node, second path information of the directory node to the next file, and The first path information pointing to the next-level directory node.

Further, the directory node in this embodiment may also be empty, that is, the directory node has neither a file nor a next-level directory node, and the directory metadata information includes only the attribute information of the directory node.

It can be seen from the above that the data storage device in the file system of the embodiment is activated at the time of the file. The system reads the metadata file from the disk space to the memory, and operates the metadata file in the memory when storing the file or creating the new directory node, so that the metadata file stores all the directory nodes and the directory node and the lower part thereof. The path information between the files, and then synchronize the metadata files in the disk space according to a preset period. Since the metadata file of this embodiment is a variant trie structure, each trie node may store attribute information of the directory node, path information of the directory node to the next file or the next-level directory node, attribute information of the file, or storage. The information such as the label of the data block of the file, the namespace of the file system is managed by the metadata file, and the namespace is associated with an overall data structure. On the one hand, the file system is convenient for data storage, thereby improving storage efficiency. When searching, the storage location of the target directory or the target file can be found by reading the disk space at one time, which reduces the number of interactions between the memory and the disk space, and improves the efficiency of the file system to access the disk space.

To further illustrate the flow of the data storage method in the file system provided by the embodiment of the present invention, the following describes the storage root directory /home/picture/flower.jpg file as an example.

When the storage device (or system) of the data in the file system is started, the file system reads the metadata file from the disk space into the memory, and then, during the storage of the file or the directory establishment process, the element in the memory The data file is operated.

In this embodiment, it is assumed that the root directory already exists in the metadata file read from the disk space into the memory. Based on this, the file system creates a home directory under the root directory of the metadata file in memory, and stores the directory metadata information of the home directory into the metadata file in the memory. For example, the file system stores the directory name home, creation time, and the like of the home directory into a metadata file in memory (or a data structure stored in the variant trie). Next, the file system creates a picture directory in the home directory, stores the directory metadata information of the picture directory into the metadata file in the memory, and adds path information to the picture directory in the directory metadata information of the home directory. The path information may include: a directory name of the picture directory and a pointer to the picture directory, so that the next directory of the home directory may be identified, and the next level directory may be found. Then, the file system creates a flower.jpg file in the picture directory, stores the file metadata information of the flower.jpg file into the metadata file in the memory, and adds a pointer to flower.jpg in the catalog metadata information of the picture directory. Path information for the file. The path information may include: a file name of the flower.jpg file and a pointer to the flower.jpg file, so that it can identify which file is in the picture directory. You can also find the files under it.

After the above operation, the information stored in the metadata file in the memory can be represented as a tree structure as shown in FIG. 2A. Assuming that the preset period arrives at this time, the file system writes the metadata file shown in Fig. 2A into the first storage portion of the disk space.

Further, assuming that the file system also needs to create a configuration (config) file under the root directory node, the file system stores the file metadata information of the config file into the metadata file in the memory, and the directory metadata information in the root directory node. Add path information to the config file. Further, assuming that the file system also creates a video (Video) directory in the home directory, the file system stores the directory metadata information of the Video directory into the metadata file in the memory, and adds the directory metadata information in the home directory. Path information to the Video directory. Going a step further, suppose the file system also creates the today ( today ) directory in the picture directory and creates the sun . bmp ( sun.bmp ) file in the today directory. After the above operation, the information stored in the metadata file in the memory can be represented as a tree structure as shown in Fig. 2B.

In Fig. 2A and Fig. 2B, the circle indicates a non-empty directory node (i.e., there is a subordinate directory node under the directory node or a file exists or a file and a directory node at the same time). A triangle indicates an empty directory node (that is, there is neither a file nor a next-level directory node under the directory node). The box represents the file node.

Further, when the preset period arrives again, the file system writes the metadata file in the memory to the disk space, so that the metadata file in the disk space is consistent with the metadata file in the memory.

Compared with the standard trie, the variant trie shown in FIG. 2A and FIG. 2B mainly has the following differences: the standard trie stores one character per node, and the variant trie of this embodiment stores a string for each node. For example, directory name, file name, path information, and so on. Unlike the standard trie, all nodes of the variant trie of this embodiment can record information.

In summary, the path of the namespace of the file system corresponds to the variant trie structure of the embodiment, that is, a path from the root node to one leaf node through one or more intermediate nodes. The variant trie of this embodiment is consistent with the directory tree structure of the file system itself, and can preserve the path complete semantics. In the variant trie of this embodiment, only the non-leaf node records the directory metadata information of the directory node, and does not need to record the correspondence between the directory node and the data block label, because in the variant trie structure, the next-level directory node of the directory node is not stored. In a data block, but directly The relationship between the directory nodes is maintained by the structure of the variant trie itself. In the variant trie of the embodiment, the leaf node records the file metadata information of the ordinary file and the corresponding data block label, and queries the leaf node, thereby obtaining the data block number of the storage file, which is equivalent to obtaining the content of the file. Each leaf node of the variant trie of this embodiment can not only correspond to the file system file but also corresponding to the empty directory, and can record the directory metadata information of the empty directory. The data structure design itself can support the establishment of an empty directory, support for reading the attribute information of the directory, support the traversal of the next level of the directory, and the operation of the file system directory can be correspondingly, which lays a foundation for the file system to access the disk space.

Further, in the foregoing embodiments, the file system may perform a linear table, a binary tree, or a hash in a metadata file in the memory for each directory node and/or each directory node and its lower file node. Organization and storage management.

An embodiment of the present invention provides a method for accessing data in a file system. The access method is implemented based on a method for storing data in a file system provided by the foregoing embodiment. In this embodiment, the access device for data in the file system also includes a file system and a disk space corresponding to the file system. The disk space is divided into a first storage portion for storing a metadata file, and a second storage portion for storing all path information required to access the disk space. As described in the foregoing storage method embodiment, the metadata file stores directory metadata information of each directory node and file metadata information of a file stored in the disk space, and the file metadata information includes file attribute information and a disk. The label of the data block storing the file in the space, and the directory metadata information of the directory nodes of each level includes the attribute information of the directory node. If the directory node has a lower-level directory node, the directory metadata information further includes first path information corresponding to the next-level directory node; if the file exists under the directory node, the directory metadata information further includes a file pointing to the next file. Second path information. The second storage portion is for storing file data. When the access device of the data in the file system is powered on, the file system reads the metadata file from disk space into memory, and then accesses the disk space based on the path information in the metadata file in the memory. The access method of this embodiment includes: an operation of the file system to access attribute information of a directory node and an operation of the file system to access a file stored in the disk storage space.

A preferred implementation is as follows: The directory nodes at each level in the metadata file form a tree structure.

Since this embodiment manages all path information through a metadata file, regardless of the file system Which access operation is performed can be based on metadata files in memory.

If the data to be accessed by the file system is a directory node (denoted as a target directory node), the file system starts from the root node in the metadata file in the memory, according to the directory metadata information of each directory node. The first path information pointing to the next-level directory node performs lookup matching in the directory metadata information of each directory node until the target directory node is found or the search fails.

If the file system finds the target directory node in the metadata file in the memory, the file system reads the attribute information of the target directory node from the directory metadata information of the target directory node to implement reading or accessing the target directory node. It can be seen that all the path information is managed by the metadata file, so that the file system only needs to interact between the memory and the disk space when accessing the directory node (that is, the file system will be activated when the access device of the data in the file system is started). The metadata file is read from the disk space into the memory), the access is completed, the number of interactions between the memory and the disk space is reduced, and the access efficiency is improved.

Further, if the data to be accessed by the file system is a file (for the difference, it is recorded as the target file), the file system starts from the root node in the metadata file in the memory, according to the directory metadata information of the directory nodes of each level. The first path information pointing to the next-level directory node is searched and matched in the directory metadata information of each directory node, the last-level directory node where the target file is located, and then the directory metadata of the last-level directory node is obtained. The second path information in the information pointing to the next file is searched and matched in the file element data information of the file under the last level directory node until the target file is found or the search fails.

If the file system finds the target file from the metadata file in the memory, the file system obtains the label of the data block of the storage target file in the disk space from the file metadata information of the target file, and then the file system obtains the label of the obtained data block. The target file is read in the data block of the identified disk space. That is, after obtaining the label of the data block where the target file is located, the file system reads the target file in the corresponding data block in the disk space, and reads the target file into the memory.

The attribute information of the directory nodes at each level includes: one of a directory name, a type, a size, and a creation time of each directory node or any combination thereof. The attribute information of the file includes: one of the file name, type, size, and creation time of the file or any combination thereof.

Further, in combination with the file system management mode of the directory nodes and file nodes in the metadata file, the file system can use a linear table, a binary tree or a hash mode in each directory. The directory metadata information of the node is searched and matched in the file metadata information of the file under the last level directory node where the target file is located.

The data access method in the file system provided by the embodiment is matched with the data storage method in the file system provided by the foregoing embodiment, and the file system namespace is matched with the entire data structure by using the storage structure of the metadata file. The file system reads the metadata file from the disk space into the memory when the access device of the data in the file system is started. The metadata file in the memory makes the parsing of the namespace directly from the path to the data block, and is no longer the path to The process of indexing the node to the data block, changing the disk space to search for multiple times, reduces the number of interactions between the file system and the disk space, and improves the efficiency of finding or accessing disk space.

The following describes the flow of the file access method provided by the embodiment of the present invention by taking the home/picture/flower .jpg file as an example in the metadata file structure shown in FIG. 2B.

In this embodiment, the file system uses the hash algorithm when looking up to the next level. The process of accessing the file is as follows:

Step 1. The file system reads the metadata file from the disk space into the memory when the access device of the data in the file system is started.

Step 2. The file system starts from the root node of the metadata file in the memory, and finds the home directory of the next-level directory node of the root node by hash.

Step 3. The file system continues to find the next-level directory node of the home directory node through the hash method, that is, the icture directory.

Step 4. The file system continues to find the flower.jpg file under the picture directory node by hash.

Step 5. The file system obtains the data block number of the flower.jpg file stored in the disk space from the file metadata information of the flower.jpg file.

In steps 2 through 5 above, the file system always looks in the metadata file in memory, does not interact with disk space, and can find the storage location of the target file in disk space by one search.

Step 6. The file system reads the flower.jpg file from the corresponding data block in the disk space according to the obtained data block label.

Further, assuming that the file system continues to look up the home/picture/today directory in the metadata file structure shown in FIG. 2B, the flow of the directory access method is as follows: Step a: The file system starts from the root node of the metadata file in the memory, and finds the home directory of the next-level directory node of the root node by hash.

Step b. The file system continues to find the directory of the next level directory of the home directory node by hash.

Step c. The file system continues to find the next directory directory of the picture directory node by hash.

Step d: The file system obtains the attribute information of the today directory from the directory metadata information of the today directory.

The metadata file is the file system that reads from the disk space into the memory when the access device of the data in the file system is started, so the metadata file in the memory is directly used in the subsequent access process. In the above steps a-step d, the file system is also searched in the metadata file in the memory, there is no interaction with the disk space, and the target directory node can be found in one search.

As can be seen from the access process of the foregoing file and directory, the embodiment organizes the namespace of the file system by using the metadata file, so that the file system can find the attribute information of the directory node or find the file in the disk space in the metadata file at a time. In the storage location, there is no need to repeatedly interact between memory and disk space, which improves the efficiency of searching.

In summary, the data storage method in the file system provided by the embodiment of the present invention cooperates with the data access method in the file system provided by the embodiment of the present invention, and can generate the following beneficial effects: 1. The namespace of the file system Managing an overall memory data structure using metadata files avoids multiple interactions between memory and disk space, with good overall performance, excellent performance, and high efficiency. 2. The data structure of the metadata file is simple and consistent with the directory tree structure of the file system itself, and can preserve the complete semantics of the path. 3. Through the data structure of the metadata file, the parsing process of the namespace is simplified from the path to the inode to the block process to the process directly from the path to the block, and the search is changed to the mapping one time, and the file or directory can be efficiently completed. Find. 4. Organize namespaces with metadata files to support directory operations for file systems very well and simply.

FIG. 3 is a schematic structural diagram of a data storage device in a file system according to an embodiment of the present invention. As shown in FIG. 3, the apparatus of this embodiment includes: a file system 31 and a disk space 32 corresponding to the file system 31.

The disk space 32 of this embodiment is divided into a first storage portion and a second storage portion, A storage portion is used to store metadata files that store all path information needed to access disk space 32, and the second storage portion is used to store file data.

The file system 31 of this embodiment includes: a first initial reading module 311, a first storage module 312, a second storage module 313, and a synchronization module 314.

The first initial reading module 311 is connected to the disk space 32 for reading the metadata file from the disk space 32 into the memory when the storage device of the data in the file system is started.

The first storage module 312 is connected to the first initial reading module 31 1 and configured to store the directory metadata information of the next-level directory node to the first file node when the file system establishes the next-level directory node under the first directory node. An initial reading module 31 1 reads into the metadata file in the memory, and adds the first path information pointing to the next-level directory node in the directory metadata information of the first directory node in the metadata file in the memory. The directory metadata information of the next-level directory node includes: attribute information of the next-level directory node. The attribute information of the next-level directory node includes: one of the directory name, type, size, and creation time of the next-level directory node or any combination thereof.

The second storage module 313 is connected to the first initial reading module 31 1 and the disk space 32, and is configured to acquire the free data block in the disk space 32 when the file system stores the file under the first directory node, and store the file to In the acquired data block, the file metadata information of the file is stored into the metadata file read by the first initial reading module 311 into the memory, and the element of the first directory node in the metadata file in the memory The second path information pointing to the file is added to the data information; the file metadata information of the file includes: attribute information of the file and a label of the data block of the storage file. The attribute information of the file includes: one of the file name, type, size, and creation time of the file or any combination thereof.

The synchronization module 314 is connected to the disk space 32 and is used to write the metadata file in the memory into the disk space 32 according to a preset period. More specifically, the synchronization module 314 writes the metadata file in memory to the first storage portion of the disk space according to a preset period.

Among them, the directory nodes at all levels in the metadata file can form a tree structure, that is, the entire metadata file can be managed through a tree structure. For each level of directory nodes and between directory nodes and files, you can use the linear table, binary tree or hash mode for organization and management.

Specifically, the first storage module 312 can organize the directory metadata information of the first directory node and the directory metadata information of the next directory node in a metadata file in the memory in a linear table, a binary tree, or a hash manner. . The second storage module 313 can organize the directory metadata information of the first directory node and the file metadata information of the file in a metadata file in the memory in a linear table, a binary tree, or a hash manner.

The function modules of the data storage device in the file system of the present embodiment can be used to execute the flow of the data storage method in the file system provided by the present invention. The specific working principle is not described here. For details, refer to the description of the method embodiment.

The data storage device in the file system of this embodiment stores all the path information of the disk space by using the metadata file, and reads the metadata file from the disk space into the memory when the system starts, and creates a new level. When the directory node or a file is stored under a directory node, the metadata information of the next-level directory node or the file to be stored is directly stored in the metadata file in the memory, and the element of the upper-level directory node is Add the path information to the next-level directory node or the file to be stored in the data information, and organize the management of the namespace by using the metadata file, so that only the space for storing the metadata file and the data block of the storage file are divided in the disk space. However, the spatial division is not scattered as in the prior art, which improves the efficiency of the file system for storing data in the disk space, and lays a foundation for improving the efficiency of the file system to access data in the disk space.

FIG. 4 is a schematic structural diagram of an apparatus for accessing data in a file system according to an embodiment of the present invention. As shown in FIG. 4, the apparatus of this embodiment includes: a file system 41 and a disk space 42 corresponding to the file system 41.

The disk space 42 of this embodiment is divided into a first storage portion and a second storage portion, the first storage portion is configured to store a metadata file, and the metadata file stores all path information required to access the disk space 32. The second storage portion is for storing file data.

The file system 41 of this embodiment includes: a second initial reading module 411, a first access module 412, and a second access module 413.

The second initial reading module 411 is connected to the disk space 42 for reading the metadata file from the disk space 42 into the memory when the access device of the data in the file system is started.

The first access module 412 is connected to the second initial reading module 41 1 for reading from the second initial reading module 41 1 into the metadata file in the memory when the file system accesses the target directory node. The root directory node starts, according to the directory path metadata information of the directory nodes of each level, the first path information pointing to the next-level directory node is in the directory metadata information of each directory node. Perform a search match until the target directory node is found or the search fails. If the target directory node is found, the attribute information of the target directory node is obtained from the directory metadata information of the target directory node.

The attribute information of the directory nodes at each level includes: one of a directory name, a type, a size, and a creation time of each directory node or any combination thereof.

The second access module 413 is connected to the second initial reading module 41 1 and the disk space 42 for reading the metadata file in the memory from the second initial reading module 411 when the file system accesses the target file. Starting at the root directory node, searching and matching in the directory metadata information of each directory node according to the first path information pointing to the next-level directory node in the directory metadata information of each directory node, and obtaining the last of the target file The first-level directory node performs matching matching in the file metadata information of the file under the last-level directory node according to the second path information in the directory metadata information of the last-level directory node, until the target is found. The file or the search fails. If the target file is found, the label of the data block storing the target file in the disk space 42 is obtained from the file metadata information of the target file, and then the disk space 42 identified from the obtained data block is identified. The target file is read in the data block.

The attribute information of the file includes: one of the file name, type, size, and creation time of the file or any combination thereof.

Preferably, the metadata file of the embodiment is managed in a tree structure as a whole, that is, the directory nodes at each level in the metadata file constitute a tree structure.

The first access module 412 may specifically use a linear table, a binary tree, or a hash mode in each of the directory nodes of the metadata file and the directory node and the file to be organized and managed by using a linear table, a binary tree, or a hash mode. Lookup matching is performed in the directory metadata information of the level directory node. The second access module 413 may specifically perform lookup matching in the directory metadata information of the directory nodes of each level and the file metadata information of the files in the last level directory node where the target file is located in a linear table, a binary tree or a hash manner.

The function modules of the data accessing device in the file system of the present embodiment can be used to execute the flow of the data access method in the file system provided by the embodiment of the present invention, and the specific working principle is not described again.

The data access device in the file system of the embodiment cooperates with the data storage device in the file system provided in the above embodiment, and the file system only needs to be in the disk space when the system is booted. The metadata file is read into the memory space, and the metadata information (that is, the path information required to access the entire disk space) is searched according to the directory nodes stored in the metadata file and the files in the directories at all levels, through the memory. The metadata file can find the target file or the target directory at one time, which reduces the number of interactions between the memory and the disk space, and greatly improves the access efficiency.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The foregoing steps include the steps of the foregoing method embodiments; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

It should be noted that the above embodiments are only for explaining the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that: The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; and the modifications or substitutions do not deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

Claim

A method for storing data in a file system, wherein a disk space corresponding to a file system is divided into a first storage portion and a second storage portion, and the first storage portion is configured to store a metadata file, The metadata file stores all path information required to access the disk space; the second storage portion is configured to store file data; and the file system converts the metadata file when the data storage device in the file system is started Read from the disk space into the memory, the storage method includes:

If the file system establishes a next-level directory node under the first directory node, the file system stores the directory metadata information of the next-level directory node into a metadata file in the memory, and in the memory The first path information of the next-level directory node is added to the directory metadata information of the first directory node in the metadata file; the directory metadata information of the next-level directory node includes: the next Attribute information of the level directory node;

If the file system stores a file under the first directory node, the file system acquires a free data block in the disk space, stores the file in the acquired data block, and files the file Metadata information is stored in the metadata file in the memory, and second path information pointing to the file is added to the directory metadata information of the first directory node in the metadata file in the memory; The file metadata information of the file includes: attribute information of the file and a label of a data block storing the file;

2. The method for storing data in a file system according to claim 1, wherein the directory nodes at each level in the metadata file form a tree structure.

The method for storing data in a file system according to claim 1, wherein the file system is in the linear table, the binary tree or the hash mode, and the first file is in the metadata file in the memory. The directory metadata information of the directory node, the directory metadata information of the next-level directory node, and the file metadata information of the file are organized and stored.

The method for storing data in the file system according to claim 1 or 2 or 3, wherein the attribute information of the next-level directory node includes: a directory name and a type of the next-level directory node, One or any combination of size and creation time;

The attribute information of the file includes: a file name, a type, a size, and a creation time of the file. One of them or any combination thereof.

A method for accessing data in a file system, wherein a disk space corresponding to a file system is divided into a first storage portion and a second storage portion, and the first storage portion is configured to store a metadata file, The metadata file stores all path information required to access the disk space; the second storage portion is configured to store file data; and the file system activates the metadata file when the data access device of the file system is started Read from the disk space into the memory, the access method includes:

The method for accessing data in a file system according to claim 5, wherein the directory nodes at each level in the metadata file form a tree structure.

The method for accessing data in a file system according to claim 5, wherein the file system is in a linear table, a binary tree or a hash manner in directory metadata information of each directory node and at the last Find matching in the file metadata information of the file under the primary directory node.

8. A method of accessing data in a file system according to claim 5 or 6 or 7, The attribute information of the directory nodes of the levels includes: one of a directory name, a type, a size, and a creation time of the directory nodes of the levels; or any combination thereof;

The attribute information of the file includes: one of a file name, a type, a size, and a creation time of the file, or any combination thereof.

9. A storage device for data in a file system, comprising: a file system and a disk space corresponding to the file system;

The disk space is divided into a first storage portion and a second storage portion, where the first storage portion is configured to store a metadata file, where the metadata file stores all path information required to access the disk space, The second storage portion is configured to store file data;

The file system includes:

a first initial reading module, configured to read the metadata file from the disk space into a memory when a storage device of data in the file system is started;

And a synchronization module, configured to write the metadata file in the memory into the disk space according to a preset period.

The data storage device in the file system according to claim 9, wherein the directory nodes at each level in the metadata file form a tree structure.

The storage device for data in the file system according to claim 9, wherein the first storage module is in a metadata file in the memory in a linear table, a binary tree or a hash mode. Directory metadata information of the first directory node and the next-level directory node Directory metadata information is organized for storage;

The second storage module organizes the directory metadata information of the first directory node and the file metadata information of the file in a metadata file in the memory in a linear table, a binary tree or a hash manner. .

The storage device for data in the file system according to claim 9 or 10 or 11, wherein the attribute information of the next-level directory node comprises: a directory name and a type of the next-level directory node. One of, size and creation time, or any combination thereof;

13. A device for accessing data in a file system, comprising: a file system and a disk space corresponding to the file system;

The file system includes:

a second initial reading module, configured to read the metadata file from the disk space into a memory when the access device of the data in the file system is started;

a first access module, configured to start from a root node in the metadata file in the memory when the file system accesses the target directory node, and point to the next according to the directory metadata information of each directory node The first path information of the level directory node performs lookup matching in the directory metadata information of the directory nodes of each level until the target directory node is found or the search fails, and if the target directory node is found, the target directory node Obtaining attribute information of the target directory node in the directory metadata information;

a second access module, configured to: when the file system accesses the target file, start from a root node in the metadata file in the memory, and point to a next level according to directory metadata information of each directory node The first path information of the directory node is searched and matched in the directory metadata information of each directory node, and the last-level directory node where the target file is located is obtained, and then according to the directory metadata information of the last-level directory node. The second path information pointing to the file below is searched and matched in the file metadata information of the file under the last level directory node until the target file is found or the search fails, if the target file is found Obtaining, from the file metadata information of the target file, a label of a data block in the disk space in which the target file is stored, and then in a data block of the disk space identified by a label of the acquired data block Read the target file.

14. The data access device of the file system according to claim 13, wherein the directory nodes at each level in the metadata file form a tree structure.

The device for accessing data in the file system according to claim 13, wherein the first access module searches for directory metadata information of each directory node in a linear table, a binary tree or a hash manner. Match

The second access module performs search matching in the directory metadata information of each directory node and the file metadata information of the file under the last-level directory node in a linear table, a binary tree or a hash manner.

The device for accessing data in the file system according to claim 13 or 14 or 15, wherein the attribute information of the directory nodes of the levels includes: a directory name, a type, a size, and One of the creation times or any combination thereof;