KR100648065B1 - File system for the hardward using the i/o acceleration technique and method for processing data in the same - Google Patents

Abstract

The present invention relates to a file system for hardware to which input / output acceleration technology is applied and a data processing method in the file system. The method manager of this file system receives a method requiring access to a file stored on a disk, and controls access to the file by selecting either block device processing or character device processing for the hardware to which the I / O acceleration technology is applied. do. Slow-Path Manager Method Performs block device processing for hardware with I / O acceleration technology under the control of the manager. The fast-path manager performs character device processing on hardware with I / O acceleration technology under the control of the method manager. The cache manager, under the control of the slow-path manager, performs file access stored on the disk through a cache that caches a portion of the data stored on the disk. Under the control of the fast-path manager, the input / output acceleration manager controls the copy operation of the data stored in the disk to the memory area for input / output acceleration through the hardware to which the input / output acceleration technology is applied. According to the present invention, even hardware using I / O acceleration technology can be supported through a standard interface of the existing Unix file system.

File System, Linux, I / O Acceleration, EXT2, EXT3, Character Device, PMEM

Description

FILE SYSTEM FOR THE HARDWARD USING THE I / O ACCELERATION TECHNIQUE AND METHOD FOR PROCESSING DATA IN THE SAME}

1 is a schematic block diagram of a server to which a file system for hardware to which an input / output acceleration technology according to an embodiment of the present invention is applied is applied.

FIG. 2 is a detailed block diagram of the EXT3NS file system shown in FIG. 1.

3 is a layout view of the disk shown in FIG.

FIG. 4 is a diagram showing the structure of a block group of the disk shown in FIG.

5 is a structural diagram of a data block indexing method in a file system for hardware to which an input / output acceleration technology according to an exemplary embodiment of the present invention is applied.

FIG. 6 is a diagram for describing a 4KB block indexing method in the disk shown in FIG. 3.

7 is a flowchart illustrating a method in which a file system for hardware to which an input / output acceleration technology is applied according to an embodiment of the present invention supports a standard file interface by performing a low-path.

8 is a flowchart illustrating a method of supporting a standard file interface by performing a fast-path in a hardware file system to which an input / output acceleration technology according to an embodiment of the present invention is applied.

The present invention relates to a file system, and more particularly, to a file system that provides an interface to an application program that requires a standard file system interface on hardware to which an input / output acceleration technology having a special mechanism is applied, and data processing in the file system. It is about a method.

Recently, the biggest problem in providing a streaming service in the trend of high-definition video is the limitation of server capacity. Typically, the server reads input data from the disk and sends it through the network card. In this operation, the processor of the server is involved. Therefore, when the I / O workload is high, such as streaming, the performance of the I / O device will determine the performance of the server. In order to solve this problem, the necessity of input / output acceleration technology has emerged, and research on this has been performed.

Recent researches on I / O acceleration technology have implemented I / O acceleration function through hardware without software dependence, so that it can secure enough bandwidth for multimedia streaming while reducing the load on system processor. In the hardware with I / O acceleration technology, the basic disk block size is set at least 64 times larger than the 4KB used in the existing file system. In addition, the data read from the disk can be serviced directly through the network. This has a structure suitable for serving high performance files having continuity such as media data.

On the other hand, the standard interface used in the file system on the operating system follows the specification defined in the POSIX (Portable Operating System Interface for UNIX), an OS standard application program interface (API) established by the IEEE standards organization. Most file systems in the Linux kernel follow this one, such as the Ext (Extended file system) 2 file system or the Ext3 file system. However, in the case of a file system that aims for input / output performance as a priority, such as multimedia or parallel processing, it has been common to provide an interface different from the standard in order to provide high performance to the user. This is because the existing application has the disadvantage of restricting the use of the provided file system without changing the source code. Therefore, the file systems for high performance I / O are considered for the above design because the implementation mechanism is not suitable to support the standard file interface. As a result, there is a problem in that existing file systems such as Linux's various file systems cannot be used due to their structural characteristics even on the hardware to which the I / O acceleration technology for providing the I / O function of the high performance server is applied.

Therefore, in order to solve the above problems, the present invention provides a hardware for which I / O acceleration technology is applied to alleviate the bottleneck of the processor and the bus while supporting the interface of the existing Unix file system in a high performance server using I / O acceleration hardware. A file system and a data processing method in the file system are provided.

File system according to an aspect of the present invention for achieving the above object,

A file system for managing files on a disk connected to hardware to which I / O acceleration technology is applied.

A method manager that receives a method requiring access to a file stored on a disk and selects any one of block device processing and character device processing for hardware to which the input / output acceleration technology is applied to control access to the file; A slow-path manager performing block device processing on hardware to which the input / output acceleration technology is applied under the control of the method manager; A fast-path manager that performs character device processing on hardware to which the input / output acceleration technology is applied under the control of the method manager; A cache manager, under the control of the slow-path manager, performing file access stored on the disk through a cache that caches a portion of data stored on the disk; And an input / output acceleration manager that controls, by the control of the fast-path manager, a copy operation of the data stored in the disk to the memory area for the input / output acceleration through the hardware to which the input / output acceleration technology is applied.

Here, the size of the data block processed by the slow-path manager and the fast-path manager is different.

A data processing method in a file system according to another aspect of the present invention,

A data processing method in a file system for managing files on a disk connected to hardware to which input / output acceleration technology is applied,

a) receiving a method call for processing a file stored on the disk; b) determining whether a buffer specified in the called method corresponds to a memory area for accelerating input / output; c) In step b), if it is determined that the buffer corresponds to the memory area for accelerating the I / O, the character data processing for the hardware to which the I / O acceleration technology is applied is performed so that the file data specified in the method is displayed. Controlling copying to a memory area for input / output acceleration; And d) in step b), if it is determined that the buffer does not correspond to the memory area for the I / O acceleration, perform block device processing on the hardware to which the I / O acceleration technology is applied to perform file data specified in the method. Performing access to the.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the accompanying drawings, parts irrelevant to the description of the present invention are omitted in order to clearly describe the present invention, and the same or similar parts are denoted by the same reference numerals.

1 is a schematic block diagram of a server to which a file system for hardware to which an input / output acceleration technology according to an embodiment of the present invention is applied is applied.

In an embodiment of the present invention, for convenience of description, the hardware to which the input / output acceleration technology is applied is referred to as a 'network storage card', and a file system according to an embodiment of the present invention for supporting a standard interface to such an NS card is provided. Is described as 'EXT3NS'.

As shown in FIG. 1, in a server providing a high-definition video streaming service, the Linux kernel 100 uses an NS card 300 to which an input / output acceleration technology connected through a Peripheral Component Interconnect (PCI) bus 200 is applied. It provides high performance I / O service.

The NS card 300 to which the input / output acceleration technology is applied includes a network interface card (NIC) unit 310, a disk unit 320, and a PMEM (PCI memory) unit 330.

The disk unit 320 reads data from the disk 400 or writes data to the disk 400 under the control of the Linux kernel 100.

The NIC unit 310 transmits data stored in the memory read by the disk unit 320 to an external network or receives data input from an external network and stores the data in the memory through the disk unit 320 and then stores the disk ( To be written at 400).

The PMEM unit 330 reads data stored in the disk 400 in a substantially zero-copy form and transmits the data directly to an external network.

Meanwhile, the Linux kernel 100 may include a virtual file system (Virtual Filing System 110), an EXT2 file system 120, an EXT3 file system 130, an EXT3NS file system 140, a page cache / buffer cache ( Buffer Cache) 150, block device driver 160, and NS driver 170.

The virtual file system 110 is a kernel software layer that handles all system calls related to the standard Unix file system. A virtual file system 110 handles system files and files that are directed to the necessary control functions in the physical file system code that control I / O. It is a purposeless layer. This virtual file system 110 is an abstraction layer between the implementation of the actual file system and the user process, and includes various types of files including the EXT2 file system 120, the EXT3 file system 130, and the EXT3NS file system 140. It provides a common interface common to the system.

The EXT2 file system 120 and the EXT3 file system 130 are standard file systems of the Linux system. The EXT2 file system 120 is an asynchronous file system that does not simultaneously store data and metadata of a file, and the EXT3 file system 130 ) Is a journaling file system applying the journaling technique used in the database to the EXT2 file system 120. Journaling technology means that when a user enters or modifies data, the relevant data is recorded in the log before the data is immediately written to the hard disk, and if an abnormal shutdown occurs due to a power failure or other reason, which is the most frequent problem during recording, It is a technology that allows you to rewrite or recover by referring to the data recorded in the log when booting. That is, the EXT3 file system 130 is a file system with backup and recovery capability, leaving a portion for recording.

The EXT3NS file system 140 is a file system according to an embodiment of the present invention and is a file system capable of supporting a standard interface for the NS card 300 to which an input / output acceleration technology is applied. The EXT3NS file system 140 is implemented to enable standard interface support for the NS card 300 while keeping its foundation in the EXT3 file system, which is a standard file system of Linux. The EXT3NS file system 140 is located under the virtual file system 110, such as the EXT2 file system 120 and the EXT3 file system 130, and the page cache / buffer cache 150, like the EXT3 file system 130. ) Can be accessed as a block device for the NS card 300, and also through the NS driver 170, the NS card 300 can be accessed in the form of a character device. That is, the EXT3NS file system 140 processes the NS card 300 as a block device when performing a slow-path for providing an input / output function between the server main memory and the disk 400. In the case of performing a fast-path for providing a zero-copy input / output function between the disk 400 and the PMEM, the NS card 300 is treated as a character device and accessed.

When slow-pathing is performed, a read is performed from the disk 400 to the size corresponding to the page cache (usually 4 KB) via the NS card 300, and this data is stored in a user-provided buffer, i.e., the main memory of the server. Copied to the area.

When the fast-path is performed, reading is performed from the disk 400 through the NS card 300 in a size corresponding to a logical block (mainly an integer multiple of 256 KB), and this data is copied to the PMEM area.

On the other hand, in the page cache / buffer cache 150, the page cache is used to speed up access to images and data on disk. This is used to cache the logical contents of a file page by page, accessed through the file and its offset within the file. When pages are read from disk 400 into the server's memory, the pages are cached in the page cache.

The buffer cache also contains data buffers used by the block device driver 160. These buffers have a fixed size (e.g., 512 bytes) and have a block of data read from or written to NS card 300. These buffer caches are indexed by device identifiers and desired block numbers, which can be used to find data blocks quickly. When performing the low-path in the EXT2 file system 120, the EXT3 file system 130, and the EXT3NS file system 140, the NS card 300 is accessed only through a buffer cache. At this time, if data is found in the buffer cache, access to the disk 400 through the NS card 300 is not necessary.

The block device driver 160 blocks the NS card 300 through the page cache / buffer cache 150 by file systems such as the EXT2 file system 120, the EXT3 file system 130, and the EXT3NS file system 140. Provide an interface for access to the device.

The NS driver 170 provides an interface so that the EXT3NS file system 140 can directly access the NS card 300 to the character device device.

Meanwhile, the PCI bus 200 is a path for data that connects peripheral devices such as the disk 300 connected through the NS card 300 to the system, in this case, the Linux kernel 100, and the PCI bus 200 is The Linux kernel 100 allows each device to be accessed independently of one another.

The disk 400 is connected to the Linux kernel 100 through the NS card 300, and stores and manages various data.

2 is a detailed block diagram of the EXT3NS file system 140 shown in FIG.

As shown in FIG. 2, the EXT3NS file system 140, which is a hardware file system to which an input / output acceleration technology is applied according to an embodiment of the present invention, has a method manager 141, a slow-path manager 143, and a cache. Manager 145, fast-path manager 147 and NS manager 149.

The method manager 141 processes a method called from the virtual file system 110 by generating a system call related to reading or writing a file in an application program. The method manager 141 examines the buffer arguments passed to the method called from the virtual file system 110 and blocks the disk 400 through the NS card 300 if the buffer used is the server's main memory. Instructs the slow-path manager 143 to perform the work to access the device. If the buffer used is the PMEM area, the fast-path manager 147 for accessing the disk 400 through the NS card 300 to the character device is instructed to perform a task.

The slow-path manager 143 performs block device processing for the NS card 300 according to the instruction of the method manager 141. That is, the slow-path manager 143 performs the same data processing as in the conventional EXT3 file system 130, and the NS card 300 connected through the page cache / buffer cache 150 and the block device driver 160. The disk 400 is accessed in the form of a block device to read or write data.

The cache manager 145 controls the page cache / buffer cache 150 under the control of the slow-path manager 143 to perform related data processing.

The fast-path manager 147 performs character device processing for the NS card 300 according to the instruction of the method manager 141. That is, the fast-path manager 147 accesses the disk 400 in the form of a character device through the NS card 300 connected through the NS driver 170 to control data to be zero-copyed through the PMEM area. do.

The NS manager 149 controls the NS driver 170 under the control of the fast-path manager 147 to perform related data processing.

Slow-path manager 143 reads data from disk 400 via NS card 300 in size corresponding to page cache, while fast-path manager 147 reads disk 400 through NS card 300. ) Read the data in the size corresponding to the logical block.

3 is a layout view of the disk 400 shown in FIG.

As shown in FIG. 3, the arrangement of the disk 400 in the EXT3NS file system 140 assumes that M physical disks are mounted on the NS card 300, and the M disks mounted in this manner are connected in parallel. One logical block (logical block 0, logical block 1, logical block 2, etc.) is formed. In such a situation, the NS card 300 streams one logical block into M physical disks in a size obtained by dividing M into logical blocks.

On the other hand, a plurality of logical blocks are configured to form one block group (block group 0, block group 1, ..., block group N). In order to increase the availability of the EXT3NS file system 140, the same metadata is included at the beginning of each block group. Finally, the block groups thus configured form logical disks again. The EXT3NS file system 140 operates on the logical disk thus formed.

4 is a diagram showing the structure of a block group of the disk 400 shown in FIG.

As shown in FIG. 4, each block group in the EXT3NS file system 140 contains metadata of the critical file system that controls the information.

The meta data includes a super block (410) and a group description block (420), a block bitmap (430), an inode bitmap (440), and an inode table (Inode table). , 450) and a data block 460.

The super block 410 stores and manages information on the basic size or shape of the EXT3NS file system 140. The slow-path manager 143 and the fast-path manager 147 of the EXT3NS file system 140 may utilize and maintain the file system using the information of this super block. The super block 410 records information such as the total number of inodes, the size of the file system, the size of the block, the number of inodes per group, and the like. Here, the inode is the most basic unit of the file system and has all the information of the file except the file name. The file name is stored in the directory along with the inode number.

The group description block 420 is a bitmap indicating the allocation state of each block in the block group, the number of which is equal to the number of blocks. For example, a block bitmap field is assigned a block number of a block bitmap, an inode bitmap field is assigned a block number of an inode bitmap, and an inode table field is assigned a first number of an inode table 450. The block number of the inode table block is assigned.

The block bitmap 430 is a bitmap indicating the allocation state of the blocks, the number of which is equal to the number of blocks. The block bitmap 430 is information referred to when a block is allocated or released. Thus, in the case of the block bitmap 430, one bit is mapped to one logical block.

The inode bitmap 440 is a bitmap indicating the inode allocation state of the block, and the number thereof is the same as the number of blocks as in the block bitmap. The inode bitmap 440 is information referred to when the inode is allocated or released.

The inode table 450 is composed of adjacent contiguous blocks, and each block corresponds to an inode. These inodes contain information such as file type, access rights, file length, owner identifier, last access time, and inode last change time.

Access to such metadata is handled only through the slow-path. That is, system calls for metadata are handled by the slow-path manager 143. Thus, all metadata is manipulated and cached based on page cache size. This is to reduce the cost of access to metadata, which is different from the size of the logical block handled by NS driver 170 used with PMEM.

The ext3ns_sb_info structure of the super block 410 includes s_es, which is a pointer to a super block in the buffer pool 500, s_sbh, which is a pointer to a buffer 510 having a super block, and a buffer 520 having a group description block. ), S_group_count, which is a pointer to), s_block_bitmap, which is a field for the cache 530 of the block bitmap, and s_inode_bitmap, which is a field for the cache 540 of the inode bitmap.

The "struct inode" structure of the inode table 450 includes the structure of inode information "ext3ns_inode_info", which includes i_data [15], which is a set of pointers 550 pointing to a data block, and which the inode belongs. I_block_group, the index for the block group, is included. Among these, i_data [15] has a value of 15 bits in total, and a total of 12 bits from bit 0 to bit 11 directly point to a data block, and the rest have entries of 12, 13, and 14, and indirect. ), Double indirect, triple indirect. Points to a data block with a pointer to the data block.

A method of mapping each position of the file to the disk 400 will be described with reference to FIG. 5. Used in this mapping is the indirect block mechanism among the metadata. That is, mapping is performed using i_data [15] included in the structure of the inode table 450.

Referring to FIG. 5, the i_data [15] array is the first start of this indirect block mechanism. The first 12 elements of this array point directly to the logical block. The following elements indicate indirect logical blocks, double indirect logical blocks, and triple indirect logical blocks, respectively, in that order.

The ext3ns_block_to_path () function in the EXT3NS file system 140 enables mapping from the offset of the file to the logical block of the logical disk 400. This function retrieves a logical block that stores the logical block index corresponding to the offset position of the file. For example, with reference to FIG. 6, if the size of the logical block is 256n and the offset in the logical block is d, it corresponds to the quotient of d / (4K / 4) among contiguous 4KB small blocks in the 1MB block. The small blocks are read and brought to the page cache / buffer cache 150. Then, the logical block index at the offset corresponding to the remainder of d / (4K / 4) in the small block is obtained. The index thus obtained corresponds to the sequence number of consecutive logical blocks in the logical disk 400. For example, if the obtained index is i, the position is 256n * i. The operation up to this point is a process of obtaining a logical block necessary for the fast-path according to an embodiment of the present invention.

If you want to go through the slow-path and read the small blocks into the page cache / buffer cache 150, some steps are added before and after the fast-path progression described above. The first half of the work that needs to be added divides the 256n from the file's offset to get the remainder, then divides the remainder by 4K to get the quotient. The later work that needs to be added is to read the small blocks into the page cache / buffer cache 150 through the shares obtained from the previous work from the logical blocks obtained through the ext3ns_block_to_path () function. In the example shown in FIG. 6, when the quotient obtained through the first half operation is k, the small block position "256n * i + k" should be actually read through the block device driver 160 in the low-path.

Hereinafter, a method of supporting a standard file interface by the EXT3NS file system 140, which is a hardware file system to which an input / output acceleration technology according to an embodiment of the present invention is applied, will be described in detail with reference to FIGS. 7 and 8.

First, a low-path for accessing metadata or the like in a system or an application program will be described with reference to FIG. 7.

When a read or write system call to a file is called, the virtual file system 110 operates to handle it.

The virtual file system 110 performs a preliminary work to process the called system call, and then calls the corresponding method of the corresponding file system, here EXT3NS file system 140 (S1).

Next, the method manager 141 of the EXT3NS file system 140 examines the buffer argument passed to the method called by the virtual file system 110 to determine whether the buffer is the main memory area of the server. Since slow-path performance is described here, the buffer corresponds to the main memory area of the server. Therefore, the method manager 141 instructs the slow-path manager 143 to process the NS card 300 as a block device (S2), and the slow-path manager 143 to perform the corresponding task. Related data is requested to the page cache / buffer cache 150 connected to the cache manager 145 (S3 and S4).

Therefore, the page cache / buffer cache 150 has a size corresponding to the page cache from the disk 400 connected to the NS card 300 through the block device driver 160 except that there is data corresponding to the inside thereof. 4KB) and read data (S5, S6), and the data read from the disk 400 is cached in the page cache / buffer cache 150 (S7, S8).

Thereafter, the page cache / buffer cache 160 delivers the cached data to the method manager 141 through the cache manager 145 and the slow-path manager 143 (S9, S10, and S11). 141 copies the transferred data to the buffer area in the main memory area provided by the user through the virtual file system 110 as the data for the corresponding method (S12).

Next, a fast-path for providing a high quality video streaming service will be described with reference to FIG. 8.

When a read or write system call for a file for a streaming service is called, the virtual file system 110 operates to process it.

The virtual file system 110 performs a preliminary operation to process the called system call, and then calls a corresponding method of the corresponding file system, here EXT3NS file system 140 (S21).

Next, the method manager 141 of the EXT3NS file system 140 examines the buffer argument passed to the method called by the virtual file system 110 to determine whether the buffer is a PMEM region. Since fast-path performance is described here, the buffer corresponds to the PMEM area. Thus, the method manager 141 instructs the fast-path manager 147 to process the NS card 300 as a character device (S22), and the fast-path manager 147 to perform the corresponding task. The NS driver 170 connected to the NS manager 149 requests related data processing (S23 and S24).

Therefore, the NS driver 170 drives the NS card 300 as a character device (S25), and the disk unit 320 of the NS card 300 is a size corresponding to a logical block of the requested data (mainly an integer multiple of 256KB). A read operation is performed from the disk 400, and the read data is copied to the PMEM area (S25).

Therefore, the NS card 300 transmits the result of the processing in which the relevant data is copied to the PMEM area to the method manager 141 through the NS driver 170, the NS manager 149, and the fast-path manager 147 (S26). (S27, S28), the method manager 141 returns the related data processing result to the user through the virtual file system 110 as a processing result for the called method (S29).

As described above, the data processing through the slow-path can maintain the existing standard file interface through the EXT3NS file system 140 according to the embodiment of the present invention, even for data processing through the fast-path for high speed video streaming. .

Although the present invention has been described with reference to the most practical and preferred embodiments, the present invention is not limited to the above-described embodiments, but includes various modifications and equivalents within the scope of the following claims.

According to the present invention, even hardware using I / O acceleration technology can be supported through a standard interface of the existing Unix file system.

Therefore, I / O acceleration technology can be used in the same application environment.

Claims (8)

A file system for managing files on a disk connected to hardware to which input / output acceleration technology is applied, A method manager that receives a method requiring access to a file stored on a disk and selects any one of block device processing and character device processing for hardware to which the input / output acceleration technology is applied to control access to the file; A slow-path manager performing block device processing on hardware to which the input / output acceleration technology is applied under the control of the method manager; A fast-path manager that performs character device processing on hardware to which the input / output acceleration technology is applied under the control of the method manager; A cache manager, under the control of the slow-path manager, performing file access stored on the disk through a cache that caches a portion of data stored on the disk; And Under the control of the fast-path manager, an I / O acceleration manager for controlling a copy operation of data stored in the disk to a memory area for I / O acceleration through hardware to which the I / O acceleration technology is applied. File system that includes. The method of claim 1, The slow-path manager and fast-path manager, A file system for controlling file data through metadata stored on the disk. The method of claim 2, The meta data includes an inode table that stores information corresponding to an inode, which is a basic unit of the file system. The file data processed by the slow-path manager and the fast-path manager are data blocks mapped by an inode structure that is a basic structure of the inode table. File system. The method of claim 3, And the size of the data block processed by the slow-path manager and the fast-path manager is different. The method according to any one of claims 1 to 4, The cache manager performs file access using data cached in the cache from the disk through a block device driver that enables access to a block device device for hardware to which the input / output acceleration technology is applied. system. The method according to any one of claims 1 to 4, And a memory area for accelerating the input / output is PMEM (PCI Memory). A data processing method in a file system for managing a file on a disk connected to hardware to which input / output acceleration technology is applied, a) receiving a method call for processing a file stored on the disk; b) determining whether a buffer specified in the called method corresponds to a memory area for accelerating input / output; c) In step b), if it is determined that the buffer corresponds to the memory area for accelerating the I / O, the character data processing for the hardware to which the I / O acceleration technology is applied is performed so that the file data specified in the method is displayed. Controlling copying to a memory area for input / output acceleration; And d) In step b), if it is determined that the buffer does not correspond to the memory area for the I / O acceleration, block device processing is performed on the hardware to which the I / O acceleration technology is applied to the file data specified in the method. To perform access to Data processing method in the file system comprising a. The method of claim 7, wherein And a block size of the file data processed in the character device in step c) and a block size of the file data processed in the block device in step d) are different.

Images