KR100758272B1 - File movement method supporting data zero-copy technique - Google Patents

Abstract

A method for moving a file without copying data is provided to enhance whole performance with reduction of CPU load and costs for copying the data to a user area by moving the file between a source and target device with a buffer page shared in a kernel area. A file moving request is received from an application providing of the user area. The buffer page for the source device is allocated in the kernel area(401). The data of the moved file stored in the source device is copied to the buffer page in a DMA(Direct Memory Access) mode(402). The buffer page is removed from a cache and page management information is changed to allocate the buffer page to the target device(405). The data of the buffer page is copied to the target device in the DMA mode(406).

Description

File movement method supporting data zero-copy technique}

1 is a diagram illustrating a file moving method using data copying between a user area and a kernel area according to the prior art;

2 is a diagram illustrating an embodiment of a data copyless file moving method according to the present invention;

3 is an exemplary explanatory diagram showing an available moving path supported by a data copyless file moving method according to the present invention;

4 is a flowchart illustrating an embodiment of a method for moving a data copyless file according to the present invention.

Explanation of symbols on the main parts of the drawings

101, 201: source device 100, 200: destination device

102, 202: Kernel Area 103, 203: Application

104: user buffer 105, 106: page

107, 108, 205, 206: socket buffer 204: buffer page

The present invention relates to a method of moving a dataless copy file, and more particularly, to share a buffer page existing in a kernel area in order to prevent data copying between a user area and a kernel area performed by a CPU. Dataless copy file movement method to improve overall performance by performing file movement between target devices (e.g. disks, network cards), thereby reducing the load on the central processing unit and its associated costs of copying data into the user area. It is about.

In systems that are requested to move large files, such as multimedia streaming servers, data processing costs increase as the file size increases, and the availability of a central processing unit (CPU) is a major issue. Doing.

In other words, as data throughput increases due to file movement, the cost of copying data between system and user area and calling a system call increases, resulting in a decrease in utilization of the central processing unit per unit operation. There are a number of ways for this.

More specifically, in order to move a file, a system call must be repeatedly called. In each call, two CPUs copy data between user / kernel areas and process switching of a process. . This is an operation that increases the cost of the central processing unit resources and adds a load to the system that handles a large number of large data movement requests.

In order to solve this problem, a file moving technique using a dataless copy method has been developed in a hardware manner and a software manner. The hardware method is mainly researched in the network card field. By connecting the memory in the network card with the address space in a certain area in the user area, the data transferred to the network is not copied into the kernel area. It is a method of copying data directly from a network device to a network device. In this case, the data copy is copied using a direct memory access (DMA) method, not a central processing unit.

In addition, the software method mainly connects a certain address space of the user area and a predetermined memory area allocated to the kernel area so that data in the designated memory area can be directly accessed without copying from the user area or the kernel area.

All of the above methods are performed through re-mapping between designated memory areas. In order to use these methods, memory such as a creation process for a designated memory area, memory allocation and deallocation for processing a request, and the like are performed. There was a problem that additional management in the area was required.

Meanwhile, a method for moving a file between a user area and a kernel area according to the prior art will be described below.

1 is a diagram illustrating a file moving method using data copying between a user area and a kernel area according to the prior art.

When the kernel performs a general file move operation, the application program 103 located in the user area requests reading a file and receives a page 105 from the kernel for reading data stored in the source device 101. At this time, when the source device 101 is a network card, the socket buffer 107 is allocated. After the data is first copied from the source device 101 to the allocated page 105 or the socket buffer 107 by a direct memory access (DMA) method, the page 105 is executed by the central processing unit (CPU). Or data in socket buffer 107 is secondary copied to user buffer 104 (110, 114).

Subsequently, a request for movement to the target device 100 is requested for data stored in the user buffer 104 in the user area, and a page 106 or socket buffer 108 for this is allocated from the kernel. Data stored in the user buffer 104 by the central processing unit is first copied to the allocated page 106 or the socket buffer 108, and then secondly copied to the target device 100 by a direct memory approach. (112, 116).

In the copying process, all data included in the moving target file is repeatedly moved in the user area to the allocated memory area size. Whenever the file is moved from the kernel area to the user area or vice versa, the contest switching of the corresponding process ( context switching occurs. In addition, the request in the user area means a system call call. In order to move data in a certain unit of file between devices (disk, network card, etc.), two central processing units copy and two system call calls. Is required.

This time, the prior art related to file movement will be described with specific examples.

As a first prior art, a Korean patent "zero-copy method between a kernel and a user in an operating system (published no. 10-2000-0065846, published on November 15, 2000)", in particular, between a kernel region and a user region in a Solaris operating system Since the kernel uses virtual memory instead of copying a buffer in the user area when transferring a data block, a technique is disclosed to reduce time overhead and memory waste in transferring a data block. That is, the first prior art initializes the shared storage area in the user area and the kernel area and performs sharing by memory mapping (in this case, additional management such as area allocation and release must be performed). This has the effect of saving memory resources.

By the way, the first prior art provides the effect of cost reduction and performance improvement in general data block transmission, but is not a technology specialized for file movement.

In addition, referring to the Republic of Korea patent "Network card and server implementing the zero-copy transmission function and its transmission method (published No. 10-2005-0065133, published on June 29, 2005) as a second prior art, PCI without copying between memories (Peripheral Component Interconnect) A technology that supports zero-copy transfer using memory and the existing general-purpose Berkeley Software Distribution (BSD) socket application programmable interface (API). Map a region of memory to a user memory region to get a virtual address for that PCI memory region. Thereafter, when a user program requests reading a block of a disk into the PCI memory area, the disk controller reads the disk block and stores data in the PCI memory area using a direct memory approach. When the user program completes reading, it sends a request to the network for the PCI memory area, and the transmission control protocol / Internet protocol (TCP / IP) stack in the kernel responsible for network processing confirms that the address passed by the user program is the PCI memory area. Therefore, only the header is written and the address of the PCI memory area is transferred to the TOE (TCP / IP Offload Engine) network controller. The TOE network controller reads a header from a local memory area and a payload from a corresponding PCI memory area by a direct memory access method, performs a checksum, and then transmits it to the network.

However, the second prior art has the effect of improving the data transmission speed while preventing the performance degradation of the host system by transmitting data to the zero-copy network without an intermediate copying process, while in a general system other than the network. There was a problem that the application between the disks is not easy.

In addition, referring to the Republic of Korea patent "zero-copy message passing method (published No. 10-2000-0038076, 2000.07.05)" as the third prior art, to eliminate the memory copy overhead incurred when message passing between hosts The process for message passing, daemons, and network cards all share the memory used for message passing, thereby minimizing the performance degradation due to memory copying in a networked parallel computing environment.

However, the third prior art can improve the performance of the existing message passing system by eliminating the memory copy overhead incurred when exchanging messages with the remote process, but eliminate the data copying process between the user area and the kernel area in the system. There was no number of problems.

The present invention has been proposed in order to solve the above problems, in order to prevent data copying between the user area and the kernel area executed by the CPU, the buffer page existing in the kernel area is shared to provide a source / purpose. By moving files between devices (e.g. disks, network cards), we provide a data-free file transfer method to improve overall performance by reducing the load on the central processing unit and its associated costs of copying data into the user area. Its purpose is to.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an aspect of the present invention, there is provided a data copyless file moving method comprising: a request receiving step of requesting a file moving from an application program in a user area; An allocation step of allocating a buffer page for a source device to a kernel region; Copying data of a movement target file stored in the source device to the buffer page according to a direct memory access method; A setting change step of removing the buffer page from the page cache and performing a change process on page management information so that the buffer page is allocated for the target device; And a second copying step of copying data of the buffer page to the destination device according to a direct memory approach.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention eliminates the data copying process by the central processing unit between the user area and the kernel area and reduces the number of system call calls, thereby increasing the availability of the central processing unit and improving the overall performance. At this time, the copy process by the direct memory access (DMA) method is excluded from the object of removal.

In addition, the data copyless file moving method according to the present invention may be implemented as a component in a kernel to provide an interface to a user area.

2 is a diagram illustrating an embodiment of a data copyless file moving method according to the present invention.

As shown in FIG. 2, an application program located in a user area requests file movement between devices (disk, network card, etc.). Accordingly, the kernel allocates a buffer page 204 for the source device in the kernel area to read data from a disk or network card that is the source device 201. At this time, the allocated buffer page 204 is added to the page cache of the kernel. The page size data is then copied from the source device 201 to the allocated buffer page 204 by a direct memory approach. Here, when the source device 201 is a network card, a copy process by a direct memory access method is first performed to the socket buffer 205 (207), and then the allocated buffer page 204 from the socket buffer 205 is performed. The copying process by the central processing unit is performed once.

The allocated buffer page 204 is used for the purpose of sending data to the destination device 200 (ie, to reuse the page allocated for the source device for data movement for the destination device). After removing from the page cache and changing the page management information accordingly, it is added to the page cache as the buffer page 204 for the target device. The configuration change process is performed by the kernel. The data of the buffer page 204 is copied to the target device 200 by a direct memory access method. In this case, when the target device 200 is a network card, a copy process by the central processing unit from the buffer page 204 to the socket buffer 206 is performed once, and then the target device 200 from the socket buffer 206. The copy process is performed by the direct memory approach.

The above process is repeated until all data in the movement target file is moved. Here, the buffer page for the source device and the buffer page for the destination device are the same buffer page, and are shared by the source / target device during data input / output.

The operation flow in the above components will be described later with reference to FIG. 4.

3 is an exemplary diagram illustrating an available moving path supported by the data copyless file moving method according to the present invention.

Here, the paths 310, 311, 312, and 313 of file movement to which the data zero-copy file movement method according to the present invention are applied will be described.

The case where the data copyless file movement can be applied includes the case where the movement target file is stored on the disk 303 of the local system and the case where the movement target file is located on another system 304 on the network. When the moving target file is stored in the disk 303 of the local system, the moving target file may be moved to the same disk or another disk 310 or transferred to a network card for transmission 311 to another system on the network.

In addition, when the file to be moved is located on another system 304 on the network, a path for moving the file received through the network card to the disk 312 in the system or resending 313 to another system on the network is possible.

That is, the present invention can provide free file movement between a disk and a network card existing in the system. Here, the source devices 303 and 304 and the destination devices 301 and 302 may be identical to each other.

In order to enable free file movement between the network and the disk without data being copied to the user area as described above, a read module 306 for reading data from the disk, a write module 308 for writing data to the disk, and a network The data processing of each of the receiving module 307 for receiving data from and the transmitting module 309 for transmitting data to the network should be accompanied.

At this time, for the functional connection between the four components (306 to 309), using a descriptor that can identify the device with each other and assign the buffer page allocated for bringing data from the source device to the destination device. Share.

That is, the buffer page allocated for the source device receives some data of the file from the source device, and then transfers the data to the desired destination device without copying by the central processing unit (without copying from the kernel area to the user area). Pass it to the corresponding output processing module to pass the data. In this case, since there is no data copying process by the central processing unit between the user area and the kernel area, the system call call process in the user area is also unnecessary.

Here, the configuration change process (performed by the kernel) for sharing the buffer page 204 will be described in detail.

The configuration change process for sharing the buffer page 204 includes adding and deleting a corresponding page to a page cache of a kernel region, and changing information about each moving target file of the page. .

The read module 306 and the receive module 307 for reading data from the disk / network allocate (add) new pages to the page cache in the same manner as the existing pages when allocating pages. In this case, the reception module 307 calls the transmission module 309 for transmitting data to the network to transmit the data of the shared buffer page 204 and then releases the page added to the page cache. Do it together. This is because page caching is unnecessary in a data transmission / reception process through a network.

Meanwhile, the write module 308 for writing data to the disk releases the corresponding page (which is a buffer page) added to the existing page cache before using the page transferred from the read module 306 or the receiving module 307. . This is to prevent system problems that can occur as the same page is used as a page cache by different devices. Therefore, after releasing the existing page added to the page cache, the process includes changing the data of the page with the file information for the target device. Therefore, when both the source device and the target device are disks, only pages for the target device exist in the page cache.

4 is a flowchart illustrating an embodiment of a method for moving a data copyless file according to the present invention.

First, in response to a request for file movement between devices from an application program, the kernel allocates a buffer page for a source device in the kernel region (401). At this time, the allocated buffer page 204 is added to the page cache of the kernel region.

The kernel copies the data stored in the source device to the allocated buffer page according to a direct memory access method (402). In this case, when the source device is a network card, the data of the moving object file is first copied to the socket buffer by a direct memory access method, and then secondly copied from the socket buffer to the buffer page by the central processing unit.

Thereafter, the kernel determines whether there is no space available in the buffer page or whether all data of the moving target file has been copied to the buffer page (403). As a result of the determination 403, when there is no space available in the buffer page or when all data is copied, the process proceeds to the next step. On the other hand, if the available space remains in the buffer page or the data copy is not completed, the determination result 403, after adjusting the offset of the buffer page (404), copying data from the source device ( Proceed to 402).

Thereafter, the kernel removes the buffer page from the page cache of the kernel region and performs a change process on page management information (405). This is to prevent a system problem caused by sharing the buffer page. As a result of performing the configuration change process, the buffer page is added to the page cache as a buffer page for the target device while maintaining the changed information.

Thereafter, the kernel copies the data of the buffer page to the target device according to a direct memory approach (406). In this case, when the target device is a network card, the data is first copied to the socket buffer by the central processing unit, and then secondly copied from the socket buffer to the target device by a direct memory approach.

Thereafter, the kernel determines whether all data in the movement target file has been copied from the source device to the destination device (407), and if the copy is completed (results return to the user area), if the copy is not completed After changing the size of the total data to be processed (408), the process proceeds to step 401 of allocating a buffer page for the source device.

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form. Since this process can be easily implemented by those skilled in the art will not be described in more detail.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

As described above, the present invention eliminates the process of copying data between user space and kernel space, and reduces the number of system call calls and consequent switching of a process, thereby increasing the availability of a central processing unit (CPU) and improving overall performance. It can be effected.

In addition, the present invention does not create, manage, or use a separate designated shared memory area, but allocates and uses a shared buffer page in the kernel area, thereby reducing the additional cost burden.

Claims (5)

In the method of moving a dataless copy file, A request receiving step of requesting a file move from an application program in a user area; An allocation step of allocating a buffer page for a source device to a kernel region; Copying data of a movement target file stored in the source device to the buffer page according to a direct memory access method; A setting change step of removing the buffer page from the page cache and performing a change process on page management information so that the buffer page is allocated for the target device; And A second copy step of copying data of the buffer page to the destination device according to a direct memory approach How to move a data copy-free file that includes. The method of claim 1, The setting change step, Deleting a buffer page from the page cache of a kernel, adding a new buffer page, and changing data of the added buffer page with file information for the target device. The method according to claim 1 or 2, The source device and the target device, A method of moving a dataless copy file, characterized in that it is either a disk or a network card of a local system. The method of claim 3, wherein The first copy step, As the source device is the network card, data of the moving object file stored on the network card by the direct memory access method is copied to the socket buffer, and then the data stored in the socket buffer by the central processing unit. Copying the data to the buffer page. The method of claim 3, wherein The second copy step, As the target device is the network card, data stored in the buffer page by the central processing unit is copied to the socket buffer, and then data stored in the socket buffer by a direct memory approach. And copying the data to the network card.

Images