KR100654190B1 - communication interface method for controlling actions between Socket interface and TOETCP offload Engine on Kernel - Google Patents

Abstract

The present invention relates to a communication interface method for controlling a connection between a socket interface used in an application and a TCP offload engine (TOE).

The present invention can implement a TOE socket interface that is fully compatible with the existing socket interface.

Therefore, the general socket switch layer and the offload protocol processing layer that do not depend on a specific TOE can be manufactured to support a compatible socket interface on various TOEs. In addition, the data copying function implemented in the offload protocol processing layer can improve the network performance of the server using the TOE.

TCP Offlond Engine (TOE), Kernel Sockets, Socket Switch Layer, Offload Protocol Layer, TOE Device Driver

Description

Communication interface method for controlling actions between Socket interface and TCP offload engine (TOE) on Kernel}

1 is a view showing the configuration of a conventional network system.

2 is a diagram illustrating a software hierarchy structure for communication in a conventional network system.

3 is a diagram illustrating a software hierarchy for supporting a conventional TOE socket interface.

4 and 5 are diagrams for explaining a method of reducing CPU burden by not copying data during protocol processing.

6 is a diagram illustrating a software layer structure including a kernel level socket layer according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating an interface defined in each layer of FIG. 6.

8 is a diagram illustrating socket data transmission at a kernel level socket layer according to an embodiment of the present invention.

9 and 10 are diagrams for explaining a data copyless structure on the structure of FIG. 6 according to an embodiment of the present invention.

The present invention relates to a method for processing computer network protocols, and more particularly, to a communication interface method for efficiently processing a network protocol in a network computer server to increase server performance, increase throughput, and provide a standard socket interface on a network interface.

1 is a diagram showing a general configuration of a computer network system.

As shown in FIG. 1, a conventional computer network system includes one or more nodes 10, each node 10 having separate operating systems that are independent of each other, and each performing a separate application program. Each of the nodes 10 separately operated as described above is connected to the network 20 through a network interface card (NIC) 11, and in order for the applications executed in each node to communicate with each other, the application programs. Use a communication interface that can be used to write

These communication interfaces are widely used sockets interface developed by the University of California, Berkeley, the software layer that passes through the network system of the conventional Linux operating system when communicating between nodes using these socket interfaces The structure is shown in FIG.

2, a socket interface 30 is located at the top of the software hierarchy, below which the system call layer 40, kernel socket layer 41, TCP / UDP 42, and IP 43 are located. Are sequentially located, and at the bottom is a network driver 44 for driving the NIC 45. At this time, the socket interface 30 is configured in the form of a library at the user level and is provided for each application program. The system call layer 40 and the layers below the socket interface 30 are called when the socket interface 30 is called by the corresponding application program. Software module used to process.

That is, when an application program of the node 1 (10) calls the socket interface 30, the socket interface 30 passes through the system call layer 40 belonging to the operating system kernel area of the node 1 (10). Pass the call command to layer 41. The call command is then passed to the network driver 44 via the TCP / UDP 42 and IP 43 layers, whereby the NIC 45 is driven by the network driver 44.

In this process, the kernel copies data at the user level into the kernel buffer, and the data of the kernel buffer is taken by the NIC 45 through direct memory access (DMA). Only after this series of software layers will the application be able to send data over the network. However, it is known that memory copying performed when TCP protocol processing and protocol processing in the software layer as described above occupy the most CPU and cause network performance degradation.

The following describes how to reduce the burden on the CPU that causes such network performance degradation.

3 is a diagram illustrating a software structure for supporting a conventional TOE socket interface.

The method used to reduce the burden on the conventional CPU is to make TCP protocol processing a separate hardware (TOE: TCP Offload Engine).

As shown in FIG. 3, since a network interface device such as the TOE 52 is not supported by Linux, the TOE provider creates a separate TOE device driver 51, and the TOE 52 on the TOE device driver 51. Must provide a socket interface that applies only to Meanwhile, in order to develop the TOE 52 and provide a socket interface applicable only to the TOE 52, both the TOE device driver 51 and the socket module, that is, the TOE support module 50 in FIG. 3 must be developed. The burden arises. In addition, the TOE support module 50 has a problem that is incompatible with the standard socket interface.

4 and 5 are diagrams for explaining a method of reducing CPU burden by not copying data during protocol processing.

4 is a method of avoiding memory copy by allowing a user to directly access the memory of the NIC.

However, the application program must allocate the memory of the NIC as a socket buffer and use the address of the buffer when sending and receiving sockets. In addition, an application must use the special APIs provided by the NIC to use the NIC's memory.

5 illustrates a method of avoiding memory copying between a user and the kernel by using a memory shared by the kernel and the user.

However, similar to the case of FIG. 4, the application program should allocate shared memory as a socket transmit / receive buffer and use a separate API for this purpose.

As described above, according to the method for reducing the burden on the CPU, a separate API must be used to use the TOE device driver 51 or to use a specific memory, thereby causing compatibility problems between applications. In addition, when the above method is applied to an existing socket application program, a special API must be modified to be used. In addition, when the method of FIG. 5 is used, the socket buffer must be additionally arranged in units of pages.

An object of the present invention is to provide a communication interface method for controlling a connection between a socket interface used in an application program and a TOE so as to implement a TOE socket interface compatible with an existing socket interface.

In the communication interface method for controlling a connection between a socket interface in the application and a TCP Offload Engine (TOE) according to an aspect of the present invention for achieving this object, the socket is a socket switching layer, an offload protocol processing layer, And a TOE device driver layer. The method includes: a) when a user calls a socket interface, determines whether the socket currently used by the socket switch layer is connected to the TOE, and according to the determination result. Selecting a protocol stack; b) when a TOE protocol stack is selected in the socket switch layer, converting a request of a user transferred from the socket switch layer into a descriptor form in the offload protocol processing layer; c) receiving the descriptor from the offload protocol processing layer at the TOE device driver layer, storing the descriptor in a specified memory of the TOE, and returning control to the offload protocol processing layer.

delete

delete

delete

According to another aspect of the present invention, a communication interface method using a kernel-level socket for controlling a connection between a socket interface in an application program and a TOE is provided. Receiving an address of the memory through a socket transmission / reception interface from an application to which the memory is allocated; Converting the address of the received memory into an actual memory address that can be recognized by a DMA engine of a TOE; And transmitting the converted real memory address to the TOE.

delete

delete

delete

delete

delete

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention. Like parts are designated by like reference numerals throughout the specification.

Hereinafter, a kernel level socket layer and a communication interface method using the same will be described in detail with reference to the accompanying drawings.

6 is a diagram illustrating a software layer structure including a kernel level socket layer according to an embodiment of the present invention.

Referring to FIG. 6, the socket interface 100 is positioned at the top of the software hierarchy, and the system call layer 200 and the BSD socket layer 210 are sequentially positioned below the software interface. Next, the TOE device driver 280 and the offload protocol layer (OP layer) 270 for driving the TOE 300 are sequentially located at the lowermost level, and the NIC device driver for driving the NIC 400 ( 250), TCP / IP 240 and INET socket layer 230 are sequentially located. In addition, a socket switch layer 260 having a function of selecting each protocol processing stack passing through a layer from a system call layer of a kernel to the TOE device driver 280 or the NIC device driver 250 to implement the socket interface. ) Is located.

As described above, under a software hierarchy including a kernel level socket layer according to an embodiment of the present invention, an application program may use an existing socket interface as it is, and a kernel level system call layer 200 and a BSD socket layer 210. When the call command is transmitted through the ()), the socket switch layer 260 may turn off or turn off the existing protocol stack passing through the NIC device driver 250 at the INET socket layer 230 depending on whether the socket 300 is currently connected to the TOE 300. The TOE protocol stack passing through the load protocol layer 270 and the TOE device driver 280 is selected. As such, the switch layer is limited to having only the ability to select a protocol stack.

Meanwhile, the offload protocol processing layer 270 is a layer that implements a function independent of a specific TOE. The offload protocol processing layer 270 includes functions such as a data non-copy function, a mapping between a user socket and a TCP session of the TOE. The offload protocol processing layer 270 implements an additional function for the TOE 300 and is a means for overcoming semantic differences between the socket interface and the TOE driver interface. The TOE driver interface provides only a very low level interface and is very different from the socket interface so that the switch layer 260 is not suitable for direct calls.

The TOE device driver layer 280 implements the interface provided by the TOE 300 in software. The interface provided by most of the TOE 300 is a form of posting a socket, data transmission, and the like to a memory shared with the TOE 300 and notifying the processing result through a hardware interrupt. The TOE device driver 280 implements this hardware interface as a software interface for the offload protocol processing layer 260 to use. Thus, the initiative to store or retrieve descriptors in shared memory resides in the offload protocol processing layer 260.

In addition, when the transmission of the data posted to the shared memory is completed, a hardware interrupt occurs and the TOE device driver 280 performs the minimum function in the interrupt handler, and then completes the information from the shared memory and the remaining socket related processing is turned off. The load protocol processing layer 280 is in charge.

FIG. 7 is a diagram illustrating an interface defined in each layer of FIG. 6.

When a user invokes a socket interface, the call is connected to an interface with the same prefix and different name and argument, up to socket switch layer 260 and offload protocol layer 270. The offload protocol layer 270 creates a TOE request descriptor based on the meaning of the called function and the passed arguments, and delivers the descriptor to the TOE device driver 280. The TOE device driver 280 refers to the descriptor and classifies the socket call into a command message such as socket, bind, listen, connect, accept, and the like, and a data transmission / reception message such as sendmsg and recvmsg. For example, toe_post_cmd delivers command descriptors, and toe_post_send and toe_post_recv deliver data transmission and reception descriptors to the TOE 300, respectively.

8 is a diagram illustrating socket data transmission at a kernel level socket layer according to an embodiment of the present invention.

When an application calls the socket interface, that is, the socket send function send (socket_id, socket_buffer, buffer_size), the socket switch layer 260 determines whether the socket (socket_id) currently being used is connected to the TOE 300. do. As a result, if the socket is connected to the TOE 300, the function call of the user is transferred to the offload protocol processing layer 270 as it is. Then, the offload protocol processing layer 270 converts the user's transmission request into a descriptor form (op_sendmsg) and transmits the request to the TOE device driver 280. Next, the TOE device driver 280 stores this descriptor (toe_post_send) in shared memory with the TOE 300 and returns control to the offload protocol processing layer 270. The offload protocol processing layer 270 waits for the request process until the requested descriptor processing is completed and a predetermined completion processing function is called. When the TOE 300 receives the hardware interrupt, it calls the completion processing function, takes the descriptor stored in the completion queue, and wakes up the process waiting for completion.

9 and 10 are diagrams for describing a data non-copy structure on the structure of FIG. 7 according to an embodiment of the present invention. In particular, FIG. 9 is a diagram illustrating a copyless structure when the start addresses of data are aligned, and FIG. 10 is a diagram illustrating a copyless structure when the start addresses of data are not aligned.

First, referring to FIG. 9, an application program allocates an arbitrary user level virtual memory and transfers the memory to the offload protocol layer 270 through a socket transmission / reception interface. Then, the offload protocol layer 270 converts the memory into an actual memory address that can be recognized by the DMA engine of the TOE 300 and transfers the memory to the TOE 300.

When the user-specified memory is larger than the page or crosses the page boundary, the TOE 300 cuts the real memory based on the page boundary and generates a list (sg list, scatter / gather list) of actual memory addresses and memory sizes. Create and deliver to TOE 300. Since this memory must stay in real memory until the transmission and reception is completed, the page map table must be searched for all pages included in the list, and the reserved bit of the page table of the corresponding page must be set.

On the other hand, if the actual address of the address specified by the user does not satisfy the byte alignment requested by the TOE 300 will be described below with reference to FIG.

Referring to FIG. 10, when the actual address of the address specified by the user does not satisfy the byte alignment requested by the TOE 300, only the misaligned portion of the front part of the memory is stored and processed in the aligned kernel memory.

The above method can be applied to data transmission and reception, and since the TOE 300 manages a buffer necessary for processing the TCP protocol, utilizing this buffer can avoid memory copying at the time of reception as well as transmission for the sorted user memory. .

Although the embodiments of the present invention have been described in detail above, the present invention is not limited thereto, and various other changes and modifications are possible.

According to the present invention, a general socket switch layer and an offload protocol processing layer that do not depend on a specific TOE can be manufactured to support compatible socket interfaces on various TOEs. In addition, the data copying function implemented in the offload protocol processing layer can improve the network performance of the server using the TOE.

Claims (14)

In a communication interface method in which a socket controls a connection between a socket interface in an application program and a TCP offload engine (TOE), The socket is implemented in a form including a socket switching layer, an offload protocol processing layer, and a TOE device driver layer, a) when the user calls the socket interface, determining whether the socket currently being used by the socket switch layer is connected to the TOE, and selecting a protocol stack according to the determination result; b) when a TOE protocol stack is selected in the socket switch layer, converting a request of a user transferred from the socket switch layer into a descriptor form in the offload protocol processing layer; c) receiving the descriptor from the offload protocol processing layer at the TOE device driver layer, storing the descriptor in a specified memory of the TOE, and returning control to the offload protocol processing layer. Communication interface method comprising a. The method of claim 1, In step b), the offload protocol processing layer provides a descriptor to the TOE device driver, and performs a task of retrieving the processing result of the provided descriptor from the TOE device driver. The method of claim 2, The offload protocol processing layer waits for a user's request process until a task corresponding to a descriptor corresponding to a user's transmission request stored in a specified memory of the TOE is completed and a predetermined completion processing function is called. Communication interface method. The method of claim 3, The offload protocol processing layer calls a completion processing function when the TOE receives a hardware interrupt, takes a descriptor stored in a completion queue, and wakes up a process waiting for completion. The method of claim 1, And in the TOE device driver layer, when a task corresponding to a descriptor stored in a specified memory of the TOE is completed and a hardware interrupt occurs, retrieving completion information from the designated memory. The method of claim 5, The TOE device driver layer is a communication interface method, characterized in that the software implemented by the interface provided by the TOE. The method of claim 5, The descriptor is generated according to the meaning of the called function and the passed argument. The method of claim 1, If the user calls a socket interface in step a), the call is connected to an interface that differs only in prefix and has the same name and argument across the socket switch layer and offload protocol processing layer. The method of claim 1, And said offload protocol processing layer includes a data copy-free function and a mapping function of a user socket and a TCP session of said ROE. In the communication interface method using a kernel-level socket that controls the connection between the socket interface in the application and the TOE, In the offload protocol processing layer of the kernel-level socket, Receiving an address of the memory through a socket transmit / receive interface from an application program allocated to any user level virtual memory; Converting the address of the received memory into a real memory address that can be recognized by a direct memory access (DMA) engine of a TOE; And Delivering the translated real memory address to the TOE Communication interface method comprising a. The method of claim 10, If the memory specified by the user is larger than the page or crosses the page boundary, the communication interface is characterized by cutting the page based on the page boundary and creating a list of the actual memory address and the memory size and transmitting the same to the TOE. Way. The method of claim 11, Wherein the list is an sg list and a scatter / gather list. The method of claim 11, Search for a page map table for all pages included in the list to set the reserved bit of the page table of the corresponding page so that the user stays in the real memory until the transmission and reception of the memory designated by the user is completed. Communication interface method. The method of claim 11, If the actual address of the memory address specified by the user does not satisfy the byte alignment requested by the TOE, only the misaligned portion of the memory is stored in the aligned kernel memory and processed.

Images