DE202017105777U1 - System for hardware-independent RDMA - Google Patents

Abstract

A system comprising: a memory storing packet processing instructions; and one or more dedicated packet processing units in communication with the random access memory and with one or more application processing units, wherein the one or more dedicated packet processing units are dedicated to perform packet processing, wherein each of the one or more dedicated packet processing units is assigned a processing thread, the one or the plurality of dedicated packet processing units are configured to: receive a packet from an application executing in one of the application processing units in a first processing thread; processing by software defined handler operations initiated by instructions in the received packet without the operation to submit to an application thread in the first processing thread.

Description

BACKGROUND

Some high-performance computer systems use network interface controllers (NICs), which enable CPU-efficient communication with low latency between machines. In particular, Remote Direct Memory Access (RDMA) technology is a technique in which a NIC hardware device implements sufficient functionality to allow one machine to efficiently access the memory of another machine without the overhead of Suffer from universal software networking stacks running in shared host processors. NICs with this functionality are sometimes referred to as "smart NICs" because they implement much more functionality than a generic NIC that simply sends and receives packets generated by the host. The functions of a smart NIC include reliable transport in the NIC as well as a series of "activities" that implement specific functions such as an RDMA read and an RDMA write.

Universal host networking stacks running on legacy NICs are unable to provide the same low-latency credentials as dedicated hardware-enabled smart NICs. First, kernel-based networking stacks are primarily designed to time-multiplex the cores with other applications and services. Because of interrupt handling and context switching with unpredictable scheduler delays, time-division multiplexed use of a processor inevitably results in a latency impact. Second, time-multiplexed networking stacks must be invoked using an operating system call, which is a non-trivial overhead. Third, conventional host networking stacks do not implement activities to perform well-defined functions such as RDMA read and RDMA write inline with the transport. Instead, they must make an incoming request to an application thread, and the action of dropping an application thread causes significant CPU overhead and significant CPU latency.

Hardware-based Smart NICs have several issues. First, they are inflexible against change. It is either difficult or impossible to extend the NIC with new activities, new overload defense algorithms, or bug fixes. Second, hardware-based smart NICs often have different requirements for the surrounding systems, such as requiring a lossless network fabric.

In accordance with the requirements of the utility model law to protect the utility model and subject of the utility model are only devices as defined in the appended claims, but no method. Where the description refers to methods, these references are merely illustrative of the device or devices for which protection is sought from the appended claims.

SUMMARY

The present disclosure provides a computer system that devotes one or more processing units to package software, with other processing units simultaneously executing application software. In some examples, the system uses dynamic load information to dynamically increase and decrease the number of processing units dedicated to packet processing. The system may use signals such as queue length, queue enqueue latency and share of busy processing loops to detect overload conditions. Further, the system may include a mechanism for establishing shared memory areas to interact with other applications. The shared memory mechanisms create an abstraction of a per-application "command" and per-application "completion queues". The system can query pro-application command queues to detect the arrival of new requests. In addition, the mechanism provides detection of application termination as well as an ability for an application to expose portions of its address space for receipt and transmission of data. In some examples, the system further includes a framework for executing software-defined handlers, such as "tasks" in-line with threads executing packet processing and transport software. The handlers can implement RDMA capabilities. Further, the system may implement packet capture, packet processing, software-defined handler processing, and reliable transport operation, all within the same thread.

One aspect of the disclosure provides a system that stores a memory that stores packet processing instructions and one or more dedicated packet processing units in conjunction with the memory and with one or more application processing units contains. The one or more dedicated packet processing units are dedicated to performing packet processing, and each of the one or more dedicated packet processing units is assigned a processing thread. The one or more dedicated packet processing units are for receiving a packet from an application executing in one of the application processing units in a first processing thread and for processing by software-defined handler operations initiated by instructions in the received packet without Send operation to an application thread configured in the first processing thread. In some examples, the one or more processors are further configured to perform a transport for the received packets and to perform a transport for processed packets to be sent, wherein the carrying out of the transports is performed in the first processing thread. The transport may be connected to the one or more application processing units without context switching or system calls via an interface. Furthermore, the one or more processors may be configured to query one or more command queues, each command queue being specific to an application, the received packet being staged by the one or more command queues, and configured to enter completions into a completion queue. The command and completion queues may be included in a shared memory where the shared memory is shared between the one or more processing units and a second processing unit specific to the application. The one or more dedicated packet processing units may be processing cores of a multi-core processor. In some examples, such a multi-core processor resides in a network interface controller (NIC) interfaced with an engine that includes a memory that stores the assigned threads.

In another aspect of the disclosure, a system is provided that includes one or more random access memories, a first set of processing units in communication with the one or more random access memories, wherein the first set of processing units are configured to execute exclusively the packet processing, and a second Amount of processing units associated with the one or more random access memories and with the first set of processing units, wherein the second set of processing units is configured to execute only applications. Each of the first processing units is assigned a processing thread, the processing thread being executable by one of the first processing units to process received packets, carry out a transport for the received packets, process software-defined handler operations, and one for packets to be sent Transport. Further, the system may include a shared memory such as shared memory rings between the first set of processing units and the second set of processing units. In some examples, each of the second set of processing units communicates via a separate set of shared memory rings, the set comprising transmitter rings and receive rings. Further, the shared memory may include at least one instruction queue and at least one completion queue. The first set of processing units may be configured to query instruction queues for each application and to enter completions after completion of packet processing operations in the completion queues.

In yet another aspect of the disclosure, there is provided a method of receiving a packet from an application executing in a second processor dedicated for application processing, having a first processor operative to perform packet processing and for is dedicated to executing a first processing thread, and processing by software defined handler operations initiated by instructions in the received packet, without invoking additional threads, in the first processing thread. Further, the method may include performing a transport for the received packets with the first processor executing the first processing thread and performing a transport for processed packets to be sent with the first processor containing the first processing thread executes included.

list of figures

• 1 FIG. 10 is a block diagram of an exemplary system in accordance with aspects of the disclosure. FIG.
• 2 Figure 12 is a block diagram of an example engine of the system 1 ,
• 3 FIG. 10 is a schematic diagram illustrating communications using shared memory in accordance with aspects of the disclosure. FIG.
• 4 FIG. 13 is a functional diagram illustrating accessing memory in accordance with aspects of the disclosure. FIG.
• 5 FIG. 10 is a schematic diagram illustrating communication between two exemplary systems in accordance with aspects of the disclosure. FIG.
• 6 illustrates an exemplary package in accordance with aspects of the disclosure.
• 7 FIG. 10 is a flowchart of an example method in accordance with aspects of the disclosure. FIG.
• 8th illustrates a processing loop partition processing time among activities in accordance with aspects of the disclosure.

DETAILED DESCRIPTION

Overview

The disclosure is directed to a system including a plurality of processors, such as processing cores, wherein one or more first processors of the plurality are dedicated to packet processing software and wherein one or more second processors of the plurality are dedicated to executing application software. The one or more first processors dedicated to packet processing may be universal cores. A suitable number of dedicated processors are assigned one or more software threads. Each processing thread executes the following actions in a busy processing loop, but not necessarily in the specified order: processing incoming packets, performing a reliable transport for received packets, processing by software defined handler operations (e.g., task operations), executing reliable transport for transmission packets, command queue queries for each application. Thus, z. For example, a thread such as by monitoring a network problem (e.g., congestion, failure, delay, etc. signals), changing transport parameters in response thereto, and notifying any packet loss in the transmission, will send receive queues (Rx queues) to incoming packets and it performs a reliable transport for the incoming packets from the Rx queue. In some examples, the thread that performs reliable transport for inbound packets from the Rx queue, such as receiving a NAK or a duplicate ACK, could trigger retransmission of lost packets. Executing the reliable transport generates acknowledgments and schedules the processing of task operations. For example, for each operation operation, a particular state machine may be selected to process that operation. Executing the activity operations may include scheduling transmission packets (Tx packets). For the Tx packets, such as by generating acknowledgments and retransmitting lost packets, reliable transport is also performed. The thread asks one or more shared memory command queues for each application that communicates with the NIC. B. on additional to be processed commands. While any of these actions are being performed, the thread may also write completions to notify applications of a completed work. For example, the state machine may write metadata to shared memory connections that are shared among the dedicated processor and the one or more second processors.

By running in dedicated processing cores, a kernel is always available for immediate processing of incoming packets. Packets can be sent to and received from the hardware NIC without system call overhead and interrupts. The software threads can be executed in a privileged or in an unprivileged mode.

For interaction between the NIC and applications, command and completion queues may be used, and they may be non-blocking. An application wishing to use the NIC establishes communication with the NIC via an intra-machine connection using an existing communication primitive such as a Unix domain socket or the like via an initial channel. When this initial channel has been established, the application uses operating system mechanisms to build a shared memory area with the NIC.

A basic structure is implemented to perform activities inline with a transport processing thread. The basic structure is designed with extensibility so that application developers can create new activities without significant rework. The wire format contains an "operation code" to specify the operation to be performed.

Exemplary systems

1 represents a system 100 the application processors 110 . 120 in communication with a NIC 150 about shared random access memory 132 . 134 contains. The system 100 can be implemented as a system of devices on a motherboard or as any other configuration of electronically coupled computing devices. For example, the system may include a backplane interconnecting peripherals, an expansion card interface, or the like. The connection between the computing devices may be hard-wired connections, wireless connections, or any other type of connection. As yet another example, the computer system 100 TCP / IP, 802.33, Ethernet, InfiniBand, or any other type of network.

Every application processor 110 . 120 may be dedicated to running application software. Every application processor 110 . 120 can run one or more applications and the applications running in one processor may be different from the applications running in other processors. For example only, the processor 110 run a remote procedure call (RPC) application while the processor is running 120 running a video streaming application. Although only two processors 110 . 120 It should be understood that any number of processors may be included and that they may carry any number of different applications.

Each application can have an associated client library 112 . 122 have. The client libraries 112 . 122 can code, z. For example, for the applications to interact with application programming interfaces (APIs). The APIs can z. For example, in-memory storage systems that use RDMA, RPC, or any other type of API may be.

The shared memory 132 . 134 can z. Shared memory rings or any other type of memory. The shared memory 132 . 134 can provide command-line and completion queues based on memory. The shared memory is in conjunction with the following 3 described in more detail. In some examples, the shared memory is 132 . 134 supported by a temporary file store. In addition, the shared memory 132 . 134 Exclude application data payloads. Instead, application data can be sent to the NIC 150 Memory registration, shared interprocess memory, anonymously supported heap memory, and so on, to the NIC 150 be transmitted.

The engine 145 may be a packet processing pipeline for a particular function anchored to a reserved core. The engine 145 can z. In a framework supporting high performance packet processing entities. The engine 145 is in connection with 2 further described.

The engine 145 is interfaced to one or more input / output (I / O) drivers 160 which provide means for interfacing with hardware components, for communicating between threads, and / or for communicating with other user space processors. The I / O drivers 160 may all implement a same unified API. Although multiple I / O drivers 160 are shown, any number of I / O drivers may be included. In some examples, the number of I / O drivers 160 may be a number of cores in the NIC 150 correspond.

The engine 145 contains one or more processing cores, each with a dedicated thread. The one or more processing cores may be dedicated for packet processing. Using dedicated cores offers several performance benefits. For example, dedicated cores save significant overhead required by time-multiplexing a core. In addition, dedicated cores provide higher microarchitectural performance as caches, translation lookaside buffers (TLBs), and branch predictors remain warmer. Again, dedicated cores avoid significant synchronization overhead associated with shared data structures and work distribution.

2 puts the engine 145 As shown, the engine contains 145 a send queue (Tx queue) 252 , a receive queue (Rx queue) 254 and a plurality of processing cores 270 . 280 . 290 , The Tx queue 25 and the Rx queue 254 can be used to send and receive packets over the NIC to a network. For example, the Tx queue 252 and the Rx queue 254 Be part of the I / O driver 160 above in conjunction with 1 connected is. Furthermore, the engine contains 145 a working memory 260 and one or more processors 210 in connection with the working memory 260 ,

The working memory 260 stores information by the processor 210 can be accessed, including instructions 264 that through the processor 210 can be executed. In addition, the memory contains data 262 which can be read out, manipulated or stored by the processor. Of the Memory may be any type that can store information that can be accessed by the processor, such as a hard disk drive, a ROM, a RAM, a DVD, a CD-ROM, a writable random access memory, and a read-only memory , The working memory 260 contains data 262 by the processor in accordance with the instructions 264 can be read out, manipulated or stored. Although the system and method are not limited by any particular data structure, the data may e.g. For example, in computer registers, in a relational database as a table with several different fields and records, or as XML documents. In addition, the data may be formatted in any computer-readable format such as, but not limited to, binary, ASCII, or Unicode. In addition, the data may contain any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other random access memory (including other network locations), or information used by a function are included to calculate relevant data.

The instructions 264 may be any set of instructions to be executed by the processor directly (such as machine code) or indirectly (such as scripts). In this regard, the terms "instructions", "steps" and "programs" may be used interchangeably herein. The instructions may be stored for direct processing by the processor in object code format or in any other computer language that may contain scripts or collections of independent source code modules that may be interpreted or compiled in advance as needed.

The one or more processors 210 may be a commercially available multicore processor containing a plurality of processing units. In accordance with other examples, the one or more processors may 210 a collection of specialized or commercially available single processing units such as application specific integrated circuits (ASICs), microprocessors, state machines, etc. Although the majority of the processing cores are shown as being within the same box of the engine 145 It should be noted that the cores may be single cores of processing located in separate devices. For example, the cores may reside in a NIC, in an accelerator device, independently, and so on.

Every core 270 . 280 . 290 owns an assigned thread 275 . 285 . 295 , Every thread 275 . 285 . 295 In a busy processing loop, performs the following actions, but not necessarily in the specified order: processing incoming packets, performing a reliable transport for received packets, processing task operations, performing reliable transport for transmit packets, querying command queues for each application. Thus, a thread performing a function, such as processing incoming packets, may also process an operation without forming another thread.

The through the threads 275 . 285 . 295 Processed operations may be any logical actions that interact with remote resources. The operations may be initiated by instructions, although not all instructions initiate operations. For example, some commands may change a local state. Although operations begin with an instruction that expresses an intent of an application user, the operations may include a wide range of resources including local and remote condition tracking objects, memory, wire messages, communication channels conveying messages, and finally one or more completions performed by local and remote engines are generated. Some example operations include remote memory access reads (RMA reads) and memory access writes (RMA writes), atomic operations such as compare and swap operations (CAS operations), and fetch and append operations or send and receive operations.

In accordance with an example, dynamic load information may be used to dynamically increase or decrease a number of processing units dedicated for packet processing. For example, a first number of cores can execute processes while the engine 145 monitors a load on the first number of cores. The system may use signals such as queue length, queue enqueue delay, and busy processing loop fraction to detect overload load conditions. If the engine 145 detects that the load is approaching or exceeding a threshold, it may assign another core to share the load. In assigning the further core, a further engine can be generated and a part of the load of the first number of cores can be migrated into the further core. Conversely, it may be determined that more cores than necessary are allocated for packet processing if the engine detects that a load in one or more of the first number of cores is below a different threshold. Accordingly, the assignment of one or more several of the first number of cores can be detached and their load can be migrated to one of the remaining cores or shared among several of the remaining cores. In this regard, cores are dynamically dedicated over coarser timescales than in existing stacks that use interrupts. Furthermore, no other tasks can be scheduled for a resource if it is dedicated to packet processing. Rather, the kernel is assigned exclusively for a known, ongoing period of time.

Although 2 the processor 210 and the memory 260 As is functional within the same block, it will be appreciated that the processor and memory may actually include multiple processors and memory that may or may not reside within the same physical enclosure. For example, some of the instructions and data may be stored in a removable CD-ROM and others within a read-only computer chip. Some or all of the instructions and data may be stored in a location provided by the processor 210 is physically remote, but he can still access. Similarly, the processor 210 actually include a collection of processors that can work in parallel or not in parallel.

Although 2 the engine 145 In addition, as a single block containing its own processors and its own memory, the operations described herein may also involve many computer devices, e.g. In the "cloud". For example, various operations, which are described below as including a single computing device, may include a plurality of computing devices, such as multiple processors in a load balanced server farm. Similarly, memory components at different locations may have different portions of instructions 264 store and together form a medium for storing the instructions. Various operations described herein as being performed by a computing device may be performed by a virtual machine. The instructions can be exemplary 264 be specific to a first type of server, but the relevant operations are performed by a second type of server executing a hypervisor that emulates the first type of server. In addition, the operations by a container, for. By a computer environment that does not rely on an operating system tied to specific hardware types.

3 provides a more detailed example of the shared memory between the client library 312 and the engine 345 The shared memory contains one or more shared memory rings. As shown, the shared memory contains z. For example, a shared Tx memory ring 332 and a shared Rx memory ring 334 , In accordance with some examples, each client library in the plurality may use a separate pair of shared memory rings where a plurality of client libraries are connected to the engine 345 communicate.

Initially, the client library builds 312 via a control plane communication channel (CPCC) 314 a communication with the engine 345 on. The CPCC 314 can use operating system services provided by an operating system. The CPCC 314 can z. B. be an RPC system. The CPCC 314 may be based on a Unix domain socket or other interprocess communication system (IPC system) that provides the ability to pass file descriptors over the socket. Accordingly, the memory, for. By sharing a file descriptor / tmpfs among processors. If the communication between the client library 312 and the engine 345 via the CPCC 314 has been established, the CPCC 314 used to share the shared memory rings 332 . 334 build. The CPCC 314 can be considered as a slow path while the shared memory rings 332 . 334 be regarded as a fast way.

The shared Tx memory ring 332 and the shared Rx memory ring 334 can be shared memory rings. The shared Tx memory ring 332 and the shared Rx memory ring 334 can be supported by a temporary file memory memory. For example, the shared Tx memory ring 332 through a temporary file store 372 supported during the shared Rx memory ring 334 through a temporary file store 374 is supported. In other examples, a backup storage unit may include both the shared Tx memory ring 332 as well as the shared Rx memory ring 334 support.

About the shared memory rings 332 . 334 find data path operations. The shared memory rings 332 . 334 provide memory-based command and completion queues. For example, the shared Tx memory ring 332 serve as a command queue and can the shared Rx Memory ring 334 serve as a completion queue. The shared Tx memory ring 332 can only support a single producer thread. The shared Rx memory ring 334 can only support a single consumer thread. The shared memory rings provide clients with a non-blocking mechanism to interact with the engine 345 co. Between the shared Tx memory ring 332 and the shared Rx memory ring 334 no synchronization is required.

In accordance with an example, the client library tries 312 to connect across all containers in a machine with a well-known name in a global namespace. As part of a handshake of the CPCC 314 a client API version number is exchanged. The engine 345 implements the API version number of any applications in use.

The shared memory rings can store messages of variable length and store them out of them. As in 3 is shown, the client writes to the shared Tx memory ring 332 z. For example, commands issued by the engine 345 to be read. If the engine 345 Coordinated with the NIC executing instructions initiated by the commands, it writes completions to the completion queue in the shared Rx memory ring 334 ,

In some examples, a backpressure may be applied to the client application when the command queue becomes full. For example, the engine 345 be occupied, in which case it is blocked that the client stores more work. If the client application stops, completions from the engine 345 Storing the engine stops 345 pending further commands from the command queue until space is available in the completion queue.

4 provides an example of how client applications describe local memory for the engine running in different processes. Areas such as an area 449 can be used to allocate memory in an address space 447 the engine 445 to identify.

The engine 445 Lists the local memory over area objects, which can be a base pointer and a length. In some examples, applications may create areas by sharing shared memory 419 in a client process and then re-map this memory in the engine process. In other examples, applications may have existing memory at the engine 445 to register. If the client 410 one or more areas 449 Describing the local memory of the engine 445 an area handling access number and an {offset, length} within that area 449 require. Virtual addresses can be ignored via process limits.

In accordance with an example, the client saves 410 a command in the engine 445 The command initiates and initiates operations such as RMA reads or RMA writes, atomic operations, or transmit and receive operations. The command can be the area 449 of the working memory. For example, RMA reads and RMA writes both specify a remote area of memory in an address space 417 of the client 410 as well as a local area 449 of the working memory in the address space 447 the engine 445 , To allow a remote application to the area 449 can access the working memory area 449 one or more windows 475 be appropriate to explicitly grant access to this memory. The window 475 can also be used to manage a fine-grained access control. If an application wishes a remote application to be able to read a portion of its local memory, it will create a window bound to a previously created area and then announce that window access number for the remote application. When a read command is submitted, the remote application specifies a local area and the previously advertised window, and vice versa for a write command.

In accordance with another example, "send operations" allow an application to send unsolicited data from local memory to a buffer provided by the remote application. The area 449 is used to describe the local memory, but no description of the remote memory is necessary. In some examples, a buffer pool may manage the local receive memory, allowing any incoming send share the same pool of buffers.

Between two clients a bidirectional sequence of operations can be exchanged. This sequence of operations may be referred to as a data stream. When to the engine 445 any of the commands discussed above, the application also specifies a data stream on which the command should be executed. The engine 445 can generate internal operations to track the progress of each instruction and those operations into its specified data stream classified. Completion of data streams are provided in the instruction submission order. However, the effects of the operations, especially for multipath communications, may not occur in turn. For example, the effect of subsequent writes may occur in any order, including nested. The data stream can be aborted at any time, ending any work on that data stream, providing error completions for all pending work.

Every window 475 and data stream may be assigned an identifier such as a color. On the windows 475 can only be accessed by data streams of the same color. Colors are stored and checked locally, which can prevent spoofing. A remote engine 445 can on windows 475 only access a stream of matching color. Data streams can be both through the client application 410 as well as through the engine 445 be generated cooperatively so that all access to memory windows should be explicitly granted.

5 illustrates an example of communication between two engines. Applications can begin a connection setup process over HTTP to the application's HTTP server. A local engine exports an address to its clients. The address may be a log buffer that tells a remote engine how to connect to the local engine. To connect the local engine to the remote engine, the addresses can be exchanged. As in 5 is shown, sends z. For example, a client sends an address of the local engine to the server, e.g. To the HTTP server. The server resolves the received address and gets an address for the remote engine. The server sends the remote engine address to the client, which resolves the address of the remote engine. Accordingly, a bidirectional connection is established between the local engine and the remote engine.

Reliable, unordered datagram transport between two engines can be referred to as a network flow. Data streams are multiplexed onto a network flow. A network flow is assigned a congestion window. A many-to-many map between network flows and data streams may be referred to as a flow mapper. The Flow Mapper is responsible for planning data flows on network flows. Each flow mapper corresponds to a remote engine. Thus every data stream via its address corresponds to a flow mapper. Each flow mapper has one or more network flows to this remote engine. When a network flow has an opportunity to send a packet, it asks the flow mapper, who in turn chooses streams to fill the packet. Small messages from multiple streams are multiplexed onto the same stream and can be stacked in a single packet. Multiple data streams, including data streams from different clients, can be multiplexed to each flow, including at the sub-packet level.

6 represents an exemplary package 600 which is used for a parametric and reliable transport protocol. The package 600 contains lower layers 602 like starting blocks, traits 604 , a wire header 606 and payload 608 ,

The transport protocol is an implementation of network flow and provides reliable, orderly delivery of any datagrams. The parametric section of the transport protocol is in 6 referred to as "traits". A specific collection of traits cases defines a concrete wire format. For example, a concrete trait may be congestion control, timing information, and so forth. The wire format may include an operation code to specify which operation to perform. The package can be changed based on the traits. Although the wire header 606 and the payload 608 remain the same, z. For example, other headers in the packet may be changed depending on the traits. IPv4 traits provide an Ethernet / UDP / IPv4 header. The transport protocol can contain an acknowledgment (ACK) in each packet, which acknowledges all packets up to a consecutive number. If no work is available, it can generate "nop" packets that contain only one ACK. If it detects a discard, it generates an explicit NACK. The protocol also supports path migration. If it detects that a current way is properly crowded, it may change a source user datagram protocol port (source user UDP port) hoping to move a flow to a less crowded path.

Exemplary methods

7 provides an exemplary schedule, a procedure 700 for processing transactions using a dedicated kernel for packet processing. The procedure 700 can z. By a system called an engine having a NIC with multiple processing cores connected to a packet processing pipeline via an interface. Further, the engine may include one or more processors programmed to perform the packet processing. Of course, the operations included in the following procedures need not be performed in the order described. Rather, different operations can be handled in a different order or simultaneously and Operations can be added or omitted.

In the block 705 the packet queries command queues for packets. Each command queue may be specific to an application executing in a separate dedicated processing unit. The command queues may e.g. In a shared memory between the NIC / Engine and the separate dedicated processing unit for the application.

In the block 710 the system receives a packet from the application processing unit. For example, the system will save the packet from the command queue.

In the block 715 the system processes the received package. For example, the engine detects what is in the package and interprets it.

In the block 720 The system performs a reliable transport for the received packet. For example, the received packet may be one as described above 6 have described format. The package may contain an acknowledgment. Accordingly, the system confirms whether the packet has been received. If the packet has not been received, a negative acknowledgment may be sent so that the packet can be resent or otherwise retried to receive the packet.

In the block 725 a software-defined handler is processed inline with a same thread in which the packet was received and processed. The received package can z. B. contain an operation code that specifies an activity to be performed. Instead of waking up a new thread to perform the activity, it will run in the same thread.

In the block 730 The system performs reliable transport for sent packets. For example, the system transports the processed packet back to the application processing unit or to other devices in the network.

In the block 735 the system writes completions to a completion queue. The completion queue may be included in the shared memory between the application processor and the engine / NIC. For example, the completion queue may be accessed by the application processor to determine which tasks have been completed. Further, the application processor may update the command queue and the completion queue in response to the read completions. For example, the application processor may clear completion from the completion queue and the corresponding command from the command queue when the application processor reads that completion has occurred.

Any of the above tasks can be performed in the same processing thread. For example, unlike a new thread being woken up to perform a particular operation, such operations may be performed in the same thread as the others.

Since the system described herein and the methods described herein emulate a "smart" NIC using general-purpose computing devices, these systems and methods provide significant cost savings. For example, existing hardware can be updated rather than replacing large amounts of hardware. Moreover, by having dedicated processing cores for packet processing, resources for incoming instructions should always be available. In some examples, additional processing cores may be mandated under significant loads. For example, additional processors may be assigned threads to perform processes such as those associated with 7 Described to execute if the command queue is filled.

8th represents a processing loop that divides processing time among activities. An execution loop 810 divides the processor time between sending work (Tx work) 822 , Reception work (Rx work) 824 and command work 826 , The Tx work 822 is through a first Tx flow 831 and through a second Tx flow 832 although it will be appreciated that additional or fewer flows may be used in other examples. The first Tx-river 831 operates the negotiation routines defined by software 842 . 844 while the second Tx flow 832 the software-defined handler 846 served. The Rx work 824 is through a first Rx flow 833 and through a second Rx flow 834 although it will be appreciated that additional flows can be used in other examples. The command work is done by applications 835 . 836 generated. In accordance with an example, the software defined treatment routines 842 . 846 Work for the client application 835 while the software-defined handler 844 does work for the client application 836 performs. In this regard, the Tx work 822 in terms of software-defined treatment routines 842 - 846 generally fair. In some examples, Rx does work 842 Fairness in line with network usage, thus supporting fairness in terms of overload protection and other feedback mechanisms. The command work 826 is fair to the client. For example, each of the multiple clients receives approximately the same percentage of resources if multiple clients provide approximately equal amounts of work.

As these and other changes and combinations of the features discussed above may be utilized without departing from the subject matter as defined by the claims, the foregoing description of exemplary aspects is to be considered as illustrative rather than restrictive of the subject matter as defined by the claims. Of course, the illustration of the examples given herein (as well as statements formulated as "such as", "for example", "containing" and the like) should not be interpreted as limiting the claimed subject matter to the specific examples; rather, the examples are just a few of many possible aspects.

Claims (20)

System comprising: a working memory storing packet processing instructions; and one or more dedicated packet processing units in communication with the random access memory and with one or more application processing units, wherein the one or more dedicated packet processing units are dedicated to perform the packet processing, wherein each of the one or more dedicated packet processing units is assigned a processing thread, the one or the multiple dedicated packet processing units are configured to: Receiving a packet from an application executing in one of the application processing units in a first processing thread; Processing by software defined handler operations initiated by instructions in the received packet, without dispatching the operation to an application thread, in the first processing thread. System after Claim 1 wherein the one or more processors are further configured to: perform a transport for the received packets; and performing a transport for processed packets to be sent; wherein the carrying out of the transports is performed in the first processing thread. System after Claim 2 wherein the transport is interfaced with the one or more application processing units without context switching or system calls. System after Claim 2 or 3 wherein the one or more processors, when performing the transport for the received packets, are configured to: detect a network problem, change transport parameters in response to detecting a network problem, and notify of any packets lost during transport. System after Claim 1 wherein the one or more processors are further configured to: query one or more instruction queues, each instruction queue being specific to an application, wherein the received packet is paged out of the one or more instruction queues; and entering completions into a completion queue. System after Claim 5 wherein the command and completion queues are included in a shared memory, wherein the shared memory is shared between the one or more processing units and a second processing unit specific to the application. System according to one of Claims 1 to 6 wherein the one or more dedicated packet processing units are processing cores of a multi-core processor. System according to one of Claims 1 to 7 wherein the one or more processors are further configured to dynamically increase or decrease a number of the one or more dedicated packet processing units. System comprising: one or more working memory; a first set of processing units in communication with the one or more random access memories, wherein the first set of processing units are configured to execute only the packet processing; a second set of processing units in communication with the one or more random access memories and with the first set of processing units, wherein the second set of processing units are configured to execute applications exclusively. System after Claim 9 . wherein each of the first processing units is assigned a processing thread, wherein the processing thread is executable by one of the first processing units to: process received packets; to carry out a transport for the received packets; processing software-defined handler operations; and to carry out a transport for packets to be sent. System after Claim 9 or 10 further comprising a shared memory between the first set of processing units and the second set of processing units. System after Claim 11 where the shared memory includes shared memory rings. System after Claim 12 wherein each of the second set of processing units communicates via a separate set of shared memory rings, the set comprising transmitter rings and receive rings. System according to one of Claims 11 to 13 wherein the shared memory includes at least one instruction queue and at least one completion queue. System after Claim 14 wherein the first set of processing units are configured to: query instruction queues for each application; and entering completions in the completion queues after completing packet processing operations. A computer-readable storage medium comprising instructions that, when executed, cause at least one processor of a computer system to perform operations that include: Receiving a packet from an application executing in a second processor dedicated for application processing with a first processor dedicated to performing packet processing and executing a first processing thread; Processing by software defined handler operations initiated by instructions in the received packet without invoking additional threads in the first processing thread. Computer readable storage medium after Claim 16 wherein the operations further comprise: performing a transport for the received packets with the first processor executing the first processing thread; and performing a transport for processed packets to be sent with the first processor executing the first processing thread. Computer readable storage medium after Claim 17 wherein performing the transport for the received packets comprises: detecting a network problem, changing transport parameters in response to detecting a network problem, and retransmitting any packet lost in the transmission. Computer readable storage medium according to one of Claims 16 to 18 wherein receiving the packet comprises: querying one or more command queues in a shared memory between the first processor and the second processor with the first processor executing the first processing thread, each command queue specific to one executing in the second processor Application is; and removing the packet from the one or more command queues. Computer readable storage medium after Claim 19 The operations further include entering completions into a completion queue in the shared memory.

Images