US8566833B1 - Combined network and application processing in a multiprocessing environment - Google Patents

Combined network and application processing in a multiprocessing environment Download PDF

Info

Publication number
US8566833B1
US8566833B1 US12045991 US4599108A US8566833B1 US 8566833 B1 US8566833 B1 US 8566833B1 US 12045991 US12045991 US 12045991 US 4599108 A US4599108 A US 4599108A US 8566833 B1 US8566833 B1 US 8566833B1
Authority
US
Grant status
Grant
Patent type
Prior art keywords
network context
network
processing
context
memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12045991
Inventor
Anumita Biswas
Vijay Singh
Sonny Son
Bill Berryman
Dave Noveck
Peter Shah
Jason Goldschmidt
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NetApp Inc
Original Assignee
NetApp Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5017Task decomposition
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5018Thread allocation

Abstract

According to a novel mechanism, each processing device (e.g., a central processing unit (CPU) in a multi-processor system) is assigned to process a single execution thread for a task and the execution thread is processed across various layers of the multi-processor system (such as a network layer and application layer) without being divided into separate threads. Advantageously, upon initialization of the multi-processor system, network context data structures are created equal to the number of processing devices in the system. As used herein, a network context is a logical entity to which zero or more connections are bound during their lifetime. Rather than sharing data structures among execution threads, a multi-processor system allocates memory resources per each network context during initialization of the system. As a result, an execution thread processing a task queued to a particular network context accesses memory resources allocated for that network context only.

Description

RELATED PATENT APPLICATIONS

This application is related to U.S. patent application Ser. No. 11/936,141, entitled “APPLICATION-CONTROLLED NETWORK PACKET CLASSIFICATION,” by Anumita Biswas, filed on Nov. 7, 2007.

FIELD OF THE INVENTION

The present invention relates to transmission of data packets across a network, and more particularly, to parallelizing processing of data packets in a multi-processor system.

BACKGROUND

It is known to parallelize processing of data packets so that more than one data packet can be executed at the same time in a multi-processor system. Multi-processor is useful when a single task takes long to complete and processing data packets serially (e.g., one at a time) would slow down the overall data packet throughput. A data packet is a unit of data transmitted as a discrete entity between devices over a network.

Some multi-processor systems follow the ISO/OSI (International Standards Organization/Open System Interconnection) model, which defines a networking framework for implementing protocols in seven layers. These layers are an application layer, a presentation layer, a session layer, a transport layer, a network layer, a data link layer, and a physical layer. These layers will be described in greater detail in reference to FIG. 2. In such a system, control is passed from one layer to the next during execution of a data packet.

Increasing overall system performance is a goal that most multi-processor systems aim to achieve. Existing multi-processor systems usually parallelize processing of data packets in a network layer of the multi-processor system and divide the data packet processing between a network layer and an application layer. The network and transport layers provide identifiers, known as internet protocol (IP) addresses and ports that uniquely identify an endpoint of a two way communication in what is termed as a connection. The network layer also determines the route from the source device (e.g., a device that sends a request) to the destination device (a device that receives the request). The transport layer performs data flow control and provides recovery mechanisms from packet loss during network transmission. The application layer is the layer at which applications utilize network services in the layers below it to do various user driven functions like file transfer, database transactions and the like. Typically, first, a network layer processes inbound data packets and after processing them signals an application thread (which runs in the application layer) to continue the processing. A thread is a separate stream of packet execution. An application thread at some point picks up the packet and performs processing in a separate application thread. Thus, dividing processing of data packets between the two layers presents scheduling delays and impacts overall system's performance.

Moreover, conventionally, an application layer would execute application threads to process data packets arriving from the network layer. Once the network layer processes a data packet on one processor, it then hands off a data packet to the application layer, which runs separate application threads on another processor. Even though the processors are capable of operating independently, still the processor executing the application thread may need to wait for the results of the execution of the network thread on another processor. As a result, at least two processors were required to process a data packet from a network layer up to the application layer.

Furthermore, when one or more application threads are executed in parallel in a multi-processor system, they often need to access common resources (such as data structures). If more than one thread tries to access the same resource to process a data packet, a lock is needed to prevent two threads from accessing the same resource(s). Implementing locks, however, can be burdensome and expensive and can slow down performance of the system.

Accordingly, what is needed is a mechanism that can increase overall system's performance by eliminating shortcomings of dividing data packet processing between network processing and application processing without locking requisite resources.

SUMMARY OF THE INVENTION

According to a novel mechanism, each processing device (e.g., a central processing unit (CPU) in a multi-processor system) is assigned to process an execution thread for a task across various layers of the multi-processor system (such as across a network layer and application layer) so that execution of a single thread is not divided between a network layer and an application layer. Since a single processing device can perform processing across the layers, the need to move a thread from one processing device to another can be eliminated. This in turn, allows other processing devices in the multi-processor system to be efficiently used to execute more threads. As a result, throughput of a multi-processor system increases.

Advantageously, upon initialization of the multi-processor system, network context data structures are created equal to the number of processing devices in the system. As used herein, a network context is a logical entity to which zero or more connections are bound during their lifetime. Rather than sharing data structures among execution threads, a multi-processor system allocates memory resources per each network context during initialization of the system. As a result, an execution thread processing a task queued to a particular network context accesses memory resources allocated for that network context only. Since each execution thread can access a pool of resources assigned to a corresponding network context, the need to lock various memory resources, which are typically accessed by more than one thread, can be eliminated. As a result, more than one execution threads running across various layers of the system can be executed in the multi-processor system.

Furthermore, as discussed herein, each network context is associated with its own pool of resources (e.g., application data structures and memory sub-regions for storing thread messages) which are not accessed by any other execution thread. For example, a memory of the multi-processor system can be segmented into a plurality of memory sub-regions, with each sub-region storing data structures for each network context. Thus, for ‘n’ network contexts, there are ‘n’ memory sub-regions corresponding to each network context. Each execution thread can use its own memory sub-region to pass messages without clashing with another execution thread. As a result, more than one execution thread can be processed simultaneously in the multi-processor system without requiring burdensome and expensive process of locking data structures and memory. Furthermore, combining network and application threads into a single execution thread provides performance benefits as it avoids delays associated with queuing data packets between the network layer and the application layer.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present invention are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:

FIG. 1 shows network environment that includes a source network device and a destination network device that communicate with each other over a network according to an embodiment of the present invention;

FIG. 2 is a block diagram of components of an operating system of the destination network device shown in FIG. 1;

FIG. 3 is a block diagram of distribution of data packets among network contexts according to an embodiment of the present invention;

FIG. 4 is a block diagram of various fields of a data packet transmitted between a source network device and a destination network device shown in FIG. 1;

FIG. 5 is a diagram of an exemplary network data structure used to process data packets on a connection;

FIG. 6A is a flow diagram of the steps performed by a multi-processor system during initialization according to embodiments of the present invention;

FIG. 6B is a flow diagram of the steps performed by a multi-processor system to process a data packet as a single execution thread across layers of an operating system according to embodiments of the present invention;

FIG. 7 is a diagram showing a plurality of network data structures, each data structure associated with a network context according to an embodiment of the present invention; and

FIG. 8 is a diagram showing memory subdivisions for allocating data structures per each network context in the multi-processor system shown in FIG. 1 according to embodiments of the present invention.

DETAILED DESCRIPTION Network and Storage Server Environment

FIG. 1 is a schematic block diagram of a storage system environment that includes two interconnected network devices, such as a source network device (e.g., client 170) and a destination network device (e.g., storage system 120) communicating with each other by exchanging data packets over a network 118, such as a local or wide area network. Although one client 170 is shown in FIG. 1, those skilled in the art would understand that any number of clients can communicate with storage system 120 over network 118. Furthermore, a source network device can be a storage system in communication with destination storage system 120. As used herein, the term “source” can be broadly defined as a location from which data travels, and the term “destination” can be defined as the location to which the data travels.

The client 170 transmits requests for data to the destination storage system 120 and receives responses to the requests over network 118. Data is transferred between the client 170 and the storage system 120 using data packets 155. If the client 170 executes the Windows® operating system (not shown in FIG. 1), data packets can be transmitted using the Common Internet File System (CIFS) protocol over TCP/IP. Client 170 can run the UNIX® operating system. In such an embodiment, client 170 may communicate with the storage system 120 using either the Network File System (NFS) protocol over TCP/IP or the Direct Access File System (DAFS) protocol over a virtual interface (VI) transport in accordance with a remote DMA (RDMA) protocol over TCP/IP. It will be apparent to those skilled in the art that client systems running other types of operating systems may also communicate with the storage system using other file access protocols. Furthermore, client 170 may communicate with destination storage system 120 using block-based protocols, such as Internet Small Computer System Interface (iSCSI) or Fibre Channel (FC) protocol.

As shown in FIG. 1, client 170 executes client software 172 (such as CIFS client software, NFS client software, DAFS client software, or any other application resident in memory 173 and executed by processor 175 on a client 170). Client 170 further includes a protocol stack 174, which may include TCP/IP protocol stack (Internet Protocol module and its supporting transport mechanisms, the Transport Control Protocol module and the User Datagram Protocol module). These modules are not shown in FIG. 1.

Destination storage system 120 can be a multiprocessing network device that manages storage of one or more volumes, which in turn, may comprise an array of storage devices 160. Storage system 120 comprises a plurality of processors 122 (also referred to herein as “processing device”), a memory 125, a network adapter 130, and a storage adapter 140 interconnected by a system bus 145. These components of the storage system 120 are briefly described below.

Processors 122 can be any machine, which are capable of executing a sequence of instructions (e.g., macro- or microinstructions). Processors 122 may be implemented as general purpose microprocessors or special purpose microprocessors. As will be described in greater detail herein, each processor is assigned to process an execution thread for a task and processes the execution thread across various layers of the multi-processor system (such as across a network layer and application layer). Unlike prior art implementations, which required at least one processor to execute a task in the network layer and another processor to execute the same task in the application layer, according to novel embodiments, a single processor is assigned to execute a single execution thread across various layers. As a result, other processors in the multi-processor system 120 can be efficiently used to execute more threads. This, in turn, increases throughput of the multi-processor system 120.

In the illustrative embodiment, the memory 125 comprises storage locations that are addressable by the processors and adapters for storing software program code. The memory 125 can be a random access memory (RAM). The processors and adapters may, in turn, comprise processing elements and/or logic circuitry configured to execute the software code and manipulate the data structures stored in memory. The operating system 200 (shown in more detail in FIG. 2), portions of which are typically resident in memory, functionally organizes the storage system by, inter alia, invoking storage operations in support of a file service or block service implemented by the storage system. It will be apparent to those skilled in the art that other processing and memory means, including various computer readable media, may be used for storing and executing program instructions pertaining to the inventive technique described herein.

The network adapter 130 comprises the mechanical, electrical and signaling circuitry needed to connect storage system 120 to the network 118, which may comprise a point-to-point connection or a shared medium, such as a local area network.

The storage adapter 140 cooperates with the operating system 200 executing on the storage system to access data requested by the client system (such as client 170). The data may be stored on storage devices 160 that are attached, via the storage adapter 140, for example, to storage system 120.

In one exemplary implementation, storage system 120 can include a nonvolatile random access memory (NVRAM) 135 that provides fault-tolerant backup of data, enabling the integrity of storage system transactions to survive a service interruption based upon a power failure, or other fault.

As will be described in more detail with reference to FIG. 2, storage operating system 200 includes a file system 210, application layer 220 (which may include HTTP server 222, NFS server 224, CIFS server 226, SnapMirror application 228, etc), and a network layer 270 (these components are shown and described in greater detail in reference to FIG. 2).

Storage Operating System

FIG. 2 is a schematic block diagram of the components of the storage operating system 200 of storage system 120 that may be advantageously used with embodiments of the present invention. In the illustrative embodiment, the storage operating system 200 can be the Data ONTAP™ operating system available from Network Appliance Inc., of Sunnyvale, Calif., that implements a Write Anywhere File Layout (WAFL™) file system, such as file system 210. Those skilled in the art would understand that a file system other than WAFL™ can be used to implement a novel mechanism described herein. As used herein, the term “storage operating system” generally refers to the computer-executable code operable on a computer that manages data access. The storage operating system 200 can be implemented as an application program operating over a general-purpose operating system, such as UNIX® or Windows NT®, or as a general-purpose operating system, which is configured for storage applications as described herein. The storage operating system 200 further includes a disk management module (not shown in FIG. 2) that manages disk access operations. The disk management module may include a disk storage module that implements a disk storage protocol and a disk driver module that implements a disk access protocol (these modules are not shown in FIG. 2).

Still referring to FIG. 2, the storage operating system 200 includes a series of modules organized to form an integrated protocol stack 280 or, more generally, a multi-protocol engine that provides data paths for clients (e.g., client 170) or other storage systems to access data stored on the storage system 120 using block and/or file access protocols. In one implementation, the protocol stack 280 can follow the OSI model that defines a network framework for implementing protocols in seven layers so that control is passed from one layer to the next, starting from the top layer (e.g., application layer 220) and proceeding to the bottom layer (e.g., the physical layer, which corresponds to the media access module 240 shown in FIG. 2). The media access module 240 comprises network drivers 244 (e.g., gigabit Ethernet drivers). The media access module 240 as a physical layer is responsible for transmitting bits of data from one network device to another network device and regulates the transmission of a stream of bits over a physical medium. For example, media access module 240 defines how a cable is attached to the network adapter (such as network adapter 130 shown in FIG. 1) and what transmission technique is used to transfer the data over the cable.

The media access module 240 interfaces to the classifier module 242 (corresponding to the data link layer in the OSI model). The classifier module 242 is responsible for maintaining a pool of network threads and their associated queues (referred to herein as network contexts and shown in more detail in FIG. 3). A network thread is a process that executes network functionality. A network context includes work items to be processed. A work item generally includes data packets to be processed and a function pointer that needs to be executed with the packets.

The IP module 236 (corresponding to the network layer in the OSI model) is responsible for translating logical addresses into physical addresses. A logical address refers to a network layer address, such as an IP address. A physical address refers to an address of the physical communications device in a system. The IP module 236 is further responsible for managing flow control problems, such as switching and routing. As known in the art, the IP module 236 may be used to implement the Internet Protocol for routing data packets to their intended destinations over a network. TCP module 234 corresponds to the transport layer in the OSI model. TCP module 234 handles error recognition and recovery. It also repackages long messages, when necessary, into small packets for transmission at the receiving end and rebuilds packets into the original message. The TCP module 234 may implement the Transport Control Protocol for ensuring reliable transport of data packets over a network.

The socket interface 232 corresponds to the session layer in the OSI model. Socket interface 232 allows applications on different devices to establish, use and terminate a session/connection. Socket interface 232 regulates which device transmits data, when and how long the transmission takes place. The socket interface 232 can be implemented as Berkeley Software Distribution (BSD) interface.

Media access module 240, classifier module 242, IP module 236, TCP module 234 and socket interface module 232 are part of a network layer 270.

Application layer 220 (which corresponds to application layer in the OSI model) is the layer at which applications access network services. Application layer 220 represents the services that directly support applications, such as software for file transfers, database access, and electronic mail. For example, in one embodiment, application layer 220 may include Hypertext Transfer Protocol (HTTP) server 222 for providing services for file transfer in HTTP format, NFS server 224, CIFS server 226, SnapMirror® application 228, or any other application resident in memory 125 and executed by processors 122 on storage system 120.

Typically, for each layer, the operating system 200 defines one or more processing threads that process data packets and then passes the data packets to the next thread in the next layer for further processing. When processing data packets, the processing threads may pass messages (thread messages) to each other using a message passing scheme provided by the operating system 200. A thread message contains data needed to properly process data packets between the threads. For example, a thread message may include an indication of data location, such as a file offset and a length of data to retrieve. Conventionally, an application layer would execute application threads to process data packets arriving from the network layer. Thus, once the network layer processes a data packet on one processor, it then hands off a data packet to the application layer, which runs separate application threads on another processor. Even though the processors are capable of operating independently, still the processor executing the application thread may need to wait for the results of the execution of the network thread on another processor. As a result, at least two processors were required to process a data packet from a network layer up to the application layer. Splitting processing into two parts thus presents delays in processing of data packets and results in inefficient utilization of processing resources, such as CPUs. Furthermore, an application layer often needs to access various application data structures (such as a reply cache 129) to perform a common task while processing data packets. As a result, multiple application threads had to access common data structures at the same time. To avoid this from happening, a shared data structure was typically locked while accessed by an application thread.

According to a novel mechanism, each processing device (e.g., a central processing unit (CPU) in a multi-processor system) is assigned to process a single execution thread for a task and the execution thread is processed across various layers of the multi-processor system (such as a across a network layer and application layer) without being divided into separate threads. Advantageously, upon initialization of the multi-processor system, network context data structures are created equal to the number of processing devices in the system. As used herein, a network context is a logical entity to which zero or more connections are bound during their lifetime. Rather than sharing data structures among execution threads, a multi-processor system allocates memory resources per each network context during initialization of the system. As a result, an execution thread processing a task queued to a particular network context accesses memory resources allocated for that network context only. Since each execution thread can access a pool of resources assigned to a corresponding network context, the need to lock various memory resources, which are typically accessed by more than one thread, can be eliminated. As a result, more than one execution threads running across various layers of the system can be executed in the multi-processor system.

Furthermore, since conventionally, data packet processing was divided between the network layer and the application layer, at least two CPUs in the multi-processor system were occupied at the same time to process a data packet given that there is interdependency between processing a network thread and an application thread. According to the novel mechanism, since a single execution thread processes a data packet across various layers of the operating system in the same network context, the need to utilize two CPUs to process a thread is eliminated. Rather, a single CPU can be assigned to process the execution thread. As a result, remaining CPUs in the multi-processor system can be utilized to process other threads.

Methods of Operation

Referring now to FIG. 6A, it illustrates the steps performed by a multi-processor system (such as storage system 120 shown in FIG. 1). During initialization, operating system creates a number of network contexts equal to the number of CPUs in the system (step 610). The number of network contexts is then used by the storage system 120 to allocate memory resources for each network context (step 620). For example, referring now to FIG. 8, memory 125 (at the storage system 120) is segmented into a plurality of memory sub-regions so that each sub-region can store data structures for a particular network context (e.g., mbuf data structure 810 for holding data packet data; a message data structure 820 for communicating with a file system; a counter data structure 830 that indicates a number of processed read and write requests for each network contexts; a receive buffer 840; and a send buffer 850, as well as other data structures). As shown in FIG. 8, each memory sub-region has data structures corresponding to a particular network context. Thus, for ‘n’ network contexts, there are ‘n’ memory regions, each memory region corresponding to each network context. Since no two threads can access the same data structures, two or more threads can be executed at the same time, with each thread executing a data packet across network and application layer, without clashing for the same resource (e.g., data structures).

Still with reference to FIG. 6A, as part of the initialization of the multi-processor system, a single CPU is assigned to process an execution thread across layers of the operating system (step 630).

Referring now to FIG. 6B, it illustrates steps performed by storage system 120, implemented as a multi-processor system, to process data packets across layers (e.g., a network layer and an application layer) of an operating system according to an embodiment of the present invention.

Storage system 120 receives a request to establish a network connection from a client system 170. Such a request may include a source IP address, a source port and a destination port. As discussed herein, the TCP module 234 may be used to implement the Transport Control Protocol for ensuring reliable transport of data packets over a network. The TCP module 234 may provide a plurality of TCP ports, each port being identified by a port number and capable of establishing a TCP connection over the network. Thus, each TCP connection may be identified by a source IP address, source port number, a destination IP address, and a destination port number. A destination network device (such as storage system 120) may allocate a temporary port number for a source network device that establishes connection with a destination storage device to use in creating a TCP connection. A TCP connection is typically received through a well-known port number at the destination network device. Each network service has a well-known TCP port number defined by the network service standards and may “listen” on that port for incoming connections. Multiple connections may use the same TCP port number if they are connecting to the same network service.

Each network driver device 244 may maintain multiple connections through the network through multiple physical ports. Each port (a physical socket with supporting circuitry) maintains a specific connection. Each port of a network driver device 244 may operate as a separate device having a media access control (MAC) address and its own assigned IP address (so that a connection to the IP address will always be handled by that physical port of the network device). Each port can handle multiple TCP/IP connections.

Thus, initially, a request to establish connection is received by network drivers of storage system 120. A request is in the form of a data packet. The network drivers call the classifier module 242, which is responsible for identifying a network context to which the data packet will be queued (step 640). To this end, in one implementation, classifier module 242 calculates a value based on a 3-tuple (source IP address, source port, and destination port) indicated in the packet. The following hash function can be used to determine a network context:
Destination port 1:y=hash1(x);

wherein ‘x’ is a logical combination of a source IP address of the incoming data packet, a source port, and a destination port (also referred to herein as a 3-tuple); ‘y’ represents a hash number, which then becomes an input to another function that maps the hash to a network context number ‘y1’ as follows: y1=map_hash_q(y); wherein “q” represents a network context queue. Those skilled in the art would understand that other mechanisms can be used to compute a logical value based on a 3-tuple.

At step 650, IP processing of the data packet is performed by IP module 236. IP module 236 validates the data packet and sends it off to the TCP module 234 along with the network context identification (ID) to which the data packet is queued.

The TCP module 234 receives the data packet and the network context identifier (ID) and determines how to process the data packet (step 660). To this end, TCP module 234 then uses a 3-tuple as an index to a TCP connection table (such as the one shown in FIG. 7) corresponding to the received network context (step 660). At this point, it is advantageous to describe differences between TCP connection tables according to prior art implementations and a novel mechanism by which each network context is associated with its own TCP connection table.

Referring now to FIG. 5, it illustrates TCP connection table 500 (according to a prior art implementation), which represents an array, with each index 510 in the array being a linked list of PCBs 520. An index is also referred to herein as a “hash bucket.” Each PCB stores data related to one TCP connection. Such data may relate to establishing a connection, managing data transmission via data packets, managing data receipt, and termination of the connection. TCP connection table 500 is accessed by more than one thread to access PCB to obtain information about a particular connection. As a result, locks are typically implemented to prevent more than one thread from accessing the same PCB.

According to a novel technique, which is described in a commonly-owned patent application U.S. patent application Ser. No. 11/936,141, entitled “APPLICATION-CONTROLLED NETWORK PACKET CLASSIFICATION,” by Anumita Biswas, the disclosure of which is incorporated by referenced herein, rather than having one TCP data structure that is accessed by all threads (such as shown in FIG. 5), each network context is associated with its own TCP data structure that stores a linked list of PCBs for connections queued to that network context. Thus, FIG. 7 illustrates a plurality of TCP tables 700. Each TCP table 700 is associated with one network context (which, in turn, is served by an execution thread that processes data packets queued to that network context). TCP table 700 includes an array of hash buckets 710, with each hash bucket 710 being linked to a list of PCBs 720. TCP table 700 is established upon initialization of storage system 120 and can be maintained in memory 125.

Thus, to determine how to process a data packet received on a connection, TCP module 234 indexes into a TCP table corresponding to the received network context ID. An index to access the TCP table can be computed as a value based on a 3-tuple (a source IP address, a source port, and a destination port indicated in an initial data packet to establish the connection). TCP module 234 compares the computed value with data in PCBs (step 664). If there is a match (e.g., the PCB block for the requested connection already exists), TCP module 234 uses the data stored in the PCB to process data packets arriving on the connection (step 668). In addition, the TCP module 234 puts the data packet on a socket indicated in the PCB (e.g., queues the data packet in memory 125 and chains mbuf data structures in memory). As noted herein, mbuf data structures are allocated for each network context and are used for holding data packet data during data packet processing.

If, at step 664, PCB is not found in the TCP table, it may indicate that a connection is a new one. TCP module treats it as a new connection (step 666). To this end, TCP module 234 creates a new PCB and attaches it to a linked list of PCBs in the TCP table corresponding to the context ID. The new PCB block includes data for the initiated connection (such as data related to a) establishing a connection, b) managing data transmission, c) managing data receipt, and d) termination of connection)).

After TCP processing at step 668, a determination is made whether there are data in a data packet (step 670). If there are data (such as a header and a payload), the mbuf data structure is passed to application layer (step 674). Mbuf data structure includes, for example, a network context ID, a receive buffer, and a send buffer.

Application layer 220 receives the receive buffer and reads a header of the data packet to determine the type of a request (e.g., read, write, etc) (step 676)

An exemplary data packet 155 transmitted between client 170 and storage system 120 is shown in FIG. 4. Data packet 155 includes various fields, such as IP header 420, TCP header 430, application header 440, and payload 450.

IP header 420, in turn, includes fields for storing a source IP address (also referred to herein as a “foreign IP address”) and a destination IP address (also referred to herein as “a local IP address”).

TCP header 430 includes fields for storing a source port (also referred to herein as a “foreign port”) and a destination port (also referred to herein as a “local port”).

Application header 440 includes control data provided by an application when processing a data packet. If the client 170 executes, for example, CIFS client software, the application header 440 stores control data related to CIFS protocol. Similarly, in the case of NFS software, the header stores control data related to NFS protocol. If the application is a replication application (such as SnapMirror) executed at the source network device and a destination network device, the application header includes control data inserted by the replication application. Payload 450 includes actual data to be stored at the destination storage system 120.

Still with reference to FIG. 6B, application layer 220 determines (step 680) (e.g., for a read request) whether requested data are in a reply cache (such as the one shown in FIG. 1). A reply cache is used to cache replies to non-idempotent requests, so that in case of retransmission of a client request, the correct reply can be sent back to the client. Non-idempotent requests refer to operations that do not return same results if repeated (e.g., remove, create, etc).

As described herein, each reply cache is associated with a network context. Thus, application layer 220 determines whether the requested data are in the reply cache associated with the network context in which the data packet is executed. If the data are in the reply cache, the execution thread receives the data, frames a response data packet (step 690) and puts the response into the mbuf data structure allocated to the network context to which the data packet was originally queued. As part of framing a response data packet, the execution thread populates the data packet with a protocol header (such as NFS header, CIFS header, etc). The execution thread returns a memory address(s) for the mbuf data structure to the sub-region of memory 125 corresponding to the network context (so that the mbuf data structure can be used later by another execution thread). The execution thread then puts the data packet on a socket and calls TCP module 234 (Step 692). A counter data structure (such as 830 shown in FIG. 8) for the network context is incremented to indicate that the network context performed a read request.

TCP module 234 receives the packet, adds a TCP header, and hands off the packet to IP module 236. IP module 236 looks at the destination address in the data packet and sends the data packet to an appropriate network driver 244. The network driver 244 transmits the packet over the network 118 to the source network device (such as client 170).

If the data are not in the reply cache (in step 680), execution thread sends a message data structure to the file system (step 682). Exemplary message data structures 820 are shown in FIG. 8. Each message data structure is allocated to a network context. The message data structure is used to communicate with the file system. The message data structure may include a request instructing the file system to perform an access request (read or write data). For example, a request may indicate: read 100 bytes at offset 20 from a file (identified by a file name). Importantly, the message data structure is associated with the network context to which the data packet is queued so that no other execution thread can access the same message data structure.

File system 210, in turn, uses a disk storage module and then a disk access module (not shown in FIGS. 1-8) to load the requested data in memory 125 for processing by the storage system 120. Application layer obtains the network context ID to which the data request had first arrived, from a shared data structure between the TCP layer and application layer (the data structure is not shown in Figures). The application layer 220 queues the response to the network context as a work item to the execution thread. The execution thread then performs TCP and IP processing prior to sending the data packet to network driver 244. The network driver 244 transmits the packet over the network 118 to the source network device (such as client 170). This completes processing of the data packet (step 683).

Referring now to FIG. 3, it illustrates execution thread 1, execution thread 2, and execution thread n, which are executed in the storage system 120 across the network layer 270 and the application layer 220. Each execution thread (1 though n) is associated with its own network context. For example, execution thread 1 is associated with network context 1, execution thread 2 is associated with network context 2, and execution thread n is associated with network context n. As described herein, each network context represents a logical entity to which one or more data packets are queued for execution. Each network context is associated with its own pool of resources (e.g., various network and application layer data structures). When a data packet arrives, network driver 244 calls the classifier module 242 for identifying a network context to which the data packet will be queued. As shown in FIG. 3, classifier module 242 queues data packets to various network contexts. Thus, a data packet queued to a particular network context is executed within that network context by an execution thread across the network layer and the application layer. The execution thread uses data structures assigned to the network context to process a data packet.

Thus, a novel mechanism is provided by which processing of a data packet by the network layer 270 and application layer 220 is combined into the same execution thread. Since a single CPU can perform processing across the application layer and network layer, the need to move a thread from one CPU to another is eliminated. This in turn, allows other CPUs in the multi-processor system to be utilized by processing other execution threads. As a result, throughput of a multi-processor system increases. Furthermore, since each execution thread is associated with its own pool of resources and no two threads can access the same resource, a plurality of execution threads can run simultaneously in the multi-processor system without the need to lock the resources.

The present invention can be adapted to multi-processor systems that can be implemented as a network-attached storage (NAS) device (that can provide clients with a file-level access to data stored in a set of mass storage devices) or as a Storage Area Network (SAN) device that can provide clients with a block-level access to stored data. Those of skill in the art would understand that although embodiments of the present invention were described in the context of using hash functions, any other suitable method for turning data into another number can be used with the present invention.

Furthermore, the use of the phrase “one embodiment” throughout does not necessarily mean the same embodiment. Although these particular embodiments of the invention have been described, the invention should not be limited to these particular embodiments. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense.

Unless specifically stated otherwise, it is to be appreciated that throughout the discussions utilizing terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like refer to the action and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (e.g. electronic) quantities within the computer systems registers and memories into other data similarly represented as physical quantities within the computer system.

The present invention can be implemented by apparatuses for performing the operations herein. These apparatuses may be specially constructed for the required purposes or they may comprise a machine, such as a general-purpose computer selectively activated or reconfigured by a computer program (such as a collection of instructions for execution by a machine or processor for example) stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to any type of disk including floppy disks, optical disks, magnetic optical disks, read-only memories, random access memories, EPROMS, EEPROMS, magnetic or optical cards or any type of media suitable for storing physical (e.g. electronic) constructions and each coupled to a computer system bus. Each of these media may be coupled to a computer system bus through use of an appropriate device for reading and or for writing the media.

Claims (18)

What is claimed is:
1. A method for processing tasks of a protocol stack on a multi-processor system, comprising:
creating, at initialization of the multi-processor system, a number of network context data structures equal to a number of processing devices;
allocating a memory resource for each network context so that an execution thread queued to a particular network context accesses the memory resource allocated for the particular network context, the execution thread associated with a network connection data structure representing one or more transport layer connections queued to the particular network context, the memory resource allocated for the particular network context represents the network connection data structure, the network connection data structure comprising a transport layer connection table, wherein each entry in the transport layer connection table references a process control block data structure (PCB), wherein each PCB associated with a transport layer connection is accessed by the execution thread processing a task queued to the particular network context; and
assigning a single processing device of the number of processing devices in the multi-processor system to process the execution thread using the memory resource and processing of the execution thread across layers of the protocol stack, thereby not requiring a lock of the memory resource.
2. The method of claim 1, further comprising:
receiving, by a network layer, a data packet to perform the task,
determining a network context ID to queue the data packet, the context ID determined using information indicated in the data packet, the information selected from a group consisting of a source address, a source port, a destination address, a destination port, and combinations thereof; performing network processing of the data packet using the memory resource, the memory resource associated with the network context ID;
providing, by the network layer, to an application layer, the network context ID and the memory resource associated with the network context ID to further process the data packet; and
processing, by the application layer, the data packet using data structures associated with the network context so that the task is executed by the single processing device, the application layer comprising a protocol chosen from a group consisting of HTTP, NFS, CIFS, RDMA over TCP/IP, and iSCSI.
3. The method of claim 1, wherein the memory resource allocated for the particular network context represents a message instructing a file system to access a file, the message processed by the task queued to the particular network context.
4. The method of claim 1, wherein the memory resource allocated for the particular network context represents a data structure for holding a data packet, the packet processed by the task queued to the particular network context.
5. The method of claim 1, wherein the memory resource allocated for the particular network context represents a counter data structure that indicates a number of processed I/O requests for the particular network context.
6. The method of claim 1, further comprising storing memory resources for network contexts in sub-regions of memory of the multi-processor system such that each memory sub-region maintains memory resources corresponding to the particular network context.
7. The method of claim 1, wherein processing the execution thread across layers comprises processing the execution thread across a network layer configured to perform network processing of the task and an application layer configured to perform application services.
8. The method of claim 1, wherein the execution thread is processed by one processing device of the number of processing devices across layers of the protocol stack so that the processing of the thread is not divided among two or more processing devices in the multi-processor system.
9. The method of claim 1, further comprising:
providing a network context identification (ID) to an application layer of the multi-processor system while processing the execution thread so that the thread is processed at the application layer in the particular network context by using resources allocated to the particular network context identified by the network context ID, wherein the context ID identifies the particular network layer connection, wherein context ID is determined using information indicated in a data packet, the information selected from a group consisting of a source address, a source port, a destination address, a destination port, and combinations thereof.
10. A method for processing tasks of a protocol stack on a multi-processor system, comprising:
maintaining a number of network context data structures equal to a number of processing devices that process one or more tasks assigned to a particular network context;
maintaining memory resources for each network context so that a single execution thread that process a first task queued to a first network context accesses a memory resource allocated for the first network context, the memory resource represents a network connection data structure, the network connection data structure comprising a transport layer connection table, wherein each entry in the transport layer connection table references a process control block data structure (PCB), wherein each PCB associated with a transport layer connection is accessed by the execution thread processing a task queued to the particular network context; and
processing, by a single processing device of the number of processing devices, the single execution thread for the first task using the memory resources for the first network context and processing the single execution thread for the first task across a network layer and an application layer of the multi-processor system, wherein each network context is associated with a transport layer connection for a lifetime of the connection.
11. The method of claim 10, further comprising storing the memory resources for the first network context in a sub-region of memory of the multi-processor systems such that each memory sub-region maintains the memory resources corresponding to a single network context.
12. The method of claim 10, wherein processing the single execution thread does not lock the memory resources.
13. A system for processing tasks of a protocol stack, the system comprising:
a plurality of network context data structures maintained in a memory, equal to a number of processing devices for processing tasks assigned to a particular network context;
a plurality of memory resources, each resource associated with a network context so that a single execution thread processing a task queued to a particular network context accesses memory resources allocated for that network context, the protocol stack comprising a transport layer and an application layer, each network context associated with a transport layer connection for a lifetime of that connection;
two or more processing devices for processing tasks assigned to the particular network context, each processing device assigned to process a single execution thread for a task using the allocated memory resources, wherein a memory resource of the plurality of memory resources allocated for the particular network context represents a transport control protocol (TCP) data structure accessed by the single execution thread processing tasks queued to the particular network context, wherein the TCP data structure represents an array of indexes, each index corresponding to a linked list of one or more process control blocks (PCB) storing data about processing the task; and
processing the execution thread for the task across layers of the protocol stack, whereby the processing is performed without requiring to lock the allocated memory resources.
14. The system of claim 13, wherein a memory resource of the plurality of memory resources allocated for the particular network context represent a message data structure for passing data to and from a file system to process the task queued to the particular network context.
15. The system of claim 13, wherein a memory resource of the plurality of memory resources allocated for the particular network context represents a data structure for holding a data packet including the task queued to the particular network context, the particular network context is determined using information indicated in the data packet, the information selected from a group consisting of a source address, a source port, a destination address, a destination port, and combinations thereof.
16. The system of claim 15, further comprising:
one or more network drivers configured to receive one or more data packets over a network to process the task and to forward the data packet to a classifier module; and
the classifier module configured to distribute the received data packets among one or more of the network contexts and to queue the data packets to the one or more network contexts for processing.
17. The system of claim 13, wherein a memory resource of the plurality of memory resources allocated for the particular network context represents a counter data structure that indicates a number of processed I/O requests for the particular network context.
18. The system of claim 13, wherein the plurality of memory resources are maintained in sub-regions of the memory such that each memory sub-region is associated with a corresponding network context.
US12045991 2008-03-11 2008-03-11 Combined network and application processing in a multiprocessing environment Active 2031-10-25 US8566833B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12045991 US8566833B1 (en) 2008-03-11 2008-03-11 Combined network and application processing in a multiprocessing environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12045991 US8566833B1 (en) 2008-03-11 2008-03-11 Combined network and application processing in a multiprocessing environment

Publications (1)

Publication Number Publication Date
US8566833B1 true US8566833B1 (en) 2013-10-22

Family

ID=49355415

Family Applications (1)

Application Number Title Priority Date Filing Date
US12045991 Active 2031-10-25 US8566833B1 (en) 2008-03-11 2008-03-11 Combined network and application processing in a multiprocessing environment

Country Status (1)

Country Link
US (1) US8566833B1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9535759B2 (en) * 2014-05-23 2017-01-03 Osr Open Systems Resources, Inc. Work queue thread balancing
US20170052864A1 (en) * 2015-08-21 2017-02-23 International Business Machines Corporation Automatic connection detection in a data center
US9785473B2 (en) 2014-07-14 2017-10-10 Nxp Usa, Inc. Configurable per-task state counters for processing cores in multi-tasking processing systems
US9961021B2 (en) 2012-04-19 2018-05-01 Cisco Technology, Inc. Enabling applications in a multi-transport stack environment

Citations (66)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4847755A (en) * 1985-10-31 1989-07-11 Mcc Development, Ltd. Parallel processing method and apparatus for increasing processing throughout by parallel processing low level instructions having natural concurrencies
US5021945A (en) * 1985-10-31 1991-06-04 Mcc Development, Ltd. Parallel processor system for processing natural concurrencies and method therefor
US5410675A (en) * 1989-08-21 1995-04-25 Lisa M. Shreve Method of conforming input data to an output data structure and engine for accomplishing same
US5535338A (en) * 1993-07-28 1996-07-09 3Com Corporation Multifunction network station with network addresses for functional units
US6078564A (en) 1996-08-30 2000-06-20 Lucent Technologies, Inc. System for improving data throughput of a TCP/IP network connection with slow return channel
US6338078B1 (en) 1998-12-17 2002-01-08 International Business Machines Corporation System and method for sequencing packets for multiprocessor parallelization in a computer network system
US6363477B1 (en) * 1998-08-28 2002-03-26 3Com Corporation Method for analyzing network application flows in an encrypted environment
US6480507B1 (en) * 1998-11-19 2002-11-12 Nortel Networks Limited Communication protocol stack apparatus and method of implementing same
US20020194338A1 (en) 2001-06-19 2002-12-19 Elving Christopher H. Dynamic data buffer allocation tuning
US20030069920A1 (en) 2001-09-28 2003-04-10 Melvin Stephen W. Multi-threaded packet processing engine for stateful packet processing
US20030081615A1 (en) 2001-10-22 2003-05-01 Sun Microsystems, Inc. Method and apparatus for a packet classifier
US20030140088A1 (en) * 2002-01-24 2003-07-24 Robinson Scott H. Context-based information processing
US20040030745A1 (en) 1997-10-14 2004-02-12 Boucher Laurence B. Method and apparatus for distributing network traffic processing on a multiprocessor computer
US20040148376A1 (en) 2002-06-28 2004-07-29 Brocade Communications Systems, Inc. Storage area network processing device
US20040193678A1 (en) * 2003-03-27 2004-09-30 Microsoft Corporation Notifications for shared resources
US20040205754A1 (en) 2002-08-26 2004-10-14 Interdigital Technology Corporation Software porting layer
US6820127B2 (en) 2001-05-03 2004-11-16 International Business Machines Corporation Method, system, and product for improving performance of network connections
US20040246956A1 (en) * 2003-06-06 2004-12-09 Meng David Qiang Parallel packet receiving, routing and forwarding
US20050060365A1 (en) * 2002-01-24 2005-03-17 Robinson Scott L. Context-based information processing
US6904040B2 (en) 2001-10-05 2005-06-07 International Business Machines Corporaiton Packet preprocessing interface for multiprocessor network handler
US20050138091A1 (en) 2003-12-22 2005-06-23 Jean-Pierre Bono Prefetching and multithreading for improved file read performance
US20050154783A1 (en) * 2001-02-27 2005-07-14 Microsoft Corporation Intent based processing
US6920501B2 (en) 2001-12-17 2005-07-19 Ntt Docomo, Inc. Communication socket migration among different devices
US6922557B2 (en) * 2000-10-18 2005-07-26 Psion Teklogix Inc. Wireless communication system
US20050226235A1 (en) 2004-04-08 2005-10-13 Alok Kumar Apparatus and method for two-stage packet classification using most specific filter matching and transport level sharing
US6981140B1 (en) * 1999-08-17 2005-12-27 Hewlett-Packard Development Company, L.P. Robust encryption and decryption of packetized data transferred across communications networks
US20060013253A1 (en) 2004-07-16 2006-01-19 Hufferd John L Method, system, and program for forwarding messages between nodes
US20060013222A1 (en) 2002-06-28 2006-01-19 Brocade Communications Systems, Inc. Apparatus and method for internet protocol data processing in a storage processing device
US6990662B2 (en) * 2001-10-31 2006-01-24 Hewlett-Packard Development Company, L.P. Method and system for offloading execution and resources for resource-constrained networked devices
US20060059287A1 (en) * 2004-09-10 2006-03-16 Pleora Technologies Inc. Methods and apparatus for enabling bus connectivity over a data network
US7020713B1 (en) 2000-10-10 2006-03-28 Novell, Inc. System and method for balancing TCP/IP/workload of multi-processor system based on hash buckets
US7065568B2 (en) * 2000-11-30 2006-06-20 Microsoft Corporation System and method for managing states and user context over stateless protocols
US7127717B2 (en) * 2001-12-25 2006-10-24 Hitachi, Ltd. Hierarchical server system
US20060242532A1 (en) 2005-04-26 2006-10-26 Intel Corporation Techniques to provide information validation and transfer
US20060271734A1 (en) 2003-03-21 2006-11-30 Strange Stephen H Location-independent RAID group virtual block management
US20070019636A1 (en) 2005-06-30 2007-01-25 Victor Lau Multi-threaded transmit transport engine for storage devices
US20070026887A1 (en) * 2005-07-27 2007-02-01 Mitsubishi Denki Kabushiki Kaisha Method for transferring information related to at least a mobile terminal in a mobile telecommunication network
US20070070975A1 (en) 2005-09-26 2007-03-29 Toshio Otani Storage system and storage device
US20070078819A1 (en) 2003-03-14 2007-04-05 Zayas Edward R Efficient, robust file handle invalidation
US20070162631A1 (en) 2005-12-28 2007-07-12 International Business Machines Corporation Method for selectable software-hardware internet SCSI
US20070168693A1 (en) 2005-11-29 2007-07-19 Pittman Joseph C System and method for failover of iSCSI target portal groups in a cluster environment
US20070174850A1 (en) 2006-01-20 2007-07-26 Uri El Zur Method and System for HBA Assisted Storage Virtualization
US20070208821A1 (en) 2003-12-29 2007-09-06 Pittman Joseph C System and method for zero copy block protocol write operations
US7287090B1 (en) 2000-12-21 2007-10-23 Noatak Software, Llc Method and system for identifying a computing device in response to a request packet
US7287258B1 (en) 2001-08-31 2007-10-23 International Business Machines Corporation Thread consistency support system and method
US20070291779A1 (en) * 2006-06-20 2007-12-20 Wunderlich Mark W Techniques to process network protocol units
US20080082966A1 (en) 2006-09-29 2008-04-03 Siemens Aktiengesellschaft System for creating and running a software application for medical imaging
US7373646B1 (en) * 2003-04-04 2008-05-13 Nortel Network Limited Method and apparatus for sharing stack space between multiple processes in a network device
US20080114814A1 (en) * 2006-11-14 2008-05-15 Microsoft Corporation Managing user customizations of pre-provisioned contexts
US20080134196A1 (en) 2005-05-19 2008-06-05 Intel Corporation Apparatus, System, and Method of a Memory Arrangement for Speculative Multithreading
US20080163215A1 (en) 2006-12-30 2008-07-03 Hong Jiang Thread queuing method and apparatus
US20080184233A1 (en) 2007-01-30 2008-07-31 Norton Scott J Abstracting a multithreaded processor core to a single threaded processor core
US20080244118A1 (en) 2007-03-28 2008-10-02 Jos Manuel Accapadi Method and apparatus for sharing buffers
US7443836B2 (en) 2003-06-16 2008-10-28 Intel Corporation Processing a data packet
US20080301299A1 (en) * 2007-05-29 2008-12-04 Microsoft Corporation Automatically targeting and filtering shared network resources
US20090006502A1 (en) 2007-06-26 2009-01-01 Microsoft Corporation Application-Specific Heap Management
US7500239B2 (en) 2003-05-23 2009-03-03 Intel Corporation Packet processing system
US7502870B1 (en) * 2004-08-13 2009-03-10 Sun Microsystems, Inc. Method for receiving network communication and apparatus for performing the same
US20090125907A1 (en) 2006-01-19 2009-05-14 Xingzhi Wen System and method for thread handling in multithreaded parallel computing of nested threads
US7581163B1 (en) * 2006-06-01 2009-08-25 Juniper Networks, Inc. Detection of corrupted memory pointers within a packet-processing device
US7630402B2 (en) * 2006-01-06 2009-12-08 Fujitsu Limited Media access layer processors for network communications
US7664823B1 (en) 2003-09-24 2010-02-16 Cisco Technology, Inc. Partitioned packet processing in a multiprocessor environment
US7664938B1 (en) 2004-01-07 2010-02-16 Xambala Corporation Semantic processor systems and methods
US7899044B2 (en) * 2006-06-08 2011-03-01 Alcatel Lucent Method and system for optimizing resources for establishing pseudo-wires in a multiprotocol label switching network
US7953878B1 (en) 2007-10-09 2011-05-31 Netapp, Inc. Multi-threaded internet small computer system interface (iSCSI) socket layer
US8135033B2 (en) * 2005-06-01 2012-03-13 Agient Technologies, Inc. Method of communicating between layers of a protocol stack and apparatus therefor

Patent Citations (72)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5021945A (en) * 1985-10-31 1991-06-04 Mcc Development, Ltd. Parallel processor system for processing natural concurrencies and method therefor
US4847755A (en) * 1985-10-31 1989-07-11 Mcc Development, Ltd. Parallel processing method and apparatus for increasing processing throughout by parallel processing low level instructions having natural concurrencies
US5410675A (en) * 1989-08-21 1995-04-25 Lisa M. Shreve Method of conforming input data to an output data structure and engine for accomplishing same
US5535338A (en) * 1993-07-28 1996-07-09 3Com Corporation Multifunction network station with network addresses for functional units
US6078564A (en) 1996-08-30 2000-06-20 Lucent Technologies, Inc. System for improving data throughput of a TCP/IP network connection with slow return channel
US20040030745A1 (en) 1997-10-14 2004-02-12 Boucher Laurence B. Method and apparatus for distributing network traffic processing on a multiprocessor computer
US6363477B1 (en) * 1998-08-28 2002-03-26 3Com Corporation Method for analyzing network application flows in an encrypted environment
US6480507B1 (en) * 1998-11-19 2002-11-12 Nortel Networks Limited Communication protocol stack apparatus and method of implementing same
US6338078B1 (en) 1998-12-17 2002-01-08 International Business Machines Corporation System and method for sequencing packets for multiprocessor parallelization in a computer network system
US6981140B1 (en) * 1999-08-17 2005-12-27 Hewlett-Packard Development Company, L.P. Robust encryption and decryption of packetized data transferred across communications networks
US7020713B1 (en) 2000-10-10 2006-03-28 Novell, Inc. System and method for balancing TCP/IP/workload of multi-processor system based on hash buckets
US6922557B2 (en) * 2000-10-18 2005-07-26 Psion Teklogix Inc. Wireless communication system
US7065568B2 (en) * 2000-11-30 2006-06-20 Microsoft Corporation System and method for managing states and user context over stateless protocols
US7353266B2 (en) * 2000-11-30 2008-04-01 Microsoft Corporation System and method for managing states and user context over stateless protocols
US7287090B1 (en) 2000-12-21 2007-10-23 Noatak Software, Llc Method and system for identifying a computing device in response to a request packet
US7290039B1 (en) * 2001-02-27 2007-10-30 Microsoft Corporation Intent based processing
US7349953B2 (en) * 2001-02-27 2008-03-25 Microsoft Corporation Intent based processing
US20050154783A1 (en) * 2001-02-27 2005-07-14 Microsoft Corporation Intent based processing
US6820127B2 (en) 2001-05-03 2004-11-16 International Business Machines Corporation Method, system, and product for improving performance of network connections
US20020194338A1 (en) 2001-06-19 2002-12-19 Elving Christopher H. Dynamic data buffer allocation tuning
US7287258B1 (en) 2001-08-31 2007-10-23 International Business Machines Corporation Thread consistency support system and method
US20050243734A1 (en) 2001-09-28 2005-11-03 Nemirovsky Mario D Multi-threaded packet processing engine for stateful packet processing
US20030069920A1 (en) 2001-09-28 2003-04-10 Melvin Stephen W. Multi-threaded packet processing engine for stateful packet processing
US6904040B2 (en) 2001-10-05 2005-06-07 International Business Machines Corporaiton Packet preprocessing interface for multiprocessor network handler
US20030081615A1 (en) 2001-10-22 2003-05-01 Sun Microsystems, Inc. Method and apparatus for a packet classifier
US6990662B2 (en) * 2001-10-31 2006-01-24 Hewlett-Packard Development Company, L.P. Method and system for offloading execution and resources for resource-constrained networked devices
US6920501B2 (en) 2001-12-17 2005-07-19 Ntt Docomo, Inc. Communication socket migration among different devices
US7127717B2 (en) * 2001-12-25 2006-10-24 Hitachi, Ltd. Hierarchical server system
US20050060365A1 (en) * 2002-01-24 2005-03-17 Robinson Scott L. Context-based information processing
US20030140088A1 (en) * 2002-01-24 2003-07-24 Robinson Scott H. Context-based information processing
US20060013222A1 (en) 2002-06-28 2006-01-19 Brocade Communications Systems, Inc. Apparatus and method for internet protocol data processing in a storage processing device
US20040148376A1 (en) 2002-06-28 2004-07-29 Brocade Communications Systems, Inc. Storage area network processing device
US20040216125A1 (en) 2002-08-26 2004-10-28 Interdigital Technology Corporation Wireless device operating system (OS) application programmer's interface (API)
US20040205754A1 (en) 2002-08-26 2004-10-14 Interdigital Technology Corporation Software porting layer
US20070078819A1 (en) 2003-03-14 2007-04-05 Zayas Edward R Efficient, robust file handle invalidation
US20060271734A1 (en) 2003-03-21 2006-11-30 Strange Stephen H Location-independent RAID group virtual block management
US7529823B2 (en) * 2003-03-27 2009-05-05 Microsoft Corporation Notifications for shared resources
US20040193678A1 (en) * 2003-03-27 2004-09-30 Microsoft Corporation Notifications for shared resources
US7373646B1 (en) * 2003-04-04 2008-05-13 Nortel Network Limited Method and apparatus for sharing stack space between multiple processes in a network device
US7500239B2 (en) 2003-05-23 2009-03-03 Intel Corporation Packet processing system
US20040246956A1 (en) * 2003-06-06 2004-12-09 Meng David Qiang Parallel packet receiving, routing and forwarding
US7443836B2 (en) 2003-06-16 2008-10-28 Intel Corporation Processing a data packet
US7664823B1 (en) 2003-09-24 2010-02-16 Cisco Technology, Inc. Partitioned packet processing in a multiprocessor environment
US20050138091A1 (en) 2003-12-22 2005-06-23 Jean-Pierre Bono Prefetching and multithreading for improved file read performance
US20070208821A1 (en) 2003-12-29 2007-09-06 Pittman Joseph C System and method for zero copy block protocol write operations
US7664938B1 (en) 2004-01-07 2010-02-16 Xambala Corporation Semantic processor systems and methods
US20050226235A1 (en) 2004-04-08 2005-10-13 Alok Kumar Apparatus and method for two-stage packet classification using most specific filter matching and transport level sharing
US20060013253A1 (en) 2004-07-16 2006-01-19 Hufferd John L Method, system, and program for forwarding messages between nodes
US7502870B1 (en) * 2004-08-13 2009-03-10 Sun Microsystems, Inc. Method for receiving network communication and apparatus for performing the same
US20060059287A1 (en) * 2004-09-10 2006-03-16 Pleora Technologies Inc. Methods and apparatus for enabling bus connectivity over a data network
US20060242532A1 (en) 2005-04-26 2006-10-26 Intel Corporation Techniques to provide information validation and transfer
US20080134196A1 (en) 2005-05-19 2008-06-05 Intel Corporation Apparatus, System, and Method of a Memory Arrangement for Speculative Multithreading
US8135033B2 (en) * 2005-06-01 2012-03-13 Agient Technologies, Inc. Method of communicating between layers of a protocol stack and apparatus therefor
US20070019636A1 (en) 2005-06-30 2007-01-25 Victor Lau Multi-threaded transmit transport engine for storage devices
US20070026887A1 (en) * 2005-07-27 2007-02-01 Mitsubishi Denki Kabushiki Kaisha Method for transferring information related to at least a mobile terminal in a mobile telecommunication network
US20070070975A1 (en) 2005-09-26 2007-03-29 Toshio Otani Storage system and storage device
US20070168693A1 (en) 2005-11-29 2007-07-19 Pittman Joseph C System and method for failover of iSCSI target portal groups in a cluster environment
US20070162631A1 (en) 2005-12-28 2007-07-12 International Business Machines Corporation Method for selectable software-hardware internet SCSI
US7630402B2 (en) * 2006-01-06 2009-12-08 Fujitsu Limited Media access layer processors for network communications
US20090125907A1 (en) 2006-01-19 2009-05-14 Xingzhi Wen System and method for thread handling in multithreaded parallel computing of nested threads
US20070174850A1 (en) 2006-01-20 2007-07-26 Uri El Zur Method and System for HBA Assisted Storage Virtualization
US7581163B1 (en) * 2006-06-01 2009-08-25 Juniper Networks, Inc. Detection of corrupted memory pointers within a packet-processing device
US7899044B2 (en) * 2006-06-08 2011-03-01 Alcatel Lucent Method and system for optimizing resources for establishing pseudo-wires in a multiprotocol label switching network
US20070291779A1 (en) * 2006-06-20 2007-12-20 Wunderlich Mark W Techniques to process network protocol units
US20080082966A1 (en) 2006-09-29 2008-04-03 Siemens Aktiengesellschaft System for creating and running a software application for medical imaging
US20080114814A1 (en) * 2006-11-14 2008-05-15 Microsoft Corporation Managing user customizations of pre-provisioned contexts
US20080163215A1 (en) 2006-12-30 2008-07-03 Hong Jiang Thread queuing method and apparatus
US20080184233A1 (en) 2007-01-30 2008-07-31 Norton Scott J Abstracting a multithreaded processor core to a single threaded processor core
US20080244118A1 (en) 2007-03-28 2008-10-02 Jos Manuel Accapadi Method and apparatus for sharing buffers
US20080301299A1 (en) * 2007-05-29 2008-12-04 Microsoft Corporation Automatically targeting and filtering shared network resources
US20090006502A1 (en) 2007-06-26 2009-01-01 Microsoft Corporation Application-Specific Heap Management
US7953878B1 (en) 2007-10-09 2011-05-31 Netapp, Inc. Multi-threaded internet small computer system interface (iSCSI) socket layer

Non-Patent Citations (15)

* Cited by examiner, † Cited by third party
Title
Biswas, Anumita. "Application-Controlled Network Packet Classification", U.S. Appl. No. 11/936,141, filed Nov. 7, 2007, 52 pages.
Bjorkman, Mats et al, "Performance Modeling of Multiprocessor Implementations of Protocols," IEEE/ACM Transactions on Networking, vol. 6, No. 3, Jun. 1998.
Ito, Robert Mabo et al., "A Multiprocessor Approach for Meeting the Processing Requirements for OSI," IEEE Journal on Selected Areas in Communications, vol. 11, No. 2, Feb. 1993.
Kaiserwerth, Matthias, "The Parallel Protocol Engine," IEEE/ACM Transactions on Networking, vol. 1, No. 6, Dec. 1993.
Leue, Stephan et al., "On Parallelizing and Optimizing the Implementation of Communication Protocols," IEEE/ACM Transactions on Networking, vol. 4, No. 1, Feb. 1996.
Lin, Xiaojun; Shroff, Ness B.; Srikant, R., "A Tutorial on Cross-Layer Optimization in Wireless Networks", © 2006, IEEE, IEEE Journal on Selected Areas in Communications, vol. 24, No. 8, pp. 1452-1463. *
Millberg, Mikael; Nillson, Erland; Thid Rikard; Kumar, Shashi; Jantsch, Axel, "The Nostrum Backbone-a Communication Protocol Stack for Networks on Chip", © 2004, IEEE, Proceedings of the 17th Internation Conference on VLSI Design. *
Millberg, Mikael; Nillson, Erland; Thid Rikard; Kumar, Shashi; Jantsch, Axel, "The Nostrum Backbone—a Communication Protocol Stack for Networks on Chip", © 2004, IEEE, Proceedings of the 17th Internation Conference on VLSI Design. *
Nahum, Erich M et al., "Performance Issues in Parallelized Network Protocols," Proceedings of the First Symposium on Operating Systems Design and Implementation, Usenix Association, Nov. 1994.
Salehi, James D. et al., "The Effectiveness of Affinity-Based Scheduling in Multiprocessor Network Protocol Processing (Extended Version)," IEEE/ACM Transactions on Networking, vol. 4, No. 4, Aug. 1996.
Schmidt, Douglas C. et al., "Measuring the Performance of Parallel Message-based Process Architectures," INFOCOM '95: Proceedings of the Fourteenth Annual Joint Conference of the IEEE Computer and Communication Societies (vol. 2), Apr. 1995.
TechTarget, "OSI (Open System Interconnection)", © Oct. 10, 2006 http://searchnetworking.techtarget.com/definition/OSI. *
Wikipedia, "OSI model", © Dec. 23, 2004 http://en.wikipedia.org/w/index.php?title=OSI-model&oldid=9028529. *
Wikipedia, "OSI model", © Dec. 23, 2004 http://en.wikipedia.org/w/index.php?title=OSI—model&oldid=9028529. *
Yates, David J. et al., "Networking Support for Large Scale Multiprocessor Servers," Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, May 1996.

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9961021B2 (en) 2012-04-19 2018-05-01 Cisco Technology, Inc. Enabling applications in a multi-transport stack environment
US9535759B2 (en) * 2014-05-23 2017-01-03 Osr Open Systems Resources, Inc. Work queue thread balancing
US9785473B2 (en) 2014-07-14 2017-10-10 Nxp Usa, Inc. Configurable per-task state counters for processing cores in multi-tasking processing systems
US20170052864A1 (en) * 2015-08-21 2017-02-23 International Business Machines Corporation Automatic connection detection in a data center
US9875168B2 (en) * 2015-08-21 2018-01-23 International Business Machines Corporation Automatic connection detection in a data center

Similar Documents

Publication Publication Date Title
Pfister An introduction to the infiniband architecture
Prylli et al. BIP: a new protocol designed for high performance networking on myrinet
Dunning et al. The virtual interface architecture
Jose et al. Memcached design on high performance rdma capable interconnects
US6535518B1 (en) System for bypassing a server to achieve higher throughput between data network and data storage system
US5805823A (en) System and method for optimal multiplexed message aggregation between client applications in client-server networks
US6424621B1 (en) Software interface between switching module and operating system of a data packet switching and load balancing system
US6901580B2 (en) Configuration parameter sequencing and sequencer
US6272522B1 (en) Computer data packet switching and load balancing system using a general-purpose multiprocessor architecture
US6014699A (en) Internet protocol assists for high performance LAN connections
US7321926B1 (en) Method of and system for allocating resources to resource requests
US7502884B1 (en) Resource virtualization switch
US6847647B1 (en) Method and apparatus for distributing traffic over multiple switched fiber channel routes
US6888792B2 (en) Technique to provide automatic failover for channel-based communications
US6928478B1 (en) Method and apparatus for implementing a MAC address pool for assignment to a virtual interface aggregate
US6308238B1 (en) System and method for managing connections between clients and a server with independent connection and data buffers
Chun et al. Virtual network transport protocols for Myrinet
US5999974A (en) Internet protocol assists for high performance LAN connections
US6614808B1 (en) Network packet aggregation
US6084859A (en) Internet protocol assists using multi-path channel protocol
US20090064140A1 (en) System and Method for Providing a Fully Non-Blocking Switch in a Supernode of a Multi-Tiered Full-Graph Interconnect Architecture
US7274706B1 (en) Methods and systems for processing network data
US20020133593A1 (en) Systems and methods for the deterministic management of information
US5951706A (en) Method of independent simultaneous queueing of message descriptors
US20040037313A1 (en) Packet data service over hyper transport link(s)

Legal Events

Date Code Title Description
AS Assignment

Owner name: NETAPP, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BISWAS, ANUMITA;SINGH, VIJAY;SON, SONNY;AND OTHERS;SIGNING DATES FROM 20080313 TO 20080321;REEL/FRAME:020767/0737

FPAY Fee payment

Year of fee payment: 4