JP2002305535A - Method and apparatus for providing a reliable protocol for transferring data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method, apparatus and computer implemented instructions for transferring data. SOLUTION: A request is sent by a requester to a responder. The request includes the amount of available processing space at the requester. When the request is received from the responder, data are identified using the request. The data are placed into a plurality of subsequences of data packets for transfer to the requester, wherein each packet within the set of subsequences hold data in the amount less than or equal to the amount of available space. These subsequences are then sent to the requester one subsequence at a time. A new subsequence is sent each time the available processing space at the requester becomes free to process data from another subsequence.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates generally to improved network data processing systems and, more particularly, to methods and apparatus for managing network data processing systems. More specifically, the present invention relates to a method and apparatus for transferring data using a set of data packets.

[0002]

2. Description of the Related Art A system area network (SA)
In N), a message passing mechanism is provided by hardware, which can be used for input / output devices and inter-process communication (IPC) between general purpose computing nodes. A process running on the device accesses the SAN message passing hardware by posting a send / receive message to access the SAN channel
Send / receive the work queue on the adapter (CA). These processes are also referred to as "consumers." The send / receive work queue (WQ) is assigned to consumers as a queue pair (QP). The message has five different transport types: Reliable Con
nected (RC, reliable connection), Reliable datagram
(RD, reliable datagram), Unreliable Conne
cted (UC, unreliable connection), Unreliable Datagra
m (UD, unreliable datagram), Raw Datagram
(RawD, raw datagram). The consumer completes the SAN transmission / reception work (W
Retrieve the results of these messages from the completion queue (CQ) via C). The source channel adapter is
Handles outbound message segmentation and transmission to the destination. Destination·
The channel adapter handles the reassembly of inbound messages and their placement in the memory space specified by the consumer of the destination. Two channel adapter types, a host channel adapter (HCA) and a target
There is a channel adapter (TCA). Host channel adapters are used by general purpose computing nodes to access the SAN fabric. Consumers use the SAN verb to
Access channel adapter functions. Software that interprets verbs and accesses the channel adapter directly is called a channel interface (CI).

[0003]

The network management operations, especially during configuration, are often performed by relatively simple routines having limited information transfer and storage capabilities. For this reason, the unreliable datagram message type is used and the datagram length is limited to a fixed small value. Many management operations can be accomplished by forwarding a single datagram or a request / response datagram pair. If an error occurs during these simple operations, the entire operation can be repeated without incurring excessive overhead. Other management operations require the transfer of large amounts of data that cannot be contained in a single datagram. These operations require means for correlating the datagrams involved, recognizing lost datagrams, and recovering lost datagrams. Also, in a network management "Get" operation where the requester requests data from the database, the requester does not know the amount of data to be returned. There is a need for a scheme in which the requester sets an upper limit on the amount of data returned and continues to return more data about the request if additional data is available. Finally, in a network management "Set" operation where a large amount of data is sent to the receiver, the data receiver may not be initially ready to receive all the data.

Accordingly, it would be advantageous to have an improved method and apparatus that limit the amount of initial data transmitted and provide the receiving party with a means to request the next data when it is ready to receive the data. It is.

[0005]

SUMMARY OF THE INVENTION The present invention provides a method, apparatus, and computer-implemented instructions for transferring data in response to a request and for transferring data to a receiver without receiving a previous request. . When data is transferred in response to a request, the request is sent by the request issuer to the responder. The request includes the amount of processing space available on the request issuer side. When a request is received from the responder, data is identified using the request. The data is arranged into a plurality of data packets for transfer to the requestor, with each packet of the set of packets holding a fixed amount of data. Packets are grouped into sub-sequences of packets, each of which holds an amount of data less than or equal to the amount of available space. These subsequences are then sent to the requester. The new subsequence has available processing space available on the request issuer side,
Sent whenever data from another subsequence can be processed. The requester responds to the request each time the amount of available processing space becomes available.
A sub-sequence of packets is received, and the data in each sub-sequence of the data packet fits into the amount of available processing space.

When data is sent to a receiver without a previous request, the sender first sends a sub-sequence of data packets holding up to a default amount of data. The sender then sends an additional subsequence of data each time the amount of processing space on the receiver becomes free.

[0007] The novel features believed characteristic of the invention are set forth in the following claims. However, the invention itself, as well as preferred aspects, further objects, and advantages of its use, are best understood by referring to the following detailed description of exemplary embodiments in conjunction with the accompanying drawings.

[0008]

Referring now to the drawings, and in particular to FIG. 1, there is shown a diagram of a network global change computing system according to a preferred embodiment of the present invention. The distributed computer shown in FIG.
The system uses a System Area Network (SA)
N) The embodiments of the present invention, which are in the form of 100 and are provided merely by way of example, and described below, may be implemented on many other types and configurations of computer systems. For example, a computer system embodying the present invention can be a small server with one processor and a few I / O adapters to a massively parallel supercomputer with hundreds or thousands of processors and thousands of I / O adapters -It can extend to the system. Further, the invention can be implemented within the infrastructure of a remote computer system connected by the Internet or an intranet. SAN 100 is a high bandwidth, low latency network interconnect node in a distributed computer system. A node is a component that is connected to one or more links of a network and that forms the origin or destination of a message in the network.
In the illustrated example, the SAN 100 has a host processor
Node 102, host processor node 104, R
An AID (new magnetic disk control) subsystem node 106 and nodes in the form of I / O chassis nodes 108 are included. The nodes shown in FIG. 1 are for illustration purposes only. The SAN 100 can be connected to any number of all types of independent processing nodes, I / O adapter nodes, and I / O device nodes. Any of these nodes can function as end nodes, defined herein as devices that originate or ultimately consume messages or frames within SAN 100.

In one embodiment of the present invention, an error handling mechanism is present in the distributed computer system, and the error handling mechanism allows reliable c between end nodes in a distributed computing system such as SAN 100.
Onnection or reliable datagram communication becomes possible.

[0010] A message, as used herein, is an application-defined unit of data exchange and a primitive unit of communication between cooperating processes. A packet is a unit of data encapsulated by a networking protocol header or networking protocol trailer. The header is generally
Provides control and routing information for transmitting frames over the SAN. Trailers typically include control data and cyclic redundancy check (CRC) data to prevent packets from being destroyed and delivered. SAN 100 includes a communication management infrastructure that supports both input / output and inter-process communication (IPC) in a distributed computer system. The SAN 100 shown in FIG. 1 includes a switched communication fabric 116 that allows multiple devices to transfer data in parallel in a protected and remotely managed environment with high bandwidth and low latency. Become like End nodes can communicate over multiple ports and can use multiple paths through the SAN fabric. Multiple ports and SA shown in FIG.
The path through N can be used for data transfer with fault tolerance and improved bandwidth.

The SAN 100 shown in FIG.
2, switch 114, switch 146, and router 1
17 are included. A switch connects multiple links together and uses a small header Destination Local Identifier
A device that uses the (DLID) field to route packets from one link to another within a subnet. A router connects multiple subnets together and creates a Destination Globally Uni
A device that can route frames from one link in a first subnet to another link in a second subnet using a que Identifier (DGUID).

In one embodiment, the link is a full-duplex channel between two network fabric elements, such as end nodes, switches, or routers. Examples of suitable links include, but are not limited to, copper cables, fiber optic cables, backplanes and printed circuit copper traces on printed circuit boards.

For a trusted service type, the host processor end node and the I / O adapter
An end node, such as an end node, generates a request packet and returns an acknowledgment packet. Switches and routers pass packets from the source to the destination.
The switch passes the packet unchanged, except for a modified CRC trailer field that is updated at each stage of the network. The router updates the modified CRC trailer field and changes other fields in the header as the packet is routed.

In the SAN 100 shown in FIG. 1, the host processor node 102, the host processor node 104, and the input / output chassis node 108
At least one to interface with AN100
One channel adapter (CA) is included. In one embodiment, each channel adapter is an endpoint that implements the channel adapter interface in sufficient detail to become a source or sink of packets transmitted on SAN 100. The host processor node 102 has a host channel adapter 118 and a host
A channel adapter in the form of a channel adapter 120 is included. Host processor node 104 includes host channel adapter 122 and host channel adapter 124. Host processor
Node 102 also includes central processing units 126-130 and memory 132 interconnected by a bus system 134. Host processor node 1
04 also includes central processing units 136-140 and memory 142 interconnected by a bus system 144.

Host channel adapters 118 and 120 provide connection to switch 112 and host
Channel adapters 122 and 124 are connected to switch 1
Provide connections to 12 and 114. In one embodiment, the host channel adapter is implemented in hardware. In this embodiment, the host channel adapter hardware offloads much of the central processing unit and I / O adapter communication overhead. This hardware implementation of the host channel adapter also allows multiple simultaneous communications over a switched network without the traditional overhead associated with communication protocols. In one embodiment, the host channel adapter and SAN 100 of FIG.
Provides zero processor copy data transfer without operating system kernel processes to the system's I / O and inter-process communication (IPC) consumers and uses hardware to provide reliable fault-tolerant communication I will provide a.

As shown in FIG. 1, the router 117
Is coupled to a wide area network (WAN) connection or a local area network (LAN) connection to other hosts or other routers.

The input / output chassis node 108 of FIG. 1 includes a switch 146 and a plurality of input / output modules 14.
8 to 156 are included. In this example, the input / output module takes the form of an adapter card. The example adapter card shown in FIG. 1 includes a SCSI adapter card for the input / output module 148 and a SCSI adapter card for the input / output module 152.
fiber channel hub and fiber channel-arbitrated lo
The adapter card to the op (FC-AL) device, the Ethernet adapter card for the I / O module 150, the graphics adapter card for the I / O module 154, and the video adapter card for the I / O module 156 included. Any of the known types of adapter cards can be implemented. The I / O adapter also includes a switch in the I / O adapter backplane for coupling the adapter card to the SAN fabric. These modules also include target channel adapters 158-166. In this example, the RAID subsystem node 106 of FIG. 1 includes a processor 168, memory 170, a target channel adapter (TCA) 172, and a plurality of redundant or striped storage disk devices 174. Target channel adapter 172 may be a fully functional host channel adapter.

The SAN 100 processes data communication related to input / output and communication between processors. SAN100
It supports the high bandwidth and scalability required for I / O, and also supports the extremely low latency and low CPU overhead required for inter-processor communication.
User clients can bypass operating system kernel processes and use efficient message passing, such as host channel adapters.
It has direct access to the network communication hardware that enables the protocol. SAN100 is
Suitable for modern computing models, it is a fundamental component of new forms of I / O and computer cluster communication. Further, using the SAN 100 of FIG.
The I / O adapter nodes will be able to communicate between themselves or with any or all of the processor nodes in the distributed computer system. For I / O adapters added to SAN 100, the resulting I / O adapter node is SAN 100
It has substantially the same communication functions as the host processor nodes within.

Turning now to FIG. 2, a functional block diagram of a host processor node according to a preferred embodiment of the present invention is shown. Host processor node 200
Is an example of a host processor node, such as host processor node 102 of FIG.

In this example, the host processor node 200 shown in FIG. 2 includes a set of consumers 202-208, which are processes running on the host processor node 200. Host processor node 200 also includes channel adapter 210 and channel adapter 212. Channel adapter 210 includes ports 214 and 216,
Channel adapter 212 has ports 218 and 2
20 are included. Each port is connected to a link. These ports are connected to a single S, such as SAN 100 in FIG.
It can be connected to an AN subnet or multiple SAN subnets. In this example, the channel adapter is
Takes the form of a host channel adapter. Consumers 202 through 208 are connected to verb interface 222
And forward the message to the SAN via the message and data service 224. The verb interface is essentially an abstraction of the functionality of the host channel adapter. The operating system can expose some or all of the verb functionality through its programming interface. Basically, this interface defines the behavior of the host.

Further, the host processor node 20
0 includes the message and data service 224,
Is a higher-level interface than the verb layer, with channel adapters 210 and 212
Used for processing messages and data received via Message and data service 224
Provides an interface to consumers 202-208 for processing messages and other data.

Referring now to FIG. 3, a host channel adapter according to a preferred embodiment of the present invention is shown. Host channel adapter 3 shown in FIG.
00, a set of queue pairs (QPs) 302-310
And these queue pairs are used to transfer messages to the host channel adapter ports 312-316.

Host channel adapter port 31
The buffering of data into 2-316 is channelized via virtual lanes (VL) 318-334, each VL having its own flow control. A subnet manager configures the channel adapter with the local address of each physical port, or port LID. A subnet manager agent (SMA) 336 is an entity that communicates with the subnet manager to configure a channel adapter. The memory translation and protection (MTP) 338 is a mechanism that translates a virtual address into a physical address and performs a validity check of an access right. Direct memory access (DMA) 340
Is stored in memory 342 for queue pairs 302-310.
Provides a direct memory access operation using

A single channel adapter, such as the host channel adapter 300 shown in FIG. 3, can support thousands of queue pairs. In contrast, target channel adapters in I / O adapters typically support a much smaller number of queue pairs.

Each queue pair has a transmission work queue (SWQ).
And a receiving work queue. The send work queue is used to send channel semantic messages and memory semantic messages.
The receive work queue receives channel semantic messages. The consumer invokes an operating system-specific programming interface, referred to herein as a verb, to place a work request (WR) on a work queue.

Referring now to FIG. 4, there is shown a diagram illustrating the processing of a work request according to a preferred embodiment of the present invention. In FIG. 4, the reception work queue 400, the transmission work queue 402, and the completion queue 404 are
Exist to handle requests from the consumer 06 and requests for the consumer 406. These consumers 40
6 is finally sent to the hardware 408. In this example, the consumer 406 determines that the work request 41
Generate 0 and 412 and receive work done 414.
As can be seen from FIG. 4, the work request placed in the work queue is called a work queue element (WQE). The transmission work queue 402 includes work queue elements (WQEs) 422 to 42 that describe data to be transmitted to the SAN fabric.
8 is included. The receive work queue 400 has a work queue element (WQE) 41 that describes where to put incoming channel semantic data from the SAN fabric.
6 to 420 are included. Work queue elements are handled by hardware 408 in the host channel adapter.

Verbs also provide a mechanism to retrieve completed work from completion queue 404. As can be seen from FIG.
The completion queue 404 has a completion queue element (CQE) 43
0 to 436 are included. The completion queue element includes information about a previously completed work queue element. Completion queue 404 is used to create a single point of completion notification for multiple queue pairs. The completion queue element is
It is a data structure on a completion queue. This element describes a completed work queue element. The completion queue element contains sufficient information to determine the completed queue pair and the particular work queue element. A completion queue context is a block of information that contains pointers to individual completion queues, lengths, and other information needed to manage each completion queue.

An example of a work request supported for the send work queue 402 shown in FIG. 4 is as follows.
The send work request is sent to the data segment referenced by the receive work queue element at the remote node.
A channel semantic operation that pushes a set of data segments. For example, work queue element 4
28, data segment 4 438, data segment 5 440, and data segment 6 44
2 is included. Each of the data segments of the transmission work request includes a virtually contiguous memory area. The virtual address used to reference the local data segment is in the address context of the process that created the local queue pair.

Remote direct memory access (RDM
A) A read operation request provides a memory semantic operation for reading a virtually contiguous memory space on a remote node. The memory space can be either part of a memory area or part of a memory window. The memory area refers to a set registered before a virtual contiguous memory address defined by a virtual address and a length. A memory window refers to a set of virtually contiguous memory addresses bound to a previously registered area.

An RDMA Read operation request reads a virtually contiguous memory space on a remote end node and writes the data to a virtually contiguous local memory space. Similar to the send work request, the virtual address used by the RDMA Read work queue element that references the local data segment is in the address context of the process that created the local queue pair. For example, the work queue element 416 of the receive work queue 400
Segment 1 444, data segment 2 44
6, and data segment 3 448.
The remote virtual address is in the address context of the process that owns the remote queue pair targeted by the RDMA Read work queue element.

The RDMA Write work queue element allows virtually contiguous memory on a remote node.
A memory semantic operation for writing to a space is provided. The RDMA Write work queue element contains a distributed list of virtual addresses in the local virtual contiguous memory space and the remote memory space where the local memory space is written.

The RDMA FetchOp work queue element provides a memory semantic operation that performs atomic operations on remote words. The RDMA FetchOp work queue element is a combination of an RDMA Read operation, a Modify operation, and an RDMA Write operation. RDMA
The FetchOp work queue element is Compare and Swap i
Multiple read-modify-write operations, such as f equal, can be supported.

A host channel that modifies (destroys) a memory window by associating (disassociating) the memory window with a memory area through a bind (unbind) remote access key (R_Key) work queue element. -Commands for the adapter hardware are provided. R_Key is part of each RDMA access and is used to verify that the remote process has granted access to the buffer.

In one embodiment, the receiving work queue 4 of FIG.
00 supports only one type of work queue element, called a receive work queue element. The receive work queue element provides a channel semantic operation that describes the local memory space to which incoming sent messages are written. The receive work queue element includes a distributed list describing a plurality of virtually contiguous memory spaces. Incoming send messages are written to these memory spaces. The virtual address is in the address context of the process that created the local queue pair.

Regarding the inter-process communication, the user mode
The software process transfers data via the queue pair directly from where the buffer resides in memory. In one embodiment, the transfer over the queue pair bypasses the operating system and consumes fewer host instruction cycles. Queue pairs allow zero processor copy data transfers without operating system kernel intervention. 0 processor copy
Data transfer provides efficient support for high bandwidth, low latency communication.

When a queue pair is created, the queue pair:
Set to provide the selected type of transport service. In one embodiment, a distributed computer system implementing the present invention supports four types of transport services.

[0037] Reliable connected service and Unreli
In the able connected service, the local queue pair is
Associated with only one remote queue pair. Conn
The expected service is a process where one process is
You need to create a queue pair, and this queue pair
For communicating over a SAN fabric.
You. Therefore, the N host processor nodes
Each contains P processes, and each node has P
All of the processes are running on all other nodes
If you want to communicate with the process,
The P ^Two× (N-1) queue pairs are required
is there. In addition, the process uses the queue pair as the same host
To another queue pair on the channel adapter
Can be.

In the Reliable datagram service, a local end-to-end (EE) context is associated with only one remote end-to-end context. The reliable datagram service allows a client process in one queue pair to communicate with any other queue pair on any other remote node. In the receive work queue, reliable datag
The ram service allows incoming messages from any outgoing work queue on any other remote node. Because reliable datagram services are connectionless, scalability is significantly improved with reliable datagram services. Therefore, an end node having a fixed number of queue pairs is a reliable c
A much larger number of processes and end nodes using reliable datagram services can be communicated than with the onnection transport service. For example, if each of the N host processor nodes contains P processes, and all of the P processes at each node want to communicate with all the processes at all other nodes, In the reliable connection service, each node requires P ² × (N−1) queue pairs. In comparison, connectionless reliable
The datagram service requires only P queue pairs + (N-1) EE contexts at each node for exactly the same number of communications.

The unreliable datagram service is connectionless. unreliable datagram service
Used by management applications to discover new switches, routers, and end nodes and integrate them into a given distributed computer system. unreliable
The datagram service does not provide reliable connection service or reliable datagram service reliability guarantee. Therefore, the unreliable datagram service
It operates with less state information maintained at each end node. Turning now to FIG. 5, a diagram of a data packet is shown in accordance with a preferred embodiment of the present invention. Message data 500 includes data segment 1 502, data segment 2 504, and data segment 3 506, which are similar to the data segments shown in FIG. In this example, these data segments form packet 508, which is placed in packet payload 510 within data packet 512. In addition, the data packet 512 may include a CRC used for error checking.
514 are included. In addition, a routing header 516 and a transport header 518 are present in the data packet 512. The routing header 516 is used to identify the source port and the destination port of the data packet 512. In the transport header 518 in this example, the data packet 5
Twelve destination queue pairs are designated.

Further, the transport header 518
Also provides information about the data packet 512, such as instruction code, packet sequence number, and partition. The opcode identifies whether the packet is the beginning, end, middle, or only packet of a message. Operation is transmitted RD according to the instruction code
It is also specified whether it is an MA write, read, or atomic. The packet sequence number is initialized when communication is established and is incremented each time a queue pair creates a new packet. The ports of the end nodes can be configured to be members of one or more possibly overlapping sets called partitions.

When a reliable transport service is used, when the request packet reaches its destination end node, the destination
The end node uses the acknowledgment packet to inform the request packet sender that the request packet has been verified at the destination and accepted. The acknowledgment packet acknowledges one or more valid and accepted request packets. The requester may have multiple pending request packets before receiving the acknowledgment. In one embodiment, the number of multiple pending messages is determined when creating the QP.

The present invention provides a mechanism for managing the transfer of data between a requester requesting data and a responder sending the requested data back to the requester. The present invention also provides for managing the transfer of data between a sender and a receiver that has not previously sent a request for data. These mechanisms use data packets to provide reliable data transfer through requester and responder or sender and receiver processes while using unreliable datagrams referred to as managed datagrams (MADs). This is implemented by placing the fields within. The mechanism by which the requester requests data includes the request issuer sending a query or request for data. In this case, the size of the response and the amount of data are unknown. The request includes the amount of buffer space available at the requestor to process the data returned in the response. The responder responds by sending a data packet containing data that responds to the request. In these examples, the data is included in the sequence of the MAD. MAD
Does not exceed the amount of buffer space available at the requestor. Instead of transmitting all data in a single sequence of data packets, a series of MADs is transmitted in this manner. These sequences of data packets are also called subsequences.

After transmitting a subsequence, the responder waits for a response indicating the correct reception of the previous subsequence and the availability of buffer space before transmitting the next subsequence. If an error occurs, the subsequence can be retransmitted instead of retransmitting the entire response.

Furthermore, the amount of buffer space can be changed, and this change in the amount of available buffer space can be reflected in the response from the requestor. Also, the last data packet may not contain data up to the amount of available buffer space. In other words, this data packet may contain less data than the amount of available buffer space. In such a case, the fragment flag in the data packet is set to indicate how full the data packet is, and the current packet is
Last transmitted data, including data responding to requests
It can be identified as a packet.

Turning now to FIG. 6, a diagram of a MAD according to a preferred embodiment of the present invention is shown. MAD600
Is an example of a packet payload field of a data packet, such as the packet payload 510 of FIG. Segment number field 602, payload length field 604, fragment flag field 606, and window parameter field 6
08 is an additional field used to provide reliable data transfer between the requester and responder. In these examples, the request issuer may be the host processor node 102 of FIG. 1 and the responder may be the RAID subsystem node 106 of FIG.

[0046] The segment number field 602 identifies the relative position of the packet in the request or response. For example, a particular packet may be the first packet in a series of packets to be processed. Alternatively, the packet may be the last packet or somewhere in the middle with respect to the data located in other packets. Segment number field 6
02 identifies the relative position of the data in the packet,
As a result, this data can be reassembled in the correct order with data from other data packets.

In these examples, the payload length field 604 is valid for the first packet of a multiple packet request and multiple packet response. The payload length field 604 specifies the overall expected length of a multiple packet request or response. The Payload Length field is also valid in the last packet of a multi-packet transmission operation or response to specify how much data is included in the last packet of the operation. To specify the number of valid data bytes in the last packet when the actual amount of data transmitted is not equal to the expected amount of data indicated in the payload length field of the first packet of the operation First, it is necessary to include the payload length in the last packet of the multi-packet operation. In the fragment flag field 606, the packet is (1)
The first or last packet of a request or response, (2) an acknowledgment packet acknowledging receipt of the packet, (3) a retransmission request packet requesting retransmission of the packet, and (4) resetting the transaction. It is specified which of the packets requires a timer to perform.

Window parameter field 60
At 8, the amount of buffer space available at the receiver for the subsequent subsequence is specified. The window parameter field is valid in request or acknowledgment packets.

FIGS. 7 and 8 show the process of a protocol that includes a multiple packet request and is transmitted in response to a request for data. 9 and 10 illustrate the process of a protocol involving the transmission of a multi-packet message without receiving a previous request.

Turning now to FIG. 7, a flowchart of a process used for requesting and receiving data is shown in accordance with a preferred embodiment of the present invention. The process shown in FIG. 7 can be performed on the request issuer side.

The process begins by sending a request (step 700). In this request, the requester includes a window field, thereby specifying the amount of buffer space or other memory space available for processing blocks of data that can be transmitted in a subsequence. . Next, a timer is started (step 702). If a subsequence is received after starting the timer, an acknowledgment is sent (step 704). This acknowledgment acknowledges receipt of the subsequence. Thereafter, a determination is made as to whether the last data packet of the entire request has been received (step 706). Such identification can be performed by using a fragment flag located in a fragment flag field, such as the fragment flag field 606 of FIG. If the last data packet has been received, the process ends.

Referring again to step 702, if a timeout or error occurs after starting the timer, a determination is made as to whether the maximum number of retries has been exceeded (step 708). If the maximum number of allowed retries has not been exceeded, a retransmission request is sent (step 710) and the process returns to step 702. The timeout occurs when the timer expires. An error can occur if a data packet was not received correctly, or if the data packet contains an error after error checking. If the maximum number of allowed retries has been exceeded, the resources allocated to the operation are released (step 712), with the process terminating thereafter.

If a keep-alive response is received after starting the timer, the process returns to step 702. In these examples, a keep-alive response is received from the responder if additional time is required to transfer the data to the requestor.

Referring again to step 706, if the last data packet has not been received, the process returns to step 702.

Turning now to FIG. 8, there is illustrated a flowchart of a process used to process a request for data, according to a preferred embodiment of the present invention. The process illustrated in FIG. 8 may be performed on the responder in these examples.

The process begins by receiving a request from a request issuer (step 800). next,
Access data (step 802). This data is data corresponding to the request. The amount of data accessed is equal to the length specified in the request's window field. Then, the data is transmitted (step 8).
04). The data transmitted in step 804 is transmitted in a sub-sequence that includes an amount of data up to the amount of space made available by the requestor specified in the window field. Further, if the data is for the last data packet sent to the user, a fragment flag is set to indicate that the data packet is the last data packet. Next, the process waits for an acknowledgment from the requestor (step 806). If an acknowledgment is received, a determination is made as to whether the last data packet has been transmitted (step 808). Last data
If the packet has been sent, the process ends.

Referring again to step 802, if more time is required to access the data,
A keepalive is sent (step 810) and the process returns to step 802.

Referring again to step 806, if a retransmission request is received while waiting for an acknowledgment, the process returns to step 804. This retransmission request may be received in response to an error in the data received by the request issuer. If a timeout is received while waiting for an acknowledgment, resources are released (step 812), and the process ends thereafter.

Turning now to FIG. 9, there is shown a flowchart of a process used to receive data from a sender that has not sent a previous request, according to a preferred embodiment of the present invention.

The process begins by receiving the first subsequence (step 900). This first subsequence contains a default amount of data and contains information specifying the overall amount of data transmitted for the transmission operation. Next, an acknowledgment is transmitted to the transmitting side (step 902). A timer is started (step 904). In step 904, when the entire sub-sequence is received, an acknowledgment is sent back to the transmitting side (step 906). After that, the last data
A determination is made as to whether a packet has been received (step 908). If the last data packet of the entire transmission operation has been received, the process ends. This data packet is the last data packet of the last subsequence of the data packet.

Referring again to step 904, if a keepalive has been received, the process proceeds to step 9
Return to 04. If an error or timeout has occurred, a determination is made as to whether the maximum number of retries has been exceeded (step 910). If the maximum number of retries has not been exceeded, a retransmission request is sent to the sender (step 912) and the process returns to step 904. In step 910, if the maximum number of retries has been exceeded,
Release all resources allocated to the operation (step 914) and the process ends. Referring back to step 908, if the last data packet has not been received, the process returns to step 904.

Turning now to FIG. 10, there is shown a flowchart of a process used to receive data for which a previous request has not been transmitted, in accordance with a preferred embodiment of the present invention.

The process is started by the sender transmitting the first subsequence (step 100).
0). This subsequence contains a default amount of data. Next, a timer is started to time the receipt of the acknowledgment (step 1002). If an acknowledgment is received, additional data is accessed (step 1).
004). The amount of data accessed is equal to the value specified in the window parameter field of the acknowledgment packet. If the data is accessed within the set time, the data is transmitted (step 100).
6) The process waits for an acknowledgment (step 1)
008). The data is transmitted in a sub-sequence and the amount of data in the sub-sequence is determined by the memory or buffer available at the receiver, as indicated by the window parameter field of the first acknowledgment packet received from the receiver. Less than space. If an acknowledgment is received for the transmitted data, a determination is made as to whether the last packet has been transmitted (step 101).
0). If the last packet has been sent, the process ends.

Referring again to step 1002,
If a timeout occurs, a determination is made as to whether the maximum number of retries has been exceeded (step 101).
2). If the maximum number of retries has not been exceeded, the operation is retried (step 1014) and the process returns to step 1000. If the maximum number of retries allowed has been exceeded, the process is terminated and all resources allocated for the operation are released.

Referring again to step 1004,
If more time is required to access the data, a keep-alive response is sent to the receiving side (step 101).
6) The process returns to step 1004.

Referring again to step 1008,
If a timeout occurs before receiving either an acknowledgment or a retransmission request, the operation ends and resources are released at the sender (step 1018). If the maximum number of retries is exceeded in step 1012, the process also proceeds to step 1018.

Referring again to step 1010,
If the last data packet has not been transmitted, the process returns to step 1004.

Accordingly, the present invention provides a method, apparatus, and computer-implemented instructions for transferring data. This mechanism provides a reliable transfer using MAD. The data is transmitted in an amount that does not exceed the memory space available for processing the data at the receiving end. Additional data is sent when an acknowledgment is received that the data has been processed. In this manner, the data responding to the request is transmitted in a sub-sequence of multiple MADs instead of a single MAD.

Although the invention has been described in terms of a fully functioning data processing system, it should be understood that the process of the invention can be distributed in the form of computer readable media of instructions and in various forms, and that the invention has been described in terms of performing the distribution. It is important to note that those skilled in the art will appreciate that the same applies irrespective of the particular type of signal-bearing medium actually used. Examples of computer readable media include a floppy disk, a hard disk,
Recordable media such as AM, CD-ROM, DVD-ROM, digital communication link, analog communication link,
Transmission media such as a wired or wireless communication link using transmission forms such as wireless and lightwave transmission are included. The computer readable medium may take the form of an encoded format that is decoded for actual use in a particular data processing system.

The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or to declare the invention in the form disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments are described in order to best explain the principles of the invention, practical applications, and for those skilled in the art to make various modifications with various modifications suitable for the particular use contemplated. Was selected and described in order to be able to understand.

In summary, the following matters are disclosed regarding the configuration of the present invention.

(1) A method in a data processing system for transferring data, the method comprising: sending a request, the request including an amount of processing space available in the data processing system. Receiving a sub-sequence of data packets from a set of sub-sequences of data packets in response to said request each time said amount of available processing space becomes available,
The data in each subsequence in said set of subsequences of data packets fit into said amount of available processing space. (2) The method according to (1), wherein the data packet is a management datagram. (3) The method of (1) above, wherein a particular sub-sequence of data packets in the set of sub-sequences of data packets has less data than the amount of available processing space. (4) The specific data packet in the subsequence is
The method of claim 1, further comprising a fragment flag indicating whether the particular data packet is the first data packet or the last data packet of a data transfer operation. (5) The method of (1) above, wherein the particular data packet is the last data packet in the set of data packet sub-sequences. (6) The method of (1) above, wherein each subsequence in the set of subsequences of data packets has a different amount of data. (7) The method according to (1), wherein the data packet includes a segment number. (8) The method according to (1), further comprising the step of reassembling the data in the data packet into a correct order. (9) each data packet in the set of sub-sequences of data packets includes a segment number, and the data is reassembled using the segment number;
The method according to the above (8). (10) The method of (1) above, wherein the amount of available space is a buffer in the data processing system. (11) The method of (1) above, wherein the amount of available processing space is a buffer allocated in memory of the data processing system. (12) A method in a data processing system for transferring data, comprising: receiving a request from a request issuer, wherein the request includes an amount of available space; and using the request. Identifying the data and placing the data in a plurality of sub-sequences of data packets, each sub-sequence in the set of sub-sequences having an amount less than or equal to the amount of available space. Retaining the data; and transmitting the plurality of sub-sequences of data packets to the requestor. (13) The method according to (12), wherein the first data packet and the last data packet in the plurality of sub-sequences of data packets include a payload length. (14) A fragment packet indicating whether the data packet in the plurality of sub-sequences of a data packet is the first data packet or the last data packet transmitted for a data transfer operation. The method according to (12), including a flag. (15) the transmitting comprises: transmitting an untransmitted sub-sequence of data packets in the plurality of sub-sequences of data packets to the requestor; Monitoring for a response at the issuer indicating that it is free, detecting that another untransmitted subsequence of the data packet is present in the plurality of subsequences of the data packet, and detecting the response Repeating the transmitting step and the monitoring step in response to the method. (16) A data processing system, comprising: a bus system; and a communication unit connected to the bus system, wherein data is transmitted and received using the communication unit. A memory connected to the bus system, wherein the set of instructions is located in the memory; and a processor unit connected to the bus system, wherein the processor unit comprises: Executing the set of instructions to send a request, wherein the request includes an amount of processing space available in the data processing system and the processor unit causes the amount of available processing space to become free. Receiving a sub-sequence of data packets from the set of data packets in response to said request Data of the inner is fit to the amount of available processing space, data processing system including a processor unit. (17) The data processing system according to (16), wherein the bus system includes a main bus and a sub bus. (18) The data processing system according to (16), wherein the processor unit includes a single processor. (19) The data processing system according to (16), wherein the processor unit includes a plurality of processors. (20) The data processing system according to (16), wherein the communication unit is an Ethernet® adapter. (21) A data processing system, comprising: a bus system; and a communication unit connected to the bus system, wherein data is transmitted and received using the communication unit. A memory connected to the bus system, wherein the set of instructions is located in the memory; and a processor unit connected to the bus system, wherein the processor unit comprises: Executing the set of instructions to receive a request from a request issuer, wherein the request includes an amount of available space, and wherein the processor unit identifies data using a response, and A plurality of sub-sequences of data packets, wherein each sub-sequence in said set of sub-sequences has less than said amount of available space Held, the processor unit, for transmitting the plurality of sub-sequence of the data packets to the requestor, the data processing system including a processor unit. (22) A data processing system for transferring data, the transmitting unit transmitting a request, wherein the request includes an amount of processing space available in the data processing system; Receiving means for receiving a sub-sequence of data packets from a set of sub-sequences of data packets in response to said request each time said amount of processing space becomes free, comprising: Receiving means wherein the data within the data fits into said amount of available processing space. (23) The data processing system according to (22), wherein the data packet is a management datagram. (24) The specific data packet in the set of sub-sequences of data packets has a smaller amount of data than the amount of available processing space.
A data processing system according to claim 1. (22) The specific data packet in the sub-sequence includes a fragment flag indicating whether the specific packet is the first data packet or the last data packet of a data transfer operation.
A data processing system according to claim 1. (26) The data processing system according to (22), wherein the specific data packet is the last data packet in the set of data packet sub-sequences. (27) The data processing system according to (22), wherein each subsequence in the set of subsequences of data packets has a different amount of data. (28) The data processing system according to (22), wherein the data packet includes a segment number. (29) The data processing system according to (22), further including reassembly means for reassembling data in the data packet in a correct order. (30) The data processing of (29), wherein each data packet in the set of sub-sequences of data packets includes a segment number, and wherein the data is reassembled using the segment number. system. (31) The data processing system according to (22), wherein the amount of available space is a buffer in the data processing system. (32) The data processing system according to (22), wherein the amount of available processing space is a buffer allocated in a memory of the data processing system. (33) A data processing system for transferring data, a receiving means for receiving a request from a request issuer,
Receiving means including the amount of available space, identifying means for identifying data using a response, and arranging means for arranging the data in a plurality of subsequences of data packets; Arranging means, wherein each subsequence in the plurality of subsequences of data packets holds an amount of data less than or equal to the amount of available space; and the plurality of subsequences of data packets to the requestor. And a transmission means for transmitting the data. (34) The data processing system according to (33), wherein the first data packet and the last data packet in the plurality of sub-sequences of a data packet include a payload length. (35) A fragment indicating whether the data packet in the plurality of sub-sequences of a data packet is the first data packet or the last data packet transmitted for a data transfer operation. The data processing system according to (33), including a flag. (36) transmitting means for transmitting an untransmitted data packet in the plurality of sub-sequences of data packets to the request issuer, and wherein the amount of available space is on the request issuer side. Monitoring means for monitoring a response indicating that the data is free, and another untransmitted data
In response to the packet being present in the plurality of sub-sequences of a data packet and detecting the response,
The data processing system according to (33), further including: a repeating unit that repeats the start of the transmitting unit and the monitoring unit. (37) A computer in a computer-readable medium used to transfer data in the data processing system.
A program product, wherein the first instruction to send a request, wherein the request comprises an amount of processing space available in the data processing system; A second instruction for receiving a sub-sequence of data packets from the set of sub-sequences of data packets in response to the request each time the data is available, wherein the data in each sub-sequence of data packets is available for processing A second instruction that fits in said amount of space. (38) A computer program product in a computer readable medium for transferring data in a data processing system, the first instruction receiving a request from a request issuer, the request comprising: A first instruction that includes an amount, a second instruction that identifies data using the response, and a third instruction that places the data in a plurality of subsequences of the data packet. A computer including a third instruction wherein each subsequence holds less than or equal to the amount of available space, and a fourth instruction for transmitting the plurality of subsequences of data packets to the requestor.・ Program products.

[Brief description of the drawings]

FIG. 1 illustrates a network global change computing system according to a preferred embodiment of the present invention.

FIG. 2 is a functional block diagram of a host processor node according to a preferred embodiment of the present invention.

FIG. 3 illustrates a host channel adapter according to a preferred embodiment of the present invention.

FIG. 4 illustrates the processing of a work request according to a preferred embodiment of the present invention.

FIG. 5 illustrates a data packet according to a preferred embodiment of the present invention.

FIG. 6 illustrates a management datagram data packet according to a preferred embodiment of the present invention.

FIG. 7 is a flowchart of a process used for requesting and receiving data, according to a preferred embodiment of the present invention.

FIG. 8 is a flowchart of a process used to process a request for data, according to a preferred embodiment of the present invention.

FIG. 9 is a flowchart of a process used to receive data from a sender without sending a previous request for data, according to a preferred embodiment of the present invention.

FIG. 10 is a flowchart of a process used to send data without receiving a previous request for data, according to a preferred embodiment of the present invention.

[Explanation of symbols]

 500 message data 502 data segment 1 504 data segment 2 506 data segment 3 508 packet 510 packet payload 512 data packet 514 CRC 516 routing header 518 transport header 600 MAD 602 segment number field 604 payload length field 606 Fragment flag field 608 Window parameter field

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Gilles Roger Freitzer United States 78703 Austin, Texas Northumberland Road 1604 (72) Inventor Gregory Francis Pfister United States 78746 Austin, TX Sir Iver Cove 5905 (72) Inventor Renato John Ratio United States 78759 Austin, Texas Wimpeg Cove 6707 F-term (reference) 5B089 GA02 GB01 JA11 JB14 KD09 5K030 HA08 HB28 JA05 KA03 LA01 LC01 MB15 5K033 BA04 CB01 CB06 CC01 DB12 5K03 A

Claims (38)

[Claims] 1. A method in a data processing system for transferring data, the method comprising: sending a request, the request including an amount of processing space available in the data processing system; Receiving a sub-sequence of data packets from a set of sub-sequences of data packets in response to the request each time the amount of available processing space becomes available, wherein the set of sub-sequences of data packets is The data in each subsequence within the said amount of available processing space. 2. The method of claim 1, wherein said data packet is a management datagram. 3. The method of claim 1, wherein a particular subsequence of data packets in the set of data packet subsequences has less data than the amount of available processing space. 4. A particular data packet in the subsequence includes a fragment flag indicating whether the particular data packet is the first data packet or the last data packet of a data transfer operation. The method of claim 1. 5. The method of claim 1, wherein the particular data packet is the last data packet in the set of data packet subsequences. 6. The method of claim 1, wherein each subsequence in the set of subsequences of data packets has a different amount of data. 7. The method of claim 1, wherein said data packet includes a segment number. 8. The method of claim 1, further comprising the step of reassembling the data in said data packets into the correct order. 9. Each data packet in said set of sub-sequences of data packets includes a segment number;
The method of claim 8, wherein the data is reassembled using the segment number. 10. The method of claim 1, wherein said amount of available space is a buffer in said data processing system. 11. The method of claim 1, wherein said amount of available processing space is a buffer allocated in memory of said data processing system. 12. A method in a data processing system for transferring data, the method comprising: receiving a request from a request issuer, wherein the request comprises an amount of available space; Identifying the data using sub-sequences, and placing the data in a plurality of sub-sequences of data packets, each sub-sequence in the set of sub-sequences being less than or equal to the amount of available space. Maintaining a quantity of data; and transmitting the plurality of sub-sequences of data packets to the requestor. 13. The method of claim 12, wherein a first data packet and a last data packet in said plurality of sub-sequences of a data packet include a payload length. 14. The data packet in the plurality of sub-sequences of data packets indicates whether the data packet is the first data packet or the last data packet transmitted for a data transfer operation. 13. The method of claim 12, comprising a fragment flag. 15. The method according to claim 15, wherein said transmitting comprises: transmitting an untransmitted sub-sequence of data packets in said plurality of sub-sequences of data packets to said requester; Monitoring the requester for a response indicating that it is empty; another unsent subsequence of data packets being present in the plurality of subsequences of data packets; Repeating the transmitting and monitoring steps in response to the detection of 16. A data processing system, comprising: a bus system; and a communication unit connected to the bus system, wherein data is transmitted using the communication unit.
A communication unit received, a memory connected to the bus system, wherein the set of instructions is located in the memory; and a processor unit connected to the bus system. The processor unit executing the set of instructions to send a request, the request including an amount of processing space available in the data processing system;
The processor unit receives sub-sequences of data packets from the set of data packets in response to the request each time the amount of available processing space becomes free, and within each sub-sequence of data packets. Data fit into said amount of available processing space,
A data processing system including a processor unit. 17. The data processing system according to claim 16, wherein said bus system includes a main bus and a sub bus. 18. The data processing system according to claim 16, wherein said processor unit comprises a single processor. 19. The data processing system according to claim 16, wherein said processor unit includes a plurality of processors. 20. The data processing system according to claim 16, wherein said communication unit is an Ethernet® adapter. 21. A data processing system, comprising: a bus system; and a communication unit connected to the bus system, wherein data is transmitted using the communication unit.
A communication unit received, a memory connected to the bus system, wherein the set of instructions is located in the memory; and a processor unit connected to the bus system. Wherein the processor unit executes the set of instructions to receive a request from a request issuer, wherein the request includes an amount of available space, and wherein the processor unit uses Identifying data and placing said data in a plurality of sub-sequences of data packets, wherein each sub-sequence in said set of sub-sequences comprises:
A data processing system, the processor unit holding an amount of data less than or equal to the amount of available space, wherein the processor unit sends the plurality of sub-sequences of data packets to the requestor. 22. A data processing system for transferring data, the transmitting means for transmitting a request, wherein the request includes an amount of processing space available in the data processing system. Each time said amount of available processing space becomes available,
Receiving means for receiving a sub-sequence of data packets from a set of sub-sequences of data packets in response to the request, wherein data in each sub-sequence of data packets comprises a sub-sequence of available processing space. A data processing system that includes receiving means that fits in volume. 23. The data processing system according to claim 22, wherein said data packet is a management datagram. 24. The data processing system of claim 22, wherein a particular data packet in the set of data packet subsequences has less data than the amount of available processing space. 25. A particular data packet in a subsequence includes a fragment flag indicating whether the particular packet is the first data packet or the last data packet of a data transfer operation. 2
3. The data processing system according to 2. 26. The data processing system of claim 22, wherein said particular data packet is the last data packet in said set of data packet subsequences. 27. The data processing system of claim 22, wherein each subsequence in the set of subsequences of data packets has a different amount of data. 28. The data processing system according to claim 22, wherein said data packet includes a segment number. 29. The apparatus of claim 2, further comprising reassembly means for reassembling the data in said data packets in the correct order.
3. The data processing system according to 2. 30. The data of claim 29, wherein each data packet in the set of data packet subsequences includes a segment number, and wherein the data is reassembled using the segment number. Processing system. 31. The data processing system of claim 22, wherein said amount of available space is a buffer in said data processing system. 32. The data processing system of claim 22, wherein said amount of available processing space is a buffer allocated in memory of said data processing system. 33. A data processing system for transferring data, said receiving means receiving a request from a request issuer, said request including an amount of available space, and using a response. Identification means for identifying the data in a plurality of sub-sequences of a data packet, wherein each of the sub-sequences in the plurality of sub-sequences of a data packet has an available space. A data processing system, comprising: an arrangement unit that holds an amount of data equal to or less than the amount of data packets; 34. The data processing system of claim 33, wherein a first data packet and a last data packet in the plurality of sub-sequences of a data packet include a payload length. 35. The data packet in the plurality of sub-sequences of a data packet indicates whether the data packet is a first data packet or a last data packet transmitted for a data transfer operation. The data processing system of claim 33, comprising a fragment flag. 36. A transmitting means for transmitting unsent data packets in said plurality of sub-sequences of data packets to said request issuer; and wherein said amount of available space is determined by said request issuer. Monitoring means for monitoring a response indicating that the data packet is free on the side of the user; and responding to the presence of another untransmitted data packet in the plurality of subsequences of the data packet and detecting the response. 34. The data processing system according to claim 33, further comprising: repetition means for repeating the start of the transmission means and the monitoring means. 37. A computer program product in a computer readable medium used to transfer data in a data processing system, the first instructions for sending a request, the request comprising the data A first instruction including an amount of processing space available in the processing system; and a sub-sequence of the data packet from the set of data packet sub-sequences in response to the request each time the amount of available processing space becomes available. A second instruction for receiving the sequence, wherein the data in each subsequence of the data packet fits in said amount of available processing space;
Computer program product comprising the second instruction. 38. A computer program product in a computer readable medium for transferring data within a data processing system, wherein the first instructions receive a request from a request issuer, the request comprising: A first instruction that includes an amount of space; a second instruction that identifies data using a response; and a third instruction that places the data in a plurality of subsequences of a data packet, the set of subsequences. A third instruction in which each subsequence holds less than or equal to the amount of available space; and a fourth instruction that sends the plurality of subsequences of data packets to the requestor. Including computer program products.