EP1684189B1 - Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field - Google Patents

Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field Download PDF

Info

Publication number
EP1684189B1
EP1684189B1 EP06009235A EP06009235A EP1684189B1 EP 1684189 B1 EP1684189 B1 EP 1684189B1 EP 06009235 A EP06009235 A EP 06009235A EP 06009235 A EP06009235 A EP 06009235A EP 1684189 B1 EP1684189 B1 EP 1684189B1
Authority
EP
European Patent Office
Prior art keywords
field
packet
completion
type
packet header
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP06009235A
Other languages
German (de)
French (fr)
Other versions
EP1684189A2 (en
EP1684189A3 (en
Inventor
David Harriman
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Intel Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corp filed Critical Intel Corp
Publication of EP1684189A2 publication Critical patent/EP1684189A2/en
Publication of EP1684189A3 publication Critical patent/EP1684189A3/en
Application granted granted Critical
Publication of EP1684189B1 publication Critical patent/EP1684189B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/42Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/40Bus structure
    • G06F13/4004Coupling between buses
    • G06F13/4022Coupling between buses using switching circuits, e.g. switching matrix, connection or expansion network
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/912Applications of a database
    • Y10S707/922Communications

Definitions

  • the present invention pertains to the field of computer systems. More particularly, this invention pertains to the field of high speed point-to-point interconnections and communications architectures.
  • Computing appliances e.g., computer systems, servers, networking switches and routers, wireless communication devices, and the like are typically comprised of a number of disparate elements. Such elements often include a processor, system control logic, a memory system, input and output interfaces, and the like. To facilitate communication between such elements, computing appliances have long relied on general purpose input/output busses to enable these disparate elements of the computing system to communicate with one another in support of the myriad of applications offered by such appliances.
  • PCI Peripheral Component Interconnect
  • PCI Peripheral Component Interconnect
  • the PCI bus standard Peripheral Component Interconnect (PCI) Local Bus Specification, Rev. 2.2, released December 18, 1998 ) defines a multi-drop, parallel bus architecture for interconnecting chips, expansion boards, and processor/memory subsystems in an arbitrated fashion within a computing appliance. While typical PCI bus implementations have a 133Mbps throughput (i.e., 32bits at 33MHz), the PCI 2.2 standard allows for 64bits per pin of the parallel connection clocked at up to 133MHz resulting in a theoretical throughput of just over 1 Gbps.
  • the throughput provided by the PCI bus architectures has, until recently, provided adequate bandwidth to accommodate the internal communication needs of even the most advanced of computing appliances (e.g., multiprocessor server applications, network appliances, etc.).
  • computing appliances e.g., multiprocessor server applications, network appliances, etc.
  • recent advances in processing power and increasing input/output bandwidth demands create a situation where prior general purpose architectures such as the PCI bus architecture have become processing bottlenecks within such computing appliances.
  • An example of an isochronous data stream is a multimedia data stream which requires a transport mechanism to ensure that the data is consumed as fast as it is received and to ensure that the audio portion is synchronized with the video portion.
  • Conventional general purpose input/output architectures process data asynchronously, or in random intervals as bandwidth permits. Such asynchronous processing of multimedia streams data can result in lost data and/or misaligned audio and video.
  • Figure 1 is a block diagram of one embodiment of a computer system.
  • Figure 2 is a graphical illustration of an example enhanced general input/output port.
  • Figure 3 is a diagram showing the format of one embodiment of the start of a transaction layer packet header.
  • Figure 4 is a diagram of a request packet header supporting a 32-bit address format.
  • Figure 5 is a diagram of a request packet header supporting a 64-bit address format.
  • Figure 6 is a diagram of a packet header for a Message.
  • Figure 7 is a diagram showing a request header format for a configuration transaction.
  • Figure 8 is a diagram showing one embodiment of a format for a completion header.
  • Figures 9a and 9b combined form a flow diagram of an example embodiment of a method for handling received transaction layer packets.
  • Figure 10 is a flow diagram of one embodiment of a method for handling error conditions associated with received request packets.
  • Figure 11 is a flow diagram of one embodiment of a method for handling a completion packet that is not expected by a system agent.
  • Figure 12 is a flow diagram of one embodiment of a method for a requesting device handling a completion packet with a completion status other than "Successful Completion.”
  • Figure 13 is a flow diagram of one embodiment of a method for a completing device handling a completion packet with a completion status other than "Successful Completion.”
  • the disclosed embodiments involve an enhanced general input/output interconnection architecture and associated communications protocol.
  • One example embodiment includes one or more of a root complex including a host bridge, a switch, or endpoints, each incorporating at least a subset of enhanced general input/output features to support enhanced general input/output communication between such elements.
  • Communication between the enhanced general input/output facilities of such elements is performed in one embodiment using serial communication channels employing a communication protocol which supports one or more innovative features including, but not limited to, virtual communication channels, tailer-based error forwarding ("tailers" are appended to transaction layer packets to indicate an error condition), support for legacy PCI-based devices, multiple request response types, flow control and/or data integrity management facilities.
  • the communication protocol supported in this embodiment includes a communication protocol stack including a physical layer, a data link layer and a transaction layer.
  • a communications agent incorporates an enhanced general input/output engine comprising a subset of the foregoing features.
  • one or more elements of the various embodiments may be implemented in hardware, software, a propagated signal, or a combination thereof.
  • FIG. 1 is a block diagram of an electronic appliance 100, which, for this embodiment, is a computer system.
  • the system 100 includes a processor 102, a host bridge 103 included as part of a root complex 104, switch 108 and endpoint 110, each coupled as shown.
  • the root complex 104, switch 108, and endpoint 110 include one or more instances of an enhanced general input/output communication port 106.
  • each of the elements 102, 104, 108 and 110 are coupled to at least one other element through a communication link 112 supporting one or more enhanced general input/output communication channels via the enhanced general input/output communication port.
  • the system 100 is intended to represent any of a wide variety of traditional and non-traditional computing systems, servers, network switches, network routers, wireless communication subscriber units, wireless communication telephony infrastructure elements, personal digital assistants, set-top boxes, or any electric appliance that would benefit from the communication resources introduced through integration of at least a subset of the enhanced general input/output interconnection architecture and/or communications protocol described herein.
  • processor 102 controls one or more aspects of the functional capability of the electronic appliance 100.
  • the processor 102 is representative of any of a wide variety of control logic devices including, but not limited to, one or more of a microprocessor, a programmable logic device (PLD), programmable logic array (PLA), application specific integrated circuit (ASIC), a microcontroller, and the like.
  • the root complex 104 provides a communications interface between the processor 102 and the switch 108 and endpoint 110.
  • the term "root complex” refers to a logical entity of an enhanced general input/output hierarchy that is closest to a host controller, a memory controller hub, an IO controller hub, or any combination of the above, or some combination of chipset/CPU elements (i.e., in a computing system environment). Although depicted in Figure 1 as a single unit, the root complex 104 may be implemented with multiple physical components.
  • the root complex 104 is populated with one or more enhanced general input/output ports 106 to facilitate communication with other peripheral devices, e.g., the switch 108, endpoint 110 and, although not particularly depicted, the legacy bridges 114, or 116.
  • each enhanced general input/output interface port represents a different hierarchy domain.
  • the embodiment of Figure 1 denotes a root complex 104 with three hierarchy domains.
  • FIG. 2 is a graphical illustration of an example enhanced general input/output port 106.
  • the enhanced general input/output port 106 implements a communication stack comprising a transaction layer 202, a data link layer 204 and a physical layer 206 including a logical sub-block 208, and a physical sub-block 210, as shown. Elements of the transaction layer will be discussed below in more detail.
  • the transaction layer 202 provides an interface between the enhanced general input/output architecture and a device core.
  • a primary responsibility of the transaction layer 202 is the assembly and disassembly of packets for one or more logical devices within an agent.
  • the transaction layer implements a pipelined full split-transaction protocol as well as mechanisms for differentiating the ordering and processing requirements of transaction layer packets.
  • the transaction layer further comprehends transaction layer packet construction and processing.
  • One embodiment of the enhanced general input/output architecture supports the following basic transaction types and address spaces: Memory, I/O, Configuration, and Message. Two addressing types are supported: 32bit and 64bit.
  • Request and Completion packets which may be referred to simply as Requests and Completions.
  • Completions are used only where required, e.g.: to return read data, or to acknowledge completion of I/O and configuration write transactions.
  • Completions are associated with their corresponding Requests by the value in the Requester ID field of the packet header (discussed below).
  • All transaction layer packets in this embodiment start with a defined header.
  • Some transaction layer packets include data following the header as determined by the format field specified in the transaction layer packet header.
  • the transaction layer packet is limited in size by a predetermined maximum payload size value.
  • the transaction layer packet data in this embodiment is four-byte naturally aligned and in increments of four-byte double-words.
  • FIG 3 is a diagram showing the format of one embodiment of the start of a transaction layer packet header.
  • Each transaction layer packet header includes a three-bit format field (Fmt[2:0]).
  • the transaction layer packet header also includes a four-bit type field (Type[3:0]). Both the Fmt and Type fields need to be decoded in order to determine the transaction layer packet format.
  • Table 1 below shows example encodings for the Fmt field.
  • the transaction layer header for this embodiment also includes a two-bit Extended Type/Extended Length field (Et/E1). This field is used to extend either the Type field or the Length field, depending on the value in the Type field.
  • the Length field for this embodiment is ordinarily an eight-bit field, but may be extended to become a ten-bit field if the value in the Type field indicates that the Et/El field is to be used to extend the Length field.
  • the Type field can be extended to become a six-bit field by appending the Et/El field, depending on the value in the Type[3:0] field. See Table 2 below for example Fmt, Type, and Et/El field encodings (alternative embodiments may use other encoding schemes).
  • the Et/El field is used as an extension of the Type field except where noted.
  • Table 2 - Fmt, Type, and Et/El Encodings Packet Type Fmt[2:0] Type[3:0] Et/E1[1:0] Description MRd 001 010 1001 E19 E18 Memory Read Request Et/E1 field used for Length[9:8] MRdLk 001 010 1011 00 Memory Read Request - Locked MWr 101 110 0001 E19 E18 Memory Write Request - Posted Et/E1 field used for Length[9:8] IORd 001 1010 00 IO Read Request IOWr 101 1010 00 IO Write Request CfgRd0 001 1010 01 Configuration Read Type 0 CfgWr0 101 1010 01 Configuration Write Type 0 CfgRd1 001 1010 11 Configuration Read Type 1 CfgWr1 101 1010 11 Configuration Write Type 1 Msg 010 011s 2 s 1 s
  • the Message field must be decoded to determine specific cycle including if a Completion is required or not.
  • the sub-field s[2:0] specifies a group of messages.
  • the Message field must be decoded to determine specific cycle including if a Completion is required or not.
  • the sub-field c[2:0] specifies the message type: 000 - Unicast, Data Packet 001 - Multicast, Data Packet 010 - Signaling Packet, without interrupt 011 - Reserved 100 - Null signaling Packet, interrupt to Host in the destination Hierarchy 101 - Null signaling Packet, interrupt to destination device 110 - Signaling Packet, with interrupt to Host in the destination Hierarchy 111 - Signaling Packet, with interrupt to destination device Cpl 001 0100 00 Completion without Data - used for IO and Configuration Write Completions, and Memory Read Completions with Completion Status other than "Successful Completion.”
  • Request packets include a Request Header which for some types of Request packets will be followed by some number of double-words of data.
  • double-word indicates a 32-bit length of data.
  • the Length field for Message Request Headers is not used except for Messages that explicitly refer to a data length.
  • the El/Et field is concatenated with the Length field to form a ten-bit length field.
  • the ten-bit length field allows read and write requests indicating up to 4kB of data.
  • Other types of transaction layer packets are limited by the size of the Length[7:0] field to indicating up to 1kB of data.
  • the amount of data included in any transaction layer packet is limited in one embodiment to a predetermined maximum payload size.
  • the value in the Length field and the actual amount of data should be equal. If the receiver determines that the Length field value and the actual amount of data do not match, then the packet is treated as a Malformed Transaction Layer Packet. Malformed Transaction Layer Packets are described below.
  • Figure 4 is a diagram of a request packet header supporting a 32-bit address format
  • Figure 5 is a diagram of a request packet header supporting a 64-bit address format.
  • memory read requests and memory write requests case use either the 32-bit address format or the 64-bit address format.
  • the 32-bit format is used.
  • the request packet headers of Figures 4 and 5 also include a First Double Word Byte Enable field (1 st DW BE) and a Last Double Word Byte Enable field (Last DW BE).
  • the First Double Word Byte Enable field contains byte enables for the first double word of any memory read or write request. This field also contains byte enables for the only double word of an input/output or configuration request.
  • the Last Double Word Byte Enable field contains byte enables for the last double word of any memory read or write request.
  • the byte enable fields are not used with Messages because these fields overlap the message code field for a message request header (See Figure 7, discussed below).
  • bit 0 corresponds to byte 0 of the first double word of data.
  • Bit 1 corresponds to byte 1 of the first double word of data.
  • Bit 2 corresponds to byte 2 of the first double word of data.
  • Bit 3 corresponds to byte 3 of the first double word of data.
  • bit 0 corresponds to byte 0 of the last double word of data.
  • Bit 1 corresponds to byte 1 of the last double word of data.
  • Bit 2 corresponds to byte 2 of the last double word of data.
  • Bit 3 corresponds to byte 3 of the last double word of data.
  • the example packet headers of Figures 4, 5, 6, and 8 include a Requestor ID field, a Tag field, an Attribute field, and a Virtual Channel ID field.
  • the Requestor ID field and the Tag field together form a Transaction ID field.
  • the Requestor ID field is divided into a Bus Number field, a Device Number field, and a Function Number field.
  • the Tag field is a 5-bit field generated by each requesting device.
  • the tag value is unique for all outstanding requests that require a completion for that requesting device.
  • the Transaction ID field is included with all Requests and Completions.
  • the Requestor ID field for these example embodiments is a 16-bit value that is unique for every function (a function is one independent section of a multi-function device identified in configuration space by a unique function number). Functions capture the Bus Number supplied with all configuration writes completed by the function, and supply this number in the Bus Number section of the Requestor ID field.
  • Each logical device in a component is designed to respond to a unique Device Number for configuration requests addressing that component.
  • a component may contain many (perhaps up to several dozen) logical devices.
  • Each function associated with a logical device in a component is designed to respond to a unique function number for configuration requests addressing that component and logical device.
  • Each logical device may contain up to eight logical functions.
  • the Attributes field specifies characteristics of the transaction. Attributes that may be specified in the attributes field include a priority attribute, transaction ordering attributes, and cache coherency management attributes.
  • the Virtual Channel ID field identifies the virtual channel.
  • the Virtual Channel ID field is a 4-bit field that allows identification of up to 16 virtual channels on a per transaction basis.
  • virtual channel 0 is used for general purpose traffic and a virtual channel other than 0 is used for isochronous traffic.
  • Figure 6 is a diagram of a packet header for a Message. As seen in Table 2, messages may or may not include data and may or may not require completion. Messages are decoded by all devices in a system that support the enhanced general input/output interconnection architecture.
  • the Message field is decoded in order to determine the specific cycle and to determine whether the message includes data and whether the message requires completion.
  • the Message field for this embodiment is an 8-bit field located where the byte enable fields normally reside for other transaction types. Unsupported messages are treated by the receiving device as no-completion-required (no-completion-required transactions discussed below).
  • Messages for this example embodiment are divided into groups. There are eight groups that include data with the request and eight groups that do not. Other embodiments are possible using different numbers of groups. For this embodiment, as shown in Table 2, the eight groups that include data with the requests have a value of b110 in the Fmt field. The eight groups that do not include data have a value of b010 in the Fmt field.
  • the sub-field s[2:0] incorporates one bit from the Type field and the two bits from the Et/El field. The sub-field s[2:0] indicates one of eight groups.
  • Examples of various messages include, but are not limited to, the following: messages for unlocking devices; messages for resetting devices; messages indicating a correctable error condition; messages indicating an uncorrectable error condition; messages indicating a fatal error condition; messages used to report bad request packets; messages relating to power management; messages related to ordering control/management; and messages for emulating legacy (e.g., PCI) interrupt signals (or other legacy signals).
  • legacy e.g., PCI
  • Figure 7 is a diagram showing a request header format for a configuration transaction.
  • the configuration space is one of the four supported address spaces for these example embodiments.
  • Figure 8 is a diagram showing one embodiment of a format for a completion header. All read requests and some write requests require completion. Completions include a completion header that, for some types of completions, are followed by some number of double-words of data.
  • the Completion Status[2:0] field shown in Figure 8 indicates the status for a completion.
  • Table 3 shows one example encoding scheme. Table 3 - Completion Status Field Encoding Scheme Completion Status[2:0] Status 000 Successful Completion 001 Unsupported Request - Expecting Completion 011 Reserved 100 Completer Abort
  • the Completer ID[15:0] field contains the same type of information as the Requestor ID field described above.
  • the value provided in the Completer ID field corresponds to the bus/device/function of the agent that completes the request.
  • Completion headers contain the same values for Requester ID, Tag, and Channel ID as were supplied in the header of the request packet.
  • Completion headers also contain the same value in the Attribute field as was initially supplied with the header of the request.
  • Completion packets are routed by switches and root complexes to the port that initiated the corresponding request transaction.
  • individual completion packets may provide less than the full amount of data requested by the corresponding read request so long as all of the completion packets associated with the corresponding read request, when combined, return the amount of data specified.
  • I/O and Configuration read requests are completed with exactly one completion packet.
  • a completion that includes data specifies the amount of data in the packet header. If the completion packet actually contains an amount of data that differs from the specified amount, a malformed transaction layer packet results.
  • Figures 9a and 9b combined form a flow diagram of an example embodiment of a method for handling received transaction layer packets.
  • the operations described below do not necessarily need to occur in a serial fashion. Some embodiments may perform some operations simultaneously.
  • a check is made to determine whether the values contained in the Fmt and Length fields of the received packet match the actual size of the packet. A mismatch indicates a malformed packet and an error case results as shown at block 925. Error case handling will be discussed below. If the actual size of the received packet does not indicate a mismatch with the Fmt and Length fields, then processing continues at block 910.
  • the address bits [63:32] are checked to see whether any of the address bits [63:32] are non-zero. If none of the address bits [63:32] are non-zero, then the result is a malformed packet and processing proceeds to the error case block 925. If at least one of the address bits [63:32] are non-zero, then processing continues at block 915.
  • the completing device is unable to respond to the request packet due to an internal error, then the result is a "completer abort" and processing proceeds to error case block 925. Otherwise, the request is serviced at block 950. In servicing the request, it may be necessary to repeat the processing indicated by blocks 940 and 945.
  • Figure 10 is a flow diagram of one embodiment of a method for handling error conditions associated with received request packets.
  • a completion with the appropriate completion status is transmitted at block 1020.
  • the completion is routed back to the requesting device.
  • an error message is transmitted to the requesting device at block 1030.
  • the error is reported to the system at block 1040.
  • the error message transmitting operating denoted at block 1030 may be implemented as a programmable option.
  • Some systems may include one or more PCI buses in addition to the previously discussed enhanced general input/output interconnection architecture.
  • the completion status for these embodiments represents the actual PCI termination for the cycle.
  • a non-posted PCI cycle must actually be serviced on the PCI bus before a completion status can be determined.
  • the completion status values are defined as discussed below.
  • the resulting completion status value is "Successful Completion" (encoded in the completion status field as "000" for this embodiment as indicated in Table 3).
  • a read request from a host bridge is routed through a switch to a completer endpoint.
  • the Completer responds with a completion packet indicating a successful completion status and also responds with the data for the read request.
  • the switch routes this completion packet back to the host bridge.
  • the resulting completion status is "Unsupported Request” (encoded in the completion status field as "001" for this embodiment as indicated in Table 3).
  • Unsupported Request would be a memory read request to an out-of-range address. In this case, the completer is not able to support the request and the requester is expecting a completion.
  • the resulting completion status is an unsupported request. Since the requesting device is not expecting a completion, the completion status is communicated to the requesting device via a message as described above in connection with Figure 10.
  • An example of an unsupported request where the requesting device is not expecting a completion is a memory write transaction to an out-of-range address.
  • the communication of the completion status via a message may be implemented as an optional feature.
  • the resulting completion status is a "Completer Abort" (encoded in the completion status field as "100" for this embodiment).
  • a completing device When a completing device receives a packet that violates packet formation rules, the result is a "Malformed Packet.” The completing device responds to this situation by transmitting a "Malformed Packet" error message that is routed to the requesting device.
  • a switch that receives a malformed packet must, for this embodiment, route the packet to the upstream port if no other port can be positively identified as the intended destination port.
  • a read completion When a read completion has a completion status other than "Successful Completion," no data is returned with the completion packet.
  • the read completion with the non-successful completion status is the last completion transmitted for the request. For example, a completer may split a read request into four parts for servicing and the second completion packet results in a completer abort completion status. The final two completion packets are not transmitted.
  • the requesting device once it receives the completion packet with the non-successful completion status, considers the request terminated and should not expect additional completion packets corresponding to that read request.
  • Figure 11 is a flow diagram of one embodiment of a method for handling a completion packet that is not expected by a system agent.
  • An "unexpected completion" occurs when an agent receives a completion that does not correspond to any outstanding requests issued by that same agent.
  • block 1110 indicates that if there is no unexpected completion then normal operation continues at block 1120. If, however, an unexpected completion is received, the unexpected completion packet is discarded at block 1130. After the packet is discarded, the above error condition may be reported to the system at block 1140. For this example embodiment, the reporting of the error may be an option that is programmable by software.
  • Figure 12 is a flow diagram of one embodiment of a method for a requesting device handling a completion packet with a completion status other than "Successful Completion.”
  • Block 1210 indicates that if the completion status is "Successful Completion", then normal operation continues at block 1220. If the completion status is other than "Successful Completion", then at block 1230 the value of the Completer ID field is recorded. For this embodiment the Completer ID value is stored in a register. Then, at block 1240, a "Received Unsuccessful Completion" bit is set in a register in the requesting device for this embodiment. The above error condition may be reported at block 1250. The reporting of the error condition may be implemented as a programmable option. A software agent may use the Completer ID value and the "Received Unsuccessful Completion" bit to track down the source of the error condition.
  • Figure 13 is a flow diagram of one embodiment of a method for a completing device handling a completion packet with a completion status other than "Successful Completion.”
  • Block 1310 indicates that if the completion status of a transmitted completion packet is "Successful Completion", then normal operation continues at block 1320. If the completion status is other than "Successful Completion", then at block 1330 the value of the Requester ID and Tag fields are recorded. For this embodiment, the Requester ID and Tag values are stored in one or more registers. Then, at block 1340, a "Transmitted Unsuccessful Completion" bit is set in a register in the completing device for this embodiment. The above error condition may be reported at block 1350. The reporting of the error condition may be implemented as a programmable option. A software agent may use the Requester ID and Tag values and the "Transmitted Unsuccessful Completion" bit to track down the source of the error condition.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computer Hardware Design (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Communication Control (AREA)
  • Computer And Data Communications (AREA)
  • Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A transmitting device and a receiving device are coupled via a high-speed serial interface within a computer system. The transmitting device transmits a packet that includes an extended type/extended length field that extends either a type field or a length field depending on the value of the type field. The extended type/extended length field extends the length field when the type field indicates a memory read request transaction.

Description

    Field Of The Invention
  • The present invention pertains to the field of computer systems. More particularly, this invention pertains to the field of high speed point-to-point interconnections and communications architectures.
  • Background of the Invention
  • Computing appliances, e.g., computer systems, servers, networking switches and routers, wireless communication devices, and the like are typically comprised of a number of disparate elements. Such elements often include a processor, system control logic, a memory system, input and output interfaces, and the like. To facilitate communication between such elements, computing appliances have long relied on general purpose input/output busses to enable these disparate elements of the computing system to communicate with one another in support of the myriad of applications offered by such appliances.
  • Perhaps one of the most pervasive of such general purpose bus architectures is the Peripheral Component Interconnect (PCI) bus. The PCI bus standard (Peripheral Component Interconnect (PCI) Local Bus Specification, Rev. 2.2, released December 18, 1998) defines a multi-drop, parallel bus architecture for interconnecting chips, expansion boards, and processor/memory subsystems in an arbitrated fashion within a computing appliance. While typical PCI bus implementations have a 133Mbps throughput (i.e., 32bits at 33MHz), the PCI 2.2 standard allows for 64bits per pin of the parallel connection clocked at up to 133MHz resulting in a theoretical throughput of just over 1 Gbps.
  • The throughput provided by the PCI bus architectures has, until recently, provided adequate bandwidth to accommodate the internal communication needs of even the most advanced of computing appliances (e.g., multiprocessor server applications, network appliances, etc.). However, recent advances in processing power and increasing input/output bandwidth demands create a situation where prior general purpose architectures such as the PCI bus architecture have become processing bottlenecks within such computing appliances.
  • Another limitation associated with prior architectures is that they are typically not well-suited to process isochronous (time dependent) data streams. An example of an isochronous data stream is a multimedia data stream which requires a transport mechanism to ensure that the data is consumed as fast as it is received and to ensure that the audio portion is synchronized with the video portion. Conventional general purpose input/output architectures process data asynchronously, or in random intervals as bandwidth permits. Such asynchronous processing of multimedia streams data can result in lost data and/or misaligned audio and video.
    The invention is as set out in the appended claims 1,7 and 13.
  • Brief Description of the Drawings
  • The invention will be understood more fully from the detailed description given below and from the accompanying drawings of embodiments of the invention which, however, should not be taken to limit the invention to the specific embodiments described, but are for explanation and understanding only.
  • Figure 1 is a block diagram of one embodiment of a computer system.
  • Figure 2 is a graphical illustration of an example enhanced general input/output port.
  • Figure 3 is a diagram showing the format of one embodiment of the start of a transaction layer packet header.
  • Figure 4 is a diagram of a request packet header supporting a 32-bit address format.
  • Figure 5 is a diagram of a request packet header supporting a 64-bit address format.
  • Figure 6 is a diagram of a packet header for a Message.
  • Figure 7 is a diagram showing a request header format for a configuration transaction.
  • Figure 8 is a diagram showing one embodiment of a format for a completion header.
  • Figures 9a and 9b combined form a flow diagram of an example embodiment of a method for handling received transaction layer packets.
  • Figure 10 is a flow diagram of one embodiment of a method for handling error conditions associated with received request packets.
  • Figure 11 is a flow diagram of one embodiment of a method for handling a completion packet that is not expected by a system agent.
  • Figure 12 is a flow diagram of one embodiment of a method for a requesting device handling a completion packet with a completion status other than "Successful Completion."
  • Figure 13 is a flow diagram of one embodiment of a method for a completing device handling a completion packet with a completion status other than "Successful Completion."
  • Detailed Description
  • Described below are embodiments of a point-to-point packet-based interconnection architecture, communication protocol and related methods to provide a scalable and extensible general input/output communication platform for deployment within an electronic appliance. The disclosed embodiments involve an enhanced general input/output interconnection architecture and associated communications protocol. One example embodiment includes one or more of a root complex including a host bridge, a switch, or endpoints, each incorporating at least a subset of enhanced general input/output features to support enhanced general input/output communication between such elements.
  • Communication between the enhanced general input/output facilities of such elements is performed in one embodiment using serial communication channels employing a communication protocol which supports one or more innovative features including, but not limited to, virtual communication channels, tailer-based error forwarding ("tailers" are appended to transaction layer packets to indicate an error condition), support for legacy PCI-based devices, multiple request response types, flow control and/or data integrity management facilities. The communication protocol supported in this embodiment includes a communication protocol stack including a physical layer, a data link layer and a transaction layer.
  • In an alternate embodiment, a communications agent incorporates an enhanced general input/output engine comprising a subset of the foregoing features. Further, one or more elements of the various embodiments may be implemented in hardware, software, a propagated signal, or a combination thereof.
  • Figure 1 is a block diagram of an electronic appliance 100, which, for this embodiment, is a computer system. The system 100 includes a processor 102, a host bridge 103 included as part of a root complex 104, switch 108 and endpoint 110, each coupled as shown. The root complex 104, switch 108, and endpoint 110 include one or more instances of an enhanced general input/output communication port 106. As shown, each of the elements 102, 104, 108 and 110 are coupled to at least one other element through a communication link 112 supporting one or more enhanced general input/output communication channels via the enhanced general input/output communication port. The system 100 is intended to represent any of a wide variety of traditional and non-traditional computing systems, servers, network switches, network routers, wireless communication subscriber units, wireless communication telephony infrastructure elements, personal digital assistants, set-top boxes, or any electric appliance that would benefit from the communication resources introduced through integration of at least a subset of the enhanced general input/output interconnection architecture and/or communications protocol described herein.
  • In this example embodiment, processor 102 controls one or more aspects of the functional capability of the electronic appliance 100. In this regard, the processor 102 is representative of any of a wide variety of control logic devices including, but not limited to, one or more of a microprocessor, a programmable logic device (PLD), programmable logic array (PLA), application specific integrated circuit (ASIC), a microcontroller, and the like.
  • The root complex 104 provides a communications interface between the processor 102 and the switch 108 and endpoint 110. As used herein, the term "root complex" refers to a logical entity of an enhanced general input/output hierarchy that is closest to a host controller, a memory controller hub, an IO controller hub, or any combination of the above, or some combination of chipset/CPU elements (i.e., in a computing system environment). Although depicted in Figure 1 as a single unit, the root complex 104 may be implemented with multiple physical components. The root complex 104 is populated with one or more enhanced general input/output ports 106 to facilitate communication with other peripheral devices, e.g., the switch 108, endpoint 110 and, although not particularly depicted, the legacy bridges 114, or 116. In one embodiment, each enhanced general input/output interface port represents a different hierarchy domain. In this regard, the embodiment of Figure 1 denotes a root complex 104 with three hierarchy domains.
  • Figure 2 is a graphical illustration of an example enhanced general input/output port 106. In this embodiment, the enhanced general input/output port 106 implements a communication stack comprising a transaction layer 202, a data link layer 204 and a physical layer 206 including a logical sub-block 208, and a physical sub-block 210, as shown. Elements of the transaction layer will be discussed below in more detail.
  • The transaction layer 202 provides an interface between the enhanced general input/output architecture and a device core. A primary responsibility of the transaction layer 202 is the assembly and disassembly of packets for one or more logical devices within an agent.
  • One of the primary goals of the enhanced general input/output architecture is to maximize the efficiency of communication between devices. In one embodiment, the transaction layer implements a pipelined full split-transaction protocol as well as mechanisms for differentiating the ordering and processing requirements of transaction layer packets. The transaction layer further comprehends transaction layer packet construction and processing.
  • One embodiment of the enhanced general input/output architecture supports the following basic transaction types and address spaces: Memory, I/O, Configuration, and Message. Two addressing types are supported: 32bit and 64bit.
  • Transactions are carried using Request and Completion packets, which may be referred to simply as Requests and Completions. Completions are used only where required, e.g.: to return read data, or to acknowledge completion of I/O and configuration write transactions. Completions are associated with their corresponding Requests by the value in the Requester ID field of the packet header (discussed below).
  • All transaction layer packets in this embodiment start with a defined header. Some transaction layer packets include data following the header as determined by the format field specified in the transaction layer packet header. The transaction layer packet is limited in size by a predetermined maximum payload size value. The transaction layer packet data in this embodiment is four-byte naturally aligned and in increments of four-byte double-words.
  • Figure 3 is a diagram showing the format of one embodiment of the start of a transaction layer packet header. Each transaction layer packet header includes a three-bit format field (Fmt[2:0]). The transaction layer packet header also includes a four-bit type field (Type[3:0]). Both the Fmt and Type fields need to be decoded in order to determine the transaction layer packet format. Table 1 below shows example encodings for the Fmt field. Table 1 - Fmt Field Encodings
    000 2 double-word header, no data
    001 3 double-word header, no data
    010 4 double-word header, no data
    101 3 double-word header, with data
    110 4 double-word header, with data
  • The transaction layer header for this embodiment also includes a two-bit Extended Type/Extended Length field (Et/E1). This field is used to extend either the Type field or the Length field, depending on the value in the Type field. The Length field for this embodiment is ordinarily an eight-bit field, but may be extended to become a ten-bit field if the value in the Type field indicates that the Et/El field is to be used to extend the Length field. The Type field can be extended to become a six-bit field by appending the Et/El field, depending on the value in the Type[3:0] field. See Table 2 below for example Fmt, Type, and Et/El field encodings (alternative embodiments may use other encoding schemes). The Et/El field is used as an extension of the Type field except where noted. Table 2 - Fmt, Type, and Et/El Encodings
    Packet Type Fmt[2:0] Type[3:0] Et/E1[1:0] Description
    MRd 001 010 1001 E19 E18 Memory Read Request Et/E1 field used for Length[9:8]
    MRdLk 001 010 1011 00 Memory Read Request - Locked
    MWr 101 110 0001 E19 E18 Memory Write Request - Posted Et/E1 field used for Length[9:8]
    IORd 001 1010 00 IO Read Request
    IOWr 101 1010 00 IO Write Request
    CfgRd0 001 1010 01 Configuration Read Type 0
    CfgWr0 101 1010 01 Configuration Write Type 0
    CfgRd1 001 1010 11 Configuration Read Type 1
    CfgWr1 101 1010 11 Configuration Write Type 1
    Msg 010 011s2 s1s0 Message Request - The subfield s[2:0] specifies a group of messages. The Message field must be decoded to determine specific cycle including if a Completion is required or not.
    MsgD 110 011s2 s1s0 Message Request with Data - The sub-field s[2:0] specifies a group of messages. The Message field must be decoded to determine specific cycle including if a Completion is required or not.
    MsgComm 110 110c2 c1 c0 Message for Advanced Switching - The sub-field c[2:0] specifies the message type:
    000 - Unicast, Data Packet
    001 - Multicast, Data Packet
    010 - Signaling Packet, without interrupt
    011 - Reserved
    100 - Null signaling Packet, interrupt to Host in the destination Hierarchy
    101 - Null signaling Packet, interrupt to destination device
    110 - Signaling Packet, with interrupt to Host in the destination Hierarchy
    111 - Signaling Packet, with interrupt to destination device
    Cpl 001 0100 00 Completion without Data - used for IO and Configuration Write Completions, and Memory Read Completions with Completion Status other than "Successful Completion."
    CplD 101 0100 E19 E18 Completion with Data-used for Memory, IO, and Configuration Read Completions Et/E1 field used for Length[9:8]
    CplDLk 101 0101 01 Completion for Locked Memory Read
  • Request packets include a Request Header which for some types of Request packets will be followed by some number of double-words of data. The term "double-word" as used herein indicates a 32-bit length of data. For this example embodiment, the Length field for Message Request Headers is not used except for Messages that explicitly refer to a data length. Also for this embodiment, for Memory Read Requests and Memory Write Requests, the El/Et field is concatenated with the Length field to form a ten-bit length field. The ten-bit length field allows read and write requests indicating up to 4kB of data. Other types of transaction layer packets are limited by the size of the Length[7:0] field to indicating up to 1kB of data. The amount of data included in any transaction layer packet is limited in one embodiment to a predetermined maximum payload size. For transaction layer packets that include data, the value in the Length field and the actual amount of data should be equal. If the receiver determines that the Length field value and the actual amount of data do not match, then the packet is treated as a Malformed Transaction Layer Packet. Malformed Transaction Layer Packets are described below.
  • Figure 4 is a diagram of a request packet header supporting a 32-bit address format and Figure 5 is a diagram of a request packet header supporting a 64-bit address format. For one embodiment, memory read requests and memory write requests case use either the 32-bit address format or the 64-bit address format. For addresses below 4GB, the 32-bit format is used.
  • The request packet headers of Figures 4 and 5 also include a First Double Word Byte Enable field (1st DW BE) and a Last Double Word Byte Enable field (Last DW BE). The First Double Word Byte Enable field contains byte enables for the first double word of any memory read or write request. This field also contains byte enables for the only double word of an input/output or configuration request. The Last Double Word Byte Enable field contains byte enables for the last double word of any memory read or write request. The byte enable fields are not used with Messages because these fields overlap the message code field for a message request header (See Figure 7, discussed below).
  • For one embodiment, for each bit in the byte enable fields, a value of "0" indicates that the corresponding byte of data is not written or, if non-prefetchable, read at a completor. The term "completor" as used herein is meant to indicate a logical device addressed by a request packet header. A value of "1" indicates that the corresponding byte of data is written or, if non-prefetchable, read at the completor. For the First Double Word Byte Enables field, bit 0 corresponds to byte 0 of the first double word of data. Bit 1 corresponds to byte 1 of the first double word of data. Bit 2 corresponds to byte 2 of the first double word of data. Bit 3 corresponds to byte 3 of the first double word of data. For the Last Double Word Byte Enables field, bit 0 corresponds to byte 0 of the last double word of data. Bit 1 corresponds to byte 1 of the last double word of data. Bit 2 corresponds to byte 2 of the last double word of data. Bit 3 corresponds to byte 3 of the last double word of data.
  • The example packet headers of Figures 4, 5, 6, and 8 include a Requestor ID field, a Tag field, an Attribute field, and a Virtual Channel ID field. The Requestor ID field and the Tag field together form a Transaction ID field. The Requestor ID field is divided into a Bus Number field, a Device Number field, and a Function Number field.
  • The Tag field is a 5-bit field generated by each requesting device. The tag value is unique for all outstanding requests that require a completion for that requesting device. The Transaction ID field is included with all Requests and Completions. The Requestor ID field for these example embodiments is a 16-bit value that is unique for every function (a function is one independent section of a multi-function device identified in configuration space by a unique function number). Functions capture the Bus Number supplied with all configuration writes completed by the function, and supply this number in the Bus Number section of the Requestor ID field. Each logical device in a component is designed to respond to a unique Device Number for configuration requests addressing that component. For these example embodiments a component may contain many (perhaps up to several dozen) logical devices. Each function associated with a logical device in a component is designed to respond to a unique function number for configuration requests addressing that component and logical device. Each logical device may contain up to eight logical functions.
  • The Attributes field specifies characteristics of the transaction. Attributes that may be specified in the attributes field include a priority attribute, transaction ordering attributes, and cache coherency management attributes.
  • The Virtual Channel ID field identifies the virtual channel. For these example embodiments, the Virtual Channel ID field is a 4-bit field that allows identification of up to 16 virtual channels on a per transaction basis. For these example embodiments, virtual channel 0 is used for general purpose traffic and a virtual channel other than 0 is used for isochronous traffic.
  • Figure 6 is a diagram of a packet header for a Message. As seen in Table 2, messages may or may not include data and may or may not require completion. Messages are decoded by all devices in a system that support the enhanced general input/output interconnection architecture.
  • For message requests, the Message field is decoded in order to determine the specific cycle and to determine whether the message includes data and whether the message requires completion. The Message field for this embodiment is an 8-bit field located where the byte enable fields normally reside for other transaction types. Unsupported messages are treated by the receiving device as no-completion-required (no-completion-required transactions discussed below).
  • Messages for this example embodiment are divided into groups. There are eight groups that include data with the request and eight groups that do not. Other embodiments are possible using different numbers of groups. For this embodiment, as shown in Table 2, the eight groups that include data with the requests have a value of b110 in the Fmt field. The eight groups that do not include data have a value of b010 in the Fmt field. The sub-field s[2:0] incorporates one bit from the Type field and the two bits from the Et/El field. The sub-field s[2:0] indicates one of eight groups.
  • Examples of various messages that may be implemented include, but are not limited to, the following: messages for unlocking devices; messages for resetting devices; messages indicating a correctable error condition; messages indicating an uncorrectable error condition; messages indicating a fatal error condition; messages used to report bad request packets; messages relating to power management; messages related to ordering control/management; and messages for emulating legacy (e.g., PCI) interrupt signals (or other legacy signals). These various message types can be subdivided into one of the previously discussed groups. For example, all of the power management messages may be included in one group and the interrupt signaling messages may be included in another.
  • Figure 7 is a diagram showing a request header format for a configuration transaction. The configuration space is one of the four supported address spaces for these example embodiments.
  • Figure 8 is a diagram showing one embodiment of a format for a completion header. All read requests and some write requests require completion. Completions include a completion header that, for some types of completions, are followed by some number of double-words of data. The Completion Status[2:0] field shown in Figure 8 indicates the status for a completion. Table 3 shows one example encoding scheme. Table 3 - Completion Status Field Encoding Scheme
    Completion Status[2:0] Status
    000 Successful Completion
    001 Unsupported Request - Expecting Completion
    011 Reserved
    100 Completer Abort
  • The Completer ID[15:0] field contains the same type of information as the Requestor ID field described above. The value provided in the Completer ID field corresponds to the bus/device/function of the agent that completes the request. Completion headers contain the same values for Requester ID, Tag, and Channel ID as were supplied in the header of the request packet. Completion headers also contain the same value in the Attribute field as was initially supplied with the header of the request. Completion packets are routed by switches and root complexes to the port that initiated the corresponding request transaction.
  • For memory read request transactions, individual completion packets may provide less than the full amount of data requested by the corresponding read request so long as all of the completion packets associated with the corresponding read request, when combined, return the amount of data specified. For these example embodiments, I/O and Configuration read requests are completed with exactly one completion packet.
  • A completion that includes data specifies the amount of data in the packet header. If the completion packet actually contains an amount of data that differs from the specified amount, a malformed transaction layer packet results.
  • Figures 9a and 9b combined form a flow diagram of an example embodiment of a method for handling received transaction layer packets. The operations described below do not necessarily need to occur in a serial fashion. Some embodiments may perform some operations simultaneously. At block 905, a check is made to determine whether the values contained in the Fmt and Length fields of the received packet match the actual size of the packet. A mismatch indicates a malformed packet and an error case results as shown at block 925. Error case handling will be discussed below. If the actual size of the received packet does not indicate a mismatch with the Fmt and Length fields, then processing continues at block 910.
  • If the received packet is a memory request using 64 bit addressing, then at block 910 the address bits [63:32] are checked to see whether any of the address bits [63:32] are non-zero. If none of the address bits [63:32] are non-zero, then the result is a malformed packet and processing proceeds to the error case block 925. If at least one of the address bits [63:32] are non-zero, then processing continues at block 915.
  • At block 915, a check is made to determine whether any fields in the packet header contain reserved values. If reserved values are found, then the result is an malformed packet and processing proceeds to block 925. If no reserved values are found, then processing continues at block 930.
  • At block 930, a determination is made as to whether the packet is a request packet or a completion packet. If the packet is a completion packet, then processing proceeds to the completion handling block 935. Completion handling will be discussed more fully below. If the received packet is not a completion packet, then processing continues at block 940. Note that all packets that flow to block 940 are request packets.
  • At block 940, a check is made to determine whether the request packet is a request type that is supported by the completing device. If the request type is not supported, then the result is an unsupported request and processing proceeds to the error case block 925. For this example embodiment, if the unsupported request type is a broadcast message or a message using an encoding reserved for broadcast messages, then the packet is silently dropped and no error case results. If the request type is supported by the completing device, then processing continues at block 945.
  • If, as shown at block 945, the completing device is unable to respond to the request packet due to an internal error, then the result is a "completer abort" and processing proceeds to error case block 925. Otherwise, the request is serviced at block 950. In servicing the request, it may be necessary to repeat the processing indicated by blocks 940 and 945.
  • Once the request is serviced successfully, then processing continues at block 955. As indicated by block 955, if the processed request requires a completion, then a completion packet is returned at block 960.
  • Figure 10 is a flow diagram of one embodiment of a method for handling error conditions associated with received request packets. As seen at block 1010, if the received request is expecting a completion, then a completion with the appropriate completion status is transmitted at block 1020. The completion is routed back to the requesting device. If the received request is not expecting a completion, then an error message is transmitted to the requesting device at block 1030. The error is reported to the system at block 1040. The error message transmitting operating denoted at block 1030 may be implemented as a programmable option.
  • Some systems may include one or more PCI buses in addition to the previously discussed enhanced general input/output interconnection architecture. For memory, I/O, and configuration requests traveling through the enhanced general input/output interconnection architecture and destined to a device on a PCI bus, the completion status for these embodiments represents the actual PCI termination for the cycle. For example, a non-posted PCI cycle must actually be serviced on the PCI bus before a completion status can be determined. For all other cases, the completion status values are defined as discussed below.
  • When a request is completed successfully by the completing device, the resulting completion status value is "Successful Completion" (encoded in the completion status field as "000" for this embodiment as indicated in Table 3). For example, a read request from a host bridge is routed through a switch to a completer endpoint. The Completer responds with a completion packet indicating a successful completion status and also responds with the data for the read request. The switch routes this completion packet back to the host bridge.
  • When a request is received and decoded by the completing device, but the completing device does not support the requested transaction and the request requires a completion, the resulting completion status is "Unsupported Request" (encoded in the completion status field as "001" for this embodiment as indicated in Table 3). One example of an unsupported request would be a memory read request to an out-of-range address. In this case, the completer is not able to support the request and the requester is expecting a completion.
  • For the case where a request is received and decoded by the completing device and the completing device is unable to support the requested transaction and the requesting device in not expecting a completion, the resulting completion status is an unsupported request. Since the requesting device is not expecting a completion, the completion status is communicated to the requesting device via a message as described above in connection with Figure 10. An example of an unsupported request where the requesting device is not expecting a completion is a memory write transaction to an out-of-range address. The communication of the completion status via a message may be implemented as an optional feature.
  • When a completing device receives and decodes a request, but the completing device is unable to respond due to an internal error, the resulting completion status is a "Completer Abort" (encoded in the completion status field as "100" for this embodiment).
  • When a completing device receives a packet that violates packet formation rules, the result is a "Malformed Packet." The completing device responds to this situation by transmitting a "Malformed Packet" error message that is routed to the requesting device. A switch that receives a malformed packet must, for this embodiment, route the packet to the upstream port if no other port can be positively identified as the intended destination port.
  • When a read completion has a completion status other than "Successful Completion," no data is returned with the completion packet. The read completion with the non-successful completion status is the last completion transmitted for the request. For example, a completer may split a read request into four parts for servicing and the second completion packet results in a completer abort completion status. The final two completion packets are not transmitted. The requesting device, once it receives the completion packet with the non-successful completion status, considers the request terminated and should not expect additional completion packets corresponding to that read request.
  • Figure 11 is a flow diagram of one embodiment of a method for handling a completion packet that is not expected by a system agent. An "unexpected completion" occurs when an agent receives a completion that does not correspond to any outstanding requests issued by that same agent. For the example method of Figure 11, block 1110 indicates that if there is no unexpected completion then normal operation continues at block 1120. If, however, an unexpected completion is received, the unexpected completion packet is discarded at block 1130. After the packet is discarded, the above error condition may be reported to the system at block 1140. For this example embodiment, the reporting of the error may be an option that is programmable by software.
  • Figure 12 is a flow diagram of one embodiment of a method for a requesting device handling a completion packet with a completion status other than "Successful Completion." Block 1210 indicates that if the completion status is "Successful Completion", then normal operation continues at block 1220. If the completion status is other than "Successful Completion", then at block 1230 the value of the Completer ID field is recorded. For this embodiment the Completer ID value is stored in a register. Then, at block 1240, a "Received Unsuccessful Completion" bit is set in a register in the requesting device for this embodiment. The above error condition may be reported at block 1250. The reporting of the error condition may be implemented as a programmable option. A software agent may use the Completer ID value and the "Received Unsuccessful Completion" bit to track down the source of the error condition.
  • Figure 13 is a flow diagram of one embodiment of a method for a completing device handling a completion packet with a completion status other than "Successful Completion." Block 1310 indicates that if the completion status of a transmitted completion packet is "Successful Completion", then normal operation continues at block 1320. If the completion status is other than "Successful Completion", then at block 1330 the value of the Requester ID and Tag fields are recorded. For this embodiment, the Requester ID and Tag values are stored in one or more registers. Then, at block 1340, a "Transmitted Unsuccessful Completion" bit is set in a register in the completing device for this embodiment. The above error condition may be reported at block 1350. The reporting of the error condition may be implemented as a programmable option. A software agent may use the Requester ID and Tag values and the "Transmitted Unsuccessful Completion" bit to track down the source of the error condition.
  • In the foregoing specification the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than in a restrictive sense.
  • Reference in the specification to "an embodiment," "one embodiment," "some embodiments," or "other embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments, of the invention. The various appearances of "an embodiment," "one embodiment," or "some embodiments" are not necessarily all referring to the same embodiments.

Claims (18)

  1. An apparatus of a data processing system, comprising:
    a communication stack to communicate with another apparatus within the data processing system over a point-to-point interconnect, the communication stack having a transaction layer, a data link layer, and a physical layer, the transaction layer to transmit a plurality of packets to the another apparatus in serial over the point-to-point interconnect, each packet having a packet header, characterized in that the packet header including a first field to extend one of a second field and a third field depending on the contents of the second field.
  2. The apparatus of claim 1, wherein the second field is a type field.
  3. The apparatus of claim 2, wherein the third field is a length field.
  4. The apparatus of claim 3, wherein the first field is used to extend the length field when the type field indicates a memory read request transaction.
  5. The apparatus of claim 4, wherein the first field is located between and immediately adjacent to the type field and the length field in the packet header.
  6. The apparatus of claim 5, wherein the type field is located in the first byte of the packet header to be output by the data path output unit.
  7. An apparatus of a data processing system, comprising:
    a communication stack to communicate with another apparatus within the data processing system over a point-to-point interconnect, the communication stack having a transaction layer, a data link layer, and a physical layer, the transaction layer to receive a plurality of packets from the another apparatus in serial over the point-to-point interconnect, each packet having a packet header, characterized in that the packet header including a first field to extend one of a second field or a third field depending on the contents of the second field.
  8. The apparatus of claim 7, wherein the second field is a type field.
  9. The apparatus of claim 8, wherein the third field is a length field.
  10. The apparatus of claim 9, wherein the first field is used to extend the length field when the type field indicates a memory read request transaction.
  11. The apparatus of claim 10, wherein the first field is located between and immediately adjacent to the type field and the length field in the packet header.
  12. The apparatus of claim 11, wherein the type field is located in the first byte of the packet header to be output by the data path output unit.
  13. A system, comprising:
    a point-to-point interconnect;
    a transmitting device having a first communication stack including a transaction layer, a data link layer, and a physical layer, the transaction layer to transmit a plurality of packets serially onto the point-to-point interconnect, each packet having a packet header , and
    a receiving device communicatively couple to the transmitting device over the point-to-point interconnect, the receiving device having a second communication stack including a transaction layer, a data link layer, and a physical layer, the transaction layer to receive the plurality of packets serially from the transmitting device over the point-to-point interconnect characterized in that the packet header including a first field to extend one of a second field or a third field depending on the contents of the second field.
  14. The system of claim 13, wherein the second field is a type field.
  15. The system of claim 14, wherein the third field is a length field.
  16. The system of claim 15, wherein the first field is used to extend the length field when the type field indicates a memory read request transaction.
  17. The system of claim 16, wherein the first field is located between and immediately adjacent to the type field and the length field in the packet header.
  18. The system of claim 17, wherein the type field is located in the first byte of the packet header to be output by the data path output unit.
EP06009235A 2001-12-28 2002-12-05 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field Expired - Lifetime EP1684189B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/041,028 US6944617B2 (en) 2001-12-28 2001-12-28 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field
EP02797233A EP1459195B1 (en) 2001-12-28 2002-12-05 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
EP02797233A Division EP1459195B1 (en) 2001-12-28 2002-12-05 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field

Publications (3)

Publication Number Publication Date
EP1684189A2 EP1684189A2 (en) 2006-07-26
EP1684189A3 EP1684189A3 (en) 2006-08-02
EP1684189B1 true EP1684189B1 (en) 2007-08-29

Family

ID=21914326

Family Applications (2)

Application Number Title Priority Date Filing Date
EP06009235A Expired - Lifetime EP1684189B1 (en) 2001-12-28 2002-12-05 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field
EP02797233A Expired - Lifetime EP1459195B1 (en) 2001-12-28 2002-12-05 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP02797233A Expired - Lifetime EP1459195B1 (en) 2001-12-28 2002-12-05 Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field

Country Status (9)

Country Link
US (1) US6944617B2 (en)
EP (2) EP1684189B1 (en)
JP (1) JP4044523B2 (en)
CN (1) CN1608255B (en)
AT (2) ATE371900T1 (en)
AU (1) AU2002362100A1 (en)
DE (2) DE60212190T2 (en)
TW (1) TWI274497B (en)
WO (1) WO2003058469A1 (en)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7536495B2 (en) * 2001-09-28 2009-05-19 Dot Hill Systems Corporation Certified memory-to-memory data transfer between active-active raid controllers
US7146448B2 (en) * 2001-09-28 2006-12-05 Dot Hill Systems Corporation Apparatus and method for adopting an orphan I/O port in a redundant storage controller
US7403525B2 (en) * 2002-05-15 2008-07-22 Broadcom Corporation Efficient routing of packet data in a scalable processing resource
US7917658B2 (en) 2003-01-21 2011-03-29 Emulex Design And Manufacturing Corporation Switching apparatus and method for link initialization in a shared I/O environment
US7836211B2 (en) 2003-01-21 2010-11-16 Emulex Design And Manufacturing Corporation Shared input/output load-store architecture
US7046668B2 (en) * 2003-01-21 2006-05-16 Pettey Christopher J Method and apparatus for shared I/O in a load/store fabric
US8032659B2 (en) * 2003-01-21 2011-10-04 Nextio Inc. Method and apparatus for a shared I/O network interface controller
US7103064B2 (en) 2003-01-21 2006-09-05 Nextio Inc. Method and apparatus for shared I/O in a load/store fabric
US8346884B2 (en) 2003-01-21 2013-01-01 Nextio Inc. Method and apparatus for a shared I/O network interface controller
US7664909B2 (en) * 2003-04-18 2010-02-16 Nextio, Inc. Method and apparatus for a shared I/O serial ATA controller
US7953074B2 (en) 2003-01-21 2011-05-31 Emulex Design And Manufacturing Corporation Apparatus and method for port polarity initialization in a shared I/O device
US7617333B2 (en) * 2003-01-21 2009-11-10 Nextio Inc. Fibre channel controller shareable by a plurality of operating system domains within a load-store architecture
US8102843B2 (en) * 2003-01-21 2012-01-24 Emulex Design And Manufacturing Corporation Switching apparatus and method for providing shared I/O within a load-store fabric
US7698483B2 (en) * 2003-01-21 2010-04-13 Nextio, Inc. Switching apparatus and method for link initialization in a shared I/O environment
JP4564740B2 (en) * 2003-11-12 2010-10-20 株式会社リコー Imaging equipment system
US7411591B2 (en) * 2003-12-24 2008-08-12 Intel Corporation Graphics memory switch
US7606933B2 (en) * 2004-02-11 2009-10-20 Cray Canada Corporation Shared memory and high performance communication using interconnect tunneling
US7543096B2 (en) 2005-01-20 2009-06-02 Dot Hill Systems Corporation Safe message transfers on PCI-Express link from RAID controller to receiver-programmable window of partner RAID controller CPU memory
US7487274B2 (en) * 2005-08-01 2009-02-03 Asic Architect, Inc. Method and apparatus for generating unique identification numbers for PCI express transactions with substantially increased performance
US7949794B2 (en) 2006-11-02 2011-05-24 Intel Corporation PCI express enhancements and extensions
US7681089B2 (en) * 2007-02-20 2010-03-16 Dot Hill Systems Corporation Redundant storage controller system with enhanced failure analysis capability
US20090310489A1 (en) * 2008-06-17 2009-12-17 Bennett Andrew M Methods and apparatus using a serial data interface to transmit/receive data corresponding to each of a plurality of logical data streams
US8055805B2 (en) * 2009-03-31 2011-11-08 Intel Corporation Opportunistic improvement of MMIO request handling based on target reporting of space requirements
US8839057B2 (en) * 2011-02-03 2014-09-16 Arm Limited Integrated circuit and method for testing memory on the integrated circuit
JP6035704B2 (en) * 2011-03-18 2016-11-30 セイコーエプソン株式会社 Peripheral device, management device, and model information transmission method
US9575922B2 (en) * 2013-09-27 2017-02-21 Intel Corporation Apparatus, system, and method for improving equalization with a software equalization algorithm
CN110858834B (en) * 2018-08-23 2022-02-08 中国电信股份有限公司 User information transmission method, device, system and computer readable storage medium
CN113498596B (en) * 2020-01-22 2022-10-11 华为技术有限公司 PCIe-based data transmission method and device
CN115334169B (en) * 2022-04-28 2023-06-06 深圳证券通信有限公司 Communication protocol coding method capable of saving network bandwidth

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3332601A1 (en) 1983-09-09 1985-03-28 Siemens AG, 1000 Berlin und 8000 München CIRCUIT ARRANGEMENT FOR REGISTERING ADDRESSES OF MEMORY CELLS HAVING DEFECTIVE CONTENT CONTENTS
JPH0746322B2 (en) 1988-05-23 1995-05-17 日本電気株式会社 Faulty device identification system
AU665521B2 (en) 1990-10-03 1996-01-11 Thinking Machines Corporation Parallel computer system
CA2065578C (en) 1991-04-22 1999-02-23 David W. Carr Packet-based data compression method
JP2700843B2 (en) 1991-12-10 1998-01-21 三菱電機株式会社 Multiplex communication controller
JP3625302B2 (en) 1994-07-19 2005-03-02 株式会社東芝 Data delivery apparatus and method for network system
US5678007A (en) 1994-11-22 1997-10-14 Microsoft Corporation Method and apparatus for supporting multiple outstanding network requests on a single connection
US5905729A (en) 1995-07-19 1999-05-18 Fujitsu Network Communications, Inc. Mapping a data cell in a communication switch
EP1012730A1 (en) * 1996-01-31 2000-06-28 Ipsilon Networks, Inc. Improved method and apparatus for dynamically shifting between routing and switching packets in a transmission network
US5968197A (en) 1996-04-01 1999-10-19 Ericsson Inc. Method and apparatus for data recovery
US5917830A (en) * 1996-10-18 1999-06-29 General Instrument Corporation Splicing compressed packetized digital video streams
JPH10341247A (en) 1997-06-10 1998-12-22 Sony Corp Data transmitter, data receiver, data transmission system and data transmission method
US6084869A (en) 1997-10-06 2000-07-04 Hughes Electronics Corporation Resource reservation for packet-switched multiple-path communication system
US6055236A (en) * 1998-03-05 2000-04-25 3Com Corporation Method and system for locating network services with distributed network address translation
US6154839A (en) * 1998-04-23 2000-11-28 Vpnet Technologies, Inc. Translating packet addresses based upon a user identifier
JP2000003330A (en) 1998-06-16 2000-01-07 Sony Corp Device and method for information processing, and provision medium
US6515967B1 (en) 1998-06-30 2003-02-04 Cisco Technology, Inc. Method and apparatus for detecting a fault in a multicast routing infrastructure
US6359877B1 (en) 1998-07-21 2002-03-19 Telefonaktiebolaget Lm Ericsson (Publ) Method and apparatus for minimizing overhead in a communication system
US6389016B1 (en) 1998-10-14 2002-05-14 Nortel Networks Limited Data communication system and method for transporting data
US6424625B1 (en) 1998-10-28 2002-07-23 Telefonaktiebolaget Lm Ericsson (Publ) Method and apparatus for discarding packets in a data network having automatic repeat request
JP2000209220A (en) * 1999-01-14 2000-07-28 Toshiba Corp Computer, network controller, system using them and remote start method
KR100429187B1 (en) * 1999-05-11 2004-04-28 엘지전자 주식회사 ATM Packet Network and Method for Transmitting Packet
DE60038264T2 (en) * 1999-05-31 2009-04-23 Thomson Licensing Method for preprocessing data packets in a bus interface
CN1136706C (en) * 1999-09-01 2004-01-28 信息产业部武汉邮电科学研究院 Adaption method for integrating Internet with quasi-synchronous digital system
US6519731B1 (en) 1999-10-22 2003-02-11 Ericsson Inc. Assuring sequence number availability in an adaptive hybrid-ARQ coding system
US7333514B2 (en) 2001-08-09 2008-02-19 Telefonaktiebolaget Lm Ericsson (Publ) Flexible frame scheduler for simultaneous circuit-and packet-switched communication
US6691192B2 (en) * 2001-08-24 2004-02-10 Intel Corporation Enhanced general input/output architecture and related methods for establishing virtual channels therein
US6965571B2 (en) 2001-08-27 2005-11-15 Sun Microsystems, Inc. Precise error reporting

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
TW200305323A (en) 2003-10-16
CN1608255B (en) 2012-12-05
US6944617B2 (en) 2005-09-13
DE60222191T2 (en) 2008-01-03
EP1459195A1 (en) 2004-09-22
AU2002362100A1 (en) 2003-07-24
JP4044523B2 (en) 2008-02-06
WO2003058469A1 (en) 2003-07-17
EP1459195B1 (en) 2006-06-07
TWI274497B (en) 2007-02-21
EP1684189A2 (en) 2006-07-26
EP1684189A3 (en) 2006-08-02
US20030126281A1 (en) 2003-07-03
ATE329315T1 (en) 2006-06-15
DE60212190D1 (en) 2006-07-20
ATE371900T1 (en) 2007-09-15
CN1608255A (en) 2005-04-20
DE60212190T2 (en) 2007-04-19
JP2005514838A (en) 2005-05-19
DE60222191D1 (en) 2007-10-11

Similar Documents

Publication Publication Date Title
US10884971B2 (en) Communicating a message request transaction to a logical device
EP1684189B1 (en) Communicating transaction types between agents in a computer system using packet headers including an extended type/extended length field
KR20040037215A (en) An enhanced general input/output architecture and related methods for establishing virtual channels therein
EP1585029B1 (en) Method for handling completion packets with a non-successful completion status
US7191375B2 (en) Method and apparatus for signaling an error condition to an agent not expecting a completion
US7581026B2 (en) Communicating transaction types between agents in a computer system using packet headers including format and type fields

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

17P Request for examination filed

Effective date: 20060504

AC Divisional application: reference to earlier application

Ref document number: 1459195

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 13/40 20060101ALI20060627BHEP

Ipc: G06F 13/42 20060101AFI20060616BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AC Divisional application: reference to earlier application

Ref document number: 1459195

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 60222191

Country of ref document: DE

Date of ref document: 20071011

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071210

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

Ref country code: LI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

EN Fr: translation not filed
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071130

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080129

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071129

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20071231

26N No opposition filed

Effective date: 20080530

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20071205

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20071205

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20071205

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20071231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071129

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20071205

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20071231

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20200922

Year of fee payment: 19

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60222191

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220701