EP2062147A1 - Method and apparatus for conditional broadcast of barrier operations - Google Patents

Method and apparatus for conditional broadcast of barrier operations

Info

Publication number
EP2062147A1
EP2062147A1 EP07814600A EP07814600A EP2062147A1 EP 2062147 A1 EP2062147 A1 EP 2062147A1 EP 07814600 A EP07814600 A EP 07814600A EP 07814600 A EP07814600 A EP 07814600A EP 2062147 A1 EP2062147 A1 EP 2062147A1
Authority
EP
European Patent Office
Prior art keywords
data transfer
bus
bus transaction
strongly
ordered
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP07814600A
Other languages
German (de)
French (fr)
Other versions
EP2062147B1 (en
Inventor
Jim Sullivan
Barry Wolford
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of EP2062147A1 publication Critical patent/EP2062147A1/en
Application granted granted Critical
Publication of EP2062147B1 publication Critical patent/EP2062147B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/42Bus transfer protocol, e.g. handshake; Synchronisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14Handling requests for interconnection or transfer
    • G06F13/16Handling requests for interconnection or transfer for access to memory bus
    • G06F13/1605Handling requests for interconnection or transfer for access to memory bus based on arbitration
    • G06F13/161Handling requests for interconnection or transfer for access to memory bus based on arbitration with latency improvement
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14Handling requests for interconnection or transfer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus
    • G06F13/40Bus structure

Definitions

  • the present invention relates generally to the field of computers and in particular to a method and apparatus for conditionally broadcasting memory barrier bus operations.
  • Computers and other electronic systems and devices perform computational tasks in a wide variety of applications. These systems and devices increasingly integrate functionality beyond straight computation, often by integrating a variety of independent, function-specific circuits or modules, such as processors, mathematical co-processors, video and graphics engines, DMA controllers, GPS receivers, dedicated compression or encryption circuits, and the like. High-bandwidth data transfer between such devices and memory, as well as between the devices themselves, is critical to achieving the desired level of performance.
  • a data communication bus is a well-known structure providing a shared communication link between devices or modules in a processing system.
  • a common logical dichotomy of devices on a shared bus comprises “master” and “slave” devices.
  • Master devices initiate bus transactions, and commonly arbitrate among themselves for access to the bus, and, in some systems, for a share of the bus bandwidth.
  • Slave devices respond to data transfer bus transactions initiated by master devices, accepting data from the master device in response to write bus transaction and providing data to the master device in response to read bus transaction.
  • Most slave devices execute data transfer operations in the order in which the corresponding bus transactions occur on the shared bus.
  • system performance may be optimized by allowing data transfer operations - such as, for example, memory accesses - to be performed out of order.
  • a sequence of memory operations may be reordered to allow all operations to the same page in memory to be executed before a new page is opened.
  • Processing systems that are allowed to re-order memory operations are generally referred to as "weakly-ordered" processing systems.
  • processing systems that require memory operations to appear to be performed in the same order as their corresponding bus transactions are referred to as “strongly-ordered" processing systems.
  • slave devices in strongly- ordered systems may actually perform memory operations out of bus transaction order, so long as the memory state at any time appears to the processor(s) as if the memory operations had been performed in order. This characteristic is known as “global observability.”
  • Simple slave devices that always execute data transfer operations in the order received are inherently globally observable.
  • Other slave devices that may execute data transfer operations out of order "snoop" the data transfer operation addresses, and execute data transfer operations to the same address in a bus transaction order. These types of slave devices are also globally observable. Slave devices that execute data transfer operations without regard to bus transaction order are not globally observable.
  • a memory barrier is a bus transaction that ensures that all data transfer bus transactions issued by a master device prior to issuance of the memory barrier are executed, or appear to have been executed, before any data transfer bus transaction issued by the master device after the memory barrier.
  • Any memory barrier is a bus transaction that does not involve any data transfer between master and slave devices.
  • a memory barrier operation may be explicitly initiated by a master device. Alternatively, or additionally, a memory barrier operation may be generated by a bus controller in response to a strongly-ordered data transfer operation initiated by a master device.
  • a memory barrier transaction could be issued by the processor before issuing the read bus transaction.
  • the memory barrier would ensure that the write operation (as well as any other previously issued data transfer operation) is executed before the read operation is executed.
  • Memory barriers are described in co-pending US Patent Application, "Enforcing Strongly-Ordered Requests In A Weakly-Ordered Processing System," Serial No. 11/253,307, filed October 19, 2005, and assigned to the assignee of the present application, which is incorporated herein by reference in its entirety. [0009]
  • the memory barrier may be inefficient in processing systems with multiple slave devices.
  • a memory barrier transaction must be propagated to every slave device that can be accessed by the master device issuing a strongly-ordered data transfer bus transaction or memory barrier operation.
  • An acknowledgment of the memory barrier must be received from each of the slave devices before the strongly-ordered the data transfer bus transaction, or bus transaction following a memory barrier operation, is issued.
  • the delay imposed by the memory barrier is determined by the slowest slave device to respond. This may adversely affect performance, particularly where the slower slave devices perform data transfer operations in bus transaction order regardless of the memory barrier.
  • a status register associated with each slave device indicates, on a per-master basis, whether the slave currently has a pending (unexecuted) data transfer operation from each master device from which the slave may receive data transfer bus transactions. If a particular slave device indicates that it does not have any pending data transfer operations from a particular master device, a memory barrier from that master device need not be propagated to that slave device.
  • memory barriers are propagated only where necessary to enforce bus transaction ordering. That is, the memory barrier is directed only to slave devices that have pending (previously issued) data transfer operations from the master device requiring a strongly-ordered data transfer bus transaction or memory barrier operation.
  • Most conventional systems include at least some slave devices that inherently provide global observability. With respect to such slave devices, there is no need to dynamically monitor whether the slave device has pending data transfer operations from a particular master to determine whether or not to direct a memory barrier transaction to the slave device.
  • a slave device that ensures global observability many "opt out" of the memory barrier protocol.
  • the opt-out decision may be made dynamically by each slave device asserting a signal, may be set system-wide during a Power-On Self Test (POST) by polling status registers in the slave devices to ascertain their global observability, may be set by software, or it may be hardwired by system designers, so that only slave devices capable of performing out-of-order data transfer operations participate in the memory barrier protocol.
  • POST Power-On Self Test
  • This opt-out decision improves the system performance and reduces power consumption by only propagating memory barrier operations to slave devices as necessary, thus eliminating the delay associated with a memory barrier acknowledgment from a globally observable slave device.
  • One embodiment relates to a weakly-ordered processing system.
  • the system includes a plurality of slave devices and two or more master devices, each configured to direct data transfer bus transactions to one or more slave devices.
  • the system further includes a bus interconnect configured to implement data transfer bus transactions between master and slave devices, and further configured to direct an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to a strongly-ordered data transfer bus transaction request from a master device.
  • Another embodiment relates to a bus interconnect operative to direct data transfer bus transactions from one or more master devices to two or more slave devices in a weakly-ordered processing system.
  • the bus interconnect includes a bus register operative to queue data transfer bus transaction requests and a controller operative to control the issuance of data transfer bus transactions from the bus register and further operative to issue an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to a strongly-ordered data transfer bus transaction request from a master device.
  • Yet another embodiment relates to a method of executing a strongly- ordered data transfer bus transaction in a weakly-ordered processing system including one or more master devices and two or more slave devices. An indication of which of the slave devices are globally observable is maintained. An execution synchronization bus transaction is issued to one or more slave devices that are not globally observable in response to a strongly-ordered data transfer bus transaction request from a master device.
  • FIG. 1 is a functional block diagram of a weakly-ordered processing system
  • FIG. 2 is a functional block diagram of a bus interconnect in a weakly- ordered processing system
  • FIG. 3 is a functional block diagram of one embodiment of a controller in a bus interconnect for a weakly-ordered processing system
  • FIG. 4 is a functional block diagram of another embodiment of a controller in a bus interconnect for a weakly-ordered processing system.
  • FIG. 1 is a functional block diagram of a weakly-ordered processing system.
  • the processing system 100 may be disposed of in a computer or other computational system, including a portable electronic device, embedded system, distributed system, or the like.
  • the processing system 100 may be implemented as an integrated circuit, discrete components, or any combination thereof. Only those portions of the processing system 100 necessary for an explanation of embodiments of the present disclosure are depicted in FIG. 1. Those skilled in the art will recognize how best to implement the processing system 100 for each particular application.
  • the processing system 100 includes processors 102a-102c in communication with memory devices 104a-104c over a shared bus 106.
  • a bus interconnect 108 may be used to manage bus transactions between the processors 102a-102c and memory devices 104a-104c using point-to- point switching connections. In at least one embodiment of the bus interconnect 108, multiple direct links may be provided to allow two or more bus transactions to occur simultaneously.
  • One or more of the processors 102a-102c may be configured to execute instructions under control of an operating system or other software.
  • the instructions may reside in one or more of the memory devices 104a-104c.
  • Data may also be stored in the memory devices 104a-104c, and retrieved by the processors 102a-102c to execute certain instructions. The new data resulting from the execution of these instructions may be written back into the memory devices 104a-104c.
  • Each memory device 104a-104c may include a memory controller (not shown) and a storage medium (not shown), as known in the art.
  • Each processor 102a-102c may be provided with a dedicated channel 106a- 106c on the bus 106 for communicating with the bus interconnect 108.
  • the bus interconnect 108 may use a dedicated channel 106d-106f on the bus to communicate with each memory device 104a-104c.
  • a first processor 102a can access a target memory device 104b by sending a data transfer bus transaction request over its dedicated channel 106a on the bus 106.
  • the bus interconnect 108 determines the target memory device 104b from the address of the data transfer bus transaction request and issues a data transfer bus transaction to the target memory device 104b over the appropriate channel 106e on the bus 106.
  • a data transfer bus transaction may be a write transaction, a read transaction, or any other bus transaction related to a data transfer.
  • An originating processor 102a-102c may issue a write transaction to a target memory device 104a-104c by placing the appropriate address with a payload on the bus 106 and asserting a write enable signal.
  • An originating processor 102a-102c may issue a read transaction to a target memory device 104a-104c by placing the appropriate address on the bus 106 and asserting a read enable signal. In response to the read request, the target memory device 104a- 104c will send the payload back to the originating processor 102a-102c.
  • An originating processor 102a-102c may also issue bus transactions that are not data transfer bus transactions, such as a memory barrier transaction.
  • the processors 102a-102c may transmit an attribute with each memory access request.
  • the attribute may be any parameter that describes the nature of the data transfer bus transaction.
  • the attribute may be transmitted with the address over the address channel. Alternatively, the attribute may be transmitted using sideband signaling or some other methodology.
  • the attribute may be used to indicate whether or not the data transfer bus transaction request is strongly-ordered. A "strongly-ordered" request refers to a data transfer bus transaction request that cannot be executed out of order.
  • the bus interconnect 108 may monitor the attribute for each data transfer bus transaction request from the processors 102a-102c.
  • the bus interconnect 108 may enforce an ordering constraint on that transaction to every slave device that accepts bus transactions from that master and is capable of out-of-order execution of data transfer bus transactions, except for the slave device to which the strongly-ordered data transfer bus transaction is directed.
  • a data transfer bus transaction request from a first processor 102a to a target memory device 104a may include an attribute.
  • the bus interconnect 108 may determine from the attribute whether the transaction is strongly-ordered.
  • the bus interconnect 108 determines that the transaction is strongly-ordered, it sends a memory barrier to every memory device 104b and 104c that the first processor 102a is capable of accessing and that may execute data transfer bus transactions out of bus transaction order, other than the target memory device 104a.
  • the bus interconnect 108 also sends the strongly-ordered data transfer bus transaction to the target memory 104a without a memory barrier because the target memory device 104a will implicitly handle it as a strongly-ordered request due to the attribute associated with the data transfer bus transaction.
  • the processor 102a may issue a memory barrier bus transaction prior to issuing the strongly-ordered data transfer bus transaction. [0027] FIG.
  • a bus register 202 may be used to receive and store information from the bus 106.
  • the bus register 202 may be any type of storage device such as a First-In-First-Out (FIFO) memory, or other suitable storage device.
  • FIFO First-In-First-Out
  • the information received and stored by the bus register 202 may be any bus related information, but more specifically may include the address and attribute for each data transfer bus transaction request, and in the case of a write operation, the payload.
  • the bus register 202 may also store the attribute in the case of a non-data transfer bus transaction such as a memory barrier operation issued by a master device.
  • the address for each data transfer bus transaction request is also provided to a decoder 204.
  • the decoder 204 may be used to determine the target memory device for each data transfer bus transaction request in the bus register 202. This determination is used to control a bus switch 206.
  • the bus switch 206 is used to demultiplex each data transfer bus transaction from the bus register 202 to the appropriate channel of the bus 106 for its target memory device.
  • FIG. 3 is a functional block diagram of one embodiment of a controller 208 in a bus interconnect 108 for a weakly-ordered processing system.
  • the controller 208 enforces ordering constraints on memory operations based on information it receives from the decoder 204.
  • the information may include the attribute for each bus transaction, which may be stored in a first input register 302.
  • the information may also include data that identifies each memory device, other than the target memory device, that the originating processor is capable of accessing.
  • the particular memory devices accessible by each processor are preconfigured during the design stage, and therefore, can be programmed or hardwired into the decoder 204.
  • a second input register 304 may be used to store this information.
  • the first and second input registers 302, 304 may be separate registers as shown in FIG. 3, or alternatively a single register.
  • the information from the decoder 204 may be stored in registers shared with other bus interconnect functions. Each register may be a FIFO or any other suitable storage medium.
  • the controller 208 enforces ordering constraints on data transfer operations by controlling the timing of data transfer bus transactions released from the bus register 202. The process will first be described in connection with an attribute which indicates that a strongly-ordered memory data transfer bus transaction is ready to be released from the bus register 202.
  • the attribute is provided from the first input register 302 to a memory barrier generator 306 as an enabling signal.
  • the data stored in the second input register 304 is provided to the input of the memory barrier generator 306.
  • the data stored in the second input register 304 includes data that identifies each memory device, other than the target memory device, that the originating processor is capable of accessing.
  • the memory barrier generator 306 is enabled by the attribute, this information is used to generate a memory barrier for each memory device identified by the data.
  • Each memory barrier may be provided to the appropriate memory device by issuing a memory barrier transaction directed to the identified memory devices, with an attribute identifying the originating processor which initiated the strongly-ordered request.
  • the memory barriers may be provided to the appropriate memory devices using sideband signaling, or by other suitable means.
  • the memory barrier generator 306 may also generate memory barrier bus transactions in response to memory barrier bus transaction requests from a master device, which are also stored in the bus register 202, in a manner similar to that described above.
  • the memory barrier generator 306 may be used to suppress unnecessary memory barriers. For example, a memory barrier for a memory device accessible by the originating processor is superfluous, and may be suppressed, if the memory device is inherently globally observable. Globally observable slave devices may be identified in a number of ways.
  • a logical global observability register 307 includes a bit for every slave device in the system. The state of the global observability register bit indicates whether the associated slave device is globally observable, and hence may be exempted from a memory barrier transaction. The global observability register 307 is an input to the memory barrier generator 306.
  • the global observability register 307 may comprise a physical register set by system software during a Power On Self Test (POST), following a poll of slave devices to ascertain their behavior and capabilities with respect to global observability of bus transactions, such as by reading configuration status registers (CSRs) within the respective slave devices.
  • POST Power On Self Test
  • CSRs configuration status registers
  • one or more bits of a logical global observability register 307 may be hardwired by designers to a predetermined state indicating the known global observability of a corresponding slave device. This may reduce the complexity and execution time of the POST software.
  • one or more bits of a logical global observability register 307 may comprise a dynamic, binary signal from a slave device. This allows the slave device to indicate periods of global observability. For example, a slave device may queue data transfer operations in a buffer, and execute the operations out of bus transaction order.
  • the slave device When pending data transfer operations reside in the buffer, the slave device would indicate a lack of global observability, thus requiring memory barrier bus transactions be directed to the slave device if a processor issues a strongly-ordered data transfer bus transaction or memory barrier operation. However, if the buffer is empty, the slave device can guarantee global observability for at least the next occurring data transfer bus transaction (that is, the slave device guarantees that all data transfer operations previously issued to it have been executed). In this case, the slave device may indicate via the binary signal that it need not receive memory barrier transactions, and may maintain this indication only as long as its buffer is empty. [0035] In any given implementation, the logical global observability register 307 may comprise any mix of one or more physical registers set by system software, hardwired bits, or dynamic signals from slave devices, as required or desired in a particular application.
  • the processing system may be configured such that the first processor 102a can access the first, second, and third memory devices 104a, 104b, 104c.
  • the first processor 102a can access the first, second, and third memory devices 104a, 104b, 104c.
  • the corresponding attribute from the first input register 302 enables the memory barrier generator 306.
  • the data provided to the memory barrier generator 306 from the second input register 304 identifies the memory devices, other than the target memory device, that the first processor 104a can access. In this case, the data identifies the second and third memory devices 104b, 104c.
  • the memory barrier generator 306 checks the bits 307b, 307c in the logical global observability register 307 corresponding to the second and third memory devices 104b, 104c to determine whether either of memory device 104b, 104c is globally observable. In this example, bit 307b indicates global observability, and bit 307c does not. Accordingly, a memory barrier bus transaction is issued to the third memory device 104c, and the memory barrier to the second memory device 104b is suppressed.
  • logic 308 in the controller 208 may be used to monitor feedback from the memory devices for memory barrier acknowledgements.
  • a "memory barrier acknowledgement" is a signal from a memory device indicating that every data transfer operation from the processor requiring a strongly-ordered data transfer bus transaction or issuing a memory barrier operation, that preceded the memory barrier, has been executed.
  • the data from the second input register 304 and the bits of the logical global observability register 307 are used by the logic 308 to determine which memory devices should be monitored for memory barrier acknowledgements.
  • the logic 308 determines that all necessary memory barrier acknowledgements have been received, it generates a trigger that is used to release the corresponding data transfer bus transaction from the bus register 202 (or the next pending data transfer bus transaction if the memory barrier operation was issued directly by the master device). More specifically, the attribute from the first input register 302 is provided to the input of a select multiplexer 310. The multiplexer 310 is used to couple the trigger generated by the logic 308 to the bus register 202 when the attribute indicates that the data transfer bus transaction is strongly-ordered. The release signal output from the multiplexer 310 is also coupled to the decoder to synchronize the timing of the bus switch 206 (see FIG. 2).
  • a second multiplexer 312 in the controller 208 may be used to delay the release of data from the first and second registers 302, 304 until a data transfer acknowledgement is received from the target memory device when an attribute indicating a strongly-ordered data transfer bus transaction or master device-issued memory barrier operation is applied to the select input.
  • the attribute included in the bus transaction enforces an ordering constraint on the target memory device. Namely, the target memory device executes all outstanding data transfer operations issued by the originating processor before executing the strongly-ordered data transfer operation.
  • a data transfer acknowledgement is generated by the target memory device following the execution of the strongly-ordered data transfer operation.
  • the data transfer acknowledgement is fed back to the multiplexer 312 in the controller 208, where it is used generate a trigger to release new data from the first and second registers 302, 304 corresponding to the next data transfer bus transaction in the bus register 202. If the new data includes an attribute indicating that the corresponding data transfer bus transaction in the bus register 202 is strongly-ordered or comprises a master device- issued memory barrier operation, then the same process is repeated. Otherwise, the data transfer bus transaction can be released immediately from the bus register 202.
  • the controller 208 is configured to immediately release a data transfer bus transaction from the bus register 202 when the corresponding attribute in the first input register 302 indicates that the request is not strongly-ordered or a master device- issued memory barrier operation. In that case, the attribute disables the memory barrier generator 306. In addition, the attribute forces the multiplexer 310 into a state which couples an internally generated trigger to the bus register 202 to release the data transfer bus transaction. The data transfer bus transaction is released from the bus register 202 and coupled to the target memory device through the bus switch 206 (see FIG. 2). The data corresponding to the next data transfer bus transaction is then released from the first and second registers 302, 304 by an internally generated trigger output from the second multiplexer 312 in the controller 208. [0040] FIG.
  • FIG. 4 is a functional block diagram illustrating another embodiment of a controller in a bus interconnect for a weakly-ordered processing system.
  • a strongly-ordered data transfer bus transaction is released from the bus register 202 by the controller 208 at the same time the memory barriers are provided to the appropriate memory devices.
  • the first input register 302 is used to provide the attribute for a data transfer bus transaction to the memory barrier generator
  • the memory barrier generator 306 If the attribute indicates that the corresponding data transfer bus transaction is strongly-ordered, then the memory barrier generator 306 is enabled. When the memory barrier generator 306 is enabled, the data from the second input register 304 is used to identify each memory device accessible by the originating processor, other than the target memory device. For each memory device identified, the memory barrier generator 306 checks the corresponding bit of the logical global observability register
  • a memory barrier is then generated for each memory device, other than the target memory device, that does not (at that time) indicate that it is globally observable.
  • logic 314 in the controller 208 may be used to prevent subsequent data transfer bus transactions from being released from the bus register 202 until the strongly-ordered data transfer bus transaction is executed by the target memory device.
  • a delay 316 may be used to allow an internally generated trigger to release the strongly-ordered data transfer bus transaction from the bus register 202 before the trigger is gated off by the attribute. In this way, the data transfer bus transaction can be provided to the target memory device concurrently with the memory barriers for the remaining, non-globally observable memory devices accessible by the originating processor.
  • Logic 318 may be used to monitor feedback from the memory devices for the data transfer acknowledgement from the target memory device, and the memory barrier acknowledgements.
  • the data from the second input register 304 and the bits of the logical global observability register 307 are used by the logic 318 to determine which memory devices need to be monitored for memory barrier acknowledgements.
  • the logic 318 determines that the various data transfer and/or memory barrier acknowledgements have been received, it generates a trigger to release new data from the first and second input registers 302, 304 corresponding to the next data transfer bus transaction in the bus register 202.
  • the trigger is coupled through a multiplexer 320 which is forced into the appropriate state by the attribute from the first input register 202.
  • the new data includes an attribute indicating that the corresponding data transfer bus transaction in the bus register 202 is strongly-ordered, then the same process is repeated. Otherwise, the data transfer bus transaction can be released immediately from the bus register 202 with an internally generated trigger via the logic 314. An internally generated trigger may also be coupled through the multiplexer 320 to release the data from the first and second input registers 302, 304 for the next data transfer bus transaction in the bus register 202.
  • the present invention has been described herein with respect to a controller 208 within the bus interconnect 108 of a shared bus system, those of skill in the art will readily recognize that the invention is not limited to such implementation.
  • the global observability indicator for each slave device may be propagated to or accessible by each master device, which may determine whether a memory barrier bus transaction is required, and if so, to which slave devices it should be directed.
  • the present invention has been described herein with respect to particular features, aspects and embodiments thereof, it will be apparent that numerous variations, modifications, and other embodiments are possible within the broad scope of the present invention, and accordingly, all variations, modifications and embodiments are to be regarded as being within the scope of the invention. The present embodiments are therefore to be construed in all aspects as illustrative and not restrictive and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.

Abstract

A weakly-ordered processing system implements an execution synchronization bus transaction, or “memory barrier” bus transaction, to enforce strongly-ordered data transfer bus transactions. A slave device that ensures global observability may “opt out” of the memory barrier protocol. In various embodiments, the opt-out decision may be made dynamically by each slave device asserting a signal, may be set system-wide during a Power-On Self Test (POST) by polling the slave devices and setting corresponding bits in a global observability register, or it may be hardwired by system designers so that only slave devices capable of performing out-of-order data transfer operations participate in the memory barrier protocol.

Description

METHOD AND APPARATUS FOR CONDITIONAL BROADCAST OF BARRIER
OPERATIONS
BACKGROUND
[0001] The present invention relates generally to the field of computers and in particular to a method and apparatus for conditionally broadcasting memory barrier bus operations.
[0002] Computers and other electronic systems and devices perform computational tasks in a wide variety of applications. These systems and devices increasingly integrate functionality beyond straight computation, often by integrating a variety of independent, function-specific circuits or modules, such as processors, mathematical co-processors, video and graphics engines, DMA controllers, GPS receivers, dedicated compression or encryption circuits, and the like. High-bandwidth data transfer between such devices and memory, as well as between the devices themselves, is critical to achieving the desired level of performance. A data communication bus is a well-known structure providing a shared communication link between devices or modules in a processing system.
[0003] A common logical dichotomy of devices on a shared bus comprises "master" and "slave" devices. Master devices initiate bus transactions, and commonly arbitrate among themselves for access to the bus, and, in some systems, for a share of the bus bandwidth. Slave devices respond to data transfer bus transactions initiated by master devices, accepting data from the master device in response to write bus transaction and providing data to the master device in response to read bus transaction. Most slave devices execute data transfer operations in the order in which the corresponding bus transactions occur on the shared bus. [0004] In many cases, system performance may be optimized by allowing data transfer operations - such as, for example, memory accesses - to be performed out of order. For example, a sequence of memory operations may be reordered to allow all operations to the same page in memory to be executed before a new page is opened. Processing systems that are allowed to re-order memory operations are generally referred to as "weakly-ordered" processing systems.
[0005] Conversely, processing systems that require memory operations to appear to be performed in the same order as their corresponding bus transactions are referred to as "strongly-ordered" processing systems. Note that slave devices in strongly- ordered systems may actually perform memory operations out of bus transaction order, so long as the memory state at any time appears to the processor(s) as if the memory operations had been performed in order. This characteristic is known as "global observability." Simple slave devices that always execute data transfer operations in the order received are inherently globally observable. Other slave devices that may execute data transfer operations out of order "snoop" the data transfer operation addresses, and execute data transfer operations to the same address in a bus transaction order. These types of slave devices are also globally observable. Slave devices that execute data transfer operations without regard to bus transaction order are not globally observable.
[0006] In some cases, even in weakly-ordered processing systems, data transfer operation order must be enforced to ensure correct operation. For example, an application may require a processor to write data to memory before the processor reads from that memory location. Reordering these operations would result in incorrect data being returned in the read operation.
[0007] Various conventional techniques have been employed for executing ordered data transfer operations in a weakly-ordered processing system. One technique is simply to delay a particular data transfer bus transaction until all data transfer operations before it are executed. In the previous example, the processor may delay issuing a read request until it receives an indication that guarantees that the write operation data has been written to the memory location. Halting program execution to enforce data transfer operation ordering obviously has a negative effect on performance.
[0008] Another technique for executing ordered data transfer operations in a weakly-ordered processing system is to define an execution synchronization bus transaction as part of the bus protocol, also known as a "memory barrier." A memory barrier is a bus transaction that ensures that all data transfer bus transactions issued by a master device prior to issuance of the memory barrier are executed, or appear to have been executed, before any data transfer bus transaction issued by the master device after the memory barrier. Any memory barrier is a bus transaction that does not involve any data transfer between master and slave devices. A memory barrier operation may be explicitly initiated by a master device. Alternatively, or additionally, a memory barrier operation may be generated by a bus controller in response to a strongly-ordered data transfer operation initiated by a master device. In the previous example, a memory barrier transaction could be issued by the processor before issuing the read bus transaction. The memory barrier would ensure that the write operation (as well as any other previously issued data transfer operation) is executed before the read operation is executed. Memory barriers are described in co-pending US Patent Application, "Enforcing Strongly-Ordered Requests In A Weakly-Ordered Processing System," Serial No. 11/253,307, filed October 19, 2005, and assigned to the assignee of the present application, which is incorporated herein by reference in its entirety. [0009] The memory barrier may be inefficient in processing systems with multiple slave devices. In such systems, to enforce an ordering constraint, a memory barrier transaction must be propagated to every slave device that can be accessed by the master device issuing a strongly-ordered data transfer bus transaction or memory barrier operation. An acknowledgment of the memory barrier must be received from each of the slave devices before the strongly-ordered the data transfer bus transaction, or bus transaction following a memory barrier operation, is issued. Thus, the delay imposed by the memory barrier is determined by the slowest slave device to respond. This may adversely affect performance, particularly where the slower slave devices perform data transfer operations in bus transaction order regardless of the memory barrier.
[0010] Co-pending US Patent Application, "Minimizing Memory Barriers When Enforcing Strongly-Ordered Requests in a Weakly-Ordered Processing System," Serial No. 1 1/254,939, filed October 20, 2005 and assigned to the assignee of the present application, which is incorporated herein by reference in its entirety, discloses a system and method of dynamically minimizing memory barriers. A status register associated with each slave device indicates, on a per-master basis, whether the slave currently has a pending (unexecuted) data transfer operation from each master device from which the slave may receive data transfer bus transactions. If a particular slave device indicates that it does not have any pending data transfer operations from a particular master device, a memory barrier from that master device need not be propagated to that slave device. In this manner, memory barriers are propagated only where necessary to enforce bus transaction ordering. That is, the memory barrier is directed only to slave devices that have pending (previously issued) data transfer operations from the master device requiring a strongly-ordered data transfer bus transaction or memory barrier operation.
[0011] Most conventional systems include at least some slave devices that inherently provide global observability. With respect to such slave devices, there is no need to dynamically monitor whether the slave device has pending data transfer operations from a particular master to determine whether or not to direct a memory barrier transaction to the slave device.
SUMMARY
[0012] According to one or more embodiments disclosed herein, a slave device that ensures global observability many "opt out" of the memory barrier protocol. In various embodiments, the opt-out decision may be made dynamically by each slave device asserting a signal, may be set system-wide during a Power-On Self Test (POST) by polling status registers in the slave devices to ascertain their global observability, may be set by software, or it may be hardwired by system designers, so that only slave devices capable of performing out-of-order data transfer operations participate in the memory barrier protocol. This opt-out decision improves the system performance and reduces power consumption by only propagating memory barrier operations to slave devices as necessary, thus eliminating the delay associated with a memory barrier acknowledgment from a globally observable slave device. [0013] One embodiment relates to a weakly-ordered processing system. The system includes a plurality of slave devices and two or more master devices, each configured to direct data transfer bus transactions to one or more slave devices. The system further includes a bus interconnect configured to implement data transfer bus transactions between master and slave devices, and further configured to direct an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to a strongly-ordered data transfer bus transaction request from a master device.
[0014] Another embodiment relates to a bus interconnect operative to direct data transfer bus transactions from one or more master devices to two or more slave devices in a weakly-ordered processing system. The bus interconnect includes a bus register operative to queue data transfer bus transaction requests and a controller operative to control the issuance of data transfer bus transactions from the bus register and further operative to issue an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to a strongly-ordered data transfer bus transaction request from a master device. [0015] Yet another embodiment relates to a method of executing a strongly- ordered data transfer bus transaction in a weakly-ordered processing system including one or more master devices and two or more slave devices. An indication of which of the slave devices are globally observable is maintained. An execution synchronization bus transaction is issued to one or more slave devices that are not globally observable in response to a strongly-ordered data transfer bus transaction request from a master device.
BRIEF DESCRIPTION OF DRAWINGS
[0016] FIG. 1 is a functional block diagram of a weakly-ordered processing system; [0017] FIG. 2 is a functional block diagram of a bus interconnect in a weakly- ordered processing system;
[0018] FIG. 3 is a functional block diagram of one embodiment of a controller in a bus interconnect for a weakly-ordered processing system; and
[0019] FIG. 4 is a functional block diagram of another embodiment of a controller in a bus interconnect for a weakly-ordered processing system.
DETAILED DESCRIPTION
[0020] The detailed description set forth below in connection with the appended drawings is intended as a description of various embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced. In particular, for the purpose of explanation, embodiments are described with respect to a processing system comprising one or more processors issuing memory access requests to two or more memory controllers, and a bus interconnect. However, the invention is applicable to any master devices issuing data transfer bus transactions to slave devices in a shared bus system, and is not limited to processors and memory controllers.
[0021] FIG. 1 is a functional block diagram of a weakly-ordered processing system. The processing system 100 may be disposed of in a computer or other computational system, including a portable electronic device, embedded system, distributed system, or the like. The processing system 100 may be implemented as an integrated circuit, discrete components, or any combination thereof. Only those portions of the processing system 100 necessary for an explanation of embodiments of the present disclosure are depicted in FIG. 1. Those skilled in the art will recognize how best to implement the processing system 100 for each particular application. [0022] The processing system 100, as depicted in FIG. 1 , includes processors 102a-102c in communication with memory devices 104a-104c over a shared bus 106. The actual number of processors and memory devices required for any particular application may vary depending on the computational power required and the overall design constraints. A bus interconnect 108 may be used to manage bus transactions between the processors 102a-102c and memory devices 104a-104c using point-to- point switching connections. In at least one embodiment of the bus interconnect 108, multiple direct links may be provided to allow two or more bus transactions to occur simultaneously.
[0023] One or more of the processors 102a-102c may be configured to execute instructions under control of an operating system or other software. The instructions may reside in one or more of the memory devices 104a-104c. Data may also be stored in the memory devices 104a-104c, and retrieved by the processors 102a-102c to execute certain instructions. The new data resulting from the execution of these instructions may be written back into the memory devices 104a-104c. Each memory device 104a-104c may include a memory controller (not shown) and a storage medium (not shown), as known in the art.
[0024] Each processor 102a-102c may be provided with a dedicated channel 106a- 106c on the bus 106 for communicating with the bus interconnect 108. Similarly, the bus interconnect 108 may use a dedicated channel 106d-106f on the bus to communicate with each memory device 104a-104c. By way of example, a first processor 102a can access a target memory device 104b by sending a data transfer bus transaction request over its dedicated channel 106a on the bus 106. The bus interconnect 108 determines the target memory device 104b from the address of the data transfer bus transaction request and issues a data transfer bus transaction to the target memory device 104b over the appropriate channel 106e on the bus 106. A data transfer bus transaction may be a write transaction, a read transaction, or any other bus transaction related to a data transfer. An originating processor 102a-102c may issue a write transaction to a target memory device 104a-104c by placing the appropriate address with a payload on the bus 106 and asserting a write enable signal. An originating processor 102a-102c may issue a read transaction to a target memory device 104a-104c by placing the appropriate address on the bus 106 and asserting a read enable signal. In response to the read request, the target memory device 104a- 104c will send the payload back to the originating processor 102a-102c. An originating processor 102a-102c may also issue bus transactions that are not data transfer bus transactions, such as a memory barrier transaction.
[0025] In at least one embodiment of the processing system 100, the processors 102a-102c may transmit an attribute with each memory access request. The attribute may be any parameter that describes the nature of the data transfer bus transaction. The attribute may be transmitted with the address over the address channel. Alternatively, the attribute may be transmitted using sideband signaling or some other methodology. The attribute may be used to indicate whether or not the data transfer bus transaction request is strongly-ordered. A "strongly-ordered" request refers to a data transfer bus transaction request that cannot be executed out of order. [0026] The bus interconnect 108 may monitor the attribute for each data transfer bus transaction request from the processors 102a-102c. If an attribute indicates a strongly-ordered data transfer bus transaction request, the bus interconnect 108 may enforce an ordering constraint on that transaction to every slave device that accepts bus transactions from that master and is capable of out-of-order execution of data transfer bus transactions, except for the slave device to which the strongly-ordered data transfer bus transaction is directed. By way of example, a data transfer bus transaction request from a first processor 102a to a target memory device 104a may include an attribute. The bus interconnect 108 may determine from the attribute whether the transaction is strongly-ordered. If the bus interconnect 108 determines that the transaction is strongly-ordered, it sends a memory barrier to every memory device 104b and 104c that the first processor 102a is capable of accessing and that may execute data transfer bus transactions out of bus transaction order, other than the target memory device 104a. The bus interconnect 108 also sends the strongly-ordered data transfer bus transaction to the target memory 104a without a memory barrier because the target memory device 104a will implicitly handle it as a strongly-ordered request due to the attribute associated with the data transfer bus transaction. Alternatively, the processor 102a may issue a memory barrier bus transaction prior to issuing the strongly-ordered data transfer bus transaction. [0027] FIG. 2 is a functional block diagram illustrating an example of a bus interconnect 108 in a weakly-ordered processing system. The manner in which the bus interconnect is actually implemented will depend upon design considerations. Those skilled in the art will recognize the interchangeability of various designs, and how best to implement the functionality described herein for each particular application. [0028] Referring to FIG. 2, a bus register 202 may be used to receive and store information from the bus 106. The bus register 202 may be any type of storage device such as a First-In-First-Out (FIFO) memory, or other suitable storage device. The information received and stored by the bus register 202 may be any bus related information, but more specifically may include the address and attribute for each data transfer bus transaction request, and in the case of a write operation, the payload. The bus register 202 may also store the attribute in the case of a non-data transfer bus transaction such as a memory barrier operation issued by a master device. The address for each data transfer bus transaction request is also provided to a decoder 204. The decoder 204 may be used to determine the target memory device for each data transfer bus transaction request in the bus register 202. This determination is used to control a bus switch 206. The bus switch 206 is used to demultiplex each data transfer bus transaction from the bus register 202 to the appropriate channel of the bus 106 for its target memory device. A controller 208 may be used to control the timing of the data transfer bus transactions released from the bus register 202. [0029] FIG. 3 is a functional block diagram of one embodiment of a controller 208 in a bus interconnect 108 for a weakly-ordered processing system. The controller 208 enforces ordering constraints on memory operations based on information it receives from the decoder 204. The information may include the attribute for each bus transaction, which may be stored in a first input register 302. The information may also include data that identifies each memory device, other than the target memory device, that the originating processor is capable of accessing. The particular memory devices accessible by each processor are preconfigured during the design stage, and therefore, can be programmed or hardwired into the decoder 204. In any event, a second input register 304 may be used to store this information. The first and second input registers 302, 304 may be separate registers as shown in FIG. 3, or alternatively a single register. In some embodiments of the controller 208, the information from the decoder 204 may be stored in registers shared with other bus interconnect functions. Each register may be a FIFO or any other suitable storage medium. [0030] The controller 208 enforces ordering constraints on data transfer operations by controlling the timing of data transfer bus transactions released from the bus register 202. The process will first be described in connection with an attribute which indicates that a strongly-ordered memory data transfer bus transaction is ready to be released from the bus register 202. In this case, the attribute is provided from the first input register 302 to a memory barrier generator 306 as an enabling signal. At the same time, the data stored in the second input register 304 is provided to the input of the memory barrier generator 306. As indicated above, the data stored in the second input register 304 includes data that identifies each memory device, other than the target memory device, that the originating processor is capable of accessing. When the memory barrier generator 306 is enabled by the attribute, this information is used to generate a memory barrier for each memory device identified by the data. Each memory barrier may be provided to the appropriate memory device by issuing a memory barrier transaction directed to the identified memory devices, with an attribute identifying the originating processor which initiated the strongly-ordered request. Alternatively, the memory barriers may be provided to the appropriate memory devices using sideband signaling, or by other suitable means. The memory barrier generator 306 may also generate memory barrier bus transactions in response to memory barrier bus transaction requests from a master device, which are also stored in the bus register 202, in a manner similar to that described above.
[0031] According to one or more embodiments, the memory barrier generator 306 may be used to suppress unnecessary memory barriers. For example, a memory barrier for a memory device accessible by the originating processor is superfluous, and may be suppressed, if the memory device is inherently globally observable. Globally observable slave devices may be identified in a number of ways. [0032] In one embodiment of the controller 208, a logical global observability register 307 includes a bit for every slave device in the system. The state of the global observability register bit indicates whether the associated slave device is globally observable, and hence may be exempted from a memory barrier transaction. The global observability register 307 is an input to the memory barrier generator 306. The global observability register 307 may comprise a physical register set by system software during a Power On Self Test (POST), following a poll of slave devices to ascertain their behavior and capabilities with respect to global observability of bus transactions, such as by reading configuration status registers (CSRs) within the respective slave devices.
[0033] In one embodiment, which may be particularly advantageous in an ASIC or System On Chip (SOC) environment, one or more bits of a logical global observability register 307 may be hardwired by designers to a predetermined state indicating the known global observability of a corresponding slave device. This may reduce the complexity and execution time of the POST software. [0034] In another embodiment, one or more bits of a logical global observability register 307 may comprise a dynamic, binary signal from a slave device. This allows the slave device to indicate periods of global observability. For example, a slave device may queue data transfer operations in a buffer, and execute the operations out of bus transaction order. When pending data transfer operations reside in the buffer, the slave device would indicate a lack of global observability, thus requiring memory barrier bus transactions be directed to the slave device if a processor issues a strongly-ordered data transfer bus transaction or memory barrier operation. However, if the buffer is empty, the slave device can guarantee global observability for at least the next occurring data transfer bus transaction (that is, the slave device guarantees that all data transfer operations previously issued to it have been executed). In this case, the slave device may indicate via the binary signal that it need not receive memory barrier transactions, and may maintain this indication only as long as its buffer is empty. [0035] In any given implementation, the logical global observability register 307 may comprise any mix of one or more physical registers set by system software, hardwired bits, or dynamic signals from slave devices, as required or desired in a particular application.
[0036] Referring to FIGS. 1-3, an example will now be provided to illustrate the manner in which the global observability register bits can be used to suppress memory barriers. In this example, the processing system may be configured such that the first processor 102a can access the first, second, and third memory devices 104a, 104b, 104c. When a strongly-ordered data transfer bus transaction issued by the first processor 102a to the first memory device 104a (or alternatively when a memory barrier operation issued by the first processor 102a is at the output of the bus register 202), the corresponding attribute from the first input register 302 enables the memory barrier generator 306. The data provided to the memory barrier generator 306 from the second input register 304 identifies the memory devices, other than the target memory device, that the first processor 104a can access. In this case, the data identifies the second and third memory devices 104b, 104c. The memory barrier generator 306 checks the bits 307b, 307c in the logical global observability register 307 corresponding to the second and third memory devices 104b, 104c to determine whether either of memory device 104b, 104c is globally observable. In this example, bit 307b indicates global observability, and bit 307c does not. Accordingly, a memory barrier bus transaction is issued to the third memory device 104c, and the memory barrier to the second memory device 104b is suppressed.
[0037] Returning to FIG. 3, logic 308 in the controller 208 may be used to monitor feedback from the memory devices for memory barrier acknowledgements. A "memory barrier acknowledgement" is a signal from a memory device indicating that every data transfer operation from the processor requiring a strongly-ordered data transfer bus transaction or issuing a memory barrier operation, that preceded the memory barrier, has been executed. The data from the second input register 304 and the bits of the logical global observability register 307 are used by the logic 308 to determine which memory devices should be monitored for memory barrier acknowledgements. When the logic 308 determines that all necessary memory barrier acknowledgements have been received, it generates a trigger that is used to release the corresponding data transfer bus transaction from the bus register 202 (or the next pending data transfer bus transaction if the memory barrier operation was issued directly by the master device). More specifically, the attribute from the first input register 302 is provided to the input of a select multiplexer 310. The multiplexer 310 is used to couple the trigger generated by the logic 308 to the bus register 202 when the attribute indicates that the data transfer bus transaction is strongly-ordered. The release signal output from the multiplexer 310 is also coupled to the decoder to synchronize the timing of the bus switch 206 (see FIG. 2).
[0038] Once the data transfer bus transaction is released from the bus register, it is routed to the target memory device through the bus switch 206 (see FIG. 2). A second multiplexer 312 in the controller 208 may be used to delay the release of data from the first and second registers 302, 304 until a data transfer acknowledgement is received from the target memory device when an attribute indicating a strongly-ordered data transfer bus transaction or master device-issued memory barrier operation is applied to the select input. As discussed above, the attribute included in the bus transaction enforces an ordering constraint on the target memory device. Namely, the target memory device executes all outstanding data transfer operations issued by the originating processor before executing the strongly-ordered data transfer operation. A data transfer acknowledgement is generated by the target memory device following the execution of the strongly-ordered data transfer operation. The data transfer acknowledgement is fed back to the multiplexer 312 in the controller 208, where it is used generate a trigger to release new data from the first and second registers 302, 304 corresponding to the next data transfer bus transaction in the bus register 202. If the new data includes an attribute indicating that the corresponding data transfer bus transaction in the bus register 202 is strongly-ordered or comprises a master device- issued memory barrier operation, then the same process is repeated. Otherwise, the data transfer bus transaction can be released immediately from the bus register 202. [0039] The controller 208 is configured to immediately release a data transfer bus transaction from the bus register 202 when the corresponding attribute in the first input register 302 indicates that the request is not strongly-ordered or a master device- issued memory barrier operation. In that case, the attribute disables the memory barrier generator 306. In addition, the attribute forces the multiplexer 310 into a state which couples an internally generated trigger to the bus register 202 to release the data transfer bus transaction. The data transfer bus transaction is released from the bus register 202 and coupled to the target memory device through the bus switch 206 (see FIG. 2). The data corresponding to the next data transfer bus transaction is then released from the first and second registers 302, 304 by an internally generated trigger output from the second multiplexer 312 in the controller 208. [0040] FIG. 4 is a functional block diagram illustrating another embodiment of a controller in a bus interconnect for a weakly-ordered processing system. In this embodiment, a strongly-ordered data transfer bus transaction is released from the bus register 202 by the controller 208 at the same time the memory barriers are provided to the appropriate memory devices. More specifically, the first input register 302 is used to provide the attribute for a data transfer bus transaction to the memory barrier generator
306. If the attribute indicates that the corresponding data transfer bus transaction is strongly-ordered, then the memory barrier generator 306 is enabled. When the memory barrier generator 306 is enabled, the data from the second input register 304 is used to identify each memory device accessible by the originating processor, other than the target memory device. For each memory device identified, the memory barrier generator 306 checks the corresponding bit of the logical global observability register
307. A memory barrier is then generated for each memory device, other than the target memory device, that does not (at that time) indicate that it is globally observable. [0041] With the memory barrier generator 306 enabled, logic 314 in the controller 208 may be used to prevent subsequent data transfer bus transactions from being released from the bus register 202 until the strongly-ordered data transfer bus transaction is executed by the target memory device. A delay 316 may be used to allow an internally generated trigger to release the strongly-ordered data transfer bus transaction from the bus register 202 before the trigger is gated off by the attribute. In this way, the data transfer bus transaction can be provided to the target memory device concurrently with the memory barriers for the remaining, non-globally observable memory devices accessible by the originating processor.
[0042] Logic 318 may be used to monitor feedback from the memory devices for the data transfer acknowledgement from the target memory device, and the memory barrier acknowledgements. The data from the second input register 304 and the bits of the logical global observability register 307 are used by the logic 318 to determine which memory devices need to be monitored for memory barrier acknowledgements. When the logic 318 determines that the various data transfer and/or memory barrier acknowledgements have been received, it generates a trigger to release new data from the first and second input registers 302, 304 corresponding to the next data transfer bus transaction in the bus register 202. The trigger is coupled through a multiplexer 320 which is forced into the appropriate state by the attribute from the first input register 202. If the new data includes an attribute indicating that the corresponding data transfer bus transaction in the bus register 202 is strongly-ordered, then the same process is repeated. Otherwise, the data transfer bus transaction can be released immediately from the bus register 202 with an internally generated trigger via the logic 314. An internally generated trigger may also be coupled through the multiplexer 320 to release the data from the first and second input registers 302, 304 for the next data transfer bus transaction in the bus register 202.
[0043] Although the present invention has been described herein with respect to a controller 208 within the bus interconnect 108 of a shared bus system, those of skill in the art will readily recognize that the invention is not limited to such implementation. In particular, the global observability indicator for each slave device may be propagated to or accessible by each master device, which may determine whether a memory barrier bus transaction is required, and if so, to which slave devices it should be directed. [0044] Although the present invention has been described herein with respect to particular features, aspects and embodiments thereof, it will be apparent that numerous variations, modifications, and other embodiments are possible within the broad scope of the present invention, and accordingly, all variations, modifications and embodiments are to be regarded as being within the scope of the invention. The present embodiments are therefore to be construed in all aspects as illustrative and not restrictive and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.

Claims

CLAIMSWhat is claimed is:
1. A weakly-ordered processing system, comprising: a plurality of slave devices; at least one master device configured to direct data transfer bus transactions to one or more slave devices; and a bus interconnect configured to implement data transfer bus transactions between master and slave devices, and further configured to direct an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to an execution synchronization bus transaction request from a master device.
2. The system of claim 1 wherein the bus interconnect includes a logical global observability register comprising a plurality of bits, each bit corresponding to a slave device and indicating whether the slave device maintains global observability.
3. The system of claim 2 wherein the global observability register comprises one or more physical registers set by system software during system initialization.
4. The system of claim 3 wherein the system software polls configuration registers in slave devices to ascertain their global observability.
5. The system of claim 2 wherein one or more bits of the logical global observability register are hard-wired.
6. The system of claim 2 wherein one or more bits of the logical global observability register comprise dynamic binary signals from slave devices.
7. The system of claim 6 wherein a slave device buffers data transfer operations prior to executing the data transfer operations, and wherein the slave device indicates it is globally observable via a dynamic binary signal to the logical global observability register when its buffer is empty.
8. The system of claim 1 wherein data transfer bus transaction requests from the master devices include an attribute indicating whether the data transfer bus transaction is strongly-ordered.
9. The system of claim 1 wherein each slave device receiving the execution synchronization bus transaction executes all previously received data transfer operations from at least the master device issuing the strongly-ordered data transfer bus transaction.
10. The system of claim 1 wherein the slave device to which the strongly-ordered data transfer bus transaction is directed appears to master devices to have executed all previously received data transfer operations from at least the master device issuing the strongly-ordered data transfer bus transaction, prior to executing the strongly- ordered data transfer bus transaction.
11. The system of claim 1 wherein the bus interconnect directs the execution synchronization bus transaction only to non-globally observable slave devices to which the master device issuing the strongly-ordered data transfer bus transaction request may direct data transfer bus transactions.
12. The system of claim 1 wherein the bus interconnect is further configured to direct an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to a strongly ordered data transfer bus transaction request.
13. A bus interconnect operative to direct data transfer bus transactions from one or more master devices to two or more slave devices in a weakly-ordered processing system, comprising: a bus register operative to queue data transfer bus transaction requests; and a controller operative to control the issuance of data transfer bus transactions from the bus register and further operative to issue an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to an execution synchronization bus transaction request from a master device.
14. The bus interconnect of claim 13, wherein the controller includes a logical global observability register indicating which slave devices are globally observable.
15. The bus interconnect of claim 14 wherein the logical global observability register comprises a physical register set by system software.
16. The bus interconnect of claim 15 wherein the system software polls status registers in slave devices to ascertain their global observability, prior to setting the global observability register.
17. The bus interconnect of claim 14 wherein one or more bits of the logical global observability register are hardwired by system designers.
18. The bus interconnect of claim 14 wherein one or more bits of the logical global observability register comprise dynamic binary signals from slave devices.
19. The bus interconnect of claim 18 wherein a slave device is operative to buffer data transfer operations prior to executing them, the slave device indicating global observability via a dynamic binary signal when the buffer is empty.
20. The bus interconnect of claim 12, further comprising a decoder logically connected to the controller and operative to ascertain to which slave device a pending data transfer bus transaction is directed, and further operative to detect strongly- ordered data transfer bus transactions.
21. The bus interconnect of claim 14, further comprising a bus switch receiving data transfer bus transactions from the bus register, the bus switch operative to direct the data transfer bus transactions to slave devices under the control of the decoder.
22. The bus interconnect of claim 13 wherein the controller is further operative to issue an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to a strongly ordered data transfer bus transaction request.
23. A method of executing a strongly-ordered data transfer bus transaction in a weakly-ordered processing system including one or more master devices and two or more slave devices, comprising: maintaining an indication of which of the slave devices are globally observable; and issuing an execution synchronization bus transaction to one or more slave devices that are not globally observable in response to an execution synchronization bus transaction request from a master device.
24. The method of claim 23 further comprising detecting a strongly-ordered data transfer bus transaction by decoding an attribute of each data transfer bus transaction request received from a master device.
25. The method of claim 23 wherein the execution synchronization bus transaction is issued only to non-globally observable slave devices to which the master device issuing a strongly-ordered data transfer bus transaction request may direct data transfer bus transactions.
26. The method of claim 23 wherein in maintaining an indication of which of the slave devices are globally observable comprises maintaining a logical global observability status register, one bit of which corresponds to each slave device.
27. The method of claim 26 further comprising: polling status registers in slave devices during initialization to ascertain each slave device's global observability; and setting a physical global observability status register.
28. The method of claim 26 wherein maintaining an indication of which of the slave devices are globally observable comprises receiving a dynamic binary signal from one or more slave devices indicating the slave device's global observability.
29. The method of claim 23 further comprising, for each a slave device receiving an execution synchronization bus transaction, executing all pending data transfer operations from at least the master device issuing the strongly-ordered data transfer bus transaction request.
30. The method of claim 23 further comprising, for the slave device receiving the strongly-ordered data transfer bus transaction, executing all pending data transfer operations from at least the master device issuing the strongly-ordered data transfer bus transaction request, prior to executing the strongly-ordered data transfer bus transaction.
31. The method of claim 23 further comprising:
Receiving a strongly-ordered data transfer bus transaction request.
EP07814600A 2006-08-31 2007-08-31 Method and apparatus for conditional broadcast of barrier operations Not-in-force EP2062147B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/468,894 US7783817B2 (en) 2006-08-31 2006-08-31 Method and apparatus for conditional broadcast of barrier operations
PCT/US2007/077336 WO2008028101A1 (en) 2006-08-31 2007-08-31 Method and apparatus for conditional broadcast of barrier operations

Publications (2)

Publication Number Publication Date
EP2062147A1 true EP2062147A1 (en) 2009-05-27
EP2062147B1 EP2062147B1 (en) 2011-04-27

Family

ID=38943823

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07814600A Not-in-force EP2062147B1 (en) 2006-08-31 2007-08-31 Method and apparatus for conditional broadcast of barrier operations

Country Status (9)

Country Link
US (1) US7783817B2 (en)
EP (1) EP2062147B1 (en)
JP (1) JP4891405B2 (en)
KR (1) KR101056153B1 (en)
CN (1) CN101506783B (en)
AT (1) ATE507530T1 (en)
DE (1) DE602007014226D1 (en)
TW (1) TW200819989A (en)
WO (1) WO2008028101A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7237098B2 (en) * 2003-09-08 2007-06-26 Ip-First, Llc Apparatus and method for selectively overriding return stack prediction in response to detection of non-standard return sequence
US9026744B2 (en) * 2005-03-23 2015-05-05 Qualcomm Incorporated Enforcing strongly-ordered requests in a weakly-ordered processing
US7500045B2 (en) * 2005-03-23 2009-03-03 Qualcomm Incorporated Minimizing memory barriers when enforcing strongly-ordered requests in a weakly-ordered processing system
US7917676B2 (en) 2006-03-10 2011-03-29 Qualcomm, Incorporated Efficient execution of memory barrier bus commands with order constrained memory accesses
US8352682B2 (en) * 2009-05-26 2013-01-08 Qualcomm Incorporated Methods and apparatus for issuing memory barrier commands in a weakly ordered storage system
GB2474446A (en) * 2009-10-13 2011-04-20 Advanced Risc Mach Ltd Barrier requests to maintain transaction order in an interconnect with multiple paths
US8285937B2 (en) * 2010-02-24 2012-10-09 Apple Inc. Fused store exclusive/memory barrier operation
US8577986B2 (en) 2010-04-02 2013-11-05 Microsoft Corporation Mapping RDMA semantics to high speed storage
US9350806B2 (en) 2012-09-07 2016-05-24 International Business Machines Corporation Zero copy data transfers without modifying host side protocol stack parameters
US9489307B2 (en) * 2012-10-24 2016-11-08 Texas Instruments Incorporated Multi domain bridge with auto snoop response

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04190435A (en) 1990-11-26 1992-07-08 Hitachi Ltd Memory access order guarantee method for multiprocessor system
US5778438A (en) 1995-12-06 1998-07-07 Intel Corporation Method and apparatus for maintaining cache coherency in a computer system with a highly pipelined bus and multiple conflicting snoop requests
US5893165A (en) * 1996-07-01 1999-04-06 Sun Microsystems, Inc. System and method for parallel execution of memory transactions using multiple memory models, including SSO, TSO, PSO and RMO
US6047334A (en) 1997-06-17 2000-04-04 Intel Corporation System for delaying dequeue of commands received prior to fence command until commands received before fence command are ordered for execution in a fixed sequence
US6088771A (en) 1997-10-24 2000-07-11 Digital Equipment Corporation Mechanism for reducing latency of memory barrier operations on a multiprocessor system
US6370632B1 (en) 1997-11-18 2002-04-09 Intrinsity, Inc. Method and apparatus that enforces a regional memory model in hierarchical memory systems
US6038646A (en) 1998-01-23 2000-03-14 Sun Microsystems, Inc. Method and apparatus for enforcing ordered execution of reads and writes across a memory interface
US6247102B1 (en) * 1998-03-25 2001-06-12 Compaq Computer Corporation Computer system employing memory controller and bridge interface permitting concurrent operation
US6073210A (en) 1998-03-31 2000-06-06 Intel Corporation Synchronization of weakly ordered write combining operations using a fencing mechanism
US6816934B2 (en) * 2000-12-22 2004-11-09 Hewlett-Packard Development Company, L.P. Computer system with registered peripheral component interconnect device for processing extended commands and attributes according to a registered peripheral component interconnect protocol
US6167492A (en) 1998-12-23 2000-12-26 Advanced Micro Devices, Inc. Circuit and method for maintaining order of memory access requests initiated by devices coupled to a multiprocessor system
US6275913B1 (en) 1999-10-15 2001-08-14 Micron Technology, Inc. Method for preserving memory request ordering across multiple memory controllers
US6708269B1 (en) * 1999-12-30 2004-03-16 Intel Corporation Method and apparatus for multi-mode fencing in a microprocessor system
US6963967B1 (en) 2000-06-06 2005-11-08 International Business Machines Corporation System and method for enabling weak consistent storage advantage to a firmly consistent storage architecture
US6609192B1 (en) 2000-06-06 2003-08-19 International Business Machines Corporation System and method for asynchronously overlapping storage barrier operations with old and new storage operations
JP3999943B2 (en) * 2001-03-13 2007-10-31 株式会社東芝 Multi-bank access control device and multi-bank access control method
US7398376B2 (en) * 2001-03-23 2008-07-08 International Business Machines Corporation Instructions for ordering execution in pipelined processes
US6996812B2 (en) * 2001-06-18 2006-02-07 International Business Machines Corporation Software implementation of synchronous memory barriers
ATE335237T1 (en) * 2001-08-24 2006-08-15 Intel Corp A GENERAL INPUT/OUTPUT ARCHITECTURE, PROTOCOL AND CORRESPONDING PROCEDURES FOR IMPLEMENTING FLOW CONTROL
US20030131175A1 (en) 2001-12-24 2003-07-10 Heynemann Tom A. Method and apparatus for ensuring multi-threaded transaction ordering in a strongly ordered computer interconnect
US6976115B2 (en) 2002-03-28 2005-12-13 Intel Corporation Peer-to-peer bus segment bridging
US7490218B2 (en) 2004-01-22 2009-02-10 University Of Washington Building a wavecache
WO2005121948A1 (en) 2004-06-02 2005-12-22 Sun Microsystems, Inc. Method and apparatus for enforcing membar instruction semantics in an execute-ahead processor
US7644409B2 (en) 2004-06-04 2010-01-05 Sun Microsystems, Inc. Techniques for accessing a shared resource using an improved synchronization mechanism
US7725618B2 (en) 2004-07-29 2010-05-25 International Business Machines Corporation Memory barriers primitives in an asymmetric heterogeneous multiprocessor environment
US7454570B2 (en) 2004-12-07 2008-11-18 International Business Machines Corporation Efficient memory update process for on-the-fly instruction translation for well behaved applications executing on a weakly-ordered processor
US7500045B2 (en) 2005-03-23 2009-03-03 Qualcomm Incorporated Minimizing memory barriers when enforcing strongly-ordered requests in a weakly-ordered processing system
US9026744B2 (en) 2005-03-23 2015-05-05 Qualcomm Incorporated Enforcing strongly-ordered requests in a weakly-ordered processing
US7917676B2 (en) 2006-03-10 2011-03-29 Qualcomm, Incorporated Efficient execution of memory barrier bus commands with order constrained memory accesses
JP2009269935A (en) 2008-04-30 2009-11-19 Sumitomo Metal Mining Co Ltd Silver film having golden metallic luster

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2008028101A1 *

Also Published As

Publication number Publication date
US7783817B2 (en) 2010-08-24
JP4891405B2 (en) 2012-03-07
US20080059683A1 (en) 2008-03-06
EP2062147B1 (en) 2011-04-27
KR101056153B1 (en) 2011-08-11
TW200819989A (en) 2008-05-01
JP2010501962A (en) 2010-01-21
WO2008028101A1 (en) 2008-03-06
DE602007014226D1 (en) 2011-06-09
CN101506783A (en) 2009-08-12
CN101506783B (en) 2011-04-20
ATE507530T1 (en) 2011-05-15
KR20090051238A (en) 2009-05-21

Similar Documents

Publication Publication Date Title
EP2062147B1 (en) Method and apparatus for conditional broadcast of barrier operations
US5535341A (en) Apparatus and method for determining the status of data buffers in a bridge between two buses during a flush operation
EP1466255B1 (en) Supercharge message exchanger
US5919254A (en) Method and apparatus for switching between source-synchronous and common clock data transfer modes in a multiple processing system
US8825922B2 (en) Arrangement for processing trace data information, integrated circuits and a method for processing trace data information
KR19980032140A (en) Shared bus system with transaction and destination ID
US6675251B1 (en) Bridge device for connecting multiple devices to one slot
JPH08235103A (en) Bridge circuit between two buses of computer system
JPH04218861A (en) Multiple-cluster-signal processor
US8990456B2 (en) Method and apparatus for memory write performance optimization in architectures with out-of-order read/request-for-ownership response
JP2007220046A (en) Bus device, bus system and information transfer method
US5519872A (en) Fast address latch with automatic address incrementing
CN110858187A (en) Multiprocessor system with distributed mailbox structure and processor error checking method thereof
KR20020008955A (en) Bus system and execution scheduling method for access commands thereof
US7774513B2 (en) DMA circuit and computer system
JP3600536B2 (en) Method and system for limiting corruption of write data and PCI bus system
US6633927B1 (en) Device and method to minimize data latency and maximize data throughput using multiple data valid signals
US9858222B2 (en) Register access control among multiple devices
US5799161A (en) Method and apparatus for concurrent data routing
US5809534A (en) Performing a write cycle to memory in a multi-processor system
KR100276136B1 (en) Method and apparatus for enabling cache streaming
KR100278805B1 (en) Data Mediation Device and Method in Multi-Processing System
EP1416393B1 (en) Processor interface having a stream register and a FIFO
KR20050067324A (en) Interface device between master/slave devices and method thereof

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20090326

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK RS

17Q First examination report despatched

Effective date: 20090622

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

DAX Request for extension of the european patent (deleted)
AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 602007014226

Country of ref document: DE

Date of ref document: 20110609

Kind code of ref document: P

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602007014226

Country of ref document: DE

Effective date: 20110609

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20110427

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20110427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110829

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110827

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110728

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110807

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20120130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007014226

Country of ref document: DE

Effective date: 20120130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110727

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110427

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20180716

Year of fee payment: 12

Ref country code: FR

Payment date: 20180718

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20180726

Year of fee payment: 12

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602007014226

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20190831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190831

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200303

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190831