RU164236U1 - PCI Express-based CLUSTER CONTROL SWITCH - Google Patents

Abstract

A control switch for a cluster based on the PCI Express protocol, which has a processor (CPU), memory (RAM) and its own address space and performs the functions of routing traffic by translating the addresses of cluster nodes located in different address spaces by constructing internal translation tables of the switch addresses and search for the translated address in these tables according to an algorithm based on a comparison of the high order bits of the original non-translated address with the high bits of the translated address, containing non-transparent ports for connecting cluster nodes via a PCI Express cable system that have a PCI Express cable adapter with transparent ports, an internal PCI Express bus, a computational core consisting of a CPU and RAM, and its own address space.

Description

The utility model relates to computing, namely, to a switching environment based on the cable specification of the PCI Express computer bus.

As a prototype adopted a cluster system with direct switching channels (RU No. 2461055, G06F 13/00; G06F 15/163). The structure of the computing cluster on PCI Express has also been described in patent US 2013/0227193 A1, G06F 13/40, 08.29.2013.

The disadvantage of this system is the need to use a dedicated computing node as a master device, which is responsible for the distribution of address space between slave computers, which complicates the system setup. In addition, a single master device is a critical point of failure of the entire cluster system, which leads to a decrease in its reliability.

The task to which the claimed utility model is directed is to accelerate the process of transferring traffic from node to node within a PCI Express-based cluster through a management switch.

The technical result is the use of a control external switch at the base of the cluster, which allows to accelerate the transfer of traffic between the cluster nodes and simplify the translation of the addresses of the recipient nodes.

The specified technical result is achieved through the use of a control external switch as the master device of the PCI Express cluster, which uses the proposed algorithm for translating recipient addresses developed by the authors.

A control switch for a cluster based on the PCI Express protocol, which has a processor (CPU), memory (RAM) and its own address space and performs the functions of routing traffic by translating the addresses of cluster nodes located in different address spaces by constructing internal translation tables of the switch addresses and search for the translated address in these tables according to an algorithm based on a comparison of the high order bits of the original non-translated address with the high bits of the translated address, containing non-transparent ports for connecting cluster nodes via a PCI Express cable system that have a PCI Express cable adapter with transparent ports, an internal PCI Express bus, a computational core consisting of a CPU and RAM, and its own address space.

The proposed system is a computing cluster consisting of computing nodes supporting the use of the intra-system PCI Express bus and a specialized PCI Express control switch. In general, computing nodes can be personal computers with a PCI Express bus on the motherboard, server platforms. All computing nodes are switched into a single computing cluster through the use of a specialized cable system designed in accordance with the PCI Express cable specification. To be able to connect to the network, the node must have a specialized bus adapter. The bus adapter operates in transparent bridge mode. The core of the cluster is an external PCI Express control switch, which itself builds a routing table based on the traffic passing through it. All switch ports are made using an opaque bridge technology.

In this patent, all the computing nodes in the computing cluster are equal in relation to each other, all the functions for managing address spaces and traffic routing are concentrated in the PCI Express switch.

In FIG. 1 shows a general view of the cluster topology; in FIG. 2 shows the structure of an opaque port; in FIG. 3 shows a computing node; in FIG. 4 shows the format of the routing table, FIG. Figure 5 shows the developed algorithm for address translation in the opaque ports of the management switch.

General view of the cluster topology (Fig. 1) includes:

- PCI Express Management Switch - 1

- switching unit - 2

- opaque bridges - 3

- interface cables - 4

- transparent bridges - 5

- computing nodes - 6.

The switching environment itself is a combination of a PCI Express switch (1), with bus network adapters (8) connected to it via interface cables (4) installed in the computing nodes (6) of the computing cluster.

The PCI Express switch (1) is a fully autonomous system and has its own internal resources, including a central processing unit (CPU) (10) and random access memory (RAM) (11), which has an autonomous internal address space, the switching unit (2) has a specialized switching matrix (KM) (12), which is associated with opaque bridges (NM) (3) on the ports of the PCI Express control switch. The use of opaque ports on the PCI Express control switch is due to the fact that they delimit the address spaces of the control switch and all computing nodes from each other, thereby preventing conflicts during system initialization when configuration messages are exchanged on the PCI Express bus. The PCI Express control switch connects all devices using the point-to-point connection type, this type of connection is the only one possible on the PCI Express bus. In order for computing nodes to be able to “see” each other in the address space, so-called cross-access windows are created in the PCI Express control switch through which any of the computing nodes can “see” other nodes. Cross-access windows are special areas in the memory of the management switch that are accessible to all nodes connected to the switch. All compute nodes (6) in the bus network adapter (CA) (8) use transparent ports (5), which allow configuration messages to pass through to the PCI Express control switch.

Logically opaque bridge (3) (Fig. 2) is the terminal device for the PCI Express control switch itself and for computing nodes. Each of these terminals is assigned addresses from different address spaces of two paired systems. Using the base address registers (BRA) (7), addresses are translated when the computing node accesses the PCI Express control switch. Thus, using opaque bridges on the ports of the PCI Express control switch, the computing nodes are interfaced with the internal system of the PCI Express control switch.

A computing node in the general case is a computer with an intra-system PCI Express bus and a PCI Express bus adapter for connecting interface cables (4) (Fig. 3).

Computing node (6) has its own computing core (9), consisting of a processor (10) and memory (11), has its own address space, which is limited by opaque bridges on the ports of the control switch.

For routing traffic, the control switch uses a mechanism for translating the addresses of computing nodes. For this, in the process of traffic passing through the PCI Express 3.0 switch, an internal routing table is built (Fig. 4). The following necessary information is entered in the table: the most significant bits of the untranslated (source) address of the recipient computing node (13); full translated address of the receiving node (14), a reality bit (15), which indicates whether the current record in the routing table is valid (“0” is not a valid record, “1” is a valid record).

The address translation mechanism in the control switch works according to the algorithm developed by the authors (Fig. 5). At the beginning of the transaction, the node transmitting the data records the data that it transfers to the outgoing window of its local memory (16). The outgoing window is the range of the host address space, which is formed in each device connected to the PCI Express control switch at the time of initialization and serves to write data to it intended for transmission to another node. At the next step (17), the generated protocol packet is sent to the NM of the control switch. Since the present patent considers only a tabular method of address translation, the NM register “BarSetup” is deliberately configured to use this method (18). Then, the second double word of the PCI Express protocol transport layer packet (TLP), containing the broadcast address, is written into the address register of the NM RgA (19). In step (20), the base address of the broadcast address contained in the NM register RgA [Base] is compared with the base address contained in the NM BAR register [2]. If the addresses match, then the high bits of the source address (RgA [High]) are compared with the value (LT [AHigh]) in the lookup table (22). In the case of a successful search and validation of the validity field (23) (LT [Valid]), which confirms the relevance of the entry in the search table, the base address contained in RgA [Base] is replaced by the base address of the corresponding entry in the search table (LT [TAdr]) , in other words, the address is translated (24). Then, the translated base address from LT [TAdr] is recorded in the translated address register (RgAT) and the offset from the source address stored in the RgA [Ofs] register is added (25). The LT [Par] lookup table parameter, which determines the port number to which the packet with the translated address should be transmitted, is used to determine the output port number.

Claims (1)

A control switch for a cluster based on the PCI Express protocol, which has a processor (CPU), memory (RAM) and its own address space and performs the functions of routing traffic by translating the addresses of cluster nodes located in different address spaces by constructing internal translation tables of the switch addresses and search for the translated address in these tables according to an algorithm based on a comparison of the high order bits of the original non-translated address with the high bits of the translated address, containing non-transparent ports for connecting cluster nodes via a PCI Express cable system that have a PCI Express cable adapter with transparent ports, an internal PCI Express bus, a computational core consisting of a CPU and RAM, and its own address space.

Images