WO2013044469A1

WO2013044469A1 - Deadlock-free adaptive routing algorithm

Info

Publication number: WO2013044469A1
Application number: PCT/CN2011/080304
Authority: WO
Inventors: Dong Xiang
Original assignee: Dong Xiang
Priority date: 2011-09-28
Filing date: 2011-09-28
Publication date: 2013-04-04

Abstract

A new deadlock-free routing scheme for meshes is proposed based on a new virtual network partitioning scheme, called channel overlapping. Two virtual networks can share some common virtual channels based on the new virtual network partitioning scheme. The deadlock-free adaptive routing method is then extended to deadlock-free adaptive fault-tolerant routing in 3D meshes still with two virtual channels. A few faulty nodes can make a higher dimensional mesh unsafe for fault-tolerant routing methods based on the block fault model, where the whole system (n-dimensional space) forms a fault block. Planar safety information in meshes is proposed to guide fault-tolerant routing and classifies fault-free nodes inside 2D planes. Many nodes globally marked as unsafe in the whole system become locally enabled inside 2D planes. This fault-tolerant deadlock-free adaptive routing algorithm is also extended to the one in n-dimensional meshes with two virtual channels.

Description

DEADLOCK-FREE ADAPTIVE ROUTING ALGORITHM

Field of the Invention

The present invention relates to parallel interconnection networks, more particularly to deadlock-free adaptive routing algorithms in mesh networks with fault-tolerance ability.

Background of the Invention

Torus and mesh networks have been widely used in recent experimental or commercial multi-computers. How to control the number of virtual channels required for deadlock-free adaptive routing in meshes/tori is a challenging problem for high-performance multi-computer systems, which is closely related to the amount of buffer and area overhead of a router. This is especially important for on-chip networks.

Mesh is a topology which has n-dimensional grid structure with k nodes in each dimension. In wormhole switching, every packet is divided into several flow control digits or flits. The first flit of a packet (called the header flit) contains the information for routing. The packet is pipelined through the network at flit level. When a header flit is blocked during transmission, all flits wait at their local nodes. In VCT switched networks, packets can cut through to the next router before the complete packet has been received. The main difference between a VCT-switched and wormhole-switched network is that routers need buffer a whole packet when blocking occurs. Therefore, a VCT switched router needs more buffers than a wormhole-switched router.

A physical channel can be divided into several logical or virtual channels. Each virtual channel operates as if each were using a distinct physical channel operating at part of the speed. Virtual channels were originally introduced to solve the problem of deadlock in networks. Deadlock is a network state where no messages can advance because each message requires a channel occupied by another message.

A routing algorithm determines the sequence of channels for a packet to traverse from the source to the destination. The routing scheme is important to performance of an interconnection network. It is essential to present a more effective routing algorithm for mesh networks. Adaptive routing algorithms use information about network traffic and/or channel status to avoid congested or faulty regions of the network. It has the ability to route packets through alternative paths in the presence of contention or faulty components.

Several routing algorithms were proposed for meshes and torus. The X-Y routing in a 2D mesh delivers a packet first along dimension X and then along dimension Y, after all hops along dimension X have been eliminated. Turn model was proposed for designing partially adaptive deadlock-free algorithms in a mesh. It prevents the minimum number of turns to avoid cyclic channel dependencies.

A virtual network is a subset of channels that are used to route packets toward a particular set of destinations. The channel sets corresponding to different virtual networks are disjoint. Depending on the destination, each packet is injected into a particular virtual network, where it is routed until it arrives at its destination. Virtual networks can be implemented by using disjoint sets of virtual channels for each virtual network and mapping those channels over the same set of physical channels.

Summary of the Invention

A new deadlock-free adaptive routing algorithm for meshes was proposed for 3D meshes based on a new virtual network partitioning scheme called channel overlapping. The channel overlapping virtual network partitioning scheme presents deadlock-free fully adaptive routing for most packets with only two virtual channels. Some special turns and prevented turns are introduced in order to break some potential cyclic channel dependencies generated by the channel overlapping scheme. A new fault model was proposed by constructing fault blocks inside separate planes. The proposed deadlock-free adaptive routing algorithm was extended to a deadlock-free adaptive fault-tolerant routing algorithm in 3D meshes with the same deadlock avoidance technique. Only two virtual channels are sufficient to support deadlock-free adaptive routing in 3D meshes with arbitrarily shaped faults.

Brief Description of Figures

Fig 1 - Fig 3 illustrate deadlock-free routing using channel overlapping.

Fig 4 - Fig 6 illustrate deadlock-free adaptive routing algorithm based on channel overlapping in 3D meshes.

Fig 7 illustrates an example of unsafe 5-ary 3D mesh.

Fig 8 - Fig 9 illustrate the planar safety information collecting method.

Fig 10 - Fig 12 illustrate deadlock-free adaptive fault-tolerant routing in 3D meshes.

Detailed Description of Preferred Embodiments

Fig.l presents that a 2D mesh can be partitioned into four separate virtual networks x+y+, x+y~, x^~y+, and x-y- in a traditional way. Each virtual network must be assigned separate virtual channels. Messages inserted into the network are partitioned into multiple different classes, where each class falls into a single virtual network. The virtual channels are assigned as x+y+ (c_x,\+, c ,i+), x+y- (c_xj+, c_y^—), x~y+ (c_{x \}-, Cy_s2+), and x—y- (c_x,2~, c_yjr . Here, "+" and "-" represents the increasing and decreasing directions, respectively. Labels in the brackets show virtual channel assignments of a virtual network. For example, means the first virtual channel along x+ direction. As for a 3D mesh, it can be partitioned into eight virtual networks: (1) x+y+z+, (2) x+y+z- (3) x+y-z+, (4) x+y-z- (5) x-y+z+, (6) x-y+z- (7) x-y-z+, and (8) x-y-z—. Four virtual channels are required to avoid deadlocks based on the partitioning scheme described earlier.

Fig.2 presents that the original four virtual networks in the assignment scheme shown in Fig.l, can be merged into two virtual networks, x+y* and x~y*, where "*" represents both "+" and "-" directions. The virtual channel assignment for the merged virtual network x+y* is presented in Fig. 2a. The virtual channel assignment for the merged virtual network χ-y* is presented in Fig. 2b. No packet uses the virtual channel c_x,₂+ and c_x,₂-. These virtual channels can be used as adaptive channels to improve performance and adaptivity of the routing algorithm.

As for 3D meshes the eight virtual networks can be merged into four different virtual networks: x+y+z* (c_x,i+, c_y,i+, c_Zjl), x+y-z* (c_x,₂+, c_y\^~, c_Zjl), x-y*z+ (<¾-, Cy,₂, c_z,₂+), and x^~y*z- (c_x,₂-, Cy,₂, c_Zj2-). In the first two merged virtual networks x+y+z*, x+y-z*, all z channels share the same virtual channel c_Zjl; in the last two virtual networks x-y*z+ and x-y*z-, all y channels share the same virtual channels <¾₂. Therefore only two virtual channels are used.

The proposed virtual network partitioning scheme is called channel overlapping. It can reduce the number of virtual channels for deadlock-free adaptive routing. Almost all of the previous methods based on network partitioning assign different virtual channels to different virtual networks. The proposed channel overlapping scheme makes some of the virtual networks share some common virtual channels, which provides deadlock-free adaptive routing and reduces the number of required virtual channels. Only two virtual channels are sufficient to support deadlock-free adaptive routing and deadlock-free adaptive fault-tolerant routing in 3D meshes.

As shown in Fig.3, let all packets be classified corresponding to all eight original virtual networks. The labels on the arrowed lines represent the classes of packets. Fig.3a presents the cyclic dependencies in x+y+z* and x+y-z*. Fig.3b presents acyclic dependencies of the first two virtual networks in xy- and yz-planes. Fig.3c presents cyclic dependencies in x-y*z+ and x-y*z^~. Fig3.d presents acyclic dependencies in planes xy and yz in the last two virtual networks. All cyclic channel dependencies can be removed by the special turns as presented by the dashed arrowed lines in Figs.3a and 3c. The way to avoid cyclic dependencies as shown in Fig.3a for packet class 3 is to use c_z,₂+ instead of c_Zjl+ after a turn from a y- channel to a z+ channel, while a turn from a y— channel to a z+ channel for packet class 4 also uses channel c_Zj2- instead of c_Zjl-.We shall show later in this section that only the four special turns are still not enough to avoid all potential cyclic channel dependencies.

Special turns are also necessary to remove the potential cyclic channel dependencies as presented in Fig.3c similarly. As shown in Fig.3c, a turn from a y- channel to a z- channel for packet class 8 uses c_Zjl- instead of c_Zj2-, and a turn from a y- channel to a z+ channel for packet class 6 uses virtual channel c_Zjl+ instead of c_z,₂+. As for packet classes 5 and 7, no constraint is necessary. Any turn from the first and second virtual network to the last two virtual networks uses only the z channels. It returns to the original virtual network after all consecutive z hops. The situation is similar to all special turns from the last two virtual networks to the first virtual networks. All the above special turns cannot have negative impact on performance.

Two prevented turns are necessary as presented in Fig.3a and 3c. A class 3 packet, which occupies a c_Zjl+, is prevented turning to a c_y - channel, as shown in Fig.3a. A class 8 packet is prevented turning to a c_y2 ^~ channel from a c_Zj2- channel, as shown in Fig.3c.

As shown in Fig.3, a packet that falls into the virtual network x+y+z* can be delivered with full adaptivity without any constraint. Similarly, any packet inside virtual network x^~y*z+ can also be delivered with full adaptivity. However, a packet inside virtual networks x+y—z* or x^~y*z^~ should be delivered with some constraints as presented in Figs.3a and 3c because some prevented turns are necessary in both virtual networks. Three bits can be used in the header flit to indicate the classes of a packet. The extra cost on latency to deliver a packet is trivial.

Fig.4 presents the deadlock-free adaptive routing algorithm. Parameters (x_s, y_s, z_s) and (xd, yd, z ) represent coordinates of the source and the destination, respectively, where (x_s, y_s, z_s) can be the coordinate of an intermediate node. Fig.5 presents the adaptive deadlock- free procedure for a 3D mesh in the virtual network x+y+z*. As mentioned earlier, no special turn and prevented turn is necessary in the virtual network. Therefore, a packet can be delivered with full adaptivity along all minimum paths. As shown in Fig.4, x_0jf, y_0jf, and z₀g represent the offsets along dimension x, y, and z, respectively. One of the channels <¾+, <¾ i+, and c_Zjl can be selected as the next hop when the offsets along dimensions x, y, and z are all not eliminated. One of the channels c_y,i+ and c_Zjl can be selected as the next hop when the offset along dimension x has been eliminated. One of the channels c_Xjl+ and c_y,i can be selected if z_o =0. Similarly, one of the channels c_Xjl+ and c_Zjl can be selected as the next hop when the offset along dimension y has been eliminated.

There exist a special turn and a prevented turn when a packet falls into the virtual network x^~y*z^~. The special turn includes a packet of class 8, which occupies a y— channel, can only request a c_Zjl- channel instead of a c_z,₂ ^_ channel. A packet of class 8, which occupies a y— channel, is prevented traversing to a c_y r channel, as presented in Fig.3c.

The details are presented in the routing procedure in the virtual network x^~y*z- in Fig.6. A packet of class 8, which occupies a c_y r channel, requests one of c_y r, c_x>2 ^~, and c_zj- when offsets along three dimensions are not eliminated. It requests one of c_x,2^~ and c_z,₂ ^~ when the packet of class 8 occupies a z-channel because of the prevented turn in this case. In any other cases, the packet can be delivered along any one of the following channels c_x,₂-, <¾₂, and c_z,₂ along the minimum feasible paths. The next hop can be selected similarly when one or two of the offsets along three dimensions have been eliminated.

A fault block is a set of connected faulty and unsafe nodes. A node is unsafe if it has at least two neighbors that is faulty or unsafe. Fig.7 presents that only seven faulty nodes make all fault-free nodes in the 5^χ5^χ5 mesh unsafe based on the block fault model in a traditional way. However, reliable packet routing can still be conducted inside many sub-meshes in a mesh in many cases although the system is unsafe. In our method, we would like to consider the safety of each fault-free node inside each plane, which does not consider the effects of any faulty components outside of the plane. The above scheme can enable many globally unsafe nodes with respect to the planes.

A fault-free node can have different states in different planes. We consider the safety of all fault-free nodes inside various 2D sub-meshes (planes). Many nodes become safe inside different planes although they are unsafe in the whole system. Consider node (0, 0, 0) in the faulty 5-ary 3D mesh, as shown in Fig.7. All fault-free nodes are unsafe in the whole system, but all of them except (0, 0, 0) and (1 , 1 , 0) are enabled in all planes. Nodes (0, 0, 0) and (1 , 1 , 0) are locally unsafe in the plane (*, *, 0) because they have two faulty neighbors along different dimensions in the plane. Faults out of the plane under consideration do not have any influence on the safety of nodes in the plane when the planar fault model is considered. Safety information based on the planar fault model is called planar safety information. Planar safety information of a fault-free node in the whole network or any planes is obtained. The planar safety information of a fault-free node can be obtained by the procedure given in Fig.8 and 9.

The planar safety information of a system can be updated now and then when some new faults occur. The proposed method can also tolerate dynamic faults. A new fault can occur in the process of delivering a packet, where the packet can be stored in multiple nodes. The packet must be consumed at one of the routers that keeps one or more data flits. A signal must be sent back to the source at the header flit. The packet is sent again after the planar safety information has been updated.

Fig.10 presents a new deadlock- free adaptive fault-tolerant routing algorithm for meshes. A packet is routed from a fault- free source to a fault- free destination. The deadlock avoidance scheme is the same as the one presented in Fig.4. The deadlock-free adaptive fault-tolerant routing scheme supports misroutes along a dimension with a "*" label for any packet. Any packet inside virtual networks x+y+z* and x^~y*z+ can be provided full adaptivity without any constraints. A special turn and a prevented turn for packets delivered inside the virtual networks χ-y+z* and x^~y*z- are required for packet classes 3 and 8, respectively.

As shown in Fig.11 , the adaptive deadlock- free fault-tolerant routing scheme is provided for a 3D mesh in the virtual network x^~y*z+. In the procedure as presented in Fig.11 , let (x_c, y_c, z_c) and (¾ y_d, z ) represent the coordinates of the current node and the destination. The offsets x₀g =x_c ^~Xd, y₀ff =y_c ^~yd, and z₀/f =z_c-Zd. The routing algorithm selects one of <¾-, Cy,₂, and c_z,₂+ as the next minimum hop when the offsets along three dimensions are not zero, where the function se(a, b, c) selects one of the three channels a, b, and c. The proposed algorithm adaptively selects one of the remaining channels along the minimum paths when the offset along one dimension has been eliminated, which routes the packet via the given virtual channel along the minimum path if one or two of the three neighbors along minimum paths are faulty or unsafe. The packet can be misrouted along dimension z when only one of the offsets (x_off and y₀f) is not zero and the neighbor in a minimum path along that dimension is faulty or unsafe.

As shown in Fig.12, the adaptive deadlock- free fault-tolerant routing scheme is provided for a 3D mesh in the virtual network x+y—z*. Similar to that in the fault- free meshes, there exist some special turns from a z+ channel to a y— channel for a packet of class 3, and prevented turns from a z+ channel to a y— channel for a packet of class 3. The packet uses virtual channel c_z,₂+ instead of c_Zjl+ when it turns from a c_y,_\— to a z+ channel for a packet of class 3. The deadlock- free adaptive fault-tolerant routing algorithm in the virtual network x+y—z* is quite similar to the one in Fig.11.

Claims

Claims What is clamed is:

1. A virtual network partitioning method with channel overlapping, said method comprising:

The 2D mesh network is divided into two virtual networks: x+y*(c_x,i+, Cy,i) and x—y*(c_x,i—, c_y,₂). Labels in the brackets show virtual channel assignments of a virtual network. "+" and "-" represents the increasing and decreasing directions. "*" represents both "+" and "-" directions. Virtual channel c_x,₂+ and c_x,₂ ^~ can be used as adaptive channels to improve performance and adaptivity of the routing algorithm.

The 3D mesh network can be divided into eight virtual networks: (1) x+y+z+, (2) x+y+z-, (3) x+y^~z+, (4) x+y—z-, (5) x^~y+z+, (6) χ—y+z—, (7) x-y—z+, and (8) x—y—z—. Each two of them can be merged into one virtual network, which are x+y+z* (<¾+, c_y,i+, c_z,i), x+y-z* (c_x,2+, Cy,i^~, c_z,i), x^~y*z+ (<¾-, c_y,₂, c_z,₂+), and x^~y*z^~ (c_x,₂- c_y,₂, Cz,2^~)- Only two virtual channels are used in each dimension.

2. A deadlock-free adaptive routing algorithm in meshes based on channel overlapping, said method comprising:

All packets inserted into the 3D mesh can be classified corresponding to all eight original virtual networks. The cyclic channel dependencies can be removed by the special turns. For packet class 3 is to use c_z,₂+ instead of c_{Zj l}+ after a turn from a y— channel to a z+ channel, while a turn from a y— channel to a z+ channel for packet class 4 also uses channel c_z,₂ ^~ instead of c_{Zj l}-. A turn from a y— channel to a z— channel for packet class 8 uses c_{Zj l}- instead of c_z,₂ ^~, and a turn from a y— channel to a z+ channel for packet class 6 uses virtual channel c_Zjl+ instead of c_z,₂+. As for packet classes 5 and 7, no constraint is necessary. Any turn from the first and second virtual network to the last two virtual networks uses only the z channels. It returns to the original virtual network after all consecutive z hops.

Special turns are still not enough to avoid all potential cyclic channel dependencies. Two prevented turns are necessary. A class 3 packet, which occupies a c_z,i+, is prevented turning to a c_y - channel. A class 8 packet is prevented turning to a Cy,2^_ channel from a c_z,₂ ^~ channel.

3. A deadlock- free adaptive routing algorithm with fault-tolerance ability based on channel overlapping, said method comprising:

A fault block is a set of connected faulty and unsafe nodes. A node is unsafe if it has at least two neighbors that is faulty or unsafe. In our method, the safety of each fault-free node inside each plane is considered, while the effects of any faulty components outside of the plane are not. A fault-free node can have different states in different planes. We consider the safety of all fault- free nodes inside various 2D sub-meshes (planes). Safety information based on the planar fault model is called planar safety information. Planar safety information of a fault-free node in the whole network or any planes is obtained, which is updated now and then when some new faults occur. A packet is routed from a fault-free source to a fault-free destination. The deadlock avoidance scheme is the same as the one presented in Claim 2. The deadlock-free adaptive fault-tolerant routing scheme supports misroutes along a dimension with a "*" label for any packet. Any packet inside virtual networks x+y+z* and x^~y*z+ can be provided full adaptivity without any constraints. A special turn and a prevented turn for packets delivered inside the virtual networks χ-y+z* and x^~y*z- are required for packet classes 3 and 8, respectively.