JPH05204876A - Hierarchical network and multiprocessor using the same - Google Patents

Abstract

PURPOSE:To provide a hierarchical network as a new inter-processor connecting network in a multiprocessor system provided with a large number of processors. CONSTITUTION:This network is the one which comprises one network by connecting plural networks whose nodes are coupled with a connection line as a p-ary/n-dimensional cube in hierarchical fashion, and plural nodes are selected from a p-ary/n-dimensional network at a lower layer, and a p-ary/m- dimensional network at the next layer can be comprised by connecting them with each other as setting selected gate nodes as a p-ary/m-dimensional cube. Similarly, the gate nodes are selected from plural p-ary/m-dimensional networks at the next layer, and a p-ary/one-dimensional network at a more higher layer is comprised by connecting them with each other setting as a p-ary/one- dimensional cube, then, one hierarchical network can be comprised of the whole networks.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical network having a small number of paths between nodes and a multiprocessor system using the hierarchical network.

[0002]

2. Description of the Related Art A network for connecting a plurality of nodes has been researched and various networks have been devised. Regarding the connection method of these nodes, "Combining method" by Kyoichi Kurokawa and Hideo Aiso (Information Processing, vol. 27, No. 9, Sep. 1)
989, pp. 1005-1021). In this description, networks are separated into "dynamic" networks and "dynamic" networks according to whether the coupling between input and output changes dynamically. Dynamic networks include crossbar networks, baseline networks, omega networks, butterfly networks (indirect n-cubes), delta networks, banyan networks and the like. On the other hand, examples of static networks include ring networks, star networks, tree networks, lattice networks, fully connected networks, n-cubes, and CCC.

On the other hand, in the field of computer architecture,
Various multiprocessor systems to which the above-mentioned networks are applied have been researched and developed. D. The paper "BB by Brocks et al.
NTC2000 Architecture and Programming Mod
els "(Proc. of COMPCON '91 pp.46-50, 199)
TC20 using the butterfly network described in 1)
00 (BBN, USA). In addition, the duplexed Yn described in Japanese Patent "JP-A-57-201931"
A multiprocessor system using a tree network called et may be used. In recent years, a hypercube (n-cube) type has attracted a great deal of attention as a network. As a multiprocessor to which this type of network is applied, nCUBE (nCUBE, USA) and iPS are used.
C (Intel Corp.) is being developed.

As described above, the hypercube has become one of the mainstream of today's multiprocessor systems. Moreover, there is k-array n-cube (J. DALLY, “Preforma”) as a concept including most of the above static networks.
nce Analysis of k-arry n-cube Interconnection Netw
ork ”, Trans. on Computers, vol 39, No. 6, JUNE199
0). Not only hypercubes, but ring networks, lattice networks, total networks, omega networks, indirect n-cubes, etc. are all k-array n-cu
This is a specialized network of be network or a network of the same type.

[0005]

A hypercube forming a conventional network will be briefly described with reference to FIGS. 4 and 5. 4 (a) to 4 (d) show two-dimensional, three-dimensional, and four-dimensional hypercubes, respectively.
Is the number N of nodes indicating a five-dimensional hypercube and the number M of inter-node connection links from one node is N = 2 in the case of one dimension, M = 1, and N = 4 in the case of two dimensions, M = 2,
N = 8 for 3D, M = 3, N = 1 for 4D
In the case of 6, M = 4 and 5 dimensions, N = 32, M = 5, and in the case of n dimensions, N = 2̂n (operator “̂” represents exponentiation).
The same shall apply hereinafter), M = n. In the conventional hypercube, as the number of nodes increases, the number of inter-node connection buses M increases.
M increases. For example, MM = (Nlog ₂ N) / 2 in a hypercube with N (= 2 ^ n) nodes. For example, in the one-dimensional case shown in FIG. 4B, N = 2, so M
M = (2log ₂ 2) / 2 N in the three-dimensional case shown in 1
= 8, MM = (8log ₂ 8) / 2 = 12, and when the number of nodes N is 2̂16, MM = 524,288. As the number of nodes increases in this way, the number of inter-node connection buses becomes enormous. Therefore, it becomes practically difficult to configure a multiprocessor system using this network so that the processors are arranged in the respective nodes.

In order to solve this problem, each vertex node of the hypercube is made into a ring shape to expand the number of nodes C
A CC (Cube Connected Cycle) network and the like were devised (described in the above-mentioned commentary paper No. 1027). However, in this network, there is a problem that the transfer distance between nodes becomes large depending on the position of each node as compared with the hypercube having the same number of nodes. In the tree network, which is a hierarchical network described on page 1014 of the above-mentioned commentary paper, there is a problem in that the transfer distance between nodes becomes large for the half-face CCC and the same machine in which the number of connection buses between nodes becomes small. For example, in an n-stage binary tree network with (2 ^ n) -1 nodes,
The transfer distance between nodes is about twice as large as that of an n-dimensional hypercube having almost the same number of nodes. Communication time between processors is very important in a multiprocessor system. The communication time between processors usually increases in proportion to the distance of the transfer path between nodes. In the case of the hypercube network, the maximum value Dmax of the inter-node transfer distance is Dmax = log ₂ N when the number of nodes is N (= 2 ^ n). For example, in a system in which the number of nodes N = 2 ^ 16, the maximum transfer distance Dmax = 16.

In addition, there is a problem in expandability in a multiprocessor system using a hypercube network. That is, when a certain processor is added to an arbitrary hypercube system, it is necessary to increase the network connection ports of the corresponding existing processors one by one. Therefore, it is virtually impossible to add more processors than the predesigned number of ports.

Further, in the tree network, since the transfer path is uniquely determined, there is a problem that a blocked state of the communication path frequently occurs and the parallel transfer of data is hindered.

In addition, one module or LSI
The following problems arise when constructing a large-scale multiprocessor system using a plurality of 1-module chip multiprocessor systems in which a plurality of processors are mounted on a chip. For example, a multiprocessor system consisting of 2 ^ 16 processors is
It is assumed that a 1-chip multiprocessor containing 8 (= 2 ^ 7) processors is applied to a plurality of hypercube networks and realized. In order to construct a 16-dimensional hypercube system using 2 ^ 9 of these chips, it is necessary to output 1152 inter-node connection buses from one chip, which is obviously a pin neck. Such a situation becomes a serious problem as the number of processors to be configured increases.

An object of the present invention is to provide a hierarchical network which can be constructed with a small number of inter-node connection buses when constructing a network having a large number of nodes. Another object of the present invention is to provide a hierarchical network having a short maximum communication distance between nodes.

Further, according to the present invention, the coding theory is applied to a method of selecting a plurality of nodes belonging to a network of an upper layer from nodes of a network of a lower layer, thereby reducing the number of inter-node connection buses and It is an object of the present invention to provide a hierarchical network in which the maximum transfer distance between them is equal to or less than that of a single hierarchical network having the same number of nodes.

It is also an object of the present invention to provide a communication method in the above new hierarchical network.

Another object of the present invention is to provide a multiprocessor system using such a new hierarchical network.

It is another object of the present invention to provide a multiprocessor system excellent in expandability by making the network hierarchical.

Furthermore, according to the present invention, when a plurality of processors are mounted on one module or a semiconductor chip and a plurality of them are combined to construct a larger-scale multiprocessor system, a connection bus between the modules or the semiconductor chips is provided. An object of the present invention is to provide a multiprocessor system that can avoid the problem of a pin neck of a module or a semiconductor chip by reducing the number.

[0016]

According to the present invention, a plurality of nodes are selected as nodes of an upper layer network from each of a plurality of lower layer networks having a plurality of nodes (hereinafter, the selected node is referred to as a gate node). Gate nodes that belong to the same lower layer network are connected to each other so that they are shorter than the communication distance in the network, and gate nodes that do not belong to the same lower layer network are connected to each other so that they can communicate with each other. It This provides a hierarchical network that enables transfer between nodes belonging to different lower layer networks while reducing the transfer distance within the same lower layer network.

Further, according to the present invention, each node of a plurality of p-adic n-dimensional lower layer networks is given a p-adic n-digit node number that can be uniquely determined in all the lower layer networks. When these node numbers are regarded as codes, the nodes to which the node numbers corresponding to the code words are given are selected as gate nodes, and the gate nodes are mutually connected as nodes in the p-adic m-dimensional upper layer network. As a result, a network having a smaller number of inter-node connection buses and a communication distance equal to or less than that of a p-adic cube having the same number of nodes is provided.

Further, in the present invention, when a node number uniquely given to a node in each of a plurality of lower layer networks is regarded as a code, the node whose node number is a code word of the t-fold error correction code is By being selected as a gate node and constructing a hierarchical network, a network having a small number of inter-node connection buses and a communication distance equal to or less than a single-layer network having nodes having the same number is provided.

The present invention provides a multiprocessor system in which a processor is arranged at a node of the above-mentioned network.

In the present invention, in the above multiprocessor system, the node selected as the gate node is connected to the other nodes (hereinafter,
Called a leaf node) with a single processor,
It is possible to provide a multiprocessor system with excellent expandability.

In the present invention, as the interprocessor communication method of the multiprocessor system, when data is transferred from a processor arranged in a node, data is transferred to a node belonging to the same lower layer network by the shortest path in the lower layer network. Data is transferred to a node belonging to another network via a gate node connected to the upper layer network.

Further, when a t-fold error correction code is used as a means for selecting a gate node when configuring the hierarchical network, the path from the leaf node a to the gate node b is set to the leaf node a. The node number in the given lower layer network is regarded as a code, and this is corrected by the error correction means in the t-fold error correction code to obtain the code word of the code, and the obtained code word reaches the gate node b to which the code word is assigned. Data is transferred by setting a route.

[0023]

As is apparent from the above performance comparison, the multistage code cube according to the present invention can significantly reduce the number of connection buses between nodes as compared with the conventional hypercube. For example, a (7,4) linear code that is a single error correction code is used for encoding, and a 7-dimensional hypercube is 8
Hierarchical networks (hereinafter referred to as 4-stage (7,4) code cubes) in which four nodes are collected and interleaved in a 4-stage hypercube have a total number of nodes of 2 (7+ (7-4)
(4-1)), that is, 2 to the 16th power, but the number of inter-node connection buses is 262,080, and the number of inter-node connection buses of a 16-dimensional hypercube having the same number of nodes It is about half of 524,288. Generally, the number of inter-node connection buses of a multi-stage code cube in which the number of nodes is 2 to the n-th power is m, where the number of dimensions of the basic hypercube is m, In the case of 8), p is approximated by (p / (p-1)) · (2 ^ n) · / 2 (the operator “^” represents exponentiation). Therefore, the number of inter-node connection buses of the multistage code cube is determined depending on the applied code, and the dimension number m of the basic hypercube and the unit p for assembling the basic hypercube depend on the applied code. The inter-node connection bus increases only in proportion to the number of nodes.

Further, the multiprocessor system using the multi-stage code cube according to the present invention makes it possible to construct a large-scale multiprocessor system at a low cost by greatly reducing the interprocessor connection network. Especially, when the system is expanded, only the processor corresponding to the gate node needs to be expanded, so that the system expandability is very excellent.

Further, according to the present invention, the basic hypercube can be mounted on one module, and the number of connecting lines between the modules can be greatly reduced when combining them to construct a larger network. For example, in the multistage (7,4) code cube in which the basic cube is a 7-dimensional hypercube, the number of connecting lines between modules is 67 when the number of nodes is 2 16 and this is 128 per chip.
This is a number that can be fully realized when a processor is installed. On the other hand, when the 16-dimensional hypercube is mounted as the 7-dimensional hypercube as the basic cube, the number of connecting lines between the modules is 1152, which is obvious.

Further, according to the present invention, the gate node unit of the multi-stage code cube has a multiprocessor configuration in which main memory is shared, so that the function of the gate node unit can be expanded only by adding a processor module board. As a result, the system of the present system can be expanded only by adding the processor board and the network board without changing the hardware of the old system.

When the multi-stage code cube according to the present invention is applied, the internode communication time can be reduced as compared with the hypercube. For example, in the above-described 4-stage (7, 4) code cube, the maximum distance between nodes is 13, which is smaller than 16 in the hypercube having the same number of processors. Also, the number of steps required for broadcasting for the entire system is reduced accordingly, and if the broadcast node is a node belonging to the uppermost hypercube, the number of steps is further reduced. For example, in the above example, if the broadcast node is the fourth stage hypercube, it can be executed in 10 steps.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A hierarchical network according to the present invention and a multiprocessor system using the same will be described below in detail with reference to the accompanying drawings.

In the present invention, an m-dimensional hypercube is used as a basic network, some nodes are selected from each of a plurality of basic networks (hereinafter referred to as a gate node), and a higher network is selected from the selected gate nodes. It is formed. A hierarchical network is configured by recursively repeating the same processing by using the upper network as a basic network of the next stage.

Generally, when a p-adic n-digit number is considered as a code consisting of an information digit and an error correction digit, it represents one point of the error correction code space. For example, when a binary number is given to each node of the m-dimensional hypercube as a node number, the node whose node number is a code word is selected as a gate node. Accordingly, each node can determine whether its own node is a gate node by determining whether its own node number is a code capable of correcting an error (hereinafter referred to as ECC). When the own note is a node other than the gate node (hereinafter referred to as a leaf node), it is possible to communicate with the gate node closest to the own node by obtaining the ECC closest to the own node number.

By applying the code theory when selecting the gate node in this way, the address of the gate node closest to the own node can be easily obtained, and the gate node can be uniformly selected from the basic network. The advantage is that A hierarchical network that uses code theory as a method for selecting a gate node will be referred to as a "multistage code network" below, and a system that uses a "hypercube" as a basic network will be referred to as a "multistage code cube".

FIG. 3 is a diagram conceptually showing the relationship between the stages of the multi-stage code cube. In the multi-stage code cube, a gate node (5) selected from the first-stage cube 20 that is a basic cube constructs a higher-order second-stage hypercube (5). The small squares in the figure indicate the leaf nodes 23 in the basic cube. Leaf nodes exist only in the first stage basic cube. The other squares represent the gate nodes in the first stage. All gate nodes are nodes of the second stage cube. In the second-stage cube, the gate node that is the node of the third-stage cube is selected from the nodes forming them. In this way, the gate node can be classified according to the number of stages of the cube that the node belongs to. Here, the gate nodes that are the nodes up to the second stage cube are called the first stage gate nodes, and the gate nodes that are the nodes up to the third stage cube are called the second stage gate nodes. In general, a gate node that is a node up to the nth stage cube is called a (n-1) th stage gate node.
In this figure, a first stage gate node 24, a second stage gate node 25, and a third stage gate node 26 are shown.

FIG. 1B shows the structure of a three-dimensional hypercube as a basic network of this embodiment. A three-dimensional hypercube is a network with eight nodes. In the following description of this embodiment, the three-dimensional hypercube is simply called a cube. A method of constructing a network having a hierarchical structure according to this embodiment, that is, a hierarchical network will be described below.

FIG. 1A shows an embodiment of the hierarchical network according to the present invention. In this embodiment, a two-layer hierarchical network is realized by using a three-dimensional hypercube. First, line up four cubes. In this example, the three-dimensional hypercube is called a basic cube. Each basic cube is given a cube number from 00 to 11. Next, from each of the basic cubes, two nodes having communication distances greater than 1 are selected (nodes indicated by black circles in (a) of FIG. 1). Finally, the selected eight nodes are connected to each other, that is, the corresponding nodes of each basic cube are connected to each other, and the nodes in the same basic cube are connected to each other. Make up. As described above, in this embodiment, the gate nodes selected from each basic network are connected so as to form an upper layer network, that is, a three-dimensional hypercube, and the gate nodes selected from the same basic network. For example, 00000 and 00111 are connected with a communication distance of 3 in the basic network, and a communication distance of 1 in the upper layer network.

A two-layer hierarchical network can be constructed by the above procedure. In the hierarchical network according to the present invention, gate nodes selected from the basic network are connected so as to form an upper layer network. As a result, the communication distance between the gate nodes selected from the same basic network becomes shorter than the communication distance in the basic network. As a result, in the hierarchical cube network according to the present invention, the number of connecting buses is smaller than that of the hypercube network having the same number of nodes, and the communication distance between the nodes in the basic hypercube is smaller than that.

FIG. 2 shows a case where a three-dimensional hypercube is used.
It is an example of a configuration of a hierarchical network of layers. In this embodiment, the hierarchical network shown in FIG. 4 is considered to be a basic network, that is, the second-layer three-dimensional hypercube is considered to be a basic cube, and the third-layer network is configured by the procedure described above. As is apparent from this embodiment, by repeatedly applying the hierarchical network construction method according to the present invention, it becomes possible to construct a large-scale hierarchical network having multiple hierarchies.

FIG. 6 shows an example using another code. In this example, a two-layer hierarchical network using a 7-dimensional hypercube is shown.

First, eight 7-dimensional hypercubes, which are the lowest layer networks, are arranged. In this example, the 7-dimensional hypercube is called a basic cube. In the basic cube shown in FIG. 6, a part of the connecting line of the 7-dimensional hypercube is shown. Cube numbers 0 to 7 are assigned to each basic cube. Each basic cube is 128 (= 2 ^ 7)
Has nodes. Next, 16 gate nodes are selected from the 128 nodes of each basic cube.
In FIG. 6, the selected gate node is shown. 128 gate nodes selected from 8 cubes (=
16 × 8), so these nodes are connected so as to form a new 7-dimensional hypercube to form a second layer network.

By repeating the same processing as in the example of the three-dimensional hypercube, a multilayer network can be constructed.

Next, a method of selecting a gate node suitable for constructing the hierarchical network system according to the present invention will be described.

First, the lowest network, which is the basis of the hierarchical network, is a p-adic n-dimensional network. Hereinafter, this network is referred to as a basic network. Each node of the basic network is given a node number (address) of p-adic m digits. For example, the three-dimensional hypercube shown in FIG. 2 (binary three-dimensional network)
Then, each node is given a binary node number of 0000 to 111. The node number given in this way is divided into upper r digits and lower k digits (r + k = m), and one lower k digit is uniquely assigned to each upper r digit number according to the error correction theory. A node having a uniquely assigned number as a node number is selected as a gate node. In the embodiment, the binary 3-digit node number is divided into the upper 1 digit and the lower 2 digits. When the upper 1 digit is 0, the lower 2 digits are 00, and when the upper 1 digit is 1, the lower 2 Digits 11 are uniquely assigned. The node having the node numbers 000 and 111 thus determined is selected as the gate node.

The above procedure makes it possible to uniformly select gate nodes from the basic network. Further, in the above method, as a method of determining the upper digit data and the lower digit data corresponding to the upper digit data, the upper digit data is regarded as an information bit and the lower digit data is regarded as an error detection / correction bit, and There is a method of uniquely associating error detection / correction bits corresponding to bits. In the above example, binary majority code (1 digit information bit, error detection /
The gate node was selected using (correction bit 2 digits).

The number of dimensions of the basic cube as the basic network and the coding method used when selecting the gate node may be the same or different between layers. In the above example, for ease of explanation, it is assumed that the number of dimensions of the basic cube between layers and the encoding method are the same.

FIG. 7 is a diagram for explaining how to assign node numbers in a hierarchical network using a 7-dimensional hypercube, that is, a multi-stage (7, 4) code cube network.
First, as shown in FIG. 7A, a 7-bit binary in-cube node number is assigned to each node according to the physical node position in the basic cube. In the figure, the positions of the nodes on the x-axis, y-axis, and z-axis in the cube are 3
It is represented by bits, 3 bits, and 1 bit, and the node numbers in the basic cube are given by arranging in order from the lower order.
Since the network shown in the present embodiment is configured by connecting eight basic cubes, cube numbers 0 to 7 are given to each basic cube as shown in FIG. 7B. By assigning this cube number above the node number, a unique node number in the basic network is defined. That is, the node in the basic network can be identified by the format shown in FIG. In order to make the following description clearer, "G" or "L" is added to the head depending on whether the node is a gate node or a leaf node.

When selecting a gate node, one error correction code consisting of 7 bits in total, 4 bits for information bits and 3 bits for correction bits is used. Hereinafter, this code is referred to as a (7, 4) code (the code length is 7 bits and the information bits are 4 bits). The non-error codewords of the (7,4) code are 16 shown below.

Information bit Corrected bit Information bit Corrected bit A hierarchical network, "multi-stage (7,4) code cube" is configured by using a node whose number matches a non-error code as a gate node. The network shown in FIG. 6 is a two-stage (7,4) code cube.

In the following, referring to FIG. 6, two stages (7,
4) Explain the code cube. The gate node G000 of the cube # 0 is the leaf node L0 in the cube # 0.
01, L002, L004, L010, L020, L0
40, L080. On the other hand, in the second-stage 7-dimensional hypercube, the gate node G0 in cube # 0 is
13, G025, G046, and G087, respectively. In addition, gate nodes G100 and G20 of other cubes
0, connected to G400. In this way, in the second-stage network, the gate nodes are respectively connected to the four gate nodes in the same basic cube and the gate nodes at corresponding positions in the three other cubes.
That is, each gate node has seven connection lines in the basic cube and seven connection lines in the second stage network, for a total of 14 connection lines.

FIG. 8 is a diagram showing in detail the connection state of the gate nodes in the basic cube. When the gate nodes are selected using the (7,4) code, the minimum distance between the gate nodes in the same cube is 3. Therefore, any leaf node in the base cube always has a gate node at a distance of one. For example, gate node G00
0 and G013 have a distance of 3 in the basic cube. Between these two gate nodes, L001, L
Although there are leaf nodes such as 002 and L003, a gate node exists at a distance of 1. That is, L001
And L002 from the gate node G000 to L003.
Is 1 from the gate node G013. Therefore, it can be said that the communication distance from any node in the (7, 4) code cube to the gate node is within 1. Further, the gate nodes having a communication distance of 3 are directly connected. Therefore, one communication between the gate nodes via the second stage network corresponds to three communication in the basic cube.

An efficient multiprocessor system can be constructed by applying the multistage code cube as an interprocessor connection network of the multiprocessor system. For this purpose, each node of the multistage code cube includes at least one information processing device (processor), and the connection between the nodes becomes a communication line. Here, the number of dimensions of the basic cube is m, and the number of nodes of the entire system is n of 2.
It is assumed that 2 r-th gate nodes are selected from each basic cube. Leaf nodes can be constructed by using exactly the same method as each node in the system using the existing hypercube. That is, it can be configured by a processor module having m communication ports for connection between nodes in the m-dimensional hypercube. The difference between the multistage code cube multiprocessor system and the conventional hypercube multiprocessor system is the gate node.

FIG. 9 is a block diagram showing in detail the configuration of a multiprocessor system to which the two-stage code cube shown in FIG. 4 is applied. Cube 0 through Cube 3 are first
It is a multi-tiered cube, where L02 to L36 are leaf nodes and consist of processor modules.

As shown in FIG. 10A, the processor module comprises a main memory 71, a processor 72, a network router, and a network channel for connection within the same cube.
The gate nodes G00 to G37 are composed of a multiprocessor module as shown in FIG. Figure 10
Details of (b) are as shown in FIG. 11 below. Cubes 0 to 3 are first-stage cubes, and a cube composed of gate nodes G00 to G37 is a second-stage cube. Communication lines between leaf nodes are indicated by thin lines,
Communication lines between the gate nodes are indicated by thick lines.

FIG. 12 is a table showing processor numbers, cube numbers, in-cube numbers, and next-stage cube numbers in each node. The cube number of the next stage is 0 (00
0) and 7 (111) only. When four next-stage cubes are further provided, the next-stage cube number is assigned.

FIG. 11 shows an embodiment of a multiprocessor module used as a gate node in a multistage code cube multiprocessor system. This embodiment shows the configuration of the first stage gate node. Gate node 4
0 is the processor 43 on the first stage cube side, the processor 47 on the second stage side 47, the network router (NR) 46 on the first stage cube side, and the network router 5 on the second stage cube side.
0 and the inter-cube connection bus 42 in the node connecting them and the shared memory 41 between these processors.

The first stage cube in the gate node, that is, the processor 43 belonging to the first stage hierarchy is the information processing device 45.
The processor 47 belonging to the second stage cube in the gate node, that is, the second stage hierarchy has the information processing device 49 and the private memory 48.

Data transfers within the same cube are transferred via private memory within the processor. On the other hand, transfer between cubes in different layers is performed via the shared memory 41. By implementing the gate node with the multiprocessor module in this way, it is possible to easily expand the gate node to a larger number of stages. For example, when the first-stage gate node of this embodiment is expanded to the second-stage gate node, it is only necessary to connect the processors belonging to the third-stage cube to the inter-cube connection bus 42 in the node.

Although the gate node in this embodiment has a main memory shared tightly coupled multiprocessor configuration by bus coupling, it is of course possible to realize the gate node by a distributed memory type multiprocessor configuration having no shared memory. .. If the scalability is sacrificed, the gate node may be separately designed as a single processor using a processor different from the leaf node and a network router. In this case, the processor used for the gate node is required to have an integral multiple of network connection channels of the processor used for the leaf node depending on the number of stages of gate nodes.

13A and 13B show another embodiment of the node unit. In this embodiment, the leaf node is composed of a memory 71, a processor 72 for processing data, and a network router (NR) 73 for controlling data communication between the nodes, as shown in FIG.
As shown in FIG. 13B, the gate node has a plurality of network routers 46 and 50 that control the flow of messages in the networks of the respective layers, and controls the networks of the respective layers. The data transfer across the layers is performed by the bus 42 connecting the network routers.
Through. In the present embodiment, data is transferred only through the network routers 46 and 50 without being taken into the processor at the nodes on the transfer path. Therefore, the operation of the processor existing on the transfer path is not affected. Moreover, if the number of network routers is increased, it is possible to cope with network expansion.

FIG. 14B shows still another embodiment of the gate node. In the network of the present invention, a plurality of layers of networks are concentrated in the gate node. When the m-dimensional hypercube is used as the basic cube, the total number of links output from the i-th stage gate node is (m × (i
+1)) (i ≧ 0: i = 0 is a leaf node).
The network router that controls these links is realized by a crossbar switch that has (m × (i + 1) +1) input / output lines, considering that one more link is required to take in the data in its own processor. it can. Figure 1
4 (a) and 4 (b) show an example of the configuration of the leaf and gate nodes of this network using the three-dimensional hypercube as a basic cube. In this embodiment, a module having a processor, a first stage network router (4 × 4 crossbar switch), and an expansion port selector for connecting to an upper layer network is used as a basic module. As the leaf node, the basic module is used as it is as shown in FIG. On the other hand, the gate node is composed of a basic module and an expansion switch module 48, as shown in FIG. In this embodiment, the 7 × 7 crossbar switch (4) necessary for configuring the network router of the first-stage gate node is (4
+3) × (4 + 3) divided into one 4 × 4 crossbar,
One 4x3 crossbar, one 3x4 crossbar, 3x3
, And each output signal line of each crossbar is connected by a selector. Among these switches, the basic module has a built-in 4 × 4 crossbar, so that the extension switch module including the remaining three crossbars and two selectors is connected to the extension port of the basic module.

In this embodiment, since the basic module can be commonly used by all the nodes, the production cost of the basic module can be reduced. In addition, the pin bottleneck of the network router can be eliminated by dividing and realizing the network router necessary for configuring the gate node. Generally px
Crossbar switch can be divided into (q + r) × (s × t) (q + r = s + t = p), q × s, q ×
It can be realized by four crossbars of t, r × s, and r × t and a selector connecting them. It is obvious that this method of dividing the crossbar switch can be applied to other than the case of configuring the network router.

Next, the data transfer procedure according to the present invention will be described with reference to FIG.
Will be described. At the time of data transfer, the source node and the destination node are set in the message. Data transfer is performed by a process 80 for determining whether the source node and the destination node are in the same network, a process 86 for transferring the data by a transfer route in the network when the source node and the destination node are in the same network, and If not, a process 81 for obtaining the gate node p closest to the original node and a process 82 for obtaining the gate node q closest to the destination node.
A process 83 for transferring data from the source node to the gate node p using the internal network, a process 84 for transferring data from the gate node p to the gate node q via the network in the upper layer, and a process 84 for transferring data from the gate node q. Processing 85 for transferring data to the nation node using the internal network including the same. The process of transferring data from the gate node p to the gate node q is performed by recursively calling this procedure.

FIG. 16 is a diagram for explaining data transfer between nodes in a multi-stage code cube. This figure shows an example of the case where the transfer distance becomes the maximum in the inter-node communication in the 3-stage (7, 4) code cube.

The numbers of the respective nodes are given as follows. Referring to FIG. 7B, the three-stage (7,4) code cube is a cube formed by using a seven-dimensional cube as a basic cube and constructing them in three stages. 81 nodes
92 (128 × 64), and the node number can be represented by 13 bits. The lower 7 bits of the node number indicate the address in the basic cube. If a (7,4) code in which the upper 4 bits are information bits and the lower 3 bits are correction bits is selected, 16 code words whose upper 4 bits are 0000 to 1111 can be selected. ..

Therefore, it is possible to specify the node of the 7-dimensional cube of the second stage by adding 3 bits to the higher order of these upper 4 bits. The number of 7-dimensional cubes in the second stage is 8, and the number of nodes, that is, the number of gate nodes in the first stage is 1024 (16 × 64 = 128 × 8).
Is. Extension to the third stage cube is also realized by using a similar method. The number of 7-dimensional cubes in the third stage is 1, and the number of nodes, that is, the number of gate nodes in the second stage is 128 (16 × 8).

According to the above procedure, the upper 7 bits of the node number are the address in the third stage cube, the upper 4 bits to the upper 10 bits are the address in the second stage cube, and the upper 7 bits to the lowest bit are It becomes the address in the first stage cube. The address is assigned, for example, as follows.

Addresses of 7 bits (upper 7 bits to least significant bit) are assigned to the nodes of the first stage cube. The upper 4 bits of the 7 bits are information bits and the lower 3 bits are error correction. It is a bit for. 7 bits of the non-error code are assigned to the gate node.
7 bits from the upper 4 bits to the upper 10 bits are assigned as an address to the node (first stage gate node) of each basic cube of the second stage, but the address is assigned to the node of each basic cube of the first stage. The remaining 7 bits (upper 4 bits to upper 6th bit) are assigned while leaving the already assigned 3 bits from the upper 7th bit to the upper 10th bit as addresses. Addresses are assigned to the nodes of the cubes of the third stage (gate nodes of the second stage) in the same manner as the addresses are assigned in the second stage.

The 13 bits thus determined become the node number.

The transfer procedure will be described below with reference to the node numbers thus assigned.

FIG. 16 shows a route for transferring data from the leaf node 55 (node number 0000000001111) to the leaf node 60 (node number 1111111110000). The leaf node 55 belongs to the 0th cube in the first stage cube group 61, and the leaf node 60 is the 63rd cube in the first stage cube group 61.
Belongs to the second cube. Therefore, when communicating between these nodes, it is necessary to use a network between higher order cubes.

Therefore, data must first be transferred to the gate node closest to the leaf node 55. Since the gate nodes of the basic cube are selected by using one error correction code such that the distance between them is 3, the gate node always exists at a distance of 1 from each leaf node. A gate node for any node in each stage of the hypercube considers the binary address of that node in the cube as a code and corrects that address using the error correction method established in code theory, It can be easily requested.

Since the address of the leaf node 55 in the first-stage cube is (0001111), the error is corrected to (0001011) and the gate node 56 (node number 000000000111) is reached. In the same manner, the gate node corresponding to the transfer destination leaf node 60 can obtain the gate node 59 (node number 1111111110100).

If the gate nodes thus obtained are in the same cube, they can be transferred using the hypercube of the hierarchy, but in this example, the gate node 56 is different in the second stage cube group 62. Cube 65
(Cube number 0) and 66 (cube number 7). Therefore, it is understood that it is necessary to use a higher order cube when transferring between these nodes.

On the other hand, the gate nodes 56 and 59 have the first
It is a gate node of a stage, and is a leaf node in the second stage cube. Therefore, in order to connect to the third stage cube, it is necessary to transfer the data to the gate node of the second stage cube of these nodes, that is, the second stage gate node.

Since the address of the gate node 56 in the second stage cube 65 is (0000001), error correction of this address results in (0000000), and the second stage gate node 57 for transferring data can be obtained. .. Therefore, from the first stage gate node 56,
Second stage gate node 57 (node number 00000000
Data is transmitted to Similarly, the second-stage gate node 58 (node number 1111111111111) is easily obtained as the gate node corresponding to the node 59 which is the data transfer destination node in the second-stage cube group.

These second stage gate nodes are the nodes in the third stage cube 63, and the addresses in the cube are 0000000 and 1111111 respectively. Therefore, if the transfer is performed 7 times in the third stage cube, communication is performed. can do.

From the above, data transfer from the leaf node 55 to the leaf node 60 is performed by the gate nodes 56, 5
It is transferred to the third stage cube via 7, and transferred to the gate node 58 by performing transfer seven times.
Then, it is transferred to the leaf node 60 via the gate node 59. The total number of steps of these transfers is 11 when the transfer between nodes is 1. This transfer distance is the maximum transfer distance in the 3-stage (7, 4) code cube.
In the case of a hypercube having the same number of nodes, the maximum transfer distance is 13, so the multistage code cube obviously has a shorter transfer distance.

Below, the evaluation results regarding the hardware amount and transfer distance of the conventional network and the multi-stage code cube will be described. Evaluation target: Hypercube, 8
A multi-stage majority code cube (hereinafter, (3, 3) with a base n-dimensional hyper crossbar (HXB), a two-dimensional torus network, a binary tree network, and a three-dimensional hypercube
1) CCN) and a multi-stage (7, 4) code cube (hereinafter referred to as (7, 4) CCN) using a 7-dimensional hypercube as a basic cube. The structure of the above network other than the multi-stage code cube is shown below.

(1) Hypercube FIG. 17A shows the structure of a five-dimensional hypercube. n
Each node of the dimensional hypercube has n links,
It is connected to n nodes with a Hamming distance of 1. The Hamming distance is the number of bits that differ when the node number is represented by a binary number. For example, the node numbers are 20 (= 00010100) and 54 (= 00110111).
The Hamming distance of 0) is 2.

(2) Hypercrossbar FIG. 17B shows the structure of an 8 × 8 two-dimensional hypercrossbar. Each node has two links, horizontal 8x8
It is connected to the crossbar and the vertical 8 × 8 crossbar switch. In general, d of p ₁ × p ₂ × ... × pd configuration
The dimensional hypercrossbar is a network in which pi nodes are connected in the direction i in each dimension by a perfect junction network (crossbar), and each node has d links.

(3) Torus Network FIG. 17C shows the structure of a two-dimensional torus network having an 8 × 8 structure. This network has eight nodes horizontally,
It is a network in which eight nodes are connected in a ring shape in the vertical direction. A d-dimensional torus network having a structure of p ₁ × p ₂ × ... × pd is a network in which pi nodes are connected in a ring shape in each dimension i direction. Each node is connected by 2d links.

(4) Binary tree network FIG. 17D shows the configuration of the binary tree network. The binary tree network has 2n nodes and 2n-1 network routers. Each node has one link and the network router has three links.

Here, the evaluation results regarding the average number of links per node of the network and the average number of links between modules will be described.

(1) Average number of links per node The number of links connecting each node represents the number of pins and wiring amount of each LSI at the time of mounting. LSIs that make up a network
Then, it often becomes a pin count neck. Therefore, the network hardware costs are compared by the average number of links per processor. One link is shared by two nodes. Therefore, the average number of links is twice the total number of links in the network divided by the number of nodes.
FIG. 18A is a table showing the average number of links of various networks. 18B is a graph in which the horizontal axis represents the logarithm of the number of nodes in the network and the vertical axis represents the average number of links per processor. From this result, the torus network, the binary tree network, the (3,1) CCN has approximately 4 links / node regardless of the number of nodes, and the (7,4) CCN has about 8 links / node. On the other hand, in the hypercube and the hypercrossbar, the number of links increases in proportion to the logarithm of the number of nodes.

From the above results, it is understood that, considering the hardware implementation, the hierarchical network, the binary tree network, and the torus network according to the present invention are desirable when connecting a large number of nodes.

(2) Number of inter-module links When considering a parallel computer in which a plurality of processors are integrated in one chip or one module, the number of pins that can be output from one chip or module becomes a problem. Therefore, in a large scale parallel computer, it is essential to reduce the number of links between modules. For example, the number of links between modules when eight processors were integrated in one module was evaluated for each network. FIG. 19A is a table showing the calculation result.
In (b), the horizontal axis represents the logarithm of the number of nodes in the network,
The vertical axis is a graph showing the number of links between modules. From this result, it can be seen that the number of inter-module links of the hypercube and the hypercrossbar is large, and the hierarchical network, the binary tree network and the torus network according to the present invention can be realized with a relatively small number of links.

(3) Average Transfer Distance When performing random communication between all nodes to all nodes, message collision frequently occurs. In this case, the transfer time is greatly affected by the number of message collisions. The number of message collisions increases as the transfer distance increases. Therefore, the data transfer performance of the networks can be evaluated by comparing the average transfer distances of various networks. That is, it can be said that the smaller the average transfer distance, the better the transfer performance of the network.

In FIG. 20, the horizontal axis represents the logarithm of the number of nodes in the network, and the vertical axis represents the average transfer distance of the network. From the figure, it can be seen that the transfer distance of the hypercube is the shortest. If the transfer distance of Hypercube is 1, 64K
In the node configuration, the (7,4) CCN is 1.1 times that of the hypercube, the hypercrossbar is 1.2 times, and the (3,1) CCN is
N is 1.6 times, binary tree network is 3.8 times, and torus network is 16 times.
This is twice the average transfer distance. From these results, hypercube, (7,4) CCN, hypercrossbar, (3,1) C
It can be seen that the CN has high network transfer performance.

Next, the performance when various networks were constructed with the hardware cost kept constant, that is, the cost performance of the networks was compared. Assuming that the hardware cost of the network depends on the cost of the links of the network, if the hardware cost is fixed, the total number of links of the network and the transfer rate (throughput) of each link are in inverse proportion.

Therefore, when the distance of each link is normalized with the network cost kept constant, the normalized transfer rate Lt of each link is

[0089]

## EQU1 ## Lt = 1 / (average number of links) (1) Therefore, the normalized average transfer distance ND
Is

[0090]

ND = (average transfer distance) / Lt = (average transfer distance) × (average number of links) (2)

In FIG. 21, the horizontal axis shows the logarithm of the number of processors, and the vertical axis shows the normalized average distance. From this result, it can be seen that the multistage code cube has the smallest normalized average transfer distance as compared with the conventional network. For example, 6
When the normalized average transfer distance of (3,1) CCN in the 4K node configuration is 1, (7,4) CCN is 3.3,
The hypercrossbar is 5.1, the binary tree network is 5.5, the hypercube is 5.9, and the torus network is 23.6, and the cost performance of the hierarchical network according to the present invention is extremely high compared to other networks. I understand.

From the above evaluation, by using the hierarchical network according to the present invention, a network with low hardware cost and high transfer performance can be realized.

[0093]

As is apparent from the above description, the present invention provides a completely new hierarchical network topology,
In addition, the topology has a hypercube inside, and in a large-scale multiprocessor system with tens to millions of processors or more, the number of connection lines between nodes, expandability, and the number of connection lines between modules. , Has a property superior to that of the hypercube in any of the data transfer distances.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of a multi-stage code cube of the present invention.

FIG. 2 is a diagram showing another embodiment of the multistage code cube of the present invention.

FIG. 3 is a diagram conceptually showing the relationship of each stage of a multi-stage code cube.

FIG. 4 is a diagram illustrating a hypercube.

FIG. 5 is a diagram illustrating a hypercube.

FIG. 6 is a diagram showing a hierarchical network in which 7-dimensional hypercubes according to an embodiment of the present invention are stacked in two stages.

FIG. 7 is a diagram for explaining how to assign node numbers to a multi-stage (7,4) code cube.

FIG. 8 is a diagram showing a connection state of gate nodes in a basic cube of a multi-stage (7,4) code cube.

9 is a block diagram showing a configuration of a multistage code cube multiprocessor system which is a multiprocessor system to which the multistage code cube shown in FIG. 3 is applied.

FIG. 10 is a diagram of an embodiment of a leaf node and a gate node in a multiprocessor system using a multi-stage code cube.

FIG. 11 is a configuration diagram of an embodiment of a gate node in a multiprocessor system using a multi-stage code cube.

FIG. 12 is a processor number of a processor unit placed in each node of FIG. 9, a cube number, an in-cube number,
It is the figure which showed the number in the next-stage cube as a table.

FIG. 13 is a configuration diagram of another embodiment of a half node and a gate node in a multiprocessor system using a multi-stage code cube.

FIG. 14 is a configuration diagram of another embodiment of a half node and a gate node in a multiprocessor system using a multi-stage code cube.

FIG. 15 is a diagram illustrating a data transfer procedure in a network of multistage code cubes.

FIG. 16 is a diagram illustrating data transfer between nodes in a network of multi-stage code cubes.

FIG. 17 is a diagram showing a configuration of a conventional network.

FIG. 18 is a table and a graph showing a comparison of the number of links per node between the multi-stage code cube according to the present invention and the conventional network.

FIG. 19 is a table and a graph showing a comparison of the number of inter-module connection links between the multi-stage code cube according to the present invention and the conventional network.

FIG. 20 is a graph showing a comparison of average transfer distances between a multi-stage code cube according to the present invention and a conventional network.

FIG. 21 is a graph showing a comparison of normalized average transfer distance between a multi-stage code cube according to the present invention and a conventional network.

Claims (13)

[Claims] 1. A hierarchical network comprising a plurality of lower layer networks having a plurality of nodes and an upper layer network configured by hierarchically connecting these, wherein each of the plurality of lower layer networks having the plurality of nodes A plurality of gate nodes are selected to be nodes in an upper layer network, and among the selected gate nodes, the selected gate nodes belonging to the same lower layer network communicate with each other in a lower layer network where the communication distance between the gate nodes is lower. A hierarchical network, characterized in that the selected gate nodes are connected to each other so as to be shorter than the distance, and the selected gate nodes that do not belong to the same lower layer network are connected to each other so that they can communicate with each other. 2. The hierarchical network according to claim 1, wherein the network forming the i-th layer is a p _i -adic n _i -dimensional network. 3. A plurality of underlying p-adic n having a plurality of nodes
It is a hierarchical network consisting of a dimensional network and an upper layer p-adic m-dimensional network configured by hierarchically connecting these, wherein each node of a plurality of lower layer p-adic n-dimensional networks having a plurality of nodes is Given a p-adic n-digit node number that can be uniquely determined in the lower layer network, select a plurality of gate nodes corresponding to the node number corresponding to the code word when the node number is regarded as a code, and select from multiple lower layer networks. A hierarchical network, characterized in that a plurality of selected gate nodes are interconnected as nodes of a higher layer p-adic m-dimensional network. 4. The hierarchical network according to claim 1, wherein the network forming the i-th layer is composed of n _i -dimensional hypercubes. 5. When selecting a gate node belonging to an upper layer network in the hierarchical network according to claim 3 or 4, t is selected from nodes belonging to each lower layer network.
It is characterized in that a plurality of gate nodes corresponding to a node number corresponding to a code word of a multiple error correction code are selected, and a plurality of gate nodes selected from a plurality of lower layer networks are connected to each other as nodes in a higher layer network. Hierarchical network to do. 6. The hierarchical network according to claim 1, wherein a processor module is arranged in each node of the network, and a connection line of the network is configured as a communication line. Type network multiprocessor system. 7. The hierarchical network multiprocessor system according to claim 6, wherein the multiprocessor system including the lower layer network is constituted by one chip or one module. 8. The hierarchical network multiprocessor system according to claim 6, wherein a processor module constituting a node other than a gate node is a private memory, a processor connected to the private memory, the processor and the lower layer network. A processor module that includes a network router connected to each other, and that configures each gate node includes a bus, a shared memory connected to the bus, two or more processors connected to the bus, and the plurality of processors. And a plurality of network routers connected to a network of each layer, a hierarchical network multiprocessor system. 9. The hierarchical network multiprocessor system according to claim 6, wherein the processor module constituting a node other than the gate node comprises a first private memory, and a processor connected to the first private memory. A processor module that includes the processor and a network router connected to the lower layer network, and that configures each gate node includes a second private memory, a processor connected to the second private memory, and a processor connected to the processor. Hierarchical network multiprocessor system, comprising: a network bus and at least two network routers connected to the network of each layer and the bus. 10. The hierarchical network multiprocessor system according to claim 6, wherein a processor module constituting a node other than the gate node comprises a processor, an input port and an output port connected to the lowest layer network, and an expansion port. A network router connected to the processor, the input port, the output port, and the expansion port for selectively connecting the input from the input port and the output from the processor to the input to the processor and the output port And a selector for supplying one of an output from the network router and an input from the expansion port to a processor, and the processor module forming each gate node includes a node other than the gate note. The processor modules to configure and the previous Expansion port, the upper layer network, connected to the lower layer network, the input from the upper layer network, the input from the lower layer network, the output of the processor module from the expansion port, the input to the expansion port of the processor module, the An output to the upper layer network, an extension network router for selectively connecting to the output to the lower layer network, an input from the processor module, an output from the extension network router, an output of the upper layer network and the lower layer network A hierarchical network multiprocessor system including selection means for selectively supplying to the output of the. 11. The hierarchical network multiprocessor system according to claim 6, wherein when data is transferred from a processor arranged in a node, the data is transferred to a node belonging to the same lower layer network by the shortest path in the lower layer network. A hierarchical network characterized by being configured such that data is transferred to a node belonging to another network, data is transferred to a node connected to a higher-layer network, and data is transferred via a higher-level network. Multiprocessor system. 12. The hierarchical network multiprocessor system according to claim 11, wherein a path from a node a other than a node connecting the lower layer network and the upper layer network to a node b connecting the lower layer network and the upper layer network. Regards the node number in the lower layer network given to the node a as a code, corrects this by the error correction means in the t-fold error correction code, obtains the code word of the code, and assigns the obtained code word. A hierarchical network multiprocessor system having means for providing a path to the node b. 13. A multiprocessor system in which a plurality of processor modules are connected by a network, wherein the processor module includes a processor, an input port and an output port connected to the network, an expansion port, the processor, the input port, A network router connected to the output port and the expansion port for selectively connecting an input from the input port and an output from the processor to an input to the processor and the output port, and an output from the network router And a selector for supplying one of an input from the expansion port to a processor, the processor module includes at least one processor, a plurality of network input / output ports, a network router, and Network A multiprocessor system having an expansion port and a selector for expanding the input / output port of the network.