IE20070113A1 - An optical communication network - Google Patents

Abstract

An optical interconnection network has a plurality of network nodes (70) interconnected by optical links (76-79). The network nodes (70) each receive a global clock and switch repeatedly with a fixed periodicity their routing states according to the global clock. Each network node (70) selects a time slot, a wavelength, and an initial optical link for transmitting a packet in dependence upon the desired path for the packet through the network, and outputs the packet in the selected time slot, wavelength and link. The network topology is a torus of the Manhattan street type. Each network node (70) comprises a plurality of channel nodes (73a) for each link, a wavelength division demultiplexer (72a), a combiner (74a), and an optical space switch (75). The optical space switch (75) has a 2x2 configuration. The optical space switch (75) has a wavelength converter (42) per link feeding a passive wavelength routing device (41).

Description

The invention relates to communication networks, such as for internal processor-toprocessor communication within a supercomputer.

Prior Art Discussion Supercomputers of increasingly higher processing power are required for work on some of the most demanding problems in science and engineering. To achieve economic scaling in the design of supercomputers, a current trend is towards the use of large numbers of processors each of relatively modest individual performance and cost. In some supercomputers the processing nodes are interconnected so as to allow parallel processing with the aim of achieving very high overall performance (as measured, for example, in floating-point operations per second, FLOPs). Increasingly the critical system that determines the overall supercomputer performance is the interconnection network and this has been widely acknowledged as being a challenge in supercomputer hardware.

A known approach to packet routing is called “clockwork routing”, or “directed trail routing” (Chevalier et al, 1998; Cotter, 2004, Chevalier et al, 2000). This approach overcomes the need for a central network controller, thus reducing latency, whilst also eliminating the need for contention resolution at transit network nodes, thus reducing latency, equipment complexity and cost. The approach also avoids the use of a switch core containing large numbers of active devices. Clockwork routing consists of selecting a time slot for a packet in dependence upon the desired path for the packet through the network, outputting the packet onto the network at a source node in the selected time slot, « 070 1 1 5 «Ό 7 Ο 1 13 and switching repeatedly with a fixed periodicity the routing states of a multiplicity of nodes in the network. The correct combination of time slot selection at the source and fixed periodic switching of the transit nodes results in delivery of the packets to their destinations without the need for contention resolution at the transit nodes.

In more detail, consider the network shown in Fig. 1, which is an nxn Manhattan street network (so called because there are // rows and // columns, and the row (2) and column (3) links alternate in direction, similar to the alternating one-way traffic directions in the streets and avenues in the Manhattan district of New York city). At the intersection of a row and column is located a network node, here called a ‘torus node’ (1). The torus nodes are labelled (r,c), where r is the row number (/-0,1,.. .,//-1) and c is the column number (c=0,1,...,//-1). In the Manhattan street network each row and column is connected back on itself, as shown in Fig. 1, so that topologically the network can be considered to be located on the surface of a torus. In the example shown, n=4. Each of the torus nodes contains a 2x2 cross-bar space switch, as shown in Fig. 2. When the switch is in the cross state (Fig. 2a), the row input is connected to the row output and the column input is connected to the column output; whereas when the switch is in the bar state (Fig. 2b), the row input is connected to the column output and the column input is connected to the row output. To allow clockwork routing, the switches throughout the network operate in synchronism with a global clock. According to this global clock, time is divided into a continuous sequence of fixed, equal-length time slots. On every link of the network, each slot can contain a packet of information which is being transmitted between a pair of nodes of the network. Time slots are labelled sequentially 0,1,2,...,//-1. Time slot j is immediately followed by time slot j+1 (for/=0,1,2,... ,//-2) or by time slot 0 in the case of /=//-1; thus the labelling of time slots is continuous, as shown diagrammatically in Fig. 3. At any torus node, in time slots 1,2,. ..,//-1 the switch is in the cross state, whereas in time slot 0 the switch is in the bar position. Thus all of the switches in the network operate in time synchronism. In practice, each time slot may incorporate (usually at the beginning or end of the time slot) a short time period, known as a guard band; the switches are permitted to operate (i.e. change state from cross to bar, or vice versa) only during guard « 070 113 ΙΕο 70113 bands, and also during the guard bands no information is transmitted, thus avoiding the possibility of information being corrupted when a switch is operated. The physical length of each link joining adjacent torus nodes is chosen so that the time of flight of signals between adjacent nodes is equal to kn+\ time slots, where k is zero or any positive integer, and, as previously, n denotes the size of the nxn Manhattan street network. Thus a packet in time slot j at a network node, if transmitted onwards to an adjacent torus node, will arrive at that node in time slot (/+1) mod n. Now to effect clockwork routing of a packet starting from a particular network node (called the source) to a chosen destination node, the source should transmit the packet on a selected output port and time slot. To give an example, suppose the torus node (2, 0) is the source of a packet which is to be transmitted to a destination node (3, 2). In this case, the source could transmit the packet on the row output in a time slot labelled 2. This packet will travel to the adjacent node (2, 1) and arrive there in a time slot labelled 3.

According to Fig. 3, the switch at node (2, 1) will be in the cross position in time slot 3, and therefore the packet will continue onwards along the row. The packet therefore travels onwards to the next adjacent node (2, 2) and arrives there in a time slot labelled 0. According to Fig. 3, the switch at node (2, 2) will be in the bar position in time slot 0, and therefore the packet will be automatically diverted onto the column for onwards transmission. The automatic clockwork routing continues as follows: node (2, 2) in time slot 0; node (1, 2) in time slot 1; node (0, 2) in time slot 2; and finally the destination node (3, 2) in time slot 3. Alternatively, the source could transmit the packet on the column output in a time slot labelled 1, in which case the clockwork routing is as follows: node (2, 0) in time slot 1; node (1, 0) in time slot 2; node (0, 0) in time slot 3; node (3, 0) in time slot 0 (at which point the packet is diverted automatically to the row); node (3, 3) in time slot 1; and finally the destination node (3, 2) in time slot 2. Notice, in this case the alternative routing is longer than the first. At each node encountered by the packet, the packet destination address is compared with the address of the current node; if there is an address match, the destination has been reached and the packet will be removed from the network, whereas if the addresses do not match the packet will be allowed to continue IS 0 7 Ο 113 ΙΕπ7 Ο 1 13 uninterrupted. Notice that at each node, this simple address match is the only information processing needed to ensure the packet arrives correctly at its destination. All the routing selection is performed at the source, by the initial choice of outward link and time slot.

The method of clockwork routing suffers from a limitation in the scaling of the size of the network. For an nxn toroidal network, the logical diameter of the network (the largest minimal number of hops between any pair of nodes) is 2(n-l), whilst the number of nodes is N = n2. Therefore a limitation of this architecture is that increasing N necessarily increases the average number of hops (scaling linearly with n), resulting in both reduced throughput (scaling as 1/n) and increased latency.

The invention is directed towards providing a network with the advantages of clockwork routing including reduced latency and complexity as compared to other methods, whilst allowing a higher degree of bandwidth scaling without need for a multi-stage architecture.

REFERENCES US6,714,552 (March 30, 2004), D. Cotter, “Communications Network” “A new packet routing strategy for ultrafast photonic networks”, F Chevalier, D Cotter and D Harle, IEEE 1998 Global Telecommunications Conference (Globecom'98), (Sydney, Australia, 1998).

F Chevalier, “Performance evaluation of the clockwork routing scheme in optical packet switching networks”, Ph.D. thesis, University of Strathclyde, UK (2000).

The invention is directed towards providing an improved communications network. ιε ο 7 Ο 11 3 »070 1 1 3 -5SUMMARY OF THE INVENTION According to the invention, there is provided an optical interconnection network having a plurality of network nodes interconnected by optical links: wherein the network nodes each comprise means for receiving a global clock and for switching repeatedly with a fixed periodicity their routing states according to the global clock; and wherein each network node comprises means for selecting a time slot, a wavelength, and an initial optical link for transmitting a packet in dependence upon the desired path for the packet through the network, and for outputting the packet in the selected time slot, wavelength and link.

In one embodiment, the network topology is a torus.

In one embodiment, the network topology is of the Manhattan street type.

In one embodiment, the means for selecting a wavelength for a packet includes means for switching between a plurality of lasers.

In one embodiment, each network node comprises a plurality of channel nodes for each link, a wavelength division demultiplexer, a combiner, and an optical space switch.

In one embodiment, the optical space switch has a 2x2 configuration.

In one embodiment, the optical space switch comprises a wavelength converter per link feeding a passive wavelength routing device. Κ 070 11$ «070 113 -6Ιη one embodiment, each routing device comprises a combiner and a wavelength division demultiplexer.

In one embodiment, the routing device comprises an arrayed wavelength router.

In another embodiment, the routing device comprises a wavelength channel interleaver.

In a further embodiment, each channel node comprises a processor, and a buffer connected to a transmission wavelength generator.

In one embodiment, the processor performs look-ups to a routing table.

In a further embodiment, the channel node buffer comprises means for determining a destination network node address of a packet.

In one embodiment, the transmission wavelength generator comprises: an interface to receive data in electrical form; an interface to the optical node processor; and a switch to transfer the data signal to one of a plurality of lasers each emitting at a different wavelength.

In one embodiment, each laser is a directly-modulated distributed feedback laser.

In another embodiment, each laser is an integrated laser modulator semiconductor laser.

In one embodiment, the means for selecting a wavelength for a packet includes means for switching a laser between a plurality of emission wavelengths.

In another aspect, the invention provides a supercomputer comprising a plurality of computing processors and any interconnect network as defined above.

IBo7 Ο 1 13 « 070 113 -7- 1^7(1 Of In one embodiment, each network node comprises a plurality of channel nodes for each network link, and each processor is connected to a channel node.

In one embodiment, each processor is connected to a buffer of a channel node.

In one embodiment, the computing processors are connected to the network by a parallel bus.

DETAILED DESCRIPTION OF THE INVENTION Brief Description of the Drawings The invention will be more clearly understood from the following description of some embodiments thereof, given by way of example only with reference to the accompanying drawings in which:Fig. 1 is a diagram showing a 4 x 4 “clockwork” type network of the prior art discussed in the INTRODUCTION above and also applying to the invention at this level; Fig. 2 is a diagram of a 2x2 crossbar switch for a node of the network of Fig. 1, showing the cross (a) and bar (b) states, discussed in the INTRODUCTION above and also applying to the invention at this level; Fig. 3 is a timing diagram showing a continuous sequence of time slots for the prior art network of Fig. 1, discussed in the INTRODUCTION above and also applying to the invention at this level; Fig. 4 is a block diagram showing the functional components of a network or “torus” node; »070 1 13 -8Figs. 5(a) and (b) are diagrams of a 2x2 crossbar optical space switch of the network node; Fig. 6 shows a configuration of optical components to implement a passive optical routing device of the switch of Fig. 5; Fig. 7 shows a configuration of opto-electronic components to implement an optical wavelength converter of the switch of Fig. 5.

Fig. 8 is a block diagram showing the functional components of a wavelength or “optical” channel node of the network node of Fig. 4; Fig. 9 shows a configuration of optical and opto-electronic components to implement a transmission wavelength selector of the channel node of Fig. 8; and Figs. 10 and 11 show alternative implementations of the transmission wavelength selector.

Description of the Embodiments An optical communication network uses packet routing to transmit data in the optical domain according to the “clockwork” arrangement, without contention at nodes. The network is as represented in Fig. 1 at a high level, the invention lying in the manner of implementing the network or “torus” nodes. Each network node acts at any one time as a sender node and/or a transit node and/or a receiver node. Each network node contains several optical channel nodes. Each channel node can host a number (maybe several tens or even hundreds) of processors. The total number of processors (number of processors hosted in each channel node x number of channel nodes in a torus node x number of torus nodes in the network) together constitute a supercomputer. «070 113 -9VEw 7 0 113 7 9 91 ; The size of the network can be kept small, with n being only of the order of 8 for example while providing a much higher network capacity. This is achieved because each node selects not only a time slot and outgoing link (row or column), but also a wavelength for transmission of each packet. The network is of particular benefit as a supercomputer architecture that consists of large numbers of interconnected individual processors. This allows supercomputers to be scaled to substantially greater computing power than has been achieved to date.

By using WDM as part of the routing process, it is possible to host a much larger number of supercomputer processors, whilst maintaining the available interconnection bandwidth per processor. That is an important advantage of the invention. The scaling limitation of the prior clockwork architecture is that increasing the size of the torus increases the number of hops between any pair of nodes or processors, thus reducing the available interconnection bandwidth per processor. The invention overcomes this scaling limitation. Thus, if m (the number of distinct wavelengths used on the row or column directions) is 32 in a network with 8x8 torus nodes, the number of channel nodes in each torus node is 2x32 (for the example configuration of Fig. 4), and so the overall number of wavelength channel nodes is hx«x»jx2 = 8x8x64; yet«, the dimension of the nxn torus, is still 8, and so the logical diameter of the network remains unchanged as 2(n-1) with n=8. Because the logical diameter is unchanged, the average number of hops between any pair of nodes (or processors) in the network - and thus the available interconnection bandwidth per processor - is unchanged, despite the much greater number of processors hosted.

Fig. 4 shows the internal configuration of a network node 70 to achieve the required functionality. Components and combinations of components other than that shown in Fig. could be used to achieve the same or similar functionality. A row input 76 and a row output 78 carry signals on wavelengths λι, λ2, ..., λη,, and a column input 77 and a column output 79 carry signals on wavelengths λι', λ2', ..., λ„'. An optical space switch It η 7 ο 1 13 Κ 070 113 -ίο75, an example of which is shown in Figs. 5(a) and 5(b), directs signals in the row or column directions according to their time slot.

The Fig. 4 architecture is one example; it shows m channel nodes in each of the row and column inputs (i.e. a total of 2m channel nodes per torus node. An alternative configuration has m (not 2m) channel nodes per torus node, and each channel node can transmit and receive on both the row and column directions. Alternatively, an arrayed waveguide router with similar functionality could be used as the switch 75.

A wavelength demultiplexer 72a splits the signals received at the row input 76 into the separate wavelengths λι, λ2, ..., each of which is passed to the input of one of m channel nodes 73a. We assume here that the channel node p receives signals on wavelength λρ. The optical output from those channel nodes is combined by a combiner 74a and passed to the switch 75. Similarly, a wavelength demultiplexer 72b splits the signals received at a column input 77 into the separate wavelengths λι’, λ2*, ..., λ„’, each of which is passed to the input of one of m optical channel nodes 73b. We assume here that the channel nodep' receives signals on wavelength λ/. The optical output from those optical channel nodes is combined by a combiner 74b and passed to the switch 75.

Thus in this example each network node can accommodate 2m channel nodes, and therefore the number of channel nodes in the network is 2n2m. Each optical channel node could host a plurality of processors, each interconnected by a dedicated local network within the channel node. If the number of processors hosted by a channel node is 2, then the number of processors in the network is 2n2mz. A contention-resolution unit 71 receives control signals from each of the optical channel nodes within the torus node, and these control signals indicate requests for time slots and wavelengths for the insertion of packets into the network. The contention-resolution unit 71 detects the occurrence of contending requests, and returns control signals to the optical channel nodes indicating grants or denials of permission to use particular time slots and wavelengths. * 07011? «070 1 1 3 iR/oou The 2x2 cross-bar switch 75 for a network node is in one embodiment implemented using a passive optical routing device in combination with optical wavelength converters. The architecture is shown in Fig. 5, in which a passive routing device 41 of a node directs an optical signal with wavelength λι received from input a or b to output c, and directs an optical signal with different wavelength λι' received from input at a or b to output d.

Items 42 and 43 are optical wavelength converters which can be switched on or off by control signals at inputs 44 and 45. In the cross state (Fig. 5a), the wavelength converters 42 and 43 are switched off, and a signal with optical wavelength λι at input port a is directed to output port c, whilst a signal with a different optical wavelength λ/ at input port b is directed to output port d. However, in the bar state (Fig. 5b), the wavelength converters 42 and 43 are switched on. The converter 42 converts the wavelength of an optical signal from λι to λι’, and the converter 43 converts the wavelength of an optical signal from λι’ to λι. A signal with wavelength λι at the input a is converted to wavelength λι' and directed to output port d, whilst a signal with wavelength λ/ at input port b is converted to the wavelength λι and directed to output port c. Thus, switching on or off the wavelength converters will effect the action of a 2x2 cross-bar switch.

Fig. 6 shows the hardware implementation of the passive routing device 41. It comprises an optical combiner 51 of the type JDSU Corp, fusion coupler FFC-1K31AB100) and an optical band splitter 52 (of type JDSU Corp, bandsplitter DWBW2). The combiner 51 is almost independent of wavelength for its operation (within limits specified by the manufacturer), whereas the band splitter 52 is designed to direct input signals of wavelength λι to output port c and input signals of wavelength λΤ to output port d.

Different types of optical wavelength converters may be used, including opticalelectrical-optical (OEO) converters and all-optical converters. Fig. 7 shows an example of an OEO converter 42. An optical receiver 61 converts an optical signal input on an optical waveguide 66 with wavelength λρ to an electrical data signal. A laser 62 is of a type whose output wavelength can be switched between two values, λρ and λρ', according to the value of a control signal at a control input 64. The output from the laser is II 070 1II Μ -12ι eo f 0'001 modulated by a modulator 63, which is driven by the electrical data signal obtained from the receiver 61. The modulator 63 is of a type capable of operating correctly at either of the selected wavelengths λρ and λ/. A control signal input 65 may also be applied to the modulator in order to optimise the operation of the modulator according to the wavelength of the laser output. The optical output is emitted on an optical waveguide 67.

The invention achieves the benefits of clockwork routing including reduced latency and complexity, whilst greatly increasing the bandwidth-carrying capacity of the network. In the prior clockwork routing at any instant and fixed position in the network a maximum of only one information signal is carried on a link between adjacent torus nodes. In this invention, clockwork routing is extended so as to be capable of carrying simultaneously a plurality of information signals on a link between adjacent torus nodes by means of wavelength division multiplexing. Crucially, when a packet is to be transmitted across the network from a source node to a destination node, the source node predetermines the path to be followed by the packet on its entire route from the source to the destination by selecting the link, time slot and wavelength with which the packet leaves the source. The selection is performed with the benefit of a routing table at each source node. While the table is more extensive than in prior clockwork networks, it is still of a very manageable size and the look-up does not introduce any appreciable complexity.

To transmit a packet starting from node (2,0) to reach node (3,2), the node (2,0) selects the row output and time slot 2. The packet is held in an electronic buffer before conversion to an optical signal and transmission into the optical network, and the route selection can be performed in software using a look-up table.

Consider the routing of a packet from a channel node p in network (or “torus”) node (2,0) to reach optical channel node q in torus node (3,2). The channel node p in torus node (2,0) selects the row output, wavelength lambda subscript q and time slot 2. As before, the packet is held in an electronic buffer (item 802 in Fig. 8) prior to transmission, and the route selection can be performed in software using a look-up table which specifies Κ 070 11J ·13' route(s) to be used for packets transmitted from channel node p in torus node (2,0) to reach channel node 1 in torus node (3,2).

Take as an example an «xn Manhattan street network, as illustrated in Fig. 1 for the case n=4. The network comprises n2 torus nodes and the links between them. In this invention each link may carry up to m signals simultaneously, each on one of up to m fixed and distinct optical wavelengths. Links interconnecting adjacent torus nodes located on the same row may carry signals on wavelengths λι, λ2, ..., Xm, and links interconnecting adjacent torus nodes located on the same column may carry signals on wavelengths λι', k2,..., Tm'.

Fig. 8 shows an example of the internal configuration of a channel node 73a of the network node 70 (Fig. 4). An optical signal at an input 81 is on a fixed predetermined wavelength, as defined by the wavelength demultiplexer 72a or 72b. The signal enters a receiver 82 where it is converted to an electrical data signal which is transferred to a control unit 83 and a serial delay unit 84. The control unit 83 performs a number of functions, including: determining whether or not a time slot contains an incoming packet; determining the destination address of an incoming packet; determining other control data associated with an incoming packet such as its priority class; receiving from processors hosted within the optical channel node 80 requests for time slots and wavelengths for the insertion of packets into the network. The information about incoming packets and outstanding requests to transmit is conveyed from the control unit 83 to the contention-resolution unit 71, which subsequently relays control messages back to the control unit 83. On the basis of the information and control messages received, the control unit 83 generates control signals 800, 803, 805, 64 and 65. The function of serial the delay unit 84 is to provide a fixed predetermined time delay sufficient to allow the control processing and signalling to be completed, before transferring the electrical data signal to the switch 85. In accordance with control signal 800, the switch 85 will divert the electrical data signal to either the input buffer 86 or optical modulator 65. The electrical data signal will be diverted to the input buffer 86 in the case that the destination I6o 7 0 1 13 ΕΚ» 7 C Ο ί j -14address of the incoming packet is within the optical channel node 80, otherwise the electrical data signal will be diverted to the optical modulator 65 for immediate onward transmission in the network. The input buffer 86 transfers an incoming packet via output 87 to its required destination.

In another implementation the channel node 73 a may also be configured to pass the electrical data signal to both 86 and 65 simultaneously in the case that the incoming packet is assigned a plurality of destination addresses in different channel nodes or is to be broadcast to a multiplicity of channel nodes. In accordance with control signal 64, the laser 62 emits light on either one of two distinct predetermined optical wavelengths, and this light is modulated by 63 in accordance with the electrical data signal received from switch 85.

Packets to be inserted onto the network are received from processors in the channel node via 801 and are held in output buffer 802. In accordance with control signal 803, data packets will be transferred at appropriate times to transmission wavelength generator unit 804. In accordance with control signal 805, the transmission wavelength generator unit 804 emits optical data packets at one of a plurality of wavelengths λι, λ2, ..., ληι, λ/, λ2', ..., The outputs from wavelength selector unit 804 and modulator 63 are combined at optical coupler 806 and pass to optical output 807. Control signal 805 tells the transmission wavelength generator unit 804 which output wavelength is required. This control signal is derived from the routing table. For instance, in an example described above, the wavelength selected for the required route is q. The control circuitry is not shown explicitly in Fig. 8. A dedicated control processor chip runs the software needed to hold the routing table and make the appropriate route selection depending on the destination address of a packet. An output from that route selection process is therefore the control signal 805 instructing the transmission wavelength generator unit 804 to select wavelength q.

S6« 7 ο 1 1 3 « Ο 7 Ο 11 Λ Whereas in Fig. 8 only one output buffer 802 is shown, it may be advantageous to have a plurality of such buffers which queue outgoing packets according to their priority class or other attributes. Whereas in Fig. 8 only one transmission wavelength generator unit 804 is shown, it may be advantageous to have a plurality of such units which are capable of transmitting packets with different wavelengths simultaneously in the same time slot, in which case the outputs from this plurality of transmission wavelength generator units are combined optically together with that from modulator 63 and pass to optical output 807. Whereas in Fig. 8 an electrical serial delay unit 84 is used, it may advantageous in terms of performance or cost to use a configuration in which optical signals are delayed using a length of optical fibre or planar waveguide.

In this invention, when a packet is inserted onto the network by an optical channel node, its path through the network from the source to the ultimate destination is entirely determined at the source node by selecting a time slot and wavelength for the packet. This, together with the global synchronised switching with a fixed periodicity by the transit nodes passed by the packet on its route from source to destination, ensures delivery to the destination. The transit nodes are not involved in path selection. For the purpose of routing a packet, the only processing required at a transit or destination optical channel node is to determine whether the destination address of the incoming packet matches an address in the node; if there is a match then the destination is reached, otherwise the packet is transmitted onwards.

Fig. 9 shows one implementation of a transmission wavelength generator 804 of the wavelength channel node 73a. The data signal to be transmitted is input in electrical form at 91 and passes to switch 93. In accordance with control signal 805, switch 93 transfers the data signal to one of a plurality of lasers 94 each emitting on a different wavelength. For example, the lasers 94 could be directly-modulated distributed feedback semiconductor lasers, or could be integrated laser modulator semiconductor devices. Only the laser selected by switch 93 emits an optical signal, and this signal is modulated IE 0 70 115 ’k<)701l3 ·16· '1^07081 J in accordance with the data signal to be transmitted. The outputs from the lasers are combined in a wavelength multiplexer 95 to provide the optical output 92.

Fig. 10 shows an alternative implementation of a transmission wavelength generator 804.

The data signal to be transmitted is input in electrical form at 101 and passes to an electro-optic modulator 103. In accordance with control signal 805, the driver unit 106 switches on a selected one of a plurality of continuous-wave lasers 104, such as distributed feedback semiconductor lasers. The optical outputs from the lasers are combined in a wavelength multiplexer 105, and then passes to modulator 103 where the optical wave is modulated in accordance with the data signal to be transmitted, to produce the optical output 102.

Fig. 11 shows a further alternative implementation of a transmission wavelength generator 804. The data signal to be transmitted is input in electrical form at 111 and passes to electro-optic modulator 113. In accordance with control signal 805, the control unit 115 produces a control signal causing tuneable laser 114 to emit light in a single wavelength selected from a plurality of predetermined wavelengths. For example, the tuneable laser could be a sampled-grating distributed feedback semiconductor diode laser. The optical output from the laser passes to a modulator 113 where the optical wave is modulated in accordance with the data signal to be transmitted, to produce the optical output 112.

Whereas the description above refers to a Manhattan street network, the invention is not limited to that case. For example, the network could be a torus in which adjacent rows and columns do not alternate in direction. Furthermore, whereas Fig. 1 and the description given above suggests that the links between adjacent nodes are unidirectional, the invention is not limited to that case. It may be advantageous for the network to contain bi-directional links, in which case each torus node contains two 2x2 crossbar switches and an increased number of wavelength channel nodes each of which takes as its input one wavelength channel on one of the four links entering the torus node (those four links being two opposing row links and two opposing column links). This 070 1 1 1 ,6”'317 «W70 01J configuration with bi-directional links is of greater complexity and requires approximately twice as many components; however the advantages include double the communications bandwidth and reduced average shortest path between any pair of torus nodes thus reducing the average communications latency.

The method of “clockwork’' routing of data packets in a network has known benefits, including avoidance of the need for a central network controller, thus reducing latency, whilst also eliminating the need for contention resolution at transit network nodes, thus reducing latency, equipment complexity and cost. However, for application as an interconnection network for processors in a supercomputer, the principal drawback of clockwork routing until now has been the limited bandwidth available and the limited ability to scale the network to large size.

A benefit of this invention is that now clockwork routing can be used to provide an interconnection network for the processors in high-end computing, capable of scaling to sizes substantially greater than the world’s largest supercomputers today. This can be illustrated by the following example: • Assume packet size = 512 bits; • Single-channel transmission rate = 40 Gbit/s; • Therefore, duration of data packet = 12.8 ns; • Assume guard band = 1 ns; • Therefore, time slot = 13.8 ns; * · 8 • Assume transmission group velocity = c/1.5 = 2x10 m.s ; • Therefore, physical length of time slot = 2.76 m; • Assume distance between torus nodes is equivalent to 1 time slot; • Assume Manhattan street network of modest size 16x16 (i.e. n=l 6); • Assume number of wavelengths per link m= 16; • Assume bidirectional link configuration; • Therefore, number of channel nodes 2n m = 8192 ?fl 7 0 1 13 IE Ο 7 Ο 113 -18Therefore: • Network bisection bandwidth = 2mn x 40 Gbit/s = 20.48 Tbit/s (in the field of communication networks, the bisection bandwidth is defined as the sum of the bandwidth of the minimum set of links that, if removed, partition the network into two equal unconnected sets of nodes); • Bisection bandwidth per optical channel node = 2.5 Gbit/s; • Number of processors that can be hosted at 44 Mbit/s bisection bandwidth per processor = 20.48xl012/44xl06 = 465,000 processors (equivalent to 56 processors per channel node in this example). For a supercomputer, the processors that are hosted by a channel node may be interconnected by a dedicated local interconnection such as a high speed parallel bus. This may be connected to the buffer 802 for the channel node 73 a. Thus if a processor wishes to transmit a packet to a processor hosted on a different channel node, it sends the packet to the output buffer 802 where it awaits transmission onwards into the optical interconnection network using the clockwork routing described above. In this way all of the processors are interconnected together to form a single supercomputer.

This final result can be compared with the figure of 64,000 processors that can be served at 44 Mbit/s bisection bandwidth per processor in the fully-implemented version of Blue Gene/L.

It will be appreciated that the invention retains the important advantages of clockwork routing including reduced latency and complexity as compared to the prior art, whilst now allowing a higher degree of bandwidth scaling (or equivalently, whilst allowing the network to interconnect a much greater number of devices).

The invention is not limited to the embodiments described but may be varied in construction and detail.

Claims (21)

1. An optical interconnection network having a plurality of network nodes interconnected by optical links: wherein the network nodes (70) each comprise means for receiving a global clock and for switching repeatedly with a fixed periodicity their routing states according to the global clock; and wherein each network node (70) comprises means for selecting a time slot, a wavelength, and an initial optical link for transmitting a packet in dependence upon the desired path for the packet through the network, and for outputting the packet in the selected time slot, wavelength and link.

2. An optical interconnection network as claimed in claim 1, wherein the network topology is a torus.

3. An optical interconnection network as claimed in claims 1 or 2, wherein the network topology is of the Manhattan street type.

4. An optical interconnection network as claimed in any of claims 1 to 3, wherein the means for selecting a wavelength for a packet includes means (93) for switching between a plurality of lasers (94).

5. An optical interconnection network as claimed in any preceding claim, wherein each network node (70) comprises a plurality of channel nodes (73 a) for each link, a wavelength division demultiplexer (72a), a combiner (74a), and an optical space switch (75). ΙΕ λ j ι 070113 2 °· gfSOOOU

6. An optical interconnection network as claimed in claim 5, wherein the optical space switch (75) has a 2x2 configuration.

7. An optical interconnection network as claimed in either of claims 5 or 6, wherein the optical space switch (75) comprises a wavelength converter (42) per link feeding a passive wavelength routing device (41).

8. An optical interconnection network as claimed in claim 7, wherein each routing device (41) comprises a combiner (51) and a wavelength division demultiplexer (52).

9. An optical interconnection network as claimed in claim 7, wherein the routing device comprises an arrayed wavelength router.

10. An optical interconnection network as claimed in claim 7, wherein the routing device comprises a wavelength channel interleaver.

11. An optical interconnection network as claimed in any of claims 5 to 10, wherein each channel node (73 a) comprises a processor, and a buffer (802) connected to a transmission wavelength generator (804).

12. An optical interconnection network as claimed in claims 10 or 11, wherein the processor performs look-ups to a routing table.

13. An optical interconnection network as claimed in claim 12, wherein the channel node buffer comprises means for determining a destination network node address of a packet.

14. An optical interconnection network as claimed in claims 12 or 13, wherein the transmission wavelength generator (804) comprises: Ito 7 Ο 1 13 «070 11J -/ι- '1^70 01 < an interface to receive data in electrical form; an interface (805) to the optical node processor; and 5 a switch (93) to transfer the data signal to one of a plurality of lasers (94) each emitting at a different wavelength.

15. An optical interconnection network as claimed in claim 14, wherein each laser (94) is a directly-modulated distributed feedback laser.

16. An optical interconnection network as claimed in claim 14, wherein each laser is an integrated laser modulator semiconductor laser.

17. An optical interconnection network as claimed in any preceding claim, wherein 15 the means for selecting a wavelength for a packet includes means for switching a laser between a plurality of emission wavelengths.

18. A supercomputer comprising a plurality of computing processors and an interconnect network as claimed in any preceding claim.

19. A supercomputer as claimed in claim 18, wherein each network node (70) comprises a plurality of channel nodes (73a) for each network link (76-79), and each processor is connected to a channel node. 25

20. A supercomputer as claimed in claim 19, wherein each processor is connected to a buffer (802) of a channel node (73 a).

21. A supercomputer as claimed in any of claims 18 to 20, wherein the computing processors are connected to the network by a parallel bus.