WO2020227982A1

WO2020227982A1 - System and method for flexible optical interconnect in data centers

Info

Publication number: WO2020227982A1
Application number: PCT/CN2019/087082
Authority: WO
Inventors: Chongjin Xie; Rui LU; Anbin Wang
Original assignee: Alibaba Group Holding Limited
Priority date: 2019-05-15
Filing date: 2019-05-15
Publication date: 2020-11-19
Also published as: CN113518980A; CN113518980B

Abstract

One embodiment described herein provides a communication system. The communication system can include a first switch that includes one or more optical transceiver modules and a plurality of independent optical cables coupled to a respective optical transceiver module on the first switch.

Description

SYSTEM AND METHOD FOR FLEXIBLE OPTICAL INTERCONNECT IN DATA CENTERS

Inventors: Chongjin Xie, Rui Lu, and Anbin Wang

BACKGROUND Field

This disclosure is generally related to data center designs. More specifically, this disclosure is related to designing optical interconnects among switches in a data center.

Related Art

The rapid growth in cloud users' computing needs continues to drive the computing capabilities of cloud servers, thus increasing the speed and scalability requirements for data centers. A typical data center can be a pool of resources (e.g., computational, storage, and network resources) interconnected using a communication network. The data center network plays an important role in the data center, because it interconnects all of the data center resources.

The data center network needs to be scalable and efficient in order to connect tens or even hundreds of thousands of servers. The key characteristics of the data center network can include bandwidth, scale, and latency. Designers of data center networks often pursue the goal of large scale, low latency, and low cost. Designing large-scale data center networks while keeping the cost and latency low can be a challenge.

SUMMARY

One embodiment described herein provides a communication system. The communication system can include a first switch that includes one or more optical transceiver modules and a plurality of independent optical cables coupled to a respective optical transceiver module on the first switch, thereby allowing the first switch to couple to a plurality of other switches in the communication system via at least the plurality of independent optical cables.

In a variation on this embodiment, the respective optical transceiver module can include one or more of: an SN ^TM-based optical interface, an MDC-based optical interface, and a multi-fiber push on (MPO) optical interface.

In a further variation, the respective optical transceiver module can include multiple SN-or MDC-based interfaces, and a respective optical cable can include an SN-or MDC-based connector, respectively, for coupling the respective optical cable to the optical transceiver module.

In a further variation, the respective optical transceiver module can include an MPO optical interface, and a respective optical cable can include an MPO connector or a duplex LC connector for coupling the respective optical cable to the optical transceiver module.

In a further variation, the respective optical transceiver module can include an MPO optical interface, and a respective optical cable can couple to the MPO optical interface via an MPO converter that converts a higher fiber count MPO interface to a lower fiber count MPO interface.

In a variation on this embodiment, each of the plurality of independent optical cables can be coupled to a different switch in the communication system.

In a variation on this embodiment, a respective switch in the communication system can have an input/output (I/O) capacity of 256 × 50 Gbps (gigabits per second) or 512 × 50 Gbps.

In a further variation, the respective optical module can have a speed of 200 Gbps or 400 Gbps.

In a variation on this embodiment, the first switch can include N optical transceiver modules, and each optical transceiver module can be coupled to M independent optical cables. The first switch can be coupled to M×N other switches in the communication system.

In a further variation, the communication system can include two levels of switches. The first switch is on a first level and a respective port on a downlink of the first switch is labeled using a 3-tuple (i ₁, j ₁, k ₁) with i, j, and k representing a switch sequence number, an optical module sequence number, and a port sequence number, respectively. The port (i ₁, j ₁, k ₁) is coupled to a port (i ₂, j ₂, k ₂) belonging to an uplink of a second switch on a second level, where i ₂ = j ₁*M+k ₁, j ₂ = i ₁%M, and k ₂ = mod (i ₁, M) .

One embodiment can provide a coupling mechanism for coupling among switches in a data center network. The coupling mechanism can include a plurality of independent optical cables coupled to a respective optical transceiver module on a respective switch, thereby allowing the respective switch to couple to a plurality of other switches in the data center network.

One embodiment can provide a method for coupling among switches in a data center network. The method can include selecting a switch, coupling a first end of a plurality of independent optical cables to a respective optical transceiver module on the selected switch, and coupling a second end of the plurality of optical cables to a plurality of other switches in the data center network.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates an exemplary network infrastructure of a data center.

FIGs. 2A and 2B show exemplary interconnections among switches of two different levels, according to prior art.

FIG. 3A shows exemplary interconnections among switches, according to one embodiment.

FIG. 3B shows exemplary interconnections among switches, according to one embodiment.

FIG. 4 shows an exemplary network having two levels of switches, according to one embodiment.

FIG. 5A shows an exemplary optical interface of an optical transceiver module.

FIG. 5B shows an exemplary optical interface configuration of an optical module, according to prior art.

FIG. 5C shows an exemplary output configuration of an optical module, according to one embodiment.

FIG. 5D shows an exemplary output configuration of an optical module, according to one embodiment.

FIG. 6A illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment.

FIG. 6B illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment.

FIG. 7A shows an exemplary output configuration of an optical module, according to one embodiment.

FIG. 7B shows an exemplary output configuration of an optical module, according to one embodiment.

FIG. 8A illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment.

FIG. 8B illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment.

FIG. 9 presents a flowchart illustrating an exemplary process for establishing a data center network, according to one embodiment.

In the figures, like reference numerals refer to the same figure elements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

Overview

Embodiments described herein solve the technical problem of providing efficient and flexible optical interconnects among switches in a data center. More specifically, by implementing multiple parallel fiber outputs on each individual optical module, some embodiments increase the number of independent ports on each switch without the need to modify the internal structure of the switch or the need to increase the count of optical modules on the switch. In some embodiments, the optical modules may conform to a standard form factor, such as quad small form-factor pluggable (QSFP) or octal small form-factor pluggable (OSFP) , and various types of optical interface can be used to couple an optical module to multiple optical cables, with each optical cable corresponding to a switch port. In some embodiments, optical cables from a particular switch on a particular switch level can be arranged in such a way that they are coupled to other individual switches, one switch per optical cable, on a next level. Given a switch equipped with N optical modules, with each optical module being coupled to M parallel output optical cables, the switch can couple to up to M × N other switches.

Optical Interconnect in Data Center

FIG. 1 illustrates an exemplary network infrastructure of a data center. In FIG. 1, data center network 100 can include three layers of network switches, namely an access layer, an aggregate layer, and a core layer. The servers are connected to switches within the access layer. The aggregate layer switches interconnect multiple access layer switches. The aggregate layer modules can also provide various important services, such as content-switching, firewall, Secure Socket Layer (SSL) offloading, intrusion detection, network analysis, etc. All of the aggregate layer switches are connected to each other by core layer switches. The core layer switches are also responsible for connecting the data center to networks outside the data center (e.g., the Internet) . To meet the ever-increasing user demand for computing power, the data center should include a large number of interconnected high-performance servers.

Hardware modules used to form the data center network (e.g., data center network 100 shown in FIG. 1) can include electrical switches, optical transceiver modules (or simply optical modules) , and optical cables. Current data centers typically implement a network having a speed of 100 Gbps (gigabits per second) , with the electrical switch chip having an input/output (I/O) capacity of 128 × 25 Gbps or 256 × 25 Gbps and the optical transceiver module having a speed of 100 Gbps. The network interface card (NIC) of each server can have a speed of 25 Gbps or higher. The next generation data center may implement a 400 Gbps network, with the switch chip having an I/O capacity of 256 × 50 Gbps or 512 × 50 Gbps and the optical transceiver module having a speed of 400 Gbps. The NIC speed of the servers in the next generation data center can be up to 100 Gbps.

For high-speed (e.g., 100 Gbps and beyond) data center networks, optical interconnects (e.g., optical transceivers and optical cables) can be essential in realizing interconnections among the switches and the coupling between the access layer switches and the servers. For a large-scale network, it is desirable to have as many switch ports as possible. However, the mismatch in speed between the switch chip I/O and the optical transceiver can often lead to a reduced number of ports on a switch, which can in turn limit the scale of the data center network. For example, a switch chip I/O can include 512 electrical channels, with each channel running at a speed of 50 GHz. On the other hand, the optical transceiver module may have a speed of 400 Gbps, meaning that up to eight electrical channels would need to be combined onto a single optical transceiver, which is often coupled to a single optical cable to serve as a single switch port. Therefore, instead of providing 512 switch ports, the switch module can only provide up to 64 switch ports.

One possible solution for increasing the port count on a switch is to increase the number of switch chips included in the switch. By cascading a plurality of switch chips, one can increase the total port count of a switch. However, such a solution can be expensive and can also increase the network latency.

Another solution is to reduce the speed of the optical transceiver and use a much larger number of transceivers to serve each switch, thus equivalently increasing the number of ports on each switch. FIGs. 2A and 2B show exemplary interconnections among switches of two different levels, according to prior art. In FIG. 2A, each switch is equipped with one optical module having a single optical cable output. More specifically, upper level switch 202 is equipped with a high-speed optical transceiver module 204 (e.g., a 400G optical module for 400G networks) , and lower level switch 212 is equipped with a high-speed optical transceiver module 214. The output of each optical module is a single optical cable (e.g., optical cable 206) . As one can see, a switch at each level can only couple to a single switch at a different level, because each switch only has one port for each link (e.g., the uplink or the downlink) . Note that here each switch in fact includes both the uplink and the downlink. In the figures, for simplicity purposes, only one link (uplink or downlink) is shown. On the other hand, in the example shown FIG. 2B, the uplink or the downlink of each switch is equipped with two optical modules having reduced speed. For example, instead of a 400G optical transceiver, each switch can be equipped with two 200G optical transceivers. More specifically, the downlink of upper level switch 222 is equipped with

optical transceiver modules

224 and 226, and the uplink of lower level switch 232 is equipped with

optical transceiver modules

234 and 236. Similar to the example shown in FIG. 2A, each optical transceiver module has a single output optical cable. For example, optical module 224 has an output cable 228 and optical module 226 has an output cable 230. As a result, each switch now has two switch ports capable of connecting to two other switches. In the example shown in FIG. 2B, each of the two upper level switches (e.g., switch 222 or 242) can couple to two lower level switches (e.g., switches 232 and 252) . Compared to the example shown in FIG. 2A, the scale of the network shown in FIG. 2B is twice as large.

However, increasing the number of optical transceivers without increasing the network speed can be cost-ineffective, because of the increased cost-per-bit of the optical modules. Moreover, the increased number of optical transceivers can also lead to an enlarged size of the switch box, which is not desirable in data centers.

To overcome these problems, some embodiments increase the number of switch ports on a switch without the need to decrease the speed of its optical modules. More specifically, in some embodiments, instead of a single optical cable, an optical transceiver module can be coupled to multiple parallel optical cables, with each cable including a single fiber or a fiber bundle. In other words, instead of having a single input/output optical cable to facilitate a single switch port, the optical module now has multiple parallel input/output optical cables to facilitate multiple switch ports, thus increasing the number of ports on the switch without the need to reduce the speed of the optical module. Note that each single switch port includes both input and output optical cables.

FIG. 3A shows exemplary interconnections among switches, according to one embodiment. In FIG. 3A, each link of each switch includes one optical module and each optical module includes two independent optical cables. For example, switch 302 is equipped with an optical module 304 in its downlink, which is coupled to two optical cables,

cables

306 and 308. Similarly, switch 312 is equipped with an optical module 314, and the inputs/outputs of optical module 314 are carried by two optical cables,

cables

316 and 318. Note that each optical cable can include a single optical fiber or a bundle of optical fibers.

FIG. 3A also shows that the two optical cables of an optical switch can couple to the two optical cables of two different optical switches. For example, upper level switch 302 can separately couple to lower level switches 322 and 324 via

optical cables

306 and 308. Similarly, upper level switch 312 can also separately couple to lower level switches 322 and 324 via

optical cables

316 and 318. Compared to the example shown in FIG. 2A, the scale of the network shown in FIG. 3A is twice as large, while keeping the speed of the optical modules (e.g., optical modules 304 and 314) unchanged.

The scale of the network can further be increased by equipping multiple optical modules on each switch. FIG. 3B shows exemplary interconnections among switches, according to one embodiment. In FIG. 3B, each switch includes two optical modules in one link (e.g., uplink or downlink) and each optical module includes two input/output optical cables. Consequently, a link of each switch now is provided with four switch ports, making it possible for the switch to couple to up to four other switches in uplink or downlink. In the example shown in FIG. 3B, upper level switch 342 can couple to lower level switches 352, 354, 356, and 358 via the four optical cables of switch 342 in its downlink. Similarly, upper level switch 344 can also couple to the same four lower level switches via its four optical cables in its downlink. Compared to the example shown in FIG. 2B, the scale of the network has been doubled while keeping the switch architecture as well as the speed of the optical modules unchanged.

The scale of the network can be determined based on the number of optical modules per switch and the number of independent inputs/outputs per optical module. FIG. 4 shows an exemplary network having two levels of switches, according to one embodiment. Each switch can be equipped with N optical modules in its uplink or downlink, and each optical module can have M independent inputs/outputs, with each input/output being an individual optical cable. Note that an individual optical cable can either be a single optical fiber or a single bundle of multiple fibers. In the example shown in FIG. 4, optical modules included in the downlink of upper level switch 402 can be labeled as module_0 (or MOD_0) through module_N-1 (or MOD_N-1) , and the switch ports (i.e., the input/output optical cables) of optical module 404 can be labeled as port_0 through port_M-1. As one can see from FIG. 4, the downlink or uplink of each switch has N×Mports, making it possible for the switch to couple to up to N×M other switches in its downlink or uplink.

In some embodiments, a switch can have various I/O capacities, such as 256 × 50 Gbps and 512 × 50 Gbps. The scope of this disclosure is not limited by the type or capacity of the switch module. Similarly, an optical module equipped on the switch can have various speeds and conform to various types of form factor. In some embodiments, an optical module can have a speed of 200 Gbps or 400 Gbps. The optical module can have a form factor of octal small form-factor (OSFP) , quad small form-factor double-density (QSFP-DD) , or QSFP. Moreover, the optical Physical Media Dependent (PMD) types used by the optical module can include SR8 (which refers to 8 pairs of multi-mode fibers) , DR4 (which refers to 4 pairs of single-mode fibers) , and SR4.2 (which refers to 4 pairs of multi-mode fibers, with 2 wavelength channels per fiber) .

In the example shown in FIG. 4, each switch level includes N×M switches, and each switch is coupled to N×M switches located at a different level, thus maximizing the scale of the network. In some embodiments, a switch port at the upper level can be defined as (i ₁, j ₁, k ₁) and a switch port at the lower level can be defined as (i ₂, j ₂, k ₂) , where i, j, k refer to the numberings of the switch, optical module on a switch, and output optical cable of an optical module, respectively. The numbering (or sequence number) of the switches (i.e., i) can range from 0 to N×M-1, the numbering of the optical modules on a switch can range from 0 to N-1, and the numbering of the output optical cables on an optical module can range from 0 to M-1. The subscript (1 or 2) refers to the switch level (upper or lower, respectively) . In some embodiments, the connections among the switch ports can be arranged using the following formula:

i ₂ = j ₁*M+k ₁

j ₂ = i ₁%M ,

k ₂ = mod (i ₁, M)

where M refers to the number of optical cables per optical module, and the %symbol indicates an integer division operation. According to the above formula, the leftmost switch port in the upper level (i.e., port (0, 0, 0) ) should be connected to switch port (0, 0, 0) , which is the leftmost port in the lower level. Similarly, the adjacent switch port (i.e., port (0, 0, 1) ) should be connected to switch port (1, 0, 0) , meaning that it is connected to the leftmost port on the second leftmost switch at the lower level. One can see that the above formula describes the connections shown in FIG. 3B. Note that a symmetric pattern can be observed where connections from switch ports on one side of the top level can be a mirror image of the connections from the switch ports on the other side. In the example shown in FIG. 3B, the connections from the left two switches on the top level are symmetric to connections from the right two switches. For example, switch ports on the leftmost switch are sequentially (from left to right) coupled to the leftmost port on each lower level switch, and switch ports on the rightmost switch are sequentially (from right to left) coupled to the rightmost port on each lower level switch. Such an arrangement ensures maximized connections among all switches (i.e., each individual switch is connected to the maximum number of other switches) while keeping the total length of the connecting cables short.

It is also possible for the switch ports to have different types of connection relationships other than the one defined by the aforementioned formula, as long as the number of interconnected switches can be maximized. For example, it is also possible to have more or fewer switch levels. In the event of three switch levels, a middle level switch may have some ports coupled to the upper level switches and some ports coupled to the lower level switches. The scope of this disclosure is not limited by the actual connection pattern among switch ports.

Depending on the type of optical modules used in the switch, the optical cables can be assembled differently. More particularly, depending on the type of optical interface provided by the optical modules, the optical cables can be assembled differently.

In some embodiments, the optical module may include a 400G-QSFP-DD-DR4 optical module having 4 pluggable fiber interfaces, such as SN ^TM (trademark of Senko Advanced Components of Marlborough, MA) interfaces and MDC interfaces manufactured by US Conec, Ltd. of Hickory, NC. FIG. 5A shows an exemplary optical interface of an optical transceiver module. In FIG. 5A, optical interface 500 of an optical module can include 4 pairs of fiber connectors, such as fiber connector pairs 502 and 504. In some embodiments, each fiber connector can include an LC-style connector. Each fiber connector can facilitate the coupling between a single fiber and the optical module. In some embodiments, each fiber pair (i.e., the pair of fibers coupled to a connector pair) can separately carry the transmitted and received optical signals. In the example shown in FIG. 5A, up to 8 individual optical fibers can be coupled to the 4 pairs of fiber connectors.

In conventional data center settings, the output of an optical module only includes a single optical cable, which can include a bundle of fibers with one half of the fibers carrying transmitted optical signals and the other half carrying received optical signals. FIG. 5B shows an exemplary optical interface configuration of an optical module, according to prior art. In FIG. 5B, optical module 510 on a switch has a single optical cable 512. Note that, although not shown in FIG. 5B, single optical cable 512 in fact can include four pairs of optical fibers. A fiber pair always stays together as it represents the different directions of a single optical link. In the example shown in FIG. 5B, the four pairs of optical fibers can be either multi-mode fibers (MMFs) or single-mode fibers (SMFs) . As discussed previously, the switch can couple to another switch via optical cable 512. Optical module 510 shown in FIG. 5B at most provides a single switch port for coupling to a different switch. FIG. 5B also shows, in a dashed ellipse, the front view of connector 514 that couples single optical cable 512 to optical interface 516 on optical module 510. More specifically, connector 514 can include four individual fiber connectors, with each connector having both an input port and an output port.

To increase the number of switch ports on each switch module, in some embodiments, the individual optical cable coupled to the optical module can be separated into multiple groups. FIG. 5C shows an exemplary output configuration of an optical module, according to one embodiment. In FIG. 5C, the eight fibers coupled to optical module 520 are grouped into two separate groups, with each fiber group forming an independent switch port, such as

switch ports

522 and 524. More specifically, each fiber group can include an optical cable representing a single optical link, and each optical cable can include four optical fibers bundled together. The optical fibers can include single-or multi-mode optical fibers. Because optical module 520 includes two independent switch ports (i.e., switch ports 522 and 524) , a switch equipped with optical module 520 can couple to at least two other switches via the two switch ports (i.e., switch ports 522 and 524) of optical module 520. FIG. 5C also shows, in a dashed circle, the front view of connector 526 that couples an

optical cable

522 or 524 to optical interface 528 of optical module 520. More specifically, connector 526 can include two individual fiber connectors. In some embodiments, optical interface 528 can include SN, MDC, or duplex LC connectors, and one can select connector 526 based on the type of fiber connectors located on optical interface 528.

FIG. 5D shows an exemplary output configuration of an optical module, according to one embodiment. In FIG. 5D, the eight fibers coupled to optical module 530 can be grouped into four separate groups, with each fiber group forming an independent switch port, such as

switch port

532 or 534. More specifically, each fiber group can include a pair of fibers representing a single optical link. Each optical fiber within the pair of fibers carries optical signals in one direction. The optical fibers can include single-or multi-mode optical fibers. Because optical module 530 includes four independent switch ports, a switch equipped with optical module 530 can couple to at least four other switches via the four switch ports of optical module 530. FIG. 5D also shows, in a dashed circle, the front view of connector 536 that couples a fiber pair to optical interface 538 on optical module 530. More specifically, connector 536 can include an individual fiber connector. In some embodiments, optical interface 538 can include SN, MDC, or duplex LC connectors, and one can select connector 536 based on the type of fiber connectors located on optical interface 538. For example, if optical interface 538 includes four SN connectors, connector 536 can include a corresponding SN connector.

FIG. 6A illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment. In FIG. 6A, optical fiber bundle 602 can include eight optical fibers (SMFs or MMFs) grouped into two groups. Each group can include four optical fibers bundled together as an individual optical cable (e.g., optical cables 604 and 606) . A four-fiber optical connector (e.g., two SN, MDC, or duplex LC connectors grouped together) can be attached to each end of each optical cable. For example, four-fiber

optical connectors

608 and 610 can be attached to the left and right ends, respectively, of optical cable 604. Each four-fiber optical connector can be used to couple an end of a corresponding optical cable to the optical interface of an optical transceiver module. For example,

connectors

608 and 610 can couple to

optical interfaces

612 and 614, respectively. More specifically, the optical or fiber connector on an optical cable matches the corresponding optical interface, such that the optical connector can mate with the corresponding optical interface to achieve low-loss coupling. For example, if the optical interface on the optical module is SN, the optical connector on the corresponding optical cable would be an SN connector. Note that, in real life implementations, optical fiber bundle 602 can be kept together as a single bundle before it fans out at each end to allow the individual optical cables to couple to different switches. This can reduce the number of dangling cables in the data center.

In some embodiments, an optical module can include an eight-fiber optical interface, thus allowing up to eight optical fibers to simultaneously couple to the optical module. By assembling the eight fibers that are coupled to the same optical module into two separate optical cables and by attaching individual connectors onto the optical cables, in a way similar to what is shown in FIG. 6A, some embodiments provide a single optical module the ability to couple to two other optical modules, without making any change to the design of the optical interface on the optical module. If the single optical module belongs to a particular switch, and the two other optical modules belong to two other, different switches, this particular fiber arrangement can now allow that particular switch to couple to the two other, different switches, thus increasing the scale of the network formed by the switches.

FIG. 6B illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment. In FIG. 6B, optical fiber bundle 622 can include eight optical fibers (SMFs or MMFs) grouped into four groups. Each group can include two optical fibers bundled together as an individual optical cable (e.g., optical cables 624 and 626) . A two-fiber optical connector (e.g., an SN, MDC, or LC connector) can be attached to each end of each optical cable. For example, two-fiber

optical connectors

628 and 630 can be attached to the left and right ends, respectively, of optical cable 624. Each two-fiber optical connector can be used to couple an end of a corresponding optical cable to the optical interface on an optical transceiver module. For example,

connectors

628 and 630 can couple to

optical interfaces

632 and 634, respectively. By assembling the eight fibers into four separate optical cables and by attaching individual optical connectors onto the ends of the optical cables, some embodiments allow each end of optical fiber bundle 622 to couple to up to four other optical modules and, hence, up to four other switches.

In some embodiments, an optical module equipped on a switch may have a multi-fiber push on (MPO) interface, where multiple fibers can couple to the optical module via the MPO interface. Moreover, to group the multiple fibers into separate groups, additional MPO connectors can be employed. FIG. 7A shows an exemplary output configuration of an optical module, according to one embodiment. In FIG. 7A, optical module 700 has an MPO interface 702. A fiber bundle having an MPO connector 704 can be coupled to optical module 700 via MPO interface 702 and MPO connector 704. In some embodiments, MPO connector 704 can be a 16-fiber MPO connector and the bundle of fibers can include 16 SMFs or MMFs that are individually contained in their fiber jackets.

To achieve a desired number of switch ports supported by the optical module, in some embodiments, the bundle of individual fibers can be separated into multiple groups. In the example shown in FIG. 7A, the 16 fibers have been separated into two groups, with each group comprising eight fibers. Each group of eight fibers can also couple to a 2nd-level MPO connector (e.g., an eight-fiber MPO connector 706 or 708) , thus allowing these 16 fibers to couple to other optical modules via their MPO connector. Note that the eight fibers in each group can also be divided into two subgroups with each subgroup carrying signals in one direction. In some embodiments,

MPO connectors

706 and 708 can have a different gender than that of MPO connector 704. For example, if MPO connector 704 is male,

MPO connectors

706 and 708 can be female, thus allowing optical cables having male connectors to couple to

MPO connectors

706 and 708. In general, MPO connectors 704-708 can function as a converter that converts the 16-fiber MPO interface 702 to two eight-fiber MPO interfaces 706 and 708.

As one can see in FIG. 7A, optical module 700 now includes two separate ports (i.e., MPO connectors 706 and 708) , thus allowing optical module 700 to couple to up to two other optical modules. In other words, a switch equipped with optical module 700 can couple to up to two other switches via

MPO connectors

704, 706, and 708. In this example, the high-fiber-count to low-fiber-count MPO converter (which includes MPO connectors 704-708) allows the optical module to have two independent inputs/outputs, thus making it possible for the optical module to couple to up to two other optical modules.

FIG. 7B shows an exemplary output configuration of an optical module, according to one embodiment. In FIG. 7B, optical module 720 has an MPO interface 722. A fiber bundle having an MPO connector 724 can couple to optical module 720 via MPO interface 722 and MPO connector 724. In some embodiments, MPO connector 724 can be a 16-fiber MPO connector and the bundle of fibers can include 16 SMFs or MMFs. The 16 fibers attached to MPO connector 724 can be grouped into four groups, with each group including four fibers. A 2nd-level MPO connector (e.g., four-fiber MPO connector 726 or 728) can be attached to the other end of each group. In addition to the four-fiber MPO connector, other types of connector, such as a pair of duplex LC or SN connectors, can also be used at the other end of each four-fiber group. In the example shown in FIG. 7B, the converter that includes a 16-fiber MPO connector and four four-fiber connectors allows the optical module to have four independent switch ports, thus making it possible for the optical module to couple to up to four other optical modules.

FIG. 8A illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment. In FIG. 8A, optical fiber bundle 802 can include sixteen optical fibers (SMFs or MMFs) grouped into two groups. Each group can include eight optical fibers bundled together as an individual optical cable (e.g., optical cables 804 and 806) . An eight-fiber MPO connector can be attached to each end of each optical cable. For example, eight-

fiber MPO connectors

808 and 810 can be attached to the left and right ends, respectively, of optical cable 804. Each eight-fiber MPO connector can couple an optical cable to a corresponding MPO interface on an optical transceiver module.

FIG. 8B illustrates an exemplary arrangement of a plurality of optical cables, according to one embodiment. In FIG. 8B, optical fiber bundle 822 can include sixteen optical fibers (SMFs or MMFs) grouped into four groups. Each group can include four optical fibers bundled together as an individual optical cable (e.g., optical cables 824 and 826) . A four-fiber optical connector (e.g., an MPO or quad-LC connector) can be attached to each end of each optical cable. For example, four-

fiber MPO connectors

828 and 830 can be attached to the left and right ends, respectively, of optical cable 824. Each four-fiber optical connector can facilitate the coupling between an optical cable and a corresponding optical transceiver module. By assembling the 16 fibers into four independent optical cables, some embodiments allow one optical module to couple to up to four other optical modules, thus increasing the scale of the network fourfold.

FIG. 9 presents a flowchart illustrating an exemplary process for establishing a data center network, according to one embodiment. During operation, a high-performance data center switch can be selected (operation 902) . The switch can be equipped with a number of optical transceiver modules, which typically conform to a standard form factor (e.g., SFP, QSFP, QSFP-DD, OSFP, etc. ) . The PMD type of an optical transceiver module can be SR8, DR4, or SR4.2. The optical interface on the optical transceiver module can be SN, MDC, or MPO. Each optical interface may allow coupling of a number of individual fibers. For example, an optical module can allow up to 16 individual fibers (SMFs or MMFs) to be coupled to its optical interface. The coupling between the fibers and the optical module can be accomplished via an SN, MDC, or MPO type of optical interface.

Subsequent to selecting the switch, one can couple a number of individual optical cables to the one or more optical modules on the switch (operation 904) . In some embodiments, multiple optical cables can couple to a single optical module. Each optical cable can include a single fiber or a bundle of multiple fibers. For example, two optical cables with each optical cable including four individual fibers can couple to an optical module having an eight-fiber optical interface. Depending on the type of interface on the optical transceiver module, each optical cable may be equipped with an appropriate optical connector. For example, if the optical transceiver module has an SN or MDC type of optical interface, each optical cable will have an SN or MDC connector. For example, if an optical module has an eight-fiber SN interface, the two optical cables can each have a four-fiber dual-SN connector for plugging into the SN interface on the optical module. In a different example, four optical cables can couple to the eight-fiber SN interface on the optical module, with each optical cable having an SN connector. On the other hand, if the optical transceiver module has an MPO interface, each optical cable can also have an MPO connector but with a smaller fiber count. An MPO converter, similar to one formed by MPO connectors 704-708 shown in 7A, can facilitate the coupling between the optical cable and the optical transceiver module.

Subsequent to coupling multiple optical cables to each optical module on the switch, one can couple the other end of each optical cable to a different switch on the other level according to a predetermined switch-interconnection map (operation 906) . For example, each switch port (i.e., each individual optical cable) can be labeled using a 3-tuple (i _a, j _a, k _a) , where subscript a specifies the switch level, and i, j, and k specify the switch number, optical module number, and optical cable number, respectively. In some embodiments, there are two switch levels. An optical cable of a particular switch port on the upper level (e.g., port (i ₁, j ₁, k ₁) ) can be coupled to a predetermined switch port on the lower lever (e.g., port (i ₂, j ₂, k ₂) ) . More specifically, the lower level port can be determined based on the following formula:

i ₂ = j ₁*M+k ₁

j ₂ = i ₁%M .

k ₂ = mod (i ₁, M)

Subsequent to coupling all individual optical cables from one switch to the different switches on the next level, one can determine whether all switches have been connected (operation 908) . If not, a different switch is selected (operation 902) and the process repeats until every switch has been connected to a maximum number of other switches.

In general, embodiments of the present invention provide a solution for enlarging the scale of a data center network by increasing the number switch ports supported by each switch without making changes to the switch architecture as well as the optical transceiver modules. More specifically, to increase the number of ports per switch, some embodiments allow multiple independent optical cables to couple to a single optical module, with each optical cable representing an individual switch port. The independent optical cables can be routed to other, different switches, thus connecting the switch to those other switches. Because the number of optical cables coupled to a switch can be larger than the number of optical modules on the switch, compared to conventional approaches where a single optical cable is coupled to each optical module, some embodiments can significantly increase the number of ports per switch and, hence, the scale of the network. In a particular scenario where there are two levels of switches, the coupling among the switch ports of the two levels can follow a predetermined pattern, which can ensure that each switch is coupled to a maximum number of other switches while keeping the total cable length short. Various types of optical interface can be employed by the optical transceiver modules. Accordingly, the optical cables may employ different types of optical connector. The scope of this disclosure is not limited by the particular type of connector used by each optical cable.

The foregoing descriptions of various embodiments have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. Additionally, the above disclosure is not intended to limit the present invention. The scope of the embodiments described herein is defined by the appended claims.

Claims

A communication system, comprising:

a first switch, wherein the first switch comprises one or more optical transceiver modules; and

a plurality of independent optical cables coupled to a respective optical transceiver module of the first switch.
The communication system of claim 1, wherein the respective optical transceiver module comprises one or more of:

an SN-based optical interface;

an MDC-based optical interface; and

a multi-fiber push on (MPO) optical interface.
The communication system of claim 2, wherein the respective optical transceiver module comprises multiple SN-or MDC-based interfaces, and wherein a respective optical cable comprises an SN-or MDC-based connector, respectively, for coupling the respective optical cable to the optical transceiver module.
The communication system of claim 2, wherein the respective optical transceiver module comprises an MPO optical interface, and wherein a respective optical cable comprises an MPO connector or a duplex LC connector for coupling the respective optical cable to the optical transceiver module.
The communication system of claim 2, wherein the respective optical transceiver module comprises an MPO optical interface, and wherein a respective optical cable can couple to the MPO optical interface via an MPO converter that converts a higher fiber count MPO interface to a lower fiber count MPO interface.
The communication system of claim 1, wherein each of the plurality of independent optical cables is coupled to a different switch in the communication system.
The communication system of claim 1, wherein a respective switch in the communication system has an input/output (I/O) capacity of 256 ×50 Gbps (gigabits per second) or 512 × 50 Gbps.
The communication system of claim 7, wherein the respective optical module has a speed of 200 Gbps (gigabits per second) or 400 Gbps.
The communication system of claim 1, wherein the first switch comprises N optical transceiver modules, wherein each optical transceiver module is coupled to M independent optical cables, and wherein the first switch is coupled to M×N other switches in the communication system.
The communication system of claim 9, wherein the communication system comprises at least two levels of switches; wherein the first switch is on a first level and a respective port on a downlink of the first switch is labeled using a 3-tuple (i ₁, j ₁, k ₁) with i, j, and k representing a switch sequence number, an optical module sequence number, and a port sequence number, respectively; and wherein the port (i ₁, j ₁, k ₁) is coupled to a port (i ₂, j ₂, k ₂) belonging to an uplink of a second switch on a second level, where i ₂=j ₁*M+k ₁, j ₂=i ₁%M, and k ₂=mod (i ₁, M) .
A coupling mechanism for coupling among switches in a data center network, comprising:

a plurality of independent optical cables coupled to a respective optical transceiver module of a respective switch, wherein the plurality of independent optical cables are coupled to a number of different switches in the data center network.
The coupling mechanism of claim 11, wherein the respective optical transceiver module comprises multiple SN-or MDC-based interfaces, and wherein a respective optical cable comprises an SN-or MDC-based connector, respectively, for coupling the respective optical cable to the optical transceiver module.
The coupling mechanism of claim 11, wherein the respective optical transceiver module comprises an MPO optical interface, and wherein a respective optical cable comprises an MPO connector or a duplex LC connector for coupling the respective optical cable to the optical transceiver module.
The coupling mechanism of claim 11, wherein the respective switch comprises N optical transceiver modules, wherein each optical transceiver module is coupled to M independent optical cables, and wherein M×N independent optical cables are used to couple the respective switch to M×N other switches in the data center network.
The coupling mechanism of claim 14, wherein the data center comprises at least two levels of switches; wherein a respective port on a downlink of a first level is labeled using a 3-tuple (i ₁, j ₁, k ₁) with i, j, and k representing a switch sequence number, an optical module sequence number, and a port sequence number, respectively; and wherein the coupling mechanism is configured such that the port (i ₁, j ₁, k ₁) is coupled to a port (i ₂, j ₂, k ₂) belonging to an uplink of a second level, where i ₂=j ₁*M+k ₁, j ₂=i ₁%M, and k ₂=mod (i ₁, M) .
A method for coupling among switches in a data center network, the method comprising:

selecting a switch;

coupling a first end of a plurality of independent optical cables to a respective optical transceiver module on the selected switch; and

coupling a second end of the plurality of optical cables to a plurality of other switches in the data center network.
The method of claim 16, wherein the respective optical transceiver module comprises multiple SN-or MDC-based interfaces, and wherein coupling the first end of a respective optical cable to the respective optical transceiver module comprises coupling an SN-or MDC-based connector on the respective optical cable to a corresponding SN-or MDC-based interface, respectively, on the respective optical transceiver module.
The method of claim 16, wherein the respective optical transceiver module comprises an MPO optical interface, and wherein coupling the first end of a respective optical cable to the respective optical transceiver module comprises coupling an MPO connector or a duplex LC connector on the respective optical cable to the MPO optical interface.
The method of claim 16, wherein the selected switch comprises N optical transceiver modules, wherein the method further comprises:

coupling M independent optical cables to each optical transceiver module; and

coupling the selected switch to M×N other switches in the data center network.
The method of claim 19, wherein the data center comprises at least two levels of switches; wherein a respective port on a downlink of a first level is labeled using a 3-tuple (i ₁, j ₁, k ₁) with i, j, and k representing a switch sequence number, an optical module sequence number, and a port sequence number, respectively; and wherein the method further comprises coupling the port (i ₁, j ₁, k ₁) to a port (i ₂, j ₂, k ₂) belonging to an uplink of a second level, where i ₂=j ₁*M+k ₁, j ₂=i ₁%M, and k ₂=mod (i ₁, M) .