EP1732271A1 - Data communication system and method with virtual ports - Google Patents

Data communication system and method with virtual ports Download PDF

Info

Publication number
EP1732271A1
EP1732271A1 EP05012567A EP05012567A EP1732271A1 EP 1732271 A1 EP1732271 A1 EP 1732271A1 EP 05012567 A EP05012567 A EP 05012567A EP 05012567 A EP05012567 A EP 05012567A EP 1732271 A1 EP1732271 A1 EP 1732271A1
Authority
EP
European Patent Office
Prior art keywords
input
network
group
virtual sub
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP05012567A
Other languages
German (de)
French (fr)
Other versions
EP1732271B1 (en
Inventor
Norbert Dr. Eicker
Thomas Prof. Dr. Dr. Lippert
Jens Hauke
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Forschungszentrum Juelich GmbH
Partec Cluster Competence Center GmbH
Original Assignee
Forschungszentrum Juelich GmbH
Partec Cluster Competence Center GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to EP05012567A priority Critical patent/EP1732271B1/en
Application filed by Forschungszentrum Juelich GmbH, Partec Cluster Competence Center GmbH filed Critical Forschungszentrum Juelich GmbH
Priority to AT05012567T priority patent/ATE544266T1/en
Priority to ES05012567T priority patent/ES2381007T3/en
Priority to CN2006800204816A priority patent/CN101283550B/en
Priority to PCT/EP2006/005562 priority patent/WO2006131390A2/en
Priority to US11/916,844 priority patent/US9077648B2/en
Priority to JP2008515149A priority patent/JP4892550B2/en
Priority to KR1020077030537A priority patent/KR101228284B1/en
Priority to EP06754264A priority patent/EP1889414A2/en
Publication of EP1732271A1 publication Critical patent/EP1732271A1/en
Application granted granted Critical
Publication of EP1732271B1 publication Critical patent/EP1732271B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/24Multipath
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/44Star or tree networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00Routing or path finding of packets in data switching networks
    • H04L45/18Loop-free operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/15Interconnection of switching modules
    • H04L49/1515Non-blocking multistage, e.g. Clos
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00Packet switching elements
    • H04L49/35Switches specially adapted for specific applications
    • H04L49/354Switches specially adapted for specific applications for supporting virtual local area networks [VLAN]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40Network security protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/44Star or tree networks
    • H04L2012/445Star or tree networks with switching in a hub, e.g. ETHERNET switch

Definitions

  • the present invention refers to data communication systems.
  • a data communication system with a plurality of switching units and communication links forming a network having at least one loop, whereby the network is logically divided in at least two virtual sub-networks, each forming a spanning tree.
  • the present invention refers to a interconnecting network structure in an Ethernet environment.
  • Ethernet is turning out to be the preferred networking technology for recent deployments of Metropolitan Area Networks and Cluster networks. economies of scale, ease of service provisioning, high bandwidth, ease of interconnection with LANs, and scalability are some of the prominent reasons for this preferential status of Ethernets.
  • Ethernet is a preferred technology for metro and cluster networks, it has certain serious shortcomings.
  • the spanning tree based switching mechanism in Ethernets utilizes at most N - 1 links in a network of N switches. This limited utilization produces an imbalance of load which is impractical in MAN and cluster networks from a performance perspective.
  • Viking A Multi-Spanning-Tree Ethernet Architecture for Metropolitan Area and Cluster Networks
  • IEEE INFOCOM 2004 presents a system called 'Viking' addressing the aforementioned issues.
  • the core idea of the Viking system is to use multiple spanning trees in conjunction with VLAN technology to maximize the overall throughput performance of the network by utilizing multiple redundant links.
  • Viking provides fault tolerant features by providing a mechanism to divert the affected communication over to alternate paths after detecting failures. In effect, Viking strives to provide a fault-tolerant traffic engineering solution for Metro Ethernets and Cluster Networks.
  • Viking relies on Virtual LAN technology for selection of appropriate switching paths. VLANs are conventionally used to simplify network administration, reduce cost of segregation, and improve security.
  • Viking deviates from this conventional paradigm and uses tag based VLANs to select the desired switching path between a pair of end-hosts. All paths which can possibly be used as switching paths are absorbed in different spanning trees. Since each spanning tree instance corresponds to a particular VLAN, explicit selection of a VLAN results in an implicit selection of the switching path associated with the corresponding spanning tree. In case of failures, the end-hosts merely need to change the VLAN id in subsequent frames to select an alternate switching path. Each of the end-hosts needs to run a Viking Node Controller (VNC) module which is responsible for VLAN selection during network operation.
  • VNC Viking Node Controller
  • Object of the present invention is to provide a data communication system with improved applicability in different computing environments.
  • a data communication system having a plurality of input/output ports for processing incoming data packets arriving at a first input/output port to be out-put via a second input/output port.
  • the system of the present invention includes a first group of switching units and a second group of switching units. Each switching unit has a plurality of input/output interfaces, whereby at least one of the plurality of input/output interfaces of each switching unit of the first group forms an input/output port of the communications system itself.
  • each switching unit of the first group is connected via a communication link with an input/output interface of one of the switching units of the second group, so that a network is formed having at least one loop.
  • the network is logically divided in at least two virtual sub-networks.
  • each switching unit of the first group is configured to assign an incoming data packet arriving at a particular input/output port to a predetermined one of the at least two virtual sub-networks.
  • the active network elements of each of the respective virtual sub-networks form a spanning tree topology.
  • the present invention is directed to be implemented in an Ethernet environment. Hence, only spanning tree topology is allowed for each virtual sub-network. This is normally ensured by running the spanning tree protocol, which deactivates those links, which would form a loop. Therefore, it would be possible to define virtual sub-networks having loops, which would in the network start-up phase be removed. Preferably, the virtual sub-networks are already defined not to have loops. This would facilitate and quicken the start-up phase of the network for operation. Furthermore, it could be guaranteed which portions of each virtual sub-network are active during operation independently from the used algorithm in the spanning tree protocol.
  • the input/output port of the data communication system is basically an interface through which data are sent and received.
  • the input/output ports are used to connect the data communication system of the present invention to computer networks or single computing devices such as personal computers, server computers, mainframes and the like.
  • the communication system connects networks and/or computing devices attached to the individual input/output ports with each other.
  • each input/output port can be reached from another input/output port via different physical communication paths within the data communication system.
  • the internal structure of the communication system is arranged to have full bi-sectional bandwidth.
  • the data communication system processes data packets.
  • a data packet is a block of user data together with necessary address and administration information attached, to allow the network to deliver the data to the correct destination.
  • the data packet may be formed, e.g., by an IP (Internet Protocol) packet.
  • the internal structure of the data communication device is a network.
  • the network may be seen as a plurality of network segments, which are connected by the switching units.
  • the switching units allow a physical and logical star topology.
  • the switching units may be formed by a network switch. It should, however, be noted that more than one switching units may be implemented in one single physical network switch. Whenever a data packet enters a switching unit, the switching unit saves the originating MAC (Media Access Control) address and the originating input/output interface (also referred to as "port") in the switch's MAC address table. The switch then selectively transmits the data packet from specific input/output interfaces based on the data packet's destination MAC address. If the destination input/output interface is the same as the originating one, the data packet is filtered out and not forwarded.
  • MAC Media Access Control
  • the input/output interfaces of the switching units basically correspond to the ports of the respective network switch.
  • a first group and a second group of switching units are differentiated.
  • the first group of switching units include input/output interfaces forming input/output ports of the data communication system, i.e., they are reachable from the outside, by external computer networks and/or computing devices attached to the data communication system.
  • the second group of switching units do not share any input/output interfaces with the outside world of the data communication system. They are only connected via communication links to other switching units of the data communication system.
  • a multi-stage interconnect network is formed by the switching devices and the respective communication links.
  • Such a multi-stage interconnect network usually provides different physical communication paths between at least one pair of input/output ports and/or input/output interfaces.
  • Such a network topology is considered to have loops.
  • Logically dividing the network means to define a logical network on top of the physical network structure, in order to form virtual sub-networks.
  • Virtual sub-networks may have a different topology than the physical network topology.
  • the network topology form a spanning tree, i.e., a tree of a connected, undirected graph is which includes every vertex of that graph.
  • each defined virtual sub-network would anyhow only have active network elements, such as communication links and routing units, forming a network topology according to a spanning tree.
  • VLAN Virtual Local Area Network
  • a packet that is assigned to a virtual sub-network may only be routed in the virtual sub-network, i.e., forwarding of the packet functions correspondingly to a network only including the network elements, such as the switching units and communication links, of the virtual sub-network.
  • incoming data packets arriving at a particular input/output port are always assigned to the same predetermined one of the at least two virtual sub-networks.
  • the decision over which virtual sub-network a particular data packet is to be transported is solely dependent on the input/output interface through which the data packet entered the switching unit.
  • This allows to determine a virtual sub-network without the need to consult any central controlling unit or any external device.
  • data packet transmission is entirely transparent when looking from the outside at the data communication system.
  • the attached external networks and/or computing devices do not need to be aware of the internal functionality of the data communication system, no changes to such equipment need to be done, no additional software or hardware components need to be installed.
  • each of the other input/output interfaces of the switching units of the first group belong to one predetermined one of the at least two virtual sub-networks.
  • the other input/output interfaces are the ones connecting a switching unit of the first group ⁇ i.e., those switching units having connections to external networks and/or computing devices ⁇ with switching units of the second group ⁇ i.e., those switching units being connected via communication links with other switching devices, of the first and/or of the second group.
  • Data packets which are switched to another input/output interface being an input/output port, i.e., leading to the outside, of the same switch, are switched directly without being further routed through the internal network structure of the data communication system.
  • the data packet needs to be transported to the respective switching device. This is done by sending the data packet via the input/output interface belonging to the respective virtual sub-network the data packet got assigned to, over the respective communication link to another switching unit.
  • another switching step is necessary before the data packet reaches the switching unit that includes the input/output interface forming the destination input/output port.
  • Input/output interfaces of switching units of the first group, which are forming input/output ports of the system are members of all virtual sub-networks. This advantageously allows to forward data packets having another input/output port of the same switching unit as its destination, directly to the respective input/output port. Furthermore, data packets reaching a switching unit via an internal communication link may advantageously switched to any of the input/output ports.
  • each of the other input/output interfaces of a particular switching unit of the first group belongs to one single virtual sub-network. This advantageously reduces the risk of overloading one single input/output interface and the respectively connected communication link.
  • each of the other input/output interfaces of a particular switching unit of the first group belong to different virtual sub-networks. This advantageously distributes the load of data packets to be forwarded internally over all internal input/output interfaces of the respective switching unit.
  • each communication link only belongs to one virtual sub-network. This strict segregation of communication resources to different virtual sub-networks further decreases the risk of overloading single communication links. It should however be noted that a communication has duplex capability and it could be implemented that a particular link belongs to one virtual sub-network for one direction, whereas the other direction is assigned to a different virtual sub-network.
  • each of the other input/output interfaces of the switching units of the first group belong to the same virtual sub-network as the respectively connected communication link.
  • communication links and respectively connected input/output interfaces are members of the same virtual sub-network.
  • each virtual sub-network allows communication from any one of the switching units of the first group to all other switching units of the first group via at least one switching unit of the second group, assuming that the data packet is assigned to the corresponding virtual sub-network.
  • each virtual sub-network advantageously includes all input/output interfaces, which are forming input/output ports of the system.
  • each virtual sub-network advantageously includes a subset of other input/output interfaces and communication links, which allow to reach all other switching devices having input/output interfaces forming input/output ports, so that a data packet being assigned to its virtual sub-network is able to be routed to any input/output port of the data communication system.
  • all input/output ports of a particular switching unit of the second group belong to the same virtual sub-network. This allows to forward data packets without the need to assign them to a different virtual sub-network. However, it should be acknowledged that it could also be possible to assign data packets to a different virtual sub-network.
  • each switching unit of the second group belongs to a different virtual sub-network.
  • This facilitates the routing effort when setting up the system and significantly simplifies the processing within each switching unit.
  • the switching units basically function in a way as no virtual sub-network would be defined.
  • each switching unit of the first group has the same number of input/output interfaces forming input/output ports of the system and other input/output interfaces being connected via respective communication links with input/output interfaces of switching units of the second group.
  • the switching units of the second group advantageously have the same number of input/output interfaces as the ones of the first group.
  • each of the switching units of the second group is connected via a communication link with each of the switching units of the first group, as to form a star topology around the respective switching device.
  • the switching units of the second group would belong to the second stage. In this scenario the claimed configuration would then allow that from one second-stage switching device each of the switching devices of the first and third stage could be reached over the respective virtual sub-network.
  • the first and the second group of switching elements together with the communication links form a portion of a computer network.
  • the data communication system according to the present invention could therefore advantageously be employed as a portion of a larger computer network or a computer switch or an interconnecting network between computing devices.
  • Clos networks are a class of multi-stage switching network topologies that provide alternate paths between inputs and outputs, making it possible to minimize or eliminate the blocking that can otherwise occur in such networks.
  • other kind of networks may be used, such as Banyan, Delta, Butterfly or Benes networks.
  • one or more switching units may be formed by one physical network switch.
  • wire-based connections Although different wireless connections could be used for implementing the communication links, it is preferred to use wire-based connections, since they allow a higher bandwidth and better shielding between the different connections.
  • the virtual sub-networks are formed by Virtual Local Area Networks (VLAN) in an Ethernet environment.
  • VLAN Virtual Local Area Networks
  • Ethernet switches are frequently provided with VLAN capability, since this is normally used for network segregation for security or maintenance reason.
  • the present invention may advantageously be employed in a network switch.
  • a network switch may than be a portion of a computer network.
  • the data communication device may advantageously be employed to connect computing nodes of a computer cluster, in which at least a portion of the interconnecting computer network is formed by a data communication system according to the present invention.
  • a method for processing data packets in a data communication system with a plurality of input/output ports connected via a network having at least one loop, whereby at least two virtual sub-networks are provided, whereby in operation the active network elements of each of the virtual sub-networks have a network topology according to a spanning tree.
  • the method includes the following steps: Firstly, an incoming data packet arriving at a first input/output port of a first switching unit to be out-put via a destination input/output port is received. Then, in the first switching unit the incoming data packet is assigned to one of the at least two virtual sub-networks. Subsequently, the data packet is transmitted to the destination input/output port via the assigned virtual sub-network and out-put via the destination input/output port.
  • FIG. 1 there is depicted a system 100 according to the present invention using 8-port switches to implement a 16-port data communications system.
  • the system includes eight 8-port switches 110, 111, 112, 113, 114, 115, 116 and 117 forming the switching units of the system 100.
  • Each switch 110 to 117 is provided with eight input/output interfaces, the first switch 110 for example is provided with the input/output interfaces 110/0, 110/1, 110/2, 110/3, 110/4, 110/5, 110/6 and 110/7, the other input/output interfaces are denoted respectively.
  • the first switch 110, the second switch 111, the third switch 112 and the fourth switch 113 belong to a first group of switches.
  • Each of the switches 110 to 113 of the first group have four input/output interfaces providing input/output ports for connecting 16 computing devices 120 to 135 also referred to as computing nodes.
  • the first input/output interface 110/0 of the first switch 110 is, e.g., connected to the first node 120, whereas the sixteenth node 135 is connected to the fourth input/output interface 113/3 of the fourth switch 113, respectively.
  • Communication link 140 connects input/output interface 110/4 with input/output interface 114/0, whereas communication link 155 connects input/output interface 113/7 with input/output interface 117/3, respectively.
  • the fifth to the eighth switch 114 to 117 belong to a second group of switches only having internal connections to communications links. In other words, none of the input/output interfaces of those switches form an input/output port of the system. Furthermore it gets apparent from Fig. 1 that each switch of the first group 110 to 113 has one communication link connecting it to each of the switches of the second group 114 to 117. For example, from the first switch 110, communication link 140 connects to switch 114, connection link 141 connects to switch 115, communication link 142 connects to switch 116 and communication link 143 connects to switch 117. The other switches 111 to 113 have corresponding communication links 144 to 155 connecting them to switches 114 to 117 respectively.
  • the following virtual sub-networks are defined in the setup of Fig. 1.
  • the first virtual sub-network includes the following communication links: 140, 144, 148 and 152, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • the second virtual sub-network includes the following communication links: 141, 145, 149 and 153, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • the third virtual sub-network includes the following communication links: 142, 146, 150 and 154, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • the fourth virtual sub-network includes the following communication links: 143, 147, 151 and 155, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • Each switch of the first group 110 to 113 is configured to assign incoming data packets to a virtual sub-network, depending on the input/output port the packets were arriving.
  • data packets coming from the first node 120 arrive at input/output port 110/0 and are assigned to the first virtual sub-network.
  • Data packets coming from the second node 121 arrive at input/output port 110/1 and are assigned to the second virtual sub-network.
  • Data packets coming from the third node 122 arrive at input/output port 110/2 and are assigned to the third virtual sub-network.
  • Data packets coming from the fourth node 123 arrive at input/output port 110/3 and are assigned to the fourth virtual sub-network.
  • incoming data packets are assigned to the virtual data network the input/output interface belongs to which is drawn opposite of the input/output interface at which the data packet arrives. It should be acknowledged that bidirectional communication between to different nodes may us different virtual sub-networks per direction.
  • the switching units being implemented in switches 114 to 117 could be combined in a lower number of switches.
  • the switching units implemented in the switches 114 and 115 could be implemented in one single 8-port switch, and the switching units implemented in the switches 116 and 117 could be implemented in another single 8-port switch. This would lead to a configuration in which the entire data communication system would only require a total number of six 8-port switches instead of eight as shown in Fig. 1. It should be acknowledged that different hardware implementations of the present invention are possible.
  • Switch 210 has 8 input/output interfaces 210/0 to 210/7. Four of which forming input/output ports of the system, namely input/output interfaces 210/0 to 210/3. The remaining input/output interfaces 210/4 to 210/7 provide connection to internal communication links (not shown).
  • the input/output ports 210/0 to 210/3 are members of all virtual sub-networks, whereas each of the internal input/output interfaces 210/4 to 210/7 are members of a dedicated virtual sub-network, i.e., input/output interface 210/4 belongs to the first virtual sub-network, Virtual Local Area Network 1, input/output interface 210/5 belongs to the second virtual sub-network, Virtual Local Area Network 2, input/output interface 210/6 belongs to the third virtual sub-network, Virtual Local Area Network 3, and input/output interface 210/7 belongs to the forth virtual sub-network, Virtual Local Area Network 4.
  • the switch uses the default VLAN functionality to assign the respective virtual sub-network to the incoming data packets.
  • FIG. 3 depicting a system according to the present invention using 8-port switches to implement a 32-port data communications system.
  • the system includes twelve 8-port switches 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320 and 321 forming the switching units of the system 300.
  • Each switch 310 to 321 is provided with eight input/output interfaces, the first switch 310 for example is provided with the input/output interfaces 310/0, 310/1, 310/2, 310/3, 310/4, 310/5, 310/6 and 310/7, the other input/output interfaces are denoted respectively.
  • the first switch 310, the second switch 311, the third switch 312 and the fourth switch 313 as well as the ninth switch 318, the tenth switch 319, the eleventh switch 320 and the twelfth switch 321 belong to a first group of switches.
  • Each of the switches 310 to 313 and 318 to 321 of the first group have four input/output interfaces providing input/output ports for connecting 32 computing devices node0 to node31.
  • the first input/output interface 310/0 of the first switch 310 is, e.g., connected to the first node0, whereas the sixteenth node15 is connected to the fourth input/output interface 313/3 of the fourth switch 313, respectively.
  • the remaining four input/output interfaces of the switches 310 to 313 are used to attach communication links 340 to 355.
  • Communication link 340 connects input/output interface 310/4 with input/output interface 314/0, whereas communication link 355 connects input/output interface 313/7 with input/output interface 317/3, respectively.
  • Communication link 360 connects input/output interface 318/0 with input/output interface 314/4, whereas communication link 375 connects input/output interface 321/3 with input/output interface 317/7, respectively.
  • the fifth to the eighth switch 314 to 317 belong to a second group of switches only having internal connections to communications links. In other words, none of the input/output interfaces of those switches form an input/output port of the system. Furthermore it gets apparent from Fig. 3 that each switch of the first group 310 to 313 and 318 to 321 has one communication link connecting it to each of the switches of the second group 314 to 317. For example, from the first switch 310, communication link 340 connects to switch 314, connection link 341 connects to switch 315, communication link 342 connects to switch 316 and communication link 343 connects to switch 317. The other switches 311 to 313 have corresponding communication links 344 to 355 connecting them to switches 314 to 317 respectively.
  • the following virtual sub-networks are defined in the setup of Fig. 3.
  • the first virtual sub-network includes the following communication links: 340, 344, 348, 352, 360, 364, 368 and 372 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • the second virtual sub-network includes the following communication links: 341, 345, 349, 353, 361, 365, 369 and 373 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • the third virtual sub-network includes the following communication links: 342, 346, 350, 354, 362, 366, 370 and 374 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • the fourth virtual sub-network includes the following communication links: 343, 347, 351, 355, 363, 367, 371 and 375 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • Each switch of the first group 310 to 313 and 318 to 321 is configured to assign incoming data packets to a virtual sub-network, depending on the input/output port the packets were arriving.
  • data packets coming from the first node 320 arrive at input/output port 310/0 and are assigned to the first virtual sub-network.
  • Data packets coming from the second node 321 arrive at input/output port 310/1 and are assigned to the second virtual sub-network.
  • Data packets coming from the third node 322 arrive at input/output port 310/2 and are assigned to the third virtual sub-network.
  • Data packets coming from the fourth node 323 arrive at input/output port 310/3 and are assigned to the fourth virtual sub-network.
  • Data packets coming from the fifth node 324 arrive at input/output port 311/0 and are assigned to the first virtual sub-network again.
  • incoming data packets are assigned to the virtual data network the input/output interface belongs to which is drawn opposite of the input/output interface at which the data packet arrives. It should be acknowledged that bidirectional communication between to different nodes may us different virtual sub-networks per direction.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Small-Scale Networks (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

A data communication system having a plurality of input/output ports, a first group and a second group of switching units. Each switching unit has a plurality of input/output interfaces, whereby at least one of the plurality of input/output interfaces of each switching unit of the first group forms an input/output port of the communications system, and at least one other input/output interface of each of the switching units of the first group is connected via a communication link with an input/output interface of one of the switching units of the second group, so that a network is formed having at least one loop. The network is logically divided in at least two virtual sub-networks, each forming a spanning tree. Each switching unit of the first group is configured to assign an incoming data packet arriving at a particular input/output port to a predetermined one of the at least two virtual sub-networks. An advantage of the present invention is the ability to transparently employ the taught data communication device in different computing environments, without the requirement to alter any external equipment.

Description

    Background of the Invention
  • The present invention refers to data communication systems. In particular, to a data communication system with a plurality of switching units and communication links forming a network having at least one loop, whereby the network is logically divided in at least two virtual sub-networks, each forming a spanning tree. More particularly, the present invention refers to a interconnecting network structure in an Ethernet environment.
  • Ethernet is turning out to be the preferred networking technology for recent deployments of Metropolitan Area Networks and Cluster networks. Economies of scale, ease of service provisioning, high bandwidth, ease of interconnection with LANs, and scalability are some of the prominent reasons for this preferential status of Ethernets.
  • Though Ethernet is a preferred technology for metro and cluster networks, it has certain serious shortcomings. Primarily, the spanning tree based switching mechanism in Ethernets utilizes at most N - 1 links in a network of N switches. This limited utilization produces an imbalance of load which is impractical in MAN and cluster networks from a performance perspective.
  • Sharma et al.: "Viking: A Multi-Spanning-Tree Ethernet Architecture for Metropolitan Area and Cluster Networks", IEEE INFOCOM 2004, presents a system called 'Viking' addressing the aforementioned issues. The core idea of the Viking system is to use multiple spanning trees in conjunction with VLAN technology to maximize the overall throughput performance of the network by utilizing multiple redundant links. Further, Viking provides fault tolerant features by providing a mechanism to divert the affected communication over to alternate paths after detecting failures. In effect, Viking strives to provide a fault-tolerant traffic engineering solution for Metro Ethernets and Cluster Networks. Viking relies on Virtual LAN technology for selection of appropriate switching paths. VLANs are conventionally used to simplify network administration, reduce cost of segregation, and improve security. Viking deviates from this conventional paradigm and uses tag based VLANs to select the desired switching path between a pair of end-hosts. All paths which can possibly be used as switching paths are absorbed in different spanning trees. Since each spanning tree instance corresponds to a particular VLAN, explicit selection of a VLAN results in an implicit selection of the switching path associated with the corresponding spanning tree. In case of failures, the end-hosts merely need to change the VLAN id in subsequent frames to select an alternate switching path. Each of the end-hosts needs to run a Viking Node Controller (VNC) module which is responsible for VLAN selection during network operation.
  • Object of the present invention is to provide a data communication system with improved applicability in different computing environments.
  • Summary of the Invention
  • The object is achieved by a system and a method according to the independent claims. According to the present invention, a data communication system is provided having a plurality of input/output ports for processing incoming data packets arriving at a first input/output port to be out-put via a second input/output port. The system of the present invention includes a first group of switching units and a second group of switching units. Each switching unit has a plurality of input/output interfaces, whereby at least one of the plurality of input/output interfaces of each switching unit of the first group forms an input/output port of the communications system itself. On the other hand, at least one other input/output interface of each of the switching units of the first group is connected via a communication link with an input/output interface of one of the switching units of the second group, so that a network is formed having at least one loop. The network is logically divided in at least two virtual sub-networks. According to the present invention, each switching unit of the first group is configured to assign an incoming data packet arriving at a particular input/output port to a predetermined one of the at least two virtual sub-networks. An advantage of the present invention is the ability to transparently employ the taught data communication device in different computing environments, such as computing clusters etc., without the requirement to alter any external equipment.
  • In operation, the active network elements of each of the respective virtual sub-networks form a spanning tree topology. As aforementioned, the present invention is directed to be implemented in an Ethernet environment. Hence, only spanning tree topology is allowed for each virtual sub-network. This is normally ensured by running the spanning tree protocol, which deactivates those links, which would form a loop. Therefore, it would be possible to define virtual sub-networks having loops, which would in the network start-up phase be removed. Preferably, the virtual sub-networks are already defined not to have loops. This would facilitate and quicken the start-up phase of the network for operation. Furthermore, it could be guaranteed which portions of each virtual sub-network are active during operation independently from the used algorithm in the spanning tree protocol.
  • The input/output port of the data communication system is basically an interface through which data are sent and received. The input/output ports are used to connect the data communication system of the present invention to computer networks or single computing devices such as personal computers, server computers, mainframes and the like. In other words, the communication system connects networks and/or computing devices attached to the individual input/output ports with each other.
  • In order to achieve a high bandwidth, each input/output port can be reached from another input/output port via different physical communication paths within the data communication system. Preferably, the internal structure of the communication system is arranged to have full bi-sectional bandwidth.
  • The data communication system processes data packets. A data packet is a block of user data together with necessary address and administration information attached, to allow the network to deliver the data to the correct destination. The data packet may be formed, e.g., by an IP (Internet Protocol) packet.
  • As already mentioned above, the internal structure of the data communication device is a network. The network may be seen as a plurality of network segments, which are connected by the switching units. The switching units allow a physical and logical star topology.
  • The switching units may be formed by a network switch. It should, however, be noted that more than one switching units may be implemented in one single physical network switch. Whenever a data packet enters a switching unit, the switching unit saves the originating MAC (Media Access Control) address and the originating input/output interface (also referred to as "port") in the switch's MAC address table. The switch then selectively transmits the data packet from specific input/output interfaces based on the data packet's destination MAC address. If the destination input/output interface is the same as the originating one, the data packet is filtered out and not forwarded.
  • The input/output interfaces of the switching units basically correspond to the ports of the respective network switch.
  • A first group and a second group of switching units are differentiated. According to the claimed invention, the first group of switching units include input/output interfaces forming input/output ports of the data communication system, i.e., they are reachable from the outside, by external computer networks and/or computing devices attached to the data communication system. The second group of switching units do not share any input/output interfaces with the outside world of the data communication system. They are only connected via communication links to other switching units of the data communication system. Hence, a multi-stage interconnect network is formed by the switching devices and the respective communication links.
  • Such a multi-stage interconnect network usually provides different physical communication paths between at least one pair of input/output ports and/or input/output interfaces. Such a network topology is considered to have loops.
  • Logically dividing the network means to define a logical network on top of the physical network structure, in order to form virtual sub-networks. Virtual sub-networks may have a different topology than the physical network topology. According to the present invention, the network topology form a spanning tree, i.e., a tree of a connected, undirected graph is which includes every vertex of that graph.
  • It should be acknowledged that the virtual sub-networks could also be defined to have loops. However, since a spanning tree protocol is executed per virtual sub-network, i.e., per Virtual Local Area Network (VLAN), in operation each defined virtual sub-network would anyhow only have active network elements, such as communication links and routing units, forming a network topology according to a spanning tree.
  • A packet that is assigned to a virtual sub-network may only be routed in the virtual sub-network, i.e., forwarding of the packet functions correspondingly to a network only including the network elements, such as the switching units and communication links, of the virtual sub-network.
  • Advantageously, incoming data packets arriving at a particular input/output port are always assigned to the same predetermined one of the at least two virtual sub-networks. Hence, the decision over which virtual sub-network a particular data packet is to be transported is solely dependent on the input/output interface through which the data packet entered the switching unit. This allows to determine a virtual sub-network without the need to consult any central controlling unit or any external device. Furthermore, data packet transmission is entirely transparent when looking from the outside at the data communication system. The attached external networks and/or computing devices do not need to be aware of the internal functionality of the data communication system, no changes to such equipment need to be done, no additional software or hardware components need to be installed.
  • According to a preferred embodiment of the present invention, each of the other input/output interfaces of the switching units of the first group belong to one predetermined one of the at least two virtual sub-networks. As aforementioned the other input/output interfaces are the ones connecting a switching unit of the first group ― i.e., those switching units having connections to external networks and/or computing devices ― with switching units of the second group ― i.e., those switching units being connected via communication links with other switching devices, of the first and/or of the second group. Data packets which are switched to another input/output interface being an input/output port, i.e., leading to the outside, of the same switch, are switched directly without being further routed through the internal network structure of the data communication system. However, if the destination input/output port is situated at another switching device, the data packet needs to be transported to the respective switching device. This is done by sending the data packet via the input/output interface belonging to the respective virtual sub-network the data packet got assigned to, over the respective communication link to another switching unit. In a three-stage interconnecting network, another switching step is necessary before the data packet reaches the switching unit that includes the input/output interface forming the destination input/output port.
  • It should be acknowledged that also multi-stage interconnecting networks with more than tree stages may be used. In this case additional switching and transmission steps within the data communication system would be necessary. However, in order to keep the latency for transporting the data packets from an incoming input/output port to an outgoing input/output port low, the number of stages should be kept small.
  • Input/output interfaces of switching units of the first group, which are forming input/output ports of the system are members of all virtual sub-networks. This advantageously allows to forward data packets having another input/output port of the same switching unit as its destination, directly to the respective input/output port. Furthermore, data packets reaching a switching unit via an internal communication link may advantageously switched to any of the input/output ports.
  • In a preferred embodiment of the present invention, each of the other input/output interfaces of a particular switching unit of the first group belongs to one single virtual sub-network. This advantageously reduces the risk of overloading one single input/output interface and the respectively connected communication link.
  • In a further advantageous embodiment, each of the other input/output interfaces of a particular switching unit of the first group belong to different virtual sub-networks. This advantageously distributes the load of data packets to be forwarded internally over all internal input/output interfaces of the respective switching unit.
  • According to the present invention, each communication link only belongs to one virtual sub-network. This strict segregation of communication resources to different virtual sub-networks further decreases the risk of overloading single communication links. It should however be noted that a communication has duplex capability and it could be implemented that a particular link belongs to one virtual sub-network for one direction, whereas the other direction is assigned to a different virtual sub-network.
  • Advantageously, each of the other input/output interfaces of the switching units of the first group belong to the same virtual sub-network as the respectively connected communication link. In other words, communication links and respectively connected input/output interfaces are members of the same virtual sub-network.
  • According to a preferred embodiment of the present invention, each virtual sub-network allows communication from any one of the switching units of the first group to all other switching units of the first group via at least one switching unit of the second group, assuming that the data packet is assigned to the corresponding virtual sub-network. I.e., each virtual sub-network advantageously includes all input/output interfaces, which are forming input/output ports of the system. Furthermore, each virtual sub-network advantageously includes a subset of other input/output interfaces and communication links, which allow to reach all other switching devices having input/output interfaces forming input/output ports, so that a data packet being assigned to its virtual sub-network is able to be routed to any input/output port of the data communication system.
  • Advantageously, all input/output ports of a particular switching unit of the second group belong to the same virtual sub-network. This allows to forward data packets without the need to assign them to a different virtual sub-network. However, it should be acknowledged that it could also be possible to assign data packets to a different virtual sub-network.
  • In a preferred embodiment of the present invention, there is also a segregation of entire switching units to different virtual sub-networks, i.e., each switching unit of the second group belongs to a different virtual sub-network. This facilitates the routing effort when setting up the system and significantly simplifies the processing within each switching unit. The switching units basically function in a way as no virtual sub-network would be defined.
  • Advantageously, each switching unit of the first group has the same number of input/output interfaces forming input/output ports of the system and other input/output interfaces being connected via respective communication links with input/output interfaces of switching units of the second group. In a preferred configuration each switching device has N = 2·n input/output interfaces with (n = 2, 3,...), whereby N/2 input/output interfaces form input/output ports and the other N/2 input/output interfaces are used for internal connections.
  • The switching units of the second group advantageously have the same number of input/output interfaces as the ones of the first group.
  • In a preferred embodiment of the present invention, each of the switching units of the second group is connected via a communication link with each of the switching units of the first group, as to form a star topology around the respective switching device. In a three-stage interconnecting network, the switching units of the second group would belong to the second stage. In this scenario the claimed configuration would then allow that from one second-stage switching device each of the switching devices of the first and third stage could be reached over the respective virtual sub-network.
  • As aforementioned, the first and the second group of switching elements together with the communication links form a portion of a computer network. Hence, the data communication system according to the present invention could therefore advantageously be employed as a portion of a larger computer network or a computer switch or an interconnecting network between computing devices.
  • In a preferred embodiment of the present invention, the first and the second group of switching elements together with the communication links are arranged to form a Clos network (also known as Omega network). Clos networks are a class of multi-stage switching network topologies that provide alternate paths between inputs and outputs, making it possible to minimize or eliminate the blocking that can otherwise occur in such networks. Alternatively, other kind of networks may be used, such as Banyan, Delta, Butterfly or Benes networks.
  • When implementing a data communication system according to the present invention, one or more switching units may be formed by one physical network switch.
  • Although different wireless connections could be used for implementing the communication links, it is preferred to use wire-based connections, since they allow a higher bandwidth and better shielding between the different connections.
  • Advantageously, the virtual sub-networks are formed by Virtual Local Area Networks (VLAN) in an Ethernet environment. Ethernet switches are frequently provided with VLAN capability, since this is normally used for network segregation for security or maintenance reason.
  • The present invention may advantageously be employed in a network switch. Such a network switch may than be a portion of a computer network.
  • Furthermore, the data communication device according to the present invention may advantageously be employed to connect computing nodes of a computer cluster, in which at least a portion of the interconnecting computer network is formed by a data communication system according to the present invention. The same applies to an interconnecting network used in a computer grid.
  • According to the present invention a method is provided for processing data packets in a data communication system with a plurality of input/output ports connected via a network having at least one loop, whereby at least two virtual sub-networks are provided, whereby in operation the active network elements of each of the virtual sub-networks have a network topology according to a spanning tree. The method includes the following steps: Firstly, an incoming data packet arriving at a first input/output port of a first switching unit to be out-put via a destination input/output port is received. Then, in the first switching unit the incoming data packet is assigned to one of the at least two virtual sub-networks. Subsequently, the data packet is transmitted to the destination input/output port via the assigned virtual sub-network and out-put via the destination input/output port.
  • Brief Description of the Drawings
  • As aforementioned a key advantage of the present invention is the provision of a data communication system which may easily be employed. The following presented preferred embodiments constitute only a small selection of possible implementations. They will now be described with reference to the attached figures, wherein:
    • Fig 1 shows a system according to the present invention using 8-port switches to implement a 16-port data communications system;
    • Fig. 2 shows a detail view of an 8-port switch as used in the systems according to the present invention; and
    • Fig. 3 shows a system according to the present invention using 8-port switches to implement a 32-port data communications system.
    Detailed Description
  • With reference to Fig. 1, there is depicted a system 100 according to the present invention using 8-port switches to implement a 16-port data communications system. The system includes eight 8- port switches 110, 111, 112, 113, 114, 115, 116 and 117 forming the switching units of the system 100. Each switch 110 to 117 is provided with eight input/output interfaces, the first switch 110 for example is provided with the input/output interfaces 110/0, 110/1, 110/2, 110/3, 110/4, 110/5, 110/6 and 110/7, the other input/output interfaces are denoted respectively.
  • The first switch 110, the second switch 111, the third switch 112 and the fourth switch 113 belong to a first group of switches. Each of the switches 110 to 113 of the first group have four input/output interfaces providing input/output ports for connecting 16 computing devices 120 to 135 also referred to as computing nodes. The first input/output interface 110/0 of the first switch 110 is, e.g., connected to the first node 120, whereas the sixteenth node 135 is connected to the fourth input/output interface 113/3 of the fourth switch 113, respectively.
  • The remaining four input/output interfaces of the switches 110 to 113 are used to attach communication links 140 to 155. Communication link 140 connects input/output interface 110/4 with input/output interface 114/0, whereas communication link 155 connects input/output interface 113/7 with input/output interface 117/3, respectively.
  • The fifth to the eighth switch 114 to 117 belong to a second group of switches only having internal connections to communications links. In other words, none of the input/output interfaces of those switches form an input/output port of the system. Furthermore it gets apparent from Fig. 1 that each switch of the first group 110 to 113 has one communication link connecting it to each of the switches of the second group 114 to 117. For example, from the first switch 110, communication link 140 connects to switch 114, connection link 141 connects to switch 115, communication link 142 connects to switch 116 and communication link 143 connects to switch 117. The other switches 111 to 113 have corresponding communication links 144 to 155 connecting them to switches 114 to 117 respectively.
  • The following virtual sub-networks are defined in the setup of Fig. 1. The first virtual sub-network includes the following communication links: 140, 144, 148 and 152, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • The second virtual sub-network includes the following communication links: 141, 145, 149 and 153, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • The third virtual sub-network includes the following communication links: 142, 146, 150 and 154, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • The fourth virtual sub-network includes the following communication links: 143, 147, 151 and 155, and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • Each switch of the first group 110 to 113 is configured to assign incoming data packets to a virtual sub-network, depending on the input/output port the packets were arriving. In the presently depicted implementation example, data packets coming from the first node 120 arrive at input/output port 110/0 and are assigned to the first virtual sub-network. Data packets coming from the second node 121 arrive at input/output port 110/1 and are assigned to the second virtual sub-network. Data packets coming from the third node 122 arrive at input/output port 110/2 and are assigned to the third virtual sub-network. Data packets coming from the fourth node 123 arrive at input/output port 110/3 and are assigned to the fourth virtual sub-network. As apparent from this description, incoming data packets are assigned to the virtual data network the input/output interface belongs to which is drawn opposite of the input/output interface at which the data packet arrives. It should be acknowledged that bidirectional communication between to different nodes may us different virtual sub-networks per direction.
  • Since only four input/output interfaces of the switches 114 to 117 are used according to the example of Fig. 1, the switching units being implemented in switches 114 to 117 could be combined in a lower number of switches. E.g., the switching units implemented in the switches 114 and 115 could be implemented in one single 8-port switch, and the switching units implemented in the switches 116 and 117 could be implemented in another single 8-port switch. This would lead to a configuration in which the entire data communication system would only require a total number of six 8-port switches instead of eight as shown in Fig. 1. It should be acknowledged that different hardware implementations of the present invention are possible.
  • With reference now to Fig. 2, there is depicted a detail view of an 8-port switch 210 as used in the systems according to Fig. 1 and 3. Switch 210 has 8 input/output interfaces 210/0 to 210/7. Four of which forming input/output ports of the system, namely input/output interfaces 210/0 to 210/3. The remaining input/output interfaces 210/4 to 210/7 provide connection to internal communication links (not shown).
  • As already explained in view of Fig. 1, the input/output ports 210/0 to 210/3 are members of all virtual sub-networks, whereas each of the internal input/output interfaces 210/4 to 210/7 are members of a dedicated virtual sub-network, i.e., input/output interface 210/4 belongs to the first virtual sub-network, Virtual Local Area Network 1, input/output interface 210/5 belongs to the second virtual sub-network, Virtual Local Area Network 2, input/output interface 210/6 belongs to the third virtual sub-network, Virtual Local Area Network 3, and input/output interface 210/7 belongs to the forth virtual sub-network, Virtual Local Area Network 4.
  • The switch uses the default VLAN functionality to assign the respective virtual sub-network to the incoming data packets.
  • Now with reference to Fig. 3 depicting a system according to the present invention using 8-port switches to implement a 32-port data communications system. The system includes twelve 8- port switches 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320 and 321 forming the switching units of the system 300. Each switch 310 to 321 is provided with eight input/output interfaces, the first switch 310 for example is provided with the input/output interfaces 310/0, 310/1, 310/2, 310/3, 310/4, 310/5, 310/6 and 310/7, the other input/output interfaces are denoted respectively.
  • The first switch 310, the second switch 311, the third switch 312 and the fourth switch 313 as well as the ninth switch 318, the tenth switch 319, the eleventh switch 320 and the twelfth switch 321 belong to a first group of switches. Each of the switches 310 to 313 and 318 to 321 of the first group have four input/output interfaces providing input/output ports for connecting 32 computing devices node0 to node31. The first input/output interface 310/0 of the first switch 310 is, e.g., connected to the first node0, whereas the sixteenth node15 is connected to the fourth input/output interface 313/3 of the fourth switch 313, respectively.
  • The remaining four input/output interfaces of the switches 310 to 313 are used to attach communication links 340 to 355. Communication link 340 connects input/output interface 310/4 with input/output interface 314/0, whereas communication link 355 connects input/output interface 313/7 with input/output interface 317/3, respectively.
  • The remaining four input/output interfaces of the switches 318 to 321 are used to attach communication links 360 to 375. Communication link 360 connects input/output interface 318/0 with input/output interface 314/4, whereas communication link 375 connects input/output interface 321/3 with input/output interface 317/7, respectively.
  • The fifth to the eighth switch 314 to 317 belong to a second group of switches only having internal connections to communications links. In other words, none of the input/output interfaces of those switches form an input/output port of the system. Furthermore it gets apparent from Fig. 3 that each switch of the first group 310 to 313 and 318 to 321 has one communication link connecting it to each of the switches of the second group 314 to 317. For example, from the first switch 310, communication link 340 connects to switch 314, connection link 341 connects to switch 315, communication link 342 connects to switch 316 and communication link 343 connects to switch 317. The other switches 311 to 313 have corresponding communication links 344 to 355 connecting them to switches 314 to 317 respectively.
  • The following virtual sub-networks are defined in the setup of Fig. 3. The first virtual sub-network includes the following communication links: 340, 344, 348, 352, 360, 364, 368 and 372 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • The second virtual sub-network includes the following communication links: 341, 345, 349, 353, 361, 365, 369 and 373 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • The third virtual sub-network includes the following communication links: 342, 346, 350, 354, 362, 366, 370 and 374 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • The fourth virtual sub-network includes the following communication links: 343, 347, 351, 355, 363, 367, 371 and 375 and all input/output interfaces connected with those communication links. Additionally all input/output interfaces forming input/output ports.
  • Each switch of the first group 310 to 313 and 318 to 321 is configured to assign incoming data packets to a virtual sub-network, depending on the input/output port the packets were arriving. In the presently depicted implementation example, data packets coming from the first node 320 arrive at input/output port 310/0 and are assigned to the first virtual sub-network. Data packets coming from the second node 321 arrive at input/output port 310/1 and are assigned to the second virtual sub-network. Data packets coming from the third node 322 arrive at input/output port 310/2 and are assigned to the third virtual sub-network. Data packets coming from the fourth node 323 arrive at input/output port 310/3 and are assigned to the fourth virtual sub-network. Data packets coming from the fifth node 324 arrive at input/output port 311/0 and are assigned to the first virtual sub-network again. As apparent from this description, incoming data packets are assigned to the virtual data network the input/output interface belongs to which is drawn opposite of the input/output interface at which the data packet arrives. It should be acknowledged that bidirectional communication between to different nodes may us different virtual sub-networks per direction.

Claims (23)

  1. A data communication system with a plurality of input/output ports for processing incoming data packets arriving at a first input/output port to be out-put via a second input/output port, said system including:
    a first group of switching units and
    a second group of switching units,
    whereby each switching unit has a plurality of input/output interfaces,
    whereby at least one of said plurality of input/output interfaces of each switching unit of said first group forms an input/output port of said communications system,
    whereby at least one other input/output interface of each of said switching units of said first group is connected via a communication link with an input/output interface of one of the switching units of said second group, so that a network is formed having at least one loop,
    whereby the network is logically divided in at least two virtual sub-networks, whereby in operation the active network elements of each virtual sub-network have a spanning tree topology, characterized in that
    each switching unit of said first group is configured to assign an incoming data packet arriving at a particular input/output port to a predetermined one of said at least two virtual sub-networks.
  2. The system according to claim 1, characterized in that incoming data packets arriving at a particular input/output port are always assigned to the same predetermined one of said at least two virtual sub-networks.
  3. The system according to one of the preceding claims, characterized in that each of said other input/output interfaces of said switching units of said first group belong to one predetermined one of said at least two virtual sub-networks.
  4. The system according to one of the preceding claims, characterized in that the input/output interfaces of the switching units of said first group forming input/output ports of the system belong to all virtual sub-networks.
  5. The system according to one of the preceding claims, characterized in that each of the other input/output interfaces of a particular switching unit of said first group belongs to one single virtual sub-network.
  6. The system according to one of the preceding claims, characterized in that each of said other input/output interfaces of a particular switching unit of said first group belongs to different virtual sub-networks.
  7. The system according to one of the preceding claims, characterized in that each communication link only belongs to one virtual sub-network.
  8. The system according to claim 7, characterized in that each of said other input/output interfaces of the switching units of said first group belongs to the same virtual sub-network as the respectively connected communication link.
  9. The system according to one of the preceding claims, characterized in that each virtual sub-network allows communication from any one of the switching units of said first group to all other switching units of said first group via at least one switching unit of said second group.
  10. The system according to one of the preceding claims, characterized in that all input/output ports of a particular switching unit of said second group belong to the same virtual sub-network.
  11. The system according to claim 10, characterized in that each switching unit of said second group belong to a different virtual sub-network.
  12. The system according to one of the preceding claims, characterized in that each switching unit of said first group has the same number of input/output interfaces forming input/output ports of the system and other input/output interfaces being connected via respective communication links with input/output interfaces of switching units of said second group.
  13. The system according to one of the preceding claims, characterized in that each of said switching units of said second group is connected via a communication link with each of the switching units of said first group.
  14. The system according to one of the preceding claims, characterized in that said first and said second group of switching elements together with said communication links form a portion of a computer network.
  15. The system according to one of the preceding claims, characterized in that said first and said second group of switching elements together with said communication links are arranged to form a clos network.
  16. The system according to one of the preceding claims, characterized in that one or more switching units may be formed by one physical network switch.
  17. The system according to one of the preceding claims, characterized in that said communication links are formed by wire-based connections.
  18. The system according to one of the preceding claims, characterized in that said virtual sub-networks are formed by Virtual Local Area Networks in an Ethernet environment.
  19. A network switch being formed by a data communication system according to one of the preceding claims.
  20. A computer network including a network switch according to claim 19.
  21. A computer cluster including a plurality of computing nodes being connected via a computer network, characterized in that at least a portion of said computer network is formed by a data communication system according to one of the claims 1 to 18.
  22. A computer grid including a data communication system according to one of the claims 1 to 18.
  23. A method for processing data packets in a data communication system with a plurality of input/output ports connected via a network having at least one loop, whereby at least two virtual sub-networks are provided, whereby in operation the active network elements of each virtual sub-network have a spanning tree topology, the method including the following steps:
    receiving an incoming data packet arriving at a first input/output port of a first switching unit to be out-put via a destination input/output port;
    assigning in said first switching unit said incoming data packet to one of said at least two virtual sub-networks;
    transmitting said data packet to said destination input/output port via said assigned virtual sub-network; and
    out-putting said data packet via said destination input/output port.
EP05012567A 2005-06-10 2005-06-10 Data communication system and method with virtual ports Active EP1732271B1 (en)

Priority Applications (9)

Application Number Priority Date Filing Date Title
AT05012567T ATE544266T1 (en) 2005-06-10 2005-06-10 DATA COMMUNICATION METHOD AND SYSTEM WITH VIRTUAL PORTS
ES05012567T ES2381007T3 (en) 2005-06-10 2005-06-10 Method and data communication system with virtual ports
EP05012567A EP1732271B1 (en) 2005-06-10 2005-06-10 Data communication system and method with virtual ports
PCT/EP2006/005562 WO2006131390A2 (en) 2005-06-10 2006-06-09 Data communication system and method with virtual ports
CN2006800204816A CN101283550B (en) 2005-06-10 2006-06-09 Data communication system and method with virtual ports
US11/916,844 US9077648B2 (en) 2005-06-10 2006-06-09 Data communication system with multiple input/outputs and method of processing data packets
JP2008515149A JP4892550B2 (en) 2005-06-10 2006-06-09 Data communication system and method
KR1020077030537A KR101228284B1 (en) 2005-06-10 2006-06-09 Data communication system and method
EP06754264A EP1889414A2 (en) 2005-06-10 2006-06-09 Data communication system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP05012567A EP1732271B1 (en) 2005-06-10 2005-06-10 Data communication system and method with virtual ports

Publications (2)

Publication Number Publication Date
EP1732271A1 true EP1732271A1 (en) 2006-12-13
EP1732271B1 EP1732271B1 (en) 2012-02-01

Family

ID=36601223

Family Applications (2)

Application Number Title Priority Date Filing Date
EP05012567A Active EP1732271B1 (en) 2005-06-10 2005-06-10 Data communication system and method with virtual ports
EP06754264A Withdrawn EP1889414A2 (en) 2005-06-10 2006-06-09 Data communication system and method

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP06754264A Withdrawn EP1889414A2 (en) 2005-06-10 2006-06-09 Data communication system and method

Country Status (8)

Country Link
US (1) US9077648B2 (en)
EP (2) EP1732271B1 (en)
JP (1) JP4892550B2 (en)
KR (1) KR101228284B1 (en)
CN (1) CN101283550B (en)
AT (1) ATE544266T1 (en)
ES (1) ES2381007T3 (en)
WO (1) WO2006131390A2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009039384A2 (en) * 2007-09-20 2009-03-26 C & S Operations, Inc. Computer system with multiple terminals
US7983253B2 (en) * 2007-04-20 2011-07-19 Nuon, Inc. Time-space carrier sense multiple access
US20130003550A1 (en) * 2011-06-29 2013-01-03 Broadcom Corporation System and Method for Priority Based Flow Control Between Nodes

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8000266B2 (en) * 2006-12-19 2011-08-16 Cisco Technology, Inc. Shared virtual device ports
KR101232601B1 (en) * 2009-12-18 2013-02-12 한국전자통신연구원 Communication node in combined node system and communication method thereof
CN101873478B (en) * 2010-06-18 2013-07-17 杭州海康威视数字技术股份有限公司 Control method and related device of network monitoring system
US8837517B2 (en) 2010-09-22 2014-09-16 Amazon Technologies, Inc. Transpose boxes for network interconnection
US8621111B2 (en) 2010-09-22 2013-12-31 Amazon Technologies, Inc. Transpose box based network scaling
US20120250679A1 (en) * 2011-03-29 2012-10-04 Amazon Technologies, Inc. Network Transpose Box and Switch Operation Based on Backplane Ethernet
US9363208B1 (en) 2011-03-29 2016-06-07 Amazon Technologies, Inc. Logical switches
JP5720524B2 (en) 2011-10-12 2015-05-20 富士通株式会社 RELAY PROGRAM, RELAY DEVICE, AND CONTROL METHOD
US8848509B2 (en) * 2012-04-27 2014-09-30 Telefonaktiebolaget L M Ericsson (Publ) Three stage folded Clos optimization for 802.1aq
CN103841055A (en) * 2012-11-22 2014-06-04 西安邮电大学 ODU2 data sequence rearrangement circuit
US9860116B1 (en) * 2013-06-18 2018-01-02 Cisco Technology, Inc. Physical network orchestration for data centers
DE102014219472A1 (en) * 2014-09-25 2016-03-31 Siemens Aktiengesellschaft Method for transmitting data, network nodes and network
CN105262704B (en) * 2015-09-15 2018-02-23 中国人民解放军信息工程大学 Mapping device and its mapping method based on controller and interchanger
CN113131996B (en) * 2018-12-06 2022-07-12 长沙天仪空间科技研究院有限公司 Communication optimization method based on ground station

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1049299A1 (en) * 1999-04-30 2000-11-02 Nortel Networks Limited Method and apparatus for bandwidth management of aggregate data flows
US20020163922A1 (en) * 2001-05-01 2002-11-07 Dooley David L. Network switch port traffic manager having configurable packet and cell servicing
WO2005015855A1 (en) * 2003-07-28 2005-02-17 Diseño De Sistemas En Silicio, S.A. Method of switching packets in a transmission medium comprising multiple stations which are connected using different links

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4914429A (en) * 1988-12-09 1990-04-03 Transwitch Corp. Switch components and multiple data rate non-blocking switch network utilizing the same
ATE149070T1 (en) * 1990-08-08 1997-03-15 Siemens Ag METHOD AND CIRCUIT ARRANGEMENT FOR DETERMINING THE QUALITY OF VIRTUAL CONNECTIONS OVER AN ATM SWITCHING DEVICE
US5394402A (en) * 1993-06-17 1995-02-28 Ascom Timeplex Trading Ag Hub for segmented virtual local area network with shared media access
EP1162788B1 (en) * 2000-06-09 2012-11-21 Broadcom Corporation Trunking and mirroring across stacked gigabit switches
US6990106B2 (en) * 2001-03-19 2006-01-24 Alcatel Classification and tagging rules for switching nodes
US7397794B1 (en) * 2002-11-21 2008-07-08 Juniper Networks, Inc. Systems and methods for implementing virtual switch planes in a physical switch fabric
US7483374B2 (en) * 2003-08-05 2009-01-27 Scalent Systems, Inc. Method and apparatus for achieving dynamic capacity and high availability in multi-stage data networks using adaptive flow-based routing
WO2005027391A2 (en) * 2003-09-06 2005-03-24 Teak Technologies, Inc. Strictly nonblocking multicast linear-time multi-stage networks
US7463588B1 (en) * 2004-02-18 2008-12-09 Woven Systems, Inc. Mechanism for enabling load balancing to be achieved in a loop-free switching path, reverse path learning network
US7672314B2 (en) * 2004-07-09 2010-03-02 Cisco Technology, Inc. Scaling VLANs in a data network
CA2595788C (en) * 2005-02-04 2013-06-25 Level 3 Communications, Inc. Ethernet-based systems and methods for improved network routing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1049299A1 (en) * 1999-04-30 2000-11-02 Nortel Networks Limited Method and apparatus for bandwidth management of aggregate data flows
US20020163922A1 (en) * 2001-05-01 2002-11-07 Dooley David L. Network switch port traffic manager having configurable packet and cell servicing
WO2005015855A1 (en) * 2003-07-28 2005-02-17 Diseño De Sistemas En Silicio, S.A. Method of switching packets in a transmission medium comprising multiple stations which are connected using different links
EP1662720A1 (en) * 2003-07-28 2006-05-31 Diseno de Sistemas en Silicio S.A. Method of switching packets in a transmission medium comprising multiple stations which are connected using different links

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7983253B2 (en) * 2007-04-20 2011-07-19 Nuon, Inc. Time-space carrier sense multiple access
WO2009039384A2 (en) * 2007-09-20 2009-03-26 C & S Operations, Inc. Computer system with multiple terminals
WO2009039384A3 (en) * 2007-09-20 2009-05-14 C & S Operations Inc Computer system with multiple terminals
US7882274B2 (en) 2007-09-20 2011-02-01 Virtual Desktop Technologies, Inc. Computer system with multiple terminals
US20130003550A1 (en) * 2011-06-29 2013-01-03 Broadcom Corporation System and Method for Priority Based Flow Control Between Nodes
US9124524B2 (en) * 2011-06-29 2015-09-01 Broadcom Corporation System and method for priority based flow control between nodes

Also Published As

Publication number Publication date
CN101283550A (en) 2008-10-08
JP4892550B2 (en) 2012-03-07
WO2006131390A2 (en) 2006-12-14
US9077648B2 (en) 2015-07-07
KR20080037625A (en) 2008-04-30
EP1889414A2 (en) 2008-02-20
CN101283550B (en) 2012-05-23
JP2008546332A (en) 2008-12-18
WO2006131390A3 (en) 2008-02-21
EP1732271B1 (en) 2012-02-01
US20090213866A1 (en) 2009-08-27
KR101228284B1 (en) 2013-01-30
ES2381007T3 (en) 2012-05-22
ATE544266T1 (en) 2012-02-15

Similar Documents

Publication Publication Date Title
US9077648B2 (en) Data communication system with multiple input/outputs and method of processing data packets
US11632606B2 (en) Data center network having optical permutors
US12120016B2 (en) Data center network with packet spraying
US8289977B2 (en) Two-layer switch apparatus avoiding first layer inter-switch traffic in steering packets through the apparatus
US8654680B2 (en) Packet forwarding using multiple stacked chassis
US8243729B2 (en) Multiple chassis stacking using front end ports
US20070036178A1 (en) Layer 2 virtual switching environment
US8442045B2 (en) Multicast packet forwarding using multiple stacked chassis
US10644995B2 (en) Adaptive routing in a box
US20050249229A1 (en) Dynamically scalable edge router
Cisco Introduction to LightStream 2020
Cisco Introduction to LightStream 2020
Cisco Introduction to LightStream 2020
Cisco Introduction to LightStream 2020
Cisco Introduction to LightStream 2020
Cisco Introduction to LightStream 2020
Cisco Introduction to LightStream 2020
US7366167B2 (en) Apparatus and method for hairpinning data packets in an Ethernet MAC chip
Chang et al. Quasi-circuit switching and quasi-circuit switches

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR LV MK YU

17P Request for examination filed

Effective date: 20070524

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

17Q First examination report despatched

Effective date: 20070726

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 544266

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602005032472

Country of ref document: DE

Effective date: 20120329

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: HEPP WENGER RYFFEL AG

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2381007

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20120522

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20120201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120601

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120502

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120601

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 544266

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20121105

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120630

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602005032472

Country of ref document: DE

Effective date: 20121105

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120501

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20050610

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20230719

Year of fee payment: 19

Ref country code: CH

Payment date: 20230702

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230830

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240619

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240626

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240624

Year of fee payment: 20

Ref country code: FI

Payment date: 20240619

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20240624

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20240621

Year of fee payment: 20