WO2020221416A1

WO2020221416A1 - Method and apparatus for selecting a communication device of a plurality of communication devices for communication packet processing

Info

Publication number: WO2020221416A1
Application number: PCT/EP2019/060911
Authority: WO
Inventors: Ermin SAKIC
Original assignee: Siemens Aktiengesellschaft
Priority date: 2019-04-29
Filing date: 2019-04-29
Publication date: 2020-11-05

Abstract

The invention relates to a method and an apparatus for selecting a communication device of a plurality of communication devices for communication packet processing of a communication network, wherein the communication device is controlled by a plurality of control devices connected to the communication network. The method comprises the steps of: Receiving communication packets from the control devices over the communication network, wherein the control devices comprise identically configured und arranged state machines, which compute an identical output for an identical input, if operating in an error-free state; Comparing the received communication packets to exclude on the basis of a fault tolerant operation the control devices operating in an none-error-free state of the communication network; Determining a number of matching communication packets based on a number of the control devices of the communication network and a number of tolerated none-error-free operating control devices; Reassigning at least one communication connection in the communication network by providing configuration tables to at least one of the plurality of communication devices in the communication network based on the comparison result; Computing the correct number of matching communication packets to determine a correct communication package; and Reporting the correct communication package to the plurality of communication devices to identify the communication device for efficient communication packet processing.

Description

Method and apparatus for selecting a communication device of a plurality of communication devices for communication packet processing

The invention relates to a method and apparatus for selecting a communication device of a plurality of communication devices for communication packet processing.

The present invention is focused on the Distributed Software Defined Networks (SDN) as they are a practical and easy-to- understand application domains. The highly available / dis tributed SDNs were introduced as a way to improve the scala bility of the control plane and mitigate the

problem of single-point-of-failure of the network controller. However, the control plane correctness may be endangered by malicious controllers that enforce incorrect configurations, i.e., because of an inconsistency, corrupt/buggy state or malicious behavior. A Byzantine Fault Tolerant (BFT) SDN con trol plane allows for runtime detection of malicious

controllers and their dynamic exclusion from the system con figuration .

This framework is based on the working principle that at the switches the packets originating from all SDN controllers are collected and compared. In a further step the "correct" con trol message is deduced and the correct configuration message is locally applied. In the network phase where no

malicious or unavailable controllers are identified, such an approach results in a high system footprint (each message up date results on average in C transmissions and packet compari sons, where C is the number of deployed controllers) . Moreo ver, the current data plane implementations of SDN-enabled switches may not have the sufficient capacity to forward and process these packets for large request number and thus could easily lead to an overloaded control plane. Moreover, the general problem of tolerating Byzantine Faults for centralized Network Control applications is a novel area in distributed network control and management research. Cur rently, all solutions assume redundant computation result com parison .

In Fig. 4 a known SDN-Network is shown comprising a plurality of network controller, e.g. SDN-Network controller 12, which communicate via a communication network 11 with the communica tion devices 10. The communication devices 10 are controlled and/or managed by the SDN-Network controller 12. The communi cation devices 10 can be configured as a switch. In Fig. 4, the SDN-Network controller 12 are tasked with a configuration of the communication device 10 that receives the communication arrows. The communication devices are configured to forward the packets through the communication network 11. In the SDN- Network the configuration data and payload data are trans ferred. Each SDN-controller performs a bandwidth reservation (load) for transferring configuration data to an appropriate switch. In Fig. 4 the bandwidth reservation is marked by the "+1". At each port, where the packet has to be transferred, the bandwidth of "+1" is reserved. In Fig. 4, in total a sum of eight transferring ports is necessary to configure the com munication device 10. Therefore, a total overhead (total foot print) of eight in the network is given. Moreover, a worst- case delay of two hops is given. The worst-case delay de scribes the longest way through the communication network 11. In this SDN-Network no optimization was performed that leads to a high overhead and delay.

Accordingly, it is an object of the present invention to pro vide a method and an apparatus which allows for offloading the comparison procedure of redundant controllers' outputs from the control plane of the switches to their data plane by se lecting a communication device of communication devices for the communication packet processing. This object is achieved according to a first aspect of the present invention by a method for selecting a communication device of a plurality of communication devices for communica tion packet processing comprising the features of claim 1.

The invention provides according to the first aspect a method for selecting a communication device of a plurality of commu nication devices for communication packet processing of a com munication network, wherein the communication device is con trolled by a plurality of control devices connected to the communication network, comprising the method steps of:

Receiving communication packets from the control devices over the communication network, wherein the control devices com prise identically configured und arranged state machines, which compute an identical output for an identical input, if operating in an error-free state;

Comparing the received communication packets to exclude on the basis of a fault tolerant operation the control devices oper ating in an none-error-free state of the communication net work;

Determining a number of matching communication packets based on a number of the control devices of the communication net work and a number of tolerated none-error-free operating con trol devices;

Reassigning at least one communication connection in the com munication network by providing configuration tables to at least one of the plurality of communication devices in the communication network based on the comparison result; Compu ting the correct number of matching communication packets to determine a correct communication package; and Reporting the correct communication package to the plurality of communica tion devices to identify the communication device for effi cient communication packet processing.

An advantage of the method and apparatus is the efficient pro cessing of redundant controller messages stemming from multi ple co- or dislocated controller instances, using data plane mechanisms which allow a comparison of packets at the optimal location in the network (i.e., according to variable optimiza tion criteria, such as total communication overhead or experi enced control plane delay) . Therefore, hardware acceleration can be used for redundant packet removal instead of the lim ited-capacity on-board CPU. Moreover, the comparison does not depend on the software and its programming and can therefore processed faster.

In addition, the present invention allows for optimal identi fication of the node that will process packets destined to any switch in the network and thus allows for a lowered system footprint .

Further, by allowing for deduction of correct messages in switches that may not necessarily be the switch is the desti nation of the target configuration, so in order to save on the total communication overhead and the system delay/response time .

In a possible embodiment, the communication network is a dis tributed software defined network, SDN.

SDNs allow network administrators to more easily manage the network by abstracting the lower levels of functionality into virtual services. This has become increasingly important with the introduction of virtualization, in which a larger data center has to increasingly create and configure virtual sys tems over the network and generate associated firewall rules and network addresses. There are several approaches to gener ate virtual networks (VLANs) analogously, but this leads to a high level of complexity. Moreover, SDN gives network adminis trators programmable, central control of network traffic with out having to manually access the individual physical network components .

SDN decouples the system that decides where the data is sent to (the control plane) from the underlying system that for wards the data to the selected destination (the data plane) . These systems simplify network administration and enables new applications such as network virtualization, where the control plane is separated from the data plane and implemented as a pure application.

In a further possible embodiment, the communication devices are assigned to the distributed software defined network that comprises a control plane and a data plane, wherein the con trol devices are assigned to the control plane and the commu nication devices are assigned to the data plane.

Communication devices are commonly separated into a control plane and a data plane (sometimes referred to as a forwarding plane or a media plane) . In the case that the communication device is a switch or router (or is implementing routing func tionality) , the control plane typically determines how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing port for that data) , and the data plane is in charge of forwarding that data. For instance, the con trol plane typically includes one or more routing protocols (e.g., an exterior gateway protocol such as Border Gateway Protocol (BGP) (RFC 4271), Interior Gateway Protocol (s) (IGP) (e.g., Open Shortest Path First (OSPF) (RFC 2328 and 5340), Intermediate System to Intermediate System (IS-IS) (RFC 1142), Routing Information Protocol (RIP) (version 1 RFC 1058, ver sion 2 RFC 2453, and next generation RFC 2080)), Label Distri bution Protocol (LDP) (RFC 5036) , Resource Reservation Proto col (RSVP) (RFC 2205, 2210, 2211, 2212, as well as RSVP- Traffic Engineering (TE) : Extensions to RSVP for LSP Tunnels RFC 3209, Generalized Multi-Protocol Label Switching (GMPLS) Signaling RSVP-TE RFC 3473, RFC 3936, 4495, and 4558)) that communicate with other communication devices to exchange routes and select those routes based on one or more routing metrics. In addition, the control plane also typically in cludes ISO layer 2 control protocols such as Rapid Spanning Tree Protocol (RSTP) , Multiple Spanning Tree Protocol (MSTP) , and SPB (Shortest Path Bridging) , which have been standardized by various standard bodies. Routes and adjacencies are stored in one or more routing structures (e.g., Routing Information Base (RIB), Label Infor mation Base (LIB), one or more adjacency structures) on the control plane. The control plane programs the data plane with information (e.g., adjacency and route information) based on the routing structure (s) . For instance, the control plane pro grams the adjacency and route information into one or more forwarding structures (e.g., Forwarding Information Base

(FIB) , Label Forwarding Information Base (LFIB) , and one or more adjacency structures) on the data plane. The data plane uses these forwarding and adjacency structures when forwarding traffic .

Typically, a communication device includes a set of one or more line cards, a set of one or more control cards, and op tionally a set of one or more service cards (sometimes re ferred to as resource cards) . These cards are coupled together through one or more interconnect mechanisms (e.g., a first full mesh coupling the line cards and a second full mesh cou pling all of the cards) . The set of line cards make up the da ta plane, while the set of control cards provide the control plane and exchange packets with external communication devices through the line cards. The set of service cards can provide specialized processing (e.g., Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec) (RFC 4301 and 4309), Intrusion Detection System (IDS), peer-to-peer (P2P) , Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN) , Evolved Packet Core (EPC) Gateway) ) . By way of example, a service card may be used to terminate IPsec tunnels and execute the attendant authentication and encryp tion algorithms.

In a further possible embodiment, the method step comparison of the received communication packets is processed in the data plane . In this way, advantageously, the comparison of packets can be processed in hardware and a processing with greater perfor mance is achieved.

In a further possible embodiment, the fault tolerant operation comprises a Byzantine Fault Tolerant operation. A Byzantine fault (also interactive consistency, source congruency, error avalanche, Byzantine agreement problem, Byzantine generals' problem, and Byzantine failure) is a condition of a computer system, particularly distributed computing systems, where com ponents may fail and there is imperfect information on whether a component has failed. The Byzantine fault is any fault pre senting different symptoms to different observers. A Byzantine failure is the loss of a system service due to a Byzantine fault in systems that require consensus. The objective of Byz antine fault tolerance is to be able to defend against fail ures of system components with or without symptoms that pre vent other components of the system from reaching an agreement among themselves, where such an agreement is needed for the correct operation of the system.

In a further possible embodiment, the configuration tables comprise forwarding tables and processing tables.

The configuration tables hold by the communication devices provide the main mechanism to define how to process any incom ing packet. The tables are populated by a reassigner component at the system startup and are re-optimized at runtime. The configuration tables comprise four distinct tables to provide the necessary functionalities for. A "processing table" de clares which switch should process the packets destined for any other switch. A "process forwarding table" declares which port (virtual or physical) the packets should be sent out to for further processing (internal port if processing locally, or an egress (existing) port if processing on a remote

switch) . A "L2 forwarding table" for pushing the processed packet to the configuration target switch. A "hashed flow ta ble" to be able to uniquely identify the decision of the cor- rect controller, a hash value is generated on the data field of each incoming packet during processing. The counting of the incoming packets is done in this table by updating values in previously declared registers.

In a further possible embodiment, the communication connection is further reassigned in consideration by at least the con straint of the current topology state. In this way, advanta geously, the "incorrect" controllers can be distinguished and the reassigner is notified of a required controller-switch re assignment procedure.

In a further possible embodiment, the communication connection is further reassigned in consideration by at least the con straint of the packet processing and forwarding capacities of the communication device. In this way, advantageously, the op timal processing node for packets to any of the switches in the network can be found.

In a further possible embodiment, the communication connection is further reassigned in consideration by at least the con straint of the packet processing and forwarding capacities of the control device. In this way, advantageously, the optimal processing node for packets to any of the switches in the net work can be found.

In a further possible embodiment, the communication connection is further reassigned in consideration by at least the con straint of the communication connection delay.

In a further possible embodiment, the communication connection is further reassigned in consideration by at least the con straint of the processing delay. With a minimized access to the software (control plane) the processing delay is mini mized. As a result, the configuration time of a communication device is also minimized. In a further possible embodiment, the communication device comprises at least a switch, a router, and/or a firewall.

The switch is the component for enablement of arbitrary packet processing responsibilities. The switch provides an interface for specification of processing (who processes packets for whom) and forwarding matrices (how to reach the destination / how to reach the processing node for a particular destina tion) . The switch is the component for packet processing in Byzantine fault-tolerant manner (i.e., the

hashing, counting evaluation procedure) . The switch merges the packets and output of a single "correct" packet based on the criteria that enables Byzantine Fault Tolerant operation

(i.e., definition of the minimum number of matching correct packets) .

In a further possible embodiment, the state machine of the re dundant control devices is a replicated state machine. The state machine replication or state machine approach is a gen eral method for implementing a fault-tolerant service by rep licating servers and coordinating client interactions with server replicas. The approach also provides a framework for understanding and designing replication management protocols. In the replicated state machine model, each "correct" control ler contains the exactly equal state information (e.g., net work topology, resource reservations) as any other correct controller. In the case of a controller with diverged state, i.e., as a result of corrupted/buggy operation or malicious adversary take-over, the "incorrect" controllers' computations outputs may differentiate from the correct ones. In this way, the "incorrect" controller can be distinguished.

In a further possible embodiment, the control device comprises at least a virtual machine, a container, or a server.

Up to now, the invention has been described with respect to the claimed method. Features, advantages or alternative embod iments herein can be assigned to the other claimed objects (e.g. the computer program or a device, i.e. the apparatus or a computer program product) and vice versa. In other words, the subject matter which is claimed or described with respect to the device can be improved with features described or claimed in the context of the method and vice versa. In this case, the functional features of the method are embodied by structural units of the system and vice versa, respectively. Generally, in computer science a software implementation and a corresponding hardware implementation are equivalent. Thus, for example, a method step for "storing" data may be performed with a storage unit and respective instructions to write data into the storage. For the sake of avoiding redundancy, alt hough the apparatus may also be used in the alternative embod iments described with reference to the method, these embodi ments are not explicitly described again for the apparatus.

The invention further provides according to a second aspect an apparatus for selecting a communication device of a plurality of communication devices for communication packet processing comprising the features of claim 11.

The invention further provides according to a second aspect an apparatus for selecting a communication device of a plurality of communication devices for communication packet processing of a communication network, wherein the communication device is controlled by a plurality of control devices connected to the communication network, the apparatus comprises a central processing unit, configured to:

receive communication packets from the control devices over the communication network via a communication interface, wherein the control devices comprise identically configured und arranged state machines, which compute an identical output for an identical input, if operating in an error-free state; compare the received communication packets to exclude on the basis of a fault tolerant operation the control devices oper ating in an none-error-free state of the communication net work; determine a number of matching communication packets based on a number of the control devices of the communication network and a number of tolerated none-error-free operating control devices ;

reassign at least one communication connection in the communi cation network by providing configuration tables to at least one of the plurality of communication devices in the communi cation network based on the comparison result;

compute the correct number of matching communication packets to determine a correct communication package; and

report the correct communication package to the plurality of communication devices to identify the communication device for efficient communication packet processing.

The invention further provides according to a third aspect a computer program product for selecting a communication device of a plurality of communication devices for communication packet processing comprising the features of claim 12.

In the third aspect the invention relates to a computer pro gram product comprising a computer program, the computer pro gram being loadable into a memory unit of a computing unit, including program code sections to make the computing unit ex ecute the method for selecting a communication device of a plurality of communication devices for communication packet processing according to the first aspect of the invention, when the computer program is executed in said computing unit.

The invention further provides according to a fourth aspect a computer-readable medium for selecting a communication device of a plurality of communication devices for communication packet processing comprising the features of claim 13.

In the fourth aspect the invention relates to a computer- readable medium, on which program code sections of a computer program are stored or saved, said program code sections being loadable into and/or executable in a computing unit to make the computing unit execute the method for selecting a communi- cation device of a plurality of communication devices for com munication packet processing according to an aspect of the in vention, when the program code sections are executed in the computing unit.

The properties, features and advantages of this invention de scribed above, as well as the manner they are achieved, become clearer and more understandable in the light of the following description and embodiments, which will be described in more detail in the context of the drawings. This following descrip tion does not limit the invention on the contained embodi ments. Same components or parts can be labeled with the same reference signs in different figures. In general, the figures are not for scale. It shall be understood that a preferred em bodiment of the present invention can also be any combination of the dependent claims or above embodiments with the respec tive independent claim.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter .

In the following possible embodiments of the different aspects of the present invention are described in more detail with reference to the enclosed figures.

Fig. 1 shows a schematic diagram of a first specific example for illustrating the operation of a method and an ap paratus selecting a communication device of a plural ity of communication devices for communication packet processing;

Fig. 2 shows a flowchart of a possible exemplary embodiment of a method for selecting a communication device of a plurality of communication devices for communication packet processing; Fig . 3 shows a block diagram of a possible exemplary embodi ment of an apparatus for selecting a communication device of a plurality of communication devices for communication packet processing;

Fig. 4 shows a schematic diagram of an example for illus trating the operation of a method selecting a commu nication device of a plurality of communication de vices for communication packet processing according to the prior art;

Fig . 5 shows a schematic diagram of a second specific exam ple for illustrating the operation of a method and an apparatus selecting a communication device of a plu rality of communication devices for communication packet processing;

Fig . 6 shows a flowchart of a possible exemplary embodiment of the switch data plane mechanism for selecting a communication device of a plurality of communication devices for communication packet processing;

Fig. 7 shows a flowchart of a possible exemplary embodiment of the switch control plane mechanism for selecting a communication device of a plurality of communication devices for communication packet processing;

Fig. 8 shows schematically a possible exemplary embodiment of the interaction between switches and the reassign- er for selecting a communication device of a plurali ty of communication devices for communication packet processing .

Fig. 1 shows schematically a first specific example for illus trating the operation of a method and an apparatus selecting a communication device of a plurality of communication devices for communication packet processing. In Fig. 1 an SDN-Network is shown comprising a plurality of network controller 12, e.g. SDN-Network controller 12 (control device) , which communicate via a communication network 11 with the communication devices 10. The controller 12 are the cen tralized control plane entities that enforce configurations on the switches based on their decision-making processing. Each controller 12 serves each client request. The controller 12 runs on a dedicated hardware, e.g. a computer or can be imple mented as a virtual machine or container. The replicated state machine model in which each "correct" controller 12 contains the exactly equal state information as any other correct con troller 12. In the case of a controller 12 with diverged state, i.e., as a result of corrupted/buggy operation or mali cious adversary take-over, the "incorrect" controllers' compu tations outputs may differentiate from the correct ones. The overall goal of comparison procedure in switches is to distin guish "incorrect" controllers 12 and notify the reassigner of a required controller-switch reassignment procedure.

The communication devices 10 are controlled and/or managed by the SDN-Network controller 12. The communication devices 10 can be configured as a switch and represents the configuration target. The switches are the forwarding devices that push the packets and depending on the output of the optimization func tion executed in reassigner, the switches are assigned the re sponsibility for matching outputs computed by different con trollers, for themselves or for various other configuration targets. The switches collect and compare the messages sent out by different controllers 12 and distinguish the correct message. On identification of a discrepancy/corrupt control ler, the switches have the task to declare the corrupt-or- buggy controllers to the reassigner component.

The reassigner is a component with functionalities, which at runtime identifies the switches that are to process the redun dant messages computed by multiple controllers 12. The reas signer can be an instance that can be hosted in the communica tion network 11 next to the controller 12 or on a separate electronic device that is configured to process the reassigner functionality. The reassigner dynamically reassigns the con troller-switch connections based on the information received from the detection mechanism of the switches. The reassigner calculates the mapping for the communication devices 10. The output of the assignment of comparison and packet forwarding tasks are the processing and forwarding tables. Additionally, based on the number of controllers 12 and a number of maximum tolerated Byzantine failures, the reassigner computes and re ports to the switches the number of required matching messages required to detect the correct message. Based on the result of the optimization function executed in the reassigner, the switches may gain the responsibility of control packets pro cessing. In an embodiment, the reassigner can be secured by a trusted platform module or implemented in a trusted platform module to avoid providing corrupted messages. During the start (boot) process of the components connected to the communica tion network 11, each communication device 10 achieve the re quired information, which communication device 10 has to be configured and started. The messages for the start process are provided by the reassigner. If a corrupted controller device 12 (controller) is detected or a switch malfunctions during runtime, the reassigner reconfigure the routing tables or starts a remapping, respectively and provides the reconfigured routing tables to the communication devices 10 of the communi cation network 11.

The reassigner component includes the logic for solving the optimization problem. It takes as input the current topology state, available packet processing and forwarding capacities of switches and controllers, link and processing delays (con straints) . The reassigner outputs the forwarding tables and processing tables, as well as the number of required matching packets to detect the correct message. The reassigner inter acts with the switches in the network by processing (packet comparison) responsibility table exchange and forwarding ta bles exchanges (containing next port to processing destination / next L2 switching output port) . The reassigner provides the minimum number of required matching packets for detecting the correct outcome. The reassigner reacts on asynchronous events, such as switch failure, controller failure/controller status change from correct to incorrect.

In Fig. 1, the SDN-Network controller 12 are tasked with a configuration of the communication device 10 that receives the communication arrows. The communication devices 10 are config ured to forward the packets through the communication network 11. In the SDN-Network the configuration data and payload data are transferred. Each SDN-controller performs a bandwidth res ervation (load) for transferring configuration data to an ap propriate switch. In Fig. 1 the bandwidth reservation is marked by the "+1". At each port, where the packet has to be transferred, the bandwidth of "+1" is reserved. In Fig. 1, in total a sum of six transferring ports are necessary to config ure the communication devices 10. Therefore, a total overhead (total footprint) of six in the network is given. Moreover, a worst-case delay of two hops is given. The worst-case delay describes the longest way through the communication network 11.

Fig.l shows an optimized case according to the present inven tion with a multi-objective function (communication overhead footprint and control plane delay) . The communication device that receives the communication arrows represent the optimal processing node according to the two optimization objectives (footprint and delay) . With the present invention, it is pos sible to specify an optimization objective so to minimize the experienced delay in the control plane, as well as the total number of control packets flowing through the network. Based on the present invention for control packet processing, if a switch is assigned to process the packets destined for another switch, in the end it will only

forward one packet with the correct decision. In other words, if the processing of the packets was to be done only at the destination switch, a request for the communication devices, which receives the communication arrows, would create a total footprint of 8 packets in the network (see Fig. 4) . In Fig. 1 an optimization by delay metric is considered as well, the total experienced critical path is equal to 2 hops in the worst case in addition for having an optimal communication overhead .

Fig. 2 shows schematically a flowchart of a possible exemplary embodiment of a method for selecting a communication device of a plurality of communication devices for communication packet processing .

The method comprises in the illustrated exemplary embodiment several main steps. In a first step SI, communication packets from the control devices 12 over the communication network 11 are received, wherein the control devices 12 comprise identi cally configured und arranged state machines, which compute an identical output for an identical input, if operating in an error-free state.

In a further step S2, the received communication packets are compared to exclude on the basis of a fault tolerant operation the control devices 12 operating in an none-error-free state of the communication network 11.

In a further step S3, a number of matching communication pack ets is determined based on a number of the control devices 12 of the communication network 11 and a number of tolerated none-error-free operating control devices 12.

In a further step S4, at least one communication connection in the communication network 11 is reassigned by providing con figuration tables to at least one of the plurality of communi cation devices 10 in the communication network 11 based on the comparison result.

In a further step S5, the correct number of matching communi cation packets is computed to determine a correct communica tion package. In a further step S6, the correct communication package is re ported to the plurality of communication devices 10 to identi fy the communication device 10 for efficient communication packet processing.

Fig. 3 shows schematically a block diagram of a possible exem plary embodiment of an apparatus for selecting a communication device of a plurality of communication devices for communica tion packet processing.

As can been seen from the block diagram of Fig. 3, the appa ratus 100 is used for selecting a communication device 10 of a plurality of communication devices 10 for communication packet processing .

The apparatus 100 comprises in the illustrated embodiment a central processing unit 101 and a communication interface 102. The central processing unit 101 is configured to receive com munication packets from the control device 12 over the commu nication network 11 via the communication interface 102, wherein the control devices 12 comprise identically configured and arranged state machines, which compute an identical output for an identical input, if operating in an error-free state. The central processing unit 101 is further configured to re ceive communication packets to exclude on the basis of a fault tolerant operation the control devices 12 operation in an none-error-free state of the communication network 11. The central processing unit 101 is further configured to determine a number of matching communication packets based on a number of the control devices 12 of the communication network 11 and a number of tolerated none-error-free operating control devic es 12. The central processing unit 101 is further configured to reassign at least one communication connection in the com munication network 11 by providing configuration tables to at least one of the plurality of communication devices 10 in the communication network 11 based on the comparison result. The central processing unit 101 is further configured to compute the correct number of matching communication packets to deter mine a correct communication package, and to report the cor rect communication package to the plurality of communication devices 10 to identify the communication device 10 for effi cient communication packet processing.

The apparatus for selecting a communication device 10 of a plurality of communication devices 10, mentioned above, may be a computer, personal computer or a workstation in a computer network and includes the central processing unit 101, a system memory (not shown) , and a system bus (not shown) that couples various system components including the system memory to the central processing unit 101. The system bus may be any of sev eral types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system memory may include read only memory (ROM) and/or random access memory (RAM) . A basic input/output system (BIOS), containing basic routines that help to transfer information between elements within the personal computer, such as during start-up, may be stored in ROM. The computer may also include a hard disk drive for read ing from and writing to a hard disk. The hard disk drive may be coupled with the system bus by a hard disk drive interface. The drive and its associated storage media provide nonvolatile storage of machine-readable instructions, data structures, program modules and other data for the computer. Although the exemplary environment described herein employs a hard disk, those skilled in the art will appreciate that other types of storage media, such as flash memory cards, digital video disks, random access memories (RAMs) , read only memories

(ROM) , and the like, may be used instead of, or in addition to, the storage devices introduced above. A number of program modules may be stored on the hard disk, ROM or RAM, such as an operating system, one or more application programs, like the method for selecting and/or other program modules, and/or pro gram data for example. Fig.5 shows schematically a second specific example for illus trating the operation of a method and an apparatus selecting a communication device of a plurality of communication devices for communication packet processing.

In Fig. 5 an SDN-Network is shown comprising a plurality of network controller, e.g. SDN-Network controller 12, which com municate via a communication network 11 with the communication devices 10. The communication devices 10 are controlled and/or managed by the SDN-Network controller 12. The communication devices 10 can be configured as a switch. In Fig. 5, the SDN- Network controllers 12 are tasked with a configuration of the communication device 10 that receives the communication ar rows. The communication devices are configured to forward the packets through the communication network 11. In the SDN- Network the configuration data and payload data are trans ferred. Each SDN-controller performs a bandwidth reservation (load) for transferring configuration data to an appropriate switch. In Fig. 5 the bandwidth reservation is marked by the "+1". At each port, where the packet has to be transferred, the bandwidth of "+1" is reserved. In Fig. 5, in total a sum of six transferring ports is necessary to configure the commu nication device 10. Therefore, a total overhead (total foot print) of six in the network is given. Moreover, a worst-case delay of four hops is given. The worst-case delay describes the longest way through the communication network 11. In Fig.

5 is the optimized case according to the present invention presented comprising a single-objective function considering the communication overhead footprint. Therefore, if the pro cessing has been perfomred at the communication device 10 that receives the communication arrows, the total number of packets would be six but with a higher worst-case delay of four hops. As a result, by defining the objective as minimizing the num ber of forwarded packets, the method and apparatus of the pre sent invention is able to find the optimal

processing node for packets destined to any of the switches in the network. The Switches that are declared to process controller messages for a particular configuration target (i.e., for itself, or for another switch) collect the information about the

packets stemming from different controllers 12. The switch maintains counters of matching hashes for a particular request identifier. After a sufficient number of matching packets was collected for a particular hash, the comparator switch signs and forwards one instance of the correct packet to its own control plane for any required software processing (i.e., fi nal identification of the correct hash and the identification of malicious controllers 12), and another

instance on the port that leads to the next hop and the path to the final destination target. The differentiation of al ready processed from unprocessed packets can be done by observing the inclusion of signature field.

Fig. 6 shows schematically a flowchart of a possible exemplary embodiment of the switch data plane mechanism for selecting a communication device of a plurality of communication devices for communication packet processing.

In Fig. 6, the workflow of the switch data plane mechanism af ter receiving any packet is depicted. In Fig. 6. the step D1 indicates the Start-step of the switch data plan mechanism. Step D2 checks whether the packet was received. If a packet was received (+) , step D3 checks whether the packet is signed. If no packet was received (-) the switch data plan mechanism goes back to step D1. If the packet is signed ( + ) , step D4 checks, whether the received packet is for the receiving com munication device 10 (see Fig. 1) . If the packet is unsigned (-) , in step D7 the packet is verified for processing. If the received packet is for the receiving communication device (+) , the received packet is sent to the central processing unit of the communication device 10 and the switch data plan mechanism goes back to step D1. If the received packet is not for the receiving communication device 10 (-) , in step D6 the received packet is forwarded to the destination communication device 10 and the switch data plan mechanism goes back to step D1. If the unsigned packet is processed in D7 (+) , step D8 performs a hash on the 'data' field. If the unsigned packet is not pro cessed (-) , in step D14 the unsigned packet is forwarded to the assigned processing node and the switch data plan mecha nism goes back to step D1. After performing step D8, in step D9 is check, whether the value of step D8 is in the

't_flow_hash' table. If the value is in the ' t_flow_hash' ta ble (+) , in step D10 the associated action is applied (update specific counters based on hash value) . If the value is not in the ' t_flow_hash' table (-) , in step D15 the

'req_id_pkt_counter' is updated. The 'req_id_pkt_counter' is not a specific based on the hash value. After performing step D15, an entry should add in the ' t_flow_hash' table, and step D19 is performed. In step D19 the packet is sent to the cen tral processing unit of a communication device and the switch data plan mechanism goes back to step D1. After performing step D10, step Dll checks whether the value 'valid_pkt_count ' and the value 'ncs_valid_pkts ' are equal. The value 'val- id_pkt_count ' indicates the value of the data plane packet counter for the given hash value. The value 'ncs_valid_pkts ' indicates the value calculated by the reassigner and updated by the control plane of the communication device 10, equal to the minimum number of controllers that make up the majority.

If the value 'valid_pkt_count ' and the value 'ncs_valid_pkts ' are equal (+) , step D12 creates a copy of the packet to send it later to the central processing unit of the communication device. In order to send to two destinations at the same time, in P4 it can only be done using packet cloning. P4 is a pro gramming language designed to allow programming of packet for warding planes. In contrast to general purpose languages such as C or Python, P4 is a domain-specific language with a number of constructs optimized around network data forwarding. Step D13 checks whether the packet is the created copy. If the packet is the created copy (+) , the replica is sent to the central processing unit in step D19 because the correct hash is known. If the packet is not the created copy (-) , because it is the original packet, in step D17 the packet is signed and in step D18 forwarded to the destination communication de- vice. If the value 'valid_pkt_count ' and the value

'ncs_valid_pkts ' are not equal (-) , step D16 compares the val ue ' total_pkt_count ' with the value 'num_valid_cont ' . The val ue ' total_pkt_count ' is the value of the 'req_id_pkt_counter' data plane register, given the req_id_value (unique value) .

The value 'num_valid_cont ' is the value calculated by the reasigner and updated by the control plane of the switch, equal to the total number of control devices 12 minus the num ber of the detected malicious control devices 12. If the value 'total_pkt_count ' and the value 'num_valid_cont ' are equal (+) , the central processing unit (cpu) should reset the coun ters . If the value ' total_pkt_count ' and the value

'num_valid_cont ' are not equal (-) , the switch data plan mech anism goes back to step D1.

Fig. 7 shows schematically a flowchart of a possible exemplary embodiment of the switch control plane mechanism for selecting a communication device of a plurality of communication devices for communication packet processing.

The workflow of the switch control plane mechanism after re ceiving a copy of correct packet is depicted in Fig. 7. The switch has to identify any incorrect control devices 12 (i.e., those whose output hashes diverge from the deduced correct hash) and subsequently notify the centralized reassigner com ponent. The connection device 10 might also simply need to ap ply the configuration message if it is itself the configura tion destination (configuration target) . Additionally, the switch control plane will clear registers associated with that message, so to save the memory space for future request se quence IDs (e.g. comparisons).

In Fig. 7 the step Cl indicates that the packet was sent to the central processing unit. Step C2 indicates the Start-step of the switch control plan mechanism. Step C3 checks whether a packet is received from the data plane of the communication device 10. If a packet is received from the data plane of the communication device (+) , step C4 checks whether the packet is signed. If the packet is signed (+) , there is no case when the packet is not for the communication device 10 and in step C5 the flow decision is applied and the switch control plan mech anism goes back to step C2. If no packet was received (-) in step C3, the switch control plan mechanism goes back to step C2. If the packet is not signed (-) , step C6 is performed. In step C6 is checked, whether the hash value is recognized. If the hash value is recognized (+) , step Cll checks whether the value 'valid_pkt_count ' and the value 'ncs_valid_pkts ' are equal. The value 'valid_pk_count ' is the value of the data plane packet counter for the given hash value. The value

'ncs_valid_pkts ' is the value calculated by the reassigner, equal to the minimum number of control devices 12 that make up the majority. If the hash value is not recognized (-) in step C6, step C7 add the value in the recognized hashes array. Fol lowing step C8 decides which data packet counter will be asso ciated to this hash value. In step C9, an entry is added to the ' t_flow_hash' table and in step CIO the counters associat ed to this hash value are updated. In the following step Cll the value 'valid_pkt_count ' and the value 'ncs_valid_pkts ' are compared. If the value 'valid_pkt_count ' and the value

'ncs_valid_pkts ' are equal (+) , step C12 saves the hash value of the received packet as the correct hash value. If the value 'valid_pkt_count ' and the value 'ncs_valid_pkts ' are equal (- ) , step C12 is skipped. In step C13 the value 'to- tal_pkt_count ' and the value 'num_valid_cont ' are compared.

The value ' total_pkt_count ' is the value of the

'req_id_pkt_counter' data plane register, given the req_id value (unique value) . The value 'num_valid_cont ' is the value calculated by the reassigner, equal to the total number of control device 12 minus the number of the detected malicious controller. If the value ' total_pkt_count ' and the value

'num_valid_cont ' are equal (+) , step C14 checks whether any control device 12 was malicious. If the value 'to- tal_pkt_count ' and the value 'num_valid_cont ' is not equal (- ), the switch control plan mechanism goes back to step C2. If the control device 12 is malicious (+) , in step C15 the reas signer is notified. If the control device 12 is not malicious or the state is unknown (-) , step C15 is skipped and in step C16 all used registers are reset.

Fig. 8 shows schematically a possible exemplary embodiment of the interaction between switches and the reassigner for se lecting a communication device of a plurality of communication devices for communication packet processing.

The reassigner may accordingly reconfigure the processing and forwarding tables in the communication devices 10 (see Fig.

1) , e.g., a switch, after receiving a notification about de tection of a malicious controlling device 12 (see Fig. 1) (or a failed switch) , as well as the number of required matching messages ncs_valid_pkts to detect the correct message. The workflow of the interactions between the

reassigner and switches is depicted in Fig. 8.

In Fig. 8 reference numeral 13 denotes the reassigner und rev erence numeral 13 a communication device, preferably a switch. The reassigner 10 initializes in a first step the calculation of the shortest path routing and the optimal processing nodes. This information is provided to the switch. In a second step the switch uses the received information from the reassiger 13 to build the entries for the P4 tables, in particular the t_process table, t_process_forward table, and the t_12_forward table. In a third step, for instance a malicious controlling device 12 is detected. The switch notifies the ressigner. As a result, the reassigner recalculates the optimal processing nodes in a fourth step. The switches are notified about the decision, if there is any change. In a fifth step, the entries of the P4 tables are updated. In particular, the t_process ta ble and the t_process_forward table. In a sixth step, for in stance, a switch failure is detected. The switch notifies the reassigner. In a seventh step, the reassigner processes a re calculation of shortest path routing and optimal processing nodes. The switches are notified about the changes. In an eight step the entries for the P4 tables are updated, in par- ticular, the t_process table, the t_process_forward table, and the t_12_forward table.

The reassigner 13 has the role to populate the switches for warding table so that the nodes selected as processing nodes for redundant controller messages are optimal with respect to a set of given constraints (i.e., so that the total message overhead or control plane latency experienced in

control plane is minimized) .

In the present invention the switches hold the tables that provide the main mechanism to define how to process any incom ing packet. The tables are populated by the reassigner compo nent at the system startup and are re-optimized at runtime. In the P4 application, four distinct tables are implemented to provide the necessary functionalities. The processing table declares which switch should process the packets destined for any other switch. The process forwarding table declares which port (virtual or physical) the packets should be sent out to for further processing (internal port if processing locally, or an egress port if processing on a remote switch) . The L2 forwarding table is for pushing the processed packet to the configuration target switch. The hashed flow table is able to uniquely identify the decision of the correct controller and a hash value is generated on the data field of each incoming packet during processing. The counting of the incoming packets is done in this table by updating values in previously de clared registers.

The processing table, named T_process, maintains information regarding ID of the switches whose packets need to be pro cessed by the switch, which hosts this table. It is populated at the system bootstrapping time and at runtime by the reas signer, based on the networks state information provided by the routing and the optimization algorithm that executes on top of that state. The reassigner modifies the tables at runtime any time a controller was identified as incorrect (and thus may be excluded from the cluster, result ing in a different optimization function result) , and when any of the switches have been identified as failed.

As shown in Table 1 and Table 2, there is no action associated with the entries of this table because it is only used to check whether the switch should process a specific incoming packet or not. Table 1 shows the structure of table t_process and Table 2 shows an example of a populated t_process table.

Table 1: structure of table t_process

Table 2: Example of a populated t_process table

In the case when a packet should not be processed by this switch, it needs to forward it to the correct processing node (switch) using the correct output port. That requires the switch to have information about where to send this packet in such cases. This information is maintained by the

T_process_forward table. Here, each switch identifier is mapped to an output port to where the incoming packet needs to be forwarded in order to reach the designated processing node. As shown in Table 3 and Table 4, the associated action for each table entry is set out port, which takes as a parame ter the port number. Similarly, as before, this table is popu- lated by the reassigner at the system bootstrapping time and any time a new incorrect controller is identified / a switch node has failed.

Table 3: structure of table t_process_forward

Table 4: Example of a populated t_process_forward table

Forwarding to the designated processing node in not the only forwarding mechanism necessary for our application. If the switch should process the incoming control packets for another switch, the last step will be to forward it to the destination switch. Therefore, information on how to reach all the other switches in the network needs to be maintained in each switch. In an embodiment, this is also populated by the reassigner. This is done by the T_12_forward table. Similar to the

t_process_forward table, each switch id is mapped to an out port with the only difference that the packet is now forwarded to the destination switch.

The T_12_forward table is used in two cases. First when the switch has finished processing the control packets are desig nated for another switch, and second when a switch receives a signed (already processed) packet that is not for itself but for another switch in the network. The structure of the table and an example of how it may be populated are shown in Table 5 and Table 6.

Table 5: structure of table t 12 forward

Table 6: Example of a populated t_12_forward table

In a Byzantine Fault Tolerant system, for any change that will be applied in the network, each of the controlling devices 12 calculates its individual decision and attempts to forward it to the destination switch (but it is intercepted by the switch declared as the processing node for decisions destined for the destination switch) . Therefore, in order to uniquely identify packets in the network, in the present invention the availa bility of a packet header that contains information regarding the request id, controller id, destination switch id (could be an MAC/IP address) , data (which carries the decision of the controller) and signature (which denotes if a packet is al ready processed) is given. The term „processing" according to the present invention refers to checking and counting the data field of packets that have the same request id, and destina tion switch id. Furthermore, to be able to uniquely identify the decision of the controller, a hash value is generated on the data field of the packet during processing. The counting of the incoming packets is done by updating values in previ ously declared registers. These registers may be implemented as mapped to calculated hash values in a 1:1 relationship. As shown in Table 7, when a new request-sequence-id row is able is added in the table, a hash is computed over the acquired payload and the counter is incremented to 1. On each subse quent arrival of a packet with the same sequence identifier, the procedure is repeated and the hash counter is updated ac cording to the payload content.

Table 7: Model of the hash counter table

In summary, the invention relates to a method and an apparatus for selecting a communication device of a plurality of commu nication devices for communication packet processing of a com munication network, wherein the communication device is con trolled by a plurality of control devices connected to the communication network. The method comprises the steps of:

Reassigning at least one communication connection in the com munication network by providing configuration tables to at least one of the plurality of communication devices in the communication network based on the comparison result;

Computing the correct number of matching communication packets to determine a correct communication package; and

Reporting the correct communication package to the plurality of communication devices to identify the communication device for efficient communication packet processing. Reference numerals

1 method

10 communication device

11 communication network

12 control device

13 reassigner

100 apparatus

101 central processing unit

102 communication interface

D1-D19 data plane mechanism steps Cl-Cl 6 control plane mechanism steps S1-S6 method steps

Claims

Patent Claims

1. Method (1) for selecting a communication device (10) of a plurality of communication devices (10) for communication packet processing of a communication network (11), wherein the communication device (10) is controlled by a plurality of con trol devices (12) connected to the communication network (11), comprising the method steps of:

- Receiving (SI) communication packets from the control de vices (12) over the communication network (11), wherein the control devices (12) comprise identically configured und arranged state machines, which compute an identical output for an identical input, if operating in an error- free state;

- Comparing (S2) the received communication packets to ex clude on the basis of a fault tolerant operation the con trol devices (12) operating in an none-error-free state of the communication network (11);

- Determining (S3) a number of matching communication pack ets based on a number of the control devices (12) of the communication network (11) and a number of tolerated none- error-free operating control devices (12);

- Reassigning (S4) at least one communication connection in the communication network (11) by providing configuration tables to at least one of the plurality of communication devices (10) in the communication network (11) based on the comparison result;

- Computing (S5) the correct number of matching communica tion packets to determine a correct communication package; and

Reporting (S6) the correct communication package to the plurality of communication devices (10) to identify the communication device (10) for efficient communication packet processing.

2. Method according to claim 1, wherein the communication network (11) is a distributed software defined network, SDN.

3. Method according to claim 2, wherein the communication de vices (10) are assigned to the distributed software defined network that comprises a control plane and a data plane, wherein the control devices (12) are assigned to the control plane and the communication devices (10) are assigned to the data plane.

4. Method according to claim 3, wherein the method step (S2), comparison of the received communication packets, is processed in the data plane.

5. Method according to any of the preceding claims, wherein the fault tolerant operation comprises a Byzantine Fault Tol erant operation.

6. Method according to any of the preceding claims, wherein the configuration tables comprise forwarding tables and pro cessing tables.

7. Method according to any of the preceding claims, wherein the communication connection is further reassigned in consid eration by at least one of the constraints:

the current topology state;

packet processing and forwarding capacities of the commu nication device (10); packet processing and forwarding capacities of the control device ( 12 ) ;

communication connection delay, and/or

processing delay.

8. Method according to any of the preceding claims, wherein the communication device (10) comprises at least a switch, a router, and/or a firewall.

9. Method according to any of the preceding claims, wherein the state machine of the redundant control devices (12) is a replicated state machine.

10. Method according to any of the preceding claims, wherein the control device (12) comprises at least a virtual machine, a container, or a server.

11. Apparatus (100) for selecting a communication device (10) of a plurality of communication devices (10) for communication packet processing of a communication network (11), wherein the communication device (10) is controlled by a plurality of con trol devices (12) connected to the communication network (11), the apparatus comprises a central processing unit (101), con figured to: receive communication packets from the control devices (12) over the communication network (11) via a communica tion interface (102), wherein the control devices (12) comprise identically configured und arranged state ma chines, which compute an identical output for an identical input, if operating in an error-free state; compare the received communication packets to exclude on the basis of a fault tolerant operation the control de- vices (12) operating in an none-error-free state of the communication network (11); determine a number of matching communication packets based on a number of the control devices (12) of the com munication network (11) and a number of tolerated none- error-free operating control devices (12); reassign at least one communication connection in the communication network (11) by providing configuration ta bles to at least one of the plurality of communication devices (10) in the communication network (11) based on the comparison result; compute the correct number of matching communication packets to determine a correct communication package; and report the correct communication package to the plurality of communication devices (10) to identify the communica tion device (10) for efficient communication packet pro cessing .

12. A computer program product comprising program elements which induce an electronic device (D) to carry out the steps of the method for identifying a communication device (10) for efficient communication packet processing accord ing to one of the preceding method claims, when the pro gram elements are loaded into a memory of the electronic device (D) .

13. A computer-readable medium on which program elements are stored that can be read and executed by an electronic de vice (D) in order to perform steps of the method for iden tifying a communication device (10) for efficient communi cation packet processing according to one of the preceding method claims, when the program elements are executed by the electronic device (D) .