US9906437B2 - Communication system, control apparatus, control method and program - Google Patents

Communication system, control apparatus, control method and program Download PDF

Info

Publication number
US9906437B2
US9906437B2 US14/433,012 US201314433012A US9906437B2 US 9906437 B2 US9906437 B2 US 9906437B2 US 201314433012 A US201314433012 A US 201314433012A US 9906437 B2 US9906437 B2 US 9906437B2
Authority
US
United States
Prior art keywords
control apparatus
packet handling
destination
communication flows
packet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/433,012
Other languages
English (en)
Other versions
US20150263939A1 (en
Inventor
Yohei Hasegawa
Yohei Iizawa
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Asia Pacific Pte Ltd
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Assigned to NEC CORPORATION reassignment NEC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HASEGAWA, YOHEI, IIZAWA, YOHEI
Publication of US20150263939A1 publication Critical patent/US20150263939A1/en
Application granted granted Critical
Publication of US9906437B2 publication Critical patent/US9906437B2/en
Assigned to NEC Asia Pacific Pte Ltd. reassignment NEC Asia Pacific Pte Ltd. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NEC CORPORATION
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/38—Flow based routing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/02—Topology update or discovery
    • H04L45/04—Interdomain routing, e.g. hierarchical routing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/42—Centralised routing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/12—Avoiding congestion; Recovering from congestion
    • H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H04L47/78—Architectures of resource allocation
    • H04L47/781—Centralised allocation of resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H04L47/82—Miscellaneous aspects
    • H04L47/825—Involving tunnels, e.g. MPLS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/08—Annexed information, e.g. attachments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q11/00—Selecting arrangements for multiplex systems
    • H04Q11/0001—Selecting arrangements for multiplex systems using optical switching
    • H04Q11/0062—Network aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q11/00—Selecting arrangements for multiplex systems
    • H04Q11/0001—Selecting arrangements for multiplex systems using optical switching
    • H04Q11/0062—Network aspects
    • H04Q2011/0064—Arbitration, scheduling or medium access control aspects

Definitions

  • the present invention relates to a communication system, a control apparatus, a control method and a program.
  • the invention relates to a communication system, a control apparatus, a control method and a program, where a network is centrally controlled by the control apparatus.
  • OpenFlow communication is taken as end-to-end flow, and path control, failure recovery, load balancing, and optimization are performed on a per-flow basis.
  • An OpenFlow switch as specified in Non Patent Literature 2 is provided with a secure channel for communication with an OpenFlow controller, and operates according to a flow table in which appropriate addition or rewriting is instructed by the OpenFlow controller.
  • the flow table for each flow there are definitions of sets of match conditions (Match Fields) for collation with packet headers, flow statistical information (Counters), and instructions (Instructions) that define processing content (refer to “4.1 Flow Table” in Non Patent Literature 2).
  • an OpenFlow switch when an OpenFlow switch receives a packet, a search is made for an entry having a match condition (refer to “4.3 Match Fields” in Non Patent Literature 2) that matches header information of the received packet, from the flow table.
  • a match condition (refer to “4.3 Match Fields” in Non Patent Literature 2) that matches header information of the received packet, from the flow table.
  • the OpenFlow switch updates the flow statistical information (Counters) and also implements processing content (packet transmission from a specified port, flooding, dropping, and the like) described in an Instructions field of the entry in question, for the received packet.
  • the OpenFlow switch transmits a request for entry setting, to the OpenFlow controller via the secure channel, that is, a request (Packet-In message) to transmit control information for processing the received packet.
  • the OpenFlow switch receives a flow entry determined by processing content and updates the flow table. In this way, the OpenFlow switch preforms packet forwarding using entries stored in the flow table as control information.
  • Patent Literature (PTL) 1 discloses an optical network system provided with optical path establishing means, and configured from a plurality of optical edge routers that connect an external IP network to an optical network, and a plurality of optical cross-connect devices provided with switching means for connecting between the optical edge routers by optical paths on an optical path basis.
  • a destination to accommodate communication flows received by a communication apparatus belonging to a specific layer is determined without giving consideration to a packet forwarding state of a lower layer thereof (for example, a data link layer), there is a possibility of communication flows in the lower layer being unevenly distributed.
  • a specific layer for example, a network layer
  • a packet forwarding state of a lower layer thereof for example, a data link layer
  • Patent Literature 1 stops at disclosing technology for individual path control of an IP network and an optical network, and there is no disclosure of technology for realizing efficient operation of an overall network integrating both thereof.
  • a communication system comprising: a control apparatus that controls communication apparatuses included in a hierarchical network, and a first communication apparatus that forms links in a first layer of the network and also performs processing related to communication flows based on a first processing rule (i.e., packet handling rule, termed herein as “packet handling operation”); wherein the control apparatus determines a destination to accommodate communication flows specified according to the first packet handling operation, based on information related to a second layer that differs from the first layer.
  • a first processing rule i.e., packet handling rule, termed herein as “packet handling operation”
  • a control apparatus that determines a destination to accommodate communication flows via a first communication apparatus that forms links in a first layer of a hierarchical network, based on information related to a second layer that differs from the first layer.
  • a control method for a control apparatus that controls a communication apparatus included in a hierarchical network comprising a step of determining a destination to accommodate communication flows via a first communication apparatus that forms links in a first layer of the network, based on information related to a second layer that differs from the first layer, and setting a first packet handling operation in accordance with the determined destination to accommodate communication flows, in the first communication apparatus.
  • this method is associated with a particular mechanism known as a control apparatus that controls communication apparatuses included in a hierarchical network.
  • a program to execute, on a computer comprising a control apparatus that controls a communication apparatus included in a hierarchical network, a process of determining a destination to accommodate communication flows via a first communication apparatus that forms links in a first layer of the network, based on information related to a second layer that differs from the first layer, and a process of setting a first packet handling operation in accordance with the determined destination to accommodate communication flows, in the first communication apparatus.
  • this program may be recorded on a computer-readable storage medium.
  • the storage medium may be a non-transient storage medium such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium or the like.
  • the present invention may be embodied as a computer program product.
  • a communication system a control apparatus, a control method and a program for the control apparatus, which contribute, in a centrally controlled network in which control targets are expanded, to efficiently operating the overall network.
  • FIG. 1 is a diagram for describing an outline of an exemplary embodiment.
  • FIG. 2 is a diagram showing an example of a configuration of a communication system according to a first exemplary embodiment.
  • FIG. 3 is a diagram showing an example of a configuration of connections between OpenFlow switches 10 - 1 and 10 - 2 shown in FIG. 2 .
  • FIG. 4 is a diagram showing an example of an internal configuration of an OpenFlow switch 10 .
  • FIG. 5 is an example of a table set in a table DB 13 of the OpenFlow switch 10 - 1 .
  • FIG. 6 is a diagram showing an example of an internal configuration of a packet transport node 50 .
  • FIG. 7 is an example of a table set in a table DB 13 of a packet transport node 50 - 1 .
  • FIG. 8 is an example of a block diagram representing a configuration example of a control apparatus 20 .
  • FIG. 9 is a diagram showing an example in which properties related to the OpenFlow switch 10 - 1 are summarized.
  • FIG. 10 is a diagram showing an example of properties related to packet transport nodes 50 - 1 and 50 - 3 .
  • FIG. 11 is a diagram showing an example of network paths contained in a network path DB 25 .
  • FIG. 12 is a sequence diagram for describing operations of the communication system according to the first exemplary embodiment.
  • FIG. 13 is a flowchart showing an example of operations of the control apparatus 20 .
  • FIG. 14 is a diagram summarizing the total number of TCP/IP flows accommodated in candidate pseudo lines.
  • FIG. 15 is a diagram reflecting a network path corresponding to a TCP/IP flow of ID 13 with regard to FIG. 11 .
  • FIG. 16 is a diagram showing an example of a configuration of connections between the packet transport nodes 50 - 1 and 50 - 2 shown in FIG. 3 .
  • FIG. 17 is a diagram showing an example of an internal configuration of an optical cross-connect 60 .
  • FIG. 18 is an example of a table set in the table DB 13 of an optical cross-connect 60 - 1 .
  • FIG. 19 is a diagram showing an example in which properties related to optical cross-connects 60 - 1 and 60 - 3 are summarized.
  • FIG. 20 is a diagram showing an example of network paths contained in a network path DB 25 .
  • FIG. 21 is a flowchart showing an example of operations of a control apparatus 20 a.
  • FIG. 22 is a diagram summarizing the total number of TCP/IP flows accommodated in candidate optical paths.
  • FIG. 23 is a diagram reflecting a network path corresponding to a TCP/IP flow of ID 33 with respect to FIG. 20 .
  • FIG. 24 is a diagram showing an example of a configuration of connections between the packet transport nodes 50 - 1 and 50 - 2 shown in FIG. 3 .
  • FIG. 25 is a diagram showing relationships of 3 pseudo lines and optical paths included in the respective pseudo lines.
  • FIG. 26 is a diagram showing an example of network paths accommodated in a network path DB 25 .
  • FIG. 27 is a flowchart showing an example of operations of a control apparatus 20 b.
  • FIG. 28 is a diagram summarizing maximum values and average values calculated for each pseudo line.
  • FIG. 29 is a diagram reflecting a network path corresponding to a TCP/IP flow of ID 44 with respect to FIG. 26 .
  • FIG. 30 is a diagram summarizing maximum values and average values calculated for each pseudo line.
  • FIG. 31 is a diagram showing an example of respective parameters for optical fiber cables configuring the network shown in FIG. 24 .
  • FIG. 32 is a diagram summarizing bandwidth estimated values for each optical fiber cable used by the respective pseudo lines.
  • FIG. 1 a description is given concerning an outline of an exemplary embodiment using FIG. 1 . It is to be noted that reference symbols in the drawings attached to this outline are added to respective elements for convenience, as examples in order to aid understanding, and descriptions of the outline are not intended to limit the invention in any way.
  • the communication system shown in FIG. 1 includes a control apparatus 100 that controls communication apparatuses included in a hierarchical network, and first communication apparatus 101 that form links in a first layer of the network and perform processing related to communication flows based on a first processing rule (i.e., packet handling operation).
  • the control apparatus 100 determines destinations for accommodating communication flows specified according to the first packet handling operation, based on information related to a second layer that differs from the first layer.
  • the control apparatus 100 determines destinations for accommodating communication flows specified according to a packet handling operation set in a first communication apparatus 101 , based on information related to the second layer.
  • a destination to accommodate a communication flow received via the first communication apparatus 101 is determined without giving consideration to the second layer, there is a possibility of communication flows in the second layer being unevenly distributed.
  • a destination to accommodate communication flows is determined while using information related to the second layer, as in the control apparatus 100 , it is possible to avoid uneven distribution of communication flows in the second layer.
  • control apparatus 100 determines destinations for accommodating communication flows in an upper layer (for example, the first layer) so as to avoid duplication of communication flows in a lower layer (for example, the second layer) (in anticipation of a statistical multiplexing effect in the lower layer). As a result, it becomes possible to efficiently operate the network.
  • an upper layer for example, the first layer
  • a lower layer for example, the second layer
  • FIG. 2 is a diagram showing an example of a configuration of a communication system according to the first exemplary embodiment.
  • FIG. 2 shows a configuration that includes OpenFlow switches 10 - 1 to 10 - 3 (OFS 10 - 1 to OFS 10 - 3 ) that implement communication between a communication terminal 30 and servers 40 - 1 and 40 - 2 , and a control apparatus 20 that controls these OpenFlow switches 10 - 1 to 10 - 3 .
  • OpenFlow switches 10 - 1 to 10 - 3 OFS 10 - 1 to OFS 10 - 3
  • FIG. 2 shows a configuration that includes OpenFlow switches 10 - 1 to 10 - 3 (OFS 10 - 1 to OFS 10 - 3 ) that implement communication between a communication terminal 30 and servers 40 - 1 and 40 - 2 , and a control apparatus 20 that controls these OpenFlow switches 10 - 1 to 10 - 3 .
  • the control apparatus 20 sets packet processing rules (i.e., packet handling rules, termed herein as “packet handling operations”) specifying processing for received packets, with respect to the OpenFlow switches 10 - 1 to 10 - 3 .
  • packet handling rules i.e., packet handling rules, termed herein as “packet handling operations”
  • the OpenFlow switches 10 - 1 to 10 - 3 perform packet processing (packet forwarding) in accordance with the packet handling operations set by the control apparatus 20 .
  • the OpenFlow switch 10 - 1 makes an enquiry to the control apparatus 20 regarding processing for the received packet.
  • the control apparatus 20 that receives the enquiry generates a packet handling operation for the received packet and makes a setting in the OpenFlow switch 10 - 1 .
  • the control apparatus 20 may generate a packet handling operation related to a certain packet. In this case, it is possible for the control apparatus 20 to spontaneously set a packet handling operation for the OpenFlow switch 10 - 1 .
  • the configuration shown in FIG. 2 corresponds to connections on logical paths between the OpenFlow switches 10 - 1 , 10 - 2 and 10 - 3 .
  • the connections in FIG. 2 indicate connection modes in a third layer (network layer).
  • connections between respective OpenFlow switches are via a second layer (data link layer) and a first layer (physical layer).
  • FIG. 3 is a diagram showing an example of a configuration of connections between OpenFlow switches 10 - 1 and 10 - 2 shown in FIG. 2 .
  • FIG. 3 shows a configuration that includes packet transport nodes 50 - 1 to 50 - 4 (PTN 50 - 1 to PTN 50 - 4 ) that implement communication between OpenFlow switches 10 - 1 and 10 - 2 .
  • the OpenFlow switch 10 - 1 and the packet transport nodes 50 - 1 and 50 - 3 are connected via an Ethernet (registered trademark; the same applies below) cable or the like.
  • the OpenFlow switch 10 - 2 is connected to the packet transport nodes 50 - 2 and 50 - 4 .
  • the packet transport nodes 50 - 1 and 50 - 2 are connected using a transmission medium such as an Ethernet cable, an optical fiber cable, or the like.
  • the packet transport nodes 50 - 3 and 50 - 4 are similarly connected.
  • Pseudo lines connecting the OpenFlow switches 10 - 1 and 10 - 2 are formed between the packet transport nodes. More specifically, pseudo lines are formed at the respective packet transport nodes by applying MPLS-TP (Multi Protocol Label Switching-Transport Profile) technology.
  • MPLS-TP Multi Protocol Label Switching-Transport Profile
  • received packets are encapsulated using MPLS labels when packets are forwarded.
  • the packet transport nodes on the receiving side that receive the encapsulated received packets release the encapsulation of the packets (decapsulation) using MPLS labels.
  • FIG. 3 does not show connections between packet transport node 50 - 1 and packet transport node 50 - 4 , and between packet transport node 50 - 3 and packet transport node 50 - 2 .
  • the number of pseudo lines formed between the packet transport nodes is 2, but this is also an example and there is no limitation to the number of pseudo lines.
  • pseudo lines are described here but it is possible also to have examples using logical lines, logical paths, lines, paths, or the like.
  • the OpenFlow switch 10 - 1 forwards packets towards the OpenFlow switch 10 - 2 , a selection is made either to forward the packets to the packet transport node 50 - 1 , or to forward the packets to the packet transport node 50 - 3 (selection of port).
  • the OpenFlow switch 10 - 2 receives packets from the packet transport nodes 50 - 2 or 50 - 4 , and processes the packets in accordance with a packet handling operation.
  • OpenFlow switches 10 - 1 and 10 - 2 are connected by a plurality of pseudo lines formed between a plurality of packet transport nodes.
  • the control apparatus 20 sets packet handling operations specifying processing when packets are received by respective packet transport nodes, with respect to the packet transport nodes 50 - 1 to 50 - 4 . More specifically, the control apparatus 20 sets pseudo lines accommodating TCP/IP flows in response to TCP/IP flows received by the respective packet transport nodes. For example, a packet handling operation is set in the packet transport node 50 - 1 , such as a certain TCP/IP flow being accommodated in pseudo line 1 , and another TCP/IP flow being accommodated in pseudo line 2 .
  • Determination of pseudo lines accommodating TCP/IP flows is implemented by specifying MPLS level when received packets are encapsulated.
  • MPLS level By specifying the MPLS level, it is possible to specify a specific pseudo line from among a plurality of pseudo lines formed between respective packet transport nodes.
  • the packet transport nodes 50 - 1 to 50 - 4 perform packet processing (packet encapsulating) in accordance with packet handling operation(s) set by the control apparatus 20 .
  • the packet transport nodes 50 - 1 to 50 - 4 make an enquiry to the control apparatus 20 regarding processing for the received packet.
  • the control apparatus 20 that receives the enquiry generates a packet handling operation for the received packet and makes a setting in the packet transport nodes 50 - 1 to 50 - 4 .
  • OpenFlow switches and packet transport nodes are targets for control by the control apparatus 20 .
  • FIG. 4 is a diagram showing an example of an internal configuration of the OpenFlow switches 10 (described as “OpenFlow switch 10 ” below, when there is no particular need to distinguish between OpenFlow switches 10 - 1 to 10 - 3 ).
  • the OpenFlow switch 10 is configured to include a communication unit 11 , a table management unit 12 , a table database (table DB) 13 , and a forwarding processing unit 14 .
  • the communication unit 11 is a means for implementing communication with the control apparatus 20 that sets packet handling operations in the OpenFlow switch 10 .
  • the communication unit 11 communicates with the control apparatus 20 using an OpenFlow protocol of Non Patent Literature 2.
  • the communication protocol between the communication unit 11 and the control apparatus 20 is not limited to the OpenFlow protocol.
  • the communication unit 11 notifies the control apparatus 20 of properties (number of connection ports, connection destination, IP address, MAC address, and the like) related to the OpenFlow switch 10 , in accordance with a request from the control apparatus 20 .
  • the table management unit 12 is a means for managing a table held by the table DB 13 . More specifically, the table management unit 12 records packet handling operations instructed by the control apparatus 20 in the table DB 13 , and when notified that a new packet has been received from the forwarding processing unit 14 , requests the control apparatus 20 to set a packet handling operation. In a case where an expiry condition for a packet handling operation stored in respective tables is established, the table management unit 12 performs processing to delete or invalidate the packet handling operation.
  • the table DB 13 is configured by a database that can store 1 or more tables to be referred to, when the forwarding processing unit 14 performs processing of a received packet.
  • the forwarding processing unit 14 is configured to include a table retrieval unit 141 and an action execution unit 142 .
  • the table retrieval unit 141 is a means for retrieving a packet handling operation having a match field that matches a received packet, from a table stored in the table DB 13 .
  • the action execution unit 142 is a means for performing packet processing in accordance with processing connect indicated in an instruction field of the packet handling operation retrieved by the table retrieval unit 141 . In a case where a packet handling operation having a match field matching a received packet is not found, the forwarding processing unit 14 notifies this to the table management unit 12 . Furthermore, the forwarding processing unit 14 updates statistical information recorded in the table DB 13 , in response to the packet processing.
  • FIG. 5 is an example of a table set in the table DB 13 of the OpenFlow switch 10 - 1 .
  • a packet handling operation is set that implements forwarding of a TCP/IP flow received by the OpenFlow switch 10 - 1 to any of the packet transport nodes 50 - 1 to 50 - 3 .
  • a 1 is set as a transmission source IP address
  • a 2 is set as a destination IP address
  • the first processing from the top in FIG. 5 is executed.
  • the table retrieval unit 141 of the OpenFlow switch 10 - 1 retrieves the first packet handling operation from the top of the table in FIG. 5 , as a packet handling operation matching the received packet.
  • the action execution unit 142 of the OpenFlow switch 10 - 1 forwards the relevant received packet from a port connected to the packet transport node 50 - 1 , in accordance with content shown in an instruction field thereof. Similarly, in a case of receiving a TCP/IP flow where B 1 is set as a transmission source IP address and B 2 is set as a destination IP address, the OpenFlow switch 10 - 1 forwards the relevant packet from a port connected to the packet transport node 50 - 3 . It is to be noted that, in a case where no packet handling operation corresponding to the received packet exists, the OpenFlow switch 10 requests setting of a packet handling operation, with respect to the control apparatus 20 .
  • time T 1 and time T 2 are respectively set as “Time To Live” (TTL) in expiry conditions of respective packet handling operations.
  • TTL Time To Live
  • the table management unit 12 performs an operation to delete the packet handling operation in question.
  • the forwarding processing unit 14 initializes a timer that manages “Time To Live” whenever the respective packet handling operations are executed.
  • Statistical information corresponding to the respective packet handling operations is updated whenever the relevant packet handling operation is executed.
  • the packet handling operation as described above is similarly set in the OpenFlow switch 10 - 2 and the OpenFlow switch 10 - 3 .
  • FIG. 6 is a diagram showing an example of an internal configuration of a packet transport node 50 (described as “packet transport node 50 ” below when there is no particular need to distinguish between the packet transport nodes 50 - 1 to 50 - 4 ).
  • the main internal configuration of the packet transport node 50 matches the main internal configuration of the OpenFlow switch 10 shown in FIG. 4 . Accordingly, a detailed description concerning the internal configuration of the packet transport node 50 is omitted.
  • a point of difference between the OpenFlow switch 10 and the packet transport node 50 is that packet handling operations recorded in the table DB 13 are different. With the packet handling operations recorded in the table DB 13 being different, contents of packet processing executed in accordance with the relevant packet handling operations by the action execution unit 142 also differ.
  • FIG. 7 is an example of a table set in the table DB 13 of the packet transport node 50 - 1 .
  • a packet handling operation is set that implements forwarding of packets using a pseudo line 1 or a pseudo line 2 for TCP/IP flows received by the packet transport node 50 - 1 .
  • a 1 is set as a transmission source IP address
  • a 2 is set as a destination IP address
  • the first processing from the top in FIG. 7 is executed.
  • the table retrieval unit 141 of the packet transport node 50 - 1 retrieves the first packet handling operation from the top of the table in FIG. 7 , as a packet handling operation matching the received packet.
  • the action execution unit 142 of the packet transport node 50 - 1 then performs packet processing (uses the pseudo line 1 ) in accordance with content shown in an instruction field thereof. More specifically, the action execution unit 142 of the packet transport node 50 - 1 encapsulates a received packet using an MPLS label corresponding to the pseudo line 1 .
  • the “Time To Live” is set as an expiry condition, similar to the OpenFlow switch 10 .
  • the packet transport node 50 selects a pseudo line when forwarding a received packet, in accordance with a packet handling operation set by the control apparatus 20 .
  • the control apparatus 20 determines a pseudo line accommodating the relevant TCP/IP flow, for each TCP/IP flow received by the packet transport node 50 .
  • FIG. 8 is an example of a block diagram reflecting a configuration example of the control apparatus 20 .
  • the control apparatus 20 is configured to include: a network configuration management unit 21 , a network configuration database (network configuration DB) 22 , a control message processing unit 23 , a path/action calculation unit 24 , a network path database (network path DB) 25 , a packet handling operation management unit 26 , a packet handling operation database (packet handling operation DB) 27 , and a node communication unit 28 that performs communication with the OpenFlow switch 10 and the packet transport node 50 .
  • a network configuration management unit 21 a network configuration database (network configuration DB) 22 , a control message processing unit 23 , a path/action calculation unit 24 , a network path database (network path DB) 25 , a packet handling operation management unit 26 , a packet handling operation database (packet handling operation DB) 27 , and a node communication unit 28 that performs communication with the OpenFlow switch 10 and the packet
  • the network configuration management unit 21 manages network configuration of the communication system by consolidating properties of nodes (OpenFlow switch 10 and packet transport node 50 ) included in the communication system, which are control targets of the control apparatus 20 , via the node communication unit 28 . More specifically, transmission requests related to respective properties are made to the OpenFlow switch 10 and the packet transport node 50 when the control apparatus 20 is initially started up (booted), or periodically. The OpenFlow switch 10 or the like that receives the request responds with its own properties to the control apparatus 20 . The network configuration management unit 21 summarizes these responses and records them in the network configuration DB 22 .
  • FIG. 9 is a diagram showing an example in which properties related to the OpenFlow switch 10 - 1 are summarized. It may be understood from FIG. 9 that connection destinations of the OpenFlow switch 10 - 1 are the OpenFlow switches 10 - 2 and 10 - 3 . There are 2 ports existing that can be selected when forwarding a packet towards the OpenFlow switch 10 - 2 , and they are respectively connected to the packet transport nodes 50 - 1 and 50 - 3 .
  • the network configuration management unit 21 summarizes such information to be recorded in the network configuration DB 22 .
  • FIG. 10 is a diagram showing an example of properties related to packet transport nodes 50 - 1 and 50 - 3 . From FIG. 10 it may be understood that a port of the packet transport node 50 - 1 is connected to the packet transport node 50 - 2 . Similarly, a port of the packet transport node 50 - 3 is connected to the packet transport node 50 - 4 . It is understood that there are 3 lines, pseudo lines 1 - 1 to 1 - 3 , that can be formed between the packet transport node 50 - 1 and the packet transport node 50 - 2 . Furthermore, there are 2 lines, pseudo lines 2 - 1 to 2 - 2 , which can be formed between the packet transport node 50 - 3 and the packet transport node 50 - 4 . It is to be noted that the number of pseudo lines that can be formed between the packet transport nodes shown in FIG. 10 is exemplary, and there is no limitation with regard to the number of lines shown in FIG. 10 .
  • the network configuration management unit 21 summarizes such information to be recorded in the network configuration DB 22 .
  • the control message processing unit 23 analyzes control messages received from the OpenFlow switch 10 or the packet transport node 50 , and delivers the control message information to a relevant processing means within the control apparatus 20 .
  • the path/action calculation unit 24 calculates a network path for each TCP/IP flow based on the network configuration stored in the network configuration DB 22 and the network paths stored in the packet handling operation DB 27 . More specifically, the path/action calculation unit 24 determines the forwarding path of received packets in the OpenFlow switch 10 . Or, the path/action calculation unit 24 determines pseudo lines used when received packets are forwarded in the packet transport node 50 .
  • the path/action calculation unit 24 records the calculated network path in the network path DB 25 .
  • the path/action calculation unit 24 delivers the calculated network path to the packet handling operation management unit 26 .
  • FIG. 11 is a diagram showing an example of network paths stored in the network path DB 25 . It is to be noted that in the following description and drawings, TCP/IP flows are distinguished by flow ID. TCP/IP flows with different flow IDs have different combinations of transmission source IP addresses and destination IP addresses.
  • TCP/IP flows with flow IDs of ID 1 to ID 10 are forwarded by logical paths from OpenFlow switch 10 - 1 to OpenFlow switch 10 - 2 .
  • TCP/IP flows with flow IDs of ID 11 to ID 12 are forwarded by logical paths from OpenFlow switch 10 - 1 to OpenFlow switch 10 - 3 .
  • TCP/IP flows with ID 1 to ID 3 are accommodated in pseudo line 1 - 1 .
  • TCP/IP flows with ID 4 and ID 5 are accommodated in pseudo line 1 - 2 .
  • TCP/IP flows with ID 6 and ID 7 are accommodated in pseudo line 1 - 3 .
  • TCP/IP flows with ID 8 and ID 9 are accommodated in pseudo line 2 - 1 .
  • a TCP/IP flow with ID 10 is accommodated in pseudo line 2 - 2 .
  • the path/action calculation unit 24 calculates a pseudo line and a logical path accommodating an unknown TCP/IP flow, based on the network configuration recorded in the network configuration DB 22 and the network paths recorded in the network path DB 25 .
  • the path/action calculation unit 24 functions also as a means for calculating an expiry condition for each packet handling operation and determining a value to be set in an expiry condition field of each packet handling operation. For example, the path/action calculation unit 24 determines “Time To Live” of packet handling operations corresponding to respective TCP/IP flows, in the OpenFlow switch 10 or the packet transport node 50 . It is to be noted that a predetermined value can be used in determining “Time To Live” for a packet handling operation. Or, information of transmission source IP address, destination IP address, port number or the like may be used to individually set “Time To Live” in packet handling operations.
  • the packet handling operation management unit 26 manages packet handling operations set in the OpenFlow switch 10 and the packet transport node 50 . Specially, packet handling operations to be set in the OpenFlow switch 10 and the packet transport node 50 are generated based on the network path calculated in the path/action calculation unit 24 , and the packet handling operations of each thereof are recorded in the packet handling operation DB 27 . In a case where a change occurs in a packet handling operation set in the OpenFlow switch 10 , due to a packet handling operation delete notification from the OpenFlow switch 10 or the like, corresponding content of the packet handling operation DB 27 is changed.
  • the packet handling operation DB 27 stores packet handling operations to be set in the OpenFlow switch 10 and the packet transport node 50 .
  • the respective units (processing means) of the OpenFlow switch 10 and the packet transport node 50 shown in FIG. 4 and FIG. 6 , and the control apparatus 20 shown in FIG. 8 can be implemented by a computer program that executes the respective processing to be described below, on a computer configuring these devices, using hardware thereof.
  • FIG. 12 is exemplary and operations of the communication system according to the present exemplary embodiment are not limited to a sequence of FIG. 12 .
  • FIG. 12 is a sequence diagram for describing operations of the communication system according to the first exemplary embodiment.
  • network paths shown in FIG. 11 are recorded in the network path DB 25 of the control apparatus 20 .
  • properties concerning the OpenFlow switch 10 - 1 relate to content shown in FIG. 9
  • properties concerning the packet transport nodes 50 - 1 and 50 - 3 relate to content shown in FIG. 10 .
  • the OpenFlow switch 10 - 1 receives an unknown TCP/IP flow from a communication terminal 30 (step S 001 in FIG. 12 ). It is to be noted that the flow ID of the TCP/IP flow in question is ID 13 .
  • the OpenFlow switch 10 - 1 In the OpenFlow switch 10 - 1 that receives a packet corresponding to the TCP/IP flow in question, a search is made for a packet handling operation having a match field that conforms with the received packet, from the table, but such a packet handling operation is not found. Accordingly, the OpenFlow switch 10 - 1 requests the control apparatus 20 to set a packet handling operation accompanying the received packet (step S 002 in FIG. 12 ; Packet-In).
  • the control apparatus 20 that receives the request calculates a network path to accommodate a TCP/IP flow having a flow ID of ID 13 , and generates a packet handling operation to be set in the OpenFlow switch and the packet transport node.
  • the network path for the TCP/IP flow with ID 13 is calculated using a logical path between the OpenFlow switches 10 - 1 and 10 - 2 , and also a pseudo line 2 - 2 , and the relevant packet handling operation is generated. Note that details are described below concerning calculation of the network path and generation of the packet handling operation in the control apparatus 20 .
  • the control apparatus 20 that generates the packet handling operation sets the packet handling operation in the OpenFlow switch 10 - 1 and the packet transport node 50 - 3 (step S 003 in FIG. 12 ; FlowMod).
  • the control apparatus 20 returns the packet received in step S 002 to the OpenFlow switch 10 - 1 , and instructs that the packet in question be forwarded (step S 004 in FIG. 12 ; Packet-Out).
  • the OpenFlow switch 10 - 1 forwards the received packet to the packet transport node 50 - 3 (step S 005 in FIG. 12 ).
  • the packet transport node 50 - 3 that receives the packet forwarded from the OpenFlow switch 10 - 1 accommodates the TCP/IP flow with ID 13 in the pseudo line 2 - 2 , in accordance with the packet handling operation that has been set (step S 005 in FIG. 12 ).
  • the packet forwarded from the packet transport node 50 - 3 arrives at the OpenFlow switch 10 - 2 , via the packet transport node 50 - 4 .
  • the OpenFlow switch 10 - 1 and the packet transport node 50 - 3 forward the packet corresponding to the TCP flow with ID 13 to the OpenFlow switch 10 - 2 , without making an enquiry to the control apparatus 20 (step S 006 in FIG. 12 ).
  • FIG. 13 is a flowchart showing an example of operations of the control apparatus 20 . It is to be noted that a description is given of calculation of the network path and setting of the packet handling operation, taking as an example the abovementioned calculation of the network path and setting of the packet handling operation related to the TCP/IP flow with ID 13 .
  • step S 101 the path/action calculation unit 24 is instructed to perform calculation of the network path for the TCP/IP flow with ID 13 , by the control message processing unit 23 .
  • step S 102 the path/action calculation unit 24 decides whether to forward the packet to either the OpenFlow switch 10 - 2 or the OpenFlow switch 10 - 3 from the OpenFlow switch 10 - 1 , from the destination address of the received packet.
  • a logical path to accommodate the TCP/IP flow with ID 9 is determined.
  • the forwarding destination of the received packet is determined in accordance with a routing table possessed by the control apparatus 20 .
  • a decision is made that the received packet described above is a packet that should be forwarded to the OpenFlow switch 10 - 2 .
  • step S 103 the path/action calculation unit 24 lists up pseudo line candidates that can implement a logical path from the OpenFlow switch 10 - 1 to the OpenFlow switch 10 - 2 .
  • the path/action calculation unit 24 obtains properties concerning the OpenFlow switch 10 - 1 shown in FIG. 9 , and properties concerning the packet transport nodes 50 - 1 and 50 - 3 shown in FIG. 10 , from the network configuration DB 22 .
  • step S 104 the path/action calculation unit 24 performs statistical processing for each listed pseudo line. More specifically, the total number of TCP/IP flows accommodated in the respective pseudo lines is counted. Or, in the statistical processing performed by the path/action calculation unit 24 , it is possible to perform any of: calculation of an average value of the TCP/IP flows accommodated in the respective pseudo lines, calculation of a moving average, calculation of a maximum or minimum value, or to use an arbitrary combination of these.
  • the path/action calculation unit 24 uses the network path stored in the network path DB 25 . For example, candidate pseudo lines are extracted from among the pseudo lines shown in FIG. 11 and the number of candidate pseudo lines is counted.
  • FIG. 14 is a diagram summarizing the total of TCP/IP flows accommodated in the candidate pseudo lines.
  • step S 105 the path/action calculation unit 24 determines a pseudo line with the minimum number of TCP/IP flows accommodated in the candidate pseudo lines. In the example of FIG. 14 , this corresponds to pseudo line 2 - 2 . At a point in time when the present step ends, a pseudo line and a logical path accommodating the TCP/IP flow with ID 13 are confirmed.
  • step S 106 the path/action calculation unit 24 reflects the confirmed TCP/IP flow with ID 13 , in a network path stored in the network path DB 25 (update of the network path DB 25 ). More specifically, a destination where the TCP/IP flow with ID 13 is accommodated is confirmed as below.
  • the path/action calculation unit 24 records the confirmed network path in the network path DB 25 .
  • FIG. 15 is a diagram reflecting a network path corresponding to the TCP/IP flow with ID 13 , with respect to FIG. 11 .
  • the path/action calculation unit 24 delivers the confirmed network path to the packet handling operation management unit 26 .
  • step S 107 the packet handling operation management unit 26 generates packet handling operations set in the OpenFlow switch 10 - 1 and the packet transport node 50 - 3 , from the delivered network path.
  • the pseudo line 2 - 2 is used as a network path, a packet handling operation is generated to forward a received packet corresponding to the TCP/IP flow with ID 13 to a port directed towards the packet transport node 50 - 3 , with regard to the OpenFlow switch 10 - 1 .
  • the packet handling operation management unit 26 generates a packet handling operation using the pseudo line 2 - 2 , with regard to the packet transport node 50 - 3 .
  • step S 108 the packet handling operation management unit 26 records the generated packet handling operation in the packet handling operation DB 27 .
  • the packet handling operation management unit 26 gives an instruction to set the generated packet handling operation to the OpenFlow switch 10 - 1 and the packet transport node 50 - 3 , with respect to the control message processing unit 23 .
  • the description above concerns calculation of network paths and setting of packet handling operations in the control apparatus 20 .
  • the description given concerns a case where the control apparatus 20 according to the present exemplary embodiment sets packet handling operations in the OpenFlow switch 10 and the packet transport node 50 .
  • apparatus, and connection modes thereof, used in the communication system that is a control target of the control apparatus 20 where it is necessary to set a packet handling operation limited to either the OpenFlow switch 10 or the packet transport node 50 .
  • the OpenFlow switch 10 is provided with 1 output port, with the exception of where a received packet or the like is dropped a packet handling operation is set to put a packet to the output port in question.
  • assigning packets to each communication flow is implemented in the packet transport node 50 .
  • control apparatus 20 has a plurality of OpenFlow switches 10 or packet transport nodes 50 as control targets.
  • control apparatus 20 may have a plurality of apparatuses that are targets for control by the control apparatus 20 do not exist.
  • pseudo lines accommodating a TCP/IP flow with ID 13 as described above is determined, if consideration is not given to the total number of TCP/IP flows already accommodated in the pseudo lines, there is a possibility that pseudo lines formed between the packet transport nodes 50 - 1 and 50 - 2 will be used. There is a possibility that, in the OpenFlow switch 10 - 1 , a forwarding port for a received packet may be determined as a port directed towards the packet transport node 50 - 1 , rather than a port directed towards the packet transport node 50 - 3 .
  • TCP/IP flows are accommodated in pseudo lines between the packet transport nodes 50 - 1 and 50 - 2 , before the TCP/IP flow with ID 13 is generated. Furthermore, 3 TCP/IP flows are accommodated in pseudo lines between the packet transport nodes 50 - 3 and 50 - 4 (refer to FIG. 11 ).
  • having the TCP/IP flow with ID 13 accommodated in a pseudo line between the packet transport nodes 50 - 1 and 50 - 2 is unfavorable. That is, having the TCP/IP flow with ID 13 accommodated in a pseudo line between the packet transport nodes 50 - 3 and 50 - 4 contributes to distribution of load and performance improvement (securing of bandwidth, having low delay, etc.) in the communication system.
  • OpenFlow by expanding the targets of control by OpenFlow, it is possible to efficiently operate the overall communication system.
  • optical cross-connects that connect between packet transport nodes with optical fiber cable are also objects for control by a control apparatus.
  • Configurations between the OpenFlow switches and the packet transport nodes of the communication system according to the present exemplary embodiment are the same as the configurations shown in FIG. 2 and FIG. 3 . Therefore, descriptions corresponding to FIG. 2 and FIG. 3 in the communication system according to the present exemplary embodiment are omitted.
  • FIG. 16 is a diagram showing an example of a configuration connecting the packet transport nodes 50 - 1 and 50 - 2 shown in FIG. 3 .
  • a configuration is shown that includes optical cross-connects 60 - 1 to 60 - 4 (OXC 60 - 1 to OXC 60 - 4 ) that implement communication between packet transport nodes 50 - 1 and 50 - 2 .
  • the packet transport node 50 - 1 and the optical cross-connects 60 - 1 and 60 - 3 are connected via an Ethernet cable or the like.
  • the packet transport node 50 - 2 is connected to the optical cross-connects 60 - 2 and 60 - 4 .
  • the optical cross-connects 60 - 1 and 60 - 2 are connected using an optical fiber cable.
  • the optical cross-connects 60 - 3 and 60 - 4 are similarly connected.
  • an optical transmission path (optical path) is formed to connect the packet transport nodes 50 - 1 and 50 - 2 , and data is transmitted.
  • WDM Wavelength Division Multiplex
  • a plurality of optical paths can be formed by specifying a wavelength to be used when transmitting optical data. In the example shown in FIG.
  • FIG. 16 does not show connections between the optical cross-connect 60 - 1 and the optical cross-connect 60 - 4 , and between the optical cross-connect 60 - 3 and the optical cross-connect 60 - 2 .
  • the number of optical paths formed between the respective optical cross-connects is 2, but this is exemplary, and there is no limitation to the number of optical paths.
  • the packet transport node 50 - 1 forwards a packet towards the packet transport node 50 - 2
  • a selection is made either to forward the packet to the optical cross-connect 60 - 1 , or to forward the packet to the optical cross-connect 60 - 3 .
  • the packet transport node 50 - 2 receives the packet from the optical cross-connects 60 - 2 or 60 - 4 , and processes the packets in accordance with a packet handling operation. In this way, the packet transport nodes 50 - 1 and 50 - 2 are connected using a plurality of optical paths formed between a plurality of optical cross-connects.
  • a control apparatus 20 a sets packet handling operations specifying processing when packets are received by respective optical cross-connects, with respect to the optical cross-connects 60 - 1 to 60 - 4 . More specifically, the control apparatus 20 a sets optical paths accommodating TCP/IP flows, in accordance with TCP/IP flows received by the respective optical cross-connects. For example, in the optical cross-connect 60 - 1 a packet handling operation is set, such as a certain TCP/IP flow being accommodated in an optical path 1 , and another TCP/IP flow being accommodated in an optical path 2 .
  • Determination of optical paths accommodating TCP/IP flows is implemented by specifying wavelength to be used in forwarding received packets. By specifying the wavelength, it is possible to specify a specific optical path from among a plurality of optical paths that can be formed between respective optical cross-connects.
  • the optical cross-connects 60 - 1 to 60 - 4 perform packet processing in accordance with a packet handling operation set by the control apparatus 20 a . It is to be noted that in a case where no packet handling operation exists that matches a match field of a received packet, the optical cross-connects 60 - 1 to 60 - 4 make an enquiry to the control apparatus 20 a regarding processing for the received packet.
  • the control apparatus 20 a that receives the enquiry calculates a packet handling operation for the received packet and makes a setting in the optical cross-connects 60 - 1 to 60 - 4 .
  • OpenFlow switches, packet transport nodes and optical cross-connects are targets for control by the control apparatus 20 a .
  • Configurations and operations of the OpenFlow switches and the packet transport nodes included in the communication system according to the present exemplary embodiment have no points of difference from the description of the first exemplary embodiment, and therefore further descriptions are omitted.
  • FIG. 17 is a diagram showing an example of an internal configuration of an optical cross-connect 60 (described as “optical cross-connect 60 ” below when there is no particular need to distinguish between the optical cross-connects 60 - 1 to 60 - 4 ).
  • the main internal configuration of the optical cross-connect 60 conforms with the main internal configuration of the OpenFlow switch 10 shown in FIG. 4 . Accordingly, a detailed description concerning the internal configuration of the optical cross-connect 60 is omitted.
  • a point of difference between the OpenFlow switch 10 and the optical cross-connect 60 is that packet handling operations recorded in a table DB 13 are different. With the packet handling operations recorded in the table DB 13 being different, contents of packet processing executed in accordance with the relevant packet handling operations by an action execution unit 142 are also different.
  • FIG. 18 is an example of a table set in the table DB 13 of the optical cross-connect 60 - 1 .
  • a packet handling operation is set that implements forwarding of packets using an optical path 1 or an optical path 2 for TCP/IP flows received by the optical cross-connect 60 - 1 .
  • a 1 is set as a transmission source IP address
  • a 2 is set as a destination IP address
  • the first processing from the top in FIG. 18 is executed.
  • a table retrieval unit 141 of the optical cross-connect 60 - 1 retrieves the first packet handling operation from the top of the table in FIG. 18 , as a packet handling operation matching the received packet.
  • An action execution unit 142 of the optical cross-connect 60 - 1 then performs packet processing in accordance with content shown in an instruction field thereof (uses the optical path 1 ). More specifically, the action execution unit 142 of the optical cross-connect 60 - 1 uses a wavelength corresponding to the optical path 1 and forwards the received packet. The optical cross-connect 60 - 1 forwards the packet towards the optical cross-connect 60 - 2 .
  • the optical cross-connect 60 - 1 forwards the packet using the optical path 2 .
  • the optical cross-connect 60 requests setting of a packet handling operation, with respect to the control apparatus 20 a .
  • the “Time To Live” is set as an expiry condition, similar to the OpenFlow switch 10 . It is to be noted that an example is shown here using the transmission source IP address and the destination IP address of a packet as decision conditions, but it is also possible to use an arbitrary decision condition. For example, it is also possible to have an input port number as a decision condition.
  • the optical cross-connect 60 selects a pseudo line when forwarding a received packet, in accordance with a packet handling operation set by the control apparatus 20 a .
  • the control apparatus 20 a determines an optical path accommodating the relevant TCP/IP flow, for each TCP/IP flow received by the optical cross-connect 60 .
  • control apparatus 20 a Next, a description is given concerning the control apparatus 20 a.
  • control apparatus 20 a can control (can communicate with) an optical cross-connect.
  • the method of calculating a network path for an unknown flow is different. Since there is no point of difference regarding internal configuration of the control apparatus 20 and the control apparatus 20 a , a description corresponding to FIG. 8 concerning the control apparatus 20 a is omitted. It is to be noted that since the control apparatus 20 a also has the optical cross-connect 60 as a control target, a network configuration management unit 21 obtains properties of the optical cross-connect 60 via a node communication unit 28 . The network configuration management unit 21 manages network configuration of the communication system by consolidating properties of the optical cross-connect 60 , in addition to OpenFlow switches or the like.
  • FIG. 19 is a diagram showing an example in which properties related to the optical cross-connects 60 - 1 and 60 - 3 are summarized. From FIG. 19 it may be understood that a port of the optical cross-connect 60 - 1 is connected to the optical cross-connect 60 - 2 . Similarly, a port of the optical cross-connect 60 - 3 is connected to the optical cross-connect 60 - 4 . Optical paths that can be formed between the optical cross-connect 60 - 1 and the optical cross-connect 60 - 2 are the 3 optical paths 1 - 1 to 1 - 3 .
  • optical paths that can be formed between the optical cross-connect 60 - 3 and the optical cross-connect 60 - 4 are the 2 optical paths 2 - 1 and 2 - 2 . It is to be noted that the number of optical paths that can be formed between the optical cross-connects shown in FIG. 19 is exemplary, and there is no limitation to the number of optical paths shown in FIG. 19 .
  • the network configuration management unit 21 summarizes such information to be recorded in the network configuration DB 22 .
  • FIG. 20 is a diagram showing an example of network paths stored in the network path DB 25 .
  • TCP/IP flows with flow IDs of ID 21 to ID 30 are forwarded by logical paths from OpenFlow switch 10 - 1 to OpenFlow switch 10 - 2 .
  • TCP/IP flows with flow IDs of ID 31 and ID 32 are forwarded by logical paths from OpenFlow switch 10 - 1 to OpenFlow switch 10 - 3 .
  • TCP/IP flows with ID 21 to ID 23 are accommodated in pseudo line 1 - 1 .
  • TCP/IP flows with ID 24 and ID 25 are accommodated in pseudo line 1 - 2 .
  • TCP/IP flows with ID 26 and ID 27 are accommodated in pseudo line 1 - 3 .
  • TCP/IP flows with ID 28 and ID 29 are accommodated in pseudo line 2 - 1 .
  • a TCP/IP flow with ID 30 is accommodated in pseudo line 2 - 2 .
  • a TCP/IP flow with ID 21 is accommodated in optical path 1 - 1 .
  • TCP/IP flows with ID 22 and ID 30 are accommodated in optical path 1 - 2 .
  • TCP/IP flows with ID 23 and ID 29 are accommodated in optical path 1 - 3 .
  • TCP/IP flows with ID 24 and ID 28 are accommodated in optical path 2 - 1 .
  • TCP/IP flows with ID 25 to ID 27 are accommodated in optical path 2 - 2 .
  • the path/action calculation unit 24 calculates a logical path, pseudo line and optical path, to accommodate an unknown TCP/IP flow, based on the network configuration recorded in the network configuration DB 22 and the network paths recorded in the network path DB 25 .
  • control apparatus 20 a differs from the control apparatus 20 with regard to a method of calculating a network path for an unknown flow. More specifically, the calculation of the network path for each TCP/IP flow in the path/action calculation unit 24 is different. Therefore, a description is given concerning the calculation of the network path and setting of a packet handling operation in the control apparatus 20 a.
  • FIG. 21 is a flowchart showing an example of operations of the control apparatus 20 a .
  • FIG. 21 differs from the operational example shown in FIG. 13 with regard to operations of steps S 203 to S 207 . Accordingly, a description concerning other steps is omitted. It is to be noted that the flow ID of the unknown TCP/IP flow is ID 33 .
  • step S 203 the path/action calculation unit 24 lists up optical path candidates that can implement a logical path from the OpenFlow switch 10 - 1 to the OpenFlow switch 10 - 2 .
  • the path/action calculation unit 24 obtains properties concerning the OpenFlow switch 10 - 1 , properties concerning the packet transport nodes 50 - 1 and 50 - 3 , and properties concerning the optical cross-connects 60 - 1 and 60 - 3 , from the network configuration DB 22 .
  • optical path candidates there are 5 optical path candidates: the optical paths 1 - 1 to 1 - 3 formed between the optical cross-connects 60 - 1 and 60 - 2 , and the optical paths 2 - 1 to 2 - 2 formed between the optical cross-connects 60 - 3 and 60 - 4 .
  • step S 204 the path/action calculation unit 24 performs statistical processing for each listed optical path. More specifically, the total number of TCP/IP flows accommodated in the respective optical paths is counted. At this time, the path/action calculation unit 24 uses the network paths stored in the network path DB 25 . For example, candidate optical paths are extracted from among the optical paths shown in FIG. 20 and the number of candidate optical paths is calculated.
  • FIG. 22 is a diagram summarizing the total number of TCP/IP flows accommodated in candidate optical paths.
  • step S 205 the path/action calculation unit 24 determines an optical path with the minimum number of TCP/IP flows that are accommodated in a candidate optical path. In the example of FIG. 22 , this corresponds to the optical path 1 - 1 .
  • a logical path, a pseudo line and an optical path, accommodating the TCP/IP flow with ID 13 are confirmed.
  • an optical path that is to accommodate a TCP/IP flow with ID 33 is confirmed as the optical path 1 - 1 . This is because, when the optical path is confirmed, the pseudo line to be used is confirmed, and a forwarding destination of a received packet is confirmed in the OpenFlow switch 10 - 1 .
  • step S 206 the path/action calculation unit 24 reflects the confirmed TCP/IP flow with ID 33 , in a network path stored in the network path DB 25 (update of the network path DB 25 ). More specifically, a destination where the TCP/IP flow with ID 33 is accommodated is confirmed as below.
  • the path/action calculation unit 24 records the confirmed network path in the network path DB 25 .
  • FIG. 23 is a diagram reflecting a network path corresponding to the TCP/IP flow with ID 33 , with respect to FIG. 20 .
  • the confirmed network path is delivered by the path/action calculation unit 24 to the packet handling operation management unit 26 .
  • step S 207 the packet handling operation management unit 26 generates packet handling operations to be set in the OpenFlow switch 10 - 1 , the packet transport node 50 - 1 and the optical cross-connect 60 - 1 , from the delivered network path.
  • a packet handling operation is generated that forwards a received packet corresponding to the TCP/IP flow with ID 33 to a port directed towards the packet transport node 50 - 1 , with regard to the OpenFlow switch 10 - 1 .
  • the packet handling operation management unit 26 generates a packet handling operation that uses the pseudo line 1 - 1 , with regard to the packet transport node 50 - 1 .
  • the packet handling operation management unit 26 generates a packet handling operation that uses the optical path 1 - 1 , with regard to the optical cross-connect 60 - 1 .
  • the description above concerns calculation of a network path and setting of a packet handling operation in the control apparatus 20 a.
  • the TCP/IP flow with ID 33 is accommodated in pseudo line 2 - 2 .
  • 2 TCP/IP flows (ID 22 and ID 30 ) are already accommodated in the optical path 1 - 2 used by the pseudo line 2 - 2 .
  • the optical path 1 - 1 accommodates only 1 TCP/IP flow (ID 21 ). Therefore, having an unknown TCP/IP flow accommodated in the optical path 1 - 1 rather than in the optical path 1 - 2 contributes more to distribution of load and performance improvement of the communication system.
  • optical cross-connect as an example of a device forms an optical path used by a pseudo line.
  • apparatuses used are not limited to optical cross-connects. It is also possible to have a communication system in which optical paths are formed by devices such as an optical router or the like.
  • a description was given concerning a communication system provided with 1 pair of optical cross-connects between packet transport nodes.
  • a description is given concerning a communication system provided with a plurality of optical cross-connects between packet transport nodes.
  • the configurations between OpenFlow switches and packet transport nodes of the communication system according to the present exemplary embodiment are the same as the configurations shown in FIG. 2 and FIG. 3 . Therefore, descriptions corresponding to FIG. 2 and FIG. 3 in the communication system according to the present exemplary embodiment are omitted.
  • FIG. 24 is a diagram showing an example of a configuration connecting the packet transport nodes 50 - 1 and 50 - 2 shown in FIG. 3 . It is to be noted that a control apparatus 20 b according to the present exemplary embodiment is not illustrated in FIG. 24 , but optical cross-connects 70 - 1 to 70 - 7 perform packet processing in accordance with a packet handling operation set by the control apparatus 20 b.
  • FIG. 25 is a diagram showing relationships of the 3 pseudo lines and optical paths included in the respective pseudo lines. According to FIG. 25 , for example, optical paths 1 , 4 and 7 are included in the pseudo line 1 .
  • a packet forwarded by the pseudo line 1 is delivered to the packet transport node 50 - 2 , via the optical cross-connects 70 - 1 , 70 - 2 , 70 - 5 and 70 - 7 .
  • FIG. 24 is an example of the communication system, and the communication system configuration is not limited to that of FIG. 24 .
  • the control apparatus 20 b sets packet handling operations specifying processing when packets are received by respective optical cross-connects, with respect to the optical cross-connects 70 - 1 to 70 - 7 . More specifically, the control apparatus 20 b sets optical paths accommodating TCP/IP flows, in accordance with TCP/IP flows received by the respective optical cross-connects. For example, in the optical cross-connect 70 - 1 a packet handling operation is set, such as a certain TCP/IP flow being accommodated in the optical path 1 , another TCP/IP flow being accommodated in the optical path 2 , and a further TCP/IP flow being accommodated in an optical path 3 .
  • the optical cross-connects 70 - 1 to 70 - 7 perform packet processing in accordance with a packet handling operation set by the control apparatus 20 b . It is to be noted that in a case where no packet handling operation exists that matches a match field of a received packet, the optical cross-connects 70 - 1 to 70 - 7 make an enquiry to the control apparatus 20 b with respect to the received packet.
  • the control apparatus 20 b that receives the enquiry generates a packet handling operation for the received packet and makes a setting in the optical cross-connects 70 - 1 to 70 - 7 .
  • OpenFlow switches, packet transport nodes and optical cross-connects are targets for control by the control apparatus 20 b .
  • Configurations and operations of the OpenFlow switches and the packet transport nodes included in the communication system according to the present exemplary embodiment have no points of difference from the description of the first exemplary embodiment, and therefore further descriptions are omitted.
  • the optical cross-connect 70 when selecting an optical path, performs the optical path selection by switching connection ports, rather than switching wavelength when transmitting optical data. For example, in the optical cross-connect 70 - 1 , when using the optical path 1 , packets are transmitted to a port directed toward the optical cross-connect 70 - 2 . Therefore, an action execution unit 142 of the optical cross-connect 70 switches connection port by which packets are forwarded, in accordance with a packet handling operation. It is to be noted that, in a case where no packet handling operation corresponding to a received packet exists, the optical cross-connect 70 requests setting of a packet handling operation, with respect to the control apparatus 20 b.
  • the optical cross-connect 70 selects an optical path when forwarding a received packet, in accordance with a packet handling operation set by the control apparatus 20 b .
  • the control apparatus 20 b determines an optical path to accommodate the relevant TCP/IP flow, for each TCP/IP flow received by the optical cross-connect 70 .
  • control apparatus 20 b a description is given concerning the control apparatus 20 b .
  • Methods of calculation network path for an unknown flow are different for the control apparatus 20 a and the control apparatus 20 b . That is, since there is no point of difference regarding internal configuration of the control apparatus 20 and the control apparatus 20 b , a further description concerning the control apparatus 20 b is omitted. Since the control apparatus 20 b also has the optical cross-connect 70 as a control target, a network path including the optical cross-connect 70 is stored in the network path DB 25 .
  • FIG. 26 is a diagram showing an example of network paths stored in the network path DB 25 . It is to be noted that, while a description is omitted in FIG. 26 , TCP/IP flows with flow IDs of ID 41 to ID 43 are forwarded by logical paths from OpenFlow switch 10 - 1 to OpenFlow switch 10 - 2 . TCP/IP flows with ID 41 to ID 42 are respectively accommodated in pseudo lines 1 to 3 . Furthermore, a TCP/IP flow with ID 41 is accommodated in optical path 1 , optical path 4 and optical path 7 . A TCP/IP flow with ID 42 is accommodated in optical path 2 , optical path 5 and optical path 7 . A TCP/IP flow with ID 43 is accommodated in optical path 3 , optical path 6 and optical path 8 .
  • the path/action calculation unit 24 of the control apparatus 20 b calculates a logical path, pseudo line and optical path accommodating an unknown TCP/IP flow, based on the network configuration recorded in the network configuration DB 22 and the network paths recorded in the network path DB 25 .
  • control apparatus 20 b differs from the control apparatus 20 a with regard to method of calculating a network path for an unknown flow. More specifically, calculation of a network path for each TCP/IP flow in the path/action calculation unit 24 is different. Therefore, a description is given concerning calculation of the network path and setting of a packet handling operation in the control apparatus 20 b.
  • FIG. 27 is a flowchart showing an example of operations of the control apparatus 20 b .
  • FIG. 27 differs from the operational example shown in FIG. 21 with regard to operations of steps S 303 to S 307 . Accordingly, a description concerning other steps is omitted. It is to be noted that the ID of the unknown TCP/IP flow is ID 44 .
  • step S 303 the path/action calculation unit 24 lists up pseudo line candidates that can implement a logical path from the OpenFlow switch 10 - 1 to the OpenFlow switch 10 - 2 .
  • the path/action calculation unit 24 obtains properties concerning the OpenFlow switch 10 - 1 , properties concerning the packet transport nodes 50 - 1 and 50 - 3 , and properties concerning the optical cross-connects 70 - 1 to 70 - 7 , from the network configuration DB 22 .
  • pseudo lines 1 to 3 are listed up as pseudo line candidates that can implement a logical path from the OpenFlow switch 10 - 1 to the OpenFlow switch 10 - 2 .
  • step S 304 the path/action calculation unit 24 performs statistical processing based on optical paths included in the listed pseudo lines.
  • the path/action calculation unit 24 uses the network paths stored in the network path DB 25 . More specifically, the path/action calculation unit 24 calculates the maximum value of TCP/IP flows accommodated in the optical paths used by the respective pseudo lines.
  • optical path 1 , optical path 4 and optical path 7 are included in pseudo line 1 . If the number of TCP/IP flows included in each optical path is counted, optical path 1 includes 1 TCP/IP flow, optical path 4 includes 1 TCP/IP flow, and optical path 7 includes 2 TCP/IP flows. Therefore, the maximum value of TCP/IP flows accommodated in an optical path used by pseudo line 1 is 2. A calculation is similarly made for pseudo lines 2 and 3 . As a result, the maximum value of TCP/IP flows accommodated in an optical path used by pseudo line 2 is 2. The maximum value of TCP/IP flows accommodated in an optical path used by pseudo line 3 is 1.
  • the path/action calculation unit 24 calculates the value of: the total number of TCP/IP flows accommodated in the optical paths used by the respective pseudo lines, divided by the number of optical fiber cables used by each of the pseudo lines. Such a value can be regarded as an average value of TCP/IP flows accommodated in 1 optical fiber cable used by the respective pseudo lines.
  • FIG. 28 is a diagram summarizing maximum values and average values calculated for each pseudo line.
  • step S 305 the path/action calculation unit 24 determines a pseudo line for which the calculated maximum value is smallest.
  • the maximum value of pseudo line 3 is 1, pseudo line 3 is selected.
  • a pseudo line accommodating the TCP/IP flow with ID 44 is confirmed.
  • an optical path that is to accommodate a TCP/IP flow with ID 44 is confirmed as optical path 3 .
  • the optical path to be used is confirmed. This is in order to confirm a forwarding destination of an OpenFlow switch accompanying confirmation of the pseudo line.
  • step S 306 the path/action calculation unit 24 reflects the confirmed TCP/IP flow with ID 44 , in a network path stored in the network path DB 25 (updates the network path DB 25 ). More specifically, a destination where the TCP/IP flow with ID 44 is accommodated is confirmed as below.
  • the path/action calculation unit 24 records the confirmed network path in the network path DB 25 .
  • FIG. 29 is a diagram reflecting a network path corresponding to the TCP/IP flow with ID 44 , with respect to FIG. 26 .
  • the confirmed network path is delivered by the path/action calculation unit 24 to the packet handling operation management unit 26 .
  • step S 307 the packet handling operation management unit 26 generates packet handling operations to be set in the OpenFlow switch 10 - 1 , the packet transport node 50 - 1 , and the optical cross-connect 70 - 1 , from the delivered network path. Since the pseudo line 3 is used as a network path, a packet handling operation is generated that forwards a received packet corresponding to the TCP/IP flow with ID 44 to a port directed towards the packet transport node 50 - 1 , with regard to the OpenFlow switch 10 - 1 .
  • the packet handling operation management unit 26 generates a packet handling operation using the pseudo line 3 , with regard to the packet transport node 50 - 1 . Furthermore, the packet handling operation management unit 26 generates a packet handling operation to forward the TCP/IP flow with ID 44 to a port directed towards the optical cross-connect 70 - 4 , with regard to the optical cross-connect 70 - 1 .
  • the description above concerns calculation of a network path and setting of a packet handling operation in the control apparatus 20 b.
  • step S 305 in FIG. 27 the path/action calculation unit 24 determines a pseudo line for which the calculated maximum value is smallest.
  • the maximum values of TCP/IP flows accommodated by respective optical paths may coincide.
  • the path/action calculation unit 24 determines pseudo lines, giving consideration to the number of TCP/IP flows (average value as described above) accommodated in each optical fiber cable.
  • the network path has a state as in FIG. 29 .
  • an analysis should be made as to whether an unknown TCP/IP flow (with assumed ID 45 ) should be accommodated in any of the pseudo lines.
  • calculation of the maximum value and average value of optical paths, as accommodated in optical paths included in each pseudo lines, is as in FIG. 30 .
  • the path/action calculation unit 24 Since maximum values for respective pseudo lines coincide and the value thereof is 2, as shown in FIG. 30 , the path/action calculation unit 24 cannot decide, from the maximum value, whether the TCP/IP flow with ID 45 should be accommodated in any of the pseudo lines. Therefore, the path/action calculation unit 24 uses the calculated average value to determine a pseudo line to accommodate the TCP/IP flow with ID 45 . More specifically, a selection is made of a pseudo line for which the calculated average value is smallest. In the example of FIG. 30 , since the average value of pseudo line 1 or 2 is smaller than pseudo line 3 , pseudo line 1 or 2 is selected.
  • the path/action calculation unit 24 decides the pseudo line to accommodate an unknown TCP/IP flow, based not only on the maximum value of TCP/IP flows accommodated by optical paths included in the respective pseudo line, but also the average value of TCP/IP flows accommodated in 1 optical fiber cable.
  • optical path 7 For example, in the network paths shown in FIG. 26 , if it is decided to accommodate an unknown TCP/IP flow in pseudo line 1 or 2 , then 3 TCP/IP flows are accommodated in optical path 7 .
  • This state may be said to be an imbalanced state when viewed from the overall communication system. That is, avoiding usage of optical path 7 in which 2 TCP/IP flows are already accommodated, as much as possible, and using another optical path contributes to distribution of load balancing and performance improvement in the overall communication system.
  • control apparatus 20 b In the control apparatus 20 b according to the third exemplary embodiment, statistical processing is performed with regard to the number of TCP/IP flows accommodated by optical paths included in the respective pseudo lines, and a destination to accommodate the TCP/IP flows is decided.
  • pseudo lines accommodating TCP/IP flows are decided based on bandwidth (throughput; that is, data quantity forwarded by an optical fiber cable per unit time) of optical fiber cables used by respective optical paths. Since there is no point of difference between the configurations of the communication system according to the present exemplary embodiment and the communication system according to the third exemplary embodiment, further descriptions are omitted. Furthermore, since there is no point of difference concerning internal configurations of the control apparatus 20 c , OpenFlow switch 10 , packet transport node 50 and optical cross-connect 70 , further descriptions are omitted.
  • control apparatus 20 c There is a point of difference between the control apparatus 20 c and the control apparatus 20 b in a method of calculation a network path for an unknown flow by a path/action calculation unit 24 . Therefore, a description is given of the method of calculating a network path for an unknown flow in the control apparatus 20 c.
  • the path/action calculation unit 24 of the control apparatus 20 c estimates bandwidth of optical fiber cable used by respective optical paths, and sets a pseudo line including the optical fiber cable with the minimum estimated bandwidth to a pseudo line accommodating an unknown TCP/IP flow. It is to be noted that the path/action calculation unit 24 uses the following formula (1) to estimate bandwidth for each optical fiber cable.
  • Bandwidth X ⁇ ⁇ Min ⁇ ( W R , C ⁇ R ⁇ P ) ( 1 )
  • W indicates receive window
  • R indicates round trip delay time
  • P indicates packet loss rate
  • C indicates a constant
  • X indicates the number of TCP/IP flows accommodated in respective optical fiber cables
  • ⁇ indicates link bandwidth of respective optical fiber cables.
  • optical fiber cable 1 For a network configuration shown in FIG. 24 , a description is given concerning a result of estimating bandwidth of each optical fiber cable, by associating respective optical paths and optical fiber cables used by the respective optical paths.
  • associating optical paths and optical fiber cables for example, an optical fiber cable used by optical path 1 is taken as optical fiber cable 1 .
  • FIG. 31 is a diagram showing an example of respective parameters described above, for optical fiber cables configuring the network shown in FIG. 24 .
  • Optical fiber cable naming in FIG. 31 corresponds to optical path naming in FIG. 24 .
  • an optical fiber cable implementing optical path 1 is optical fiber cable 1 .
  • optical fiber cables are simply written as “cable”.
  • the number X of TCP/IP flows related to the optical fiber cable 7 corresponding to the optical path 7 is 2.
  • the number X of TCP/IP flows related to other optical fiber cables is 1.
  • the receive window W, round trip delay time R and packet loss rate P related to TCP/IP flows accommodated by the respective optical fiber cables are equivalent.
  • Respective numerical values thereof are as shown in FIG. 31 .
  • the link bandwidth of optical fiber cables 3 , 6 and 8 is smaller than that of other optical fiber cables ( 1/10 of the link bandwidth of other optical fiber cables). It is to be noted that the values of respective parameters shown in FIG. 31 are exemplary, and there is no limitation to these values.
  • the path/action calculation unit 24 of the control apparatus 20 c calculates estimated values for bandwidth for each optical fiber cable used by respective optical paths, using information related to the respective optical fiber cables, as shown in FIG. 31 .
  • FIG. 32 is a diagram summarizing bandwidth estimated values for each optical fiber cable used by the respective pseudo lines.
  • the path/action calculation unit 24 calculates the total of bandwidth estimated values of the optical fiber cables used by the respective pseudo lines, for each pseudo line. Thereupon, as shown in FIG. 32 , the total of the bandwidth estimated values for pseudo line 1 is 0.4; the total of the bandwidth estimated values for pseudo line 2 is 0.4; and the total of the bandwidth estimated values for pseudo line 3 is 3.
  • the path/action calculation unit 24 determines pseudo line 1 or 2 that has the smallest total of bandwidth estimated values, as a pseudo line to accommodate an unknown TCP/IP flow.
  • bandwidth is estimated for the optical fiber cables used by the respective pseudo lines, and a pseudo line with the smallest total of bandwidth estimated values is determined as a pseudo line to accommodate the unknown TCP/IP flow.
  • the communication system comprising a second communication apparatus that forms links in the second layer, and also performs processing related to communication flows based on a second packet handling operation, wherein the control apparatus determines a destination to accommodate communication flows specified according to the first and/or second packet handling operation, based on information related to the second layer.
  • control apparatus determines a destination to accommodate communication flows specified according to the first and/or second packet handling operation, based on information related to a first path that implements links in the second layer.
  • control apparatus determines a destination to accommodate communication flows specified according to the first and/or second packet handling operation, by statistical processing of the number of communication flows accommodated in the first path.
  • control apparatus counts the number of communication flows accommodated in each of a plurality of the first paths, determines the first path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generates the first and/or second packet handling operation in accordance with the determined destination to accommodate the communication flows.
  • the communication system including a third communication apparatus that forms links in a third layer different from the first and second layers, and also performs processing related to communication flows based on a third packet handling operation, wherein the control apparatus determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, based on information related to the third layer.
  • control apparatus determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, by statistical processing of the number of communication flows accommodated in each of a plurality of second paths implementing the third layer.
  • control apparatus counts the number of communication flows accommodated in each of the plurality of second paths, determines the second path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generates at least one of the first to third packet handling operations in accordance with the determined destination to accommodate communication flows.
  • control apparatus calculates an average value of communication flows accommodated in transmission media, based on the number of communication flows accommodated in each transmission medium used by each of the plurality of second paths and the number of transmission media used by each of the second paths, and determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations based on the average value.
  • control apparatus determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, based on an estimated value of bandwidth for each transmission medium used by the plurality of second paths.
  • control apparatus determines a destination to accommodate communication flows specified according to a first packet handling operation set in the first communication apparatus, or a second packet handling operation set in a second communication apparatus that forms links in the second layer, based on information related to the second layer.
  • control apparatus determines a destination to accommodate communication flows specified according to the first and/or second packet handling operation, based on information related to a first path that implements links in the second layer.
  • control apparatus determines a destination to accommodate communication flows specified according to the first and/or second packet handling operation, by statistical processing of the number of communication flows accommodated in the first path.
  • control apparatus counts the number of communication flows accommodated in each of a plurality of the first paths, determines the first path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generates the first and/or second packet handling operation in accordance with the determined destination to accommodate communication flows.
  • control apparatus determines a destination to accommodate communication flows specified according to at least one of the first packet handling operation, the second packet handling operation, and a third packet handling operation set in a third communication apparatus that forms links in a third layer that differs from the first and second layers, based on information related to the third layer.
  • control apparatus determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, by statistical processing of the number of communication flows accommodated in each of a plurality of second paths that implement the third layer.
  • control apparatus counts the number of communication flows accommodated in each of the plurality of second paths, determines the second path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generates at least one of the first to third packet handling operations in accordance with the determined destination to accommodate communication flows.
  • control apparatus calculates an average value of communication flows accommodated in a transmission medium, based on the number of communication flows accommodated in each transmission medium used by each of the plurality of second paths and the number of transmission media used by each of the second paths, and determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations based on the average value.
  • control apparatus determines a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, based on an estimated value of bandwidth for each transmission medium used by the plurality of second paths.
  • the control method of the control apparatus including a step of determining a destination to accommodate communication flows specified according to the first packet handling operation or a first packet handling operation set in a second communication apparatus that forms links in the second layer, based on information related to the second layer.
  • the control method of the control apparatus including a step of determining a destination to accommodate communication flows specified according to the first and/or second packet handling operation, based on information related to a first path that implements links in the second layer.
  • the control method of the control apparatus including a step of determining a destination to accommodate communication flows specified according to the first and/or second packet handling operation, by statistical processing of the number of communication flows accommodated in the first path.
  • the control method of the control apparatus including a step of counting the number of communication flows accommodated in each of a plurality of the first paths, determining the first path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generating the first and/or second packet handling operation in accordance with the determined destination to accommodate communication flows.
  • control method of the control apparatus including a step of determining a destination to accommodate communication flows specified according to at least one of the first packet handling operation, the second packet handling operation, and a third packet handling operation set in a third communication apparatus that forms links in a third layer that differs from the first and second layers, based on information related to the third layer.
  • the control method of the control apparatus including a step of determining a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, by statistical processing of the number of communication flows accommodated in each of a plurality of second paths that implement the third layer.
  • the control method of the control apparatus including a step of counting the number of communication flows accommodated in each of the plurality of second paths, determining the second path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generating at least one of the first to third packet handling operations in accordance with the determined destination to accommodate communication flows.
  • the control method of the control apparatus including a step of calculating an average value of communication flows accommodated in a transmission medium, based on the number of communication flows accommodated in each transmission medium used by each of the plurality of second paths and the number of transmission media used by each of the second paths, and determining a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations based on the average value.
  • the control method of the control apparatus including a step of determining a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, based on an estimated value of bandwidth for each transmission medium used by the plurality of second paths.
  • the program according to the thirty-first mode wherein the program executes a process of determining a destination to accommodate communication flows specified according to the first packet handling operation or a second packet handling operation set in a second communication apparatus that forms links in the second layer, based on information related to the second layer.
  • the program according to the thirty-second mode wherein the program executes a process of determining a destination to accommodate communication flows specified according to the first and/or second packet handling operation, based on information related to a first path that implements links in the second layer.
  • the program according to the thirty-third mode wherein the program executes a process of determining a destination to accommodate communication flows specified according to the first and/or second packet handling operation, by statistical processing of the number of communication flows accommodated in the first path.
  • the program according to the thirty-fourth mode wherein the program executes a process of counting the number of communication flows accommodated in each of a plurality of the first paths, determining the first path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generating the first and/or second packet handling operation in accordance with the determined destination to accommodate communication flows.
  • the program according to any one of the thirty-second to thirty-fifth modes, wherein the program executes a process of determining a destination to accommodate communication flows specified according to at least one of the first packet handling operation, the second packet handling operation, and a third packet handling operation set in a third communication apparatus that forms links in a third layer that differs from the first and second layers, based on information related to the third layer.
  • the program according to the thirty-sixth mode wherein the program executes a process of determining a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, by statistical processing of the number of communication flows accommodated in each of a plurality of second paths that implement the third layer.
  • the program according to the thirty-seventh mode wherein the program executes a process of counting the number of communication flows accommodated in each of the plurality of second paths, determining the second path with the smallest number of accommodated communication flows as a destination to accommodate communication flows, and also generating at least one of the first to third packet handling operations in accordance with the determined destination to accommodate communication flows.
  • the program according to the thirty-eighth mode wherein the program executes a process of calculating an average value of communication flows accommodated in a transmission medium, based on the number of communication flows accommodated in each transmission medium used by each of the plurality of second paths and the number of transmission media used by each of the second paths, and determining a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations based on the average value.
  • the program according to the thirty-seventh mode wherein the program executes a process of determining a destination to accommodate communication flows specified according to at least one of the first to third packet handling operations, based on an estimated value of bandwidth for each transmission medium used by the plurality of second paths.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
US14/433,012 2012-10-03 2013-10-02 Communication system, control apparatus, control method and program Active 2034-05-13 US9906437B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2012221482 2012-10-03
JP2012-221482 2012-10-03
PCT/JP2013/076823 WO2014054691A1 (ja) 2012-10-03 2013-10-02 通信システム、制御装置、制御方法及びプログラム

Publications (2)

Publication Number Publication Date
US20150263939A1 US20150263939A1 (en) 2015-09-17
US9906437B2 true US9906437B2 (en) 2018-02-27

Family

ID=50435012

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/433,012 Active 2034-05-13 US9906437B2 (en) 2012-10-03 2013-10-02 Communication system, control apparatus, control method and program

Country Status (6)

Country Link
US (1) US9906437B2 (ja)
EP (1) EP2905929B1 (ja)
JP (1) JP6492660B2 (ja)
KR (1) KR101734100B1 (ja)
SG (1) SG11201502579XA (ja)
WO (1) WO2014054691A1 (ja)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3764604A1 (en) * 2014-10-07 2021-01-13 Sedonasys Systems Ltd. Systems and methods for managing multi-layer communication networks
JP2016082344A (ja) * 2014-10-15 2016-05-16 日本電気株式会社 制御装置、情報処理システム、制御方法、及び、プログラム
EP3267635B1 (en) 2015-03-06 2020-10-21 Nec Corporation Network control device, network control method, and recording medium for program
JP6733655B2 (ja) * 2015-03-06 2020-08-05 日本電気株式会社 ネットワーク制御装置、ネットワーク制御方法、および、プログラム
CN106301541B (zh) * 2015-06-05 2019-07-30 中兴通讯股份有限公司 设置业务保护的方法、控制器和光网络设备
US10505859B2 (en) * 2016-11-10 2019-12-10 The Government Of The United States Of America, As Represented By The Secretary Of The Navy Packet deadlines in a queue to control the age of information
CN109302504B (zh) * 2017-07-25 2020-08-04 中国移动通信有限公司研究院 Ptn中控制信令通道的建立方法、ptn网元及存储介质
US20210319690A1 (en) * 2018-08-24 2021-10-14 Sumitomo Electric Industries, Ltd. Information providing device, information providing method, information providing system, computer program, and data structure

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004071033A1 (ja) 2003-02-03 2004-08-19 Nippon Telegraph And Telephone Corporation 光ネットワーク、光エッジルータ及びそのプログラム、カットスルー方法およびエッジルータ
JP2005159942A (ja) 2003-11-28 2005-06-16 Nippon Telegr & Teleph Corp <Ntt> 複数レイヤパス設定方法及び複数レイヤパス設定プログラム
JP2005223522A (ja) 2004-02-04 2005-08-18 Nippon Telegr & Teleph Corp <Ntt> 経路計算方法、経路計算制御装置および経路計算プログラム。
US20060209785A1 (en) 2002-05-17 2006-09-21 Paola Iovanna Dynamic routing in packet-switching multi-layer communications networks
US20080002664A1 (en) 2006-06-02 2008-01-03 Huawei Technologies Co., Ltd. Method and system for multi-domain route computation
US20080219268A1 (en) * 2007-03-01 2008-09-11 Dennison Larry R Software control plane for switches and routers
US20100158035A1 (en) 2008-12-19 2010-06-24 Electronics And Telecommunications Research Institute Resource management and recursive path computation for real-time automatic path setup at multi-layer transport network
JP2010148096A (ja) 2008-12-22 2010-07-01 Korea Electronics Telecommun 多階層転送網のｆａ−ｌｓｐに対する階層間ｔｅリンク資源管理方法
US7903557B1 (en) 2005-09-21 2011-03-08 Qlogic, Corporation Multipathing using multiple endpoint addresses for load balancing in a network
US20110261825A1 (en) 2009-03-09 2011-10-27 Nec Corporation OpenFlow COMMUNICATION SYSTEM AND OpenFlow COMMUNICATION METHOD
US20110286324A1 (en) * 2010-05-19 2011-11-24 Elisa Bellagamba Link Failure Detection and Traffic Redirection in an Openflow Network
US20110307628A1 (en) 2010-03-17 2011-12-15 Nec Corporation Communication system, node, control server, communication method and program
JP2012049674A (ja) 2010-08-25 2012-03-08 Nec Corp 通信装置、通信システム、通信方法、および通信プログラム
US20120314605A1 (en) * 2010-03-05 2012-12-13 Ippei Akiyoshi Communication system, path control apparatus, packet forwarding apparatus, and path control method
US9319337B2 (en) * 2011-10-25 2016-04-19 Nicira, Inc. Universal physical control plane

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002044138A (ja) * 2000-07-25 2002-02-08 Nec Corp ネットワークシステム，キャッシュサーバ，中継サーバ，ルータ，キャッシュサーバ制御方法及び記録媒体
JP4599101B2 (ja) * 2004-07-09 2010-12-15 パナソニック株式会社 伝送システム、通信部、伝送方法、プログラムおよび記録媒体
WO2011065268A1 (ja) * 2009-11-26 2011-06-03 日本電気株式会社 負荷分散システム、負荷分散方法、及びプログラム
JP5493965B2 (ja) * 2010-02-15 2014-05-14 日本電気株式会社 帯域制御システム、帯域制御装置、帯域制御方法および帯域制御プログラム

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060209785A1 (en) 2002-05-17 2006-09-21 Paola Iovanna Dynamic routing in packet-switching multi-layer communications networks
US8532087B2 (en) 2003-02-03 2013-09-10 Nippon Telegraph And Telephone Corporation Optical network, optical edge router, program thereof, cut through method, and edge router
US20060126642A1 (en) 2003-02-03 2006-06-15 Nippon Telegraph And Telephone Optical network, optical edge router, program thereof, cut through method, and edge router
WO2004071033A1 (ja) 2003-02-03 2004-08-19 Nippon Telegraph And Telephone Corporation 光ネットワーク、光エッジルータ及びそのプログラム、カットスルー方法およびエッジルータ
JP2005159942A (ja) 2003-11-28 2005-06-16 Nippon Telegr & Teleph Corp <Ntt> 複数レイヤパス設定方法及び複数レイヤパス設定プログラム
JP2005223522A (ja) 2004-02-04 2005-08-18 Nippon Telegr & Teleph Corp <Ntt> 経路計算方法、経路計算制御装置および経路計算プログラム。
US7903557B1 (en) 2005-09-21 2011-03-08 Qlogic, Corporation Multipathing using multiple endpoint addresses for load balancing in a network
US20080002664A1 (en) 2006-06-02 2008-01-03 Huawei Technologies Co., Ltd. Method and system for multi-domain route computation
US20080219268A1 (en) * 2007-03-01 2008-09-11 Dennison Larry R Software control plane for switches and routers
US20100158035A1 (en) 2008-12-19 2010-06-24 Electronics And Telecommunications Research Institute Resource management and recursive path computation for real-time automatic path setup at multi-layer transport network
JP2010148096A (ja) 2008-12-22 2010-07-01 Korea Electronics Telecommun 多階層転送網のｆａ−ｌｓｐに対する階層間ｔｅリンク資源管理方法
US20110261825A1 (en) 2009-03-09 2011-10-27 Nec Corporation OpenFlow COMMUNICATION SYSTEM AND OpenFlow COMMUNICATION METHOD
US20120314605A1 (en) * 2010-03-05 2012-12-13 Ippei Akiyoshi Communication system, path control apparatus, packet forwarding apparatus, and path control method
US20110307628A1 (en) 2010-03-17 2011-12-15 Nec Corporation Communication system, node, control server, communication method and program
US20110286324A1 (en) * 2010-05-19 2011-11-24 Elisa Bellagamba Link Failure Detection and Traffic Redirection in an Openflow Network
JP2012049674A (ja) 2010-08-25 2012-03-08 Nec Corp 通信装置、通信システム、通信方法、および通信プログラム
US9319337B2 (en) * 2011-10-25 2016-04-19 Nicira, Inc. Universal physical control plane

Non-Patent Citations (14)

* Cited by examiner, † Cited by third party
Title
"OpenFlow Switch Specification" Version 1.1.0 Implemented (Wire Protocol 0×02), [online], [searched on May 31, 2012], the Internet <URL:http://www.openflow.org/documents/openflow-spec-v1.1.0.pdf>.
Dongxu Zhang, et al: "Experimental Demonstration of OBS/WSON Multi-layer Optical Switched Networks with an Open Flow-based Unified Control Plane", Optical Network Design and Modeling (ONDM), 2012 16th International Conference on, IEEE, Apr. 17, 2012 (Apr. 17, 2012), pp. 1-6, XP032448926.
Eiichi Horiuchi, et al., "Network Control, Operations and Management for Photonic Networks", Proceedings of Workshop of the Institute of Electronics, Information and Communication Engineers, Mar. 5, 2012, the 111st volume, No. 475, p. 61 to 66.
Hideyuki Shimonishi, et al., "Programmable Network Using OpenFlow for Network Researches and Experiments", International Conference on Mobile Computing and Ubiquitous Networking (ICMU), May 2012.
International Search Report (ISR) (PCT Form PCT/ISA/210), in PCT/JP2013/076823, dated Jan. 7, 2014.
Japanese Office Action dated Nov. 28, 2017 in Japanese Application No. 2014-539784 with an English translation thereof.
King (ED) Old Dog Consulting A Farrel (ED) Old Dog Consulting D: "The Application of the Path Computation Element Architecture to the Determination of a Sequence of Domains in MPLS and GMPLS; draft-ietf-pce-hierarchy-fwk-05.txt", The Application of the Path Computation Element Architecture to the Determination of a Sequence of Domains in MPLS and GMPLS; Draft-IETF-PCE-Hierarchy-FWK-05.Txt, Internet Engineering Task Force, IETF; Standardworkingdraft, Internet Society (ISOC) 4., Aug. 29, 2012 (Aug. 29, 2012), pp. 1-31, XP015087032.
Korean Office Action dated May 19, 2016 with a Japanese and English translation thereof.
Korean Office Action dated Nov. 18, 2016 with a Japanese translation and partial English translation thereof.
L. Liu, et al: "Experimental Validation and Performance Evaluation of Open Flow-Based Wavelength Path Control in Transparent Optical Networks", Dec. 19, 2011 (Dec. 19, 2011), Retrieved from the Internet: URL: https://www.osapublishing.org/view-article.cfm?gotourl=https%3A%2F%2Fwww.osapublishing.org%2FDirectPDFAccess%2FCE8716FBE509-B308-6D27F3DF065E341 C-225848%2Foe-19-27-26578.pdf%3Fda%3D1%26id%03D225848%26seq%3D0%26mobile%3Dno&org=[retrieved on Apr. 19, 2016].
L. Liu, et al: "Experimental Validation and Performance Evaluation of Open Flow-Based Wavelength Path Control in Transparent Optical Networks", Dec. 19, 2011 (Dec. 19, 2011), Retrieved from the Internet: URL: https://www.osapublishing.org/view—article.cfm?gotourl=https%3A%2F%2Fwww.osapublishing.org%2FDirectPDFAccess%2FCE8716FBE509-B308-6D27F3DF065E341 C—225848%2Foe-19-27-26578.pdf%3Fda%3D1%26id%03D225848%26seq%3D0%26mobile%3Dno&org=[retrieved on Apr. 19, 2016].
Nick McKeown, et. al., "OpenFlow: Enabling Innovation in Campus Networks," [online], [searched on May 31, 2012], the Internet <URL:http://www.openflow.org/documents/openflow-wp-latest.pdf>.
Oki E et al: "Dynamic Multilayer Routing Schemes in GMPLS-based IP +Optical Networks", IEEE Communications Magazine, IEEE Service Center, Piscataway, US, vol. 43, No. 1, Jan. 1, 2005 (Jan 1, 2005), pp. 108-114, XP011201253.
Singaporean Written Opinion and Search Report dated Jan. 11, 2016.

Also Published As

Publication number Publication date
KR20150054927A (ko) 2015-05-20
EP2905929A1 (en) 2015-08-12
EP2905929B1 (en) 2020-02-19
KR101734100B1 (ko) 2017-05-11
JPWO2014054691A1 (ja) 2016-08-25
EP2905929A4 (en) 2016-06-01
JP6492660B2 (ja) 2019-04-03
SG11201502579XA (en) 2015-05-28
WO2014054691A1 (ja) 2014-04-10
US20150263939A1 (en) 2015-09-17

Similar Documents

Publication Publication Date Title
US9906437B2 (en) Communication system, control apparatus, control method and program
US11134011B2 (en) Communication system, control device, communication method, and program
CN111602370B (zh) 使用有限额外字节的带内遥测
EP2637363B1 (en) Communication system, control device, method for controlling packet transfer path, and program
KR101755138B1 (ko) 통신 시스템, 제어 장치, 및 네트워크 토폴로지 관리 방법
EP2652922B1 (en) Communication system, control apparatus, communication method, and program
WO2014054281A1 (en) Control apparatus, control method thereof, and program
WO2012081145A1 (en) Communication path control system, path control device, communication path control method, and path control program
JP6323547B2 (ja) 通信システム、制御装置、通信制御方法、および、プログラム
US20210306267A1 (en) Optimized network latency using in-band telemetry
US9461831B2 (en) Packet forwarding system, control apparatus, packet forwarding method, and program
EP2922250B1 (en) Control apparatus, communication system, control information creating method and program
US20150110107A1 (en) Packet forwarding system, control apparatus, packet forwarding method, and program
EP2991286A1 (en) Communication node, communication system, packet processing method and program
US20150195189A1 (en) Multiple tree routed selective randomized load balancing
US20160050654A1 (en) Control apparatus, communication system, control information transmission method and program
KR101802037B1 (ko) Ｓｄｎ 환경에서 서비스 기능 체이닝을 위한 ｏａｍ 메시지 전송 방법 및 시스템
US20140226486A1 (en) Communication Apparatus, Communication System, Communication Control Method, and Computer Program
US20150372900A1 (en) Communication system, control apparatus, communication control method, and program

Legal Events

Date Code Title Description
AS Assignment

Owner name: NEC CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HASEGAWA, YOHEI;IIZAWA, YOHEI;REEL/FRAME:035325/0419

Effective date: 20150313

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

AS Assignment

Owner name: NEC ASIA PACIFIC PTE LTD., SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NEC CORPORATION;REEL/FRAME:066867/0886

Effective date: 20240313