JP6288633B2 - Network control method - Google Patents

Description

  The present invention relates to a network control method using an SDN controller.

  As one technical element of a software-controllable network (SDN: Software Defined Networking), the function of a network node is a network routing mechanism (node data transfer function) and a controller mechanism (control software / routing control function at a node) In particular, a mechanism that allows flexible path control, configuration change, and the like to be easily realized by opening up an interface so that it can be separated and making network control programmable is drawing attention. In the SDN, the controller collects all network information, defines it centrally, and has a feature that can control the entire network based on centralized control processing utilizing software. The controller can control the operation of the switch by deploying switch control software. As one of the methods for realizing SDN, standardization of a network platform called OpenFlow (Non-Patent Document 1 or the like) has been promoted, and a system / product group by a company has been put on the market simultaneously with implementation by open source.

  However, the processing capacity of the controller is tight against the sudden increase in control traffic between the switch and the controller due to the non-setting of the operation information of the switch and the addition of new functions and services. There may be situations where controller scalability cannot be guaranteed. In addition, regarding the countermeasure when a failure occurs in the controller, the current technology is not disclosed. Regarding controllers that realize SDN, there are Trema developed by NEC and released as open source, Beacon, NOX / POX, etc. developed by Stanford University. The same applies to the case.

  In other words, one of the technical problems in realizing SDN at present is that, for services provided on the network, temporary access concentration to the network that occurs temporarily, and new traffic flows that are not registered as switch operations No mechanism has been established for rapidly increasing the processing capacity of a controller or allocating an alternative system in response to a sudden increase in traffic or a failure of a controller.

  FIG. 1 shows an example of the basic configuration of the SDN. The structure of a general OpenFlow switch is shown. In the OpenFlow switch, the functions of a conventional network device such as a router are represented by an OpenFlow controller (hereinafter referred to as OFC) 92 and an OpenFlow switch (hereinafter referred to as OFS) that transfers packets based on the contents of the flow table. .) 91. The OFC 92 controls the packet processing operation to the switch 11 in the OFS 91 and the statistical processing function. The OFS 91 includes a switch 11 and a flow table 12. The switch 11 performs processing such as routing (action) on the packet based on the flow information of the input traffic according to the instruction of the OFC 92. The network administrator registers actions such as a priority processing method, a discarding method, and a route setting method for the flow of the input traffic as instructions in the flow entry in the flow table 12 in advance. The control contents related to packet processing such as instructions in the flow entry are managed by software from the OFC 92 outside the OFS 91. The OFS 91 applies a matching rule related to packet traffic processing to the registered flow table 12, and performs transfer processing indicated by the matched flow entry for the packet.

  When an undefined traffic flow arrives in its own flow table 12, the OFS 91 may inquire of the operation (action) for the undefined flow to the OFC 92. The OFC 92 instructs the OFS 91 to operate on the undefined flow. When traffic flows that are not registered in the flow table 12 (instruction lists such as instructions) increase in advance with respect to the traffic flows input to the OFS 91, many exchanges of this control message occur between the OFS 91 and the OFC 92. Further, in the OFC 92, the processing load for determining the flow entry to be instructed to the OFS 91 increases. As described above, the amount of control traffic and the amount of traffic processing increase with respect to the open flow control mechanism (packet processing control mechanism) beyond the threshold value of the assumed load amount, making it difficult to control the network managed by the OFC 92. There was a danger.

  That is, in the current network control by the OFC (control server) 92 on the premise of the centralized control method, the processing capacity of the control traffic applied to the controller becomes excessive, and a situation where the communication service cannot be continued may occur.

  Conventionally, as a method for dealing with these traffics, a method of notifying the user to temporarily refrain from using, or forcing traffic for control to the OFC 92 in the preceding stage of the OFS 91 or within the OFS 91, The technique of blocking or discarding was common. It is necessary to improve the reliability related to the continuation of the user's communication service and to cope with an increase in communication traffic in a new form in the future, and the development of a new technology that replaces the conventional method has been demanded.

  On the other hand, even if new services need to be added due to advances in server virtualization technology, multiple physical resources can be allocated as needed to meet user demands, depending on the processing power required. Can be combined and used at the same time. For example, it is possible to appropriately deploy communication devices for network control represented by the OFC 92 to increase the processing capacity necessary to implement a new service.

  Non-Patent Document 2 and Non-Patent Document 3 show examples in which reliability and availability are guaranteed by mounting the OFC 92 on a distributed environment composed of a plurality of physical resources. Non-Patent Document 2 builds data that the OFC 92 should have on a distributed database, and distributes the OFC 92 by providing an appropriate API (Application Programming Interface) for network applications implemented via the OFC 92. Realized. Non-Patent Document 3 discloses a method of managing data on a distributed database and synchronizing by performing communication between a plurality of OFCs 92. Patent Document 1 describes a method for distributing a load, reducing power consumption, and guaranteeing availability by deploying a plurality of OFCs 92 and changing the number of OFSs 91 controlled by the OFC 92 according to the load state. It has been.

JP 2013-30863 A

Open Networking Foundation white paper, “Software-Defined Networking: The New Norm for Networks”, ONF White Paper, April 13, 2012. K. Koponen et al. "Onix: A Distributed control Platform for Large-scale Production Networks", In the Proc. of the 9th USENIX Symposium on Operating System Design and Implementation (OSDI 10), Vancouver, Canada, October 2010 A. Toyotacian and Y.C. Ganjali, “HyperFlow: A Distributed Control Plane for OpenFlow”, In the Proc. Of NSDI Internet Network Management Workshop / Workshop on Research on Enterprise Networking (INM / WREN), San Jose, CA, USA, April 2010

  However, any document discloses a technique for dynamically constructing a geographically distributed controller in real time using a live clone creation technique and utilizing it for network control as a large-scale virtual server. Absent.

  The present invention provides simple control of creating a live OFC replica using live clone technology, adding new services to the network, expanding demand, and suddenly increasing traffic for control processing. Another object of the present invention is to provide a network control method capable of providing a service.

The network control system of the present invention is
An SDN (Software Defined Networking) controller that controls the output path of the switch using software control;
An SDN controller controller that determines that the SDN controller is in an overload state based on control traffic between the switch and the SDN controller and resources of the SDN controller, and generates a live clone of the SDN controller in the overload state When,
Is provided.

  In the network control system of the present invention, the SDN controller control unit determines a condition including a processing speed or a memory capacity of a resource necessary for generating the live clone based on a resource of the SDN controller in an overload state, A virtual machine that satisfies the determined condition may be constructed.

  In the network control system of the present invention, one SDN controller includes a plurality of databases in which data is synchronized, and one of the plurality of databases is stored in the database in accordance with an instruction from the SDN controller control unit. The virtual machine may perform the connection process of the switch with reference to another available one of the plurality of databases.

  In the network control system of the present invention, a plurality of the SDN controllers control a common output path of the switch, and the database provided in the plurality of SDN controllers has the same content flow for controlling the output path of the switch. A table may be stored.

  In the network control system of the present invention, the SDN controller control unit may determine that a failure has occurred in the SDN controller and generate a live clone of the SDN controller in which the failure has occurred.

The network control method of the present invention is:
A control method of an SDN controller that controls an output path of a switch using software control,
A determination procedure for determining that the SDN controller is overloaded based on control traffic between the switch and the SDN controller and resources of the SDN controller;
A control procedure for creating a live clone of an overloaded SDN controller;
Is executed by the SDN controller control unit .

In the network control method of the present invention, in the control procedure, the SDN controller control unit sets a condition including a processing speed or a memory capacity of a resource necessary for generating the live clone as a resource of the overloaded SDN controller. A virtual machine that is determined based on the condition and satisfies the determined condition may be constructed.

  In the network control method of the present invention, one SDN controller includes a plurality of databases in which data is synchronized, and in the control procedure, one of the plurality of databases is an instruction from the SDN controller control unit. The virtual machine has a function of replicating data stored in the database, and the virtual machine performs connection processing of the switch with reference to another available one of the plurality of databases. Also good.

  In the network control method of the present invention, a plurality of the SDN controllers control a common output path of the switch, and the database provided in the plurality of SDN controllers has the same content flow for controlling the output path of the switch. A table may be stored.

In the network control method of the present invention, in the determination procedure, the SDN controller control unit determines that a failure has occurred in the SDN controller , and in the control procedure, the SDN controller control unit causes the SDN in which the failure has occurred. A live clone of the controller may be generated.

  The above inventions can be combined as much as possible.

  According to the present invention, a simple control of creating a live OFC replica by live clone technology can be used to add new services to the network, increase demand, and suddenly increase traffic for control processing. It is also possible to provide a network control method that can provide a service.

An example of the basic configuration of SDN using OFS is shown. An example of live cloning of an OpenFlow controller is shown. Embodiment 1 of a live clone is shown. Fig. 3 shows a second embodiment of a live clone. Fig. 3 shows a second embodiment of a live clone. An example of connection operation when creating a live clone is shown. An example of how to perform live cloning is shown. The structural example which combined OFS is shown. An example of a usage form of a plurality of OFCs is shown. An example of usage forms of a plurality of OFCs (using a proxy OFC) is shown. An example of connection from a cloud server group to an arbitrary OFC is shown. An example of an algorithm for optimizing allocation using one variable as a resource of a resource will be described. An example of an algorithm for optimizing resource allocation using multiple variables as resource resources will be described. An example (continued) of an algorithm for optimizing resource allocation using multiple variables as resource resources will be described. An example of a method for expanding an OpenFlow controller distributed in a region is shown.

  Embodiments of the present invention will be described with reference to the accompanying drawings. The embodiments described below are examples of the present invention, and the present invention is not limited to the following embodiments. In the present specification and drawings, the same reference numerals denote the same components.

  The present invention improves the efficiency of a virtual resource environment based on computing such as cloud computing and network resources when an unexpected increase in the amount of control traffic occurs temporarily or continuously. Technical means for dynamically securing large-scale processing capacity by utilizing a virtualized cloud by utilizing a virtualized network control device or open flow control mechanism (packet processing control mechanism) Is. The present invention can be applied to any SDN controller. In the following embodiment, OFC will be described as an example of an SDN controller.

  Considering the network control in the SDN shown in FIG. 1, a method of deploying the network application 93 above the control layer 82 where the OFC 92 is located is general. In this case, the network application 93 exists above the OFC 92 and updates the normal network management and monitoring functions such as when introducing new network control application software or changing the route calculation algorithm for network control. Use it when you want to. Standardization regarding SDN, including API, has not yet progressed sufficiently, and the current situation is that application 93 is also implemented depending on the implementation of each OFC 92. In the description of the present invention, when an existing OFC 92 and an application are implemented on one physical or virtual machine (denoted as VM in the figure), by utilizing a virtual environment such as a cloud, Disclose a network control mechanism that dynamically and simply realizes high availability, high scalability, and effective use of resources. When an API between the OFC 92 and the application 93 is determined in the future, it is also effective as an embodiment to apply the present invention to the processing capacity of the OFC 92 that is assumed to have the greatest variation in processing capacity.

  FIG. 2 shows an outline of a mechanism for creating a live clone for the OFC 92 as an embodiment of the present invention. A virtual OFC # 1 is assigned to the OFS 91 (OFS # 1 to OFS #n) as an OFC 92 that controls each OFS 91. The virtual OFC # 1 can be operated on one virtual machine 71 # 1, but can also be operated on a plurality of virtual machines 71 # 1 and 71 # 2 depending on the load state. .

  An OFC resource controller 95 is separately provided for a network composed of OFS 91 and OFC 92. The OFC 92 functions as an SDN controller, and the OFC resource controller 95 functions as an SDN controller control unit. The OFC resource controller 95 is a mechanism for managing and controlling in what virtual / physical configuration the OFC 92 is realized. The OFC resource controller 95 has an open flow control traffic monitoring function 951, an OFC resource load monitoring function 952, a live clone control function 953, and a load balance control function 954.

  The control method of the SDN controller according to the present embodiment is a method of controlling the association between the controller and the switch using software control, and has a determination procedure and a control procedure in order. In the determination procedure, the open flow control traffic monitoring function 951 and the OFC resource load monitoring function 952 are made to function. In the control procedure, the live clone control function 953 is caused to function. After the live clone control function 953 is functioned, the load balance control function 954 is functioned.

The open flow control traffic monitoring function 951 monitors the amount of control traffic transferred to the OFC 92 among the traffic flowing into the OFS 91.
The OFC resource load monitoring function 952 monitors the resources of the OFC 92 (CPU load, memory capacity, I / O transfer amount, etc.).

  The live clone control function 953 detects that the OFC 92 is overloaded. For example, the live clone control function 953 detects an overload when the amount of control traffic observed by the open flow control traffic monitoring function 951 exceeds a preset threshold. Further, the live clone control function 953 detects an overload when the resource of the OFC 92 observed by the OFC resource load monitoring function 952 exceeds a preset threshold value.

  When the live clone control function 953 detects an overload state, it determines that a live clone of the virtual machine 71 (OFC) is necessary. Also, it is determined that a live clone of the virtual machine 71 (OFC) is necessary even when there is a possibility of an overload in the future although it is not currently overloaded. The live clone control function 953 manages cloud resources, determines which resource the live clone of the virtual machine 71 is generated in, and creates a live clone. At this time, the live clone control function 953 notifies the VM 71 of a live clone creation instruction.

  When the generation of the live clone is completed, the load balance control function 954 instructs the corresponding OFS 91 and load balancer in addition to the virtual machine 71 in order to instruct the distribution of control traffic in order to perform load distribution. Here, a case where a live clone is created in order to increase the processing capacity has been shown. However, if OFC # 1 has some sort of situation (failure occurrence, implementation of another application, etc.), live migration is performed. It is also possible to perform the control operation so as to utilize other cloud resources.

  Functions that are preferably configured on a medium different from the OFS 91 and OFC 92 are collected and deployed on the OFC resource controller 95. The open flow control traffic monitoring function 951 includes a method of deploying in the OFS 91, a method of deploying in the OFC 92, and a method of deploying on the OFC resource controller 95. The OFC resource load monitoring function 952 includes a method of deploying in the OFC 92 and a method of deploying on the OFC resource controller 95. The live clone control function 953 and the load balance control function 954 are desirably provided on the OFC resource controller 95. The deployment of each function is not limited to this physical logical configuration. For example, if only the load balance control function 954 is mounted on another physical configuration, the OFC resource controller 95 is realized on the cloud and distributed. Embodiments in which control is performed are also conceivable.

  The open flow control traffic monitoring function 951 detects a high load when the traffic volume between the OFS 91 and the OFC 92 is equal to or greater than a predetermined threshold value, and notifies the live clone control function 953. Further, when the live clone control function 953 or the load balance control function 954 desires information related to the amount of open flow control traffic, the information is provided.

  When the OpenFlow control traffic monitoring function 951 is provided in the OFS 91, the controller 95 uses a mechanism for monitoring the amount of control traffic exchanged with the OFC by a mechanism completely different from OpenFlow, or using the OpenFlow mechanism. There is a method in which a traffic flow is defined in the flow table 12 and monitored using the count function of the flow table 12.

  When the OpenFlow control traffic monitoring function 951 is provided in the OFC 92, the OFC 92 can be controlled by using a mechanism for monitoring the amount of control traffic exchanged with the OFS 91 using an entirely different mechanism from that of the Open Flow or an Open Flow mechanism. There is a method in which the traffic flow is defined in the flow table 12 and the counter value in the flow table 12 is acquired via the OFC 92 for monitoring. This traffic monitoring function can be deployed on the OFC resource controller 95, and even when deployed in a monitoring center other than the OFS 91 and OFC 92, network control that realizes an equivalent function is possible. Nor.

  The OFC resource load monitoring function 952 monitors the resource of the virtual machine 71 of the OFC 92, detects that the resource usage status and load status are equal to or higher than a predetermined threshold, and detects that the load is high. Notice. Further, when the live clone control function 953 and the load balance control function 954 desire the resource information of the virtual machine 71 of each OFC 92, the information is provided.

Live clone control function 953
(1) Which virtual machine 71 is selected to be replicated or degraded
(2) The procedure for performing live clones
Determine and execute. When an overload is reported by the open flow control traffic monitoring function 951 or when an overload is reported by the OFC resource load monitoring function 952, it is determined that a live clone of the virtual machine 71 configuring the OFC 92 is necessary. In addition to this, when the open flow control traffic information is periodically acquired from the open flow control traffic monitoring function 951 and the resource information is acquired from the OFC resource load monitoring function 952, the virtual It is determined that a live clone of the OFC 92 of the machine 71 is necessary.

  On the other hand, if the load on the OFS 91 steadily decreases and the monitoring result of the traffic volume shows that the resources in the cloud allocated to the OFC 91 currently deployed are clearly surplus, It is necessary to perform control (reduction) for reducing the resources for the OFC 92 described above. When deciding which number of OFCs 92 is to be reduced, it is generally safe to select in order of decreasing processing capacity reduction effect due to the effect of the reduction. However, there are cases where the priority order for the removal is different depending on the processing status of other applications in the cloud, and it is clear that the priority is not limited to the above method.

When the live clone control function 953 determines a live clone generation method and creates a live clone, the following two methods are conceivable.
First method: Information of a database storing all information of the flow table 12 is also copied together and used as a completely equivalent server.
Second method: Information necessary for flow control is copied, but not all information is copied, and only the information of the new flow table 12 is added as a server.

  The first method is a case where it is used as a live clone capable of realizing redundant control for improving reliability, and is usually used in connection with a load balancer (load balancer). On the other hand, the second method is suitable for processing performed in an emergency by predicting in advance a case where traffic is likely to increase urgently. For this reason, it is preferable to use the operation method in which the mode of the second method is shifted to the mode of the first method when there is a surplus in the server processing capability for the OFC 92, such as surplus capacity.

  In general, a replication technique for creating a live clone is used as a technique for replicating information while maintaining the consistency of the contents of a database, and is used when storing the same data in a plurality of databases. When an arbitrary OFC 92 accesses a database (DB) 72 in another OFC 92, only one OFC 92 operates as a master at a certain point in time and uses a synchronization processing mechanism for waiting for requests from other OFCs 92. Can be realized.

  For example, access to a file in a specific database between different OFCs 92 is temporarily limited to one process, and a mechanism that prevents deadlock due to access contention (replication using file lock) is used. You can also Database contents can also be replicated using a dedicated management system. When configuring a master-slave system, the master records updates and notifies the slave group of the contents. Needless to say, it is not necessary to be limited to this method. In this case, it is necessary for the slave side to send a message indicating that the update has been performed correctly and to notify that it is ready to accept a subsequent new update request.

  FIG. 3 shows a schematic diagram of a live clone. A simple live clone from OFC # 1-1 to OFC # 1-2 only needs to replicate the image file and the memory contents of the virtual machine 71 in operation, but the image file is generally copied for a long time. Therefore, a new device is necessary. Therefore, as shown in FIG. 3, the image file is roughly divided into two. One is a master image file 73 to be transferred to the clone destination virtual machine 71 # 2 with different contents at the time of live clone (creation), and the other is the clone destination virtual machine 71 with exactly the same contents at the time of live clone (creation). This is the image file 72 taken over by # 2. When the OFC 92 is targeted, the master image 73 to be inherited with different contents corresponds to a system main body such as an OS, and the image to be inherited with the same contents corresponds to a database file (DB) 72 that stores all information of the flow table 12. Further, the image file including the OS to be inherited with different contents is divided into two stages, and the standard and common OS system of all the virtual machines 71 is set as the master image 73, and different setting information and log information for each virtual machine 71. Are installed as a difference file 74 at a location closer to the virtual machine 71 than the master image 73.

  FIG. 3 shows a first replication example of the virtual machine 71 using a live clone. Although the virtual machine 71 # 2 is live cloned as a replica of the virtual machine 71 # 1, the standard and common parts such as the OS are shared as the master image 73, and the setting information unique to the virtual machine 71 # 1 and the virtual machine The changed part such as the execution log after 71 # 1 is started is stored in the difference file 74 # 1 on the virtual machine 71 # 1 side, and the setting information unique to the virtual machine 71 # 2 and the virtual machine 71 # 2 are stored. The changed part such as the execution log after the start is arranged so as to be stored in the difference file 74 # 2 on the virtual machine 71 # 2 side. Standard and common parts that do not exist in the difference file 74 go to the master image 73 for both the virtual machine 71 # 1 and the virtual machine 71 # 2. Further, the part of the DB 72 that is completely the same from the virtual machine 71 # 1 to the virtual machine 71 # 2 is assumed to access the same image. As a result, it is possible to start a new live clone only by creating the difference file 74 for the virtual machine 71 to be cloned, without copying the image at the time of the live clone.

  FIG. 4 shows a second replication example of a virtual machine by live clone. In the second replication example of the virtual machine, in addition to the configuration of FIG. 3, by performing standard replication as a database reliability technology, the DB 72 is converted into the DB 72 # 1 and the DB 72 # 2 in the standard state. Duplicated. After cloning, replication of the DB 72 is stopped, the duplicated DB 72 is disconnected, and each DB 72 is assigned to a different virtual machine, so that access to the DB 72 does not cause a decrease in performance. Furthermore, after live clone, replication for high reliability of DB72 should be started again, and another duplex should be started in each of DB72 # 1 and DB72 # 2. An example of this implementation is shown in Embodiment 2 of FIG.

  FIG. 5 shows a third replication example of the virtual machine by the live clone. In the case of the third replication example of the virtual machine, DB 72 # 3 is replicated as DB 72 # 1 accessed by the virtual machine 71 # 1, and DB 72 # is replicated as DB 72 # 2 accessed by the virtual machine 71 # 2. 4 is created and duplication is implemented. In most of the processes of the virtual machines 71 constituting the OFC 92, the flow table 12 matching the control traffic is searched from the DB 72 storing all the information of the flow table 12, and the search result is stored in the OFS 91 as a new flow table 12. It is a process to reply as. Therefore, by generating a new virtual machine 71 # 2 by live clone and expanding CPU resources and memory resources, the above processing can be distributed to a larger number of virtual machines 71. As the load applied to the machine 71 decreases, it can be expected that response processing will be speeded up.

  However, as shown in FIG. 7A, when the DB storing all the information of the flow table 12 is shared between the virtual machines 71 that are live-clone, even if there is room in CPU resources and memory resources, the DB Access to the other file becomes a bottleneck, and the expected response speed cannot be increased. In such a case, the DB 72 replication technology such as replication is linked to the live clone technology, and the DB 72 replication shown in FIG. 7A is realized, and the live clone as shown in FIG. By performing the above, it is possible to expand resources related to access to the DB file and to realize a high-speed response.

  In both the live clone embodiment 1 of FIG. 3 and the live clone embodiment 2 of FIGS. 4 and 5, the cloned virtual machine 71 includes the same DB file with the same function as the clone-source virtual machine 71. , Function as a different controller. Therefore, the session (TCP session, SSL session, OpenFlow control session) being used in the clone source virtual machine 71 cannot be taken over by the cloned virtual machine 71. A TCP session or an SSL session can be easily recovered by reconnecting the session to the virtual machine 71 in which the OFS 91 has been cloned, but care should be taken in handling an open flow control session (PacketIn → FlowMod → PacketOut). For example, when the clone virtual machine 71 takes over the control of OFS 91 #n to the cloned virtual machine 71, for the session in which FlowMod has been sent to OFS 91 #n, the subsequent PacketOut is sent from the clone source. Sessions that only receive PacketIn from OFS91 # n are not processed by the clone source, and when the remaining PacketOut is sent, the TCP connection with OFS91 # n is disconnected, so that PacketIn is sent to OFS91 # n. A process for prompting re-transmission to the cloned virtual machine 71 is required. Similar considerations are required when the virtual machine 71 is degenerated.

  In general, the live clone shown in FIG. 3 and the live clone shown in FIG. 4 are mixed while balancing both the demand for expansion of CPU resources and memory resources and the demand for the ability to access the DB file. By performing the clone sequentially, it is possible to achieve an appropriate increase in processing capacity in a virtual machine environment.

  FIG. 6 shows an example of a connection operation for enabling the OFC 92 to operate without reducing the current processing capacity when creating a live clone on the cloud by duplicating the OFC 92. When a live clone creation instruction is issued from the OFC resource controller 95 to the OFC 92 of the site 1 (S301), the OFC 92 # 1 of the first site performs connection processing with the OFS 91 on one virtual machine 71 # 1. (S302 and S303), the other virtual machine 71 # 3 creates a copy to the OFC 92 # 2 of the second site (S304). That is, the data transfer in S303 and S304 is processed in parallel, so that the OFC 92 # 1-1 of the first site can suppress deterioration in processing capability when creating a live clone in the second site. Needless to say, this mechanism can also be applied to the case where OFC92 # 1-1 and OFC92 # 1-2 are at the same site.

  When live clones are continuously created due to a sudden increase in control traffic, sharing and duplication / replication of the DB 72 storing all the information in the flow table 12 is an important issue in determining the overall performance. . For example, as shown in FIG. 6, when a live clone of the virtual machine 71 # 1 is created in the virtual machine 71 # 2 having sufficient CPU resources and memory resources, it is currently used with the virtual machine 71 # 2. When the network delay between the DBs 72 is large, or when the available transmission bandwidth is narrow, when the DB 72 is shared or duplicated, more processing time is required. Therefore, the virtual machine 71 # There is a possibility that the response processing cannot be speeded up even by performing live cloning of 2.

  Since the master image 73 is common to all the virtual machines 71, such a problem can be avoided by deploying the same image in advance in a plurality of locations in the OpenFlow cloud as shown in FIG. However, since pre-deployment of the DB 72 is difficult, a virtual machine 71 at a location connected by a low-latency and wide-band network is preferentially selected from the currently used DB 72, and a plurality of live clones are performed. It is preferable to create a replication / replication of the above.

  In FIG. 7, the live clone control function 953 judges the increase in the OFC processing capacity from the information of the open flow control traffic monitoring function 951 and the OFC resource load monitoring function 952, and sequentially creates a plurality of live live clones one after another. Two basic example methods are shown for implementation. In the first method shown in FIG. 7A, the OFC 92 # 1 is sequentially fixed from the first OFC 92 # 1 installed as a master to other resources on the cloud where the controller function can be implemented. Then, this is a method for increasing the number of live clones. This method is applicable when the processing capacity of the master OFC 92 has a considerable surplus processing capacity and is highly reliable, or when the transmission distance between the slave OFCs 92 is short, and is limited to the data center. Is possible. If it is clear that the transmission band between the master OFC 92 and the subordinate OFCs 92 under its control is tight, this method should be used in a section where the transmission band is sufficiently secured. It is preferable to use it.

  Transmission bandwidth for subordinate OFC 92 in remote locations, lack of processing capability of master OFC 92, or direct connection from master OFC 92 to several subordinate OFCs 92 as described above If this is insufficient, it is preferable to use the second method shown in FIG. In the second method, the OFC 92 # 1 to be copied is not fixed to the master controller, and the function of the master is moved every time after copying. This method has a feature that the master OFC 92 can be easily changed even when the master OFC 92 fails, and can be applied to improve the fault tolerance of the entire system.

  The above-described technology allocates resources of the virtual machine 71 in response to a request from a customer portal site in the currently serviced cloud service, and can increase the number of new virtual machines 71 step by step. This can be achieved by incorporating a mechanism. With this mechanism, even when the processing capacity of the controller in the network is tight, stable service continuity (BCP: Business Continuity Plan) can be realized.

  The load balance control function 954 performs a combination of load distribution between OFCs 92 replicated by the live clone control function 953 and load distribution among a plurality of OFCs 92.

  A control method for appropriately distributing the OpenFlow control message to the OFC 92 replicated by the live clone will be described. The distribution of the OpenFlow control message is realized by independently deploying the load balancer in the OFS 91, in the OFC resource controller 95 that manages a plurality of OFCs 92 in the cloud, or in the cloud. Since session information is also inherited by the live clone, load distribution can be realized by performing packet transfer to an appropriate virtual machine 71. Needless to say, when performing address reconfiguration when performing live clone, it is necessary to have a mechanism for maintaining the session between the OFC 92 and OFS 91 of the clone source.

FIG. 8 shows a configuration example of the OFS 91.
The OFS 91 illustrated in FIG. 8A realizes the function of the switch 11 by physically connecting an L3 (layer 3 processing) switch and a load balancer. The OFS 91 shown in FIG. 8B realizes the function of the switch 11 by physically connecting a plurality of OpenFlow switches. The OFS 91 illustrated in FIG. 8C realizes the function of the switch 11 by physically connecting a virtual L3 switch and a virtual load distribution apparatus. The apparatus configuration shown in FIG. 8 (a) is a physical configuration using only the OFS 91 shown in FIG. 8 (b) by providing a plurality of types of control software for OFS 91 in principle, and is equivalent to the logical configuration shown in FIG. 8 (c). It is also possible to configure the apparatus. The functions of the OFS 91 are not limited to the examples shown here, but examples of the types of OFS 91 action control commands and types of header information that can be defined are necessary for realizing these various functions. An example is shown below.

Examples of commands that can be specified as an action for a flow include the following.
Forward: Forwards the packet to a specific physical port.
Drop: Discards the packet.
Modify-Field: Rewrites the value of the packet header to a specified value.
Store: The packet is temporarily stored in the common buffer.
copy: Copy a packet and multicast it to multiple ports.
divide: The packet is divided and multicast to a plurality of ports.
encryption: The content of the packet is encrypted.
Enqueue: The packet is stored in the designated queue buffer.

  Examples of header information that can be defined by OpenFlow include physical port number, source MAC address, destination MAC address, Ether type, VLAN ID, VLAN priority, source IP address, destination IP address, and IP protocol type. , IPTOS information, source L4 port number, destination L4 port number, MPLS label number, and closed service identification number.

  A load balancing method among the plurality of OFCs 92 by the load balance control function 954 will be described below. As shown in FIG. 9, the OFS 91 holds a plurality of pieces of OFC 92 information that can be used as setting information for itself, and can simultaneously establish a connection for open flow control with a plurality of OFCs 92. For example, all of OFC92 # 1 to # 3 are used as OFC92 with the same control function, using one OFC92 # 2 as a master and using other OFC92 # 1 and # 3 as slaves. It is also possible to use the system so that the control traffic can be transferred to the OFC 92.

  Also, as shown in FIG. 10, the OFS 91 is connected to, for example, the OFC 92 # 1 that operates as a proxy of the OFC 92, and the OFC 92 # 1 that operates as a proxy distributes the processing to each of the OFCs 92 # 2 to # 4. Is possible as well.

  FIG. 11 shows a connection configuration in a case where the target cloud server group includes OFC 92 # 1 to OFC 92 # n having different n processing capabilities. The configuration in FIG. 11 is a generalization of the configuration in FIGS. 9 and 10. For example, the open flow control traffic monitoring function 951 that performs control traffic load measurement for delivery to the OFC 92 installed in the OFS 91 observes an overload state, and the load balance control function 954 transfers to the new OFC 92. When it is determined that the connection is necessary, the unload distribution control is performed and the new OFC 92 is connected. In this case, as an example of a method of selecting the simplest new connection destination OFC 92, one resource type is required for each OFC 92 #, and the relative value of the processing capability in this resource is different. Assuming that the processing capacity is, for example, the order of processing capacity of OFC92 # 1> processing capacity of OFC92 # 2> ...> processing capacity of OFC92 (# n-1)> processing capacity of OFC92 # n In this case, it is preferable to assign the OFC 92 of the number in descending order of processing capability.

  On the other hand, it is also effective to send an inquiry message for obtaining a response response time within a range that does not affect the processing capability of each OFC 92 periodically from the OFS 91. In this case, it is possible to determine that the OFC 92 with the shortest response response time should be selected with the highest priority. Alternatively, there is a case where a method of performing evaluation by appropriately weighting the above two determination methods may be more preferable, and an example thereof will be described below. For example, in the simplest case, there are a resource type 1 (CPU processing capacity) and a resource type 2 (excess memory capacity) as request conditions when determining the allocation of a new OFC 92. Suppose you need to be satisfied at the same time.

  Here, in order to simplify the discussion, all resource types 1 that can be the target of live clones are assumed to have the same standard (clock frequency, number of instruction execution processes per unit time, etc.), Gibson mix or commercial mix. As for the processing capability corresponding to, it is assumed that the same capability is provided when the resource type is not used. The same standard is assumed for resource type 2. In this case, for example, the resource type 1 requires 30 (relative value) capability and the resource type 2 has 40 (relative value), whereas the control traffic to be handled by the OFC 92 has increased. ) Is required.

  Actually, for example, resource type 1 needs to take into account, for example, 100 MIPS (Million Instructions Per Second) × assignable CPU usage rate, but here, those conditions that should be assigned, etc. As a value converted into a simple “relative value”, the “relative value” considering the above conditions is already used.

  Here, there are various types of assignable resource types that can be used, and there are # 5, # 6, # 7, # 8, and # 9 as usable OFC 92. For example, # 5 is (resource 1:20, resource 2: 50), # 6 (resource 1:40, resource 2:20), # 7 (resource 1:28, resource 2:35), # 8 (resource 1:30, resource 2:45), # 9 (resource 1:30, resource 2:50). In this case, # 8 and # 9 match the conditions, but since the OFC 92 # closest to the requirement is OFC 92 # 8, it is only necessary to select # 8 and create a live clone.

  As a more complicated example, assume that resource 1 requires 60 (relative value) capability and resource 2 requires 70 (relative value) capability. In this case, since a satisfactory OFC 92 # cannot be found by itself, it is necessary to combine two or more OFC 92 #. In this case, it is practically preferable that the number of combinations is as small as possible. In the case of the above conditions, it is most preferable to combine OFC92 # 5 and OFC92 # 6.

  In the above example, only the resource 1 and the resource 2 are considered. However, as the new resource 3, for example, it is considered that the short response delay time is regarded as the response capability and 50 is added as a relative value. Here, assuming that the resource 3 has the highest priority and the weighting coefficient is “2”, the necessary resource 3 can be set to 50 × 2 = 100. Therefore, in searching for an OFC 92 # that satisfies the required conditions, an OFC 92 # that satisfies the following conditions may be selected.

The OFC 92 of # 5, # 6, # 7, # 8, and # 9 considers the weighting coefficient related to the element of resource 3, for example, # 5 is (resource 1:20, resource 2:50, resource 3:70). , # 6 (resource 1:40, resource 2:20, resource 3:40), # 7 (resource 1:28, resource 2:35, resource 3: 100), # 8 (resource 1:30, resource 2) : 45, resource 3:60), # 9 (resource 1:30, resource 2:50, resource 3:65), and the preconditions are as follows.
(1) Resource 1 is 60 or more and closest to 60 (2) Resource 2 is 70 or more and is closest to 70 (3) Resource 3 is 100 or more and is closest to 100

  The single OFC 92 # cannot satisfy all the conditions related to the elements of each resource, and the method of using # 5 and # 6 in combination is most preferable.

  In the examples described above, it is assumed that each resource value is closest to the required value for a combination that satisfies the condition (including a single case) on the assumption that the value of each resource can be added and evaluated. It showed how to choose a combination. However, the present invention is not limited to this example, and it goes without saying that the most appropriate combination can be selected by the same method even when, for example, the sum of the squares of the required values of the elements of each resource is evaluated. If there is no satisfactory combination, it is practically preferable to cope with the problem by selecting the combination closest to the required condition.

  On the other hand, when the number of target OFCs 92 or the number of resource types becomes very large, it is difficult to select an appropriate OFC 92 easily heuristically. In this case, a solution means can be provided by utilizing a method similar to the method of solving the knapsack problem. First, as a first step, FIG. 12 shows an algorithm for optimizing the allocation resource of the resource type when one variable is targeted.

  Here, since the number of target OFCs 92 is very large and it is not realistic to target all OFCs 92 in the world, first, it is assumed that there are n OFCs capable of determining the resource type for the time being. Then, the contents of the control flow of FIG. 12 will be described based on the premise that the reading (sampling) of the resource amount for them is performed from a specific or appropriate OFC 92.

Step S101: A resource vector D related to the first type resource of a plurality of applicable OFCs 92 is defined as follows, where the target resource type is “D”.
D = (vector consisting of n elements) = (D0, D1,... D (n−1))
Step S102: The resource state D0 (= D [0], D [1], D [1],... D [n-1]) is recorded. Here, a is the target value of the total resource, and m is the upper limit value of the number of repetitions. ct is a repeat counter, and s0 is a provisional value of the total resources. The worst value that can be considered as an initial value is entered.
Step S103: Add until the total resource s exceeds the target value a.
s = s + D [i], i = i + 1
Step S104: When s can realize a value better than s0, the record is updated with D0 = D, i0 = i, and s0 = s.
Step S105: When ct <m, D is rearranged and the processing from step S103 is repeated.

Here, as a method of rearranging D, n is the length of the vector D,
(Equation 1)
r1 = random (n)
r2 = random (n)
x = D [r1]
D [r1] = D [r2]
D [r2] = x
As described above, there are a method of rearranging two by two, and a method of rearranging the entire D using Fisher-Yates shuffle, but it goes without saying that the method is not limited to these specific methods. What is necessary is just to select a thing with a fast convergence speed in practice.

  Step S106: If ct ≧ m, the recorded D0, i0, s0 are output and the process ends.

The above-described method can be easily extended to an algorithm for obtaining an optimal combination when there are multivariable resource types. 13 and 14 show an outline flow.
Step S201: As a resource type, data D composed of k variables is sampled from a plurality of specified or arbitrary OFCs 92. In this case, D is an n-by-k matrix.
Step S202: The resource state D0 is recorded. a = (a0, a1,..., a (k−1)) is the target value of the total resources, and m is the upper limit of the number of repetitions. ct is a repeat counter, and s00 is a provisional value of the total resource difference. The worst value that can be considered as an initial value is entered.

Steps S203 to S208: Add until the j-th resource total sj exceeds the target value aj. Here, sj is expressed by the following equation.
(Equation 1)
sj = sj + D [i, j]
s = | s0−a0 | + …… + | sj−aj | + …… + | s (k−1) −a (k−1) |
Obviously, a small s indicates that the recorded D0 is close to the optimal solution. In particular, when s00 = 0, it can be assured that an optimal solution has been obtained.

Step S204: When s can realize a value better than s00, the record is updated as D0 = D, i0 = i, and s00 = s.
Step S205: When ct <m, D is rearranged and the processing from step S203 is repeated.
Here, as a method of rearranging D, n is the number of rows of D, the r-th row vector is D [r,],
(Equation 2)
r1 = random (n)
r2 = random (n)
x = D [r1,]
D [r1,] = D [r2,]
D [r2,] = x
As described above, there are a method of rearranging two rows at a time, and a method of rearranging the entire D using Fisher-Yates shuffle, but it is needless to say that the present invention is not limited to this.
As long as the same result does not appear as a result of the rearrangement, any rearrangement algorithm may be used.

  Step S206: If ct> = m, the recorded D0, i0, s00 are output and the process is terminated.

  On the other hand, for the newly generated traffic flow, by specifying the allocation rule for the proxy OFC 92 by the method shown in FIG. 10, the number of the OFC 92 to be used is specified for each event type. It goes without saying that the OFC 92 can be deployed by function using a function distribution method for sharing the processing with a specific OFC 92.

  Further, in the master / slave system multiple OFC usage mode shown in FIG. 9, after the new OFC 92 is assigned, it is preferable to promptly notify the OFS 91 that the new controller is the master. .

  In the above, the load distribution control method between OFCs 92 replicated by the live clone control function 953 and the load distribution control method between a plurality of OFCs 92 are described separately, but control methods used in appropriate combinations can be appropriately mixed. Needless to say.

  When the processing capacity of the OFC 92 fluctuates dynamically, the OFC resource controller notifies the live clone control function 953 and the load balance control function 954 of the resource processing capacity at that time from each OFC 92 at a constant cycle. 95 can grasp the latest situation. Based on this information, the OFC resource controller 95 can determine the OFC 92 # to be allocated next and the OFC 92 # to perform live clone next.

  FIG. 15 shows that when the OFS 91 exists corresponding to the regional server groups # 1, # 2, and # 3, undefined flow traffic flows from each server group into the corresponding OFS 91 #, and enters the corresponding OFC 92 #. This shows a basic network configuration when control traffic is generated. In FIG. 15, virtual machines 71 # 2 to 71 # r are generated in the OFC 92 # 1, virtual machines 71 # 2 to 71 # s are generated in the OFC 92 # 2, and the virtual machine 71 # 2 is stored in the OFC 92 # 3. The case where -71 # t is produced | generated is shown. When it is detected that the amount of control traffic exceeds a predetermined threshold value, each OFC 92 # is connected to an open cloud site assigned in advance, and the open cloud site has a necessary processing capacity. In response, clone creation or live migration is performed sequentially. If the amount of the resource is sufficiently secured, the OFC 92 can be appropriately expanded within the range according to the request.

  When there are regional server groups # 1, # 2, and # 3, when undefined flow traffic flows from each regional server group into the OFS 91 and control traffic is generated toward the corresponding OFC 92 #, the OFC 92 When it is detected that the control traffic to be processed has exceeded a predetermined threshold value, it may be connected to an open cloud site (OFC 92 # 1) that can be commonly used. In this case, the OFC 92 # 1 is a basic configuration for sequentially performing live cloning and achieving a desired processing capacity in order to be able to cope with a necessary processing capacity according to a connection request.

As described above, the feature of the present invention is the network control method based on the centralized control.
(1) Effective use of virtual server resources in the cloud,
(2) Dynamic control of the processing capacity of the controller that centrally controls the network,
(3) According to the situation such as rapid increase in traffic or network equipment failure, etc. (4) In the real-time, within the same cloud and between different clouds, the disclosed technology is
It has the feature that a highly flexible controller (network control system) can be constructed by making maximum use of the resources held by the existing cloud so that the conditions related to the required resources can be satisfied.

  In other words, the disclosed technology makes it unnecessary to provide equipment corresponding to the maximum load from the beginning when using the cloud and constructing a network control system, and has the flexibility to increase or decrease the processing capacity as necessary. It is. Furthermore, the disclosed technology can divert an existing system by performing control using a live clone, and can achieve the above-described features with simple control (live clone duplication and deletion). With regard to network control resources that are no longer needed, by incorporating procedures such as sequentially releasing live clones, it is possible to effectively use resources in the running cloud, which matches the latest traffic processing status. A network configuration having a control system processing capability can be realized.

  In this way, it is possible to sufficiently exhibit the merit that network software can be virtualized on a large scale in the cloud. In addition, the disclosed technology can be applied not only to an open flow type network represented by SDN, but also to control of NGN and existing mobile phone networks and increase the processing capacity of an operation system. Extremely wide. As described above, this technology can be used as a basic technology for realizing a secure network, and has great significance in developing the future information and communications industry.

  The present invention can be applied to the information communication industry.

11: Flow table 12: Switch 71: Virtual machine 72: Database (DB)
73: Master image 74: Difference file 81: Infrastructure layer 82: Control layer 83: Application layer 91: Open flow switch (OFS)
92: Open Flow Controller (OFC)
93: Network application 94: Host 95: OFC resource controller 951: Open flow control traffic monitoring function 952: OFC resource load monitoring function 953: Live clone control function 954: Load balance control function

Claims (10)

An SDN (Software Defined Networking) controller that controls the output path of the switch using software control;
An SDN controller controller that determines that the SDN controller is in an overload state based on control traffic between the switch and the SDN controller and resources of the SDN controller, and generates a live clone of the SDN controller in the overload state When,
A network control system comprising: The SDN controller control unit
A condition including a processing speed or a memory capacity of a resource necessary for generating the live clone is determined based on an overloaded SDN controller resource, and a virtual machine that satisfies the determined condition is constructed.
The network control system according to claim 1. One SDN controller comprises a plurality of databases with synchronized data,
One of the plurality of databases has a function of replicating data stored in the database in accordance with an instruction from the SDN controller control unit,
The virtual machine refers to another available one of the plurality of databases and performs connection processing of the switch.
The network control system according to claim 2. A plurality of SDN controllers controlling a common output path of the switch;
The database provided in the plurality of SDN controllers stores a flow table having the same content for controlling the output path of the switch.
The network control system according to claim 3. The SDN controller control unit determines that a failure has occurred in the SDN controller, and generates a live clone of the SDN controller in which the failure has occurred.
The network control system according to claim 1. A control method of an SDN controller that controls an output path of a switch using software control,
A determination procedure for determining that the SDN controller is overloaded based on control traffic between the switch and the SDN controller and resources of the SDN controller;
A control procedure for creating a live clone of an overloaded SDN controller;
A network control method in which the SDN controller control unit executes . In the control procedure, the SDN controller control unit determines a condition including the processing speed or memory capacity of the resource necessary for generating the live clone based on the resource of the SDN controller in an overload state, and the determined condition Build a virtual machine that satisfies
The network control method according to claim 6. One SDN controller comprises a plurality of databases with synchronized data,
In the control procedure,
One of the plurality of databases has a function of replicating data stored in the database in accordance with an instruction from the SDN controller control unit,
The virtual machine refers to another available one of the plurality of databases and performs connection processing of the switch.
The network control method according to claim 7. A plurality of SDN controllers controlling a common output path of the switch;
The database provided in the plurality of SDN controllers stores a flow table having the same content for controlling the output path of the switch.
The network control method according to claim 8. In the determination procedure, the SDN controller control unit determines that a failure has occurred in the SDN controller,
In the control procedure, the SDN controller control unit generates a live clone of the SDN controller in which a failure has occurred.
The network control method according to claim 6.

Images