Background
The virtualization based on the virtual machine and the virtualization based on the container are two typical server virtualization technologies in the current cloud computing field, the former has the advantages of better resource isolation, high safety and decoupling of a client operating system and a host operating system, the latter has the advantages of light weight of a virtualization platform, convenience in service deployment and efficient elastic resource expansion, the two virtualization technologies respectively have different application scenes and can coexist in a foreseeable period of time.
For a virtual machine, the most popular virtual network solution at present is a neutron scheme of openstack, a control plane is implemented by a neutron server service on a control node, and virtual network management of a data plane is implemented by an agent on each computing node, for example, ovs bridge (br-int, br-eth, br-tun, etc.) flow table management is implemented by using ovsag, and defects of the method include: 1) centralized virtual network management results in lower performance and is not suitable for rapid elastic expansion and contraction of a container network; 2) virtualization scenarios that do not support virtual machine nesting with containers.
For a container, the currently popular virtual network solutions include Calico, Flannel, Weave, Libnetwork, and the like, but these solutions are only for the container, do not provide network support of a virtual machine, and even do not support a virtualization scene in which the virtual machine and the container are nested.
Because the virtual machine technology and the container technology are two independently developed technical routes, and the implementation modes of the virtual networks are also independently evolved, the mixed deployment of virtualization and containers, particularly the nested deployment scenario, is difficult to satisfy, and therefore a new network management method is required to provide the unified network management capability of the virtual machines and the containers.
Disclosure of Invention
Aiming at the current mixed deployment of a virtual machine and a container, in particular to a nested deployment scene, the invention provides a virtual network management method of a mixed nested architecture of the virtual machine and the container. And unified fusion management of the virtual machine network and the container network is realized.
The invention integrates the advantages of a centralized controller and a distributed controller, adopts a two-stage controller structure, sinks partial functions of a control plane into a local controller on a hypervisor server, and the centralized controller is responsible for global data maintenance and interface with a third party operation platform and is responsible for flow table maintenance on a local virtual machine switch, thereby effectively reducing the pressure of the global centralized controller and improving the performance of virtual network management.
The specific operation steps are as follows:
1) introducing a local network controller (LC) in the server;
2) the virtual machine and the container on the server are both connected to an internal virtual switch (virtual switch-L0);
3) for a nested scenario in which a container runs on a virtual machine, a container network card is connected to a virtual switch (virtual switch-L1) in the virtual machine;
4) the LC acquires necessary global information, such as basic information of virtual machines and containers of each tenant and physical distribution thereof, from a Unified Network Controller (UNC);
5) when a virtual machine or a container is created, the LC issues a flow table to virtual switch-L0 according to the basic information of the local virtual machine or the container, the access channel of the local virtual machine or the container is opened, and tenants are isolated by the vlan;
6) when a virtual machine or a container is deleted, the LC deletes the flow table related to the virtual machine on the virtual switch-L0;
7) when the virtual machine or the container is migrated, the LC deletes the flow table related to the virtual machine on virtual switch-L0 on the host before migration, and the LC of the target host re-issues the related flow table after migration is completed;
8) for a nested scene that a container runs in a virtual machine, when the container is created, deleted and migrated in the virtual machine, an LC of a host machine issues or deletes a corresponding flow table to virtual switch-L0, and a virtual switch-L1 in the virtual machine is in a common two-layer switching mode;
9) when the virtual machines or containers are communicated across nodes, virtual switch-L0 encapsulates the tenant networks into corresponding formats according to the types of the tenant networks and sends the tenant networks out, and if the vlan-type networks are encapsulated into vlan messages, and the vxlan-type networks are encapsulated into vxlan messages;
10) if the tenant network is of a vlan type, vlan configuration of a physical network is needed firstly;
11) and if the tenant network is of a vxlan type, the vxlan full-connection tunnel between the servers is established by the UNC.
Because the UNC itself is not responsible for the virtualization management work of the virtual machine and the container, the UNC needs to interact with a third-party cloud management platform, take over the virtual network management service, and monitor events such as creation, deletion, migration and the like of the virtual machine and the container.
When a virtual machine is created, the UNC notifies the LC on the hypervisor server where the virtual machine is located of the relevant information of the virtual machine, and the basic information and the distribution situation of other relevant virtual machines and containers of the current tenant network, and the LC issues a flow table to the local virtual switch-L0, and opens up the communication channel from the virtual machine to other virtual machines or containers in the tenant network. In addition, the information and the distribution condition of the virtual machine need to be notified to LCs on hypervisor servers where other virtual machines or containers are located in the network of the tenant, the LCs issue flow tables to their local virtual switch-L0, and the communication channel of the virtual machines to the newly-built virtual machine is opened, so that the bidirectional communication channel between the newly-built virtual machine and other virtual machines or containers in the network of the tenant is established.
When a virtual machine is deleted, the UNC notifies the LCs on the hypervisor server where all virtual machines or containers in the tenant network are located of the information related to the virtual machine, and the LCs delete the flow table related to the virtual machine on the local virtual switch-L0.
However, when the virtual machine is migrated, the UNC notifies the relevant location information before the virtual machine is migrated to LCs on hypervisor servers where all virtual machines or containers in the tenant network are located, the LCs delete the flow table on the local virtual switch-L0 related to the virtual machine, and simultaneously notify the location information after the virtual machine is migrated to LCs on the target hypervisor server after the migration and LCs on hypervisor servers where other virtual machines and containers in the tenant network are located, and the LCs add the flow table related to the virtual machine to the respective local virtual switch-L0.
The creating, deleting, and migrating processes of the container are substantially similar to those of the virtual machine, and are not described herein again, and a nested virtualization scenario in which the container runs in the virtual machine is described below.
When a nested container is created, a virtual machine needs to be newly created, the creation process is as described above, after the virtual machine is created, a container (created by a container management platform, such as kubernets) is created inside the virtual machine, the UNC monitors a container creation event, notifies information related to the container and information of other containers and virtual machines of a current tenant network to an LC on a hypervisor server where the virtual machine is located, and the LC issues a flow table to a local virtual switch-L0 of the local virtual switch, and gets through the container to communication channels of other virtual machines or containers in the tenant network. In addition, the information and distribution condition of the container need to be notified to LCs on hypervisor servers where other virtual machines or containers are located in the tenant network, the LCs issue flow tables to their local virtual switch-L0, and open the communication channels from the virtual machines to the newly-built virtual machine, so that the bidirectional communication channels between the newly-built virtual machine and other virtual machines or containers in the tenant network are established.
The container deletion and migration in the nested virtualization scenario in which the container runs in the virtual machine are substantially consistent with the flow in the non-nested environment, and details are not repeated here.
The communication process between virtual machines or containers in a tenant is also divided into a plurality of scenes, and the scenes specifically include the following steps:
1) communication between containers within a virtual machine. Because the containers in the virtual machines belong to the same tenant, the two conditions can be divided into two conditions, one is that two communication parties are positioned in the same subnet, the two communication parties can directly communicate through virtual switch-L1, the other is that the two communication parties are positioned in different subnets (the two communication parties cannot directly communicate between the different subnets and need to pass through a gateway), the message needs to be sent to the gateway through virtual switch-L0, the gateway changes the mac address of the message destination into the mac address of the network card of the target container, and then the message is sent back to the virtual switch-L0, and then the message is sent to the virtual switch-L1 through the virtual switch-L0 and finally sent to the target container.
2) Communication between a container within a virtual machine and a virtual machine or container on a hypervisor server. Similar to 1), the method is divided into two cases of the same subnet and different subnets, if a source and a destination belong to the same subnet, the communication can be directly established by forwarding through related flow tables on virtual switch-L0, if the source and the destination belong to different subnets, a message is firstly sent to a gateway through virtual switch-L0, the gateway changes the destination mac into the mac of a target virtual machine or container and then sends the mac back to the virtual switch-L0, and then the virtual switch-L0 forwards the mac to a specific virtual machine or container according to the flow tables.
3) And communicating between the virtual machines, between the virtual machines and the containers and between the containers on the hypervisor server. The communication process is similar to 2), and is not described in detail.
4) Communication is performed across nodes. The essential process of cross-node communication is the same as that of non-cross-node communication, the biggest difference is that when the communication is not performed, communication is directly established between a source and a destination through a local virtual machine network, extra encapsulation (such as vlan and vxlan encapsulation) is not needed, when the node communication is performed, no matter whether the source and the destination are located in the same subnet, the communication process can be established through extra encapsulation, and other processes are basically consistent.
The invention also provides a virtual network management system of the virtual machine and container mixed nested architecture, which mainly comprises a centralized controller and a distributed controller.
The centralized controller is responsible for maintaining global information, interacting with a third-party cloud management platform to acquire related information and distribution conditions of the virtual machines, the containers and the like, and monitoring related events of the virtual machines and the containers, such as creation, deletion, migration and the like.
The distributed controller (or local controller) is responsible for obtaining relevant information from the centralized controller and maintaining forwarding flow tables related to the local virtual machines or containers.
The virtualization server is internally connected by two levels of virtual machine switches to form internal virtual network connection, the L0-level virtual switch-L0 is in a flow table forwarding mode and operates on a hypervisor, and the L1-level virtual switch-L1 is in a normal forwarding mode and operates inside a virtual machine.
The tenant isolation of the virtual machine and the container is realized by the virtual switch-L0 of the L0 level virtual switch through vlan isolation on each virtual port, and each container on the virtual switch-L1 of the L1 level switch belongs to the container of the same tenant, so the isolation is not performed.
When the virtual machine or the container is created, the UNC informs LCs on all hypervisor servers in the tenant network of the virtual machine or the container basic information and the distribution position information, and each LC maintains a communication flow table of the newly created virtual machine or the container on the local virtual switch-L0 and other virtual machines and containers in the tenant network.
When the virtual machine or the container is deleted, the UNC notifies the LCs on all hypervisor servers in the tenant network of the information related to the virtual machine or the container, and the LCs delete the flow table related to the virtual machine or the container on the local virtual switch-L0.
When a virtual machine or a container is migrated, the UNC notifies the LCs on all hypervisor servers in the tenant network of the relevant information before the virtual machine or the container is migrated, each LC deletes the flow table on the local virtual switch-L0 relevant to the virtual machine or the container, in addition, the UNC notifies the LCs on all hypervisor servers in the tenant network of the relevant information after the virtual machine or the container is migrated, and each LC maintains the communication flow table between the virtual machine or the container and other virtual machines and containers in the tenant network on the local virtual switch-L0.
The invention has the advantages that
1) The virtual machine and the container are treated in the same way, so that unified network management of the virtual machine and the container can be realized;
2) supporting virtual network management of nested virtualization scenes in which containers run in a virtual machine;
3) the control plane part is arranged on the hypervisor server in a lower layer, and the virtual network management performance can be improved.
Detailed Description
The invention is explained in more detail below:
the mixed deployment of the virtual machine and the container is divided into the following scenes: 1) the virtual machine runs on the physical server; 2) the container runs on a physical server; 3) the container is nested and run in the virtual machine; 4) the virtual machine is nested and run in the container; 5) the virtual machine is nested and run in the virtual machine; 6) the containers run nested in the containers. From the current requirements of cloud computing services and application values, 1 and 2 belong to the most typical application scenarios, and scenario 3 can meet some special application requirements, for example, a container with higher requirements on resource isolation and security can be considered to be deployed in an independent virtual machine, while scenarios 4, 5, and 6 have fewer current actual applications and low application values, and are not considered temporarily, and the unified network management method provided by the invention mainly aims at scenarios 1, 2, and 3, as shown in fig. 1.
The unified network management method provided by the invention comprises the following steps:
12) introducing a local network controller (LC) in the server;
13) the virtual machine and the container on the server are both connected to an internal virtual switch (virtual switch-L0);
14) for a nested scenario in which a container runs on a virtual machine, a container network card is connected to a virtual switch (virtual switch-L1) in the virtual machine;
15) the LC acquires necessary global information, such as basic information of virtual machines and containers of each tenant and physical distribution thereof, from a Unified Network Controller (UNC);
16) when a virtual machine or a container is created, the LC issues a flow table to virtual switch-L0 according to the basic information of the local virtual machine or the container, the access channel of the local virtual machine or the container is opened, and tenants are isolated by the vlan;
17) when a virtual machine or a container is deleted, the LC deletes the flow table related to the virtual machine on the virtual switch-L0;
18) when the virtual machine or the container is migrated, the LC deletes the flow table related to the virtual machine on virtual switch-L0 on the host before migration, and the LC of the target host re-issues the related flow table after migration is completed;
19) for a nested scene that a container runs in a virtual machine, when the container is created, deleted and migrated in the virtual machine, an LC of a host machine issues or deletes a corresponding flow table to virtual switch-L0, and a virtual switch-L1 in the virtual machine is in a common two-layer switching mode;
20) when the virtual machines or containers are communicated across nodes, virtual switch-L0 encapsulates the tenant networks into corresponding formats according to the types of the tenant networks and sends the tenant networks out, and if the vlan-type networks are encapsulated into vlan messages, and the vxlan-type networks are encapsulated into vxlan messages;
21) if the tenant network is of a vlan type, vlan configuration of a physical network is needed firstly;
22) and if the tenant network is of a vxlan type, the vxlan full-connection tunnel between the servers is established by the UNC.
Fig. 2 is a connection relationship of a virtual network inside a server, where a container and a virtual machine on a hypervisor server are both connected to a local virtual switch-L0 through a tap port or a virtual port (e.g. veth), and the virtual switch-L0 operates in a flow table forwarding mode, and flow table rules thereof are issued by a local controller LC. The containers in the virtual machines are connected to a virtual switch-L1 inside the virtual machines through tap port virtual ports (such as veth), the virtual switch-L1 works in a common L2 forwarding mode (because the containers in one virtual machine belong to one tenant, and thus vlan isolation is not needed between the containers in the virtual machine), since the tap of the virtual machine is hooked to virtual switch-L0, thus virtual switch-L0 is in communication with virtual switch-L1, since there may be virtual machines or containers of multiple tenants on the hypervisor server, therefore, the vlan isolation of the virtual machine or container of the tenant needs to be started on the virtual port of the virtual switch-L0, a unique local vlan number is allocated to different tenants, when a virtual machine or a container communicates across physical nodes, the local vlan number in the message needs to be replaced with the vlan number of the actual physical network (vlan encapsulation) or vni of the vxlan (vxlan encapsulation).
Fig. 3 is an overall architecture of a unified virtual network management system, and unlike a conventional virtual network solution (e.g. neutron), the present invention employs a control plane combining centralized and distributed control, in which a global central controller UNC is maintained, the UNC is responsible for maintaining global information of a data center network and basic information of virtual machines and containers (calling related interfaces to obtain from a third-party platform, such as openstack or kubernets, etc.), and furthermore, a lightweight local controller LC is started on each hypervisor server, the LC is responsible for maintaining flow table rules related to local virtual machines and containers on a virtual switch-L0, and the hidden LC needs to obtain sufficient global information from the UNC, such as container and virtual machine basic information and distribution conditions of tenants.
The unified virtual machine network management method provided by the invention treats the virtual machine and the container in the same way and the same way, provides the virtual network management capability of the container and the virtual machine integration, can support the nested virtualization scene of the container running in the virtual machine, and enriches the virtualization application scene.