CN114070822B

CN114070822B - Kubernetes Overlay IP address management method

Info

Publication number: CN114070822B
Application number: CN202111362826.9A
Authority: CN
Inventors: 花磊; 刘学聪; 梁兵; 崔骥; 赵安全; 王亮
Original assignee: Jiangsu Boyun Technology Co ltd
Current assignee: Jiangsu Boyun Technology Co ltd
Priority date: 2021-11-17
Filing date: 2021-11-17
Publication date: 2022-10-14
Anticipated expiration: 2041-11-17
Also published as: CN114070822A

Abstract

The application relates to a Kubernets Overlay IP address management method, which belongs to the technical field of communication and comprises the steps of obtaining network information of a Kubernets cluster, wherein the network information comprises all available IP addresses, available networks and all used IP addresses in the Kubernets cluster; initializing an Overlay network based on the network information; determining whether a subCIDR block has been allocated by a subCIDR of a node; if the CIDR block is not allocated, allocating the CIDR block for the subCIDR; when an IP allocation request is received, allocating an IP address from a subbCIDR of the node, wherein the IP allocation request is used for requesting to allocate the IP address for the Pod; the problem of low IP address distribution efficiency among different nodes can be solved; when the pod of different nodes needs to distribute the IP address, the distributed IP address distribution can be realized, and the system performance is improved.

Description

Kubernetes Overlay IP address management method

[ technical field ] A method for producing a semiconductor device

The application relates to a Kubernets Overlay IP address management method, and belongs to the technical field of communication.

[ background of the invention ]

Kubernets is an open source container orchestration engine that supports automated deployment, large-scale scalable, application containerization management. Kubernets provides network functionality for all container platforms that also meet the protocol, as long as it provides a standard interface. Container Network Interface (CNI) is just such a standard Interface protocol.

At present, the mainstream CNI manages IP addresses mainly by maintaining a network segment of a current node, and preferentially allocates an IP of the network segment of the current node each time an IP is allocated to a pod, and if the IP of the current node is not enough, the IP is acquired from network segments of other nodes. When releasing the IP address, inquiring a corresponding network block according to the information of the pod, such as the network name and the container ID, and if the block does not exist, considering that the block is released; the IP corresponding to the network block is released directly and then the corresponding IP pool is updated.

However, at present, the management of the IP pool is not mature, and the allocation efficiency of the IP addresses among different nodes is low.

[ summary of the invention ]

Through the design mode of the subbCIDR, each node has the corresponding subbCIDR with the spare IP address, when the IP is allocated, the IP address in the subbCIDR of the current node can be directly used, when the pod of different nodes needs to allocate the IP address, the distributed allocation of the IP address can be realized, and the performance of the system is improved. The application provides the following technical scheme:

the method is used for nodes in a Kubernets cluster, at least one subcIDR is arranged in each node, and each subcIDR is an object responsible for allocating IP addresses; the method comprises the following steps:

acquiring network information of the Kubernets cluster, wherein the network information comprises all available IP addresses, available networks and all used IP addresses in the Kubernets cluster;

initializing an Overlay network based on the network information, and acquiring all subcIDRs;

determining whether a subCIDR block of the node has been allocated;

if the CIDR block is not allocated, allocating the CIDR block for the subbCIDR;

and when an IP allocation request is received, allocating an IP address from the subbCIDR of the node, wherein the IP allocation request is used for requesting to allocate the IP address for the Pod.

Optionally, the method further comprises:

and synchronizing a first cache every a first preset time length through a go protocol, wherein the first cache stores a corresponding relation between an available network obtained after initializing an Overlay network and the CIDR.

Optionally, the method further comprises:

and synchronizing a second cache every second preset time through a go protocol, wherein the second cache stores the network state of an available network, the CIDR state of the subbCIDR and the use state of the IP address in the subbCIDR.

Optionally, the method further comprises:

determining whether the subbCIDR exists in the node through a go coroutine;

and if the subbCIDR does not exist, creating the subbCIDR through a go protocol.

Optionally, after determining whether the subbcidr exists in the node through a go protocol, the method further includes:

if the subbCIDR exists, determining whether an available IP address exists in the subbCIDR through a go protocol;

and if no available IP address exists, creating the subbCIDR through a go protocol.

Optionally, the allocating an IP address from the subbcidr of the node when the IP allocation request is received includes:

sending the IP allocation request to a CNI through a Kubelet;

sending an HTTP request for generating an IP address to the fabric-ctl through the CNI based on the IP allocation request;

determining whether the Pod has a reservation or is in a host network mode according to Pod information carried in the HTTP request through the fabric-ctl;

in the event that the Pod has no subscription and is not in host network mode, an IP address is allocated from the subbcidr of the node.

Optionally, after determining, by the fabric-ctl according to Pod information carried in the HTTP request, whether the Pod has a reservation or is in a host network mode, the method further includes:

under the condition that the Pod has a reservation, allocating a reserved IP address for the Pod;

or,

and under the condition that the Pod is in the host network mode, allocating a host IP address for the Pod.

Optionally, after allocating an IP address from the subbcidr of the node when receiving the IP allocation request, the method further includes:

and when the IP release request of the Pod is received, releasing the IP address of the Pod.

Optionally, the releasing the IP address of the Pod when receiving the IP release request of the Pod includes:

sending the IP release request to a CNI through a Kubelet;

sending the Pod information of the Pod to a fabric-ctl through the fabric based on the IP release request;

determining whether the Pod is stopped or not according to the Pod information through the fabric-ctl;

if the Pod is not stopped, determining whether the Pod has a reservation or is in a host network mode;

and releasing the IP address allocated to the Pod in the subbCIDR of the node under the condition that the Pod has no reservation and is not in the host network mode.

Optionally, the kubernets cluster supports multiple encapsulation protocols; the Pod in the Kubernetes cluster supports multiple network cards and is isolated from each other.

The beneficial effect of this application includes at least: acquiring network information of a Kubernetes cluster, wherein the network information comprises all available IP addresses, available networks and all used IP addresses in the Kubernetes cluster; initializing an Overlay network based on network information, and acquiring all subcoIDRs; determining whether a subCIDR block has been allocated by a subCIDR of a node; if the CIDR block is not allocated, allocating the CIDR block for the subCIDR; when an IP allocation request is received, allocating an IP address from the subcIDR of the slave node, wherein the IP allocation request is used for requesting to allocate the IP address for the Pod; the problem of low IP address distribution efficiency among different nodes can be solved; because each node has its corresponding subcIDR with a vacant IP address, when allocating IP, the IP address in the subcIDR of the current node can be directly used, when the pod of different node needs to allocate IP address, the distributed allocation of IP address can be realized, and the system performance is improved.

In addition, the IP address is recovered by periodically synchronizing the network cache; the problem that network resources are wasted due to the fact that network segments distributed by nodes cannot be automatically recovered when the nodes go out of the problem can be solved; the utilization rate of the IP address can be improved.

In addition, the fixing and the reservation of the IP address of the container are supported, and the problem of the drift of the IP address can be solved.

In addition, the method supports multiple networks, and can solve the problems that only a single network is supported, the requirement of multiple networks cannot be processed, and other plug-ins of third parties are needed for assistance; because an auxiliary plug-in of a third party is not needed, the Pod is supported to correspond to a plurality of networks, and the isolation of the network layer is realized.

In addition, a plurality of encapsulation protocols (vxlan, gre, gene, etc.) are supported, and the encapsulation protocols can be selected according to actual scenes.

The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical solutions of the present application more clear and clear, and to implement the technical solutions according to the content of the description, the following detailed description is made with reference to the preferred embodiments of the present application and the accompanying drawings.

[ description of the drawings ]

Fig. 1 is a schematic structural diagram of a kubernets Overlay IP address management system according to an embodiment of the present application;

fig. 2 is a flowchart of a kubernets Overlay IP address management method according to an embodiment of the present application;

fig. 3 is a flowchart of a network initialization procedure of kubernets Overlay according to an embodiment of the present application;

fig. 4 is a flowchart of an IP address assignment process of kubernets Overlay according to an embodiment of the present application;

fig. 5 is a flowchart of an IP address release procedure of kubernets Overlay according to an embodiment of the present application.

[ detailed description ] embodiments

The following detailed description of embodiments of the present application will be made with reference to the accompanying drawings and examples. The following examples are intended to illustrate the present application, but are not intended to limit the scope of the present application.

First, a number of terms referred to in the present application will be described.

Hypertext Transfer Protocol (HTTP) and Application Programming Interface (API) are structures built based on Programming languages that provide some pre-definitions to provide functionality for applications to use without accessing source code.

Overlay-a virtualization technology model overlaid on a network fabric, nodes in an Overlay network may be considered to be connected by virtual or logical links. Where each link corresponds to a path. Nodes may also be connected by multiple physical connections in the underlying network.

Kubernetes: the abbreviation K8s is an abbreviation resulting from 8 replacing the 8 characters "ubernet" in the middle of the name. Kubernetes is an open source container orchestration engine that supports automated deployment, large-scale scalable, application containerization management. When an application is deployed in a production environment, multiple instances of the application are typically deployed to load balance application requests. In kubernets, multiple containers can be created, one application instance runs in each container, and then management, discovery and access to the group of application instances are realized through a built-in load balancing policy.

Kubelet: the core component of Kubernets is a proxy component on the working node of Kubernets, and runs on each node. The Kubelet periodically receives new or modified Pod specifications from the kube-apiserver component and ensures that the Pod and its container operate under the desired specifications. Meanwhile, the component serves as a monitoring component of the working node and reports the running condition of the host to the kube-apiserver. In other words, kubelet is responsible for the running state of each node (i.e., ensuring that all containers on the node are running properly). It handles starting, stopping and maintaining the application container Pod as instructed by the control panel.

The kube-api server component provides HTTP Rest interfaces such as add, delete, modify, check and monitor (watch) of various k8s resource objects (container set (pod), copy Controller (RC), service, etc.), and is a data bus and a data center of the whole system.

Container Network Interface (CNI): the method is a specification definition between the container runtime and the network plug-in, and software meeting the definition can provide network configuration service for the container management system. CNI is a standard interface protocol for connecting a container management system to network plugins. The operation contents of the CNI include: obtaining runtime information from a processor management system; and loading network configuration information from the configuration file of the container network, transmitting the information to the corresponding plug-in, performing specific network configuration work by the plug-in, and returning the configuration result to the container management system. The CNI plug-in is an executable file that is called by kubel.

Wherein the runtime information includes but is not limited to: a path of a network name space (network name), a container ID, and a network interface name (network interface name).

Network Protocol Address (IP Address): a string of numbers used in network protocols to identify a device that sends or receives datagrams. When a device connects to the network, the device will be assigned an IP address, which serves as an identification. Through the IP address, the devices can communicate with each other, and if the IP address is not available, the device can not know which device is the sender and which is the receiver. An IP address contains two parts: a prefix identifying the network, and the host address that follows within this network.

Classless Inter-Domain Routing (CIDR): a method for assigning IP addresses to users and for sorting IP addresses for efficient routing of IP packets over the Internet. CIDR is based on Variable Length Subnet Mask (VLSM) to assign prefixes of IP addresses of arbitrary lengths.

CIDR Block (CIDR Block): address block of the network. CIDR is primarily a bitwise, prefix-based standard for interpreting IP addresses. CIDR facilitates routing by combining multiple address blocks, i.e., CIDR blocks, into a routing table entry.

subco IDR: from the network segments divided in the network created currently, one CIDR can have a plurality of subbcidrs, the proposal is responsible for allocating objects responsible for allocating IP, each node has at least one subbcidr, when allocating an IP address for a pod, the address in the subbcidr of the node is used, and when a plurality of pods allocate IP addresses, the address can be allocated in a distributed manner.

The fabric is a CNI plug-in based on Open Virtual Switch (OVS) standard, and is used to establish a network for Pod and set an IP address.

The fabric-ctl is responsible for network and IP address management, and provides network/IP management capabilities through RESTFUL API, such as creating a network, editing a network, finding an IP, and the like. The fabric-ctl has no state, and all state information is stored in the etcd storage database.

Go program (goroutine): the method refers to a lightweight execution thread running in a background, and a Go coroutine is a key component for realizing concurrency in Go.

Fig. 1 is a schematic structural diagram of a kubernets Overlay IP address management system according to an embodiment of the present application, where the system includes at least one node 110 in a kubernets cluster. In fig. 1, the number of nodes is 2 for example, but in actual implementation, the number of nodes may be two or more, and the number of nodes is not limited in this embodiment. Each node is provided with at least one subbCIDR 111, and the subbCIDR is an object responsible for allocating IP addresses. In this embodiment, subbcidr 111 is co-processed by go.

Node 110 has kubel 112, CNI113, and fabric-ctl114 running therein to enable assignment and release of IP addresses in subcid 111.

Specifically, kubel 112 calls CNI113 through a command line, and CNI113 passes parameters to fabric-ctl114 in the form of an http api; the fabric-ctl114 also returns data via http; CNI113 passes data to kubel through the command line.

In this embodiment, based on the operating environment shown in fig. 1 and with reference to fig. 2, the kubernets Overlay IP address management method provided in this embodiment at least includes the following steps:

step 201, network information of the kubernets cluster is obtained, and the network information includes all available IP addresses, available networks and all used IP addresses in the kubernets cluster.

Step 202, initializing the Overlay network based on the network information, and acquiring all subcIDRs.

After the Overlay network is initialized, the available network is written into the first cache, and all subCIDRs are acquired.

Step 203, determine whether the subCIDR of the node has allocated a CIDR block.

And step 204, if the CIDR blocks are not allocated, allocating the CIDR blocks for the subCIDR.

Alternatively, if the CIDR block is allocated, the subbcidr allocated address in the CIDR is updated, and the used IP address and the reserved IP address are synchronized.

Step 205, when receiving the IP allocation request, allocating an IP address from the subCIDR of the node, where the IP allocation request is used to request allocation of an IP address for the Pod.

Specifically, the following IP address management method provided in this embodiment includes three stages, which are: the network initialization phase, the IP address allocation phase and the IP address release phase are described as examples.

Referring to the network initialization process shown in fig. 3, as can be seen from fig. 3, the network initialization process at least includes the following steps:

step 31, acquiring network information of Kubernets cluster, and executing

steps

34 and 392.

And step 32, synchronizing a first cache every other first preset time length through a go protocol, wherein the first cache stores a corresponding relation between an available network obtained after the Overlay network is initialized and the CIDR.

And step 33, synchronizing the second cache every second preset time length through the go routine, wherein the network state of the available network, the CIDR state of the subbCIDR and the use state of the IP address in the subbCIDR are stored in the second cache.

The present embodiment does not limit the execution sequence among step 31, step 32 and step 33.

Wherein, the first preset duration is longer than the second preset duration, for example: the first preset time period is 15 minutes, the second preset time period is 1 minute, and the values of the first preset time period and the second preset time period are not limited in this embodiment.

Step 34, initializing an overlay network, and caching network and CIDR corresponding to the network;

step 35, acquiring all subCIDRs;

step 36, judging whether subcodDRR has a corresponding CIDRBlock; if not, go to step 37; if so, go to step 38;

step 37, distributing CIDR blocks for subCIDR, and ending the initialization process;

step 38, updating the IP address allocated by subCIDR in CIDR;

step 39, synchronizing the used IP address and the reserved IP address;

step 391, updating the status of subCIDR;

step 392, determining whether a node has subcIDR which is not allocated with an unused IP address through go coroutine; if not, go to step 393; if so, go to step 392 again;

at step 393, a subCIDR is created by the go protocol.

Steps 392-393 are performed after step 31, and the order of execution between steps 392-393 and steps 34-391 is not limited in this embodiment.

Referring to the IP address assignment process shown in fig. 4, as an alternative step to step 205, as can be seen from fig. 4, the IP address assignment process includes at least the following steps:

step 41, sending an IP allocation request to the CNI through the Kubelet;

specifically, kubelet sends an IP allocation request to CNI through a command line.

Step 42, sending HTTP request for generating IP address to the fabric-ctl through CNI based on IP allocation request;

step 43, determining whether the Pod has a reservation or is in a host network mode according to Pod information carried in the HTTP request through the fabric-ctl; if yes, go to step 44; if not, go to step 48;

step 44, under the condition that the Pod has no reservation and is not in the host network mode, allocating an IP address from the subCIDR of the node;

step 45, determining whether the address allocation is successful; if successful, go to step 46; if not, go to step 47;

step 46, the fabric-ctl returns successful allocation information to the Kubelet through the CNI, and the IP address allocation process is finished;

step 47, the fabric-ctl returns the distribution failure information to the Kubelet through the CNI, and the IP address distribution process is finished;

48, distributing reserved IP addresses for the Pod under the condition that the Pod has a reservation; alternatively, in the case where the Pod is in the host network mode, a host IP address is allocated to the Pod.

Optionally, after the IP address is allocated to the Pod, the IP address of the Pod may also be released. Specifically, referring to the steps and referring to the releasing process of the IP address shown in fig. 5, as can be seen from fig. 5, after step 205, the releasing process of the IP address at least includes the following steps:

step 51, sending an IP release request to the CNI through the Kubelet;

specifically, kubelet sends an IP release request to CNI through a command line.

Step 52, the Pod information of the Pod is sent to the fabric-ctl through the fabric based on the IP release request;

step 53, determining whether the Pod is stopped or not through the fabric-ctl according to the Pod information; if not, go to step 54; if the stop is detected, go to step 57;

step 54, if the Pod is not stopped, determining whether the Pod has a reservation or is in a host network mode; if not, go to step 55; if yes, go to step 56;

and step 55, under the condition that the Pod is not reserved and is not in the host network mode, releasing the IP address allocated to the Pod in the subcIDR of the node, and executing step 57.

Step 56, releasing the reserved IP address or the host IP address;

in step 57, the fabric-ctl returns a release success message to the Kubelet through the CNI.

In this embodiment, the kubernets cluster supports multiple encapsulation protocols (e.g., vxlan, gre, gene, etc.); the Pod in the Kubernetes cluster supports multiple network cards and is isolated from each other.

In summary, in the kubernets Overlay IP address management method provided in this embodiment, network information of a kubernets cluster is obtained, where the network information includes all available IP addresses, available networks, and all used IP addresses in the kubernets cluster; initializing an Overlay network based on network information, and acquiring all subcoIDRs; determining whether a subCIDR block has been allocated by a subCIDR of a node; if the CIDR block is not allocated, allocating the CIDR block for the subCIDR; when an IP allocation request is received, allocating an IP address from a subbCIDR of the node, wherein the IP allocation request is used for requesting to allocate the IP address for the Pod; the problem of low IP address distribution efficiency among different nodes can be solved; because each node has its corresponding subcIDR with a vacant IP address, when allocating IP, the IP address in the subcIDR of the current node can be directly used, when the pod of different node needs to allocate IP address, the distributed allocation of IP address can be realized, and the system performance is improved.

All possible combinations of the technical features of the above embodiments may not be described for the sake of brevity, but should be considered as within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.

Claims

1. The Kubernets Overlay IP address management method is characterized by being used in nodes in a Kubernets cluster, wherein at least one subcIDR is arranged in each node, and each subcIDR is an object responsible for allocating an IP address; wherein said Kubernetes is an open source container orchestration engine; the subbCIDR can have a plurality of subbCIDRs from the network segments divided from the network created currently, and the CIDR is a method for allocating IP addresses and classifying the IP addresses of effective routing IP data packets on the Internet; the Overlay is a virtualization technology mode superposed on a network architecture; the method comprises the following steps:

determining whether a subCIDR block has been allocated by the subCIDR of the node;

if the CIDR block is not allocated, allocating the CIDR block for the subbCIDR;

when an IP allocation request is received, allocating an IP address from the subbCIDR of the node, wherein the IP allocation request is used for requesting to allocate the IP address for the Pod; the pod is a container set;

when an IP allocation request is received, allocating an IP address from the subbCIDR of the node comprises the following steps:

sending the IP allocation request to a CNI through a Kubelet; the Kubelet is a core component of Kubernets; the CNI is a specification definition between the container runtime and the network plug-in;

determining whether the Pod has a reservation or is in a host network mode according to Pod information carried in the HTTP request through the fabric-ctl; the fabric-ctl is responsible for network and IP address management;

in the case where the Pod is out of subscription and not in host network mode, an IP address is allocated from the subCIDR of the node.

2. The method of claim 1, further comprising:

synchronizing a first cache at intervals of a first preset time length through a go protocol, wherein the first cache stores a corresponding relation between an available network obtained after initializing an Overlay network and a CIDR; the go protocol refers to a lightweight execution thread running in the background.

3. The method of claim 1, further comprising:

and synchronizing a second cache every other second preset duration through a go routine, wherein the network state of an available network, the CIDR state of the subbCIDR and the use state of the IP address in the subbCIDR are stored in the second cache.

4. The method of claim 1, further comprising:

determining whether the subbCIDR exists in the node through a go coroutine;

5. The method of claim 4, wherein after determining whether the subbCIDR exists for the node via a go protocol, further comprising:

and if no available IP address exists, creating the subcoBIDR through a go protocol.

6. The method of claim 1, wherein after determining, by the fabric-ctl according to Pod information carried in the HTTP request, whether the Pod has a reservation or is in a host network mode, the method further comprises:

or,

7. The method of claim 1, wherein after allocating the IP address from the subCIDR of the node when the IP allocation request is received, the method further comprises:

8. The method of claim 7, wherein the releasing the IP address of the Pod upon receiving the IP release request of the Pod comprises:

sending the IP release request to the CNI through a Kubelet;

and releasing the IP address allocated to the Pod in the subcIDR of the node under the condition that the Pod has no reservation and is not in the host network mode.

9. The method according to any of claims 1 to 8, wherein said kubernets cluster supports multiple encapsulation protocols; and the Pod in the Kubernets cluster supports a plurality of network cards and is isolated from each other.