CN117811919A

CN117811919A - Communication method, system, equipment and medium of heterogeneous CNI of large-scale cloud platform

Info

Publication number: CN117811919A
Application number: CN202410034624.9A
Authority: CN
Inventors: 李阳
Original assignee: Transwarp Technology Shanghai Co Ltd
Current assignee: Transwarp Technology Shanghai Co Ltd
Priority date: 2024-01-10
Filing date: 2024-01-10
Publication date: 2024-04-02

Abstract

The invention discloses a communication method, a system, equipment and a medium of heterogeneous CNI of a large-scale cloud platform. The method comprises the steps that a second heterogeneous CNI communication agent in a second Node responds to an update event of a first Vectonode resource of a first Node in a k8s cluster, a target Vectonode resource after the first Vectonode resource is updated is obtained, and whether the types of the corresponding CNIs are the same is determined according to the target Vectonode resource and the second Vectonode resource corresponding to the second Node; under the condition that the types of CNIs are the same, configuring a communication information table for VTEP equipment in the second Node according to the target vector Node resource; and realizing the VxLAN tunnel communication between the first Node and the second Node according to the communication information table and the VTEP equipment. According to the embodiment of the invention, through the technical scheme, pod networks of different types of CNIs can be opened, so that normal access among Pods of different types of CNIs is ensured, smooth transition is realized in the CNI upgrading process, the problem that a large number of Pods are restarted at the same time to cause service influence is solved, and the service in the whole cluster is ensured not to be influenced.

Description

Communication method, system, equipment and medium of heterogeneous CNI of large-scale cloud platform

Technical Field

The invention relates to the technical field of cloud computing, in particular to a communication method, a system, equipment and a medium of heterogeneous CNI of a large-scale cloud platform.

Background

Kubernetes (k 8 s) based cloud platforms typically use network plug-ins (Container Network Interface, CNI) to provide a network for the Pod of the Node. However, when the existing CNI lacks a security policy and has advanced functions such as centralized exit, etc. for some reasons, the CNI is replaced by the whole cluster, the network between the Pod and Pod of the Node is interrupted during the process of replacing the CNI, and the Pod needs to restart to use a new CNI, and the manner of replacing the CNI has limited influence under the small-scale cluster (the CNI replacement time is short, the number of pods is small, the restart is fast, and the service recovery is fast). However, if in the large-scale cluster environment, after the CNI is replaced quickly, all Pod of all nodes need to be restarted to use the latest CNI, and restarting a large number of Pod simultaneously can cause serious influence on the service, thereby affecting the network communication inside the cluster, not being able to access normally, and the service is also affected, so a communication method for replacing the network plug-in by a large-scale cloud platform is needed to solve the above technical problems.

Disclosure of Invention

In view of this, the invention provides a communication method, a system, a device and a medium for heterogeneous CNIs of a large-scale cloud platform, which can open up Pod networks of two different types of CNIs, further ensure normal access between pods of different types of CNIs, and ensure that the service is not affected after the CNIs are replaced, thereby realizing smooth transition in the process of upgrading (replacing) the CNIs, solving the problem that a large number of pods are restarted simultaneously to cause the service to be affected, and ensuring that the service in the whole cluster is not affected.

According to one aspect of the invention, the embodiment of the invention provides a communication method of heterogeneous CNI of a large-scale cloud platform, which is applied to a second Node in a k8s cluster; the second Node is the Node of which the configuration catalog is unchanged; the method comprises the following steps:

a second heterogeneous CNI communication agent in the second Node responds to an update event of a first Vector Node resource of a first Node in the k8s cluster, acquires a target Vector Node resource updated by the first Vector Node resource and a pre-created second Vector Node resource corresponding to the second Node, and determines whether the types of network plug-in CNIs corresponding to the first Node and the second Node are the same according to the target Vector Node resource and the second Vector Node resource; the first Node is a Node with changed configuration catalogue;

Under the condition that the types of the network plug-ins CNI are different, configuring a communication information table between the first Node and the second Node according to the VTEP equipment which is pre-established in the second Node by the target Vector Node resource;

and realizing VxLAN tunnel communication between the first Node and the second Node according to the communication information table and the VTEP equipment.

According to another aspect of the present invention, an embodiment of the present invention further provides a communication system of heterogeneous CNIs of a large-scale cloud platform, where the communication system includes: a K8s cluster, wherein the K8s cluster comprises at least two Node nodes and a control Node, and the at least two Node nodes comprise a first Node and a second Node; the first Node comprises: a first Node; the second Node includes: a second heterogeneous CNI communication agent; the control node comprises: kube-apiserver component;

the first Node is a Node with changed configuration catalogue; the second Node is the Node of which the configuration catalog is unchanged;

the first Node is configured to obtain a first Vector Node resource created in advance corresponding to the first Node in response to a change in a configuration directory of a network plug-in CNI in the first Node, and update field information of the network plug-in CNI with a changed configuration target to an original CNI field in the first Vector Node resource created in advance, so as to obtain an updated target Vector Node resource;

The second Node is configured to respond to an update event of a first Vector Node resource of a first Node in the k8s cluster, obtain a target Vector Node resource after the first Vector Node resource is updated, and a second Vector Node resource created in advance and corresponding to the second Node, and determine whether types of network plug-ins CNIs corresponding to the first Node and the second Node are the same according to the target Vector Node resource and the second Vector Node resource; the first Node is a Node with changed configuration catalogue;

the second Node is further configured to configure, when the types of the network plug-ins CNIs are different, a communication information table between the first Node and the second Node for VTEP equipment created in advance in the second Node according to the target Vector Node resource;

and the second Node is also used for realizing the VxLAN tunnel communication between the first Node and the second Node according to the communication information table.

According to another aspect of the present invention, an embodiment of the present invention further provides an electronic device, including:

At least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method for large-scale cloud platform heterogeneous CNI communication according to any of the embodiments of the present invention.

According to another aspect of the present invention, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores computer instructions, where the computer instructions are configured to enable, when executed by a processor, the method for implementing the communication method of heterogeneous CNIs of a large-scale cloud platform according to any embodiment of the present invention.

According to the technical scheme of the embodiment of the invention, the second heterogeneous CNI communication proxy responds to the update event of the first Vector Node resource of the first Node in the k8s cluster, whether the types of the corresponding CNIs are the same is determined according to the updated first Vector Node resource and the second Vector Node resource, under the condition that the types of the CNIs are the same, the communication information table is configured for the VTEP equipment in the second Node according to the target Vector Node resource, and the VxLAN tunnel communication is realized according to the communication information table and the VTEP equipment, so that the Pod networks of two different CNIs are communicated with each other, normal access between the Pod of the new CNI and the Pod of the old CNI is ensured, the service is not affected after the CNI is replaced, thereby realizing smooth transition in the CNI upgrading (replacing) process, preventing the problem that the service is affected due to restarting when the CNI is replaced in a large-scale environment, and ensuring that the service in the whole cluster is not affected.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a communication method of heterogeneous CNI of a large-scale cloud platform according to an embodiment of the present invention;

fig. 2 is a flowchart of another communication method of heterogeneous CNIs of a large-scale cloud platform according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a preparation environment before replacing a CNI according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a communication method of a large-scale cloud platform heterogeneous CNI according to an embodiment of the present invention;

fig. 5 is a block diagram of a communication system of heterogeneous CNI of a large-scale cloud platform according to an embodiment of the present invention;

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In an embodiment, fig. 1 is a flowchart of a communication method of a heterogeneous CNI of a large-scale cloud platform according to an embodiment of the present invention, where the method may be performed by a communication system of a heterogeneous CNI of a large-scale cloud platform, and the communication system of the heterogeneous CNI of a large-scale cloud platform may be implemented in a form of hardware and/or software, and the communication system of the heterogeneous CNI of a large-scale cloud platform may be configured in an electronic device.

As shown in fig. 1, the communication method of heterogeneous CNI of a large-scale cloud platform in this embodiment is applied to a k8s cluster, and the method is applied to a second Node in the k8s cluster; the second Node is Node with unchanged configuration catalog, and the method comprises the following specific steps:

s110, a second heterogeneous CNI communication agent in a second Node responds to an update event of a first Vector Node resource of a first Node in a k8S cluster, acquires a target Vector Node resource updated by the first Vector Node resource and a pre-established second Vector Node resource corresponding to the second Node, and determines whether the types of network plug-in CNIs corresponding to the first Node and the second Node are the same according to the target Vector Node resource and the second Vector Node resource.

The first Node is a Node with changed configuration catalogue, and the second Node is a Node with unchanged configuration catalogue.

The configuration directory in this embodiment includes the type of CNI and other related information. In this implementation, the first Node may include one or more Node nodes, where whether the configuration directory in the first Node is changed or not may be obtained by monitoring the first heterogeneous CNI communication agent in the first Node, which may be understood that the first heterogeneous CNI communication agent in the first Node monitors whether the configuration directory of the first Node itself is changed in real time or periodically. The first heterogeneous CNI communication agent may also be referred to as a vector-agent, and may be understood as a communication agent.

In this embodiment, the first Vector Node resources are Vector Node resources corresponding to the first Node nodes created in advance, and of course, since the first Node nodes include one or more Node nodes, each first Node corresponds to a corresponding first Vector Node resource, and the first Vector Node resources may include, but are not limited to, an original Node IP, a CNI type, and a Pod network segment corresponding to the first Node. In this embodiment, after the configuration directory of the target Vector Node resource, that is, the CNI, is changed, the new CNI is replaced with the original CNI, and the updated Vector Node resource is obtained, where the target Vector Node resource in this embodiment includes: the first Node corresponds to a first Node IP, a first CNI type, a Mac address of a first VXLAN tunnel endpoint (VXLAN Tunnel Endpoint, VTEP) device and a first Pod network segment. In this embodiment, the pre-creation of the first Vector Node resource includes: after the heterogeneous CNI communication agent in the first Node is started, reading a host name, a Node IP and a Node subnet section corresponding to the first Node through a kube-apiserver; obtaining Mac addresses of corresponding VTEP equipment in the first Node, and reading configuration catalogs of CNI on the first Node; the configuration catalog comprises the types of CNIs; creating and storing a first Vector Node resource corresponding to a first Node according to the Node IP, the sub-network segment of the Node, the mac address and the type of the network plug-in CNI; the names of the first Vector Node resources are respectively set as corresponding host names.

In this embodiment, the number of the second Node nodes may be one or more; the second Vector Node resource is a Vector Node resource corresponding to the Node which is created in advance. The second Vector Node resources may include, but are not limited to including: the second Node corresponds to a second Node IP, a second CNI type, a mac address of a second VTEP device and a second Pod network segment.

In this embodiment, the first heterogeneous CNI communication proxy in the first Node may perform timing or real-time monitoring according to a preset time to determine whether a configuration directory of the CNI in the corresponding Node changes, where the first heterogeneous CNI communication proxy in the first Node responds to the change of the configuration directory of the CNI in the first Node, acquires a first Vector Node resource created in advance corresponding to the first Node, and updates field information of the CNI with the changed configuration target to an original CNI field in the first Vector Node resource created in advance, to obtain an updated target Vector Node resource. It can be understood that the Vector-agent in the first Node monitors whether the configuration directory of the CNI changes, and in the process of replacing the CNI, the file under the configuration directory changes, when the configuration directory changes, the Vector-agent of the Node replacing the CNI obtains the latest CNI Type, updates the latest CNI Type to the CNI Type field of the Node corresponding to the Vector Node resource replacing the CNI to obtain the updated target Vector Node resource, and in the K8s cluster, the Node 1, the Node 2, and the Node 3 are included, and after the configuration directories corresponding to the Node 1 and the Node 2 are replaced respectively, the configuration catalog is changed, the Vector-agent of the Node 1 detects that the target of the CNI in the Node 1 is changed, the latest CNI Type is obtained, the latest CNI Type is updated to the CNI Type field of the Vector Node resource corresponding to the Node replacing the CNI, and likewise, the Vector-agent of the Node 2 detects that the target of the CNI in the Node 2 is changed, the latest CNI Type is obtained, and the latest CNI Type is updated to the CNI Type field of the Vector Node resource corresponding to the Node replacing the CNI.

In this embodiment, the types of CNIs frequently used by the k8s platform may include, but are not limited to, a flat, a calico, a cilium, an antra, etc., where the types of CNIs of the network plug-ins are all network address assignment in units of Node, and illustratively, the entire k8s cluster is assigned one large network segment 10.224.0.0/16, each Node in the k8s cluster is assigned a sub-network segment from this large network segment, for example, node1 is assigned 10.224.0.0/24, node2 is assigned 10.224.1.0/24, node3 is assigned 10.224.2.0/24, and so on. And then, the network plug-in CNI carries out configuration of Node routing according to the sub-network segments allocated by the nodes.

In one embodiment, the K8s cluster includes at least two Node nodes and a control Node, and the control Node includes a kube-apiserver component; the VTEP equipment is also pre-established in the first Node; the virtual extensible local area network ID and the port number set by the VTEP equipment pre-established in the second Node are the same, the set local IP is the Node IP of the corresponding Node, and the state is set to be UP.

In an embodiment, the pre-creation of the second Vector Node resource includes: after the heterogeneous CNI communication agent in the second Node is started, reading the host name, the Node IP and the subnet section of the Node respectively corresponding to the second Node through a kube-apiserver; obtaining Mac addresses of corresponding VTEP equipment in the second Node, and respectively reading configuration catalogs of network plug-in CNI on the second Node; the configuration catalog comprises the types of network plug-ins CNI; creating a second vector Node resource corresponding to the second Node according to the Node IP, the Node sub-network segment, the Mac address and the CNI type, and storing the second vector Node resource; the names of the second Vector Node resources are respectively set as corresponding host names.

In this embodiment, a second heterogeneous CNI communication proxy in a second Node continuously monitors an update event of a Vector Node resource corresponding to a first Node in a cluster through kube-apis server according to a certain time, and the second heterogeneous CNI communication proxy responds to the update event of the first Vector Node resource of the first Node in a k8s cluster to obtain a pre-created second Vector Node resource corresponding to the second Node, and determines whether the types of network plug-ins CNIs corresponding to the first Node and the second Node are the same according to the target Vector Node resource and the second Vector Node resource; in some embodiments, the comparison result may be obtained by comparing the first CNI type in the target Vector Node resource with the second CNI type in the second Vector Node resource, and whether the CNI types are the same may be determined according to the comparison result, which is not limited to a specific implementation manner of determining whether the CNI types are the same.

S120, under the condition that the types of the network plug-in CNI are the same, configuring a communication information table between the first Node and the second Node according to the VTEP equipment which is pre-established in the second Node by the target Vector Node resource.

The communication information table refers to a communication correlation table configured between types of two different network plug-ins CNI of the first Node and the second Node; in this embodiment, the communication information table includes: address resolution protocol (Address Resolution Protocol, ARP) table, forwarding database (Forwarding Database, FDB) table, and routing rule table. The ARP table is a table in which IP addresses and corresponding physical Mac address mapping relationships are stored in one network device. The FDB records the mapping between Mac addresses and corresponding network interfaces (ports) for supporting packet forwarding and destination address learning in ethernet. The routing rule table may also be referred to as a Route table, which records how the path of a packet is transferred from a source host to a destination host.

In this embodiment, the VTEP device may be a physical switch or a software switch, which is responsible for encapsulating and decapsulating VXLAN messages. Each VTEP has two interfaces: one is a local interface responsible for the original ethernet frame reception and transmission, and the other is an IP interface responsible for VXLAN data frame reception and transmission. In this embodiment, VXLAN devices (e.g., vector 0) are configured to establish Virtual Local Area Networks (VLANs) between containers or virtual machines to enable them to communicate on different hosts. In this embodiment, the pre-creation of the VTEP device configuration may include: the set virtual extensible local area network ID and port number are the same, the set local IP is the Node IP of the corresponding Node, the state is set to UP, and the mac address of the VTEP device is obtained, and the VTEP device is created by using the manner of vxland (virtual extensible local area network ID) =1000, port (port) =1000, local IP (local IP) =node IP, the name is set to vector0, the device state is set to UP (container is in progress), and the mac address of the VTEP device is obtained.

In this embodiment, the type of the network plug-in CNI in the updated target Vector Node resource may be changed, that is, the type of the changed network plug-in CNI is different from the type of the original network plug-in CNI, and in the case that the type of the network plug-in CNI is different, the ARP table, the FDB table, and the routing rule table between the first Node and the second Node may be configured for the VTEP device created in advance in the second Node according to the target Vector Node resource; specifically, according to the first Node IP, the first CNI type, the mac address of the first VTEP device, and the first Pod network segment corresponding to the first Node, an ARP table, an FDB table, and a routing rule table between the first Node and the second Node may be configured for the VTEP device created in advance in the second Node. Of course, under the condition that the CNI types are the same, at least two node nodes in the k8s cluster are determined to be in a normal communication state, and no processing is performed.

S130, implementing VxLAN tunnel communication between the first Node and the second Node according to the communication information table and the VTEP equipment.

In this embodiment, vxLAN tunnel communication between the first Node and the second Node may be implemented according to the communication information table and the VTEP device, and in some embodiments, the VTEP MAC address of the VxLAN package corresponding to the first Node may be searched according to the ARP table; searching a VTEP equipment interface corresponding to the VTEP MAC address according to the FDB table; the transmitted message is imported to the VTEP equipment through a routing rule table, a VTEP equipment interface and a preconfigured IP tables packet sending rule, and the message is sent to the first Node through a VXLAN tunnel corresponding to the VTEP equipment, so that the VXLAN tunnel communication between the first Node and the second Node is realized.

In this embodiment, a VTEP device is created in each Node, where the device is used to configure a VxLAN tunnel between nodes, when two nodes use different CNIs, we configure a route of a target Node network segment on a Node, import a Pod network packet accessing the Node to the VTEP device and send the Pod network packet to the target Node through the VxLAN tunnel, so as to open the network of two Pod networks of different CNIs, and further ensure that normal access can be performed between the Pod of the new CNI and the Pod of the old CNI, and the service is not affected after the CNI is replaced.

In an embodiment, fig. 2 is a flowchart of another communication method of heterogeneous CNIs of a large-scale cloud platform according to an embodiment of the present invention, where, based on the above embodiments, whether types of CNIs corresponding to a first Node and a second Node are the same is determined according to a target Vector Node resource and a second Vector Node resource, and configuring a communication information table between the first Node and the second Node by the VTEP equipment pre-established in the second Node according to the target Vector Node resource, and realizing VxLAN tunnel communication between the first Node and the second Node according to the communication information table and the VTEP equipment.

As shown in fig. 2, the communication method of heterogeneous CNI of a large-scale cloud platform in this embodiment may specifically include the following steps:

s210, a second heterogeneous CNI communication agent in a second Node responds to an update event of a first Vector Node resource of a first Node in a k8S cluster, and acquires a target Vector Node resource after the first Vector Node resource is updated and a second Vector Node resource which is created in advance and corresponds to the second Node.

S220, extracting a first CNI type in the target Vector Node resource and a second CNI type in the second Vector Node resource.

The first CNI type refers to the CNI type of the first Node in the updated target Vector Node resource; the second CNI type refers to a second CNI type of the second Node.

In this embodiment, a second heterogeneous CNI communication proxy in a second Node responds to an update event of a first Vector Node resource of a first Node in a k8s cluster, acquires a second Vector Node resource created in advance corresponding to the second Node, extracts a corresponding second CNI type from the second Vector Node resource, and extracts a first CNI type in a target Vector Node resource.

S230, comparing the first CNI type with the second CNI type to obtain a comparison result, and determining whether the CNI network plug-in types are the same according to the comparison result.

In this embodiment, the first CNI type and the second CNI type are compared to obtain a comparison result, and whether the types of CNI network plug-ins are the same is determined according to the comparison result, and under the condition that the types of the network plug-ins CNI are different, the Pod CIDR-first IP is determined according to the first Node IP; wherein, pod CIDR-first IP is the first IP of Pod IP address field. It will be appreciated that knowing the first Node IP, the first IP of the Pod IP address field can be known, and as an example, the first Node IP is 10.224.0.0/24, and then the first IP of the Pod IP address field is 10.224.0.0/1.

S240, forming an ARP table according to the Pod CIDR-first IP and the Mac address of the first VTEP device.

In this embodiment, the ARP table may be formed according to the Pod CIDR-first IP and the Mac address of the first VTEP device, which may be understood that an ARP entry is manually added to the first VTEP device interface, and the Mac address and the first IP address of the Pod CIDR (IP address field) are written into the ARP table. Illustratively, ARP-i vector0-s < PodCIDR-first IP > < Vtep Mac >, an ARP entry is manually added to the vector0 interface. < podcddr-first IP > represents the first IP address of CIDR (IP address field) of Pod, and < Vtep Mac > represents the Mac address of VXLAN encapsulation. The purpose of this command is to tell the system that on the vector0 interface, the MAC address corresponding to the IP address < podcdr-first IP > is < Vtep MAC >.

S250, forming an FDB table according to the Mac address of the first VTEP device and the first Node IP.

In this embodiment, the FDB table may be formed according to the Mac address of the first VTEP device and the first nodeb, and illustratively, bridge FDB add < VTEP Mac > v vector0 dst < nodeb > self-agent is used to add an entry in the bridged FDB table, indicating that on the vector0 interface, a packet with the destination Mac address < VTEP Mac > should be forwarded to the local (self), and the entry is permanently validated, < nodeb > is the IP address of the Node.

S260, forming a routing rule table according to the first Pod network segment and the Pod CIDR-first IP.

In this embodiment, a routing rule table is formed according to the first Pod network segment and Pod CIDR-first IP, and an exemplary IP route add < Pod idr > via < Pod idr-first IP > dev vector0 onlink is added with a routing table entry indicating that when the target IP address in < Pod idr > is to be accessed, the data packet should pass through the vector0 interface, and specifies that the next hop address is < Pod idr-first IP >. The onlink option indicates that the next hop address is directly reachable without going through other routes.

S270, using the ARP table, the FDB table and the routing rule table as three communication information tables between the first Node and the first Node.

In this embodiment, the ARP table, the FDB table, and the routing rule table are used as three communication information tables between the first Node and the first Node, and by using these three communication information tables and establishing the VXLAN network, containers or virtual machines located on different hosts can communicate through the VXLAN tunnel.

S280, searching the VTEP MAC address of the VXLAN package corresponding to the first Node according to the ARP table.

In this embodiment, the VTEP MAC address of the VXLAN package corresponding to the first Node is searched according to the ARP table, which can be understood that when performing Pod communication, the VXLAN VTEP MAC address is first found according to ARP, that is, the MAC address of the first Node is found first, and the MAC address is sent through which interface of the current Node.

S290, searching a VTEP device interface corresponding to the VTEP MAC address according to the FDB table.

In this embodiment, the VTEP device interface corresponding to the VTEP MAC address is searched according to the FDB table, and the data packet (the message during communication) is sent to the VXLAN tunnel according to the routing segment.

S2100, importing the transmitted message to the VTEP equipment through a routing rule table, a VTEP equipment interface and a preconfigured IP tables packet sending rule, and sending the message to the first Node through a VXLAN tunnel corresponding to the VTEP equipment so as to realize the VXLAN tunnel communication between the first Node and the second Node.

In this embodiment, the transmitted message is imported to the VTEP device through the routing rule table, the VTEP device interface, and the preconfigured IP tables packet sending rule, and the message is sent to the first Node through the VXLAN tunnel corresponding to the VTEP device, so as to implement VXLAN tunnel communication between the first Node and the second Node. In this embodiment, by configuring ARP entries, FDB entries, and routing rules, normal communication in the VXLAN tunnel is ensured. When the Pod of the second node communicates with the Pod of the first node, and when it needs to communicate with a Pod in < podcddr >, it uses ARP entry to find the corresponding VXLAN VTEP MAC address, then uses FDB entry to find the correct interface vector0, and finally sends the data packet to the VXLAN tunnel through the routing rule.

In this embodiment, the configuration of the IP tables packet sending rule includes: IP tables rules are added in POSTROUTING chains in tables of network address translation NAT, the IP tables rules are the first IP of a source IP address converted into Pod CIRD by a source IP of a message sent from VTEP equipment, so that when cross-Node heterogeneous CNI communication is ensured, correct return is carried out according to the source IP of a request. Exemplary, iptables: the iptables-t nat-IPOSTROUTING 1-o vector0-j SNAT-to-source < PodCIDR-first IP >, ensures that the message source IP sent from vector0 device is both SNAT (source IP address translation) to PodCIRD, i.e., podCIDR-first IP, e.g., 10.224.0.1, ensuring that the message can be correctly returned according to the source IP of the request during subsequent cross-Node heterogeneous CNI communication.

According to the technical scheme of the embodiment of the invention, the first CNI type in the target Vector Node resource and the second CNI type in the second Vector Node resource are extracted, the first CNI type and the second CNI type are compared to obtain a comparison result, whether the types of CNI network plug-ins are the same is determined according to the comparison result, an ARP table is formed according to the Pod CIDR-first IP and the MAC address of the first VTEP device, an FDB table is formed according to the MAC address of the first VTEP device and the first Node IP, a routing rule table is formed according to the first Pod network segment and the Pod CIDR-first IP, the VTEP MAC address of the VXLAN package corresponding to the first Node is searched according to the ARP table, the VTEP device interface corresponding to the VTEP MAC address is searched according to the FDB table, the transmitted message is imported to the VTEP equipment through a routing rule table, a VTEP equipment interface and a preconfigured IP tables package sending rule, and is sent to the first Node through a VXLAN tunnel corresponding to the VTEP equipment, so that the VXLAN tunnel communication between the first Node and the second Node is realized, further, the Pod networks of two different types of CNIs can be opened, normal access between Pods of different types of CNIs is further ensured, the service is not affected after the CNIs are replaced, smooth transition is realized in the upgrading (replacing) process of the CNIs, the problem that a large number of Pods are restarted simultaneously to cause the service to be affected is solved, and the service in the whole cluster is ensured not to be affected.

For example, in order to better understand that VTEP devices are created in advance in each Node respectively, vector Node resources corresponding to the current Node are created in kube-apiserver in advance and named, and IP tables rules are configured in advance, fig. 3 is a schematic diagram of preparation of an environment before CNI replacement, which is provided in an embodiment of the present invention, as shown in fig. 3, in this embodiment, a K8s cluster includes a plurality of Node nodes and a control Node (master Node), where each Node includes a heterogeneous CNI communication agent (Vector-agent), and the master Node includes a kube-apiserver component;

after a1 and vector-agent are started, reading the hostname of the Node through a kube-apiserver, and the IP address of the Node and the sub-network segments distributed to the current Node, namely the Node Name, the Node IP and the PodCIRD.

a2, creating a VTEP device using vxland (virtual extensible local area network ID) =1000, port (port) =1000, localip (local IP) =node IP, setting the name to vector0, and setting the device state to UP (container in progress).

a3, acquiring the Mac address of the VTEP equipment, namely the VtepMac.

a4, reading the information of the configuration catalog/etc/CNI/net.d/of the CNI on the Node, and obtaining the CNI Type of the current Node, namely CNI Type.

a5, the acquired Node IP, vtepMac, CNI Type and PodCIDR, and establishing a Vectornode resource corresponding to the current Node through kube-apiserver, wherein the name of the resource is set as NodeName.

a6, adding IP tables rules in POSTROUTING chains in tables of NAT (Network Address Translation network address translation) by the vector-agent.

In an embodiment, in order to better understand a communication method of a large-scale cloud platform heterogeneous CNI, fig. 4 is a schematic diagram of a communication method of a large-scale cloud platform heterogeneous CNI according to an embodiment of the present invention. In this embodiment, the K8s cluster includes a plurality of Node nodes and a control Node (master Node), where each Node includes a heterogeneous CNI communication agent (Vector-agent), the master Node includes a kube-apiserver component, each Node respectively creates VTEP equipment in advance, creates Vector Node resources corresponding to the current Node in the kube-apiserver in advance, names the Vector Node resources, and configures IP tables in advance;

as shown in fig. 4, the communication method of heterogeneous CNI of a large-scale cloud platform in this embodiment specifically includes:

b1, the vector-agent in the first Node monitors whether the configuration catalog/etc/CNI/net.d of the CNI changes.

And b2, in the process of replacing the CNI, the file under the directory is changed, and after the file is changed, the Vector-agent of the Node for replacing the CNI acquires the latest CNI Type and updates the CNI Type field of the Vector Node resource corresponding to the Node for replacing the CNI.

b3, continuously monitoring update events of corresponding Vector Node resources of other nodes in the cluster by the Vector-agent in the second Node through the kube-api server, and judging whether the first CNI Type of the Vector Node of the first Node is identical to the second CNI Type of the Vector Node in the second Node after the CNI Type of the Vector Node of the first Node changes.

b4, if the first CNI Type and the second CNI Type are the same, not processing; if the first CNI Type and the second CNI Type are different, acquiring Node IP, vtep Mac and PodCIDR of a Vector Node of the first Node; the PodCIDR-first IP is obtained through the PodCIDR, and the configuration ARP table, the FDB table and the routing rule route table are allocated to the created device vector0 in the first Node by using the Node IP, the Vtep Mac and the PodCIDR-first IP.

b5, through the ARP table, the FDB table and the routing rule table, the VxLAN tunnel between the two Node nodes is automatically opened, and the Pods of the two different CNI nodes can be communicated with each other normally.

In the embodiment of the invention, a VTEP device is created in each Node, the device is used for configuring a VxLAN tunnel between the nodes, when two nodes use different CNIs, the Node with unchanged configuration catalog is configured with the route of the Node network segment with changed catalog, the Pod network message accessing the Node is imported into the VTEP device and is sent to the Node with changed catalog through the VxLAN tunnel, thereby the Pod networks of the two different CNIs are communicated, further ensuring that the Pod of the new CNI and the Pod of the old CNI can be normally accessed, and the service is not affected after the CNI is replaced.

In an embodiment, fig. 5 is a block diagram of a communication system of heterogeneous CNI of a large-scale cloud platform according to an embodiment of the present invention, where the system is applicable to a situation of communication pairs of heterogeneous network plugins generated after a large-scale cloud platform network plugin replacement upgrade, and the system may be implemented by hardware/software. The communication method of the large-scale cloud platform heterogeneous CNI can be configured in the electronic equipment to realize the communication method of the large-scale cloud platform heterogeneous CNI in the embodiment of the invention.

As shown in fig. 5, the communication system includes: a K8s cluster, wherein the K8s cluster comprises at least two Node nodes and a control Node, and the at least two Node nodes comprise a first Node and a second Node; the first Node comprises: a first heterogeneous CNI communication agent; the second Node includes: a second heterogeneous CNI communication agent; the control node comprises: kube-apiserver component;

The communication system of the large-scale cloud platform heterogeneous CNI provided by the embodiment of the invention can execute the communication processing method of the large-scale cloud platform heterogeneous CNI provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

In an embodiment, fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 6, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the respective methods and processes described above, such as a communication method of a large-scale cloud platform heterogeneous CNI.

In some embodiments, the communication processing method of the large-scale cloud platform heterogeneous CNI may be implemented as a computer program, which is tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the above-described communication method of large-scale cloud platform heterogeneous CNIs may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the communication method of the large-scale cloud platform heterogeneous CNI by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable large scale cloud platform heterogeneous CNI communication system, such that the computer programs, when executed by the processor, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. The communication method of the heterogeneous CNI of the large-scale cloud platform is characterized by being applied to a second Node in a k8s cluster; the second Node is Node with unchanged configuration catalog; the method comprises the following steps:

2. The method of claim 1, wherein the K8s cluster comprises at least two Node nodes and one control Node; the control node comprises a kube-apiserver; the first Node also creates VTEP equipment in advance;

the virtual extensible local area network ID and the port number set by the pre-created VTEP equipment in the second Node are the same, the set local IP is the Node IP of the corresponding Node, and the state is set to be UP.

3. The method according to any one of claims 1 or 2, wherein the pre-creation of the first Vector Node resource and the second Vector Node resource comprises:

after the heterogeneous CNI communication agents respectively corresponding to the first Node and the second Node are started, reading the host names, node IPs and sub-network segments of the nodes respectively corresponding to the first Node and the second Node through kube-apiserver;

Obtaining mac addresses corresponding to the VTEP equipment in the first Node and the second Node;

respectively reading configuration catalogues of network plug-in CNI (network plug-in) on the first Node and the second Node; the configuration catalog comprises the type of a network plug-in CNI;

creating a first Vector Node resource and a second Vector Node resource corresponding to the first Node and the second Node respectively according to the Node IP, the sub-network segment of the Node, the mac address and the type of the network plug-in CNI, and storing the first Vector Node resource and the second Vector Node resource respectively;

the names of the first Vector Node resource and the second Vector Node resource are respectively set as corresponding host names.

4. The method of claim 1, wherein the target Vector Node resources comprise: the first Node comprises a first Node IP, a first CNI type, a Mac address of first VTEP equipment and a first Pod network segment corresponding to the first Node; the second Vector Node resource includes: and the second Node IP, the second CNI type, the mac address of the second VTEP equipment and the second Pod network segment corresponding to the second Node.

5. The method of claim 4, wherein the determining whether the types of network plug-ins CNIs corresponding to the first Node and the second Node are the same according to the target Vector Node resource and the second Vector Node resource comprises:

Extracting a first CNI type in the target Vector Node resource and a second CNI type in the second Vector Node resource;

and comparing the first CNI type with the second CNI type to obtain a comparison result, and determining whether the types of CNI network plug-ins are the same according to the comparison result.

6. The method of claim 4, wherein the communication table comprises: address resolution protocol ARP table, forwarding database FDB table, and routing rule table;

correspondingly, the configuring the communication information table between the first Node and the second Node for the VTEP device created in advance in the second Node according to the target Vector Node resource includes:

determining a Pod CIDR-first IP according to the first Node IP; wherein, the Pod CIDR-first IP is the first IP of the Pod IP address field;

forming an address resolution protocol ARP table according to the Pod CIDR-first IP and the mac address of the first VTEP device;

forming a forwarding database FDB table according to the mac address of the first VTEP device and the first Node IP;

forming the routing rule table according to the first Pod network segment and the Pod CIDR-first IP;

and taking the address resolution protocol ARP table, the forwarding database FDB table and the routing rule table as three communication information tables between the first Node and the first Node.

7. The method of claim 6, wherein said implementing VxLAN tunneling between the first Node and the second Node in accordance with the communication information table and the VTEP device comprises:

searching a VTEP MAC address of the VXLAN encapsulation corresponding to the first Node according to the address resolution protocol ARP table;

searching a VTEP equipment interface corresponding to the VTEP MAC address according to the forwarding database FDB table;

importing a transmitted message to the VTEP equipment through the routing rule table, a VTEP equipment interface and the preconfigured IP tables packet sending rule, and sending the message to the first Node through a VXLAN tunnel corresponding to the VTEP equipment so as to realize the VXLAN tunnel communication between the first Node and the second Node;

the configuration of the IP tables package rule comprises the following steps: and adding an IP (Internet protocol) table rule in a POSTROUTING chain in a table of the network address translation NAT, wherein the IP table rule is the first IP of a Pod CIRD converted from a source IP address of a message sent from the VTEP equipment so as to ensure that the message is correctly returned according to the source IP of a request when the trans-Node heterogeneous CNI is communicated.

8. A communication system of heterogeneous CNIs of a large-scale cloud platform, the communication system comprising: a K8s cluster, wherein the K8s cluster comprises at least two Node nodes and a control Node, and the at least two Node nodes comprise a first Node and a second Node; the first Node comprises: a first Node; the second Node includes: a second heterogeneous CNI communication agent; the control node comprises: kube-apiserver component;

9. An electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the communication method of large-scale cloud platform heterogeneous CNI according to any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that the computer readable storage medium stores computer instructions for causing a processor to execute a communication method of heterogeneous CNI of a large-scale cloud platform according to any of claims 1 to 7.