CN115550220A - SDN cluster escape method, device and storage medium based on Openstack - Google Patents

SDN cluster escape method, device and storage medium based on Openstack Download PDF

Info

Publication number
CN115550220A
CN115550220A CN202211154940.7A CN202211154940A CN115550220A CN 115550220 A CN115550220 A CN 115550220A CN 202211154940 A CN202211154940 A CN 202211154940A CN 115550220 A CN115550220 A CN 115550220A
Authority
CN
China
Prior art keywords
sdn controller
node
virtual machine
cluster
sdn
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211154940.7A
Other languages
Chinese (zh)
Inventor
丁宇桐
王生银
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Cisco Networking Technology Co Ltd
Original Assignee
Inspur Cisco Networking Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Cisco Networking Technology Co Ltd filed Critical Inspur Cisco Networking Technology Co Ltd
Priority to CN202211154940.7A priority Critical patent/CN115550220A/en
Publication of CN115550220A publication Critical patent/CN115550220A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/10Active monitoring, e.g. heartbeat, ping or trace-route
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45562Creating, deleting, cloning virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The application discloses an SDN cluster escape method, equipment and a storage medium based on Openstack. The method comprises the following steps: deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; under the condition that the SDN controller cluster works and operates, the SDN controller main node is connected with a network through nodes and detects the connection state of SDN controller slave nodes to determine whether fault nodes occur or not; under the condition that a fault node is determined, the SDN controller main node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to the OpenStack cloud computing management platform so that the OpenStack cloud computing management platform creates a virtual machine to be applied; after the virtual machine to be applied is created, the SDN controller master node executes a node creation program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into an SDN controller cluster. The method improves the stability of the SDN cluster.

Description

SDN cluster escape method, device and storage medium based on Openstack
Technical Field
The application relates to the technical field of SDN (software defined networking), in particular to an SDN cluster escape method, device and storage medium based on Openstack.
Background
With the increasing relevance of each industry and the internet in recent years, more and more enterprises select service cloud, and build a private cloud environment locally, so that the security risk of placing core services in public cloud is reduced while the resource utilization rate of the data center is ensured. Meanwhile, with the introduction of cloud, many manufacturers can deploy an SDN (software defined network) controller in a data center networking in a matching manner, and realize automatic operation, maintenance and control of the data center network through cooperative work of the SDN controller and a private cloud platform, so that labor and practice cost is saved, and technical requirements on operation and maintenance personnel are also reduced.
Software SDN controllers can be deployed in a cloud platform, server resources are saved, and the characteristics of virtual machine drift of the cloud platform are also utilized, so that the software SDN controllers are more safe and reliable compared with physical server deployment. However, in the scheme, if the cloud platform network fails and affects the management network, the SDN controller is difficult to repair or regulate the network, and at this time, operation and maintenance personnel cannot perform network troubleshooting work by using information acquired by the SDN controller, which affects the network stability of the data center. Therefore, how to improve the stability of the SDN cluster becomes an urgent problem to be solved for a data center adopting a scheme of matching the SDN with a cloud computing platform.
Disclosure of Invention
The embodiment of the application provides an SDN cluster escape method, device and storage medium based on Openstack, and aims to solve the following technical problems: for a data center adopting a scheme of matching an SDN with a cloud computing platform, how to improve the stability of an SDN cluster is provided.
In a first aspect, an embodiment of the present application provides an SDN cluster escape method based on Openstack, where the method includes: deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for realizing connection between a plurality of SDN controller slave nodes and an SDN controller master node, and comprises the following steps: a heartbeat detection network and a cluster management network; under the condition that the SDN controller cluster works and runs, the SDN controller main node is connected with a network through nodes and detects the connection state of a plurality of SDN controller slave nodes so as to determine whether fault nodes occur in the SDN controller slave nodes or not; under the condition that a fault node occurs in a plurality of SDN controller slave nodes, an SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction; after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creating program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into an SDN controller cluster.
In an implementation manner of the present application, deploying an SDN controller cluster in a cloud platform, and configuring a node for the SDN controller cluster to connect to a network specifically includes: based on preset virtual machine creation rules, creating a primary virtual machine and a plurality of secondary virtual machines, and installing SDN applications in the primary virtual machine and the plurality of secondary virtual machines to obtain an SDN controller cluster; configuring a cluster management network of the SDN controller cluster to enable the SDN controller cluster to access a management network of the cloud platform and enable a plurality of SDN controller slave nodes to be connected with the SDN controller master node through the cluster management network; configuring a heartbeat detection network and enabling a number of SDN controller slave nodes to connect with an SDN controller master node through the heartbeat detection network.
In an implementation manner of the present application, an SDN controller master node connects to a network through a node, and performs connection state detection on a plurality of SDN controller slave nodes, specifically including: the method comprises the steps that an SDN controller master node sends periodic icmp messages to each SDN controller slave node in an SDN controller cluster through a heartbeat detection network, and confirms the connection state of a target SDN controller slave node according to whether a reply message returned by the corresponding target SDN controller slave node is received; under the condition that the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a first preset time interval, the SDN controller master node marks the target SDN controller slave node in an off-connection state and keeps sending a periodic icmp message to the target SDN controller slave node; under the condition that the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a second preset time interval, the SDN controller master node marks the target SDN controller slave node as a check state and sends a periodic icmp message to the target SDN controller slave node through a cluster management network; under the condition that the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a third preset time interval, the SDN controller master node marks the target SDN controller slave node as an offline state and determines that the target SDN controller slave node is a fault node.
In an implementation manner of the present application, the constructing a virtual machine creation instruction by an SDN controller master node specifically includes: reading the upper limit number of the configurable escape virtual machines and the number of the configured escape virtual machines in a preset fault escape resource pool; the fault escape resource pool is used for providing configuration information for constructing the virtual machine; under the condition that the upper limit number of the configurable escape virtual machines is larger than the number of the configured escape virtual machines, generating to-be-configured information of the to-be-applied virtual machines based on a preset configuration information generation rule; wherein the information to be configured at least comprises: managing network addresses and heartbeat detection network addresses; and constructing a virtual machine creating instruction based on the information to be configured of the virtual machine to be applied.
In an implementation manner of the application, after the OpenStack cloud computing management platform creates the virtual machine to be applied based on the virtual machine creation instruction, the method further includes: the SDN controller main node adds the virtual machine to be applied to a cluster management network based on the management network address and adds the virtual machine to be applied to a heartbeat detection network based on the heartbeat detection network address; the SDN controller main node sends a periodic icmp message to the virtual machine to be applied through the cluster management network, marks the virtual machine to be applied in an available state under the condition that a reply message returned by the virtual machine to be applied is received within a fourth preset time interval, and sends the periodic icmp message to the virtual machine to be applied through the heartbeat detection network; and under the condition that a reply message returned by the virtual machine to be applied is received within a fifth preset time interval, marking the virtual machine to be applied as a deployment state.
In one implementation of the present application, the method further comprises: under the condition that a reply message returned by the virtual machine to be applied is not received within a fourth preset time interval or a fifth time interval, the SDN controller main node marks the virtual machine to be applied as a node application failure, and the OpenStack cloud computing management platform is enabled to establish the virtual machine to be applied again; and under the condition that the re-creation fails, continuously retrying the creation until the number of the constructed escape virtual machines reaches the upper limit number of the constructed escape virtual machines.
In one implementation manner of the present application, after marking the virtual machine to be applied as a deployment state, the method further includes: the method comprises the steps that an SDN controller main node forbids access permission of a fault node based on a preset escape rule; and sending an instruction for deleting the relevant information of the fault node to other SDN controllers from the nodes through the cluster management network.
In an implementation manner of the present application, an SDN controller master node executes a node creation program on a virtual machine to be applied, which specifically includes: based on a preset first data transmission mode, the SDN controller main node sends SDN application to the virtual machine to be applied through a cluster management network; after the SDN application transmission is completed, based on a preset second data transmission mode, the SDN controller main node sends a deployment instruction to the virtual machine to be applied through the cluster management network; and the virtual machine to be applied installs the SDN application based on the deployment instruction so as to obtain a newly added SDN controller slave node, and sends deployment completion confirmation information to the SDN controller master node.
In a second aspect, an embodiment of the present application further provides an SDN cluster escape device based on an Openstack, where the device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to: deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for realizing connection between a plurality of SDN controller slave nodes and an SDN controller master node, and comprises: a heartbeat detection network and a cluster management network; under the condition that the SDN controller cluster works and operates, the SDN controller main node is connected with a network through nodes, and connection state detection is carried out on a plurality of SDN controller slave nodes so as to determine whether fault nodes occur in the SDN controller slave nodes; under the condition that a fault node occurs in a plurality of SDN controller slave nodes, an SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction; after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creating program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into an SDN controller cluster.
In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium for an SDN cluster escape based on an Openstack, where computer-executable instructions are stored, and the computer-executable instructions are configured to: deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for realizing connection between a plurality of SDN controller slave nodes and an SDN controller master node, and comprises the following steps: a heartbeat detection network and a cluster management network; under the condition that the SDN controller cluster works and operates, the SDN controller main node is connected with a network through nodes, and connection state detection is carried out on a plurality of SDN controller slave nodes so as to determine whether fault nodes occur in the SDN controller slave nodes; under the condition that a fault node occurs in a plurality of SDN controller slave nodes, an SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction; after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creating program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into an SDN controller cluster.
According to the SDN cluster escape method, device and storage medium based on the Openstack, the SDN controller cluster can detect abnormality of the SDN controller cluster in time in the scene of network faults or certain node breakdown and the like, and tries to deploy a newly-built node in a cloud platform in a mode of building the node on the cloud platform according to virtual machine management characteristics of the cloud platform, the newly-built node is deployed in a cloud platform computing node in a normal state, the state of the SDN controller cluster is ensured to be stable as much as possible, the safety of current service data is ensured, the SDN controller cluster can still regulate and control a data center network in a risk scene, and the network stability and reliability are enhanced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a flowchart of an SDN cluster escape method based on Openstack according to an embodiment of the present application;
fig. 2 is a schematic view of an internal structure of an SDN cluster escape device based on an Openstack according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides an SDN cluster escape method, device and storage medium based on Openstack, and aims to solve the following technical problems: for a data center adopting a scheme of matching an SDN with a cloud computing platform, how to improve the stability of an SDN cluster is provided.
The technical solutions proposed in the embodiments of the present application are explained in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of an SDN cluster escape method based on Openstack according to an embodiment of the present application. As shown in fig. 1, an SDN cluster escape method based on Openstack provided in an embodiment of the present application specifically includes the following steps:
step 101, deploying an SDN controller cluster in a cloud platform, and configuring nodes of the SDN controller cluster to connect with a network.
Firstly, it needs to be described that an SDN controller cluster in the embodiment of the present application includes an SDN controller master node and a plurality of SDN controller slave nodes; the node connection network is used for realizing connection between a plurality of SDN controller slave nodes and an SDN controller master node, and comprises: a heartbeat detection network and a cluster management network.
It is understood that before deploying the SDN controller cluster in the cloud platform, the cloud platform and the data center should have completed deployment first. The deployed cloud platform and the deployed data center have online conditions required by a service production environment, namely, a complete management network is provided.
In an embodiment of the present application, deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster specifically includes: the method comprises the steps of creating a main virtual machine and a plurality of auxiliary virtual machines based on preset virtual machine creating rules, and installing SDN applications in the main virtual machine and the auxiliary virtual machines to obtain an SDN controller cluster.
It can be understood that the configuration of the virtual machine requires configuring its specification, and the specification type at least includes parameters such as CPU, memory, hard disk, and the like. The specification of the master virtual machine and the specification of the slave virtual machines are not limited herein, and the specifications of the master virtual machine and the slave virtual machines may be the same or different, so that the master virtual machine and the slave virtual machines may be created according to a preset virtual machine creation rule, and the master virtual machine and the slave virtual machines may be created according to a corresponding creation rule determined according to a requirement without being limited herein.
Further, allocating management network addresses to the master virtual machine and the plurality of slave virtual machines to configure a cluster management network of the SDN controller cluster, to enable the SDN controller cluster to access the management network of the cloud platform, and to enable the plurality of SDN controller slave nodes to connect with the SDN controller master node through the cluster management network; allocating heartbeat detection network addresses for the main virtual machine and the plurality of slave virtual machines to configure a heartbeat detection network, and enabling the plurality of SDN controller slave nodes to be connected with the SDN controller master node through the heartbeat detection network.
It can be understood that the SDN controller is responsible for controlling various forwarding rules, and therefore, after the SDN controller cluster is deployed, various devices in the network, such as servers, switches, firewalls, and the like, need to be managed.
Specifically, management information of various network devices, including management network IPs, management user names and passwords, related configurations of SNMP and the like of various devices that can be reached by the SDN controller, can be verified in a mode such as SSH or SNMP, the device information is stored in a database of the SDN controller cluster after the verification information is correct, and in a subsequent use process, the SDN controller performs network regulation and control on a network according to the obtained device management information, and performs operations such as state monitoring and configuration management on the devices.
In an embodiment of the present application, it is further required to configure a fault escape resource pool for providing configuration information for constructing a virtual machine when a fault node is detected. The fault escape resource pool comprises information such as the upper limit number (the maximum number of nodes allowed to be newly built) of the escape virtual machines, specifications of the newly built nodes (parameter selection such as a CPU (central processing unit), a memory and a hard disk for applying for creating the nodes), a newly built node management network pool (for providing a management network address), a newly built node heartbeat detection network pool (for providing a heartbeat detection network address), a newly built node user name password, a newly built node protocol type and the like. After configuration is completed, when a fault escape mechanism is triggered, the SDN controller main node constructs the virtual machine creation instruction according to the information in the fault escape resource pool.
102, under the condition that the SDN controller cluster works and runs, the SDN controller master node is connected with a network through nodes and detects the connection state of a plurality of SDN controller slave nodes so as to determine whether fault nodes exist in the SDN controller slave nodes.
In an embodiment of the application, when an SDN controller cluster operates, an SDN controller master node is used as a source, a network is connected through nodes, a plurality of SDN controller slave nodes in the cluster send periodic icmp messages, and current network connection states of the SDN controller slave nodes are confirmed according to whether reply messages are received or not. In order to ensure the accuracy of heartbeat detection, the embodiment of the application uses two different networks, namely a management network and a heartbeat detection network, to perform detection simultaneously.
Specifically, the SDN controller master node sends a periodic icmp message to each SDN controller slave node in the SDN controller cluster through a heartbeat detection network, and confirms a connection state of a target SDN controller slave node according to whether a reply message returned by the corresponding target SDN controller slave node is received.
Further, when the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after the first preset time interval, the SDN controller master node marks the target SDN controller slave node in an off-connection state, and keeps sending a periodic icmp message to the target SDN controller slave node.
Further, when the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a second preset time interval, the SDN controller master node marks the target SDN controller slave node as a check state, and sends a periodic icmp message to the target SDN controller slave node through the cluster management network.
Further, when the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a third preset time interval, the SDN controller master node marks the target SDN controller slave node as an offline state and determines that the target SDN controller slave node is a fault node.
For example: the method comprises the steps that the SDN controller master node periodically sends a group of icmp messages with an interval of 3 seconds accounting for 10 icmp messages with a period of 30 seconds as a period so as to verify whether the network state from the SDN controller master node to the SDN controller slave node is normal or not. If the icmp message in a period does not receive a reply message sent by a certain SDN controller slave node, the SDN controller master node marks the node as an off-connection state. For the SDN controller slave nodes in the disconnection state, the SDN controller master nodes maintain icmp message sending. And if the reply message from the node is not received in ten periods, namely 300 seconds in total, marking the node as the check state. For the SDN controller slave node in the checking state, the SDN controller master node is converted into sending icmp messages to the management network address, similarly, a group of icmp messages with 3 seconds interval is periodically sent by taking 30 seconds as a period, if the management network does not receive a reply message from the node in ten periods, namely 300 seconds, the node is marked as an off-line state, and the node is judged as a fault node.
103, under the condition that a fault node occurs in a plurality of SDN controller slave nodes, the SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to the OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction.
In one embodiment of the application, in the case that a fault node occurs in a plurality of SDN controller slave nodes, an escape mechanism is triggered, an SDN controller master node builds a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform in a command line manner,
in an embodiment of the present application, the method for constructing a virtual machine creation instruction by an SDN controller master node specifically includes: reading the upper limit number of the configurable escape virtual machines and the number of the configured escape virtual machines in a preset fault escape resource pool; under the condition that the upper limit number of the configurable escape virtual machines is larger than the number of the configured escape virtual machines, generating to-be-configured information of the to-be-applied virtual machines based on a preset configuration information generation rule; wherein the information to be configured at least comprises: managing network addresses and heartbeat detection network addresses; and constructing a virtual machine creating instruction based on the information to be configured of the virtual machine to be applied.
In an embodiment of the application, after the OpenStack cloud computing management platform creates the virtual machine to be applied based on the virtual machine creation instruction, the SDN controller master node adds the virtual machine to be applied to the cluster management network based on the management network address, and adds the virtual machine to be applied to the heartbeat detection network based on the heartbeat detection network address.
Further, the SDN controller master node sends a periodic icmp message to the virtual machine to be applied through the cluster management network, marks the virtual machine to be applied in an available state under the condition that a reply message returned by the virtual machine to be applied is received within a fourth preset time interval, and sends the periodic icmp message to the virtual machine to be applied through the heartbeat detection network.
Further, under the condition that a reply message returned by the virtual machine to be applied is received within a fifth preset time interval, the virtual machine to be applied is marked as a deployment state.
In an embodiment of the application, under the condition that a reply message returned by a virtual machine to be applied is not received within a fourth preset time interval or within a fifth time interval, a primary node of an SDN controller marks the virtual machine to be applied as a node application failure, and causes an OpenStack cloud computing management platform to create the virtual machine to be applied again; and under the condition that the re-creation fails, continuously retrying the creation until the number of the constructed escape virtual machines reaches the upper limit number of the constructed escape virtual machines.
In an embodiment of the application, after the virtual machine to be applied is marked as a deployment state, the SDN controller master node also disables the access right of the failed node through a preset ACL rule or a system firewall configuration mode; in addition, the SDN controller master node sends an instruction for deleting the relevant information of the fault node to other SDN controller slave nodes through the cluster management network so as to remove the fault node in the SDN controller cluster and ensure that the fault node cannot issue a command to the current data center equipment.
And step 104, after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creation program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into the SDN controller cluster.
In an embodiment of the application, after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creation program on the virtual machine to be applied.
Specifically, based on a preset first data transmission mode, the SDN controller master node sends the SDN application to the virtual machine to be applied through a cluster management network. Optionally, the first data transmission manner includes: in the SCP or FTP mode, the method comprises the steps of,
further, after the SDN application transmission is completed, based on a preset second data transmission mode, the SDN controller master node sends a deployment instruction to the virtual machine to be applied through the cluster management network. Optionally, the second data transmission method includes: ssh approach.
Further, the virtual machine to be applied installs the SDN application based on the deployment instruction so as to obtain a newly added SDN controller slave node, and sends deployment completion confirmation information to the SDN controller master node.
Further, after receiving the deployment completion confirmation information, the SDN controller master node marks the newly added SDN controller slave node as an online state.
In an embodiment of the application, after the new slave node of the SDN controller is obtained, the new slave node of the SDN controller is subjected to data synchronization, so that the new slave node of the SDN controller joins the SDN controller cluster.
In an embodiment of the application, each SDN controller slave node in the SDN controller cluster may further monitor a program state of the SDN controller slave node, when a scene such as memory leak occurs, each SDN controller slave node sends alarm information to the SDN controller master node, a fault escape mechanism is actively triggered, at this time, the SDN controller master node directly enters a step of creating a new node, the SDN controller slave node sending the alarm information is removed from the SDN controller cluster, and an access right of a device in the environment is adjusted.
Through the method, the SDN controller cluster can detect the abnormality of the SDN controller cluster in time in the scenes of network faults or certain node breakdown and the like, and tries to deploy the newly-built node in the cloud platform computing node with normal state in the mode of newly-built node on the cloud platform by depending on the virtual machine management characteristics of the cloud platform, so that the state stability of the SDN controller cluster is ensured as far as possible, the safety of current service data is ensured, the SDN controller cluster can still regulate and control a data center network in a risk scene, and the network stability and reliability are enhanced.
The above is the method embodiment proposed by the present application. Based on the same inventive concept, the embodiment of the application further provides an SDN cluster escape device based on Openstack, and the structure of the SDN cluster escape device is shown in fig. 2.
Fig. 2 is a schematic view of an internal structure of an SDN cluster escape device based on an Openstack according to an embodiment of the present application. As shown in fig. 2, the apparatus includes:
at least one processor 201;
and a memory 202 communicatively coupled to the at least one processor;
wherein the memory 202 stores instructions executable by the at least one processor, the instructions being executable by the at least one processor 201 to enable the at least one processor 201 to:
deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for realizing connection between a plurality of SDN controller slave nodes and an SDN controller master node, and comprises: a heartbeat detection network and a cluster management network;
under the condition that the SDN controller cluster works and runs, the SDN controller main node is connected with a network through nodes and detects the connection state of a plurality of SDN controller slave nodes so as to determine whether fault nodes occur in the SDN controller slave nodes or not;
under the condition that a fault node occurs in a plurality of SDN controller slave nodes, an SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction;
after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creating program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into an SDN controller cluster.
Some embodiments of the present application provide a non-volatile computer storage medium corresponding to an Openstack-based SDN cluster escape in fig. 1, storing computer-executable instructions configured to:
deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for realizing connection between a plurality of SDN controller slave nodes and an SDN controller master node, and comprises the following steps: a heartbeat detection network and a cluster management network;
under the condition that the SDN controller cluster works and operates, the SDN controller main node is connected with a network through nodes, and connection state detection is carried out on a plurality of SDN controller slave nodes so as to determine whether fault nodes occur in the SDN controller slave nodes;
under the condition that a fault node occurs in a plurality of SDN controller slave nodes, an SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction;
after the OpenStack cloud computing management platform creates the virtual machine to be applied, the SDN controller master node executes a node creating program on the virtual machine to be applied to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node to enable the newly added SDN controller slave node to be added into an SDN controller cluster.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on differences from other embodiments. In particular, for the embodiments of the internet of things device and medium, since they are substantially similar to the method embodiments, the description is simple, and reference may be made to the partial description of the method embodiments for relevant points.
The system and the medium provided by the embodiment of the application correspond to the method one to one, so the system and the medium also have the beneficial technical effects similar to the corresponding method.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises that element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims (10)

1. An SDN cluster escape method based on Openstack, which is characterized by comprising the following steps:
deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for enabling the SDN controller slave nodes to be connected with the SDN controller master node, and comprises the following steps: a heartbeat detection network and a cluster management network;
under the condition that the SDN controller cluster works and runs, an SDN controller main node connects a network through the nodes and performs connection state detection on the SDN controller slave nodes to determine whether a fault node exists in the SDN controller slave nodes;
under the condition that a fault node occurs in the SDN controller slave nodes, the SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction;
after the OpenStack cloud computing management platform creates a virtual machine to be applied, the SDN controller master node executes a node creation program on the virtual machine to be applied so as to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node so as to enable the newly added SDN controller slave node to join the SDN controller cluster.
2. The Openstack-based SDN cluster escape method according to claim 1, wherein an SDN controller cluster is deployed in a cloud platform, and a node connection network is configured for the SDN controller cluster, and specifically comprises:
creating a main virtual machine and a plurality of auxiliary virtual machines based on preset virtual machine creating rules, and installing SDN applications in the main virtual machine and the auxiliary virtual machines to obtain an SDN controller cluster;
configuring a cluster management network of the SDN controller cluster to enable the SDN controller cluster to access a management network of the cloud platform and to enable the number of SDN controller slave nodes to connect with the SDN controller master node through the cluster management network;
configuring a heartbeat detection network and enabling the SDN controller slave nodes to connect with the SDN controller master node through the heartbeat detection network.
3. The Openstack-based SDN cluster escape method according to claim 1, wherein the SDN controller master node performs connection state detection on the SDN controller slave nodes through the node connection network, specifically including:
the method comprises the steps that an SDN controller master node sends periodic icmp messages to each SDN controller slave node in an SDN controller cluster through a heartbeat detection network, and confirms the connection state of a target SDN controller slave node according to whether a reply message returned by the corresponding target SDN controller slave node is received or not;
under the condition that the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a first preset time interval, the SDN controller master node marks the target SDN controller slave node in a disconnection state and keeps sending periodic icmp messages to the target SDN controller slave node;
under the condition that the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a second preset time interval, the SDN controller master node marks the target SDN controller slave node as a check state and sends a periodic icmp message to the target SDN controller slave node through a cluster management network;
when the SDN controller master node still does not receive a reply message returned by the target SDN controller slave node after a third preset time interval, the SDN controller master node marks the target SDN controller slave node as an offline state and determines that the target SDN controller slave node is a fault node.
4. The Openstack-based SDN cluster escape method according to claim 1, wherein the SDN controller master node constructs a virtual machine creation instruction, specifically comprising:
reading the upper limit number of the configurable escape virtual machines and the number of the configured escape virtual machines in a preset fault escape resource pool; the fault escape resource pool is used for providing configuration information for constructing a virtual machine;
under the condition that the upper limit number of the configurable escape virtual machines is larger than the number of the configured escape virtual machines, generating to-be-configured information of the to-be-applied virtual machines based on a preset configuration information generation rule; wherein the information to be configured at least comprises: managing network addresses and heartbeat detection network addresses;
and constructing the virtual machine creating instruction based on the information to be configured of the virtual machine to be applied.
5. The Openstack-based SDN cluster escape method according to claim 4, wherein after the Openstack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creation instruction, the method further comprises:
the SDN controller master node adds the virtual machine to be applied to the cluster management network based on the management network address and adds the virtual machine to be applied to the heartbeat detection network based on the heartbeat detection network address;
the SDN controller main node sends a periodic icmp message to the virtual machine to be applied through a cluster management network, marks the virtual machine to be applied in an available state under the condition that a reply message returned by the virtual machine to be applied is received within a fourth preset time interval, and sends the periodic icmp message to the virtual machine to be applied through the heartbeat detection network;
and under the condition that a reply message returned by the virtual machine to be applied is received within a fifth preset time interval, marking the virtual machine to be applied as a deployment state.
6. The Openstack-based SDN cluster escape method according to claim 5, further comprising:
under the condition that a reply message returned by the virtual machine to be applied is not received within a fourth preset time interval or a fifth time interval, the SDN controller main node marks the virtual machine to be applied as a node application failure, and enables the OpenStack cloud computing management platform to establish the virtual machine to be applied again;
and under the condition of failure in recreating, continuously retrying the creation until the number of the constructed escape virtual machines reaches the upper limit number of the constructable escape virtual machines.
7. The Openstack-based SDN cluster escape method according to claim 5, wherein after the virtual machine to be applied is marked as a deployed state, the method further comprises:
the SDN controller main node forbids the access authority of the fault node based on a preset escape rule; and (c) a second step of,
sending, by the cluster management network, an instruction to delete the information related to the failed node to the other SDN controller slave nodes.
8. The Openstack-based SDN cluster escape method according to claim 1, wherein the SDN controller master node executes a node creation program on the virtual machine to be applied, and specifically includes:
based on a preset first data transmission mode, the SDN controller main node sends SDN application to the virtual machine to be applied through the cluster management network;
after the SDN application is transmitted, based on a preset second data transmission mode, the SDN controller main node sends a deployment instruction to the virtual machine to be applied through the cluster management network;
the virtual machine to be applied installs the SDN application based on the deployment instruction to obtain a newly added SDN controller slave node, and sends deployment completion confirmation information to the SDN controller master node.
9. An Openstack-based SDN cluster escape device, comprising:
at least one processor;
and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to:
deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for enabling the SDN controller slave nodes to be connected with the SDN controller master node, and comprises the following steps: a heartbeat detection network and a cluster management network;
under the condition that the SDN controller cluster works and runs, an SDN controller main node connects a network through the nodes and performs connection state detection on the SDN controller slave nodes to determine whether a fault node exists in the SDN controller slave nodes;
under the condition that a fault node occurs in the plurality of SDN controller slave nodes, the SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction;
after the OpenStack cloud computing management platform creates a virtual machine to be applied, the SDN controller master node executes a node creation program on the virtual machine to be applied so as to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node so as to enable the newly added SDN controller slave node to join the SDN controller cluster. .
10. A non-transitory computer storage medium storing computer-executable instructions for Openstack-based SDN cluster escape, the computer-executable instructions configured to:
deploying an SDN controller cluster in a cloud platform, and configuring a node connection network for the SDN controller cluster; the SDN controller cluster comprises an SDN controller main node and a plurality of SDN controller slave nodes; the node connection network is used for enabling the SDN controller slave nodes to be connected with the SDN controller master node, and comprises the following steps: a heartbeat detection network and a cluster management network;
under the condition that the SDN controller cluster works and runs, an SDN controller main node connects a network through the nodes and detects the connection state of the SDN controller slave nodes so as to determine whether a fault node occurs in the SDN controller slave nodes;
under the condition that a fault node occurs in the plurality of SDN controller slave nodes, the SDN controller master node constructs a virtual machine creating instruction and sends the virtual machine creating instruction to an OpenStack cloud computing management platform, so that the OpenStack cloud computing management platform creates a virtual machine to be applied based on the virtual machine creating instruction;
after the OpenStack cloud computing management platform creates a virtual machine to be applied, the SDN controller master node executes a node creation program on the virtual machine to be applied so as to obtain a newly added SDN controller slave node, and performs data synchronization on the newly added SDN controller slave node so as to enable the newly added SDN controller slave node to join the SDN controller cluster.
CN202211154940.7A 2022-09-21 2022-09-21 SDN cluster escape method, device and storage medium based on Openstack Pending CN115550220A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211154940.7A CN115550220A (en) 2022-09-21 2022-09-21 SDN cluster escape method, device and storage medium based on Openstack

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211154940.7A CN115550220A (en) 2022-09-21 2022-09-21 SDN cluster escape method, device and storage medium based on Openstack

Publications (1)

Publication Number Publication Date
CN115550220A true CN115550220A (en) 2022-12-30

Family

ID=84729388

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211154940.7A Pending CN115550220A (en) 2022-09-21 2022-09-21 SDN cluster escape method, device and storage medium based on Openstack

Country Status (1)

Country Link
CN (1) CN115550220A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105610614A (en) * 2015-12-28 2016-05-25 中国银联股份有限公司 High availability access system and high availability fault switching method
US20160226701A1 (en) * 2015-01-29 2016-08-04 Futurewei Technologies, Inc. Capacity-Aware Heuristic Approach for Placing Software-Defined Networking (SDN) Switches in Hybrid SDN Networks for Single Link/Node Failure
CN107947961A (en) * 2017-10-17 2018-04-20 上海数讯信息技术有限公司 Kubernetes Network Management System and method based on SDN
US20180167448A1 (en) * 2016-12-13 2018-06-14 International Business Machines Corporation Self-Recoverable Multitenant Distributed Clustered Systems
US20190268262A1 (en) * 2015-12-31 2019-08-29 New H3C Technologies Co., Ltd Controlling packets of virtual machines
CN113709220A (en) * 2021-08-16 2021-11-26 华云数据控股集团有限公司 High-availability realization method and system of virtual load balancer and electronic equipment
CN113872997A (en) * 2020-06-30 2021-12-31 华为技术有限公司 Container group POD reconstruction method based on container cluster service and related equipment
US20220215001A1 (en) * 2021-01-07 2022-07-07 Dell Products L.P. Replacing dedicated witness node in a stretched cluster with distributed management controllers

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160226701A1 (en) * 2015-01-29 2016-08-04 Futurewei Technologies, Inc. Capacity-Aware Heuristic Approach for Placing Software-Defined Networking (SDN) Switches in Hybrid SDN Networks for Single Link/Node Failure
CN105610614A (en) * 2015-12-28 2016-05-25 中国银联股份有限公司 High availability access system and high availability fault switching method
US20190268262A1 (en) * 2015-12-31 2019-08-29 New H3C Technologies Co., Ltd Controlling packets of virtual machines
US20180167448A1 (en) * 2016-12-13 2018-06-14 International Business Machines Corporation Self-Recoverable Multitenant Distributed Clustered Systems
CN107947961A (en) * 2017-10-17 2018-04-20 上海数讯信息技术有限公司 Kubernetes Network Management System and method based on SDN
CN113872997A (en) * 2020-06-30 2021-12-31 华为技术有限公司 Container group POD reconstruction method based on container cluster service and related equipment
US20220215001A1 (en) * 2021-01-07 2022-07-07 Dell Products L.P. Replacing dedicated witness node in a stretched cluster with distributed management controllers
CN113709220A (en) * 2021-08-16 2021-11-26 华云数据控股集团有限公司 High-availability realization method and system of virtual load balancer and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
冀晓翔;李江红;任娇;吴亚锋;王可;: "基于全连通群的软件定义控制系统及节点故障分析", 西北工业大学学报, no. 06, 15 December 2019 (2019-12-15), pages 1238 - 1247 *

Similar Documents

Publication Publication Date Title
CN107809367B (en) Device online method of SDN (software defined network), SDN controller and network device
US10644952B2 (en) VNF failover method and apparatus
KR101954480B1 (en) Automated build-out of a cloud-computing stamp
US10425281B2 (en) Automated network entity replacement based on historical topology consciousness
US10798218B2 (en) Environment isolation method and device
US20180337828A1 (en) System and method for mapping a connectivity state of a network
CN104981784A (en) Software deployment and control method and system
GB2505644A (en) Managing network configurations
CN113595782B (en) Network equipment management method based on SDN
US11665023B2 (en) Configuration validation of a device
US10313201B2 (en) Modular control device of an industrial automation system, and method for configuring the modular control device
CN103200036A (en) Automated configuration method of electrical power system cloud computing platform
CN103200030A (en) Network management device and method
CN107733727B (en) Zero configuration method, device and equipment
CN108366087B (en) ISCSI service realization method and device based on distributed file system
CN108667640B (en) Communication method and device, and network access system
EP2840738B1 (en) Mep configuration method and network device
CN115550220A (en) SDN cluster escape method, device and storage medium based on Openstack
EP3719599B1 (en) Network-distributed process control system and method for managing redundancy thereof
CN111478937B (en) Load balancing method and device
EP3355530A1 (en) Method, apparatus and device for processing service failure
CN115412447B (en) Service testing method, device and medium based on SDN
CN113986267A (en) Deployment method, device and medium for SDN controller installation
CN115695169B (en) Configuration information issuing method, device and storage medium
EP4040728A1 (en) Filtering information configuration method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Country or region after: China

Address after: 250101 s01-6 / F, No. 1036, Langchao Road, high tech Zone, Jinan City, Shandong Province

Applicant after: Inspur Network Technology (Shandong) Co.,Ltd.

Address before: 250101 s01-6 / F, No. 1036, Langchao Road, high tech Zone, Jinan City, Shandong Province

Applicant before: INSPUR CISCO NETWORK TECHNOLOGY Co.,Ltd.

Country or region before: China