CN117061514A - Server resource acquisition method, intelligent network card and cloud data center system - Google Patents

Server resource acquisition method, intelligent network card and cloud data center system Download PDF

Info

Publication number
CN117061514A
CN117061514A CN202311057400.1A CN202311057400A CN117061514A CN 117061514 A CN117061514 A CN 117061514A CN 202311057400 A CN202311057400 A CN 202311057400A CN 117061514 A CN117061514 A CN 117061514A
Authority
CN
China
Prior art keywords
server
intelligent network
network card
bmc
cloud platform
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311057400.1A
Other languages
Chinese (zh)
Inventor
张志刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Maipu Communication Technology Co Ltd
Original Assignee
Maipu Communication Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Maipu Communication Technology Co Ltd filed Critical Maipu Communication Technology Co Ltd
Priority to CN202311057400.1A priority Critical patent/CN117061514A/en
Publication of CN117061514A publication Critical patent/CN117061514A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3051Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3089Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
    • G06F11/3093Configuration details thereof, e.g. installation, enabling, spatial arrangement of the probes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0866Checking the configuration
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/141Setup of application sessions
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention relates to the technical field of communication, and provides a server resource acquisition method, an intelligent network card and a cloud data center system. The cloud platform determines an initial binding relation between the BMC of the server and the intelligent network card and sends a first identification code and a random number of the BMC of the server to the intelligent network card bound by the first identification code and the random number; the intelligent network card establishes an internal communication channel between the intelligent network card and the connected server, generates a second identification code based on the system information of the connected server and the received random number, and sends an initial binding relationship correct message to the cloud platform and sends the resource information of the connected server to the cloud platform when the second identification code is the same as the received first identification code; and the cloud platform binds the intelligent network card corresponding to each initial binding relation correct message, the BMC of the server and the resource information of the server to obtain a resource binding relation. The binding relation of the intelligent network card, the BMC of the server and the server resources can be obtained and determined.

Description

Server resource acquisition method, intelligent network card and cloud data center system
Technical Field
The invention relates to the technical field of communication, in particular to a server resource acquisition method, an intelligent network card and a cloud data center system.
Background
In cloud computing data center applications, cloud service providers provide public and private cloud services to customers through a cloud computing center. The user actually purchases the corresponding cloud service by purchasing the resources, and different resources have different values for the cloud service provider. The cloud service provider may use the intelligent network card to create high value resources, thereby providing high value cloud services to the user. The cloud platform presents resources such as CPU (Central Processing Unit ) model, CPU core number, memory size and the like to clients for selection and purchase, and the intelligent network card has binding relation with the installed server. Therefore, the cloud platform needs to acquire the server resource information in the environment of the intelligent network card and know the binding relation between the intelligent network card and the resources, so that the cloud platform can be linked together to provide high-value service for clients. In the prior art, only the information of the intelligent network card and the server resources can be collected independently, and then the binding relation between the intelligent network card and each resource is configured manually, so that the operation is complex and complicated, the efficiency is low, and the accuracy is low.
Disclosure of Invention
In view of the above, the present invention aims to provide a server resource acquisition method, an intelligent network card and a cloud data center system.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
in a first aspect, the present invention provides a method for obtaining server resources, which is applied to a cloud data center system, where the cloud data center system includes a cloud platform, a plurality of servers, and a plurality of intelligent network cards, each of the intelligent network cards is connected to one of the servers, and the method includes:
the cloud platform determines an initial binding relation between a baseboard management controller BMC and an intelligent network card of each server;
the cloud platform generates a first identification code and a random number of the BMC of each server, and sends the first identification code and the random number of the BMC of each server to an intelligent network card bound by the BMC of each server according to the initial binding relation;
each intelligent network card establishes an internal communication channel between the intelligent network card and a connected server, and acquires system information of the connected server through the internal communication channel;
each intelligent network card generates a second identification code based on the received random number and the system information of the connected server, and compares the second identification code with the received first identification code;
Each intelligent network card sends an initial binding relationship correct message to the cloud platform under the condition that the second identification code is the same as the received first identification code, acquires resource information of a connected server and sends the resource information to the cloud platform;
and the cloud platform binds the intelligent network card corresponding to each initial binding relation correct message, the BMC of the server and the resource information of the server to obtain a resource binding relation.
In an optional implementation manner, the cloud platform stores a BMC address of each server and a management address of each intelligent network card; the cloud platform determines an initial binding relationship between a baseboard management controller BMC and an intelligent network card of each server, and the method comprises the following steps:
the cloud platform traverses the BMC address of each server and sends a power-on starting instruction to the server corresponding to each BMC address, wherein the power-on starting instruction is used for indicating the server corresponding to the BMC address to power on and supply power for an intelligent network card connected to the server;
after each intelligent network card is electrified, a request message carrying an own management address and running time length is sent to the cloud platform;
When the binding state of the management address in the request message is unbound and the running time in the request message is smaller than a preset threshold value, the cloud platform binds the BMC address corresponding to the current power-on instruction with the management address in the request message;
and the cloud platform obtains the intelligent network card bound by the BMC of each server to obtain the initial binding relation.
In an optional implementation manner, the cloud platform stores a BMC address of each server and a management address of each intelligent network card; the cloud platform determines an initial binding relationship between a baseboard management controller BMC and an intelligent network card of each server, and the method comprises the following steps:
the cloud platform traverses the BMC address of each server and sends a power-on starting instruction to the server corresponding to each BMC address, wherein the power-on starting instruction is used for indicating the server corresponding to the BMC address to power on and supply power for an intelligent network card connected to the server;
the cloud platform detects connectivity of each management address with the binding state being unbound, and binds the management address which is changed from unconnected to connected with the BMC address corresponding to the current power-on start instruction;
And the cloud platform obtains the intelligent network card bound by the BMC of each server to obtain the initial binding relation.
In an optional implementation manner, the cloud platform stores a BMC address of each server and a management address of each intelligent network card; the cloud platform generates a first identification code and a random number of the BMC of each server, and the cloud platform comprises:
the cloud platform obtains the device model number and the device serial number of the BMC of each server through the BMC address of each server, and generates random numbers for the BMC of each server;
and the cloud platform calculates according to a preset algorithm based on the equipment model number, the equipment serial number and the random number of the BMC of each server and the management address of the intelligent network card bound with the equipment model number, the equipment serial number and the random number of each server to obtain a first identification code of the BMC of each server.
In an alternative embodiment, each of the intelligent network cards establishes an internal communication channel with a connected server, including:
each intelligent network card establishes communication connection between a first virtual network interface provided for a connected server and a second virtual network interface of a CPU (central processing unit) of the intelligent network card;
each intelligent network card sends a DHCP response message carrying a specific IP address to the connected server when receiving the DHCP request message sent by the connected server through the first virtual network interface, and establishes an internal communication channel between the intelligent network card and the connected server based on the first virtual network interface, the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface; and/or the number of the groups of groups,
Each intelligent network card obtains system information of a connected server through the internal communication channel, and the system information comprises:
each intelligent network card receives a start completion message sent by a connected server through the internal communication channel, wherein the start completion message is sent by the connected server by using a proxy service in an operating system after the operating system is started;
and each intelligent network card sends an information acquisition request to the connected server through the internal communication channel according to the starting completion message, and receives the equipment model and the equipment serial number returned by the connected server to obtain the system information of the connected server.
In an alternative embodiment, each of the intelligent network cards generates a second identification code based on the received random number and the device information of the connected server, including:
and each intelligent network card calculates according to a preset algorithm based on the equipment model and the equipment serial number of the connected server, the received random number and the self management address to obtain the second identification code.
In an alternative embodiment, the method further comprises:
and the cloud platform sends a power-off and power-off instruction to each server, and sets the binding state of the management address of the intelligent network card bound by the BMC of each server as bound according to the resource binding relation.
In a second aspect, the present invention provides a method for obtaining server resources, which is applied to an intelligent network card, wherein the intelligent network card is connected to a server, and the intelligent network card is in communication connection with a cloud platform, and the method includes:
receiving a first identification code and a random number of a BMC (baseboard management controller) of a server which is transmitted by the cloud platform and has an initial binding relation with the intelligent network card;
establishing an internal communication channel between the intelligent network card and a connected server, and acquiring system information of the connected server through the internal communication channel;
generating a second identification code based on the random number and system information of the connected server, and comparing the second identification code with the received first identification code;
and under the condition that the second identification code is the same as the received first identification code, sending an initial binding relation correct message to the cloud platform, acquiring the resource information of the connected server, and sending the resource information to the cloud platform, so that the cloud platform binds the intelligent network card, the BMC of the server and the resource information of the server based on the initial binding relation correct message to acquire the resource binding relation.
In an alternative embodiment, the establishing an internal communication channel between the intelligent network card and the connected server includes:
establishing communication connection between a first virtual network interface provided by the intelligent network card for a connected server and a second virtual network interface of a CPU (central processing unit) of the intelligent network card;
when receiving a DHCP request message sent by a connected server through the first virtual network interface, sending a DHCP response message carrying a specific IP address to the connected server, and establishing a communication channel between the intelligent network card and the connected service based on the first virtual network interface and the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface; and/or the number of the groups of groups,
the acquiring the system information of the connected server through the internal communication channel includes:
receiving a start completion message sent by a connected server through the internal communication channel, wherein the start completion message is sent by the connected server by using a proxy service in an operating system after the operating system is started;
and sending an information acquisition request to the connected server through the internal communication channel according to the starting completion message, and receiving the equipment model and the equipment serial number returned by the connected server to obtain the system information of the connected server.
In an alternative embodiment, the generating the second identification code based on the random number and the system information of the connected server includes:
and according to a preset algorithm, calculating based on the equipment model and the equipment serial number of the connected server, the random number and the self management address to obtain the second identification code.
In a third aspect, the present invention provides an intelligent network card, comprising a processor and a memory, the memory storing a computer program, the processor implementing the method of any of the preceding embodiments when executing the computer program.
In a fourth aspect, the present invention provides a cloud data center system, where the cloud data center system includes a cloud platform, a plurality of servers, and a plurality of intelligent network cards, each of the intelligent network cards being connected to one of the servers; the cloud platform, each of the servers, and each of the intelligent network cards are configured to perform the method of any of the preceding embodiments.
According to the server resource acquisition method, the intelligent network card and the cloud data center system, the cloud platform determines an initial binding relation between the BMC of each server and the intelligent network card and sends the generated first identification code and random number of the BMC of each server to the intelligent network card bound by the BMC of each server; each intelligent network card establishes an internal communication channel between the intelligent network card and the connected server and generates a second identification code based on the system information of the connected server and the received random number, and each intelligent network card sends an initial binding relationship correct message to the cloud platform and sends the resource information of the connected server to the cloud platform under the condition that the second identification code is the same as the received first identification code; and the cloud platform binds the intelligent network card corresponding to each initial binding relation correct message, the BMC of the server and the resource information of the server to obtain the resource binding relation. Through communication interaction between the cloud platform and the intelligent network card, the binding relation among the intelligent network card, the BMC of the server and the server resources can be obtained and determined, so that the operation is simplified, and the efficiency and the accuracy are improved.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 shows a schematic diagram of a cloud data center system provided by an embodiment of the present invention;
fig. 2 is a schematic flow chart of a server resource obtaining method applied to a cloud data center system according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a server resource obtaining method applied to an intelligent network card according to an embodiment of the present invention;
fig. 4 shows a schematic diagram of an intelligent network card according to an embodiment of the present invention.
Icon: 110-a processor; 120-memory; 130-a communication module.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present invention.
It is noted that relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Fig. 1 is a schematic diagram of a cloud data center system according to an embodiment of the present invention, which includes a cloud platform, a plurality of servers, and a plurality of intelligent network cards. Each intelligent network card plugs into a PCI or PCI-Express socket of each server to establish a bus connection, and each server has a BMC (Baseboard Manager Controller, baseboard management controller). The cloud platform is in communication connection with the management interface of each intelligent network card and the BMC interface of each server, and stores a BMC address list and a management address list, wherein the BMC address list comprises the BMC address of each server, and the management address list comprises the management address of each intelligent network card.
The following will use the cloud data center system as an execution subject, and introduce each step and corresponding technical effects in each method provided by the embodiment of the present invention.
Referring to fig. 2, fig. 2 is a flowchart of a method for obtaining server resources according to an embodiment of the present invention.
Step S202, the cloud platform determines an initial binding relation between a baseboard management controller BMC and an intelligent network card of each server;
in this embodiment, the cloud platform may use a manner of performing communication interaction with the intelligent network card, or may also use a manner of performing reachability detection on a management address of the intelligent network card to preliminarily determine the intelligent network card bound by the BMC of each server, that is, obtain an initial binding relationship between the BMC of each server and the intelligent network card.
Step S204, the cloud platform generates a first identification code and a random number of the BMC of each server, and sends the first identification code and the random number of the BMC of each server to an intelligent network card bound by the BMC of each server according to an initial binding relation;
in this embodiment, the cloud platform may obtain information of the BMC of the server and generate a random number for the BMC of the server, then generate a corresponding unique identification code based on the information of the BMC of the server and the random number, so as to obtain a first identification code, then the cloud platform may use HTTP (Hyper Text Transfer Protocol ) or other protocols to generate a message carrying the first identification code of the BMC of the server and the random number, and then send the message to an intelligent network card bound by the BMC of the server according to an initial binding relationship.
For the BMC of each server, the cloud platform may generate a first identification code and a random number of the BMC of each server in a similar manner, and send the first identification code and the random number to the intelligent network card bound by the BMC of each server according to an initial binding relationship.
Step S206, each intelligent network card establishes an internal communication channel between the intelligent network card and the connected server, and acquires the system information of the connected server through the internal communication channel;
Step S208, each intelligent network card generates a second identification code based on the received random number and the system information of the connected server, and compares the second identification code with the received first identification code;
it will be appreciated that each intelligent network card performs similar operations, and for brevity, embodiments of the present invention will be described with respect to one intelligent network card as an example.
In this embodiment, the intelligent network card extracts and stores the first identification code and the random number from the message sent by the cloud platform, then establishes an internal communication channel between the intelligent network card and the connected server, acquires the system information of the connected server through the internal communication channel, generates the second identification code based on the received random number and the system information of the connected server, and compares the generated second identification code with the received first identification code to determine whether the BMC of the server bound by the intelligent network card in the initial binding relationship is correct.
Step S210, each intelligent network card sends an initial binding relationship correct message to the cloud platform under the condition that the second identification code is the same as the received first identification code, and acquires the resource information of the connected server and sends the resource information to the cloud platform;
In this embodiment, if the second identification code is the same as the received first identification code after the comparison of the intelligent network card, it is indicated that the first identification code and the random number sent to the intelligent network card by the cloud platform are the first identification code and the random number of the BMC of the server connected to the intelligent network card, that is, the BMC of the server bound to the intelligent network card in the initial binding relationship is correct, then the intelligent network card sends an initial binding relationship correct message to the cloud platform, and sends the acquired information of the resources, such as the CPU model, the CPU core number, the memory, and the like, of the connected server to the cloud platform. It should be understood that the resource information of the server may be set according to actual situations, and the embodiment of the present invention is not limited.
And step S212, the cloud platform binds the intelligent network card corresponding to each initial binding relation correct message, the BMC of the server and the resource information of the server to obtain a resource binding relation.
In this embodiment, the cloud platform receives each initial binding relationship correct message and the resource information of the server corresponding to the initial binding relationship correct message, and binds the intelligent network card corresponding to each initial binding relationship correct message, the BMC of the server and the resource information of the server to obtain the resource binding relationship.
Based on the steps, the cloud platform determines an initial binding relation between the BMC of each server and the intelligent network card and sends the generated first identification code and random number of the BMC of each server to the intelligent network card bound by the BMC of each server; each intelligent network card establishes an internal communication channel between the intelligent network card and the connected server and generates a second identification code based on the system information of the connected server and the received random number, and each intelligent network card sends an initial binding relationship correct message to the cloud platform and sends the resource information of the connected server to the cloud platform under the condition that the second identification code is the same as the received first identification code; and the cloud platform binds the intelligent network card corresponding to each initial binding relation correct message, the BMC of the server and the resource information of the server to obtain the resource binding relation. Through communication interaction between the cloud platform and the intelligent network card, the binding relation among the intelligent network card, the BMC of the server and the server resources can be obtained and determined, so that the operation is simplified, and the efficiency and the accuracy are improved.
Optionally, for step S202, one possible implementation is provided by the embodiment of the present invention.
Step S202A-1, the cloud platform traverses the BMC address of each server, and sends a power-on starting instruction to the server corresponding to each BMC address, wherein the power-on starting instruction is used for indicating the server corresponding to the BMC address to power on and supply power for an intelligent network card connected to the server;
step S202A-3, after each intelligent network card is electrified, a request message carrying an own management address and running time length is sent to a cloud platform;
in this embodiment, the cloud platform traverses each BMC address in the BMC address list to establish an initial binding relationship between the BMC of the server and the intelligent network card by binding the BMC address with a certain management address.
The cloud platform sends a power-on starting instruction to a server corresponding to each BMC address, and the server can perform power-on starting operation and supply power for an intelligent network card connected to the server after receiving the power-on starting instruction.
The intelligent network card is powered on and started, and after the starting is completed, a pre-installed reporting program is executed to automatically send a request message to the cloud platform. For example, the intelligent network card uses HTTP or other protocols to generate a request message carrying its own management address and running time length, and continuously sends the request message to the cloud platform. It should be understood that the request message may also carry other parameters, which are not limited by the embodiment of the present invention.
Step S202A-5, when the binding state of the management address in the request message is unbound and the running time in the request message is less than a preset threshold, the cloud platform binds the BMC address corresponding to the current power-on instruction with the management address in the request message;
in this embodiment, the cloud platform receives a request message sent by the intelligent network card, and extracts parameters such as a management address and an operation duration carried in the request message. If the binding state of the management address in the request message received by the cloud platform is unbound and the operation time length is less than a preset threshold value, for example, 5 minutes, the intelligent network card corresponding to the management address in the request message received by the cloud platform is indicated to be the intelligent network card connected on the server which is just powered on, and then the BMC address corresponding to the current power-on instruction is bound with the management address in the request message. It should be understood that the preset threshold may be set according to practical situations, and embodiments of the present invention are not limited.
Optionally, the cloud platform further sends a response message to the intelligent network card corresponding to the management address in the request message, so that the intelligent network card stops sending the request message after receiving the response message.
Step S202A-7, the cloud platform obtains intelligent network cards bound by BMCs of all servers to obtain an initial binding relation;
in this embodiment, the cloud platform binds the BMC address of each server and the management address of each intelligent network card one by one according to the similar manner, so as to obtain the intelligent network card bound by the BMC of each server, and thus obtain the initial binding relationship.
Optionally, for step S202, another possible implementation manner is provided by the embodiment of the present invention.
Step S202B-1, the cloud platform traverses the BMC address of each server, and sends a power-on starting instruction to the server corresponding to each BMC address, wherein the power-on starting instruction is used for indicating the server corresponding to the BMC address to power on and supply power for the intelligent network card connected to the server;
step S202B-3, the cloud platform performs connectivity detection on each management address with the binding state being unbound, and binds the management address which is changed from unconnected to connected from the connected state with the BMC address corresponding to the current power-on instruction;
in this embodiment, the cloud platform traverses each BMC address in the BMC address list to establish an initial binding relationship between the BMC of the server and the intelligent network card by detecting reachability of a management address in the management address list after a certain server is specified to be powered on, and binding the BMC address of the server with the management address whose state changes.
The cloud platform sends a power-on starting instruction to a server corresponding to each BMC address, and the server can perform power-on starting operation and supply power for an intelligent network card connected to the server after receiving the power-on starting instruction.
And then the cloud platform detects the accessibility of each management address with the binding state being unbound, namely connectivity detection, if a certain management address is not reachable to be reachable, namely the connection state is not connected to be connected, the intelligent network card corresponding to the management address is the intelligent network card connected to the server which is just powered on and started, and then the BMC address corresponding to the current power-on and starting instruction is bound with the management address.
Step S202B-5, the cloud platform obtains the intelligent network card bound by the BMC of each server, and obtains the initial binding relation.
In this embodiment, the cloud platform binds the BMC address of each server and the management address of each intelligent network card one by one according to the similar manner, so as to obtain the intelligent network card bound by the BMC of each server, and thus obtain the initial binding relationship.
Optionally, for the process of generating the first identification code and the random number of the BMC of each server by the cloud platform in step S204, the embodiment of the present invention provides a possible implementation manner.
Step S204-1, the cloud platform obtains the device model number and the device serial number of the BMC of each server through the BMC address of each server, and generates a random number for the BMC of each server;
in step S204-3, the cloud platform calculates based on the device model number, the device serial number, the random number and the management address of the intelligent network card bound by the device model number and the random number of the BMC of each server according to a preset algorithm, and obtains a first identification code of the BMC of each server.
It will be appreciated that the manner in which the cloud platform generates the first identification code and the random number of the BMC of each server is similar, and for brevity, the BMC of one server will be described below as an example.
In this embodiment, the cloud platform generates a random number, namely rand_num, for the BMC of the server, and obtains information of the BMC of the server, such as a device model, namely bmc_type, and a device Serial number, namely bmc_serial, through the BMC address of the server.
And then the cloud platform calculates according to a preset algorithm based on the device model of the BMC of the server, namely BMC_Type, the device Serial number of the server, namely BMC_Serial, the random number of the server, namely RAND_Num, and the management address of the intelligent network card bound by the BMC of the server, namely NIC_Mgmt, and then the first identification code of the BMC of the server, namely H1, is obtained. In a similar manner, for the BMC of each server, the cloud platform may generate a first identification code and a random number for the BMC of each server.
It should be understood that the preset Algorithm may be MD5 (Message-Digest 5 algorism), or may be other algorithms set according to practical situations, which is not limited in the embodiments of the present invention.
Optionally, for the process of establishing an internal communication channel between each intelligent network card and the connected server in step S206, an embodiment of the present invention provides a possible implementation manner.
Step S206-1, each intelligent network card establishes communication connection between a first virtual network interface provided for a connected server and a second virtual network interface of a CPU (Central processing Unit) of the intelligent network card;
in step S206-3, when each intelligent network card receives the DHCP request message sent by the connected server through the first virtual network interface, each intelligent network card sends a DHCP response message carrying a specific IP address to the connected server, and establishes an internal communication channel between each intelligent network card and the connected server based on the first virtual network interface and the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface.
In this embodiment, the intelligent network card may be a resource acquisition program installed in advance, or may also be a resource acquisition instruction issued by the cloud platform to establish an internal communication channel between the intelligent network card and a connected server, and acquire server resources and report the server resources to the cloud platform.
The intelligent network card can perform item configuration through the forwarding module to establish communication connection between a first virtual network interface, i.e. vnet1, provided by the intelligent network card for the connected server and a second virtual network interface, i.e. vnet2, of the CPU of the intelligent network card, so that intercommunication is realized between the first virtual network interface, i.e. vnet1, and the second virtual network interface, i.e. vnet 2. It is understood that the first dashed network interface and the second virtual network interface each refer to a logical interface in network communication.
The intelligent network card then sets a DHCP (Dynamic Host Configuration Protocol ) table entry through the forwarding module, so as to send a DHCP response message carrying a specific IP address to the connected server when receiving a DHCP request message sent by the connected server through the first virtual network interface, i.e., vnet1, and establishes an internal communication channel between the intelligent network card and the connected server based on the first virtual network interface and the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface.
Optionally, for the process that each intelligent network card obtains the system information of the connected server through the communication internal channel in step S206, a possible implementation manner is provided in the embodiment of the present invention.
Step S206-5, each intelligent network card receives a start-up completion message sent by a connected server through an internal communication channel, wherein the start-up completion message is sent by the connected server by using a proxy service in an operating system after the operating system is started;
in step S206-7, each intelligent network card sends an information acquisition request to the connected server through the internal communication channel according to the starting completion message, and receives the equipment model and the equipment serial number returned by the connected server to obtain the system information of the connected server.
In this embodiment, the server may start the operating system through the cloud disk system mounted on the intelligent network card connected to the server, or may start the operating system in other manners. The embodiment of the invention is described in a mode of starting an operating system by adopting a cloud disk system mounted by an intelligent network card.
After the server completes the starting of the operating system through the cloud disk system mounted by the intelligent network card connected with the server, the proxy service in the operating system can be utilized to send a starting completion message to the intelligent network card connected with the server through an internal communication channel.
And the intelligent network card sends an information acquisition request to the connected server through an internal communication channel according to the starting completion message, and receives the equipment model and the equipment serial number returned by the connected server, so as to obtain the system information of the connected server.
Optionally, for the process of generating the second identification code by each intelligent network card in step S208 based on the received random number and the information of the system of the connected server, the embodiment of the present invention provides a possible implementation manner, namely: and each intelligent network card calculates according to a preset algorithm based on the equipment model and the equipment serial number of the connected server, the received random number and the self management address to obtain a second identification code.
In this embodiment, the intelligent network card performs an operation according to the same algorithm as the cloud platform based on the device model of the connected server, that is, host_type, the device Serial number, that is, host_serial, the received random number, that is, rand_num, and the management address of the intelligent network card, that is, nic_self_mgmt, to obtain the second identification code, that is, H2.
Optionally, after step S210, the embodiment of the present invention further provides a possible implementation manner, namely: the cloud platform sends a power-off and power-off instruction to each server, and the binding state of the management address of the intelligent network card bound by the BMC of each server is set to be bound according to the resource binding relation.
In this embodiment, after the cloud platform obtains the resource binding relationship, a power-down shutdown instruction is sent to each server, so that the server performs power-down shutdown operation after receiving the power-down shutdown instruction, and the binding state of the management address of the intelligent network card bound by the BMC of each server is set as bound according to the resource binding relationship.
Optionally, the embodiment of the invention further provides a server resource acquisition method applied to the intelligent network card, and refer to fig. 3.
Step S302, a first identification code and a random number of a BMC (baseboard management controller) of a server with an initial binding relation with an intelligent network card, which are sent by a cloud platform, are received;
step S304, an internal communication channel between the intelligent network card and the connected server is established, and system information of the connected server is obtained through the internal communication channel;
step S306, generating a second identification code based on the random number and the system information of the connected server, and comparing the second identification code with the received first identification code;
step S308, when the second identification code is the same as the received first identification code, sending an initial binding relation correct message to the cloud platform, obtaining the resource information of the connected server, and sending the resource information to the cloud platform, so that the cloud platform binds the intelligent network card, the BMC of the server and the resource information of the server based on the initial binding relation correct message to obtain a resource binding relation.
It can be understood that, in the embodiment taking the cloud data center system as the execution subject, the implementation process of steps S202 to S210 is similar to that of steps S302 to S308 in the present embodiment, and the basic principles and the technical effects of the two are the same, so for a brief description, reference may be made to the contents of steps S202 to S210 in the above embodiment.
Optionally, for the process of establishing the internal communication channel between the intelligent network card and the connected server in step S304, an embodiment of the present invention provides a possible implementation manner.
Step S304-1, establishing communication connection between a first virtual network interface provided by an intelligent network card for a connected server and a second virtual network interface of a CPU (Central processing Unit) of the intelligent network card;
step S304-3, when receiving the DHCP request message sent by the connected server through the first virtual network interface, sending a DHCP response message carrying a specific IP address to the connected server, and establishing a communication channel between the intelligent network card and the connected service based on the first virtual network interface and the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface;
it can be understood that, in the embodiment taking the cloud data center system as the execution subject, the implementation process of the steps S206-1 to S206-3 is similar to that of the steps S304-1 to S304-3 in the present embodiment, and the basic principles and the technical effects of the two are the same, so for a brief description, reference may be made to the contents of the steps S206-1 to S206-3 in the above embodiment.
Optionally, for the process of acquiring the system information of the connected server through the internal communication channel in step S304, the embodiment of the present invention provides a possible implementation manner.
Step S304-5, receiving a start-up completion message sent by the connected server through an internal communication channel, wherein the start-up completion message is sent by the connected server by using a proxy service in an operating system after the operating system is started up;
step S304-7, according to the starting completion message, sending an information acquisition request to the connected server through an internal communication channel, and receiving the equipment model and the equipment serial number returned by the connected server to obtain the system information of the connected server.
It can be understood that, in the embodiment taking the cloud data center system as the execution subject, the implementation process of the steps S206-5 to S206-7 is similar to that of the steps S304-5 to S304-7 in the present embodiment, and the basic principles and the technical effects of the two are the same, so for a brief description, reference may be made to the contents of the steps S206-5 to S206-7 in the above embodiment.
Optionally, for the process of generating the second identification code based on the random number and the system information of the connected server in step S306, the embodiment of the present invention provides a possible implementation manner, namely: and according to a preset algorithm, calculating based on the equipment model and the equipment serial number of the connected server, the random number and the self management address to obtain a second identification code.
It can be understood that, in the embodiment taking the cloud data center system as the execution subject, the implementation process of step S208 is similar to that of step S306 in the present embodiment, and the basic principles and the technical effects of both are the same, and for a brief description, reference may be made to the content of step S208 in the above embodiment.
The embodiment of the present invention further provides an intelligent network card, please refer to fig. 4, which is a schematic diagram of the intelligent network card provided in the embodiment of the present invention, wherein the intelligent network card includes a processor 110, a memory 120 and a communication module 130, and the components are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.
The processor 110 is used to read/write data or programs stored in the memory 120 and perform corresponding functions, and may be a general-purpose processor including a CPU (Central Processing Unit ), NP (Network Processor, network processor), etc.; but may also be a DSP digital signal processor, ASIC application specific integrated circuit, FPGA off-the-shelf programmable gate array or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
The memory 120 is used to store programs or data. The Memory 120 may be RAM (Random Access Memory ), ROM (Read Only Memory), PROM (Programmable Read-Only Memory, programmable Read Only Memory), EPROM (Erasable Programmable Read-Only Memory, erasable Read Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory, electrically erasable Read Only Memory), or the like.
The communication module 130 is used for signaling or data communication with other devices.
It will be appreciated that the architecture shown in fig. 4 is merely a schematic diagram of an intelligent network card, which may also include more or fewer components than shown in fig. 4, or have a different configuration than shown in fig. 4. The components shown in fig. 4 may be implemented in hardware, software, or a combination thereof.
The memory in the intelligent network card stores a computer program, and when the processor executes the computer program, the server resource acquisition method disclosed by the embodiment of the invention is realized.
The embodiment of the invention also provides a cloud data center system, which comprises a cloud platform, a plurality of servers and a plurality of intelligent network cards, wherein each intelligent network card is connected to one of the servers; the cloud platform, each server and each intelligent network card are used for executing the server resource acquisition method disclosed by the embodiment of the invention.
In summary, compared with the prior art, the server resource acquisition method, the intelligent network card and the cloud data center system provided by the invention have the following beneficial effects: (1) And comparing the identification codes respectively generated by the cloud platform and the intelligent network card to check the initial binding relation between the BMC of the server and the intelligent network card, so that the accurate binding relation can be obtained. (2) And the internal communication channel between the intelligent network card and the connected server is established through the intelligent network card, so that the resource information of the server can be obtained without borrowing an external network. (3) The IP address is allocated to the interface of the server by adopting the DHCP mode, complicated DHCP service is not required to be deployed, the operation can be completed in the intelligent network card, and the operation is simple and convenient. (4) The communication process between the intelligent network card and the server can ensure that the addresses are not exposed to the outside, can be deployed in batch replicability at the same time, and does not have the condition of address conflict. (5) The whole process of obtaining the server resources and establishing the binding relationship is automatically executed without human intervention. (6) The server resource acquisition method provided by the embodiment of the invention has strong universality and replicability, does not have the problem of incompatibility, has no relation with server manufacturers, server models and the like, and can be deployed in a large scale.
In several embodiments provided by the present invention, it should be understood that each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present invention may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (12)

1. The method for obtaining the server resources is characterized by being applied to a cloud data center system, wherein the cloud data center system comprises a cloud platform, a plurality of servers and a plurality of intelligent network cards, each intelligent network card is connected to one of the servers, and the method comprises the following steps:
the cloud platform determines an initial binding relation between a baseboard management controller BMC and an intelligent network card of each server;
the cloud platform generates a first identification code and a random number of the BMC of each server, and sends the first identification code and the random number of the BMC of each server to an intelligent network card bound by the BMC of each server according to the initial binding relation;
each intelligent network card establishes an internal communication channel between the intelligent network card and a connected server, and acquires system information of the connected server through the internal communication channel;
each intelligent network card generates a second identification code based on the received random number and the system information of the connected server, and compares the second identification code with the received first identification code;
each intelligent network card sends an initial binding relationship correct message to the cloud platform under the condition that the second identification code is the same as the received first identification code, acquires resource information of a connected server and sends the resource information to the cloud platform;
And the cloud platform binds the intelligent network card corresponding to each initial binding relation correct message, the BMC of the server and the resource information of the server to obtain a resource binding relation.
2. The method of claim 1, wherein the cloud platform stores a BMC address of each of the servers and a management address of each of the intelligent network cards; the cloud platform determines an initial binding relationship between a baseboard management controller BMC and an intelligent network card of each server, and the method comprises the following steps:
the cloud platform traverses the BMC address of each server and sends a power-on starting instruction to the server corresponding to each BMC address, wherein the power-on starting instruction is used for indicating the server corresponding to the BMC address to power on and supply power for an intelligent network card connected to the server;
after each intelligent network card is electrified, a request message carrying an own management address and running time length is sent to the cloud platform;
when the binding state of the management address in the request message is unbound and the running time in the request message is smaller than a preset threshold value, the cloud platform binds the BMC address corresponding to the current power-on instruction with the management address in the request message;
And the cloud platform obtains the intelligent network card bound by the BMC of each server to obtain the initial binding relation.
3. The method of claim 1, wherein the cloud platform stores a BMC address of each of the servers and a management address of each of the intelligent network cards; the cloud platform determines an initial binding relationship between a baseboard management controller BMC and an intelligent network card of each server, and the method comprises the following steps:
the cloud platform traverses the BMC address of each server and sends a power-on starting instruction to the server corresponding to each BMC address, wherein the power-on starting instruction is used for indicating the server corresponding to the BMC address to power on and supply power for an intelligent network card connected to the server;
the cloud platform detects connectivity of each management address with the binding state being unbound, and binds the management address which is changed from unconnected to connected with the BMC address corresponding to the current power-on start instruction;
and the cloud platform obtains the intelligent network card bound by the BMC of each server to obtain the initial binding relation.
4. The method of claim 1, wherein the cloud platform stores a BMC address of each of the servers and a management address of each of the intelligent network cards; the cloud platform generates a first identification code and a random number of the BMC of each server, and the cloud platform comprises:
The cloud platform obtains the device model number and the device serial number of the BMC of each server through the BMC address of each server, and generates random numbers for the BMC of each server;
and the cloud platform calculates according to a preset algorithm based on the equipment model number, the equipment serial number and the random number of the BMC of each server and the management address of the intelligent network card bound with the equipment model number, the equipment serial number and the random number of each server to obtain a first identification code of the BMC of each server.
5. The method of claim 1, wherein each of the intelligent network cards establishes an internal communication channel with a connected server, comprising:
each intelligent network card establishes communication connection between a first virtual network interface provided for a connected server and a second virtual network interface of a CPU (central processing unit) of the intelligent network card;
each intelligent network card sends a DHCP response message carrying a specific IP address to the connected server when receiving the DHCP request message sent by the connected server through the first virtual network interface, and establishes an internal communication channel between the intelligent network card and the connected server based on the first virtual network interface, the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface; and/or the number of the groups of groups,
Each intelligent network card obtains system information of a connected server through the internal communication channel, and the system information comprises:
each intelligent network card receives a start completion message sent by a connected server through the internal communication channel, wherein the start completion message is sent by the connected server by using a proxy service in an operating system after the operating system is started;
and each intelligent network card sends an information acquisition request to the connected server through the internal communication channel according to the starting completion message, and receives the equipment model and the equipment serial number returned by the connected server to obtain the system information of the connected server.
6. The method of claim 1, wherein each of the intelligent network cards generates a second identification code based on the received random number and system information of the connected server, comprising:
and each intelligent network card calculates according to a preset algorithm based on the equipment model and the equipment serial number of the connected server, the received random number and the self management address to obtain the second identification code.
7. A method according to claim 2 or 3, characterized in that the method further comprises:
And the cloud platform sends a power-off and power-off instruction to each server, and sets the binding state of the management address of the intelligent network card bound by the BMC of each server as bound according to the resource binding relation.
8. The method is characterized by being applied to an intelligent network card, wherein the intelligent network card is connected to a server, and the intelligent network card is in communication connection with a cloud platform, and the method comprises the following steps:
receiving a first identification code and a random number of a BMC (baseboard management controller) of a server which is transmitted by the cloud platform and has an initial binding relation with the intelligent network card;
establishing an internal communication channel between the intelligent network card and a connected server, and acquiring system information of the connected server through the internal communication channel;
generating a second identification code based on the random number and system information of the connected server, and comparing the second identification code with the received first identification code;
and under the condition that the second identification code is the same as the received first identification code, sending an initial binding relation correct message to the cloud platform, acquiring the resource information of the connected server, and sending the resource information to the cloud platform, so that the cloud platform binds the intelligent network card, the BMC of the server and the resource information of the server based on the initial binding relation correct message to acquire the resource binding relation.
9. The method of claim 8, wherein the establishing an internal communication channel between the intelligent network card and the connected server comprises:
establishing communication connection between a first virtual network interface provided by the intelligent network card for a connected server and a second virtual network interface of a CPU (central processing unit) of the intelligent network card;
when receiving a DHCP request message sent by a connected server through the first virtual network interface, sending a DHCP response message carrying a specific IP address to the connected server, and establishing a communication channel between the intelligent network card and the connected service based on the first virtual network interface and the second virtual network interface and the IP address of the first virtual network interface and the IP address of the second virtual network interface; and/or the number of the groups of groups,
the acquiring the system information of the connected server through the internal communication channel includes:
receiving a start completion message sent by a connected server through the internal communication channel, wherein the start completion message is sent by the connected server by using a proxy service in an operating system after the operating system is started;
and sending an information acquisition request to the connected server through the internal communication channel according to the starting completion message, and receiving the equipment model and the equipment serial number returned by the connected server to obtain the system information of the connected server.
10. The method of claim 8, wherein the generating a second identification code based on the random number and system information of the connected server comprises:
and according to a preset algorithm, calculating based on the equipment model and the equipment serial number of the connected server, the random number and the self management address to obtain the second identification code.
11. An intelligent network card, comprising a processor and a memory, the memory storing a computer program, the processor implementing the method of any of claims 8-10 when executing the computer program.
12. The cloud data center system is characterized by comprising a cloud platform, a plurality of servers and a plurality of intelligent network cards, wherein each intelligent network card is connected to one of the servers; the cloud platform, each of the servers and each of the intelligent network cards are for performing the method of any of claims 1-7.
CN202311057400.1A 2023-08-21 2023-08-21 Server resource acquisition method, intelligent network card and cloud data center system Pending CN117061514A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311057400.1A CN117061514A (en) 2023-08-21 2023-08-21 Server resource acquisition method, intelligent network card and cloud data center system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311057400.1A CN117061514A (en) 2023-08-21 2023-08-21 Server resource acquisition method, intelligent network card and cloud data center system

Publications (1)

Publication Number Publication Date
CN117061514A true CN117061514A (en) 2023-11-14

Family

ID=88655102

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311057400.1A Pending CN117061514A (en) 2023-08-21 2023-08-21 Server resource acquisition method, intelligent network card and cloud data center system

Country Status (1)

Country Link
CN (1) CN117061514A (en)

Similar Documents

Publication Publication Date Title
CN112866017B (en) Method, system, medium and device for configuring BMC IP address of bare metal server
WO2019075773A1 (en) Data processing method and apparatus, computer device and storage medium
CN110856126A (en) Information reporting and receiving method, terminal equipment and storage medium
CN108880972B (en) Information processing method, server and terminal
CN108459944A (en) System operation monitoring method, device and server
CN111353136B (en) Method and device for processing operation request
CN111431931A (en) Node consensus method and device
CN109842515B (en) Method for multi-device network distribution, server and computer storage medium
CN112398738B (en) Method and device for acquiring connection relation, equipment and computer readable storage medium
CN111177053B (en) Data communication method, device and system and computer readable storage medium
CN111444074A (en) Data monitoring method and device, electronic equipment and readable storage medium
CN110830263A (en) Automatic login method and device for vehicle-mounted system
CN108282374B (en) Method, system and readable storage medium for configuring NCSI network card
CN117061514A (en) Server resource acquisition method, intelligent network card and cloud data center system
CN108737350B (en) Information processing method and client
CN113114510B (en) Network fault information synchronization method and device
CN112367415B (en) Generation method and device of attribute information, electronic equipment and computer readable medium
CN112788078B (en) Data transmission method, receiving device, sending device and computer equipment
CN111767481B (en) Access processing method, device, equipment and storage medium
CN111124445B (en) Home gateway upgrading method and home gateway
CN114185804A (en) Interface testing method and device and terminal equipment
CN114585055A (en) Vehicle-mounted WiFi access method, vehicle controller, cloud server and mobile terminal
CN115514733A (en) Configuration method and configuration device for domain name resolution service (DNS)
CN115208951B (en) Request processing method, request processing device, electronic equipment and computer readable storage medium
CN112804144B (en) Information configuration method and network equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination