CN115190046B - Detection method, detection device and computing equipment of server cluster - Google Patents

Detection method, detection device and computing equipment of server cluster Download PDF

Info

Publication number
CN115190046B
CN115190046B CN202210730202.6A CN202210730202A CN115190046B CN 115190046 B CN115190046 B CN 115190046B CN 202210730202 A CN202210730202 A CN 202210730202A CN 115190046 B CN115190046 B CN 115190046B
Authority
CN
China
Prior art keywords
server cluster
server
information
communication
brain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210730202.6A
Other languages
Chinese (zh)
Other versions
CN115190046A (en
Inventor
刘兴伟
孟杰
薛皓琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Uniontech Software Technology Co Ltd
Original Assignee
Uniontech Software Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Uniontech Software Technology Co Ltd filed Critical Uniontech Software Technology Co Ltd
Priority to CN202210730202.6A priority Critical patent/CN115190046B/en
Publication of CN115190046A publication Critical patent/CN115190046A/en
Application granted granted Critical
Publication of CN115190046B publication Critical patent/CN115190046B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/50Testing arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a detection method, a detection device and a computing device of a server cluster, wherein the method comprises the following steps: acquiring equipment information of a server cluster, and determining whether the server cluster meets basic conditions of brain fracture phenomenon simulation according to the equipment information; if the communication mode of the server cluster is met, determining a communication mode of the server cluster, and performing corresponding brain-crack phenomenon simulation on the server cluster according to the communication mode to simulate the phenomenon that a fault server node which cannot communicate with other server nodes appears in the server cluster; monitoring the server cluster state, acquiring the current server cluster state information, and comparing the current server cluster state information with the last server cluster state information; and determining whether the server cluster has the brain fracture phenomenon or not and whether the arbitration equipment executes arbitration operation or not according to the comparison information so as to determine whether the server cluster can avoid the brain fracture problem or not. According to the technical scheme of the invention, whether the server cluster has the hidden trouble of brain fracture or not can be detected.

Description

Detection method, detection device and computing equipment of server cluster
The present application is a divisional application of patent application 2022103817019 filed on day 13, 4, 2022.
Technical Field
The present invention relates to the field of server clusters, and in particular, to a method and apparatus for detecting a server cluster, and a computing device.
Background
The high availability server Cluster (High Availability Cluster, abbreviated as HA Cluster) refers to a server Cluster technology for the purpose of reducing service interruption time. The service program of the user is protected to provide uninterrupted service to the outside, so that the influence of faults caused by software, hardware and people on the service is reduced to the minimum.
Split-brain (split-brain) refers to that in an HA Cluster system composed of two machines, the heartbeat of the other party cannot be detected within a certain period of time, and the two machines are simultaneously in an active state because the two machines acquire ownership of resources and services.
In the application of a dual-server node server cluster, most of the server clusters are modes of a master server and a slave server, and the server cluster generally requires that only one server node in two server nodes of the server cluster runs service resources or provides services to the outside at the same time.
In an actual production environment, 100% of control cannot be achieved on the environment, in some special cases, when a communication link between server nodes of a server cluster fails due to the problems of network line aging, network port damage, routing failure and the like, the server cluster may possibly generate a brain burst phenomenon that the heart beat of the failed node cannot be detected, and at this time, if a brain burst processing mechanism of the server cluster is not configured, the brain burst problem that resources or services of the server cluster run on two server nodes at the same time (the two server nodes access the same shared resource at the same time) is caused, so that the integrity of service data is affected.
For this reason, a method for detecting a server cluster is needed to solve the problems in the above-mentioned solution.
Disclosure of Invention
Therefore, the present invention provides a method and apparatus for detecting a server cluster, so as to solve or at least alleviate the above problems.
According to an aspect of the present invention, there is provided a method of detecting a server cluster, executed in a computing device, the computing device being communicatively connected to a server cluster, the server cluster comprising a plurality of server nodes, an arbitration device communicatively connected to the plurality of server nodes, the method comprising the steps of: acquiring equipment information of a server cluster, and determining whether the server cluster meets basic conditions of brain fracture phenomenon simulation according to the equipment information; if the basic condition of the brain fracture phenomenon simulation is met, determining a communication mode of a server cluster, and performing corresponding brain fracture phenomenon simulation on the server cluster according to the communication mode so as to simulate the phenomenon of a fault server node which cannot communicate with other server nodes in the server cluster; monitoring the server cluster state, acquiring current server cluster state information, and comparing the current server cluster state information with the last server cluster state information to obtain comparison information; and determining whether the server cluster has a brain fracture phenomenon or not and whether the arbitration equipment executes arbitration operation or not according to the comparison information so as to determine whether the server cluster can avoid the brain fracture problem or not.
Optionally, in the method for detecting a server cluster according to the present invention, the arbitration operation includes an operation of cutting off power to a failed server node.
Optionally, in the method for detecting a server cluster according to the present invention, the computing device is communicatively connected to a client, and the method further includes the steps of: and if the server cluster is determined to have the brain-split phenomenon and the arbitration equipment executes arbitration operation, sending the comparison information to a client so as to display the comparison information on the client.
Optionally, in the method for detecting a server cluster according to the present invention, the method further includes the steps of: determining whether the server cluster state is normal or not according to the comparison information, and generating a corresponding state analysis result; and sending the state analysis result to the client so as to display the state analysis result at the client.
Optionally, in the method for detecting a server cluster according to the present invention, determining whether the server cluster status normally includes one or more of: determining whether an arbitrating device in the server cluster successfully performs a predetermined operation, determining whether the state of each server node is normal, and determining whether the cluster resource state is normal.
Optionally, in the method for detecting a server cluster according to the present invention, the server cluster state information includes one or more of arbitrated device state information and historical operations in the server cluster, state information of each server node, and cluster resource state information.
Optionally, in the method for detecting a server cluster according to the present invention, the device information includes arbitration device information and information of each server node, and determining whether the server cluster meets a basic condition of brain-split phenomenon simulation according to the device information includes: determining, according to the device information, whether the mediation device can monitor each server node in the server cluster; if yes, determining whether the server cluster meets the vote number condition of a voting mechanism; and if so, determining that the server cluster meets the basic condition of the brain fracture phenomenon simulation.
Optionally, in the method for detecting a server cluster according to the present invention, the ticket number condition is: q > 2n+1, and n > 0; q is the total ticket number, and n is the ticket number obtained by the fault server node in the server cluster.
Optionally, in the method for detecting a server cluster according to the present invention, the communication mode includes a communication protocol and a communication transmission mode, and the step of determining the communication mode of the server cluster includes: and determining a communication tool used by the server cluster according to the type of the software package used by the server cluster, and acquiring a communication protocol and a communication transmission mode of the server cluster from a configuration file of the communication tool.
Optionally, in the method for detecting a server cluster according to the present invention, the communication protocol includes one or more of UDP protocol, paxos protocol, and artem protocol; the communication transmission mode comprises one or more of unicast, multicast and broadcast.
Optionally, in the method for detecting a server cluster according to the present invention, the step of obtaining the state information of the present server cluster includes: and acquiring the state information of the current server cluster at fixed time.
Optionally, in the method for detecting a server cluster according to the present invention, the arbitration device is a stonith device.
According to one aspect of the present invention, there is provided a detection apparatus residing in a computing device, the computing device being communicatively connected to a server cluster comprising a plurality of server nodes, an arbitration device communicatively connected to the plurality of server nodes, respectively, the apparatus comprising: the brain fracture simulation module is suitable for acquiring equipment information of the server cluster and determining whether the server cluster meets the basic conditions of brain fracture phenomenon simulation according to the equipment information; if the basic condition of the brain fracture phenomenon simulation is met, determining a communication mode of a server cluster, and performing corresponding brain fracture phenomenon simulation on the server cluster according to the communication mode so as to simulate the phenomenon of a fault server node which cannot communicate with other server nodes in the server cluster; the information processing module is suitable for monitoring the server cluster state and acquiring the current server cluster state information, and comparing the current server cluster state information with the last server cluster state information to obtain comparison information; and determining whether the server cluster has a brain fracture phenomenon or not and whether the arbitration equipment executes arbitration operation or not according to the comparison information so as to determine whether the server cluster can avoid the brain fracture problem or not.
According to one aspect of the invention, there is provided a computing device comprising: at least one processor; a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of detecting a server cluster as described above.
According to one aspect of the present invention, there is provided a readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform a method of detecting a server cluster as described above.
According to the technical scheme of the invention, the method for detecting the server cluster is provided, wherein the server cluster is subjected to brain fracture phenomenon simulation, and whether the server cluster can perform arbitration operation is detected to cope with and process the brain fracture phenomenon so as to avoid the brain fracture problem of the server cluster, and accordingly whether the server cluster has the hidden danger of the brain fracture problem is determined. Specifically, when the occurrence of the brain fracture phenomenon of the server cluster is determined according to the comparison information, the arbitration device performs arbitration operation for the brain fracture phenomenon of the server cluster to process, so that whether the server cluster can avoid the brain fracture problem or not and whether the expectation of high availability of the server cluster is met is judged. In this way, by executing the method for detecting the server cluster before the server cluster is put into the production environment, whether the server cluster has the hidden trouble of brain fracture or not can be detected.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which set forth the various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to fall within the scope of the claimed subject matter. The above, as well as additional objects, features, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. Like reference numerals generally refer to like parts or elements throughout the present disclosure.
FIG. 1 shows a schematic diagram of a server cluster 100 (communicatively coupled to a computing device) in accordance with one embodiment of the invention;
FIG. 2 shows a schematic diagram of a computing device 200 according to one embodiment of the invention;
FIGS. 3 and 4 respectively illustrate a flow chart of a method 300 of detecting a server cluster according to one embodiment of the invention;
Fig. 5 shows a schematic diagram of a detection device 500 according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Fig. 1 shows a schematic diagram of a server cluster 100 according to one embodiment of the invention. The computing device 200 is communicatively coupled to the server cluster 100 to detect whether the server cluster 100 can avoid a split problem, and meet the expectations of high availability, by performing a split simulation on the server cluster 100.
As shown in fig. 1, the server cluster 100 includes, in order from bottom to top, a device layer, a messaging layer, a resource allocation layer, and a resource proxy layer.
Wherein the device layer may comprise a plurality of server nodes 150. Here, fig. 1 shows by way of example only that the server cluster 100 includes two server nodes 150, but the invention is not limited to a specific number of server nodes. In addition, the device layer also includes an arbitration device 140 communicatively coupled to the plurality of servers 150.
The arbitration device 140 is a Fence device. In one implementation, the arbitration device 140 may be implemented as a stonith device. It should be noted that stonith is an acronym of "shoot the other node in the head", which is a component of the heart bean software package that allows automatic restarting of the power supply of a failed server (failed server node) using a power device communicatively connected to a health server, the stonith device may turn off the power supply and respond to software commands, and the server running heart bean may send commands to the stonith device via a serial or network line to control the power supply of the failed server. For example, when one server node detects that another server node is malfunctioning, a command may be sent over the network to the stonith device to control the power supply of the malfunctioning server node.
Accordingly, as shown in fig. 1, the resource configuration layer includes a plurality of resource managers 120, the message communication layer includes a plurality of communication tools 130, the resource agent layer includes a plurality of resource agents 110, and the resource managers 120 are respectively in communication connection with the corresponding resource agents 110 and the communication tools 130.
The resource manager 120 may be used in combination with the communication tool 130 to schedule and manage cluster resources, wherein the resource manager 120 may utilize heartbeat information between server nodes provided by the communication tool 130 to determine whether the node status is normal in order to enable failure detection and resource recovery at the node and resource level.
In one implementation, the communication tool 130 may be implemented as a Corosync communication tool or a heart bean communication tool.
It should be noted that fig. 1 provides a block diagram of a server cluster by way of example only, and the present invention is not limited to the structure of the server cluster shown in fig. 1.
In addition, as shown in FIG. 1, the computing device 200 may also be communicatively connected to the client 201 to send status information of the server cluster to the client 201 for presentation for viewing by a user or maintenance personnel.
In an embodiment of the present invention, the computing device 200 is adapted to perform the method 300 for detecting a server cluster of the present invention to perform a brain fracture simulation on the server cluster 100 to detect whether the server cluster 100 can avoid a brain fracture problem. The method 300 for detecting a server cluster will be described in detail below.
In one embodiment, as shown in fig. 1, a detection apparatus 500 is included in a computing device 200, the detection apparatus 500 being adapted to perform the method 300 of detecting a server cluster of the present invention.
FIG. 2 shows a schematic diagram of a computing device 200 according to one embodiment of the invention.
As shown in FIG. 2, in a basic configuration 202, a computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between the processor 204 and the system memory 206.
Depending on the desired configuration, the processor 204 may be any type of processing including, but not limited to: a microprocessor (UP), a microcontroller (UC), a digital information processor (DSP), or any combination thereof. Processor 204 may include one or more levels of cache, such as a first level cache 210 and a second level cache 212, a processor core 214, and registers 216. The example processor core 214 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations, the memory controller 218 may be an internal part of the processor 204.
Depending on the desired configuration, system memory 206 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 206 may include an operating system 220, one or more applications 222, and program data 224. The application 222 is in effect a plurality of program instructions for instructing the processor 204 to perform a corresponding operation. In some implementations, the application 222 can be arranged to cause the processor 204 to operate with the program data 224 on an operating system.
Computing device 200 also includes storage device 232, storage device 232 including removable storage 236 and non-removable storage 238.
Computing device 200 may also include a storage interface bus 234. Storage interface bus 234 enables communication from storage devices 232 (e.g., removable storage 236 and non-removable storage 238) to base configuration 202 via bus/interface controller 230. At least a portion of operating system 220, applications 222, and program data 224 may be stored on removable storage 236 and/or non-removable storage 238, and loaded into system memory 206 via storage interface bus 234 and executed by one or more processors 204 when computing device 200 is powered up or application 222 is to be executed.
Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to basic configuration 202 via bus/interface controller 230. The exemplary output device 242 includes an image processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. The example peripheral interface 244 may include a serial interface controller 254 and a parallel interface controller 256, which may be configured to facilitate communication via one or more I/O ports 258 and external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.). The example communication device 246 may include a network controller 260 that may be arranged to facilitate communication with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or changed in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or special purpose network, and wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR) or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
In an embodiment according to the invention, the computing device 200 is configured to perform the method 300 of detecting a server cluster according to the invention. The computing device 200 has embodied in its application a plurality of program instructions for performing the method 300 for detecting a server cluster according to the invention, which program instructions may instruct a processor to perform the method 300 for detecting a server cluster according to the invention, so that the computing device enhances the availability of a server cluster by performing the method 300 for detecting a server cluster according to the invention.
In one embodiment, an application disposed on an operating system of computing device 200 includes detection apparatus 500, detection apparatus 500 being configured to perform detection method 300 of a server cluster in accordance with the present invention.
Fig. 3 and 4 respectively show flowcharts of a method 300 for detecting a server cluster according to an embodiment of the present invention. The method 300 is suitable for execution in a computing device, such as the computing device 200 described previously.
According to the server cluster 100 provided by the invention, the server cluster system 100 comprises a plurality of server nodes 150 and arbitration equipment 140 in communication connection with the plurality of server nodes. In one implementation, the arbitration device 140 may be implemented, for example, as a stonith device.
It should be noted that, according to the method 300 for detecting a server cluster of the present invention, before the server cluster is put into a production environment, a split phenomenon is simulated for the server cluster, and whether the server cluster can perform an arbitration operation for the split phenomenon (the server node in the server cluster fails and cannot communicate with other server nodes, and the heartbeat of the failed server node cannot be detected) is detected, so as to avoid the split problem (the situation that the resources or services of the server cluster are simultaneously running on two server nodes) caused by the split phenomenon for the server cluster.
As shown in fig. 3, the method 300 begins at step S310.
In step S310, a server cluster environment is detected, device information of the server cluster is obtained, and whether the server cluster meets a basic condition of brain fracture phenomenon simulation is determined according to the device information.
Here, the device information of the server cluster may include information of respective devices in the server cluster, including, for example, mediation device information in the server cluster, information of each server node in the server cluster, and the like.
In one embodiment, determining whether the server cluster satisfies the basic condition of the brain-split phenomenon simulation according to the device information may be specifically performed in the following manner: it is determined from the device information whether the mediation device can monitor each of the server nodes in the server cluster. If each server node in the server cluster can be monitored, it is further determined whether the server cluster satisfies a vote count condition of the voting mechanism. If the voting number condition of the voting mechanism is met, the server cluster can be determined to meet the basic condition of the brain-split phenomenon simulation.
It should be noted that in case the mediation device is able to monitor each server node in the server cluster and the server cluster satisfies the vote condition of the voting mechanism, the failure server node may be subjected to a Fence operation (mediation operation, such as an operation of cutting off the power supply) by the voting mechanism and by using the mediation device, so that the basic condition of performing the brain-split simulation on the server cluster is met.
Here, the voting mechanism may be embodied as a Quorum mechanism, whose vote count condition is more than half of the total vote count. That is, the vote count condition of the voting mechanism is satisfied when the number of votes obtained by the normal server node exceeds half of the total number of votes. The ticket number condition may be expressed by the following formula: q > 2n+1, and n > 0. Wherein Q is the total number of tickets (i.e., the total number of server nodes), and n is the number of tickets obtained by the failed server node in the server cluster. It will be appreciated that when the total number of votes satisfies Q > 2n+1, the corresponding satisfaction is that the number of votes obtained by the normal server node exceeds half the total number of votes.
For example, the server cluster includes four server nodes and arbitration device, and when one server node in the server cluster fails and cannot communicate with other server nodes (no heartbeat of the failed server node is detected), the Fence operation can be performed through the voting mechanism, specifically, after voting, the normal server node can acquire 3 votes, and the failed server node can acquire 1 vote, so that the voting condition of the voting mechanism is satisfied. On the basis of meeting the voting conditions of the voting mechanism, the method can perform the Fence operation on the fault server node through the voting mechanism of the server cluster, so that the method meets the basic conditions of performing the brain fracture simulation on the server cluster. The split problem can be avoided by performing split phenomenon simulation on the server cluster to detect whether the server cluster can perform arbitration operations when split phenomenon occurs.
If it is determined that the server cluster does not meet the basic conditions of the brain-crack phenomenon simulation, relevant information which does not meet the basic conditions can be input, for example, sent to a client in communication connection with the computing device for display, and the brain-crack simulation of the server cluster is not performed.
If it is determined that the server cluster meets the basic conditions of the split phenomenon simulation, the step S320 is continuously performed to simulate the split phenomenon on the server cluster, and steps S330 to S340 are performed to detect whether the server cluster can perform the arbitration operation when the split phenomenon occurs, so as to avoid the split problem.
In step S320, a communication mode of the server cluster is determined, and a corresponding brain-crack phenomenon simulation is performed on the server cluster according to the communication mode, so as to simulate a phenomenon that at least one fault server node which cannot communicate with other server nodes appears in the server cluster.
Here, the phenomenon of brain-splitting that simulates a server cluster refers to a phenomenon that at least one server node in the simulated server cluster fails and cannot communicate with other server nodes, and the other server nodes cannot receive heartbeats of the server node.
In one embodiment, a method for simulating a split brain phenomenon corresponding to a communication mode of a server cluster may be invoked, and the split brain phenomenon simulation may be performed on the server cluster according to the method for simulating a split brain phenomenon corresponding to the communication mode.
In one embodiment, the communication modes include a communication protocol and a communication transmission mode, that is, the communication mode of the server cluster is determined according to the communication protocol and the communication transmission mode used by the server cluster. After determining the communication protocol and the communication transmission mode used by the server cluster, the corresponding brain-cracking phenomenon simulation can be performed on the server cluster according to the communication protocol and the communication transmission mode used by the server cluster.
Here, the communication protocol includes, for example, UDP protocol, paxos protocol, and artem protocol, and the communication transmission method includes, for example, unicast, multicast (i.e., multicast), broadcast, and the like. It should be noted, however, that the present invention is not limited to the particular type of communication protocol and communication transport in a communication scheme.
In one embodiment, a communication tool used by the server cluster may be determined according to a software package type used by the server cluster, and a communication protocol and a communication transmission mode of the server cluster may be obtained from a configuration file of the communication tool.
For example, when it is determined that the communication tool used by the server cluster is Corosync (a part of the server cluster management suite, which is a group communication system), a configuration file Corosync. Conf corresponding to the communication tool Corosync is read, and fields such as a transport (communication protocol), an mcostaddr (multicast address), an mcastport (multicast port), and a broadcast (broadcast) in the configuration file are parsed to determine parameter values of the respective fields. Taking a common Corosync configuration file as an example, when the parameter value of transport (communication protocol) is UDP, mcastaddr (multicast address) and the parameter value of mcastport (multicast port) is 226.16.21.1, the communication protocol of the server cluster is UDP, the communication transmission mode is multicast, and specific information of the transmission mode (multicast address and port number) can be obtained from the configuration file. Furthermore, according to the communication mode, the server cluster brain-splitting phenomenon can be simulated by closing the corresponding port of the UDP protocol, specifically, the 5405 port of the UDP protocol of the server node can be blocked by calling the corresponding function, so that the server node cannot communicate with other server nodes, and the occurrence of the server cluster brain-splitting phenomenon can be simulated.
It should be noted that the invention analyzes the communication mode, performs the brain-crack phenomenon simulation on the server cluster, and simultaneously monitors the state of the server cluster.
Specifically, in step S330, the server cluster state is monitored, the current server cluster state information is obtained, and the current server cluster state information is compared with the last server cluster state information to obtain comparison information.
In one implementation, server cluster state information may be periodically acquired during monitoring of server cluster state. The current latest acquired server cluster state information is the current server cluster state information.
Here, the server cluster state information includes, for example, one or more of arbitrated device state information and history operations in the server cluster, state information of each server node, cluster resource state information (operation state information of cluster resources). In this way, the change of the server cluster state information relative to the last time can be determined by comparing the current server cluster state information with the last time server cluster state information.
The server cluster state information of the last time is the server cluster state information before the brain fracture phenomenon simulation is performed on the server cluster (i.e., before the brain fracture phenomenon occurs in the server cluster).
In addition, if the current server cluster state information is different from the last server cluster state information, the comparison information can be written into the log to be used as a backup for the server cluster state information after being changed, and then the comparison information can be obtained from the log for analysis.
Finally, in step S340, it is determined whether the server cluster has a brain fracture phenomenon and the arbitration device performs an arbitration operation according to the comparison information, so as to determine whether the server cluster can avoid the brain fracture problem. In other words, when the server cluster is analyzed to generate the brain fracture phenomenon according to the comparison information, the arbitration device performs arbitration operation to cope with and process the brain fracture phenomenon so as to avoid the brain fracture problem of the server cluster in the production environment.
Specifically, the comparison information includes the state information of each server node in the current server cluster state information and the comparison information between the state information of each server node in the last server cluster state information, so that whether the state information of each server node (including the heartbeat information of the server node) is normal or not can be determined. According to the state (heartbeat) of each server node, whether each server node can normally communicate with other server nodes or not and whether a fault server node which cannot communicate with other server nodes exists or not can be determined, and whether the server cluster has a brain fracture phenomenon or not is determined.
In one embodiment, the arbitration operation includes an operation to power down the failed server node, such as an operation to shut down or restart the power of the failed server node. It can be understood that the comparison information further includes the state information and the history operation of the arbitration device in the current server cluster state information, and the comparison information between the state information and the history operation of the arbitration device in the last server cluster state information, so as to determine whether the arbitration device performs the arbitration operation (the operation of cutting off the power supply).
It should be noted that, when it is determined that a server cluster has a split phenomenon (a failed server node that cannot communicate with other server nodes occurs in the server cluster) according to the comparison information, whether the server cluster can avoid the split problem or not and whether the expectation of high availability of the server cluster is satisfied is determined by detecting whether the arbitration device performs a corresponding countermeasure (arbitration operation) with respect to the split phenomenon of the server cluster. In this way, by executing the method for detecting the server cluster before the server cluster is put into the production environment, whether the server cluster has the hidden trouble of brain fracture or not can be detected.
In one embodiment, computing device 200 is communicatively connected to a client. If it is determined in step S340 that the server cluster has a brain burst phenomenon and the arbitration device performs an arbitration operation, the comparison information may be output, i.e., transmitted to the client so as to be presented at the client. In this way, a user or maintenance personnel can analyze the server cluster state according to the comparison information displayed by the client.
In one embodiment, after obtaining the comparison information, the computing device 200 may further analyze the server cluster state according to the comparison information, so as to determine whether the server cluster state is normal, and generate a corresponding state analysis result. And then, the state analysis result can be sent to the client so as to display the state analysis result of the server cluster at the client for a user or maintenance personnel to check the state analysis result.
Here, the server cluster-based state information may include one or more of arbitrated device state information and historical operations in the server cluster, state information for each server node, cluster resource state information. Accordingly, determining whether the server cluster status is normal based on the comparison information may include one or more of: determining whether an arbitration device in the server cluster successfully performs a predetermined operation; determining whether the state of each server node is normal; it is determined whether the cluster resource status is normal.
Fig. 5 shows a schematic diagram of a detection device 500 according to an embodiment of the invention. The detection apparatus 500 resides in a computing device (e.g., the aforementioned computing device 200) adapted to perform the method 300 of detecting a server cluster of the present invention.
As shown in fig. 5, the detection device 500 includes a split brain simulation module 510 and an information processing module 520 coupled to each other.
The split brain simulation module 510 may obtain device information of the server cluster, and determine whether the server cluster meets a basic condition of split brain phenomenon simulation according to the device information. If the basic condition of the brain fracture phenomenon simulation is met, determining a communication mode of the server cluster, and performing corresponding brain fracture phenomenon simulation on the server cluster according to the communication mode so as to simulate the phenomenon of a fault server node which cannot communicate with other server nodes in the server cluster.
The information processing module 520 may monitor the server cluster status and obtain the current server cluster status information, and compare the current server cluster status information with the last server cluster status information to obtain comparison information. And determining whether the server cluster has the brain fracture phenomenon or not and whether the arbitration equipment executes arbitration operation or not according to the comparison information so as to determine whether the server cluster can avoid the brain fracture problem or not.
It should be noted that the split brain simulation module 510 is used for performing the steps S310 to S320, and the information processing module 520 is used for performing the steps S330 to S340. Here, specific execution logic of the brain fracture simulation module 510 and the information processing module 520 refers to the descriptions of steps S310 to S340 in the foregoing method 300, and will not be repeated here.
According to the method 300 for detecting the server cluster, before the server cluster is put into a production environment, the server cluster is subjected to brain fracture phenomenon simulation, whether the server cluster can perform arbitration operation is detected to cope with and process the brain fracture phenomenon so as to avoid the brain fracture problem of the server cluster, and accordingly whether the server cluster has the hidden danger of the brain fracture problem is determined. Specifically, when the occurrence of the brain fracture phenomenon of the server cluster is determined according to the comparison information, the arbitration device performs arbitration operation for the brain fracture phenomenon of the server cluster to process, so that whether the server cluster can avoid the brain fracture problem or not and whether the expectation of high availability of the server cluster is met is judged. In this way, by executing the method for detecting the server cluster before the server cluster is put into the production environment, whether the server cluster has the hidden trouble of brain fracture or not can be detected.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the mobile terminal will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the method of detecting a server cluster of the invention in accordance with instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.

Claims (10)

1. A method for detecting a server cluster, executed in a computing device, the computing device being communicatively connected to a server cluster, the server cluster comprising a plurality of server nodes, and an arbitration device communicatively connected to the plurality of server nodes, respectively, the method comprising the steps of:
acquiring equipment information of a server cluster, and determining whether the server cluster meets basic conditions of brain fracture phenomenon simulation according to the equipment information;
If the basic condition of the brain-split phenomenon simulation is met, determining the communication mode of the server cluster, wherein the method comprises the following steps: determining a communication tool used by a server cluster according to the type of a software package used by the server cluster, and acquiring a communication protocol and a communication transmission mode of the server cluster from a configuration file of the communication tool; the communication tool is a Corosync communication tool or a Heartbean communication tool, the communication modes comprise a communication protocol and a communication transmission mode, the communication protocol comprises one or more of a UDP protocol, a Paxos protocol and a Totem protocol, and the communication transmission mode comprises one or more of unicast, multicast and broadcast;
calling a corresponding function to close a port corresponding to the communication protocol and the communication transmission mode, so as to simulate the corresponding brain-crack phenomenon of the server cluster, and simulate the phenomenon that a fault server node which cannot communicate with other server nodes appears in the server cluster;
monitoring the server cluster state, acquiring current server cluster state information, and comparing the current server cluster state information with the last server cluster state information to obtain comparison information;
And determining whether the server cluster has a brain fracture phenomenon or not and whether the arbitration equipment executes arbitration operation or not according to the comparison information so as to determine whether the server cluster can avoid the brain fracture problem or not.
2. The method of claim 1, wherein the arbitration operation comprises an operation to power down a failed server node.
3. The method of claim 1, wherein the computing device is communicatively connected to a client, the method further comprising the steps of:
and if the server cluster is determined to have the brain-split phenomenon and the arbitration equipment executes arbitration operation, sending the comparison information to a client so as to display the comparison information on the client.
4. A method as claimed in claim 3, further comprising the step of:
determining whether the server cluster state is normal or not according to the comparison information, and generating a corresponding state analysis result;
and sending the state analysis result to the client so as to display the state analysis result at the client.
5. The method of claim 4, wherein determining whether the server cluster status is normal comprises one or more of:
determining whether an arbitration device in the server cluster successfully performs a predetermined operation;
Determining whether the state of each server node is normal;
it is determined whether the cluster resource status is normal.
6. The method of any of claims 1-5, wherein the server cluster state information includes one or more of arbitrated device state information and historical operations in a server cluster, state information for each server node, cluster resource state information.
7. The method of any of claims 1-5, wherein the device information includes arbitration device information, information for each server node, and determining from the device information whether the server cluster satisfies a basis condition for a split brain phenomenon simulation includes:
determining, according to the device information, whether the mediation device can monitor each server node in the server cluster;
if yes, determining whether the server cluster meets the vote number condition of a voting mechanism;
and if so, determining that the server cluster meets the basic condition of the brain fracture phenomenon simulation.
8. A detection apparatus residing in a computing device, the computing device communicatively coupled to a server cluster, the server cluster comprising a plurality of server nodes, an arbitration device communicatively coupled to the plurality of server nodes, respectively, the apparatus comprising:
The brain fracture simulation module is suitable for acquiring equipment information of the server cluster and determining whether the server cluster meets the basic conditions of brain fracture phenomenon simulation according to the equipment information; if the basic condition of the brain-split phenomenon simulation is met, determining the communication mode of the server cluster, wherein the method comprises the following steps: determining a communication tool used by a server cluster according to the type of a software package used by the server cluster, and acquiring a communication protocol and a communication transmission mode of the server cluster from a configuration file of the communication tool; the communication tool is a Corosync communication tool or a Heartbean communication tool, the communication modes comprise a communication protocol and a communication transmission mode, the communication protocol comprises one or more of a UDP protocol, a Paxos protocol and a Totem protocol, and the communication transmission mode comprises one or more of unicast, multicast and broadcast; calling a corresponding function to close a port corresponding to the communication protocol and the communication transmission mode, so as to simulate the corresponding brain-crack phenomenon of the server cluster, and simulate the phenomenon that a fault server node which cannot communicate with other server nodes appears in the server cluster; and
The information processing module is suitable for monitoring the server cluster state and acquiring the current server cluster state information, and comparing the current server cluster state information with the last server cluster state information to obtain comparison information; and determining whether the server cluster has a brain fracture phenomenon or not and whether the arbitration equipment executes arbitration operation or not according to the comparison information so as to determine whether the server cluster can avoid the brain fracture problem or not.
9. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-7.
10. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method of any of claims 1-7.
CN202210730202.6A 2022-04-13 2022-04-13 Detection method, detection device and computing equipment of server cluster Active CN115190046B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210730202.6A CN115190046B (en) 2022-04-13 2022-04-13 Detection method, detection device and computing equipment of server cluster

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210730202.6A CN115190046B (en) 2022-04-13 2022-04-13 Detection method, detection device and computing equipment of server cluster
CN202210381701.9A CN114500327B (en) 2022-04-13 2022-04-13 Detection method and detection device for server cluster and computing equipment

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202210381701.9A Division CN114500327B (en) 2022-04-13 2022-04-13 Detection method and detection device for server cluster and computing equipment

Publications (2)

Publication Number Publication Date
CN115190046A CN115190046A (en) 2022-10-14
CN115190046B true CN115190046B (en) 2024-01-23

Family

ID=81488529

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202210381701.9A Active CN114500327B (en) 2022-04-13 2022-04-13 Detection method and detection device for server cluster and computing equipment
CN202210730202.6A Active CN115190046B (en) 2022-04-13 2022-04-13 Detection method, detection device and computing equipment of server cluster

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202210381701.9A Active CN114500327B (en) 2022-04-13 2022-04-13 Detection method and detection device for server cluster and computing equipment

Country Status (1)

Country Link
CN (2) CN114500327B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117851300A (en) * 2022-09-30 2024-04-09 华为云计算技术有限公司 Multi-AZ arbitration system and method

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012173996A (en) * 2011-02-22 2012-09-10 Nec Corp Cluster system, cluster management method and cluster management program
CN105472022A (en) * 2015-12-24 2016-04-06 北京同有飞骥科技股份有限公司 Method and device for solving dual-computer cluster split brain
CN105934929A (en) * 2014-12-31 2016-09-07 华为技术有限公司 Post-cluster brain split quorum processing method and quorum storage device and system
CN106789193A (en) * 2016-12-06 2017-05-31 郑州云海信息技术有限公司 A kind of cluster ballot referee method and system
WO2019119263A1 (en) * 2017-12-19 2019-06-27 海能达通信股份有限公司 Distributed cluster split-brain processing method, apparatus, and device
CN110750393A (en) * 2019-09-03 2020-02-04 北京字节跳动网络技术有限公司 Method, device, medium and equipment for avoiding network service dual-computer hot standby split brain
CN212785387U (en) * 2020-05-29 2021-03-23 深圳市瑞驰信息技术有限公司 Low-cost multi-cluster arbitration system

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103684941B (en) * 2013-11-23 2018-01-16 广东中兴新支点技术有限公司 Cluster based on arbitrating server splits brain preventing method and device
CN104158707B (en) * 2014-08-29 2017-10-17 新华三技术有限公司 A kind of method and apparatus for detecting and handling cluster fissure
CN107147540A (en) * 2017-07-19 2017-09-08 郑州云海信息技术有限公司 Fault handling method and troubleshooting cluster in highly available system
CN108845865A (en) * 2018-06-28 2018-11-20 郑州云海信息技术有限公司 A kind of monitoring service dispositions method, system and storage medium
US10909008B2 (en) * 2018-12-17 2021-02-02 Ribbon Communications Operating Company, Inc. Methods and apparatus for detecting, eliminating and/or mitigating split brain occurrences in high availability systems

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012173996A (en) * 2011-02-22 2012-09-10 Nec Corp Cluster system, cluster management method and cluster management program
CN105934929A (en) * 2014-12-31 2016-09-07 华为技术有限公司 Post-cluster brain split quorum processing method and quorum storage device and system
CN106170971A (en) * 2014-12-31 2016-11-30 华为技术有限公司 Arbitration process method, arbitration storage device and system after a kind of cluster Schizencephaly
CN105472022A (en) * 2015-12-24 2016-04-06 北京同有飞骥科技股份有限公司 Method and device for solving dual-computer cluster split brain
CN106789193A (en) * 2016-12-06 2017-05-31 郑州云海信息技术有限公司 A kind of cluster ballot referee method and system
WO2019119263A1 (en) * 2017-12-19 2019-06-27 海能达通信股份有限公司 Distributed cluster split-brain processing method, apparatus, and device
CN110750393A (en) * 2019-09-03 2020-02-04 北京字节跳动网络技术有限公司 Method, device, medium and equipment for avoiding network service dual-computer hot standby split brain
CN212785387U (en) * 2020-05-29 2021-03-23 深圳市瑞驰信息技术有限公司 Low-cost multi-cluster arbitration system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
中小型指控系统的MySQL高可用集群设计与实现;张涛;《信息化研究》;47(4);第65-70页 *

Also Published As

Publication number Publication date
CN115190046A (en) 2022-10-14
CN114500327B (en) 2022-08-12
CN114500327A (en) 2022-05-13

Similar Documents

Publication Publication Date Title
CN103201724B (en) Providing application high availability in highly-available virtual machine environments
US7281040B1 (en) Diagnostic/remote monitoring by email
US10462027B2 (en) Cloud network stability
CN105357038A (en) Method and system for monitoring virtual machine cluster
US20100043004A1 (en) Method and system for computer system diagnostic scheduling using service level objectives
EP2518627A2 (en) Partial fault processing method in computer system
CN110865907B (en) Method and system for providing service redundancy between master server and slave server
CN110109782B (en) Method, device and system for replacing fault PCIe (peripheral component interconnect express) equipment
JP2002312201A (en) Processing system for log restoration measure in logically partitioned multiprocessing system, processing method and storage medium for the same
CN115190046B (en) Detection method, detection device and computing equipment of server cluster
CN113127299A (en) Server operation and maintenance method, device and system and computer readable storage medium
CN114024834A (en) Fault positioning method and device, electronic equipment and readable storage medium
CN111897697A (en) Server hardware fault repairing method and device
US10842041B2 (en) Method for remotely clearing abnormal status of racks applied in data center
US10754722B1 (en) Method for remotely clearing abnormal status of racks applied in data center
CN113032218B (en) Server fault detection method, system and computer readable storage medium
CN106294364B (en) Method and device for realizing web crawler to capture webpage
US7206975B1 (en) Internal product fault monitoring apparatus and method
US20080216057A1 (en) Recording medium storing monitoring program, monitoring method, and monitoring system
CN106411643A (en) BMC (Baseboard Management Controller) detection method and device
US20200305300A1 (en) Method for remotely clearing abnormal status of racks applied in data center
JP6317074B2 (en) Failure notification device, failure notification program, and failure notification method
CN116192885A (en) High-availability cluster architecture artificial intelligent experiment cloud platform data processing method and system
EP3974979A1 (en) Platform and service disruption avoidance using deployment metadata
WO2022009438A1 (en) Server maintenance control device, system, control method, and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant