JP2007266776A - System for monitoring normal operation of service among plurality of servers, and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for monitoring a normal operation of a service among a plurality of servers that earlier and further completely detects a failure state of a service by monitoring not only a failure of a service but also a normal operation state of a service in a service system on the premise of operation of a redundant configuration, and its method. <P>SOLUTION: The service system constructed by a plurality of the servers is composed so that communication is regularly executed to the servers opposite to each other with a UDP or a TCP and each sever in the service system is allowed to have a service normal-operation function for judging normalcy/abnormality of a service operation state by contents of a communication packet so as to mutually execute monitoring of the normal operation of a service with a redundant-configuration server. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a service normal operation monitoring system and a method for a plurality of servers, and in particular, to provide each server in a service system with a service normal operation monitoring function, and to perform service normal operation monitoring between redundant servers. The present invention relates to a service normal operation monitoring system and a method thereof among a plurality of servers.

  In recent years, the number of server systems that provide various services is increasing, and among the services that are provided, there is an increasing demand for systems that minimize service interruption time by early failure detection. Normally, in a switching system or the like, a failure of the active server is judged by detecting a failure in its own server or detecting a communication interruption using a dedicated monitoring server, and a device system that has been made redundant in advance is used. A new method of operation (system switching) has been adopted to minimize service interruption time.

  In addition, as described above, in a service system that assumes redundant configuration operation, as a method for detecting a service failure, for example, middleware checks the existence of a service providing process that operates on the middle, and processes when the process disappears. There is a function to restart.

  In addition, by performing communication communication between middleware between redundant servers, there is also a function of detecting a failure of the opposite server due to communication interruption and switching the active server.

  Furthermore, the service providing state server monitors the service monitoring external server by periodically communicating with the service providing process.

For example, if a communication unit, a service processing unit, and a process management unit are provided in the active server and the standby server, respectively, and a failure occurs in the active server, the request reception control by the communication unit and the processing by the service processing unit are performed. A technique is known in which a standby server replaces a process related to a service to be performed by an active server by taking over execution control and request distribution control by a process management unit to a standby server. (For example, see Patent Document 1)
On the other hand, a communication means for communicating between the redundant monitoring / control processors is provided, and the events that occurred are mutually notified between the monitoring / control processors, and the system status is monitored while grasping each other's status. As a control technology, a technology is also known in which a predetermined notification event is exchanged between the redundant monitoring / monitoring control processors at regular intervals, and the operations are mutually monitored. (For example, see Patent Document 2)
However, all of these conventional techniques are limited to a redundant server or a monitoring / control processor, and the execution system is limited to promptly transferring processing to the standby system when a failure occurs. Thus, there is no invention that aims to make the monitoring dedicated server unnecessary by performing regular communication and mutually judging the operation status of the service based on the contents of the communication.

In addition, there is a method to prepare an external server dedicated to service monitoring in order to monitor the normal operation status of the service. However, since a dedicated monitoring server is built in addition to the service providing server, the introduction cost and power consumption are high. There is a demerit in that it takes.
Japanese Patent Application Laid-Open No. 08-212095 JP-A-10-154085

  The present invention provides a plurality of servers that detect a service failure state earlier and more completely by monitoring not only a service failure but also a normal operation state of the service in a service system premised on the operation of a redundant configuration. It is an object to provide a service normal operation monitoring system and method therefor.

  In a server system that provides a service constructed by a plurality of servers, a first invention for solving the above-described problem is to periodically use an upper protocol of the Internet protocol with respect to a server facing from the normal operation monitoring server side. A communication means for performing communication, a check means for checking a service normal operation state based on monitoring contents registered in advance at the opposing normal operation monitored server that has accepted the communication, and a service according to the check result Service normal operation monitoring means for determining normal / abnormal operation status.

  According to the first aspect of the present invention, there is provided a service normal operation monitoring system among a plurality of servers capable of detecting a service failure state more quickly and more completely by monitoring not only a server failure but also a normal service operation. Can be provided.

  According to a second invention, in the service normal operation monitoring system among a plurality of servers described in the first invention, each server is provided with the communication means, the check means, and the service normal operation monitoring means.

  According to the second aspect of the present invention, a service normal operation monitoring system between a plurality of servers that does not require a dedicated monitoring server can be provided by performing service normal operation monitoring with redundant configuration servers.

  According to a third aspect of the present invention, in the service normal operation monitoring system between a plurality of servers described in the first aspect, the communication unit performs communication by setting a plurality of communication routes set in each server.

  According to the third aspect of the invention, it is possible to provide a service normal operation monitoring system between a plurality of servers that minimizes anomaly detection other than a service operation such as a hardware failure or a packet loss on a communication route.

  According to a fourth aspect of the present invention, there is provided a service normality operation monitoring method for a server system that provides a service constructed by a plurality of servers. A step of performing communication, a step of checking a service normal operation state based on monitoring contents registered in advance at the opposite normal operation monitored server that has accepted the communication, and a service operation state according to the check result Service normal operation monitoring step for determining normal / abnormal of the service.

  According to the fourth aspect of the present invention, there is provided a service normal operation monitoring method between a plurality of servers capable of detecting a service failure state more completely at an early stage by monitoring not only a server failure but also a normal service operation. Can be provided.

  According to a fifth aspect of the present invention, there is provided a step of periodically communicating with a server opposed to a computer from the normal operation monitoring server side using a higher-order protocol of the Internet protocol, and a normal normal operation monitored server receiving the communication. A program for executing a service normal operation monitoring step for checking a service normal operation state based on monitoring contents registered in advance and a service normal operation monitoring step for determining normality / abnormality of the service operation state according to the check result is there.

  According to the fifth aspect of the present invention, it is possible to provide a program capable of detecting a service failure state more completely at an earlier stage by monitoring not only a server failure but also a normal operation of a service.

As described above, according to the service normal operation monitoring system and the method among a plurality of servers of the present invention,
(1) By monitoring the normal operation of the service between servers in the service system without requiring a dedicated monitoring server, the capital investment cost can be minimized.
(2) By using all the plurality of communication routes set in the server and checking the normal operation of the service, it is possible to improve the accuracy of identifying a service failure or a hardware failure.
(3) By obtaining a service normal operation monitoring trigger from an external server other than the monitoring target, periodic operation monitoring can be performed without being affected by the failure / delay of the soft clock of the monitoring target server.
(4) The monitoring source server periodically receives the software clock time from the monitored server and compares the time difference, thereby detecting a service abnormal state due to a clock failure or a software slowdown factor.
(5) By providing an interface for registering / deleting (starting / stopping monitoring) the service normal operation monitoring program on each server, the monitoring trigger for service normal operation can be changed dynamically without going through server restart etc. In addition, it is possible to increase / decrease hardware and software online and respond to failures.
(6) By preparing a program for confirming the normal operation of an application service in the monitored server and providing an interface for registering the service to be confirmed for the program, the contents of the monitored service can be dynamically changed.
(7) Change the monitoring method dynamically according to the server configuration (single / duplex etc.) by exchanging the monitoring information periodically when the server is disconnected / installed when adding / removing servers it can.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, in order to make an understanding easy through the whole figure, the same code | symbol shall be attached | subjected and shown to the same location.

  FIG. 1 is a configuration diagram of a service normal operation monitoring system among a plurality of servers according to an embodiment of the present invention. In the figure, reference numerals 1 and 2 denote an AAA network and a BBB network, respectively, which constitute a LAN to which a plurality of hubs are connected. Reference numeral 10 denotes a monitored server that provides an arbitrary service, and reference numeral 11 denotes a monitoring source server that provides an arbitrary service. Reference numeral 20 denotes a program for controlling normal service operation confirmation, and has a registration mechanism for starting operation confirmation and ending operation confirmation for the application 30 and the operation system 40. Further, the normal operation of the service is confirmed according to the contents registered for the operation confirmation at the reception destination of the monitoring notification from the program 21, and the result is returned to the program 21. If necessary, restore the system that is determined to be abnormal. Reference numeral 21 denotes a program for controlling normal service operation monitoring, which registers monitoring start and monitoring end for the program 20 on the server 10, performs monitoring, and reacts based on the monitoring result. Reference numeral 30 denotes an application providing an arbitrary service. The program 20 registers the start and end of normal operation confirmation of its own service, and periodically checks whether the service is operating normally. Reference numeral 40 denotes an operation system, which is a platform on which an application providing an arbitrary service operates. Depending on the contents registered in the program 20, it may be checked whether it is operating normally. Reference numerals 50 and 51 denote hubs (HUBs) belonging to the AAA network. Reference numerals 52 and 53 denote hubs (HUBs) belonging to the BBB network. A magnetic disk device 60 is used by the program 20, the application 30, and the like for reading and writing data. Here, a config file in which service normality operation confirmation contents are registered is stored. A magnetic disk device 61 is used by the program 21, the application 31, and the like for reading and writing data. Here, a file in which the service 21 that has detected a service operation abnormality of the program 20 writes the syslog <syslog> is stored. Syslog <syslog> is a system log output function provided in various UNIX (registered trademark). Reference numerals 70 and 71 are system monitoring facilities (SCF), and 72 is an inter-SCF connection interface (RCI: Remote Cabinet interface).

  FIG. 2 is an explanatory diagram (part 1) of a normal service operation according to an embodiment of the present invention, and FIG. 3 is an explanatory diagram (part 2) of a normal service operation according to an embodiment of the present invention.

  4 is a time chart at the time of normal service operation in one embodiment of the present invention, and shows a case of a service normal operation monitoring trigger from the server 11 in the duplex server configuration in the service normal operation monitoring system of FIG. . The operation of FIG. 4 will be described below with reference to FIGS. 2 and 3.

Operation (1)
First, the monitoring notification is registered in the program 20 that controls the service normal operation confirmation from the application 30 of the server 10 that is the monitoring target server. The registration trigger is accepted regardless of whether it is in the initial setting or in operation, and the registration method may be any method of application interface function call (API: application programming interface) / configuration setting value reading by command activation. The contents to be registered are the protocol used for API notification (UDP / TCP / msg communication) / information necessary for using the protocol (port number, message queue ID, etc.) / Notification timing (cycle) / content to be notified / normal Response contents at the time / Reaction at the time of abnormality (process restart, server restart, etc.). It is also possible to register the OS to issue commands / execute processes. The program 20 holds the registered contents in the memory and waits for communication from the program 21 thereafter.

Operation (2)
If there is a registration for monitoring start in the program 21 that monitors the normal operation of the service of the server 11 that is the monitoring source server, UDP (user datagram protocol) or TCP (TCP), which is an upper protocol of the IP, is used using the communication route of the AAA network. Transmission control protocol)) periodically communicates with the program 20 of the server 10. When monitoring stop is registered, communication with the program 20 is disconnected. The monitoring start / stop registration trigger to the program 21 is accepted regardless of whether it is in the initial setting or in operation, but the communication start (monitoring valid) trigger of the program 21 after monitoring start registration is after the server 11 is in the operating state. . The reason is to prepare for operation server conversion by server system switching at the time of server 10 abnormality detection. The registration method may be either application interface function call (API) / command activation.

Operation (3)
The program 20 that has received communication from the program 21 notifies the application 30 according to the monitoring contents registered so far, and checks the normal operation state of the service from the response contents from the application 30. When there is an interface confirmation registration with the operation system 40 at the time of registration, the designated command or program is executed and the process from execution to normal termination is confirmed. If the check result is normal, the program 20 collects the software clock time of its own server, and returns a response to the program 21 together with the service normal operation state check result.

Operation (4)
When the program 21 confirms that the check result is normal, the software clock time of the server 10 collected by the program 20 is saved, and the program 21 communicates again with the program 20 to recollect the software clock time. Compare the soft clock time that was previously responded with the time collected at the time of re-communication and check if the time exceeds the threshold, and if there is no problem, the service is considered to be operating normally, and the monitoring cycle Communication (monitoring) is performed again periodically after the elapse of time.

  FIG. 5 is an explanatory diagram at the time of service abnormal operation according to an embodiment of the present invention, and shows the flow of operations in the case of an application 30 abnormality, an operation system 40 abnormality, slowdown detection, and a communication timeout.

  FIG. 6 is a time chart (part 1) at the time of service abnormal operation according to the embodiment of the present invention, and shows a case of an abnormality in the application 30. The operation of FIG. 6 will be described below with reference to FIG.

Operation (5)
If the check result response from the application 30 is abnormal (NG response) or no response, the program 20 tries to restore the service to the application 30 by the designated reaction. At that time, the maintenance person is notified of the abnormality by the SNMP trap or the syslog, but the response to the program 21 is not returned. When there is an interface confirmation registration with the operation system 40 at the time of registration, the designated command or program is executed and the process from execution to normal termination is confirmed. If the check result is normal, the program 20 collects the software clock time of its own server, and returns a response to the program 21 together with the service normal operation state check result.

  FIG. 7 is a time chart (part 2) at the time of service abnormal operation according to the embodiment of the present invention, and shows a case where the operation system 40 is abnormal. The operation of FIG. 7 will be described below with reference to FIG.

Operation (5 ')
If the execution result of the command / program from the operation system 40 is abnormal (NG response) or no response, the program 20 tries to restore the service to the operation system 40 by the designated reaction. At that time, the maintenance person is notified of the abnormality by the SNMP trap or the syslog, but the response to the program 21 is not returned.

  FIG. 8 is a time chart (part 3) at the time of service abnormal operation in one embodiment of the present invention, and shows a case of slowdown detection. The operation of FIG. 8 will be described below with reference to FIG.

Operation (6)
The program 21 compares the previously received software clock time with the time collected at the time of re-communication, and if there is a difference greater than a certain value, it is determined that the program 20 has been slowed down, and the maintenance person is notified of the SNMP. An error is notified by a trap or syslog. Also, if there is a hardware power control device (in the example, SCF 70, 71 and RCI 72), the server that has become abnormal is stopped, and the service is automatically restored by switching the server itself to an operational server.

  FIG. 9 is a time chart (part 4) at the time of service abnormal operation in the embodiment of the present invention, and shows a case of communication timeout. The operation of FIG. 9 will be described below with reference to FIG.

Operation (7)
In the program 21, the communication response is not returned from the program 20, and when the response from the AAA network times out, the program 20 of the server 10 is again used by using the BBB network which is another communication route set in the server 11. Attempt to communicate (monitor). That is, the above-described operation (4), operation (5), and operation (6) are performed using the BBB network. As a result, if both networks are NG (response timeout), it is determined that the server 10 is abnormal.

  FIG. 10 is a data configuration diagram showing contents of monitoring notification registration in one embodiment of the present invention, (a) is a data configuration diagram of application monitoring notification registration contents, and (b) is data of operation system monitoring notification registration contents. It is a block diagram.

  In the figure, APL / OS monitoring type (selection) indicates the monitoring type of Application communication (application communication) or Operation system (operation system), and APL communication is between the normal operation monitoring program and the application. Communication is performed using a designated method (msg / UDP / TCP, etc.). The msg communication is message communication (message communication), and is a communication method via a message queue implemented in an OS such as Solaris / Linux. The msg queue ID is a common ID number recognized by communication users when performing message communication. The communication user side designates this ID number and packs information in the message queue, and the receiving user side designates this ID number and takes out the information from the message queue, whereby mutual communication between users can be realized. The ms unit of the notification response timeout time indicates a unit of 1 / 1000th of a second.

  FIG. 11 is a software configuration diagram (on the normal operation monitored server side) in an embodiment of the present invention. In the figure, the normally monitored server side is the server 10 in a dual server configuration. That is, in this case, the service normal operation monitoring trigger from the server 11 is used, but the monitoring trigger from the server 10 may be used. .

  The application process 101 and the library unit 103 exist in the application 30, and the monitoring notification registration method is an application interface function call by the monitoring notification registration (user) 102 and the monitoring start registration accepting unit (API) 104, or monitoring notification. This is performed by either reading the configuration setting value based on the command activation by the registration unit (command) 111. The normal operation monitoring program process 105 exists in the program 20 and the service normal operation check execution unit (normal / abnormal determination) 107 checks the service normal operation, and the determination result and others are registered contents (memory) 114. Registered in Further, the history of the reaction unit 113 at the time of abnormality detection is also registered in the registered content (memory) 114. Reference numeral 106 denotes a service normal operation result reception unit, 108 denotes an OS command execution unit, 109 denotes an application communication unit, 110 denotes a monitoring notification registration unit, 112 denotes a service normal operation check notification reception, and a response unit.

  FIG. 12 is a software configuration diagram (normal operation monitoring server side) in an embodiment of the present invention. In the figure, the normal operation monitoring server side is the server 11 in a dual server configuration. That is, in this case, the service normal operation monitoring trigger from the server 11 is used, but the monitoring trigger from the server 10 may be used as described above.

  The application process 101 ′ and the library unit 103 ′ exist in the application 31, and the monitoring start registration method is an application interface function call by the monitoring start registration (user) 115 and the monitoring start registration acceptance unit (API) 116, or This is performed by any of the command activations by the monitoring start registration unit (command) 118. The service normal operation monitoring unit (periodic communication, network selection) 119 exists in the program 21, and the determination result and the like by the service normal operation result determination unit 120 are registered in the registration content (memory) 114 '. In addition, the history of the abnormality detection reaction unit 121 is also registered in the registered content (memory) 114 ′. Reference numeral 117 denotes a monitoring start registration unit.

  The following additional notes are further disclosed with respect to the embodiment including the above examples.

  (Supplementary Note 1) In a server system that provides a service constructed by a plurality of servers, a communication unit that periodically communicates with a server facing from the normal operation monitoring server side using an upper protocol of the Internet protocol, and the communication In the opposite normal operation monitored server that received the service, the check means for checking the service normal operation state based on the monitoring contents registered in advance, and the normality / abnormality of the service operation state is determined according to the check result A service normal operation monitoring system among a plurality of servers, comprising service normal operation monitoring means.

  (Supplementary Note 2) In the service normal operation monitoring system between a plurality of servers according to Supplementary Note 1, each server is provided with the communication means, the check means, and the service normal operation monitoring means. Service normal operation monitoring system.

  (Supplementary note 3) In the service normal operation monitoring system between multiple servers according to supplementary note 1, the communication means performs communication by setting a plurality of communication routes set in each server. Service normal operation monitoring system.

  (Supplementary Note 4) In the service normality operation monitoring method of a server system that provides a service constructed by a plurality of servers, the normal operation monitoring server side periodically communicates with the opposite server using the higher level protocol of the Internet protocol. A step of checking the service normality operation state based on the monitoring contents registered in advance at the opposing normality operation monitored server that has accepted the communication, and the normality of the service operation state according to the check result A service normal operation monitoring method among a plurality of servers, comprising: a service normal operation monitoring step for performing an abnormality determination.

  (Supplementary Note 5) In the step of periodically performing communication with the upper protocol of the Internet protocol to the server facing the computer from the normal operation monitoring server side, and in the opposing normal operation monitored server that has received the communication, A program for executing a step of checking a service normal operation state based on monitoring contents registered in advance, and a service normal operation monitoring step of determining normal / abnormal of the service operation state according to the check result.

  (Supplementary note 6) In the service normal operation monitoring system between the plurality of servers according to supplementary note 1, a collection unit that periodically collects soft clock time from the monitored server at the monitoring source server, and a time difference by the collection unit A service normal operation monitoring system among a plurality of servers, comprising a detecting means for detecting a failure by comparing the two.

  (Supplementary note 7) In the service normal operation monitoring system between a plurality of servers described in supplementary note 1, further comprising an interface for registering or deleting a service normal operation monitoring program in each server, normal service between a plurality of servers Operation monitoring system.

  (Supplementary Note 8) In the service normal operation monitoring system between the plurality of servers described in Supplementary Note 1, the plurality of servers, further comprising an interface for registering service contents for confirming the normal service operation of the application in the monitored server. Service normal operation monitoring system between.

  (Supplementary Note 9) In the service normal operation monitoring system between the plurality of servers described in Supplementary Note 1, the system includes a means for periodically exchanging monitoring information after a redundant server having a redundant configuration is incorporated. Service normal operation monitoring system.

  The present invention can be used for monitoring normal operation of a service in a server system that provides a service constructed by a plurality of servers.

1 is a configuration diagram of a service normal operation monitoring system among a plurality of servers in an embodiment of the present invention. FIG. It is explanatory drawing (the 1) at the time of service normal operation in one Embodiment of this invention. It is explanatory drawing (the 2) at the time of service normal operation in one Embodiment of this invention. It is a time chart at the time of service normal operation in one embodiment of the present invention.

It is explanatory drawing at the time of service abnormal operation in one Embodiment of this invention. It is a time chart (the 1) at the time of service abnormal operation in one embodiment of the present invention. It is a time chart (the 2) at the time of service abnormal operation in one Embodiment of this invention. It is a time chart (the 3) at the time of service abnormal operation in one Embodiment of this invention. It is a time chart (the 4) at the time of service abnormal operation in one Embodiment of this invention. It is a data block diagram which shows the monitoring notification registration content in one Embodiment of this invention. It is a software block diagram (normal operation monitored service side) in an embodiment of the present invention. It is a software block diagram (normal operation monitoring service side) in an embodiment of the present invention.

Explanation of symbols

1 AAA network 2 BBB network 10, 11 Server 20, 21 Program 30, 31 Application 40 Operation system 51, 52, 53, 54 Hub 60, 61 Magnetic disk unit 70, 71 SCF (System Control Facility)
72 RCI (Remote Cabinet Interface)
101 Application process 102 Monitoring notification registration (user)
103 Library 104 Monitoring Notification Registration Accepting Unit (API)
105 Normal operation monitoring program process 106 Service normal operation result receiving unit 107 Service normal operation check execution unit (normal / abnormal determination)
108 OS command execution unit 109 Application communication unit 110 Monitoring notification registration unit 111 Monitoring notification registration unit (command)
112 Service normal operation check notification reception, response unit 113 abnormality detection reaction unit 114 registered content (memory)
115 Monitoring start registration (user)
116 Monitoring start registration receiving part (API)
117 Monitoring start registration unit 118 Monitoring start registration unit (command)
119 Service normal operation monitoring unit (periodic communication, network selection)
120 Service normal operation result determination unit 121 Reaction unit when abnormality is detected

Claims (5)

In a server system that provides services constructed from multiple servers,
A communication means for periodically communicating with the upper server of the Internet protocol to the server facing from the normal operation monitoring server side,
Checking means for checking the service normality operating state based on the monitoring contents registered in advance at the opposing normality operation monitored server that has accepted the communication, and normal / abnormal judgment of the service operating state according to the check result Service normal operation monitoring means for performing,
A service normal operation monitoring system among a plurality of servers. In the service normal operation monitoring system between a plurality of servers according to claim 1,
A service normal operation monitoring system among a plurality of servers, wherein each server has the communication means, the check means, and the service normal operation monitoring means. In the service normal operation monitoring system between a plurality of servers according to claim 1,
A service normal operation monitoring system between a plurality of servers, wherein the communication means performs communication by setting a plurality of communication routes set in each server. In a service normality operation monitoring method of a server system that provides a service constructed by a plurality of servers,
A step of periodically communicating with the upper server of the Internet protocol to the server facing from the normal operation monitoring server side,
At the opposing normal operation monitored server that has accepted the communication, a step of checking the service normal operation state based on the monitoring contents registered in advance, and a normal / abnormal determination of the service operation state according to the check result A service normal operation monitoring step to be performed;
A service normal operation monitoring method among a plurality of servers. A step of periodically communicating with a server facing the computer from the normal operation monitoring server side using an upper protocol of the Internet protocol,
At the opposing normal operation monitored server that has accepted the communication, a step of checking the service normal operation state based on the monitoring contents registered in advance, and a normal / abnormal determination of the service operation state according to the check result A service normal operation monitoring step to be performed;
A program for running

Images