JP4896573B2 - Fault monitoring system and method, and program - Google Patents

Description

  The present invention relates to a failure monitoring technique for monitoring the occurrence of a failure in a monitored computer system including a plurality of monitored computers connected via a network.

  With the recent advancement of network technology, computer systems composed of a plurality of computers connected via a network are used in various fields such as plant monitoring. In such a computer system, the functions of the system are handled by a plurality of computers. Therefore, even if a failure occurs in any one of the computers, the entire computer system may be affected. For this reason, various fault monitoring techniques have been developed in order to monitor the occurrence of faults using a computer system as a monitoring target.

  Conventionally, as a monitoring method of a computer system, there is a method of collecting more detailed logs to detect anomalies by simultaneously collecting and comparing logs collected from multiple computers and verifying the cause of the anomaly Exists (see, for example, Patent Document 1). In addition, the monitoring target log of the monitoring target computer is received, the log necessary for failure analysis is identified based on the log and information related to the event that indicates the occurrence of failure or performance degradation of the monitoring target, and the failure analysis is performed There is a method to do this (for example, see Patent Document 2).

JP 11-143738 A JP 2004-178336 A

  The conventional computer system fault monitoring technology as described above is to verify the cause by using the log of the monitored computer when the occurrence of a fault in the computer system is detected. The following problems exist.

  First, when the occurrence of a failure in a computer system is detected, it may be difficult to identify the cause from the surfaced failure only by verifying the cause. That is, in a failure that is surfaced by a computer, there is an increasing number of cases where a function that is not related to the problem is affected and surfaced as a result of a certain problem having a secondary or tertiary influence. Of these secondary and tertiary problems, especially those that gradually affect over time, it may be difficult to identify the cause from the details of the failure that has occurred. .

  In addition, recent computers are composed of a lot of hardware and software, but these hardware and software are becoming more and more generalized and can be used in combination with those manufactured by many manufacturers. Yes. For such a lot of hardware and software, all the matters such as whether an unknown problem is inherent, and if it is, whether it affects the computer system, are inspected and verified in advance. It is difficult to do.

  Furthermore, in the process of operating a computer system for a long period of time, it is not possible to use the same when replacing faulty hardware, and it is necessary to use a successor or other products. It is difficult to inspect and verify all such matters in advance.

  Therefore, it is required not only to verify the cause when a failure occurs, but also to predict the future failure. However, in order to predict a future failure by applying the conventional technology for verifying the cause using the log of the monitored computer as described above, it is necessary to collect all the logs of the monitored computer. In this case, there is a problem that an excessive load is applied to the network.

  The present invention has been proposed to solve the above-described problems of the prior art, and its purpose is to cause the occurrence of a failure in the monitored computer system without imposing an excessive load on the network. To provide a fault monitoring system, method, and program capable of predicting the occurrence of a fault in addition to estimation.

  In order to achieve the above object, the present invention edits a change in the configuration or state of a computer obtained from collected log information as time series data in an information collecting apparatus, and the same event in log information. By replacing the content data with the abbreviated data, while comparing the amount of data to be communicated, the state monitoring / analysis server compares the analysis information of the old and new time series data including the computer configuration and state change information. It is possible to predict the occurrence of a failure.

  The fault monitoring system of the present invention is a fault monitoring system that monitors the occurrence of a fault in a monitored computer system composed of a plurality of monitored computers connected via a network. Information collection device that collects log information, edits and saves it, analyzes the log information saved by the information collection device, acquires analysis information, and estimates the cause when a failure occurs based on the log information and analysis information And a state monitoring / analysis server for predicting failure occurrence. The information collection device and the state monitoring / analysis server further have the following characteristics.

  The information collection device has data communication means, data editing means, and log storage means. Here, the data communication means is means for receiving log information of the computer from the monitored computer and communicating data including log information with the state monitoring / analysis server. The data editing means compares the configuration information or status information of the monitored computer included in the received log information with each corresponding information in the past, and if there is a change in configuration or status, the change occurs when an event occurs. And when the received event information includes the same event content data, it is replaced with the simplified data representing the event content data. The log storage means is means for storing the edited log information.

  The state monitoring / analysis server includes data communication means, data storage means, data classification means, log analysis means, failure cause estimation means, and failure occurrence prediction means. Here, the data communication means is means for communicating data including log information with the information collecting apparatus. The data storage means is means for storing the received log information and various data acquired in the server. When the received log information includes summary data, the data classification means performs editing to restore the summary data to the original event content data, and includes information indicating the occurrence of the failure in the log information. It is means for classifying data as failure information and storing it in a data storage means. The log analysis unit is a unit that analyzes the occurrence status of the log on the time axis using the received log information to acquire the analysis information, and stores the acquired analysis information in the data storage unit.

  The failure cause estimation means detects a failure cause candidate from the failure information when a similar failure occurred in the past, and calculates the certainty of the possibility that the failure is caused by the failure information indicated by the failure information. The failure cause estimation result obtained according to the calculation result is stored in the data storage unit as a part of failure information indicating the occurrence of the failure. The failure occurrence predicting means determines whether or not past analysis information is similar to the analysis information for a given period by comparing the data on the time axis. Means for calculating the certainty of the possibility of occurrence of the failure indicated by the failure information using the failure information corresponding to the analysis information, and storing the prediction result obtained according to the calculated result in the data storage unit as the prediction information It is.

  The fault monitoring method and fault monitoring program of the present invention grasp the characteristics of the system from the viewpoints of the method and the computer program.

  According to the present invention having the above-described features, in the information collecting apparatus, not only the original time-series data indicating the occurrence of various events but also the configuration information or status information of the monitored computer in the received log information. When a change is detected in comparison with past information, this change can be handled as an occurrence of an event and edited as time-series data indicating the occurrence of a change event by adding a time.

  Therefore, in the status monitoring / analysis server, in addition to the original time-series data indicating the occurrence of various events, changes related to the configuration and status of monitored computers are also time-series edited by regarding the changes as occurrences of events. Since data can be used, all time-series data including time-series data indicating information on the configuration and state of such a computer can be analyzed, and the analysis information can be compared between new and old. Then, by comparing old and new analysis information including computer configuration and status information, past analysis information with similar configuration and status is accurately detected, and failure information is generated using failure information corresponding to the analysis information. Can be accurately predicted.

  Further, in the information collection device, when the collected event information includes the same event content data, the event content data can be replaced with summary data and edited. Therefore, the amount of data transmitted from the information collection device to the state monitoring / analysis server can be reduced, and an excessive load is not applied to the network.

  According to the present invention, a fault monitoring system, method, and program capable of predicting the occurrence of a fault in addition to estimating the cause when a fault occurs in a monitored computer system without imposing an excessive load on the network Can be provided.

[First Embodiment]
[Fault monitoring system configuration]
FIG. 1 is a block diagram showing a configuration of a failure monitoring system according to a first embodiment to which the present invention is applied.

  As shown in FIG. 1, the failure monitoring system according to the present embodiment is configured by connecting an information collection device 101 and a state monitoring / analysis server 102 via the Internet 100. The information collection apparatus 101 is connected to each of a plurality of monitoring target computers 202 constituting the monitoring target computer system 201 via a network 203. In addition, a monitoring terminal 104 in the same facility is connected to the state monitoring / analysis server 102 via a monitoring intranet 103, and a remote or mobile monitoring terminal 105 is connected via the Internet 100. .

  In FIG. 1, three monitoring target computers 202 are shown as the computers constituting the monitoring target computer system 201. However, the number of computers constituting the monitoring target computer system 201 that is the subject of the present invention is as follows. There may be four or more, or two or less, or a system connected via a plurality of networks.

  Further, the Web site 106 connected to the Internet 100 in FIG. 1 exemplarily shows one Web site that provides information related to the monitoring target computer 202 on the Web page 161.

  The information collection device 101 is a device that collects, edits and saves log information 301 of the computer stored in each monitoring target computer 202 constituting the monitoring target computer system 201. Means 112 and log storage means 113.

  The data communication unit 111 is a unit that receives log information 301 of the computer from the monitoring target computer 202 and communicates data including log information with the state monitoring / analysis server 102. The data editing unit 112 compares the configuration information or status information of the monitoring target computer 202 included in the received log information with each corresponding information in the past. And editing as time-series data with the addition of time, and when the received log information 301 includes the same event content data, means for editing by replacing the data with summary data representing the event content data It is. The log storage unit 113 is a unit that stores the edited log information 311.

  The state monitoring / analysis server 102 analyzes the log information 311 stored in the information collecting apparatus 101 to obtain analysis information, and also estimates the cause of failure occurrence and prediction of failure occurrence based on the log information and analysis information. And a data communication unit 121, a data storage unit 122, a data classification unit 123, a log analysis unit 124, a failure cause estimation unit 125, and a failure occurrence prediction unit 126.

  The data communication unit 121 is a unit that communicates data including the log information 311 with the information collection apparatus 101. The data storage unit 122 is a unit that stores various data such as normal log information 321, failure information 331, analysis information 341, and prediction information 351 acquired in the state monitoring / analysis server 102. When the received log information 311 includes summary data, the data classification unit 123 performs editing to restore the summary data to the original event content data, and the log information 311 is converted into normal log information 321. Data is classified into failure information 331 indicating the occurrence of a failure and stored in the data storage unit 122.

  In the following description, “log information 301” means log information before being edited in the information collection apparatus 101, and “log information 311” is after being edited in the information collection apparatus 101. Log information including time-series data indicating a change in configuration or status. Since “normal log information 321” and “failure information 331” are classifications of “log information 311”, “normal log information 321” and “failure information” are simply referred to as “log information”. "Log information 311" including "331".

  The log analysis unit 124 is a unit that analyzes the occurrence status of the log on the time axis using the received log information 311 to acquire the analysis information 341, and stores the acquired analysis information 341 in the data storage unit 122. The failure cause estimating means 125 detects a failure cause candidate from the failure information 331 when a similar failure has occurred in the past, with the occurrence of the failure indicated by the failure information 331, and is convinced that the candidate may be the cause. This is a means for calculating the degree, and storing in the data storage unit 122 the failure cause estimation result obtained according to the calculation result as part of the failure information 331 indicating the occurrence of the failure.

  The failure occurrence predicting means 126 determines whether or not the past analysis information 341 is similar to the analysis information 341 for a given period by comparing the data on the time axis, and when determining that they are similar Then, the failure information 331 corresponding to the past analysis information 341 is used to calculate the certainty of the possibility of occurrence of the failure indicated by the failure information 331, and the prediction result obtained according to the calculation result is used as the prediction information 351. It is means for storing in the data storage means 122.

  In FIG. 1, as an example, a case where one monitoring terminal 104 is connected via the monitoring intranet 103 is shown, but one or more monitoring terminals 104 are connected to the state monitoring / analysis server 102. You may connect directly and you may combine the monitoring terminal 104 of both connection systems. Similarly, in FIG. 1, as an example, a case where a remote or mobile monitoring terminal 105 is connected via the Internet 100 is shown, but the monitoring terminal 105 connected via the Internet 100 is A plurality of units may be provided. Further, in the present invention, such a remote or mobile monitoring terminal 105 is not an essential configuration.

  Further, FIG. 1 shows a case where only one information collection device 101 and one state monitoring / analysis server 102 are provided, but a plurality of information collection devices 101 and multiple state monitoring / analysis servers 102 are provided. You may comprise. That is, since they are connected via the Internet 100, an arbitrary number of information collection devices 101 can be freely connected to an arbitrary number of state monitoring / analysis servers 102, and the degree of freedom of the fault monitoring system configuration is high.

  The information collecting apparatus 101 as described above is realized by controlling a personal computer or various high-performance computers with software specialized for information collection. The state monitoring / analysis server 102 is realized by controlling a large-capacity, high-performance computer for a server with software specialized for the state monitoring / analysis server. Various instructions and input data are received from the person in charge through 104 and 105, and various data processing results are displayed or output to the person in charge.

[Configuration of monitored computer system]
Below, the general structure of the monitoring object computer system 201 is demonstrated with reference to FIGS.

  FIG. 2 is a block diagram illustrating a configuration example of the monitoring target computer 202, where (a) illustrates the hardware configuration 202 and (b) illustrates the software configuration 202.

  As illustrated in FIG. 2A, the computer is configured by connecting a plurality of devices (hardware) such as a CPU 212, a memory 213, a disk 214, and a communication interface 215 through a bus 211. Note that the devices shown in FIG. 2A are only basic devices that constitute the computer, and generally, a wide variety of other devices are used.

  By the way, many of these devices are generalized and can be manufactured by each company using its own technology. Select the devices to be used from various viewpoints such as price, performance, and functions. It is possible to use. In addition, in the case of performance improvement, addition of functions, replacement at the time of failure, it is also possible to add new equipment or replace with a product of a different manufacturer.

  FIG. 2B shows an example of the software configuration when communication is performed via the communication interface 215 shown in FIG. 2A as an example of the software configuration of the computer. This figure shows a case where the application 222 communicates with the communication interface 215 via the interface 223 and the driver 224 under the OS (basic software) 221.

  The software configuration shown in FIG. 2B is a driver that depends on the hardware even when a new device is added to the hardware configuration shown in FIG. The software configuration can minimize the influence on the application 222 by switching the H.224 or the interface 223.

  In general, each software, that is, OS 221, application 222, interface 223, driver 224, etc. as shown in FIG. 2B outputs log information 301 corresponding to the operation and processing contents, and the own computer It is supposed to save in.

  FIG. 3 is a data configuration diagram showing an example of the log information 301 of the own computer stored in the computer 202 as shown in FIG. The log information 301 shown in FIG. 3 is information that is automatically recognized by using a recognition function that a computer has as a standard, and includes configuration information 302, status information 303, and event log information 304.

  The configuration information 302 is information indicating the configuration of each piece of hardware and software that constitutes the computer 202, and includes hardware configuration information 302a and software configuration information 302b.

  As the configuration information 302, for example, items such as a product name, a serial number, a manufacturing date, a model, and a version are stored as information for specifying the hardware and software. Note that the configuration information does not necessarily need to be information that specifies hardware or software in detail. For example, the configuration information may be simply the name of the hardware or software, or only the date and time when the hardware or software was incorporated into the computer. It doesn't matter.

  The status information 303 is information indicating the status of each piece of hardware and software that constitutes the computer 202 and its peripheral devices, and includes hardware status information 303a and software status information 303b.

  As the status information 303, for example, information such as the load of the CPU 212 for 1 second in the case of the CPU 212, the memory usage at that time in the case of the memory 213, and the data transfer amount of 1 second in the case of the disk 214 are stored. Is done.

  Note that the latest information at the present time is generally stored as the configuration information 302 and the state information 303 as described above. That is, for example, when the communication interface 215 is replaced with another product, the information before replacement is discarded from the log information 301 and the information after replacement is added to the log information 301. However, when there is a sufficient storage capacity for the log information 301, it is not always necessary to discard the configuration information 302 and the state information 303 before replacement, and information for a certain period may be stored.

  On the other hand, the event log information 304 is information in which events (occurrence of events) detected by the own computer 202 are stored in time series, and the event log information 304 in the example of FIG. 3 is the OS log information 304a output by the OS 221. And application log information 304b output by the application 222.

  Examples of the event log information 304 include information indicating various events such as information indicating that hardware has been accessed and information indicating that some kind of abnormality has been detected in hardware or software. Is added each time is detected.

  2 and 3 show a case in which the log information 301 of the monitoring target computer 202 is stored in the own computer 202, the log information 301 is output to another computer or apparatus via the communication interface 215. The log information may be stored by the other computer or device.

  As shown in FIG. 2 and FIG. 3, the monitoring target computer 202 that stores the log information 301 stores the configuration, current state, and past event information of the local computer as a part of the log information 301. Using these pieces of information, it is possible to detect an abnormality in the monitored computer system 201. Further, in order to detect an abnormality in the monitoring target computer system 201, it is not necessary to have all of the configuration information 302, the status information 303, and the event log information 304 as the log information 301. Anomalies can be detected using only

  FIG. 4 is a block diagram illustrating a functional configuration example of the monitoring target computer system 201 in which functions are multiplexed as an example of the monitoring target computer system 201 illustrated in FIG. 1.

  In the example shown in FIG. 4, in the monitoring target computer system 201 having a plurality of functions A to F, when the functions A to F are arranged in the three computers 2021 to 2023, some of the functions are at least two or more. When an abnormality occurs in one computer, the function of the computer in which the abnormality has occurred can be replaced by another normal computer.

  For example, when an abnormality occurs in the first computer 2021 and the function A (401a), function B (402a), and function C (403a) of the computer 2021 cannot be executed, the function A (401a) of the computer 2021 is used. ) And function B (402a) are replaced by the function A (401b) and function B (402b) of the second computer 2022, and the function C (403a) of the computer 2021 is the function of the third computer 2023. Substitution processing is performed in C (403b).

  Thus, when the multiplex configuration of the monitored computer system 201 as shown in FIG. 4 is applied to, for example, a plant monitoring system that performs plant monitoring or control, an abnormality occurs in a part of the monitored computer system. Even in this case, the loss of function as a plant monitoring system can be prevented.

  Note that the example of FIG. 4 shows a case where only three functions A to C are multiplexed, but the other three functions D to F are similarly set to be replaceable by another computer. Is also possible.

[Operation of fault monitoring system]
[Outline of operation]
FIG. 5 is a flowchart showing an outline of data communicated with operations during normal operation of the fault monitoring system according to the present embodiment having the above-described configuration.

  As illustrated in FIG. 5, during normal operation, the monitoring target computer 202 transmits an information change notification to the information collection device 101 when the log information changes as log information transmission processing (S510). When a data transmission request is made from the information collecting apparatus 101, log information corresponding to the request is transmitted to the information collecting apparatus 101. The log information transmission process (S510) by the monitoring target computer 202 is continuously and repeatedly executed during normal operation.

  During normal operation, the information collection device 101 performs data collection / edit processing (S520) and request data transmission processing (S530) through the data communication unit 111 by the data editing unit 112.

  The data collection / editing process (S520) makes a data transmission request to the monitoring target computer 202, collects and edits the log information, saves it in the log storage means 113, and includes failure occurrence information in the received log information. If there is, the degree of influence of the failure occurrence is calculated, and failure occurrence notification and failure occurrence detailed data are transmitted to the state monitoring / analysis server 102. The information collection apparatus 101 also sends a list of possible transmissions and log information according to the request in the state monitoring / analysis server 102 when a data transmission request is made from the state monitoring / analysis server 102 as the request data transmission process (S530). Send to.

  In FIG. 5, for the sake of convenience, the request data transmission process (S530) is shown after the data collection / editing process (S520), but these processes are actually performed at the same time during normal operation. It is performed in parallel, and each process is repeatedly executed continuously.

  Further, during normal operation, the state monitoring / analysis server 102 performs failure occurrence emergency notification processing (S540), log information acquisition / data classification processing (S550), log analysis processing (S560), failure cause estimation processing (S570), A failure occurrence prediction process (S580) is performed.

  In the failure occurrence emergency notification process (S540), when a failure occurrence notification is made from the information collection device 101, the data classification unit 123 makes a data transmission request through the data communication unit 121, and the failure occurrence detailed data is used as the failure information 331. This is a process of obtaining an emergency notification to the monitoring terminals 104 and 105 through the data communication unit 121 when the fault information 331 having a high degree of urgency exists while being acquired and stored in the data storage unit 122.

  In the log information acquisition / data classification process (S550), the data classification unit 123 makes a data transmission request to the information collecting apparatus 101 through the data communication unit 121, acquires the transmittable list and the log information 311 and receives them. In this process, the data is classified into normal log information 321 and failure information 331 and stored in the data storage unit 122.

  In the log analysis process (S560), the log analysis unit 124 analyzes the log generation status of the new log information 311 on the time axis, and stores the analysis result as the analysis information 341 in the data storage unit 122. Is transmitted to the monitoring terminals 104 and 105.

  In the failure cause estimation process (S570), the failure cause estimation means 125 compares the failure information 331 relating to the failure that has occurred with the previous failure information 331, detects a candidate for the cause of the failure, and is convinced that it may be the cause. This is a process in which a candidate whose degree is equal to or higher than a set value is estimated as a cause of failure, stored in the data storage unit 122 as part of the failure information 331, and transmitted to the monitoring terminals 104 and 105 through the data communication unit 121.

  In the failure occurrence prediction process (S580), the failure occurrence prediction means 126 compares the analysis information 341 for a given period with the past analysis information 341 and is convinced of the possibility of occurrence of a failure corresponding to the similar analysis information 341. In this processing, a failure whose degree is equal to or greater than a set value is stored in the data storage unit 122 as prediction information 351 indicating a prediction result, and is transmitted to the monitoring terminals 104 and 105 through the data communication unit 121.

  In FIG. 5, for the sake of convenience, failure occurrence emergency notification processing (S540), log information acquisition / data classification processing (S550), log analysis processing (S560), failure cause estimation processing (S570), failure occurrence prediction Although the process (S580) is shown in this order, these processes are actually performed concurrently in normal operation, and each process is repeatedly executed continuously.

[Details of each process]
Below, the procedure of each process (S510-S580) shown in FIG. 5 as mentioned above is demonstrated with reference to drawings.

[Log information transmission processing for monitored computers]
FIG. 6 is a flowchart illustrating an example of a procedure of log information transmission processing (S510) by the monitoring target computer 202.

  As shown in FIG. 6, in the log information transmission process (S510), the monitoring target computer 202 determines whether there is any change in the log information 301 of its own computer, that is, whether new information has been added or deleted, or It is determined whether or not items such as contents have changed (S511). If a change in the log information 301 is detected (YES in S511), if the change item is an item to which time information should be added, the log information 301 has changed after the time information has been added. Is sent to the information collecting apparatus 101 (S512).

  In addition, when there is a data transmission request from the information collection apparatus 101 (YES in S513), the monitoring target computer 202 extracts the requested item from the log information 301 and transmits it to the information collection apparatus 101 (S514).

  Whether or not the time should be added to the change item when the log information 301 changes is, for example, whether or not the change of the item is recorded without adding the time. This can be realized by setting the monitoring target computer 202 so that the time is added later at the time of detection. As described above, even in the log information 301, an item that is originally recorded without time is added with the time when the change is detected by the monitoring target computer 202 itself, so that the change item is displayed. At the stage of acquisition by the information collection apparatus 101, the information collection apparatus 101 can acquire a time closer to the time when the change item has changed.

  Alternatively, a condition for determining whether or not to add time to the item of the log information 301 is prepared in advance on the information collecting apparatus 101 side, and the condition is transferred from the information collecting apparatus 101 to the monitored computer 202. It is also possible to send and save for.

  As a modification of the procedure of the log information transmission process (S510), a change in log information is not notified to the information collection apparatus 101, but is requested from the log information 301 in response to a data transmission request from the information collection apparatus 101. It is also possible to simply perform the process of transmitting the item (S513, S514).

  In order to realize the log information transmission process (S510) as described above in the monitoring target computer 202, the monitoring target computer 202 may be provided with log information communication means dedicated to log information transmission. The log information 301 can be directly extracted from the monitoring target computer 202 by the information collecting apparatus 101 using a general-purpose means for network communication provided in the computer.

[Information collection device processing]
[Data collection / edit processing]
FIG. 7 is a flowchart illustrating an example of a procedure of data collection / editing processing (S520) by the data editing unit 112 of the information collecting apparatus 101.

  As shown in FIG. 7, in the data collection / editing process (S520), the data editing unit 112 of the information collecting apparatus 101 passes through the data communication unit 111 when the execution start condition is satisfied (YES in S521). A data transmission request is sent to the monitoring target computer 202 (S522). In the example of FIG. 7, as an example, the execution start condition is when an information change notification is received from the monitoring target computer 202, or when the execution start condition is set in advance to perform periodic execution and the execution time is reached , Either.

  In response to the data transmission request, the data editing unit 112 receives the data of the log information 301 from the monitoring target computer 202 by the data communication unit 111 (YES in S523), and edits the received data in chronological order (S524). Hereinafter, an example in which the hardware configuration information 302a is edited in time series will be described as a specific method for editing data in time series.

  First, the hardware configuration information 302 a in the log information 301 received from the monitoring target computer 202 includes information on all hardware devices that make up the current monitoring target computer 202. When editing the hardware configuration information in the data received this time, the data editing unit 112 compares the current hardware configuration information with the previously received hardware configuration information and detects a configuration change. In this case, this hardware configuration change is handled as an occurrence of an event, and time is added to data indicating the hardware configuration change and edited as time series data. Thereby, the configuration change information regarding the change of the hardware configuration to which the time is not originally added can be edited as time series data.

  The time added to the data indicating the hardware configuration change is, for example, either the current reception time or processing time, or the time added by the monitoring target computer 202 in the log information transmission process (S510). is there.

  Further, the software configuration information 302b, hardware status information 303a, and software status information 303b in the log information 301 of the monitoring target computer 202 are also related to the configuration and status change using the same method as the hardware configuration information 302a. The configuration change information and the state change information can be edited as time series data.

  Next, when the data can be simplified (YES in S525), the data editing unit 112 performs data simplification (S526) as much as possible to reduce the data amount. Hereinafter, an example of simplifying the event log information 304 will be described as a specific method for simplifying data.

  First, the event log in the event log information 304 is configured in a text format in which events occurring in the monitoring target computer 202 during a certain period are arranged in time series with a time and event content as one set. That is, when the same event occurs, there are a plurality of data having different occurrence times. In addition, even if they are not the same event, there are cases where some expressions of the event content are the same. In simplification of data, the same event or the same expression is simplified by replacing it with a specific symbol, and at least one of the data before the simplification is left as original data, and the specific symbol and Make it correspond.

  Alternatively, if it is possible to correspond to the replaced symbol without leaving the data before simplification, it is possible to simplify the same event or the same expression without leaving the original data before simplification. It is.

  As shown in FIG. 8, the data editing unit 112 divides the time series data edited by the above-described series of processes at regular intervals, and logs information 311 including time series data groups G1 to Gn in units of periods. Is stored in the log storage means 113 (S527).

  Further, the data editing unit 112 performs a series of processes as shown in FIG. 9 as the influence determination / notification process (S528).

  As shown in FIG. 9, when there is failure occurrence information in the received data (YES in S5281), the data editing means 112 determines the function of the monitored computer that has caused the failure related to the failure occurrence information. The degree of influence that the occurrence of the failure has on the monitored computer system 201 is calculated (S5282). If the calculated degree of influence is equal to or greater than a preset setting value (YES in S5283), the data editing unit 112 determines the presence or absence of another computer that can be replaced (S5284). If it exists (YES in S5284), the influence is corrected downward (S5285).

  In this case, for example, a plurality of determination reference values are set in advance for determining the level of influence in a plurality of levels such as “emergency”, “warning”, and “normal”. In this case, as an example, when the calculated influence degree is “emergency”, the presence / absence of another computer that can be replaced is determined. Operation such as downward revision from “emergency” to “warning” is possible.

  The data editing unit 112, after the calculation / correction of the degree of influence, provides the state monitoring / analysis server 102 with simple data indicating only the gist of the failure occurrence, not the data indicating the detailed information about the failure occurrence through the data communication unit 111. Is transmitted as a failure occurrence notification (S5286). This failure occurrence notification is, for example, simple data that expresses items such as the failure occurrence time, the degree of influence, and the outline of the failure with short sentences or symbols.

  If the data communication unit 111 receives a data transmission request from the state monitoring / analysis server 102 after transmission of the failure occurrence notification (YES in S5287), the data editing unit 112 sends a request to the state monitoring / analysis server 102. Then, the failure occurrence detailed data indicating the detailed information on the failure occurrence is transmitted through the data communication unit 111 (S5288).

  As described above, the information collection apparatus 101 performs the data collection / editing process (S520) as shown in FIG. 7 so that original time-series data such as event log information indicating the occurrence of various events with time is provided. In addition, when a change is detected by comparing the configuration information or status information of the monitored computer in the received log information with past information, this change is treated as an event occurrence, and the change event is added with the time. Can be edited as time-series data indicating the occurrence of the occurrence.

  Therefore, by transmitting the data edited in this way by the information collection device 101 to the state monitoring / analysis server 102, the state monitoring / analysis server 102 can monitor in addition to time-series data indicating the occurrence of various events. Time series data indicating the occurrence of a change event can also be used for changes related to the configuration and state of the target computer, thereby making it possible to predict the occurrence of a failure.

  Further, in the information collecting apparatus 101, when the collected event information includes the same event content data, the event content data can be replaced with summary data and edited. Therefore, since the amount of data transmitted from the information collection apparatus 101 to the state monitoring / analysis server 102 can be reduced, an excessive load is not applied to the network such as the Internet 104 connecting the two.

  Further, by saving the simplified data, the necessary log information can be stored as compactly as possible, so that the data capacity required for the log storage means 113 can be suppressed.

  Furthermore, in the information collection apparatus 101, by performing the influence degree determination / notification process (S528) as shown in FIG. 9, when there is failure occurrence information having a high influence degree on the system in the received data, By first notifying simple data indicating only the gist of the failure occurrence, it is possible to quickly and reliably send the failure occurrence notification to the state monitoring / analysis server 102 with a small amount of data.

  Then, the failure occurrence detailed data is transmitted in response to the data transmission request from the status monitoring / analysis server 102 that has received the notification, thereby notifying the person in charge of the status monitoring / analysis server 102 of the gist of the failure occurrence. After that, detailed information regarding the occurrence of the failure can be notified, which can contribute to the realization of a prompt and appropriate response to the occurrence of the failure having a high impact on the system.

  Unlike the influence determination / notification process (S528) shown in FIG. 9, when the failure occurrence detailed data is transmitted as the failure occurrence notification from the beginning without performing the failure occurrence notification using simple data, the data amount When transmission of large failure occurrence detail data is unsuccessful, the failure occurrence notification itself is unsuccessful.

  On the other hand, since the failure occurrence notification based on simple data as in the influence determination / notification processing (S528) shown in FIG. 9 can reduce the possibility of failure as much as possible, the state monitoring / analysis server 102 It is possible to reliably notify the occurrence of a failure. Then, by transmitting the failure occurrence detailed data in response to the data transmission request from the state monitoring / analysis server 102 that has confirmed the notification, even if the failure occurrence detailed data transmission is unsuccessful, the state monitoring / In response to a data transmission request from the analysis server 102 again, it is possible to perform operations such as transmitting the failure occurrence detailed data again.

  In addition, since the failure occurrence notification based on simple data can be reliably transmitted even when the data processing capacity of the notification destination is small, in the event of an emergency, either directly from the information collection device 101 or as described later. The emergency notification to the mobile phone or the like becomes possible through the failure occurrence emergency notification processing (S540) of the state monitoring / analysis server 102.

[Request data transmission processing]
FIG. 10 is a flowchart illustrating an example of a procedure of request data transmission processing (S530) by the data editing unit 112 of the information collection apparatus 101.

  As shown in FIG. 10, in the request data transmission process (S530), the data editing unit 112 of the information collection device 101 receives a normal data transmission request from the data communication unit 111 (YES in S531). Is a list request (S532). If the request is a list request (YES in S532), a transmittable list indicating items that can be transmitted in the log information stored in the log storage unit 113 is generated as transmission data (S533).

  In this case, as shown in FIG. 8, the log information 311 stored in the log storage unit 113 is composed of time-series data groups G1 to Gn that are divided at regular intervals, so The possible list is, for example, a list in which time-series data group names in the cycle unit are listed.

  When the data transmission request is not a list request but a log information request (NO in S532), the data editing unit 112 displays the requested log information from the log information stored in the log storage unit 113. The data is taken out and transmission data is generated (S534). When the data transmission request is a log information request, generally, since it includes designation of time or time, data corresponding to the designated time or time is represented in units of time series data groups as shown in FIG. The data editing process is performed as necessary to generate transmission data.

  The data editing unit 112 transmits the transmission data of the generated transmittable list or log information 311 to the state monitoring / analysis server 102 by the data communication unit 111 (S535). When the log information 311 is transmitted, information indicating that the item of the log information 311 has been transmitted is added (S536).

  As described above, the information collection apparatus 101 makes a request for list to the information collection apparatus 101 from the state monitoring / analysis server 102 by performing the request data transmission process (S530) as shown in FIG. After confirming an item in the log information 311 that has not been received, an operation such as requesting transmission of the item and receiving the item can be performed. In this case, whether or not the state monitoring / analysis server 102 has received an item with log information may be included in the transmittable list by the transmitted information managed on the information collecting apparatus 101 side. As a modification, information indicating that the status monitoring / analysis server 102 has received the information may be managed.

  In any case, by managing information regarding items that can be transmitted and information regarding transmission or reception, the information collection device 101 sends a state monitoring / analysis server 102 to the state monitoring / analysis server 102 from the transmission possible list. Since the analysis server 102 can transmit only the necessary untransmitted items, it is possible to avoid transmitting the transmitted items wastefully, and to reduce the amount of data transmitted to the state monitoring / analysis server 102 as much as possible. Can do.

[Status monitoring / analysis server processing]
FIG. 11 is a block diagram showing each process performed by the state monitoring / analysis server 102 and data acquired / stored or used in those processes. Hereinafter, each process of the state monitoring / analysis server 102 will be sequentially described with reference to flowcharts shown in FIG. 11 and FIG.

[Emergency notification processing for failure occurrence]
FIG. 12 is a flowchart showing an example of the procedure of the failure occurrence emergency notification process (S540) by the data classification unit 123 of the state monitoring / analysis server 102.

  As shown in FIG. 12, in the failure occurrence emergency notification process (S540), the data classification unit 123 of the state monitoring / analysis server 102 receives a failure occurrence notification from the information collecting apparatus 101 by the data communication unit 121 ( In S541 (YES), it is determined whether or not the influence included in the failure occurrence notification is greater than or equal to a preset setting value (S542).

  When the influence degree is equal to or greater than the set value (YES in S542), the data classification unit 123 notifies the notification destination according to the influence degree from a plurality of preset notification destinations including the monitoring terminals 104 and 105. And the failure occurrence emergency notification is made to the selected notification destination through the data communication means 121 (S543). In this case, for example, when the degree of influence is “emergency”, all notification destinations including the monitoring terminals 104 and 105 are selected, and when the degree of influence is “warning”, only the monitoring terminals 104 and 105 are selected. Operation such as selecting is possible. The data to be transmitted as the failure occurrence emergency notification is simple data basically using the failure occurrence notification data received from the information collecting apparatus 101 as it is.

  The emergency notification of failure occurrence received by the monitoring terminals 104 and 105 is displayed on the monitoring terminal. In response to this display, the person in charge of operating the monitoring terminals 104 and 105 is confirmed to confirm the notification. When an operation is performed, or when the monitoring terminals 104 and 105 recognize a failure occurrence emergency notification, a data transmission request is transmitted from the monitoring terminals 104 and 105 to the state monitoring / analysis server 102.

  In such a case, when the status monitoring / analysis server 102 receives a data transmission request from the monitoring terminals 104 and 105 by the data communication unit 121 (YES in S544), the data classification unit 123 displays the information collection device that is the notification source. A data transmission request is sent to the terminal 101 through the data communication means 121 (S545). As a modification, the execution start condition of such a data transmission request is not limited to when the data transmission request is received from the monitoring terminals 104 and 105, but for example, a failure occurrence notification transmitted to the state monitoring / analysis server 102 For example, when the data classification unit 123 recognizes the contents, it can be changed as appropriate according to the actual system operation status.

  If the data communication unit 121 receives the failure occurrence detailed data from the requested information collection apparatus 101 after transmission of the data transmission request (YES in S546), the data classification unit 123 transmits the received failure occurrence detail data. It is classified as failure information 331 different from the normal log information 321 and stored in the data storage means 122 in chronological order (S547), and to the notification destination such as the monitoring terminals 104 and 105 that have issued the failure occurrence emergency notification The failure occurrence detailed data is transmitted (S548).

  As described above, the state monitoring / analysis server 102 performs the failure occurrence emergency notification process (S540) as shown in FIG. Since it is possible to grasp detailed information about the occurrence of a failure after grasping the gist of a failure requiring urgent action that has a high impact on the system, it is quick and appropriate for a failure that has a high impact on the system. Is possible.

  Further, similar to the impact determination / notification process (S528), the failure occurrence emergency notification using simple data can reduce the possibility of failure as much as possible, so the failure occurrence notification is sent to the monitoring terminals 104 and 105. It can be done reliably. Then, by transmitting the failure occurrence detailed data in response to the data transmission request from the monitoring terminals 104 and 105 that have confirmed the notification, even if the failure occurrence detailed data transmission is unsuccessful, the monitoring terminal 104, In response to a data transmission request from 105 again, an operation such as retransmitting the detailed failure occurrence data becomes possible.

  In addition, since the failure occurrence emergency notification using simple data can be reliably transmitted even when the data processing capacity of the notification destination is small, as described in the impact determination / notification processing (S528), the mobile phone Emergency notification to the telephone etc. becomes possible.

  Further, by storing the received failure occurrence detailed data as failure information in time series order, log analysis processing by the log analysis means 124 (S560), failure cause estimation processing by the failure cause estimation means 125 (S570), failure occurrence prediction In the failure occurrence prediction processing (S580) by the means 126, etc., it can be effectively used as time series data of the failure information 331.

[Log information acquisition / data classification processing]
FIG. 13 is a flowchart illustrating an example of a procedure of log information acquisition / data classification processing (S550) by the data classification unit 123 of the state monitoring / analysis server 102.

  As shown in FIG. 13, in the log information acquisition / data classification process (S550), the data classification unit 123 of the state monitoring / analysis server 102 performs data communication when the execution start condition is satisfied (YES in S551). Through the means 121, a data transmission request for the transmittable list is made to the information collecting apparatus 101 (S552). In the example of FIG. 13, the execution start condition is, for example, when a log information acquisition request is made from the monitoring terminals 104 and 105, or is set to perform periodic execution in advance and has reached its execution time. If any.

  In response to the data transmission request for the transmittable list, the data classifying unit 123 receives the transmittable list from the information collecting apparatus 101 by the data communication unit 121 (YES in S553), and has not yet received it in the received transmittable list. It is determined whether or not there is any log information item (S554). If there is an item of log information that has not yet been received (YES in S554), a data transmission request for unreceived data of the log information item is transmitted to the information collection device 101 through the data communication unit 121. (S555).

  The data classification unit 123 receives the data when the data communication unit 121 receives the data from the requested information collection apparatus 101 after transmitting the data transmission request for the unreceived data of the log information item (YES in S556). If there is summary data in the data, the summary data is restored to the original data (S557). The data classification unit 123 classifies the received log information item data into normal log information 321 and failure information 331, and further classifies the items by items such as the monitoring target computer and the monitoring target computer system, and then the data storage unit 122. Are stored in chronological order (S558), and a log information acquisition notification is transmitted to the monitoring terminals 104 and 105 as request sources (S559).

  If there is no item of log information that has not been received (NO in S554), there is no log information to be acquired this time through the data communication unit 121 to the information collection apparatus 101, and therefore this time there is no log information. A log information non-acquisition notification indicating that it has been acquired is transmitted (S559).

  As described above, in the status monitoring / analysis server 102, the log information acquisition / data classification process (S550) as shown in FIG. Since the data of the item can be acquired at that time, all necessary log information can be reliably acquired on the state monitoring / analysis server 102 side.

  Further, the acquired log information data is classified into failure information and normal log information and stored in chronological order, whereby log analysis processing by the log analysis unit 124 (S560) and failure cause estimation by the failure cause estimation unit 125 are performed. In the estimation process (S570), the failure occurrence prediction process (S580) by the failure occurrence prediction means 126, etc., it can be used effectively as time series data of failure information and normal log information.

[Log analysis processing]
FIG. 14 is a flowchart illustrating an example of a procedure of log analysis processing (S560) by the log analysis unit 124 of the state monitoring / analysis server 102.

  As shown in FIG. 14, in the log analysis process (S560), the log analysis unit 124 of the state monitoring / analysis server 102 acquires new failure information 331 or log information 311 as an example of a log analysis execution start condition. When the log analysis request is received from the monitoring terminals 104 and 105 (YES in S561), the occurrence status of the unanalyzed log in the target failure information 331 or log information 311 is analyzed on the time axis. (S562).

  FIG. 15 is a diagram showing an example of a specific analysis result 341a stored as the analysis information 341. The horizontal axis indicates the passage of time, and the vertical axis indicates one state quantity of the monitoring target computer 202. For the load, the result of analyzing two types of values, the maximum value and the minimum value, as event information is shown. In this case, each event indicated by a dot is obtained by calculating a single state value indicating the state amount for each preset calculation unit time. For the CPU load maximum value, two changes A1 and A2 are shown.

  In FIG. 15, for example, the calculation unit time is 5 minutes, and the average CPU load for 5 minutes is calculated as a single state value. That is, each event of the CPU load maximum value indicated by each dot in the upper row is an average value of the maximum CPU load for 5 minutes, and each event of the CPU load minimum value indicated by each dot in the lower row is the minimum value for 5 minutes. It is an average value of CPU load. Further, the length of the time axis is, for example, about 300 hours to half a year.

  FIG. 16 is a diagram showing another example 341b of specific analysis results stored as analysis information 341. The horizontal axis indicates the passage of time, and the vertical axis indicates a failure graph (failure information) of the monitored computer 202. Analysis result) 3411, an event graph (event analysis result of CPU load maximum value) 3412, a state graph (analysis result of CPU load average value) 3413, and a configuration graph (analysis result of configuration change information) 3414 are shown side by side. It is. As for the CPU load maximum value of the event graph 3412, one change B1 is shown, and it can be seen from the comparison with the failure graph 3411 that the failure C1 occurs after this change B1.

  The log analysis unit 124 stores the obtained analysis result as analysis information 341 in the data storage unit 122 (S563), and transmits it to the monitoring terminals 104 and 105 through the data communication unit 121 (S564).

  As described above, when the state monitoring / analysis server 102 performs log analysis processing (S560) as shown in FIG. 14 to obtain new fault information or time series data of log information, the log information Since it is possible to analyze the occurrence of unanalyzed logs, it is possible to reliably acquire analysis information for all the acquired log information without wastefully analyzing the same log.

  Also, by storing the analysis information obtained by analyzing on the time axis, in the failure cause estimation process (S570) by the failure cause estimation means 125, the failure occurrence prediction process (S580) by the failure occurrence prediction means 126, etc. It can be effectively used as past analysis information for comparison with other analysis information.

  Furthermore, using the time series data related to the change in the configuration and state of the monitored computer 202 included in the log information 311 acquired from the information collection apparatus 101, the change in the configuration and state of the monitored computer 202 can also be performed on the time axis. Analysis information can be obtained by analysis. That is, not only the original time-series data analysis information such as event log information indicating the occurrence of various events with time, but also the time-series data regarding the configuration and state change of the monitored computer 202 that is not originally time-series data. Analysis information can be acquired, and graphs as shown in FIGS. 15 and 16 can be displayed on the monitoring terminals 104 and 105.

  Therefore, in failure cause estimation processing (S570) by failure cause estimation means 125, which will be described later, and failure occurrence prediction processing (S580) by failure occurrence prediction means 126, not only analysis information of original time-series data such as event log information, but also The analysis information of the time series data can be easily compared with the past analysis information and the graph display or the like for the change in the configuration or state of the monitored computer 202 that is not originally time series data.

[Failure cause estimation processing]
FIG. 17 is a flowchart illustrating an example of a procedure of failure cause estimation processing (S570) by the failure cause estimation unit 125 of the state monitoring / analysis server 102.

  As shown in FIG. 17, in the failure cause estimation process (S570), the failure cause estimation unit 125 of the state monitoring / analysis server 102 uses the data classification unit 123 as an example of a failure cause estimation execution start condition. When the failure occurrence emergency notification process (S540) is performed, or when a failure cause estimation request is received from the monitoring terminals 104 and 105 (YES in S571), the target failure information 331 and past failure information 331 are Comparison is performed (S572). In other words, the target failure information 331 is compared with the previous failure information 331 in the monitoring target computer and other monitoring target computers in which the failure related to the failure information 331 has occurred.

  In comparison with the past failure information, product information of the monitoring target computer 202 may be obtained from the Web site 106 and compared with the target failure information 331. FIG. 18 is a diagram showing an example of product information posted on the Web page 161 of the Web site 106, and a plurality of products indicating product identification information 601 such as a product name and causes and countermeasures for some abnormal phenomenon. Phenomenon information 602a to 602c is shown. The product phenomenon information 602a to 602c corresponds to the past failure information 331.

  If there is past fault information 331 similar to the target fault information 331 as a result of the comparison (YES in S573), the fault cause estimation unit 125 determines the target fault information based on the past fault information. A fault cause candidate for 331 is detected (S574). Subsequently, each failure cause candidate for the target failure information 331 is based on the configuration information and status information of the computer at the time of the previous similar failure and the configuration information and status information of the computer at the time of the target failure. The certainty of the possibility of the cause is calculated (S575). In this case, as the computer configuration information and status information, configuration change information and status change information converted into time series data included in the log information at the time of the failure, or analysis information thereof can be used as appropriate.

  Then, a candidate having a certainty of the possibility of being a cause having a certainty or more is set as a failure cause estimation result, stored as a part of the failure information 331 (S576), and transmitted to the requesting monitoring terminals 104 and 105 ( S577). If similar failure information 331 does not exist (NO in S573), an estimation error notification is transmitted to the requesting monitoring terminals 104 and 105 (S577).

  As described above, the failure monitoring process (S570) as shown in FIG. 17 is performed in the state monitoring / analysis server 102, so that when the failure occurs, the current failure information related to the failure is accumulated in the past. The cause of the failure can be easily estimated by comparing with the information obtained. Also, in the cause estimation, the cause of the failure is determined according to the similarity of the computer configuration and status by calculating the certainty of the possibility that each failure cause candidate is based on the computer configuration information and status information. It can be estimated more accurately. Furthermore, the cause of the failure can be estimated with higher accuracy by using the product information posted on the Web page 106 of the Web site 106 in estimating the cause.

[Failure prediction processing]
FIG. 19 is a flowchart showing an example of the procedure of failure occurrence prediction processing (S580) by the failure occurrence prediction means 126 of the state monitoring / analysis server 102.

  As shown in FIG. 19, in the failure occurrence prediction process (S580), the failure occurrence prediction unit 126 of the state monitoring / analysis server 102 generates a failure from the monitoring terminals 104 and 105 as an example of the failure occurrence prediction start condition. When the prediction request is made, or when the execution time is set in advance and the execution time is reached (YES in S581), the analysis information 341 for a certain period given as the prediction target is obtained. The analysis information 341 at the time of past failure occurrence is compared (S582). In this case, “analysis information for a given period” is generally analysis information for a certain period including the latest analysis information acquired this time.

  If there is past analysis information 341 similar to the analysis information 341 for the certain period as a result of the comparison (YES in S583), the failure occurrence predicting means 126 further corresponds to the similar past analysis information 341. It is determined whether or not the failure information 331 exists (S584).

  For example, it is assumed that an analysis result 341a as shown in FIG. 15 is given as analysis information for a certain period for a certain monitoring target computer 202. In this case, in the analysis result 341a of FIG. 15, the first change A1 among the two changes A1 and A2 in the CPU load maximum value is a discontinuous change, and the log information 321 of the same monitored computer 202 is the same. It is determined from the time series data of the configuration change information included in that that is due to the configuration change and is irrelevant to the failure. On the other hand, the continuous increase of the next change A2 is determined as one phenomenon to be compared based on a preset comparison condition or the like, and compared with the past analysis result 341.

  On the other hand, an analysis result 341b including an analysis result of failure information as shown in FIG. 16 is stored as past analysis information 341 for the same monitoring target computer 202 or another monitoring target computer 202 having a similar configuration or operation. 15, a change A2 indicating a continuous increase in the CPU load maximum value in the analysis result 341a in FIG. 15 and a continuous increase in the CPU load maximum value in the event graph 3412 in the analysis result 341b in FIG. The change B1 shown is determined to be similar. Further, as fault information corresponding to the analysis result 341b determined to be similar, fault information indicating faults C1 and C2 that occur after the change B1 exists.

  If the failure information 331 corresponding to the similar past analysis information 341 exists (YES in S584), the failure occurrence predicting means 126 calculates the certainty of the possibility of occurrence of the failure based on the failure information 331. (S585). The failure occurrence predicting means 126 stores, as the prediction information 351, a prediction result that the failure is likely to occur when there is a failure with the calculated certainty factor equal to or higher than a preset set value (YES in S586). At the same time (S587), the request is transmitted to the monitoring terminals 104 and 105 as request sources (S588).

  For example, in the above example using the analysis results 341a and 341b in FIG. 15 and FIG. 16, there is a similar change B1 in the past analysis result 341b with respect to the change A2 in the given analysis result 341a, and A prediction result indicating that there is a high possibility that the faults C1 and C2 occurring after the change B1 occur.

  Further, when there is no failure with the calculated certainty factor equal to or higher than a preset setting value (NO in S586), or when there is no failure information 331 corresponding to similar past analysis information 341 (NO in S584). The prediction result that the possibility of failure is low is stored as the prediction information 351 (S587), and is transmitted to the requesting monitoring terminals 104 and 105 (S588). Further, if similar analysis information 341 does not exist (NO in S583), a prediction error notification is transmitted to the requesting monitoring terminals 104 and 105 (S588).

  As described above, the state monitoring / analysis server 102 performs failure occurrence prediction processing (S580) as shown in FIG. 19 to compare the given analysis information with the information accumulated in the past on the time axis. Thus, it is possible to predict the possibility of failure with high accuracy. Therefore, it is possible to catch a sign of the occurrence of a failure and accurately predict the content of the failure before the failure occurs.

  In other words, in addition to the original time-series data indicating the occurrence of various events, it is also possible to use time-series data edited by regarding the change as the occurrence of an event for changes related to the configuration and status of the monitored computer. All the time series data including the time series data indicating the configuration and state information of such a computer can be analyzed, and the analysis information can be compared with the new one. Then, by comparing old and new analysis information including computer configuration and status information, past analysis information with similar configuration and status is accurately detected, and failure information is generated using failure information corresponding to the analysis information. Can be accurately predicted.

  In addition, the failure information includes the failure cause estimation result obtained by the failure cause estimation process described above, so that the failure occurrence can be predicted more accurately by using the failure cause estimation result. It can be performed with high accuracy. Therefore, it is possible to predict not only the details of the failure but also the cause before the failure occurs, and the person in charge who confirms the prediction result on the monitoring terminal must take appropriate measures according to the cause in advance. Is possible.

  In the above example using the analysis results 341a and 341b in FIG. 15 and FIG. 16, the case of performing data comparison on the time axis of analysis information for one element called the CPU load maximum value has been described. By comparing the data on the time axis of analysis information for multiple elements in the same way, calculating the certainty of the possibility of failure according to the number of elements determined to be similar, it is more accurate Predictive results can be obtained.

  In addition, regarding failures that occurred in the past, in addition to the failure cause estimation results, in addition to the failure cause estimation results, by storing the measures for the failures, by adding information about the measures to the failure prediction results, Since it is possible to present not only the details of the failure, but also appropriate measures according to it, the person in charge who confirmed the prediction results on the monitoring terminal can take the appropriate measures presented in advance It becomes. Further, information regarding countermeasures when a failure occurs may be obtained from the product information posted on the web page 161 of the website 106 as shown in FIG.

[Second Embodiment]
FIG. 20 is a block diagram showing a configuration of a failure monitoring system according to the second embodiment to which the present invention is applied.

  As shown in FIG. 20, the fault monitoring system according to the present embodiment includes the data communication unit 111 of the information collecting apparatus 101 and the data of the state monitoring / analysis server 102 in the configuration of the fault monitoring system according to the first embodiment. The communication means 121 is connected not via the Internet 100 but via the monitoring intranet 103. Other configurations are the same as those of the first embodiment.

  According to the present embodiment having the above-described configuration, the monitoring intranet 103 performs communication between the data communication unit 111 of the information collection apparatus 101 and the data communication unit 121 of the state monitoring / analysis server 102. Thus, even when the Internet 100 cannot be used, the fault monitoring system and method of the present invention can be applied. In addition, since the monitoring intranet 103 is used, a plurality of information collection apparatuses 101 can be connected to one state monitoring / analysis server 102.

[Third Embodiment]
FIG. 21 is a block diagram showing a configuration of a failure monitoring system according to the third embodiment to which the present invention is applied.

  As shown in FIG. 21, the fault monitoring system according to the present embodiment includes the information collection apparatus 101 in the same computer as the state monitoring / analysis server 102 in the configuration of the fault monitoring system according to the first embodiment. This is realized by different programs, in which the data communication unit 111 of the information collecting apparatus 101 and the data communication unit 121 of the state monitoring / analysis server 102 are connected via inter-program communication. Other configurations are the same as those of the first embodiment.

  According to the present embodiment having the above-described configuration, the information collection apparatus 101 and the state monitoring / analysis server 102 can be realized by a single computer, so that the system configuration can be simplified and the economic efficiency can be improved.

  Furthermore, as a modification, by connecting to the information collection device 101 realized on another computer, the state monitoring / analysis server 102 can be connected not only to the information collection device 101 in the same computer but also to another information collection device 101. It is possible to connect to a plurality of information collecting apparatuses 101 including the information collecting apparatus 101.

[Other Embodiments]
It should be noted that the present invention is not limited to the above-described embodiments, and various other variations can be implemented within the scope of the present invention.

  For example, the specific configuration, processing procedure, processing content, and the like of the information collection device, the state monitoring / analysis server, and each of the units that constitute these devices are only examples. That is, according to the present invention, in the information collection device, the computer configuration or state change obtained from the collected log information is edited as time series data, and the same event content data in the log information is replaced with summary data. As long as the monitoring / analysis server compares the analysis information of the old and new time-series data and predicts the occurrence of a failure, the specific configuration, processing procedure, processing content, etc. can be freely changed.

  Although the present invention is optimal as a failure monitoring system for monitoring a computer system used for plant monitoring or the like, various computer systems connected in the same manner can be similarly applied as monitoring targets, and similarly An excellent effect can be obtained.

1 is a block diagram showing a configuration of a failure monitoring system according to a first embodiment to which the present invention is applied. FIG. 2 is a block diagram illustrating a configuration example of a monitoring target computer illustrated in FIG. 1, in which (a) is a hardware configuration, and (b) is a software configuration. The data block diagram which shows an example of the log information of the own computer preserve | saved in the monitoring object computer shown in FIG. The block diagram which shows the function structural example of the monitoring object computer system which multiplexed the function as an example of the monitoring object computer system shown in FIG. The flowchart which shows the outline of the data communicated with the operation | movement at the time of normal driving | operation of the failure monitoring system which concerns on 1st Embodiment. The flowchart which shows an example of the procedure of the log information transmission process shown in FIG. 6 is a flowchart illustrating an example of a procedure of data collection / editing processing illustrated in FIG. 5. The data block diagram which shows the preservation | save format of the time series data edited by the data collection and edit process shown in FIG. The flowchart which shows an example of the procedure of the influence determination / notification process shown in FIG. 6 is a flowchart showing an example of a procedure of request data transmission processing shown in FIG. FIG. 6 is a block diagram showing each process by the state monitoring / analysis server shown in FIG. 5 and data acquired / stored or used in those processes. The flowchart which shows an example of the procedure of the failure generation | occurrence | production emergency notification process shown in FIG. The flowchart which shows an example of the procedure of the log information acquisition / data classification process shown in FIG. The flowchart which shows an example of the procedure of the log analysis process shown in FIG. The figure which shows an example of the specific analysis result preserve | saved as analysis information by the log analysis process shown in FIG. The figure which shows another example of the specific analysis result preserve | saved as analysis information by the log analysis process shown in FIG. The flowchart which shows an example of the procedure of the failure cause estimation process shown in FIG. The figure which shows an example of the product information published on the web page of the web site shown in FIG. The flowchart which shows an example of the procedure of the failure occurrence prediction process shown in FIG. The block diagram which shows the structure of the failure monitoring system which concerns on 2nd Embodiment to which this invention is applied. The block diagram which shows the structure of the failure monitoring system which concerns on 3rd Embodiment to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Internet 101 ... Information collection apparatus 102 ... Status monitoring / analysis server 103 ... Monitoring intranet 104, 105 ... Monitoring terminal 106 ... Web site 111 ... Data communication means 112 ... Data editing means 113 ... Log storage means 121 ... Data communication means 122 ... Data storage means 123 ... Data classification means 124 ... Log analysis means 125 ... Failure cause estimation means 126 ... Failure occurrence prediction means 161 ... Web page 201 ... Monitored computer system 202 ... Monitored computer 203 ... Network 301 ... Log information 302 ... configuration information 303 ... status information 304 ... event log information 311 ... log information 321 ... (normal) log information 331 ... failure information 341 ... analysis information 351 ... prediction information

Claims (14)

In a failure monitoring system that monitors the occurrence of a failure in a monitored computer system composed of a plurality of monitored computers connected via a network,
An information collection device for collecting, editing and storing log information of the computer stored in the monitored computer;
Analyzing the log information stored in the information collection device to obtain analysis information, and equipped with a state monitoring / analysis server for estimating the cause at the time of failure occurrence and prediction of failure occurrence based on the log information and analysis information,
The information collecting device includes:
Data communication means for receiving log information of the computer from the monitoring target computer and communicating data including log information with the state monitoring / analysis server;
The configuration information or status information of the monitored computer included in the received log information is compared with each corresponding information in the past, and when there is a configuration or status change, the change is treated as an event occurrence, and the time And editing as time-series data, and when the received event information includes the same event content data, data editing means for editing by replacing with summary data representing the event content data,
Having log storage means for storing the edited log information;
The state monitoring / analysis server
Data communication means for communicating data including log information with the information collection device;
Data storage means for storing the received log information and various data acquired in the server;
If the received log information includes the abbreviated data, editing is performed to restore the abbreviated data to the original event content data, and information indicating the occurrence of a failure is included in the log information as failure information. Data classification means for classifying and storing in the data storage means;
Log analysis means for analyzing the occurrence of the log on the time axis using the received log information to acquire analysis information, and storing the acquired analysis information in the data storage means;
For the occurrence of the failure indicated by the failure information, detect the candidate for the cause of the failure from the failure information when the similar failure occurred in the past, calculate the certainty of the possibility that the candidate is the cause, and according to the calculation result Failure cause estimation means for storing the failure cause estimation result obtained as a part of failure information indicating the occurrence of the failure in the data storage means;
Whether or not past analysis information is similar to the analysis information for a given period is determined by comparing the data on the time axis, and if it is determined to be similar, it corresponds to the past analysis information Yes with fault information, and calculates the likelihood of certainty of the failure indicated by the fault information, the fault occurrence prediction means for storing in said data storage means as the prediction information prediction results obtained in accordance with the calculated results And
The failure occurrence predicting means performs a data comparison on the time axis between the analysis information of the given period and the past analysis information for a plurality of elements including various state quantities of the monitored computer. A failure monitoring system configured to calculate a certainty factor of the possibility of occurrence of the failure according to the number of elements determined to be . In a failure monitoring system that monitors the occurrence of a failure in a monitored computer system composed of a plurality of monitored computers connected via a network,
An information collection device for collecting, editing and storing log information of the computer stored in the monitored computer;
Analyzing the log information stored in the information collection device to obtain analysis information, and equipped with a state monitoring / analysis server for estimating the cause at the time of failure occurrence and prediction of failure occurrence based on the log information and analysis information,
The information collecting device includes:
Data communication means for receiving log information of the computer from the monitoring target computer and communicating data including log information with the state monitoring / analysis server;
The configuration information or status information of the monitored computer included in the received log information is compared with each corresponding information in the past, and when there is a configuration or status change, the change is treated as an event occurrence, and the time And editing as time-series data, and when the received event information includes the same event content data, data editing means for editing by replacing with summary data representing the event content data,
Log storage means for storing the edited log information ;
When the received or edited log information includes information indicating the occurrence of a failure, the degree of influence that the occurrence of the failure has on the monitored computer system is calculated, and the calculated degree of influence is greater than or equal to a preset value If it is, it has an impact determination / notification means for notifying the occurrence of the failure to the state monitoring / analysis server by the data communication means,
The state monitoring / analysis server
Data communication means for communicating data including log information with the information collection device;
Data storage means for storing the received log information and various data acquired in the server;
If the received log information includes the abbreviated data, editing is performed to restore the abbreviated data to the original event content data, and information indicating the occurrence of a failure is included in the log information as failure information. Data classification means for classifying and storing in the data storage means;
Log analysis means for analyzing the occurrence of the log on the time axis using the received log information to acquire analysis information, and storing the acquired analysis information in the data storage means;
For the occurrence of the failure indicated by the failure information, detect the candidate for the cause of the failure from the failure information when the similar failure occurred in the past, calculate the certainty of the possibility that the candidate is the cause, and according to the calculation result Failure cause estimation means for storing the failure cause estimation result obtained as a part of failure information indicating the occurrence of the failure in the data storage means;
Whether or not past analysis information is similar to the analysis information for a given period is determined by comparing the data on the time axis, and if it is determined to be similar, it corresponds to the past analysis information A failure occurrence predicting unit that calculates the certainty of occurrence possibility of the failure indicated by the failure information using the failure information, and stores the prediction result obtained according to the calculation result as the prediction information in the data storage unit. Fault monitoring system characterized by that. In the information collecting apparatus,
The degree of influence determination / notification means, when notifying the occurrence of the failure to the state monitoring / analysis server, only data indicating only the gist of the occurrence of the failure, not data indicating details of the occurrence of the failure. Sending data as transmission data by the data communication means, and transmitting data indicating details of the occurrence of the failure when a request for detailed data is received from the state monitoring / analysis server in response to the notification by the transmission data The fault monitoring system according to claim 2 , wherein the fault monitoring system is configured as follows. In the information collecting apparatus,
When the influence degree determination / notification means calculates the influence degree of the occurrence of the failure, another monitoring target computer capable of substituting the function of the monitoring target computer in which the failure has occurred is included in the monitoring target computer system. fault monitoring system according to claim 2 or claim 3 is whether determined, characterized in that it is configured to determine a degree of influence in accordance with the determination result. The state monitoring / analysis server
When receiving the notification of the occurrence of the failure from the information collection device, select a notification destination to notify the occurrence of the failure from a plurality of preset notification destinations according to the degree of influence of the occurrence of the failure and, fault monitoring system according to any one of claims 2 to 4 characterized in that it has a notifying means for notifying by said communication means the occurrence of the failure notification to the selected destination. In the state monitoring / analysis server,
The log analysis means calculates a single state value indicating the state quantity for each preset calculation unit time for data indicating an arbitrary state quantity among the log information of the monitored computer, and continuously Data indicating each state value of a plurality of calculation unit times to be stored in the data storage means as analysis information of the state quantity,
Single state value indicating the state quantity of the calculation unit time, average value, maximum value, any one of claims 1 to 5 characterized in that it is a value selected from among the minimum Fault monitoring system described in 1. In the state monitoring / analysis server,
The log analysis means calculates the frequency of occurrence within the event unit time for each event unit time set in advance for the data indicating the event that repeatedly occurs in the log information of the monitored computer, and continuously The data indicating each frequency of a plurality of event unit times is configured to be stored in the data storage unit as analysis information of the event, according to any one of claims 1 to 6. The fault monitoring system described. The state monitoring / analysis server
When the data received from the information collection device includes information indicating the occurrence of a failure requiring urgent response, the urgent response to the monitoring terminal connected to the state monitoring / analysis server fault monitoring system according to any one of claims 1 to 7, characterized in that it has a fault notifying means for notifying the occurrence of a failure that requires. The information collection device and the condition monitoring and analysis server, fault monitoring system according to any one of claims 1 to 8, characterized in that it is connected via a wide area network or LAN. The data communication means of the data communication means and the condition monitoring and analysis server of the information collection device, any one of claims 1 to 8, characterized in that it is connected via a communications program Fault monitoring system described in 1. In a failure monitoring method for monitoring the occurrence of a failure in a monitored computer system including a plurality of monitored computers connected via a network,
An information collection device for collecting, editing and storing log information of the computer stored in the monitored computer;
Using the state monitoring / analysis server that analyzes the log information stored in the information collecting device to acquire analysis information, and estimates the cause when a failure occurs and predicts the occurrence of the failure based on the log information and the analysis information ,
By the information collecting device,
A data communication step of receiving log information of the computer from the monitoring target computer and performing communication of data including log information with the state monitoring / analysis server;
The configuration information or status information of the monitored computer included in the received log information is compared with each corresponding information in the past, and when there is a configuration or status change, the change is treated as an event occurrence, and the time And editing as time series data, and when the same event content data is included in the received log information, a data editing step for editing by replacing with summary data representing the event content data,
Perform a log saving step to save the edited log information,
The state monitoring / analysis server
A data communication step of communicating data including log information with the information collection device;
A data storage step for storing the received log information and various data acquired in the server;
If the received log information includes the abbreviated data, editing is performed to restore the abbreviated data to the original event content data, and information indicating the occurrence of a failure is included in the log information as failure information. A data classification step of classifying and storing in the data storage step ;
A log analysis step for analyzing the occurrence of the log on the time axis using the received log information to acquire analysis information, and storing the acquired analysis information in the data storage step ;
For the occurrence of the failure indicated by the failure information, detect the candidate for the cause of the failure from the failure information when the similar failure occurred in the past, calculate the certainty of the possibility that the candidate is the cause, and according to the calculation result A failure cause estimation step of storing the failure cause estimation result obtained in the data storage step as part of failure information indicating the occurrence of the failure;
Whether or not past analysis information is similar to the analysis information for a given period is determined by comparing the data on the time axis, and if it is determined to be similar, it corresponds to the past analysis information Using the failure information, calculate the certainty of the possibility of occurrence of the failure indicated by the failure information, and perform a failure occurrence prediction step of storing the prediction result obtained according to the calculation result as the prediction information in the data storage step ,
In the failure occurrence predicting step, for a plurality of elements including various state quantities of the monitored computer, the analysis information of the given period and the past analysis information are compared on the time axis, A failure monitoring method , wherein a certainty factor of the possibility of occurrence of the failure is calculated according to the number of elements determined to be present . In a failure monitoring method for monitoring the occurrence of a failure in a monitored computer system including a plurality of monitored computers connected via a network,
An information collection device for collecting, editing and storing log information of the computer stored in the monitored computer;
Using the state monitoring / analysis server that analyzes the log information stored in the information collecting device to acquire analysis information, and estimates the cause when a failure occurs and predicts the occurrence of the failure based on the log information and the analysis information ,
By the information collecting device,
A data communication step of receiving log information of the computer from the monitoring target computer and performing communication of data including log information with the state monitoring / analysis server;
The configuration information or status information of the monitored computer included in the received log information is compared with each corresponding information in the past, and when there is a configuration or status change, the change is treated as an event occurrence, and the time And editing as time series data, and when the same event content data is included in the received log information, a data editing step for editing by replacing with summary data representing the event content data,
A log saving step for saving the edited log information ;
When the received or edited log information includes information indicating the occurrence of a failure, the degree of influence that the occurrence of the failure has on the monitored computer system is calculated, and the calculated degree of influence is greater than or equal to a preset value If it is, the influence determination / notification step of notifying the occurrence of the failure to the state monitoring / analysis server by the data communication step is performed,
By the state monitoring / analysis server,
A data communication step of communicating data including log information with the information collection device;
A data storage step for storing the received log information and various data acquired in the server;
If the received log information includes the abbreviated data, editing is performed to restore the abbreviated data to the original event content data, and information indicating the occurrence of a failure is included in the log information as failure information. A data classification step of classifying and storing in the data storage step ;
A log analysis step for analyzing the occurrence of the log on the time axis using the received log information to acquire analysis information, and storing the acquired analysis information in the data storage step ;
For the occurrence of the failure indicated by the failure information, detect the candidate for the cause of the failure from the failure information when the similar failure occurred in the past, calculate the certainty of the possibility that the candidate is the cause, and according to the calculation result A failure cause estimation step of storing the failure cause estimation result obtained in the data storage step as part of failure information indicating the occurrence of the failure;
Whether or not past analysis information is similar to the analysis information for a given period is determined by comparing the data on the time axis, and if it is determined to be similar, it corresponds to the past analysis information Using the failure information, calculate the certainty of the possibility of occurrence of the failure indicated by the failure information, and perform the failure occurrence prediction step of storing the prediction result obtained according to the calculation result as the prediction information in the data storage step A fault monitoring method characterized by the above. In a fault monitoring program for realizing by a computer a fault monitoring system that monitors the occurrence of a fault in a monitored computer system composed of a plurality of monitored computers connected via a network,
The previous year's fault monitoring system
An information collection device for collecting, editing and storing log information of the computer stored in the monitored computer;
A state monitoring / analysis server for analyzing the log information stored in the information collecting device to obtain analysis information, and for estimating the cause when a failure occurs and predicting the occurrence of the failure based on the log information and the analysis information is provided If
In the computer constituting the information collecting apparatus,
A data communication function for receiving log information of the computer from the monitoring target computer and performing communication of data including log information with the state monitoring / analysis server;
The configuration information or status information of the monitored computer included in the received log information is compared with each corresponding information in the past, and when there is a configuration or status change, the change is treated as an event occurrence, and the time And editing as time series data, and when the received event information includes the same event content data, a data editing function for editing by replacing it with summary data representing the event content data,
Realize log saving function to save the edited log information ,
In the computer constituting the state monitoring / analysis server,
A data communication function for communicating data including log information with the information collection device;
A data storage function for storing received log information and various data acquired in the server;
If the received log information includes the abbreviated data, editing is performed to restore the abbreviated data to the original event content data, and information indicating the occurrence of a failure is included in the log information as failure information. A data classification function for classifying and storing with the data storage function;
A log analysis function for analyzing the occurrence of a log on the time axis using the received log information to acquire analysis information, and storing the acquired analysis information by the data storage function;
For the occurrence of the failure indicated by the failure information, detect the candidate for the cause of the failure from the failure information when the similar failure occurred in the past, calculate the certainty of the possibility that the candidate is the cause, and according to the calculation result A failure cause estimation function for storing the failure cause estimation result obtained by the data storage function as a part of failure information indicating the occurrence of the failure;
Whether or not past analysis information is similar to the analysis information for a given period is determined by comparing the data on the time axis, and if it is determined to be similar, it corresponds to the past analysis information Using the failure information, calculate the certainty of the possibility of occurrence of the failure indicated by the failure information, and realize the failure occurrence prediction function that stores the prediction result obtained according to the calculation result as the prediction information by the data storage function Let
By the failure occurrence prediction function, for a plurality of elements including various state quantities of the monitored computer, the analysis information of the given period and the past analysis information are compared on the time axis, and similar. A failure monitoring program characterized by calculating a certainty of the possibility of occurrence of the failure according to the number of elements determined to be present . In a fault monitoring program for realizing by a computer a fault monitoring system that monitors the occurrence of a fault in a monitored computer system composed of a plurality of monitored computers connected via a network,
The previous year's fault monitoring system
An information collection device for collecting, editing and storing log information of the computer stored in the monitored computer;
A state monitoring / analysis server for analyzing the log information stored in the information collecting device to obtain analysis information, and for estimating the cause when a failure occurs and predicting the occurrence of the failure based on the log information and the analysis information is provided If
In the computer constituting the information collecting apparatus,
A data communication function for receiving log information of the computer from the monitoring target computer and performing communication of data including log information with the state monitoring / analysis server;
The configuration information or status information of the monitored computer included in the received log information is compared with each corresponding information in the past, and when there is a configuration or status change, the change is treated as an event occurrence, and the time And editing as time series data, and when the received event information includes the same event content data, a data editing function for editing by replacing it with summary data representing the event content data,
Log saving function to save the edited log information,
When the received or edited log information includes information indicating the occurrence of a failure, the degree of influence that the occurrence of the failure has on the monitored computer system is calculated, and the calculated degree of influence is greater than or equal to a preset value If it is, the impact determination / notification function for notifying the occurrence of the failure to the state monitoring / analysis server by the data communication function is realized,
In the computer constituting the state monitoring / analysis server,
A data communication function for communicating data including log information with the information collection device;
A data storage function for storing received log information and various data acquired in the server;
If the received log information includes the abbreviated data, editing is performed to restore the abbreviated data to the original event content data, and information indicating the occurrence of a failure is included in the log information as failure information. A data classification function for classifying and storing with the data storage function ;
A log analysis function for analyzing the occurrence of a log on the time axis using the received log information to acquire analysis information, and storing the acquired analysis information by the data storage function ;
For the occurrence of the failure indicated by the failure information, detect the candidate for the cause of the failure from the failure information when the similar failure occurred in the past, calculate the certainty of the possibility that the candidate is the cause, and according to the calculation result A failure cause estimation function for storing the failure cause estimation result obtained by the data storage function as a part of failure information indicating the occurrence of the failure;
Whether or not past analysis information is similar to the analysis information for a given period is determined by comparing the data on the time axis, and if it is determined to be similar, it corresponds to the past analysis information Using the failure information, calculate the certainty of the possibility of occurrence of the failure indicated by the failure information, and realize the failure occurrence prediction function that stores the prediction result obtained according to the calculation result as the prediction information by the data storage function Fault monitoring program characterized by causing

Images