JP2016029520A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to accurately estimate a fault occurrence portion even if configuration information is insufficient or it is difficult to change the configuration information.SOLUTION: A configuration-information-deformation-rule application unit 205 generates new configuration information for defining a system configuration of a monitoring target system 30 by a new data structure for fault occurrence portion estimation, from a copy of configuration information defining the system configuration of the monitoring target system 30 by an existing data structure. A fault portion estimation unit 202 analyzes the system configuration defined by the new configuration information and estimates a fault occurrence portion within the monitoring target system 30 when a fault occurs to the monitoring target system 30.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for estimating a fault occurrence location in a monitoring target system.

In some cases, an IT (Information Technology) service provider provides an operation monitoring service for a system built by a customer.
In the operation monitoring service, a network device or server to be monitored is monitored by a monitoring device, and a customer is notified or a failure is dealt with based on an alarm notified from there.
In such IT services, there are many cases in which standards such as Information Technology Infrastructure Library (ITIL) and ISO / IEC 20000 (Information Technology Service Management System) are introduced.
When standardizing and automating operation monitoring services using such a standard, an alarm is provided by preparing a configuration management database (CMDB) that stores configuration information of network devices or servers to be monitored. Response can be made more efficient.

Alarm handling is performed in the following manner.
First, the content of the alarm is confirmed at the operation monitoring center when an alarm occurs.
Next, the configuration information of the network device or server in which the alarm has occurred is confirmed, and it is estimated which device has the failure.
After that, failure analysis and response are performed according to the failure handling procedure manual.
If the operation monitoring center cannot cope, an alarm is notified to a predetermined contact such as a department or customer who manages the network device or server in which the failure has occurred.
If it takes time to estimate the fault location, the quality of the operation monitoring service will be degraded.
In response to an alarm, there are roughly two types of conventional techniques for confirming configuration information and estimating a fault location efficiently.

The first is estimation of a failure location by an association rule (for example, Patent Document 1).
For example, it is possible to estimate the location of failure by defining the connection conditions of the configuration, the indicators and their states, the cause, etc. as correlation rules and applying them to multiple alarms occurring within a set time. To do.

The second is estimation of a fault location using an adjacency matrix (for example, Patent Document 2).
For example, in the case of performing integrated monitoring of a system composed of a plurality of hierarchies, monitoring is performed by generating an adjacency matrix representing a connection relationship between devices and performing matrix calculation.

International Publication WO2011 / 039825 JP 2012-222371 A

FIG. 31 shows a configuration example of a monitoring target system and an alarm occurrence location.
In FIG. 31, a rectangle is a system configuration item (CI), an arrow is a dependency between CIs, a lightning bolt is an alarm generated, and a to k are CI identifiers.
In FIG. 31, the CI and the dependency relationship are that the customer “Company A” uses “mail service”, and “mail service” is configured by “mail system”. The “mail system” is composed of “server 1” to “server 7”, and “server 1” to “server 7” are influenced by “FW (firewall)”.
Further, the alarm indicates that the alarm has occurred in “server 1” to “server 7”, “FW”.
The CI that generated the alarm is also referred to as an alarm generation CI or an alarm generation system component.
In the example of FIG. 31, it is assumed that a failure has occurred at “FW” (identifier: k), and an alarm has occurred that “server 1” to “server 7” cannot confirm communication due to a link down. .
In FIG. 31, the root cause of a plurality of generated alarms can be estimated to be “FW” (identifier: k) at the dependency destination (the tip of the arrow) by following the dependency relationship of the configuration information. .

For the example shown in FIG. 31, the above-described conventional technique can be used.
As in the first prior art, it is possible to estimate the root cause by defining a correlation rule that a communication alarm occurs in another server due to an FW failure.
Further, as in the second prior art, an adjacent matrix is created from the dependency relationship between CIs to obtain a distance matrix, and an alarm is generated from the CI at the highest level (“Company A” in FIG. 31). It is also possible to estimate the CI with the longest distance among the CIs as the root cause.
FIG. 32 is an example in which an alarm occurrence location is mapped to a distance matrix created based on FIG.
32 represent a to k which are identifiers of the CI shown in FIG.
The column of the distance matrix is the CI name that is the starting point for determining the distance, and the row is the CI name for determining the distance.
For example, “1” in b row of column a means that the distance from a to b is 1.
In this example, the reverse display portion in the table indicates the alarm occurrence CI (CI where lightning is displayed in FIG. 31).
Based on a “Company A”, which is the highest CI, k “FW”, which is the farthest among the alarm generation CIs, is estimated as a root cause.

  In the case of a method for estimating the root cause by using an association rule, it is necessary to set a rule for each type of alarm that can occur and for each occurrence location, and the cost for creating and maintaining the rule becomes enormous.

In addition, when detailed configuration information cannot be used, for example, when some or all of the monitoring target systems are not under the control of the company, the failure location may not be estimated by the conventional technology.
Acquisition of detailed configuration information and the current connection state can also be realized by using an auto-discovery function attached to the CMDB product.
Auto-discovery functions can often be used without problems in monitoring the operation of in-house systems.
However, for example, when the IT service provider keeps the customer asset system in the data center by housing service, or does not know how the customer uses the virtual server by virtual server rental service Cannot use the auto-discovery function or the like without the customer's consent.
In many cases, the configuration information disclosed by the customer alone is insufficient for applying the conventional technology.
When the conventional technique is applied in such a situation, detailed information such as dependency relationships between servers and application information is insufficient, and the root cause cannot be estimated.

FIG. 33 and FIG. 34 are examples in the case where an alarm that cannot be used to estimate the failure location is generated by the managed configuration information.
FIG. 33 shows that an alarm has occurred in “Server 1” and “Server 2”.
FIG. 34 is an example in which alarm occurrence locations are mapped to the distance matrix created based on FIG.
In this example, the distance from the highest CI “Company A” to “Server 1” and “Server 2” is the same, and it cannot be estimated which is the root cause.

Furthermore, there is a problem that the data structure of the CMDB that stores the configuration information cannot be easily changed.
The data structure of the CMDB is usually determined at the time of designing the operation monitoring system and is not changed during system operation.
When the monitoring target system changes, the data structure is not changed, and the individual configuration information is updated.
Therefore, for example, when a situation occurs in which it is desired to manage new configuration information that is not in the CMDB data structure only in the operation monitoring of a certain customer, such information cannot be stored in the CMDB, and the information is monitored. It is difficult to take advantage of.

  The main object of the present invention is to solve the above-described problems, and to enable accurate estimation of a fault occurrence location even when configuration information is insufficient or even when configuration information is difficult to change. Main purpose.

An information processing apparatus according to the present invention includes:
The location of failure where the system configuration of the monitored system is defined by a data structure for estimating the location of failure, which is different from the default data structure, from a copy of the configuration information that defines the system configuration of the monitored system with the default data structure A configuration information generator for estimation that generates configuration information for estimation;
An analysis estimating unit that analyzes a system configuration defined by the configuration information for estimating a failure occurrence location and estimates a failure occurrence location in the monitored system when a failure occurs in the monitored system It is characterized by that.

  According to the present invention, even when the configuration information is insufficient or when it is difficult to change the configuration information, the failure location estimation configuration is applied to the configuration information copy by applying the failure location estimation data structure to the copy of the configuration information. While generating information and maintaining the configuration information, it is possible to accurately estimate the location of failure using the configuration information for failure location estimation.

FIG. 3 is a diagram showing a system configuration example according to the first and second embodiments. The figure which shows the example of a configuration management model. The figure which shows the example of a configuration management model. The figure which shows the example of the structure information managed by CMDB. The figure which shows the example of the structure information managed by CMDB. The figure which shows the example of the structure information managed by CMDB. The figure which shows the example of a structure information deformation | transformation rule. The figure which shows the example of an attribute addition. The figure which shows the example which deform | transformed structure information by the structure information deformation | transformation rule. The figure which shows the example which deform | transformed structure information by the structure information deformation | transformation rule. The figure which shows the example which deform | transformed structure information by the structure information deformation | transformation rule. FIG. 3 is a flowchart showing an operation flow of the monitoring system according to the first embodiment. FIG. 5 is a flowchart showing a flow of operations of a configuration information modification rule application unit according to the first embodiment. FIG. 3 is a flowchart showing a flow of operation of a distance information generation unit according to the first embodiment. FIG. 4 is a diagram showing an example of distance information according to the first embodiment. FIG. 4 is a diagram showing an example of distance information according to the first embodiment. FIG. 4 is a diagram showing an example of distance information according to the first embodiment. FIG. 4 is a diagram showing an example of distance information according to the first embodiment. The flowchart figure which shows the flow of operation | movement of the failure location estimation part which concerns on Embodiment 1. FIG. The flowchart figure which shows the flow of operation | movement of the distance information generation part which concerns on Embodiment 2. FIG. FIG. 10 is a diagram showing an example of a CI list for each distance according to the second embodiment. The flowchart figure which shows the flow of operation | movement of the failure location estimation part which concerns on Embodiment 2. FIG. FIG. 6 is a diagram showing a system configuration example according to Embodiments 3 and 4. FIG. 9 is a flowchart showing a flow of operations of a failure point estimation unit according to Embodiment 3. The figure which shows the example which traces the graph of the structure information which concerns on Embodiment 3. FIG. The flowchart figure which shows the flow of operation | movement of the failure location estimation part which concerns on Embodiment 4. FIG. FIG. 10 is a diagram illustrating an example of dependency weighting according to the fourth embodiment. FIG. 10 is a diagram showing a system configuration example according to a fifth embodiment. FIG. 10 is a diagram showing an example of configuration information modification rules according to the fifth embodiment. The figure which shows the hardware structural example of the monitoring system which concerns on Embodiment 1-5. The figure which shows the example of structure information and alarm generation. The figure which shows the example of structure information and alarm generation. The figure which shows the example of the alarm generation which cannot estimate a failure location. The figure which shows the example of the alarm generation which cannot estimate a failure location.

Embodiment 1 FIG.
In the present embodiment and subsequent embodiments, it is possible to improve the accuracy of fault location estimation when the configuration information is insufficient or when the data structure of the CMDB is difficult to change, and maintenance is also possible. A configuration capable of reducing the cost will be described.

  More specifically, with respect to insufficient configuration information stored in the CMDB, the configuration information is transformed outside the CMDB by a rule that transforms the configuration information, and the distance information between the CIs based on the transformed configuration information The structure which improves the precision of fault location estimation by calculating | requiring is demonstrated.

First, the configuration will be described.
FIG. 1 shows a system configuration example according to the present embodiment.
The system according to the present embodiment includes a client device 10, a monitoring system 20, and a monitoring target system 30.
The monitoring target system 30 is an example of a system that is monitored by the monitoring system 20, and includes network devices (FW: firewall, SW: switch), a server, and the like. The monitoring system 20 monitors the monitoring target system 30 via the Internet or an intranet.
The client device 10 is a terminal device used by an operator who performs monitoring.
The operator confirms information displayed by the monitoring system 20 from the client device 10.
The monitoring system 20 monitors the monitoring target system 30 and detects a failure.
Further, the cause of the failure is estimated from the detected failure, and the estimation result is displayed to the operator.
The monitoring system 20 corresponds to an example of an information processing device.

In the monitoring system 20, the screen display unit 201 generates a screen to be displayed on the client device 10 when the operator uses the monitoring system 20.
The operator uses the screen displayed by the screen display unit 201 through the client device 10 to check the failure that has occurred.

The failure location estimation unit 202 estimates a failure cause location using failure information generated in the monitoring target system 30 and configuration information of the monitoring target system 30.
More specifically, the failure location estimation unit 202 estimates an alarm occurrence CI (alarm generation system component) that is farthest from the highest CI as a failure occurrence location.
That is, the failure location estimation unit 202 estimates the alarm occurrence CI having the highest number of hierarchy differences from the CI of the highest hierarchy as the CI that caused the failure.
The failure location estimation unit 202 corresponds to an example of an analysis estimation unit together with a distance information generation unit 204 described later.

The failure information collection unit 203 monitors the monitoring target system 30, collects the generated failure information, and stores it in the failure information DB 211.
When displaying the failure information to the operator, the failure information collection unit 203 acquires the failure information from the failure information DB 211.
The failure information collection unit 203 has the same function as the monitoring function used in a general monitoring system.

The distance information generation unit 204 generates distance information between CIs based on the configuration information passed from the configuration information modification rule application unit 205, and stores it in the distance information DB 212.
The distance information generation unit 204 corresponds to an example of an analysis estimation unit together with the above-described failure location estimation unit 202.

The configuration information modification rule application unit 205 complements and transforms the configuration information based on the configuration information stored in the CMDB 213 and the configuration information modification rule 214 defined in advance.
More specifically, a copy of the configuration information stored in the CMDB 213 is transformed to generate new configuration information (failure location estimation configuration information) for estimating the failure location.
The new configuration information after the deformation is transferred to the distance information generation unit 204, and the distance information is generated.
The configuration information modification rule application unit 205 corresponds to an example of an estimation configuration information generation unit.

  The monitoring system 20 in FIG. 1 describes only the minimum function necessary for confirming the failure that has occurred. For example, the failure response status recording function and the failure rule based on the correlation rule described above as the prior art are used. An estimation function may be included.

Next, the information storage device in the monitoring system 20 will be described.
The failure information DB 211 holds the failure information collected by the failure information collection unit 203.
The distance information DB 212 holds distance information between CIs created by the distance information generation unit 204.
The format of the distance information may be any format such as a distance matrix or a list.
The CMDB 213 is a configuration management database having a data structure defined in the configuration management model.
An example of the configuration management model and stored configuration information will be described later.
The configuration information modification rule 214 is a rule for complementing and transforming configuration information stored in the CMDB 213.
An example of the configuration information modification rule 214 will be described later.

  Next, an example of the configuration management model of the CMDB 213 and stored configuration information will be described.

2 and 3 are examples of the configuration management model.
FIG. 2 shows an example of a CI type defined as a configuration management model.
An example of the CI type defined here will be described.
“Customer” is a CI type that stores attributes of customers who use the service, and has attributes such as customer name and abbreviation, for example.
When customer information is stored in the CMDB 213, a CI is created using a customer CI type.
FIG. 3 is an example of the dependency relationship between the CI types defined in FIG.
The arrows in the figure indicate dependency relationships.
In this example, “customer” depends on “service”, “service” depends on “system”, “system” depends on “node”, and “node” becomes “hardware” and “application”. Dependent.
When storing the CI, the dependency relationship between the CIs can be defined at the position of the dependency relationship defined in the model.
In this specification, the source of an arrow is called a dependency relationship source, and the tip of the arrow is called a dependency relationship destination.

4, 5, and 6 are examples of configuration information managed by the CMDB 213.
Here, it is assumed that the configuration management model shown in FIGS. 2 and 3 is defined.
FIG. 4 is an example of correspondence between CI names and CI type names, and shows the relationship between configuration information created as CIs and their type names.
“a” means that the CMDB 213 manages the CI “Company A” whose CI model name is “customer”.
b means that the CMDB 213 manages the CI “mail service” with the CI type name “service”.
The same applies to the subsequent steps.
FIG. 5 is an example of the dependency relationship between CIs shown in FIG.
In this example, “Company A” uses “Mail Service”, “Mail Service” is composed of “Mail System”, and “Mail System” is composed of “Server 1” to “Server 7”. It is shown that “Server 1” to “Server 7” are affected by FW.
FIG. 6 is a relationship diagram of the CI that visualizes the information of FIGS. 4 and 5.

As shown in FIGS. 4 to 6, the configuration information before transformation by the configuration information transformation rule applying unit 205 defines the system configuration of the monitoring target system 30 in a default hierarchical structure that is a default data structure.
The configuration information modification rule applying unit 205 applies the configuration information modification rule to such configuration information, and the system of the monitoring target system 30 in a new hierarchical structure that is a new data structure for estimating a failure location. New configuration information (configuration information for estimating a fault occurrence location) that defines the configuration is generated.

FIG. 7 shows an example of the configuration information modification rule 214.
No is a rule number, and the configuration information modification rule 214 is a rule definition for transforming configuration information.
The description method of the configuration information modification rule is not limited.
For example, predicate logic or tabular form may be used.
Details of the rule will be described later as a configuration information modification method based on the configuration information modification rule in conjunction with FIGS. 7, 8, and 9.

FIG. 8 shows an example of attributes to be added to complement the configuration information stored in the CMDB 213.
The example shows a case where it is desired to add a role attribute of “DB server” to the CI of “Server 1” and “Web server” to the CI of “Server 2”.
The information shown in FIG. 8 is set by the operator from the client device 10 to the monitoring system 20, for example.

9, FIG. 10 and FIG. 11 show examples in which the configuration information is modified by the configuration information modification rule.
That is, FIG. 9 is a diagram obtained by modifying the relationship diagram of the CI shown in FIG. 6 with the configuration information modification rule of FIG.
FIG. 10 is a diagram in which the correspondence between the CI name and the CI type name shown in FIG. 4 is modified by the configuration information modification rule of FIG.
FIG. 11 is a diagram in which the dependency relationship between the CIs shown in FIG. 5 is modified by the configuration information modification rule of FIG.

No. of FIG. The first rule is that, when the node attribute is a DB server, the CI type “DB” is added as the CI type, and a dependency relationship from the added “DB” to the node having the DB server attribute is added. Is.
The change of the configuration information according to this rule corresponds to the number “1” surrounded by a dotted-line square in FIG.
In other words, the dependency from the CI “DB” to the server 1 is the rule No. Added by 1.
No. of FIG. The second rule is that if the node attribute is a Web server, the CI type “Web” is added as a CI, and a dependency relationship from the added “Web” to the node having the Web server attribute is added. Is.
Changing the configuration information according to this rule corresponds to the number “2” surrounded by a dotted-line square in FIG.
In other words, the dependency relationship between the CI “Web” and the server 2 is the rule number. Added by 2.
Similarly, CIs and dependencies are added to rules 3 to 4.
As described above, the configuration information modification rule application unit 205 includes a system configuration element (CI) having a specific attribute described in the configuration information modification rule 214 in a plurality of system configuration elements (CI) included in the monitoring target system 30. Is included. If such a CI is included, a new CI and a new dependency are added according to the configuration information modification rule 214.

By adding the CI and the dependency relationship, the configuration information modification rule application unit 205 uses the hierarchical structure of the monitoring target system 30 defined in the original configuration information as shown in FIG. A suitable hierarchical structure of FIG. 9 can be changed.
As described above, the configuration information modification rule application unit 205 adds CIs and dependencies as exemplified in FIGS. 9 to 11 to the copy of the configuration information stored in the CMDB 213.
For this reason, new configuration information can be acquired for estimating the location of failure without changing the configuration information in the CMDB 213.

Next, the operation will be described.
FIG. 12 is a flowchart of fault location estimation by the monitoring system 20.

S 01 is an operation of the configuration information modification rule applying unit 205, and the configuration information is modified according to the configuration information modification rule 214.
The detailed flow of S01 will be described with reference to FIG.

S02 is an operation of the distance information generation unit 204. If the configuration information modified in S01 or the configuration information modification rule 214 is not set, the distance information is obtained based on the configuration information stored in the CMDB 213. Generate.
The detailed flow of S02 will be described with reference to FIG.

In S03, it is confirmed whether or not a failure has occurred.
If a failure has occurred, the process proceeds to S04.
If no failure has occurred, exit.

S04 is the operation of the failure location estimation unit 202, and performs failure location estimation based on the currently occurring failure information and the distance information obtained in S02.
The detailed flow of S04 will be described with reference to FIG.

  S05 is the operation of the screen display unit, and displays the event information and the result of failure location estimation on the screen.

  FIG. 13 is a flowchart of the configuration information modification rule application unit 205.

In S011, the configuration information modification rule application unit 205 acquires the configuration information modification rule 214.
In S012, the configuration information modification rule application unit 205 confirms the acquired configuration information modification rule 214 and confirms whether there is a rule for which distance information is not generated from the configuration information modified by the configuration information modification rule 214. .
That is, the presence / absence of a rule that has been changed (new / corrected / deleted) or a rule that has not been changed but for which distance information has not been generated is confirmed.
For example, when a rule is changed, a flag indicating a change content (new / correction / deletion) is attached to the rule, or distance information is generated for the rule when distance information is generated. This can be determined by providing a mechanism for attaching a flag representing a certain thing.
If no distance information has been generated, the process proceeds to S013.
If there is no configuration information transformation rule for which no distance information has been generated, the process ends.
In step S013, the configuration information modification rule application unit 205 acquires a copy of the configuration information stored in the CMDB 213.
In S014, a copy of the acquired configuration information is transformed based on the configuration information transformation rule 214, and the transformed configuration information is temporarily created.
The deformation method is as shown in FIGS.

  FIG. 14 is a flowchart of the distance information generation unit 204.

In S021, the distance information generation unit 204 determines whether the distance matrix to be generated has already been generated and stored in the distance information DB 212.
If it has been generated, the process ends.
If not generated, the process proceeds to S022.

In step S022, the distance information generation unit 204 acquires configuration information for which a distance is desired.
The configuration information here is the configuration information modified by the configuration information modification rule applying unit 205 when the configuration information modification rule 214 is defined, and the configuration information obtained from the CMDB 213 when there is no configuration information. Means no configuration information.

In S023, the distance information generation unit 204 generates a distance matrix having a different search route based on the acquired configuration information.
15, 16, 17, and 18 are examples of distance matrices generated based on the configuration information in FIG.
FIG. 15 is a normal distance matrix (distance of the shortest path from a).
FIG. 16 is a distance matrix when a route of a → b → c → m → e → k is passed.
FIG. 17 is a distance matrix when a route of a → b → c → l → d → k is passed.
FIG. 18 is a distance matrix when a route of a → b → c → m → l → d → k is passed.
The alphabet in the figure corresponds to the identifier in FIG.
Normally, the distance matrix of FIG. 15 is used, but when there are a plurality of routes, the distance along each route is reflected in the distance matrix.
In the case of FIG. 16, since the route is a → b → c → m → e → k, the distance between e and k is different from that in FIG.
In FIG. 16, FIG. 17, and FIG. 18, the CI not related to the route is set to the shortest distance value as in FIG.

  In S024, the distance information generation unit 204 stores the generated distance matrix in the distance information DB 212.

  FIG. 19 is a flowchart of the failure location estimation unit 202.

In S031, the failure location estimation unit 202 acquires currently occurring failure information from the failure information DB 211.
In S032, the failure point estimation unit 202 acquires distance information from the distance information DB 212.
In step S033, the failure location estimation unit 202 estimates the CI having the longest distance from the plurality of distance matrices as the failure location.

The description will be made with reference to FIGS. 15, 16, 17, and 18.
For example, assume that an alarm is generated at l and m.
In the normal distance matrix of FIG. 15, the distances l and m are the same, and it is impossible to estimate which is the fault location.
16 and 17 are the same as those in FIG. 15, but in FIG. 18, the distance of l is 4 and the distance of m is 4. 3
As a result, the failure location estimation unit 202 can estimate that an alarm has occurred and l that is far away is the failure location.

  As described above, the failure location estimation unit 202 can estimate a failure location by creating a distance matrix of a plurality of paths when estimating the failure location.

As described above, the monitoring system 20 according to the present embodiment makes it possible to use information that is not stored in the CMDB 213 for operation monitoring.
Therefore, it is possible to estimate a fault location with higher accuracy than only the configuration information stored in the CMDB 213.
Further, since the configuration information is deformed outside the CMDB 213, there is no need to change the structure of the CMDB 213.
Since the configuration information modification rule is considered to require a smaller number of settings than the correlation rule of the prior art, it is possible to reduce the maintenance cost of the rule.

  As described above, in the present embodiment, configuration information that has already been managed in the CMDB or the like is configured outside the CMDB according to the configuration information modification rule without changing the data structure of the CMDB or the managed configuration information. A monitoring system has been described that can improve the accuracy of fault location estimation by transforming.

  Further, in the present embodiment, a monitoring system that generates a plurality of distance matrices having different paths from the configuration information and estimates the CI that is farthest from the highest CI and unique among the alarm generation CIs as a fault location. explained.

Embodiment 2. FIG.
The system configuration according to the present embodiment is as shown in FIG. 1 and is the same as that of the first embodiment.
As for the operation, since the distance information generation unit 204 and the fault location estimation unit 202 are different from those in the first embodiment, only the difference will be described.

FIG. 20 is a flowchart of the distance information generation unit 204 according to the second embodiment.
In step S043, the distance information generation unit 204 creates a CI list for each distance from the highest CI in the configuration information to the longest path based on the acquired configuration information.
In the example of FIG. 9, the highest CI is “Company A”.
The shortest distance 1 from the top is the “mail service” of b.
Distance 2 is the “mail system” of c.
“Server 1” of d is distance 3 when the dependency is a → b → c → d, distance 4 when the dependency is a → b → c → l → d, a → b → c → m → l → d In the case of, there are three patterns of distance 5.
The longest distance 6 is “FW” of k.
An example of the CI list created in S022 is shown in FIG.
The CI of distance 1 is “mail service” of b, and the CI of distance 6 is “FW” of k.

  FIG. 22 is a flowchart of the failure location estimation unit 202 according to the second embodiment.

In S053, the failure location estimation unit 202 compares the alarm occurrence CI in order from the longest path distance list, and estimates the first occurrence of the alarm occurrence CI as the failure occurrence location.
As described above, also in the present embodiment, the failure location estimation unit 202 estimates the alarm occurrence CI having the highest number of hierarchy differences from the CI of the highest hierarchy as the CI that caused the failure.

The operation of the fault location estimation unit 202 will be described using the CI list in FIG.
It is assumed that an alarm is generated in “server 1” of d and “server 2” of e.
When comparison is made from the list of 6 of the longest path, d is found in the list of distance 5, and it can be estimated that “server 1” is the failure location.
If the comparison is made from the list of 1 of the shortest path, both d and e are found in the list of distance 3, and no more trouble spots can be narrowed down.

  As described above, in the present embodiment, when estimating a failure location, a failure location can be estimated by creating a CI list for each distance and searching for an alarm occurrence CI from the longest distance.

  As described above, in the present embodiment, a CI list for each distance is created from the configuration information, an alarm occurrence CI is searched from the CI list with the longest distance, and the longest and unique CI is estimated as a fault location. Explained the system.

Embodiment 3 FIG.
In the first and second embodiments, the distance matrix and the CI list for each distance are created when there are a plurality of routes. However, a method of counting the distance in a route having a large number of alarms on the route is also conceivable. .
In this method, a fault location is estimated by following a configuration information graph without using a distance matrix or a CI list for each distance.

A system configuration example according to the present embodiment is as shown in FIG.
In the present embodiment, the distance information generation unit 204 and the distance information DB 212 shown in FIG. 1 are unnecessary.
As for the operation, since the failure point estimation unit 202 is different from those of the first and second embodiments, only the difference will be described.

FIG. 24 is a flowchart of the failure location estimation unit 202 according to the third embodiment.
In S62, the failure location estimation unit 202 traces the configuration information and estimates the alarm occurrence CI having the longest distance as a failure location on the route having a large number of alarm occurrence CIs.

The operation of S62 will be described using an example of tracing the configuration information graph of FIG.
In this example, the following three routes are conceivable from the highest CI a to l or m where an alarm is generated.
A → b → c → l route: distance is 3, alarm occurrence CI number is 1 on route
A → b → c → m route: distance is 3 and number of alarm occurrence CIs on route is 1
A → b → c → m → l route: distance is 4 and number of alarm occurrence CIs on route is 2
In this case, since the third route has the largest number of alarms on the route, l, which is the alarm occurrence CI having the longest distance on the route, is estimated as the failure point.

  As described above, in the embodiment, when estimating a failure location, the failure location can be estimated by following a route with a large number of alarm occurrence CIs.

  As described above, in the present embodiment, the monitoring system that traces the configuration information graph, counts the number of generated alarms on the route, and estimates the farthest alarm occurrence CI on the route with the largest number of generated alarms as the failure point has been described. .

Embodiment 4 FIG.
A method of estimating a fault location that searches by weighting the dependency relationship is also conceivable.
When the distance matrix is used as in the first embodiment, the distance from the highest CI to the target CI is used.
In the fourth embodiment, the distance value is not assigned to the CI, but weighting is performed on the arrow portion indicating the dependency.
That is, also in the present embodiment, the failure location estimation unit 202 estimates the alarm occurrence CI having the highest number of hierarchy differences from the CI of the highest hierarchy as the CI that caused the failure. Instead of distance, the dependency weight is used as the difference number.

The system configuration is shown in FIG. 23 and is the same as that of the third embodiment.
As for the operation, since the failure point estimation unit 202 is different from that of the third embodiment, only the difference will be described.
FIG. 26 is a flowchart of the failure location estimation unit 202 in the fourth embodiment.
FIG. 27 is an example of dependency weighting.

In S72, the failure location estimation unit 202 weights the dependency relationship of the configuration information.
The weight of the arrow starting from the highest CI is set to 1.
The weight of the arrow with the CI that is the end point as a new start point is set to 2 by adding 1.
Similarly, the weight of the arrow is added.
In the case of a CI having a plurality of end points, the larger weight is adopted.
In S73, the failure location estimation unit 202 marks each CI based on the failure information.
In S74, the failure location estimation unit 202 estimates that the CI at the end point of the arrow having the largest weight value as the failure location, where alarm occurrence marks are attached to both ends of the dependency relationship arrows.
In FIG. 27, the filled CI is estimated as a failure location.

  As described above, in the embodiment, when the failure location is estimated, the failure location can be estimated by weighting the dependency relationship.

  As described above, in the present embodiment, weighting of the dependency information of the configuration information and marking of the alarm occurrence to the CI are performed, and the alarm occurrence mark is attached to both ends of the arrow indicating the dependency relationship, and the weight is the highest among them. A monitoring system that estimates the CI at the end point of the large arrow as the fault location has been described.

Embodiment 5 FIG.
In the first to fourth embodiments, the configuration information modification rule is applied according to the attribute of the CI role.
That is, in the first to fourth embodiments, the CI having the attributes (DB server, Web server) defined in the configuration information modification rule (FIG. 7) is specified by the information in FIG.
In the present embodiment, the configuration information transformation rule is applied according to data (alarm, event) raised from the CI in the monitoring target system 30.

FIG. 28 is a configuration diagram according to the fifth embodiment.
Only differences from the first to fourth embodiments will be described below.
The log extraction unit 221 acquires failure information (event information, alarm information, etc.) collected from the monitoring target system 30 from the failure information collection unit 203, and extracts a log as a precondition for applying the configuration information transformation rule Then, the information is passed to the configuration information modification rule application unit 205.

FIG. 29 is an example of the configuration information modification rule.
The rule application condition is a search condition used when the log extracting unit 221 extracts a log.
Corresponding when the character string included in the log and alarm generated by the CI matches the rule application condition, or when the folder name of the folder where the log file generated by the CI matches the rule application condition The configuration information transformation rule is applied.
In FIG. 29, as in the other embodiments, only the minimum necessary functions for confirming the failure that has occurred are described. For example, the failure response status recording function and the correlation described above as the conventional technology are described. A failure location estimation function based on rules may be included.

Next, the operation will be described.
Only differences from the first to fourth embodiments will be described below.
The log extraction unit 221 acquires the rule application condition from the configuration information modification rule 214.
Next, the log extraction unit 221 acquires failure information from the failure information collection unit 203 and searches for a log that matches the rule application condition.
If there is a matching log, the CI that has output the log is specified, and the CI information is passed to the configuration information modification rule application unit 205.
The configuration information modification rule application unit 205 applies the configuration information modification rule to the CI.
The application method is the same as in the first to fourth embodiments.

As described above, in the present embodiment, when estimating the failure location, the configuration information modification rule to be applied can be changed according to the content of the failure information collected from the monitoring target.
As a result, when the attribute cannot be determined for the CI, or when there is a possibility that a plurality of attributes may be entered for one CI, an appropriate configuration information transformation rule is dynamically applied according to the failure information, The fault location can be estimated.

  As described above, in the present embodiment, the monitoring system that applies the configuration information modification rule according to information such as alarms and events raised from the monitoring target system has been described.

In the first to fifth embodiments, an example of a rule for adding a CI and a dependency relationship has been described. However, a rule for deleting a CI or a dependency relationship may be defined.
In that case, the failure location can be narrowed down according to the content of the failure information.
In addition, when the dependency of the original configuration information is cyclically referenced, it is possible to delete the dependency by the configuration information modification rule corresponding to the failure information and to make the configuration information without circulation, and then specify the failure location. Is possible.

  As described in the first to fifth embodiments, by adding information that is not stored in the CMDB to the copy of the configuration information, it is possible to change the failure without changing the data structure of the CMDB or the stored configuration information. The accuracy of location estimation can be increased.

As mentioned above, although embodiment of this invention was described, you may implement in combination of 2 or more among these embodiment.
Alternatively, one of these embodiments may be partially implemented.
Alternatively, two or more of these embodiments may be partially combined.
In addition, this invention is not limited to these embodiment, A various change is possible as needed.

Finally, a hardware configuration example of the monitoring system 20 shown in the first to fifth embodiments will be described with reference to FIG.
The monitoring system 20 is a computer, and each element of the monitoring system 20 can be realized by a program.
As a hardware configuration of the monitoring system 20, an arithmetic device 901, an external storage device 902, a main storage device 903, a communication device 904, and an input / output device 905 are connected to the bus.

The arithmetic device 901 is a CPU (Central Processing Unit) that executes a program.
The external storage device 902 is, for example, a ROM (Read Only Memory), a flash memory, or a hard disk device.
The main storage device 903 is a RAM (Random Access Memory).
The communication device 904 is, for example, a NIC (Network Interface Card).
The input / output device 905 is, for example, a mouse, a keyboard, a display device or the like.

The program is normally stored in the external storage device 902, and is loaded into the main storage device 903 and sequentially read into the arithmetic device 901 and executed.
The program is a program that realizes a function described as “unit” shown in FIG.
Further, an operating system (OS) is also stored in the external storage device 902. At least a part of the OS is loaded into the main storage device 903, and the arithmetic device 901 executes the OS as shown in FIG. 1 and FIG. Executes a program that realizes the function of "~ part".
In the description of the first to fifth embodiments, “determination of”, “determination of”, “analysis of”, “estimation of”, “generation of”, “calculation of”, “ Information, data and signal values indicating the results of the processing described as "extraction of", "setting of", "transformation of", "change of", "selection of", "comparison of", etc. And variable values are stored in the main storage device 903 as files.

  Note that the configuration of FIG. 30 is merely an example of the hardware configuration of the monitoring system 20, and the hardware configuration of the monitoring system 20 is not limited to the configuration illustrated in FIG. 30, and may be other configurations. .

  In addition, the information processing method according to the present invention can be realized by the procedure shown in the first to fourth embodiments.

  DESCRIPTION OF SYMBOLS 10 Client apparatus, 20 Monitoring system, 30 Monitoring target system, 201 Screen display part, 202 Fault location estimation part, 203 Fault information collection part, 204 Distance information generation part, 205 Configuration information modification rule application part, 211 Fault information DB, 212 Distance information DB, 213 CMDB, 214 Configuration information modification rule, 221 Log extraction unit.

Claims (11)

The location of failure where the system configuration of the monitored system is defined by a data structure for estimating the location of failure, which is different from the default data structure, from a copy of the configuration information that defines the system configuration of the monitored system with the default data structure A configuration information generator for estimation that generates configuration information for estimation;
An analysis estimating unit that analyzes a system configuration defined by the configuration information for estimating a failure occurrence location and estimates a failure occurrence location in the monitored system when a failure occurs in the monitored system An information processing apparatus characterized by that. The estimation configuration information generation unit
A failure occurrence that defines a system configuration of the monitored system in a hierarchical structure for estimating a failure occurrence location different from the default hierarchical structure, from a copy of configuration information that defines a system configuration of the monitored system in a default hierarchical structure The information processing apparatus according to claim 1, wherein the configuration information for location estimation is generated. The estimation configuration information generation unit
A plurality of system components included in the monitored system are added to and deleted from the copy of the configuration information described in the predetermined hierarchical structure, and the failure occurrence location is estimated. The information processing apparatus according to claim 2, further comprising: configuration information for estimating a fault occurrence location that defines a system configuration of the monitoring target system in a hierarchical structure. The estimation configuration information generation unit
Determining whether or not system components having specific attributes are included in the plurality of system components;
The information processing apparatus according to claim 3, wherein the failure location estimation configuration information is generated when the plurality of system configuration elements include a system configuration element having the specific attribute. The estimation configuration information generation unit
The data generated by the plurality of system components is analyzed to determine whether or not the system components having the specific attribute are included in the plurality of system components. The information processing apparatus described. The estimation configuration information generation unit
Analyzing at least one of a character string included in data generated by the plurality of system components and a folder name of a folder in which the data generated by the plurality of system components is stored, and the plurality of systems 6. The information processing apparatus according to claim 5, wherein it is determined whether or not a system component having the specific attribute is included in a component. The analysis estimation unit
A plurality of alarm generation system configurations in which a plurality of system components included in the monitoring target system analyze the failure occurrence location estimation configuration information described in the failure occurrence location estimation hierarchical structure and generate an alarm About the element, for each alarm generation system component, calculate the hierarchy difference number from the highest level system component of the plurality of system components to the alarm generation system component,
The information processing apparatus according to claim 3, wherein the alarm generation system component that caused the failure is estimated from the plurality of alarm generation system components based on the calculated number of hierarchy differences. The analysis estimation unit
Estimating the alarm generating system component having the highest number of hierarchical differences from the system component of the highest hierarchy among the plurality of alarm generating system components as the alarm generating system component that caused the failure The information processing apparatus according to claim 7. The analysis estimation unit
When there are a plurality of routes from the system component of the highest hierarchy to each alarm generating system component, extract a route including the most alarm generating system components from the plurality of routes,
Of the alarm generation system components included in the extracted route, the alarm generation system component having the largest number of hierarchy differences from the system component of the highest hierarchy is estimated as the alarm generation system component that caused the failure. The information processing apparatus according to claim 7. The computer defines the system configuration of the monitored system from a copy of configuration information that defines the system configuration of the monitored system with a predetermined data structure, using a data structure for estimating a fault occurrence location that is different from the default data structure. An estimation configuration information generation step for generating fault location estimation configuration information;
When the failure occurs in the monitored system, the computer analyzes the system configuration defined by the failure location estimation configuration information, and estimates the failure location in the monitored system And an information processing method. The location of failure where the system configuration of the monitored system is defined by a data structure for estimating the location of failure, which is different from the default data structure, from a copy of the configuration information that defines the system configuration of the monitored system with the default data structure An estimation configuration information generation step for generating estimation configuration information;
An analysis and estimation step of analyzing a system configuration defined by the configuration information for estimating a fault occurrence location and estimating a fault occurrence location in the monitor target system when a fault occurs in the monitoring target system; A program characterized by being executed.

Images