JP4862446B2 - Failure cause estimation system, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an obstacle factor estimation system for estimating the fundamental factor of an obstacle without further defining by manpower complicated dependency. <P>SOLUTION: An initial model generation means 40 generates an initial obstacle generation model by modeling the correspondence relation of an event and the occurrence factor and inter-occurrence factor transition by a finite automaton based on basic model definition 20. A Baum-Welch calculation means 50 learns which probability has resulted in the transition of the status of the finite automaton corresponding to the factor, based on the initial model occurrence model and an event column 100 for learning. A Viterbi calculation means 60 searches the status transition column in which the possibility of the observation of an event column 110 for obstacle factor discovery is judged to be the highest by an obstacle occurrence model after learning. A filtering module estimates the fundamental factor of the obstacle which has occurred in a monitor object device from the status transition column. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to a failure cause estimation system, method, and program, and more particularly to a failure cause estimation system, method, and program for estimating the cause of a failure that has occurred in a monitoring target device.

  There is a system that monitors a managed device such as a computer and estimates the cause of the failure when the managed device has a failure. FIG. 7 shows the configuration of the failure cause detection device described in Patent Document 1. In the conventional failure cause detection apparatus 200, the process information acquisition unit 201 acquires process information executed on the computer system, and the environment file information acquisition unit 202 acquires system environment file information. Further, the device information acquisition unit 203 acquires information on device drivers incorporated in the computer system.

  The reference environment information acquisition unit 204 collects each information acquired by the process information acquisition unit 201, the environment file information acquisition unit 202, and the device information acquisition unit 203 when the computer system is operating normally. The information is stored in the reference environment information storage unit 205. When the inspection environment information acquisition unit 206 detects whether or not an abnormality has occurred in the computer system, each information acquired by the process information acquisition unit 201, the environment file information acquisition unit 202, and the device information acquisition unit 203 is obtained. Collected and stored in the inspection environment information storage unit 207.

  The environment information comparison / determination unit 209 compares the contents of the reference environment information storage unit 205 and the contents of the inspection environment information storage unit 207 to find a state content change. The environmental information comparison / determination unit 209 refers to the allowable range information storage unit 208 in which the state change content stores the allowable value of the state content change, and determines whether or not the found state content change exceeds the allowable range. to decide. When the abnormality cause identifying unit 211 determines that the environment information comparison / determination unit 209 exceeds the allowable range, the abnormality cause identifying unit 211 identifies the cause of the abnormality of the computer system from the state change content.

  For example, the reference environment information storage unit 205 stores “the process being executed has one nfs”. Further, the allowable range information storage unit 208 stores that the maximum value of the process being executed is “8” for the process “nfs” corresponding to the situation where the addition of the mount device is permitted. Yes. In this case, when the inspection environment information acquisition unit 206 permits the addition of the mount device and detects a state that “the process being executed has 11 nfs”, the environment information comparison determination unit 209 It is determined that the allowable range is exceeded, and an abnormality is detected. The abnormality cause identifying unit 211 identifies the cause of the abnormality being detected in the computer system because the process “nfs” has exceeded the allowable number of processes.

  Another example of a conventional system for estimating the cause of failure is described in Patent Document 2. FIG. 8 shows the configuration of a causal system model construction device used in the failure cause estimation system described in Patent Document 2. In this Patent Document 2, a causal data generation storage unit 311 which is a database representing a causal relationship, a causal data generation storage unit 312 which is a reverse mapping thereof, a plurality of events correspond to one event group, or a plurality of events A causal system model construction device that maps the relationship between a cause group and an event group by a partial causal system model configuration unit 314 using the same result data set generation unit 313 that records a relationship that causes each cause to correspond to one cause group Build 300.

  FIG. 9 shows the configuration of a cause estimation apparatus used in the failure cause estimation system described in Patent Document 2. The causal system model storage unit 324 stores the causal system model constructed by the causal system model construction apparatus 300 (FIG. 8). The cause estimation device 320 applies the cause mapping from the events of the causal system model stored in the causal system model storage unit 324 from the events recognized by the observation data recognition unit 321 to obtain the cause of the failure. Further, by applying the mapping from the cause to the event, an event that can occur from the cause is obtained. In this way, by applying the mapping from the event to the cause and the mapping from the cause to the event in a transitive manner to obtain transitive closure, the cause including a possible root cause is transitively obtained.

  Further, as a technique related to an automatic troubleshooting mechanism for troubleshooting a system, there is a technique described in Patent Document 3. In Patent Document 3, a system component that causes a system failure is modeled in a Bayesian network, which is an acyclic graph having a direction composed of nodes representing variables and arcs representing dependencies between variables, and this model is used. Automatic diagnosis of the printer system.

JP-A-8-255093 (paragraphs 0013 to 0031, FIG. 1) Japanese Patent Laid-Open No. 2004-126641 JP 2001-75808 A

  In Patent Document 1, a state content change is detected, and when the detected state content change exceeds an allowable range, the cause of the abnormality in the computer system is specified from the state change content. However, in Patent Document 1, only a direct state can be specified, and in the case where an abnormal state occurs due to the occurrence of a certain failure, the root failure cannot be specified. .

  For example, the maximum value (allowable value) of the nfs process in the situation “addition of mount device is allowed” is “12”, and the maximum value of the nfs process in the situation “addition of mount device is not allowed” is “8”. Suppose there is. If the system is currently in a state where “addition of a mount device is permitted” and the number of nfs processes is 11, the number of nfs processes is not exceeding the maximum value “12”. It is judged that there is.

  After that, it is assumed that due to the “failure of the SCSI boat”, the state where “addition of mount device is not permitted” is entered. If the number of nfs processes does not change and remains twelve, the maximum value of the nfs process in the situation “addition of mount device is not permitted” is “8”, so the number of nfs processes exceeds the maximum value. . Therefore, in Patent Document 1, it is specified as a cause of failure that the number of nfs processes exceeds an allowable value.

  However, in the above case, the original failure cause is “SCSI board failure”, and the cause that the number of nfs processes deviates from the normal value is recognized as “SCSI board failure”. If you do not respond correctly, you cannot respond correctly. In Patent Document 1, for a failure that occurs transitively, for example, a failure C that is caused directly by the failure B and that is caused directly by the failure B, only the direct failure C can be specified. There is a problem that failure A cannot be specified.

  In Patent Document 2, it is necessary to register the correspondence between the cause and the failure in advance. However, since the relationship between the general events is complicated, the correspondence is described when there are many types of failures and states. Is difficult, and it is difficult to input rules for determining the root cause of failure. In Patent Document 2, in order to reduce the difficulty of such description, failure causes are grouped and mapping between them is described. However, even when mapping is defined for each group, it is necessary to define the mapping. In addition to the time and effort required to define the group, if mapping is not done well, mapping of the cause and fault The problem remains that accuracy may drop.

FIG. 10 shows the causal relationship as a model. For example, Patent Document 2 describes an example shown in FIG. In the figure, arrows indicate causal relationships. For example, a new event y ′ is defined and y ′ is the cause of x2. This relationship
y '→ x2
Can be written. In Patent Document 2, in order to facilitate the definition of mapping, if y ′ → x2 and x2∈X1,
h (X1) = y ′
A mapping h that abstracts the causal relationship is provided. Thus, h can easily describe the causal relationship by not looking at X1 as x2 and x3. However, in order to “not look closely”, h (x3) = y ′ when x3εX1. That is, as a cause of x3, there is a problem that there is a risk that y ′, which is not causally related, is regarded as a cause.

  In Patent Document 3, DAG (a graph without a loop) is used for fault modeling. When a failure occurs, an event often occurs repeatedly from the same failure. However, since Patent Document 3 uses a DAG, there is a problem that an event that occurs repeatedly cannot be handled.

  The present invention provides a failure cause estimation system, method, and program capable of solving the above-described problems of the prior art and estimating a root cause of a failure without manually defining complicated dependency relationships. With the goal.

In order to achieve the above object, the fault cause estimation system of the present invention includes an input means for inputting a basic model definition that defines an event caused by a fault that can occur in a monitoring target device and a fault cause that causes the event, Based on the basic model definition input by the input means, an initial model generation means for generating an initial failure occurrence model in which a correspondence between an event and its occurrence cause and a transition between the occurrence causes are modeled by a finite automaton; An initial model storage unit that stores the initial failure occurrence model generated by the initial model generation unit; a learning event sequence storage unit that stores a learning event sequence that is a set of events that have occurred in the monitoring target device; The initial failure occurrence model stored in the initial model storage unit and the previous model stored in the learning event string storage unit Based on the learning event string, the transition probabilities to learn whether it has remained in a state what probability finite automaton representing the transitions between the cause of the learning event string stored in the learning event sequence storage means learning A fault occurrence model generating means for generating a fault occurrence model reflecting the transition probability learned by the transition probability learning means, and a fault occurrence model storage for storing the fault occurrence model generated by the fault occurrence model generation means According to the failure occurrence model stored in the failure occurrence model storage unit, and a failure cause discovery event sequence storage unit that stores a failure cause discovery event sequence that is a set of events that have occurred in the monitoring target device, When the failure cause finding event sequence stored in the failure cause finding event sequence storage means has the highest probability of being observed A state transition sequence generating means for obtaining the obtained state transition sequence, a state transition sequence storing means for storing the state transition sequence generated by the state transition sequence generating means, and the state transition stored in the state transition sequence storing means A column is divided based on the transition probability of the failure occurrence model stored in the failure occurrence model storage means, and the first state of each of the divided state transition sequences is determined as the root of the failure that has occurred in the monitoring target device. And a filtering module for estimating the cause.

The failure cause estimation method of the present invention is a method for estimating the root cause of a failure that has occurred in a monitoring target device using a computer, and the event caused by the failure that may occur in the monitoring target device and the event An input step for inputting a basic model definition that defines a failure cause that causes an event, and a correspondence between an event and its occurrence cause and a transition between the occurrence causes are finite based on the basic model definition input by the input step. An initial model generation step for generating an initial failure occurrence model modeled by an automaton, an initial model storage step for storing the initial failure occurrence model generated by the initial model generation step, and a set of events generated in the monitored device A learning event sequence storage step for storing a learning event sequence, and the initial model storage And the initial failure model stored in step, the based on the stored the learning event string in the learning event string storage step, the stored in the learning event string storage step of the learning event string A transition probability learning step that learns at what probability the state of the finite automaton that represents the transition between occurrence causes has changed, and a failure occurrence model that generates a failure occurrence model that reflects the transition probability learned by the transition probability learning step A failure cause storage step for storing the failure occurrence model generated by the failure occurrence model generation step, and a failure cause for storing a failure cause discovery event sequence which is a set of events generated in the monitored device The event sequence storage step for discovery and the failure stored in the failure occurrence model storage step A state transition sequence generating step for obtaining a state transition sequence that is considered to have the highest probability that the failure cause finding event sequence stored in the failure cause finding event sequence storing step is observed according to a raw model, and the state transition and a state transition sequence storing step of storing the state transition sequence of sequence generation means is generated, the state transition has been the state transition sequence stored in the string storage step, the failure model stored in the fault model storage step on the basis of the transition probabilities is divided, the head of state of each state transition sequence in which the divided, characterized in that it comprises a filtering step of estimating the fundamental cause of the failure that occurred in the monitoring target device, the.

The program of the present invention is a program for causing a computer to execute a method for estimating a root cause of a failure that has occurred in a monitoring target device, and for causing an event caused by a failure that may occur in the monitoring target device to the computer. And an input step for inputting a basic model definition that defines the cause of failure that causes the event, and a correspondence relationship between the event and the cause of the occurrence and between the causes based on the basic model definition input by the input step An initial model generation step for generating an initial failure generation model in which transitions are modeled by a finite automaton, an initial model storage step for storing the initial failure generation model generated by the initial model generation step, and an occurrence in the monitoring target device A learning event sequence storage for storing a learning event sequence that is a set of events Stored in the learning event sequence storage step based on the initial failure model stored in the initial model storage step and the learning event sequence stored in the learning event sequence storage step. A transition probability learning step for learning at which probability the state of the finite automaton representing the transition between the causes of occurrence of the learning event sequence has changed, and a failure occurrence reflecting the transition probability learned by the transition probability learning step A fault occurrence model generation step for generating a model, a fault occurrence model storage step for storing the fault occurrence model generated by the fault occurrence model generation step, and a fault cause discovery that is a set of events that have occurred in the monitored device An event sequence storage step for fault cause discovery storing an event sequence, and the failure occurrence model description In accordance with the failure occurrence model stored in the step, a state transition sequence for obtaining a state transition sequence that is considered to have the highest probability that the failure cause finding event sequence stored in the failure cause finding event sequence is observed. a generation step, a state transition sequence storing step of storing the state transition sequence of the state transition sequence generating means has generated, the state transition sequence stored in the state transition sequence storing step, to the failure model storage step A filtering step of dividing based on the transition probability of the stored failure occurrence model , and estimating a leading state of the divided state transition sequence as a root cause of a failure that has occurred in the monitoring target device; It is made to perform.

  In the failure cause estimation system, method, and program of the present invention, the initial failure occurrence model created based on the basic definition model is learned using the learning event sequence, and from the failure occurrence model obtained by learning, A state transition sequence that is considered to have the highest probability of observing a failure cause finding event sequence is obtained, and based on the state transition sequence, a root cause of a failure that has occurred in the monitoring target device is estimated. In the present invention, what needs to be defined manually is the event that can occur, the cause of the failure that causes the event, and the correspondence between them, which can be easily defined manually. Therefore, it is possible to estimate the root cause of a failure without defining a complicated cause and dependency between results.

In the failure cause estimation system, method, and program of the present invention, the state transition probability between the causes of occurrence of the learning event sequence stored in the learning event sequence storage means and each cause by the Baum-Welch algorithm. It is possible to adopt a configuration for learning the event occurrence probability. As a learning algorithm, a Baum-Welch algorithm that estimates a parameter from an output symbol string can be used.

  The fault cause estimation system, method, and program of the present invention can employ a configuration for obtaining the state transition sequence by the Viterbi algorithm. A Viterbi algorithm that estimates a state string from an output symbol string can be used as an algorithm used when obtaining a string of states from a failure occurrence model after learning.

Failure cause estimation system of the present invention, a method, and the program, the initial model the initial failure model stored in the storage means, before a set Σ of hearing vent, the normal state to the set S of the failure cause s A set of states plus ₀ , a conditional probability Pr (sj | si) _{si, sjεS} indicating the probability of transition from each state to each state, and each state is in that state at the start A configuration including an initial probability {P ⁰ _si } _siεS and Pr (ej | si ) _{s iεS, ejεΣ} indicating the probability of occurrence of the event in each state can be adopted.

In the failure cause estimation system, method, and program of the present invention, the filtering module uses the state transition sequence stored in the state transition sequence storage means as {s (0), s (1),. s (n)}, the conditional probability Pr (s (i + 1) | s (i)) from the state s (i) (0 ≦ i <n) to the next state s (i + 1) is a predetermined value. below the probability, the previous SL state transition sequence stored in the state transition sequence storage unit, {s (0), s (1), ···, s (i)} and, {s (i + 1) ,..., S (n)} can be adopted. In this case, the filtering module can estimate the leading state of each of the divided state transition sequences as the root cause of the failure. By dividing in this way, when two state sequences (sequences) due to different root causes happen to be regarded as one sequence because they happen to be probabilistically maximum values, the sequences are separated for each root cause. It can be divided into sequences, and the leading state of each sequence can be estimated as the root cause.

In the failure cause estimation system of the present invention, a configuration can be adopted in which the learning event sequence stored in the learning event sequence storage means is an event sequence monitored when the monitoring target device is trial run. Alternatively, the learning event sequence stored in the learning event sequence storage means is an event sequence monitored during operation of the monitoring target device, and the cause of the failure is an analyzed event sequence. Can be adopted.

In the failure cause estimation system of the present invention, a configuration can be adopted in which the failure cause discovery event sequence stored in the failure cause discovery event sequence storage means is an event sequence monitored during operation of the monitoring target device. In this case, it is possible to estimate the root cause of the failure that has occurred in the monitoring target device in operation.

In the failure cause estimation system of the present invention, the learning event sequence stored in the learning event sequence storage means and the failure cause discovery event sequence stored in the failure cause discovery event sequence storage means are adjacent to each other. It is possible to adopt a configuration in which the occurrence time interval between two events is equal to or less than a predetermined value. In this case, for a certain cause of failure, an event sequence for learning events and an event sequence for finding failure causes can be used.

Failure cause estimation system of the present invention, a method, and, in the program, means for memorize the state transition sequence stored in the state transition sequence storage means, stored in said failure cause found for the event sequence storage means of failure cause found for the event column, and means for the event that the cause each state, in association with each state included in the state transition sequence storage means and said state transition sequence stored in Configuration can be adopted. By referring to the cause estimation result database stored in this way, the administrator or the like can analyze the root cause of the failure, the cause that accompanies it, and the like.

  In the failure cause estimation system, method, and program of the present invention, the initial failure occurrence model created based on the basic definition model is learned using the learning event sequence, and from the failure occurrence model obtained by learning, A state transition sequence that is considered to have the highest probability of observing a failure cause finding event sequence is obtained, and based on the state transition sequence, a root cause of a failure that has occurred in the monitoring target device is estimated. In the present invention, what needs to be defined manually is the event that can occur, the cause of the failure that causes the event, and the correspondence between them, which can be easily defined manually. Therefore, it is possible to estimate the root cause of a failure without defining a complicated cause and dependency between results.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows the configuration of a failure cause estimation apparatus according to an embodiment of the present invention. The failure cause estimation apparatus 10 includes an initial model parser 30, an initial model generation means 40, a Baum-Welch calculation means 50, a Viterbi calculation means 60, a filtering module 70, a model storage database 120, an event string parser 130, an event string database 140, a cause An estimation result database 150 is provided. The failure cause estimation device 10 is configured by a computer system such as a workstation, for example, and functions of each unit are realized by operating a predetermined program.

  The initial model parser 30 reads the basic model definition 20 described by the device developer, parses it, generates syntax information, and passes it to the initial model generation means 40. The basic model definition 20 defines an event that can occur in the monitoring target device 80 and a failure cause that causes the event. Based on the syntax information generated by the initial model parser 30, the initial model generation means 40 generates an initial failure occurrence model in which the correspondence relationship between events and their occurrence causes, and transitions between occurrence causes are modeled by a finite automaton. And stored in the model storage database 120.

  The event monitor 90 monitors events that occur in the monitoring target device 80 and generates a learning event sequence 100 and a failure cause discovery event sequence 110. The event sequence parser 130 parses the learning event sequence 100 and the failure cause discovery event sequence 110 to generate event data and store it in the event sequence database 140. The Baum-Welch calculating means 50 includes an initial failure occurrence model stored in the model storage database 120 and event data corresponding to the learning event sequence 100 stored in the event sequence database 140 (hereinafter, simply referred to as the learning event sequence 100). And the probability that the state of the finite automaton corresponding to the cause has changed, and the failure occurrence model reflecting the learning result is stored in the model storage database 120.

  The Viterbi calculation means 60 is a failure occurrence model stored in the model storage database 120 and event data corresponding to the failure cause discovery event sequence 110 stored in the event sequence database 140 (hereinafter also simply referred to as a failure discovery event sequence). ) To obtain the state transition sequence of the failure occurrence model having the highest probability of occurrence and output it to the filtering module 70. At this time, the Viterbi calculation means 60 also outputs the failure cause discovery event sequence 110 that is the source for obtaining the state transition sequence.

  The filtering module 70 truncates a probabilistic low sequence from the state transition sequence (failure cause sequence) estimated by the Viterbi calculation means 60 and estimated to have occurred most, The starting state of the transition sequence is estimated as the root cause. The filtering module 70 stores in the cause estimation result database 150 a cause string that includes the estimated root cause and the derived causes that are related thereto. At that time, in the failure cause finding event sequence 110, the event having the root cause and the derived cause as the cause of occurrence is stored in the cause estimation result database 150 in association with each cause. The administrator refers to the cause estimation result database 150 to analyze the cause of the failure.

  FIG. 2 shows an operation procedure of the failure cause estimation device 10 when generating a failure occurrence model. The initial model parser 30 reads the basic model definition 20 described by the device developer, and converts the read basic model definition 20 into syntax information that can be interpreted by the initial model generation means 40 (step A1). The basic model definition 20 is defined by an event set Σ, a failure cause set S, and a function f: Σ → S. Basically, the function f is a global function, but may be a partial function.

  FIG. 3 shows a description example of the basic model definition 20. The basic model definition 20 is described in a text file as shown in FIG. In this example, a paragraph beginning with [states] assuming Windows (registered trademark) as the OS defines a set S of failure causes. In the example of FIG. 9, nine causes such as “Print” and “Application Popup” are defined. The paragraph beginning with [obsservations] defines a set of events Σ. In the event monitor tool “event viewer” of Windows (registered trademark), the event type is given by a numerical ID such as “3”, “4”, “16”, etc. Use that ID.

  In [Observations], the part delimited by the event type (ID) and “,” represents a possible cause for the event. This part defines the function (mapping) f: Σ → S from the fault to the cause. For example, the event “3” is caused by the state related to “Print”, and is defined as f (“3”) = “Print”. The initial model parser 30 reads such a text file and passes the syntax information corresponding to the described basic model definition 20 to the initial model generation means 40.

Returning to FIG. 2, the initial model generation means 40 generates an initial failure occurrence model based on the syntax information corresponding to the basic model definition 20 (step A2). The failure occurrence model M is defined by the following equation.
M = {Σ, S∪ {s ₀ }, {Pr (s _j | s _i )} _{si, sj∈S} , {P ⁰ _i } _si∈S , {Pr (e _j | s _i )} _{si∈S , Ej∈Σ} }
Pr (a | b) is a conditional probability and indicates the probability of occurrence of a under the condition of b. P ⁰ _i indicates the probability that the failure occurrence model M starts from the state S _i . s ₀ indicates a state in which the monitoring target device 80 is normal. In the failure occurrence model M, “S∪ {s ₀ }, {Pr (s _j | s _i )} _{si, sj∈S} , {P ⁰ _i } _{si, sj∈S} ” represents a finite state automaton, This indicates that the next state s∈S∪ {s ₀ } is determined only by the immediately preceding state S′∈S∪ {s ₀ } and transitions with a fixed probability Pr (s | s ′). .

Will be described in detail early failure model M ₀ initial model generating unit 40 generates. The set of events Σ handled in the initial failure occurrence model M ₀ is the same as Σ defined in the basic model definition 20. The cause set S 集合 {s ₀ } handled in the initial failure occurrence model M ₀ is a set obtained by adding the normal state s ₀ to S defined in the basic model definition 20. {Pr (s _j | s _i )} _{si and sjεS} indicate transition probabilities between causes, and the transition probabilities are equal. Specifically, Pr (s _j | s _i ) = 1 / (| S | +1), where | S | The probability of becoming a steady state may be set high by increasing only the probability Pr (s _i | s _i ) of transitioning to itself without making this probability equal. {P ⁰ _i } _siεS is set to P ⁰ ₀ = 1 and P ⁰ _i = 0 (i ≠ 0). This means that the initial failure occurrence model M ₀ starts from the normal state s ₀ .

{Pr (e _j | s _i )} _{siεS, ejεΣ} indicates the correspondence between the event and the cause, and indicates the probability that the event e _j will occur in the state S _i . {Pr (e _j | s _i )} _{si∈S, ej∈Σ} is
Pr (e | s) = k × p (when f (e) = s)
Pr (e | s) = p (when f (e) ≠ s)
It is defined as However, k is a constant of 1 or more. In addition, for all sεS∪ {s ₀ }, Σ _{{e | f (e) = s}} k × p + Σ _{{e | f (e) ≠ s}} p ≦ 1. In the above definition, f (e) = s defined in the basic model definition 20, that is, for the cause s of the event e, the probability of occurrence of e from s is the probability of probability p if f (e) ≠ s. It indicates that the setting is k times. When the function f is a partial function and f is not defined, f (e | s) is given a probability p according to the above definition.

  The administrator gives the event sequence for learning 100 to the failure cause estimating apparatus 10 online or offline (step A3). For example, the administrator gives the event sequence monitored by the event monitor 90 when the monitoring target device 80 is run as a learning event sequence 100 offline. Alternatively, among the event strings monitored by the event monitor 90 during the operation of the monitoring target device 80, an event string for which the cause of failure analysis has already been performed is given as a learning event string 100 online.

The event sequence parser 130 generates event data that can be interpreted by other modules from the given learning event sequence 100 and stores the event data in the event sequence database 140. When generating event data, the event string parser 130 divides the learning event string 100 into a plurality of event strings under a predetermined condition. FIG. 4 shows how the event sequence is divided. The event string parser 130 divides the event string when the event occurrence interval is larger than a predetermined threshold T. Specifically, for example, when there is an event sequence (e (0),..., E (n)), the time between the occurrence time of the event e (i) and the occurrence time of e (i + 1) is shortened. If it is longer than the threshold value T, the event sequence (e (0),..., E (n)), the event region R ₀ : (e (0),..., E (i)) and the event region R ₁ : (E (i + 1),..., E (n)). This means that the event occurrence interval is equal to or less than the threshold value T within the event region, and the event occurrence interval between the event regions is larger than the threshold value T. The event e (i) here is not the event type (event type) but the event itself.

Returning to FIG. 2 again, the Baum-Welch calculation means 50 generates an initial failure generated by the initial model generation means 40 using the event region of the learning event sequence 100 input from the event sequence parser 130 by the Baum-Welch algorithm. It performs learning of the model _{M 0.} The Baum-Welch calculation means 50 has a model M = {Σ, S∪ {s ₀ }, {Pr (s _j | s _i ) that has the highest probability for a given event sequence (event region) to be learned. )} _{si, sj∈S} , {P ⁰ _i } _{si, sj∈S} , {Pr (e _{j │s i} )} _si ∈ _{S, ej∈Σ} } transition probability {Pr (s _{j │s i} )} _{Si, sjεS} and event occurrence probability {Pr (e _j | s _i )} _{siεS, ejεΣ} are obtained. However, it is a maximum likelihood method for obtaining a local solution from the initial model M ₀ instead of obtaining an optimum value. The Baum-Welch algorithm is a well-known algorithm as described in, for example, “Statistical Methods for Speech Recognition (Language, Speech, and Communication)” (Frederick Jelinek), Section 9.3. Is omitted.

The Baum-Welch calculation means 50 determines the transition probability {Pr (s _j | s _i )} _{si, sj∈S} and the event occurrence probability {Pr (e _j | s _i )} _si∈S of the initial failure occurrence model M _{0. , Ε∈Σ} is replaced with the transition probability and event occurrence probability obtained by learning, respectively, and a failure occurrence model M ′ is generated and stored in the model storage database 120 (step A4). Up to this point is the failure occurrence model generation phase. Thereafter, the root cause of the fault is estimated using the fault occurrence model M ′ thus obtained.

  FIG. 5 shows an operation procedure of the failure cause estimating apparatus 10 when estimating the cause of the failure. The administrator provides the event sequence observed by the event monitor 90 on the monitoring target device 80 online to the failure cause estimation device 10 as the failure cause discovery event sequence 110 (step B1). The event sequence parser 130 divides the given failure cause discovery event sequence 110 into a plurality of event regions (FIG. 4), and delivers them to the Viterbi calculation means 60 via the event sequence database 140.

  The Viterbi calculation means 60 uses the Viterbi algorithm to input the failure cause discovery event 110 (event region) for the failure occurrence model M ′ learned in the procedure shown in FIG. 2 and stored in the model storage database 120. The sequence (s (0), s (1), s (2),..., S (n)) of the cause having the highest possibility (likelihood) of realizing is obtained (step B2). S (i) in the cause sequence column does not indicate the type of cause, but indicates the state transition sequence of the cause in time order, and the numbers in parentheses are assigned in time order. The Viterbi algorithm is a generally well-known algorithm as described in, for example, “Statistical Methods for Speech Recognition (Language, Speech, and Communication)” (by Frederick Jelinek), Chapter 5). Then, the detailed description is abbreviate | omitted.

The filtering module 70 converts the cause sequence (s (0), s (1), s (2), ..., s (n)) obtained by the Viterbi calculation means 60 into two adjacent sequences. Based on the state transition probability Pr (s _{i + 1} | s _i ) between states, it is divided into a plurality of groups (step B3). For example, if Pr (s (q + 1) | s (q)) <L, (s (0), s (1)..., S (q)) and (s (q + 1),. s (n)). The probability L for determining the division is a threshold value between 0 and 1, and is a relatively small probability value. The reason for dividing in this way is that when there are two sequences due to different root causes, it may happen to be regarded as one sequence because it happens to be the maximum value. Therefore, the filtering module 70 divides a sequence having a transition probability lower than the threshold value L by considering that it is not a sequence stochastically, but happens to overlap in time series.

The filtering module 70 stores the divided sequences (cause order sequence) in the cause estimation result database 150. Also, the head of the divided sequence is estimated as the root cause. When storing the cause sequence in the cause estimation result database 150, the filtering module 70 stores the events corresponding to the causes in the cause estimation result database 150 in association with the causes. For example, when the root cause is S _i , the basic model definition is included in the failure cause discovery event sequence 110 (e (0), e (1),..., E (n)). The event associated with the failure cause S _i at 20 is stored in the cause estimation result database 150 in association with the failure cause S _i .

Hereinafter, a specific example will be described. As the basic model definition 20, the one shown in FIG. 3 is considered. A set Σ of events handled by the initial failure occurrence model M ₀ generated by the initial model generation means 40 is
Σ = {3, 4, 16, 17, 18, 19, 20}
=> {E ₀ , e ₁ , e ₂ , e ₃ , e ₄ , e ₅ , e ₆ }
It is. The set of states is
S∪ {s ₀ } = {s ₀ , “Print”, “Windows Update Agent”, “W32Time”, “Application Popup”, “i8042prt”, “Windows Installer”, “DHCP”, “Browser”, “Tcpip” }
=> {S ₀ , s ₁ , s ₂ , s ₃ , s ₄ , s ₅ , s ₆ , s ₇ , s ₈ , s ₉ , s ₁₀ }
It is. Since the transition probability between causes {Pr (s _j | s _i )} _{si, sj∈S} has a total of 10 states,
Pr (s _j | s _i ) = 1/10
It is. The initial probabilities are P ⁰ ₀ = 1 and P ⁰ _i = 0 (i ≠ 0). In FIG. 3, it is assumed that there are seven types of events. In this case, the event occurrence probability is
Pr (e | s) = 2/8 (when f (e) = s)
Pr (e | s) = 1/8 (when f (e) ≠ s)
It is.

Let the learning event sequence 100 be {e (0),..., E (n)}. This e (i) is not an event type but an event itself, and an occurrence time is recorded. In this event sequence, the difference between the occurrence times of e (i) and e (i + 1) is 2 seconds, and the difference between the occurrence times of other events is 1 second or less. Assuming that the threshold T when the event sequence parser 130 divides the event sequence into a plurality of regions is 1 second, the learning event sequence 100 has R ₁ = {e (0),..., E (i) } And R ₂ = {e (i + 1),..., E (n)}. The Baum-Welch calculation means 50 is provided with many event regions including R ₁ and R ₂ , so that the transition probability Pr (s _j | s _i ) and the event occurrence probability Pr (e) are obtained from the initial failure occurrence model M _0. With respect to | s), the probability of most occurrence of a given event region is learned, and a failure occurrence model M ′ is obtained.

  The Viterbi calculation means 60 obtains a sequence of causes that is most likely to realize the event region of the failure cause discovery event sequence 110 for the learned failure occurrence model M ′. The filtering module 70 divides the sequence of causes obtained by the Viterbi calculation means 60 based on the state transition probability between causes of the failure occurrence model M ′, and the divided sequence sequences in the cause estimation result database 150. Store. At that time, the event corresponding to each cause in the failure cause discovery event sequence 110 is stored in the cause estimation result database 150 in association with the cause of the failure. The administrator estimates the root cause by referring to the information stored in the cause estimation result database 150.

  FIG. 6 shows a specific example of information stored in the cause estimation result database 150. In the figure, the part indicated by state corresponds to the cause. In addition, {} in the state indicates an event corresponding to the cause. In this example, the cause sequence is transitioning from bottom to top, and it is estimated that Tcpip is the root cause. The administrator refers to the information stored in the cause estimation result database 150, and the TCP / IP protocol stack error is causing the cause of failure such as “Browser”, “Dhcp”, “Windows Installer 3.1”, etc. Can know.

  In this embodiment, a failure occurrence model is generated by giving a correspondence relationship between an event and its cause, an event sequence observed by the monitoring target device 80 is given to the failure occurrence model, and the cause is determined from the transition of the event sequence. Find the sequence of. By dividing the sequence of cause obtained in this way based on the transition probability between causes, it is possible to estimate the root cause of the failure that is the source of the cause transition. In the present embodiment, the relationship between the cause of failure is obtained by giving the learning event sequence 100 to the initial failure occurrence model, and it is not necessary to manually define the dependency between causes. In this embodiment, when generating an initial failure occurrence model, it is only necessary to define an event and its cause, and the relationship between the event and its cause is relatively easy to describe. Can be estimated.

  As described above, the present invention has been described based on the preferred embodiment. However, the failure cause estimation system, method, and program of the present invention are not limited to the above-described embodiment example, and the configuration of the above-described embodiment. To which various modifications and changes are made within the scope of the present invention.

  The present invention can be applied to the use of a fault monitoring system for a network or a computer system. It can also be applied to fault detection systems in embedded systems.

The block diagram which shows the structure of the failure cause estimation apparatus of one Embodiment of this invention. The flowchart which shows the operation | movement procedure of the failure cause estimation apparatus at the time of producing | generating a failure generation model. The figure which shows the example of a description of a basic model definition. The schematic diagram which shows the mode of a division | segmentation of an event sequence. The flowchart which shows the operation | movement procedure of the failure cause estimation apparatus at the time of estimating a failure cause. The figure which shows the specific example of the information stored in the cause estimation result database. The block diagram which shows the structure of the failure cause discovery apparatus described in patent document 1. FIG. The block diagram which shows the structure of the causal system model construction apparatus used for the failure cause estimation system described in patent document 2. FIG. The block diagram which shows the structure of the cause estimation apparatus used for the failure cause estimation system described in patent document 2. FIG. The model figure which models and shows a causal relationship.

Explanation of symbols

10: Failure cause estimation device 20: Basic model definition 30: Initial model parser 40: Initial model generation means 50: Baum-Welch calculation means 60: Viterbi calculation means 70: Filtering module 80: Monitoring target device 90: Event monitor 100: Learning Event sequence 110: failure cause discovery event sequence 120: model storage database 130: event sequence parser 140: event sequence database 150: cause estimation result database

Claims (22)

An input means for inputting a basic model definition that defines an event caused by a failure that may occur in the monitoring target device and a cause of the failure that causes the event;
Based on the basic model definition input by the input means, an initial model generation means for generating an initial failure occurrence model in which a correspondence between an event and its occurrence cause and a transition between the occurrence causes are modeled by a finite automaton;
Initial model storage means for storing the initial failure occurrence model generated by the initial model generation means;
A learning event string storage unit that stores a learning event string that is a set of events that have occurred in the monitoring target device;
The learning stored in the learning event sequence storage unit based on the initial failure occurrence model stored in the initial model storage unit and the learning event sequence stored in the learning event sequence storage unit A transition probability learning means for learning at what probability the state of the finite automaton representing the transition between the occurrence causes of the event sequence for
A fault occurrence model generating means for generating a fault occurrence model reflecting the transition probability learned by the transition probability learning means;
A failure occurrence model storage means for storing the failure occurrence model generated by the failure occurrence model generation means;
A failure cause discovery event sequence storage means for storing a failure cause discovery event sequence that is a set of events that have occurred in the monitored device;
According to the failure occurrence model stored in the failure occurrence model storage unit, a state transition sequence that is considered to have the highest probability that the failure cause discovery event sequence stored in the failure cause discovery event sequence is observed. State transition sequence generation means for obtaining
State transition sequence storage means for storing the state transition sequence generated by the state transition sequence generation means;
Has been the state transition sequence stored in the state transition sequence storage means, the divided based on the transition probability of failure the failure model stored in the model storage unit, the head of each state transition sequence in which the divided A filtering module that estimates the state as the root cause of the failure that occurred in the monitored device;
A failure cause estimation system comprising: The transition probability learning means learns a state transition probability between occurrence causes of the learning event string stored in the learning event string storage means and an event occurrence probability at each cause by a Baum-Welch algorithm. The failure cause estimation system according to claim 1. The failure cause estimation system according to claim 1, wherein the state transition sequence generation unit obtains the state transition sequence by a Viterbi algorithm. The initial failure model stored in the initial model storage means, a set Σ before hearing vent, a set of state plus normal state s ₀ to the set S of the failure cause, for each state, the state Conditional probability Pr (sj | si) _{si, sjεS} indicating the probability of transition from each state to each state, and the initial probability {P ⁰ _si } _siεS at each state at the start for each state, and for each state The failure cause estimation system according to claim 1, further comprising: Pr (ej | si ) _{s iεS, ejεΣ} indicating the probability that the event will occur in the state. The filtering module, the state transition sequence storage means stored said state transition sequence of {s (0), s ( 1), ···, s (n)} when the state s (i) ( If the conditional probability Pr (s (i + 1) | s (i)) from 0 ≦ i <n) to the next state s (i + 1) is lower than a predetermined probability, it is stored in the state transition sequence storage means. and the pre-Symbol state transition sequence is divided into a {s (0), s ( 1), ···, s (i)} and, {s (i + 1) , ···, s (n)}, The failure cause estimation system according to claim 4. The failure cause estimation system according to claim 1, wherein the learning event sequence stored in the learning event sequence storage unit is an event sequence monitored when the monitoring target device is trial run. 2. The learning event sequence stored in the learning event sequence storage means is an event sequence monitored during operation of the monitoring target device, and the cause of the failure is an analyzed event sequence. Failure cause estimation system described in 1. The failure cause estimation system according to claim 1, wherein the failure cause discovery event sequence stored in the failure cause discovery event sequence storage means is an event sequence monitored during operation of the monitoring target device. An occurrence time interval between two adjacent events in each of the learning event sequence stored in the learning event sequence storage unit and the failure cause finding event sequence stored in the failure cause finding event sequence storage unit is The failure cause estimation system according to claim 1, wherein the failure cause estimation system is equal to or less than a predetermined value. The filtering module includes means for memorize the state transition sequence stored in the state transition sequence storage means,
Among the failure cause discovery event sequences stored in the failure cause discovery event sequence storage means, an event that causes each state included in the state transition sequence stored in the state transition sequence storage means to occur The failure cause estimation system according to claim 1, further comprising: means for storing the information in association with each state. A method for estimating the root cause of a failure that has occurred in a monitored device using a computer,
An input step of inputting a basic model definition that defines an event caused by a failure that can occur in the monitoring target device and a failure cause that causes the event;
Based on the basic model definition input in the input step, an initial model generation step for generating an initial failure occurrence model in which a correspondence between an event and its occurrence cause and a transition between the occurrence causes are modeled by a finite automaton;
An initial model storage step for storing the initial failure occurrence model generated by the initial model generation step;
A learning event sequence storage step for storing a learning event sequence that is a set of events that have occurred in the monitored device;
The learning stored in the learning event sequence storage step based on the initial failure occurrence model stored in the initial model storage step and the learning event sequence stored in the learning event sequence storage step A transition probability learning step for learning at which probability the state of the finite automaton representing the transition between the occurrence causes of the event sequence has changed,
A failure occurrence model generation step of generating a failure occurrence model reflecting the transition probability learned by the transition probability learning step;
A failure occurrence model storage step for storing the failure occurrence model generated by the failure occurrence model generation step;
A failure cause discovery event sequence storage step for storing a failure cause discovery event sequence that is a set of events that have occurred in the monitored device;
According to the failure occurrence model stored in the failure occurrence model storage step, the state transition sequence considered to have the highest probability that the failure cause discovery event sequence stored in the failure cause discovery event sequence is observed. A state transition sequence generation step for obtaining
A state transition sequence storage step for storing the state transition sequence generated by the state transition sequence generation means;
The state transition sequence stored in the state transition sequence storing step, the divided based on the transition probability of the failure model stored in the failure model storage step, the head of each state transition sequence in which the divided A filtering step that estimates the state as the root cause of the failure that occurred in the monitored device;
A failure cause estimation method comprising: The transition probability learning step learns the state transition probability between the causes of occurrence of the learning event sequence stored in the learning event sequence storage means and the event occurrence probability at each cause by the B aum-Welch algorithm. to, failure cause estimation method of claim 1 1. The state transition sequence generating step, the V Iterbi algorithm, determining the state transition sequence, failure cause estimation method of claim 1 1. The initial failure model stored in the initial model storage step, a set Σ before hearing vent, a set of state plus normal state s ₀ to the set S of the failure cause, for each state, the state Conditional probability Pr (sj | si) _{si, sjεS} indicating the probability of transition from each state to each state, and the initial probability {P ⁰ _si } _siεS at each state at the start for each state, and for each state , Pr indicating the probability of the event in the condition occurs (ej | _{si) s i∈S,} and a _{Ej∈shiguma,} failure cause estimation method of claim 1 1. The filtering step includes
Before SL state transition sequence storing step to said stored state transition sequence of {s (0), s ( 1), ···, s (n)} when the state s (i) (0 ≦ i < (the conditional probability Pr (s (i + 1 to i + 1)) from n) next state s | s (i)) is a predetermined lower than the probability, before Symbol state stored in the state transition sequence storing step the transition sequence, you divide the a {s (0), s ( 1), ···, s (i)} and, {s (i + 1) , ···, s (n)}, according to claim 1 4. The method for estimating a cause of failure according to 4 . It said filtering step includes the steps of memorize the state transition sequence stored in the state transition sequence storing step,
Among the failure cause discovery event sequences stored in the failure cause discovery event sequence storage step, an event having each state included in the state transition sequence stored in the state transition sequence storage step as a cause of occurrence , further comprising the steps of storing in association with each state, a failure cause estimation method of claim 1 1. A program for causing a computer to execute a method for estimating a root cause of a failure that has occurred in a monitored device, the computer comprising:
An input step of inputting a basic model definition that defines an event caused by a failure that can occur in the monitoring target device and a failure cause that causes the event;
Based on the basic model definition input in the input step, an initial model generation step for generating an initial failure occurrence model in which a correspondence between an event and its occurrence cause and a transition between the occurrence causes are modeled by a finite automaton;
An initial model storage step for storing the initial failure occurrence model generated by the initial model generation step;
A learning event sequence storage step for storing a learning event sequence that is a set of events that have occurred in the monitored device;
The learning stored in the learning event sequence storage step based on the initial failure occurrence model stored in the initial model storage step and the learning event sequence stored in the learning event sequence storage step A transition probability learning step for learning at which probability the state of the finite automaton representing the transition between the occurrence causes of the event sequence has changed,
A failure occurrence model generation step of generating a failure occurrence model reflecting the transition probability learned by the transition probability learning step;
A failure occurrence model storage step for storing the failure occurrence model generated by the failure occurrence model generation step;
A failure cause discovery event sequence storage step for storing a failure cause discovery event sequence that is a set of events that have occurred in the monitored device;
According to the failure occurrence model stored in the failure occurrence model storage step, the state transition sequence considered to have the highest probability that the failure cause discovery event sequence stored in the failure cause discovery event sequence is observed. A state transition sequence generation step for obtaining
A state transition sequence storage step for storing the state transition sequence generated by the state transition sequence generation means;
The state transition sequence stored in the state transition sequence storing step, the divided based on the transition probability of the failure model stored in the failure model storage step, the head of each state transition sequence in which the divided A filtering step that estimates the state as the root cause of the failure that occurred in the monitored device;
A failure cause estimation program characterized in that The transition probability learning step, the Baum-Welch algorithm learns the state transition probability between the cause of the learning event string stored in the learning event string storage means, and event probability of each cause The failure cause estimation program according to claim 17 . The state transition sequence generating step, the Viterbi algorithm determines the state transition sequence, failure cause estimation program according to claim 1 7. The initial failure model stored in the initial model storage step, a set Σ before hearing vent, a set of state plus normal state s ₀ to the set S of the failure cause, for each state, the state Conditional probability Pr (sj | si) _{si, sjεS} indicating the probability of transition from each state to each state, and the initial probability {P ⁰ _si } _siεS at each state at the start for each state, and for each state , Pr indicating the probability of the event in the condition occurs (ej | _{si) s i∈S,} and a _{Ej∈shiguma,} failure cause estimation program according to claim 1 7. The filtering step includes
The state transition sequence storing step wherein state transition sequence stored in the {s (0), s ( 1), ···, s (n)} when the state s (i) (0 ≦ i <n ) from the next state s (i + 1) probabilities above conditions to Pr (s (i + 1) | s (i)) is a predetermined lower than the probability, pre SL state transitions stored in said state transition sequence storing step column, you divide the a {s (0), s ( 1), ···, s (i)} and, {s (i + 1) , ···, s (n)}, according to claim 2 0 Failure cause estimation program described in 1. It said filtering step includes the steps of memorize the state transition sequence stored in the state transition sequence storing step,
Among the failure cause discovery event sequences stored in the failure cause discovery event sequence storage step, an event having each state included in the state transition sequence stored in the state transition sequence storage step as a cause of occurrence , and storing in association with each state, further to execute, failure cause estimation program according to claim 1 7.

Images