WO2023040381A1 - 告警因果关系挖掘方法、告警因果挖掘装置及存储介质 - Google Patents

告警因果关系挖掘方法、告警因果挖掘装置及存储介质 Download PDF

Info

Publication number
WO2023040381A1
WO2023040381A1 PCT/CN2022/098772 CN2022098772W WO2023040381A1 WO 2023040381 A1 WO2023040381 A1 WO 2023040381A1 CN 2022098772 W CN2022098772 W CN 2022098772W WO 2023040381 A1 WO2023040381 A1 WO 2023040381A1
Authority
WO
WIPO (PCT)
Prior art keywords
alarm
node
environment
system alarm
mining
Prior art date
Application number
PCT/CN2022/098772
Other languages
English (en)
French (fr)
Inventor
弄庆鹏
李忠良
周祥生
高洪
屠要峰
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2023040381A1 publication Critical patent/WO2023040381A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/26—Visual data mining; Browsing structured data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures

Definitions

  • the embodiments of the present application relate to the technical field of the Internet, and in particular to a method for mining an alarm causality, an alarm causality mining device, and a storage medium.
  • Causality mining algorithms are usually implemented by rule-based statistical methods. For example, mining is based on the co-occurrence frequency of variable pairs or the timing of variable fluctuations, and then uses variables to represent causal coefficients.
  • causal mining has theoretical flaws. Specifically, if causality is defined by rules, if the established rules cannot correctly represent the causal relationship, the causal relationship mined is invalid. However, based on co-occurrence frequency and timing, only the correlation between variables can be mined, but the causality cannot be represented.
  • the current emerging causal reasoning methods require expert experience to provide variable causal hypotheses for verification mining or random causal hypothesis graphs for verification mining. There are problems such as difficulty in obtaining variable causal hypotheses and too large hypothesis space.
  • the embodiment of the present application provides a method for mining causality of alarms, including: building a system alarm environment based on deep reinforcement learning based on system alarm information and root cause label data of system alarm information; intelligent agents and system alarm environment through deep reinforcement learning Interact, learn and generate alert causality models that characterize alert causality and structure.
  • An embodiment of the present application provides an alarm causality mining device, including: an agent module based on deep reinforcement learning and a system alarm environment module; the system alarm environment module builds a deep reinforcement system based on system alarm information and root cause label data of the system alarm information. Learned system alarm environment; the agent module interacts with the system alarm environment module to learn and generate an alarm causal model representing the causal relationship and structure of the alarm.
  • An embodiment of the present application provides an alarm causality mining device, including: at least one processor; and a memory connected in communication with the at least one processor; wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by at least one processor. Executed by one processor, so that at least one processor can execute the alarm causality mining method mentioned in the above embodiment.
  • An embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the alarm causality mining method mentioned in the above-mentioned embodiment is implemented.
  • Fig. 1 is the flowchart of the alarm causality mining method in the embodiment of the present application
  • FIG. 2 is a system alarm diagram generated based on Table 1 and Table 2 in the embodiment of the present application;
  • Fig. 3 is a schematic structural diagram of an alarm causal excavation device in an embodiment of the present application.
  • FIG. 4 is another system alarm diagram in the embodiment of the present application.
  • Fig. 5 is a schematic diagram of the state space of the system alarm environment in the embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of the A2C model in the embodiment of the present application.
  • FIG. 7 is a schematic diagram of the interaction between the alarm causality mining device 20 and the alarm causality mining target system 30 in the embodiment of the present application;
  • FIG. 8 is a schematic diagram of the process of obtaining an alarm causal model by the alarm causality mining device 20 in the embodiment of the present application.
  • FIG. 9 is a schematic diagram of the interaction between the alarm causality mining device 20 and the alarm causality mining target system 30 in another embodiment of the present application.
  • FIG. 10 is a second schematic diagram of the process of obtaining the alarm causality model by the alarm causality mining device 20 in the embodiment of the present application;
  • Fig. 11 is a schematic structural diagram of an alarm causal mining device in another embodiment of the present application.
  • Fig. 12 is a schematic structural diagram of an alarm causal excavation device in another embodiment of the present application.
  • the purpose of the embodiments of this application is to provide a method for mining alarm causality, an alarm causality mining device, and a storage medium, which can obtain an alarm causality model based on deep reinforcement learning, avoid the use of manual rules, and realize accurate mining and monitoring of alarm causality. characterization.
  • the alarm causality mining method shown in FIG. 1 is executed by an alarm causality mining device, at least including but not limited to the following steps.
  • Step 101 Based on the system alarm information and the root cause label data of the system alarm information, build a system alarm environment for deep reinforcement learning.
  • Step 102 The agent interacts with the system alarm environment through deep reinforcement learning, and learns and generates an alarm causal model representing the alarm causality and structure.
  • the system alarm environment of deep reinforcement learning is built according to the system alarm information and the root cause label data of the system alarm information, and the agent of deep reinforcement learning conducts exploratory interaction with the system alarm environment to continuously explore and optimize.
  • the causal relationship between alarm variables is mined and represented to obtain the alarm causal model.
  • the accurate mining and characterization of the alarm causal relationship is realized.
  • the alarm causal model is obtained through deep reinforcement learning, avoiding the use of manual rules, thereby avoiding the problem of invalid causality caused by rule errors, and also avoiding the difficulty of obtaining variable causal assumptions and the problem of excessive hypothesis space.
  • building a system alarm environment for deep reinforcement learning includes: obtaining a system alarm graph indicating the system alarm information and the root cause label data of the system alarm information;
  • the system alarm graph includes the alarm category index, alarm feature vector and root cause mark of each node of the system; according to the system alarm graph, the system alarm environment is built.
  • the alarm cause and effect excavation device includes a system alarm environment module, and the system alarm environment module builds a system alarm environment according to the system alarm map.
  • the system alarm environment is constructed through the alarm graph, which makes the relationship between each node of the system more intuitive.
  • system alarm graph can be generated by the system alarm graph generating module.
  • the process of generating the system alarm graph based on the system alarm information by the system alarm graph generating module is illustrated below with an example.
  • the system alarm information includes system alarm log data (as shown in Table 1) and alarm key performance indicator (Key Performance Indicator, KPI) information (as shown in Table 2).
  • KPI Key Performance Indicator
  • node represents a node.
  • CPU Steal Time represents the CPU stealing time.
  • CPU: *% means CPU usage *%
  • Mem *% means memory usage *%
  • InputR: *% means data input rate is *%
  • OutputR: *% means data output rate is *%.
  • the system alarm map generation module obtains the system alarm information shown in Table 1 and Table 2, and performs cleaning and convergence processing on the system alarm information to remove numerical fields, invalid character fields, etc. in the system alarm information.
  • the system alarm map generation module classifies the processed data and assigns the alarm category index (Identity Document, ID) to obtain the system alarm category set dictionary.
  • the alarm category set dictionary generated based on Table 1 is:
  • the alarm category set dictionary indicates the corresponding relationship between the alarm category index and the actual alarm category.
  • the alarm system diagram generation module can search the alarm category set dictionary according to the alarm category corresponding to the system alarm information of each node, and determine the alarm category index corresponding to the system alarm information, so that the system alarm information
  • the corresponding alarm category index is recorded in the corresponding node of the system alarm graph, for the system alarm environment to respond to the instructions issued by the agent according to the alarm category index of each node.
  • the system alarm map generation module can digitize the alarm log data and alarm KPI information of each node in the system. For example, through word2vec technology, the alarm log data and alarm KPI information are converted into vectors. For example, based on the system alarm information of node_1 to node_3 in Table 1 and Table 2, the generated vectors are shown in Table 3.
  • the system alarm map generation module uses the node's log The vector and the KPI vector are fused to obtain the alarm feature vector of the node.
  • the manner of merging the log vector and the KPI vector may be, for example, merging the log vector and the KPI vector into one vector, as shown in Table 4.
  • the system alarm map generation module After obtaining the alarm category index and alarm feature vector of each node, the system alarm map generation module generates a system topology map according to the system topology data; combined with the system topology map, root cause label data, and the alarm category index and alarm feature vector of each node , to generate a system alarm graph.
  • the topology graph includes nodes and edges. Nodes in the system can be microservices, component instances, etc., and edges can be defined as relationships such as microservice call relationships and microservice data flow directions.
  • the root cause label data is used to indicate the root cause node. Specifically, in the system, if a node fails and spreads rapidly, triggering a failure alarm of its adjacent nodes, this node can be called the root cause node.
  • the system alarm graph generating module combines the system topology graph, the root cause label data, and the alarm feature vector and alarm category index of each node to obtain the system alarm graph.
  • each node can be, for example, an instantiated node of the system (such as a microservice component), the eigenvalue v of the node is the alarm eigenvector of the node, the eigenvalue Wid of the node is the alarm category index, and the eigenvalue rcn of the node Label the root cause.
  • node_1 ["node_3”] indicates that the out-degree node of "node_1” is “node_3”
  • node_2 ["node_3”] indicates that the out-degree node of "node_2” is “node_3”
  • node_3 ["node_4"] indicates that the outgoing node of "node_3" is “node_4".
  • N1 indicates node_1, v1 indicates the alarm feature vector of node_1, N2 indicates node_2, v2 indicates the alarm feature vector of node_2, N3 indicates node_3, v3 indicates the alarm feature vector of node_3, N4 indicates node_4, and v4 indicates the alarm feature vector of node_4.
  • batch sample data can be processed to obtain a set of system alarm graphs, and a system alarm environment can be built based on the set of system alarm graphs.
  • this embodiment does not limit the number of system alarm information.
  • system alarm information can also include a kind of alarm information in the system alarm log data and alarm KPI information
  • system alarm graph generation module uses the vector of the alarm information as the alarm feature vector of the node .
  • system alarm information may also include other alarm information, which is not limited in this embodiment.
  • system alarm map generation module can be a virtual module inside the alarm cause and effect mining device, or it can be a module of other devices independent of the alarm cause and effect mining device. This example does not limit the relationship between the system alarm map generation module and the alarm causality mining device.
  • the system alarm environment can establish a corresponding relationship between the alarm category index and the mining action index according to the alarm category index in the alarm category collection dictionary, so that when the mining action index issued by the agent is received Finally, according to the corresponding relationship between the mining action index and the alarm category index, determine the next node corresponding to the mining action index, and adjust the environment state of the system alarm environment.
  • the agent interacts with the system alarm environment through deep reinforcement learning, and learns and generates an alarm causal model representing the alarm causality and structure, including: the agent selects a mining action according to the current environment state of the system alarm environment; the system The alarm environment feeds back the reward value and inherited state to the agent according to the mining action, root cause marking, state transition mechanism and reward mechanism; among them, the index of the mining action corresponds to the index of the alarm category; the agent learns and Generate an alert causality model that characterizes alert causality and structure.
  • an alarm causality mining device is constructed. As shown in FIG. 3 , the alarm causality mining device 10 includes an agent module 11 and a system alarm environment module 12 .
  • the agent module 11 is responsible for the generation and learning of alarm mining strategies, and completes the mining of the causal relationship of alarm variables and the representation of the causal structure.
  • the system alarm environment module 12 constructs the system alarm environment according to the system alarm graph set and the alarm category set generated by the system alarm graph generation module 13, manages the state transition mechanism and the action reward mechanism, and responds to the intelligent system according to the state transition mechanism and the action reward mechanism.
  • the system alarm environment module 12 can also provide an environment interaction interface, so as to provide the agent module 11 with a system alarm environment for mining the causality of alarm variables.
  • the system alarm environment module 12 stores a system alarm graph with root cause node labels, and the system alarm graph is composed of system nodes and relationship edges.
  • the nodes of the system are microservice components, and the characteristics of the nodes are the characteristics of the alarm state, that is, the alarm feature vector; the relationship edge is the business relationship between the microservices, which can be the relationship between the call relationship and the data flow relationship between the microservices.
  • the root cause node label indicates whether the node is the root cause of the system alarm graph.
  • the agent module 11 conducts exploratory interaction with the system alarm environment module 12, and explores and walks the target node (root cause node) from the initial node during the interaction process. The agent module 11 accumulates Values are continuously explored and optimized to complete the mining and characterization of the causal relationship between alarm variables to obtain an alarm causal model.
  • the system alarm environment module 12 includes: an environment state representation and transfer function component, an environment action space function component, an alarm cause and effect mining reward function component and an environment state reset function component.
  • the environmental state characterization and transfer function component is responsible for the environmental state management of the system alarm environment, and stores the characterization mechanism and state transfer mechanism of the environmental state.
  • the interactive operation between the agent module 11 and the system alarm environment module 12 includes: the agent module 11 selects the corresponding excavation action according to the environment state of the current system alarm environment, and the system alarm environment module 12 generates the system alarm environment according to the current environment state and the mining action. Transition to the corresponding inherited state.
  • the environment state of the system alarm environment is determined according to the alarm state of the current node, and the alarm state of the current node is generated according to the alarm feature vector of the current node and the alarm feature vector of the adjacent nodes of the current node .
  • the alarm status of the current node consists of two parts: the alarm status of the node itself and the alarm status of adjacent nodes.
  • the adjacent nodes of the current node are the out-degree adjacent nodes or the in-degree adjacent nodes of the current node.
  • the source node of the edge is the in-degree adjacency point of the target node
  • the target node of the edge is the out-degree adjacency point of the source node.
  • the expected successor nodes of nodes with the same alarm feature vector may be inconsistent.
  • the alarm feature vector of the current node and the alarm feature vector of adjacent nodes can be obtained according to the current node and the current system alarm graph, and these two alarm feature vectors are fused to serve as the alarm state of the current node, and the current node
  • the alarm status of is used as the environmental status of the system alarm environment.
  • the environmental state (State) of the system alarm environment is characterized by the fusion of the alarm feature vector of the current node and the alarm feature vector of the out-degree adjacent nodes (or in-degree adjacent nodes), which is convenient for the agent module 11 to select mining actions.
  • the adjacent node is an out-degree adjacent node or an in-degree adjacent node can be selected according to an application scenario, which is not limited in this embodiment.
  • the process of merging the alarm feature vector of the current node and the alarm feature vectors of the adjacent nodes of the current node to obtain the alarm status of the current node is as follows: calculate the first fusion feature vector of the alarm feature vectors of the adjacent nodes of the current node, and the second A fusion feature vector is the mean value of the alarm feature vectors of the adjacent nodes of the current node; the alarm feature vector of the current node is spliced with the first fusion feature vector to obtain a second fusion feature vector, and the second fusion feature vector is the alarm state of the current node .
  • the adjacent node is the out-degree adjacent node
  • the current node is N3 as an example to illustrate the fusion process.
  • the out-degree adjacent nodes of N3 are node N4 and node N1, and the environment state of the system alarm environment is [v3, (v4+v1)/2].
  • v3 [1,2,3]
  • v1 [2,3,4]
  • v4 [2,5,6]
  • you can get: v4+v2 [4,8,10]
  • the state transition mechanism is responsible for determining the inherited state and inherited node of the system alarm environment according to the environment state of the current system alarm environment, the mining action, and the local action space of the current node.
  • the state transition mechanism includes: when the mining action belongs to the local action space of the current node, take the adjacent node of the current node that contains the alarm category index corresponding to the index of the mining action as the successor node; As the inherited state of the system alarm environment; in the case that the mining action does not belong to the local action space of the current node, set the inherited state of the system alarm environment to a specified state, for example, zero state.
  • the definition of the node transition action in the system alarm graph may be an action selected (or executed) for transitioning from the current node to the desired node.
  • Node transfer actions are divided into global action space and local action space.
  • the global action space refers to the optional transition action space in the sample environment of the alarm map of the whole system.
  • the local action space refers to the transfer action space (effective action space) allowed by the sample environment where the current step is located.
  • the state space of a certain system alarm environment is ABCDE, and action IDs are assigned to each state: 1:A, 2:B, 3:C, 4:D, 5:E. If the current state is B, then the global action space is ⁇ 1,2,3,4,5 ⁇ . If the node where the current step is located is node B and its local action space is ⁇ 1,3,4 ⁇ , it can choose action 3 to transfer to node C, but node B cannot transfer to node E.
  • the index of the mining action is consistent with the index of the alarm category.
  • its local action space is ⁇ 2, 3 ⁇ .
  • the mining action is not 2 and 3, for example, the index of the mining action is 5, which is not in the local action space of the current node, then it is judged that the mining action is an illegal action, and the node transfer is not performed, and the system alarm environment inheritance state State_ is directly set to Specifies the state, such as the zero state.
  • the accuracy of the node feature representation and the rationality of the transfer in the system alarm graph are guaranteed through the environment state representation mechanism and the state transition mechanism.
  • the meaning of the mining action is to select the corresponding mining action according to the alarm (fruit alarm) status of the current node, and transfer to the node due to the alarm, so the mining action Definitions can be options for alerting.
  • the system alarm information is collected (including but not limited to system alarm log data and KPI data), cleaned and converged through the system diagram generation module, and then the system alarm information is classified and indexed.
  • the alarm category space is the action space
  • the environmental state representation and transition function component corresponds each mining action to each alarm category
  • the mining action index corresponds to the alarm category index one by one.
  • the alarm causality mining reward functional component is responsible for feeding back corresponding rewards to the agent module 11 according to the reward mechanism, current environment state and execution actions, and the rewards represent the quality of the mining actions performed in the current state. Because in the same environment state, the rewards obtained by performing different mining actions are different, and the goal of the rewards is to better guide the agent module 11 to swim from the initial node to the target node (for example, it can be the root node of the system alarm graph Causal node) to quickly complete the mining and characterization of the causal relationship of alarm variables.
  • the reward mechanism includes: determining the reward value according to whether the mining action belongs to the local action space of the current node, and after the mining action is performed, the type of the successor node of the system alarm environment.
  • the system alarm environment module 12 performs a state transition after executing the mining action, and transfers the environmental state of the system alarm environment to the inherited state, which is reflected in the system alarm graph as transferring from the current node to the inherited node.
  • the inherited nodes are divided into four categories: root cause nodes, in-degree adjacent nodes of root cause nodes, zero out-degree nodes (or non-root cause nodes) and normal nodes.
  • the root cause node is the root cause node of the current system alarm graph, for example, node N5 in FIG. 4 .
  • the in-degree adjacent node of the root cause node is the source node of the in-degree edge of the root cause node.
  • the root cause node N5 has an in-degree edge N4 ⁇ N5, and its source node is N4, so N4 node is the source node of the root cause node N5 In-degree neighbor nodes.
  • a zero-out-degree node is a node with an out-degree of 0 and is not a root node, such as nodes N1 and N6 in Figure 4, whose out-degree edge is equal to 0.
  • Nodes other than the above types of nodes are classified as normal nodes, such as node N2 and node N3 in Fig. 4 .
  • the system alarm environment module 12 executes the mining action and transfers the system alarm environment from the current node to the successor node.
  • the currently executed mining action is an illegal action, negative rewards will be fed back;
  • the action in the local action space of the node, and the successor node is the root cause node, then the feedback is positive reward; if the currently executed mining action belongs to the action in the local action space of the current node, and the successor node is the root cause node indegree If the currently executed mining action belongs to the local action space of the current node, and the successor node is a node with zero out-degree, then a negative reward will be fed back; if the currently executed mining action belongs to the current node’s The action in the local action space, and the successor node is a normal node, feedback 0 reward.
  • the feedback reward is 0; and when transferring from node N3 to node N4, because node N4 is the root node in-degree adjacent node, the feedback is positive Reward; when transferring from node N4 to node N5, because node N5 is the root cause node, so feedback positive reward; if transferring from node N3 to node N1, because node N1 is a node with zero out-degree node, so feedback negative reward; If N3 performs an illegal action, it will feed back negative rewards.
  • the environmental status reset functional component is responsible for resetting the system alarm environment according to the environmental status reset mechanism. Specifically, in the process of interaction between the agent and the system alarm environment, it also includes: after the system alarm environment executes the mining action sent by the agent, if the interaction termination condition is met, the environment state reset mechanism is triggered.
  • the termination condition for the interaction between the agent and the system alarm environment includes at least one of: the number of interactions reaches an interaction threshold, the successor node is the root cause node, or the mining action does not belong to the local action space of the current node.
  • the termination conditions for the interaction between the agent and the system alarm environment include but are not limited to: the number of mining explorations (i.e., the number of interactions) in the current interaction cycle reaches a specified value (i.e., the interaction threshold), and the mining exploration reaches the root cause node of the system alarm graph (i.e., the inheritance node is the root cause node) and the agent outputs illegal mining actions.
  • Illegal mining actions refer to actions that do not belong to the local action space of the current node.
  • the mechanism for resetting the environment state includes: randomly selecting a system alarm graph from the system alarm graph set generated based on the system alarm information and the root cause label data of the system alarm information as the system alarm graph of the system alarm environment; In the system alarm diagram of the alarm environment, select a node whose out-degree or in-degree is not 0 as the start node; use the alarm state of the start node as the environment state of the system alarm environment, and feed it back to the agent so that the agent and the system The alert environment is reinteracted.
  • the agent is responsible for generating corresponding mining actions (Action) to interact with the system alarm environment according to the environment state of the system alarm environment, so as to realize alarm causal structure mining and characterization, and its functions include mining and characterization.
  • the agent can adopt the A2C model of the actor-critic architecture in the deep reinforcement learning model.
  • the structure of the A2C model is shown in Figure 6.
  • the agent 11 consists of an actor (Actor) network model and a critic (Critic ) network model composition.
  • the Actor network model is responsible for generating mining actions (Action) according to the environment state (State) of the system alarm environment 12. Therefore, the input dimension of the Actor model is consistent with the environment state (State) space dimension of the system alarm environment, and the output dimension is consistent with the system The action (Action) space dimension of the alarm environment is consistent.
  • the method for mining causality of alarms mentioned in this embodiment is illustrated below in combination with different meanings of different system alarm information and topological edges in the system alarm graph.
  • the alarm causality mining device 20 is communicatively connected with the alarm causality mining target system 30 .
  • the alarm causality mining target system 30 collects system alarm logs and system topology, labels the root causes, packages them into system alarm samples, and uploads the system alarm samples to the alarm cause and effect mining device 20 .
  • the alarm causality mining device 20 includes an agent module 21 and a system alarm environment module 22, and the steps for obtaining the alarm causality model are as follows:
  • Step 201 The system alarm environment module 22 analyzes the system alarm samples uploaded by the alarm causality mining target system 30.
  • Step 202 The system alarm environment module 22 constructs a system topology graph according to the system topology data, and creates graph node attributes.
  • Node attributes include alarm feature vector v, alarm category index wid and node root cause tag rcn. Clean, converge, and classify the alarm logs of all system alarm samples, obtain the set of alarm categories, and assign indexes to the alarm categories.
  • Step 203 the system alarm environment module 22 obtains the system alarm map. Specifically, taking a single sample as an example, the system alarm environment module 22 combines word2vec technology to vectorize the alarm log of the node, obtain the feature vector of the alarm log of the node, and assign it to the attribute v of the node corresponding to the system topology map; according to the steps
  • the distribution result of the alarm category index in 203 is to obtain the alarm category index of the node, and assign it to the attribute wid of the node corresponding to the system topology map; set the attribute rcn value of the root cause node in the system topology map to 1 according to the root cause label information , and the rcn value of other node attributes is set to 0.
  • the system alarm graph obtained above is stored in the system alarm graph collection.
  • Step 204 The system alarm environment module 22 constructs the system alarm environment action space according to the set of alarm categories. Specifically, each alarm category corresponds to a mining action, and the mining action index is consistent with the alarm category index.
  • the agent module 21 may include an Actor model and a Critic model.
  • Step 206 The system alarm environment module 22 creates a system alarm environment and an interaction experience pool.
  • Step 207 The system alarm environment module 22 resets the system alarm environment. First, a system alarm graph is randomly obtained from the system alarm graph set as the current system alarm graph A, and then a node with an out-degree not 0 is randomly selected from the current system alarm graph A as the current node, and finally according to the current node and its adjacent nodes The alarm feature vector v of the system alarm environment is fused to obtain the environment state (State) of the system alarm environment, and the environment state of the system alarm environment is fed back to the agent module 21 .
  • State environment state
  • Step 208 The agent module 21 reads the environment status of the system alarm environment, generates mining action (Action) according to the environment status of the system alarm environment, and sends it to the system alarm environment.
  • Action mining action
  • Step 209 The system alarm environment module 22 judges the legality of the Action, that is, judges whether the mining action is in the local action space of the current node. If the Action is illegal, go to step 210. If legal, go to step 211.
  • Step 210 The system alarm environment module 22 does not transfer the inheritance node, directly sets the alarm environment inheritance state (State_) to zero state, sets the action reward to -10.0, and sets the period interaction end flag (done) to true (True). Then step 212 is executed.
  • State_ alarm environment inheritance state
  • Step 211 The system alarm environment module 22 transfers to the node containing the alarm category index (wid) corresponding to the mining action index, takes this node as an inherited node, and obtains the inherited state (State_) of the system alarm environment according to the inherited node.
  • Step 212 The system alerts the environment module 22 to feed back the inheritance status, action rewards and cycle interaction end mark to the agent module 21.
  • Step 213 The agent module 21 forms a quaternion (State, Action, reward, State_) from the current state, mining action, action reward, and inheritance state, and stores it in the interactive experience pool as an experience sample, and sends the system alarm environment
  • the environment state (State) is updated to inherit state State_.
  • Step 214 The agent module 21 accumulatively adds 1 to the step_counter.
  • Step 215 The agent module 21 performs subsequent operations based on the following judgment results:
  • episode_counter ⁇ episode_num go to step 207, otherwise go to step 208.
  • Step 216 Complete the alarm causality mining, and export the Actor model of the agent module 21 as the target system alarm causality and structural representation model. It can provide secondary development support for the root cause location of system failures.
  • the alarm causality mining device 20 is communicatively connected with the alarm causality mining target system 30 .
  • the alarm causality mining target system 30 collects system alarm logs, alarm KPI information, and system topology, labels them as root causes, packages them into system alarm samples, and uploads the data to the alarm causality mining device 20 .
  • the alarm causal mining device 20 includes an agent module 21 and a system alarm environment module 22, and the steps for obtaining the alarm causal model are as follows:
  • Step 301 The system alarm environment module 22 analyzes the system alarm samples uploaded by the alarm causality mining target system.
  • Step 302 The system alarm environment module 22 constructs a system topology graph according to the system topology data, and creates graph node attributes.
  • Node attributes include alarm feature vector v, alarm category index wid and node root cause tag rcn. Clean, converge, and classify the alarm logs of all system alarm samples, obtain the set of alarm categories, and assign indexes to the alarm categories.
  • Step 303 the system alarm environment module 22 obtains the system alarm map. Specifically, taking a single sample as an example, the system alarm environment module 22 combines word2vec technology to vectorize the alarm log of the node to obtain the log vector of the node; vectorize the node alarm KPI information to obtain the KPI vector of the node; Splicing and merging with the KPI vector to obtain the alarm feature vector of the node; assigning the alarm feature vector to the attribute v of the node corresponding to the system topology map; according to the assignment result of the alarm category index in step 203, obtaining the alarm category index of the node, and Assign a value to the attribute wid of the node corresponding to the system topology map; set the attribute rcn value of the root cause node in the system topology map to 1, and set the attribute rcn value of other nodes to 0 according to the root cause label information.
  • the system alarm graph obtained above is stored in the system alarm graph collection.
  • Step 304 The system alarm environment module 22 constructs the system alarm environment action space according to the set of alarm categories. Specifically, each alarm category corresponds to a mining action, and the mining action index is consistent with the alarm category index.
  • the agent module 21 may include an Actor model and a Critic model.
  • Step 306 The system alarm environment module 22 creates a system alarm environment and an interaction experience pool.
  • Step 307 The system alarm environment module 22 resets the system alarm environment. First, a system alarm graph is randomly obtained from the system alarm graph set as the current system alarm graph A, and then a node with an out-degree not 0 is randomly selected from the current system alarm graph A as the current node, and finally according to the current node and its adjacent nodes The alarm feature vector v of the system alarm environment is fused to obtain the environment state (State) of the system alarm environment, and the environment state of the system alarm environment is fed back to the agent module 21 .
  • State environment state
  • Step 308 The agent module 21 reads the environment status of the system alarm environment, generates mining action (Action) according to the environment status of the system alarm environment, and sends it to the system alarm environment.
  • Action mining action
  • Step 309 The system alarm environment module 22 judges the legality of the Action, that is, judges whether the mining action is in the local action space of the current node. If the Action is illegal, go to step 310. If legal, go to step 311.
  • Step 310 The system alarm environment module 22 does not transfer the inheritance node, directly sets the alarm environment inheritance state (State_) to zero state, sets the action reward to -10.0, and sets the period interaction end flag (done) to true (True). Then step 212 is executed.
  • State_ alarm environment inheritance state
  • Step 311 The system alarm environment module 22 transfers to the node containing the alarm category index (wid) corresponding to the mining action index, takes this node as an inherited node, and obtains the inherited state (State_) of the system alarm environment according to the inherited node.
  • Step 312 The system alerts the environment module 22 to feed back the inheritance status, action rewards and cycle interaction end mark to the agent 21.
  • Step 313 The agent module 21 forms a quaternion (State, Action, reward, State_) from the current state, mining action, action reward, and inheritance state, and stores it in the interactive experience pool as an experience sample, and sends the system alarm environment
  • the environment state (State) is updated to inherit state State_.
  • Step 314 The agent module 21 accumulatively adds 1 to the step_counter.
  • Step 315 The agent module 21 performs subsequent operations based on the following judgment results:
  • step_counter> episode_len
  • episode_counter ⁇ episode_num go to step 307, otherwise go to step 308.
  • Step 316 Complete the alarm causality mining, and export the Actor model of the agent as the target system alarm causality and structural representation model. It can provide secondary development support for the root cause location of system failures.
  • step division of the above various methods is only for the sake of clarity of description. During implementation, it can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.
  • the embodiment of the present application also provides an alarm causal mining device.
  • the alarm causal mining device 20 includes: an agent module 21 and a system alarm environment module 22 .
  • the system alarm environment module 21 can build a deep reinforcement learning system alarm environment based on the system alarm information and the root cause label data of the system alarm information; the agent module 21 based on deep reinforcement learning interacts with the system alarm environment module 22 to learn and generate representations Alarm causality model for alarm causality and structure.
  • the agent module 21 based on deep reinforcement learning interacts with the system alarm environment module 22 to learn and generate representations Alarm causality model for alarm causality and structure.
  • this embodiment is a device implementation manner corresponding to the above-mentioned method embodiment, and this embodiment can be implemented in cooperation with the above-mentioned method embodiment.
  • the relevant technical details mentioned in the foregoing method embodiments are still valid in this embodiment, and will not be repeated here in order to reduce repetition.
  • the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.
  • modules involved in this embodiment are logical modules.
  • a logical unit can be a physical unit, or a part of a physical unit, or multiple physical units. Combination of units.
  • units that are not closely related to solving the technical problem proposed in the present application are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.
  • the embodiment of the present application also provides an alarm causal mining device, as shown in FIG. 12 , including: at least one processor 31; and a memory 32 connected in communication with at least one processor 31; The instructions executed by the processor 31 are executed by at least one processor 31, so that the at least one processor 31 can execute the above method embodiments.
  • the memory 32 and the processor 31 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 31 and various circuits of the memory 32 together.
  • the bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein.
  • the bus interface provides an interface between the bus and the transceivers.
  • a transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium.
  • the data processed by the processor 31 is transmitted on the wireless medium through the antenna, further, the antenna also receives the data and transmits the data to the processor 31 .
  • Processor 31 is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interfacing, voltage regulation, power management, and other control functions. And the memory 32 may be used to store data used by the processor 31 when performing operations.
  • the embodiment of the present application also provides a computer-readable storage medium storing a computer program.
  • the above method embodiments are implemented when the computer program is executed by the processor.
  • the program is stored in a storage medium, and includes several instructions to make a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种告警因果关系挖掘方法、告警因果挖掘装置及存储介质。告警因果关系挖掘方法包括：基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境（101）；通过深度强化学习的智能体与系统告警环境交互，学习并生成表征告警因果关系和结构的告警因果模型（102）。

Description

告警因果关系挖掘方法、告警因果挖掘装置及存储介质
交叉引用
本申请基于申请号为“202111113557.2”、申请日为2021年9月18日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本申请。
技术领域
本申请实施例涉及互联网技术领域，特别涉及一种告警因果关系挖掘方法、告警因果挖掘装置及存储介质。
背景技术
在数据挖掘领域，通常需要对数据变量间的因果关系和结构进行挖掘和表征。因果关系挖掘算法通常是基于规则的统计学方法实现。例如，基于变量对的共现频率或者变量波动的时序性进行挖掘，然后使用变量对因果系数进行表征。
然而，因果挖掘存在理论缺陷。具体地，若通过规则定义因果性，当所定规则无法正确表示因果关系时，所挖掘得到的因果关系是无效的。而基于共现频率和时序性只能挖掘变量间的关联性而无法表征因果性。而当前新兴的因果论推理方法需要专家经验提供变量因果假设进行验证挖掘或者是通过随机因果假设图进行验证挖掘，存在变量因果假设获取困难和假设空间过大问题。
发明内容
本申请实施例提供了一种告警因果关系挖掘方法，包括：基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境；通过深度强化学习的智能体与系统告警环境交互，学习并生成表征告警因果关系和结构的告警因果模型。
本申请实施例提供了一种告警因果挖掘装置，包括：基于深度强化学习的智能体模块和系统告警环境模块；系统告警环境模块基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境；智能体模块与系统告警环境模块交互，学习并生成表征告警因果关系和结构的告警因果模型。
本申请实施例提供了一种告警因果挖掘装置，包括：至少一个处理器；以及，与至少一个处理器通信连接的存储器；其中，存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行上述实施例提及的告警因果关系挖掘方法。
本申请实施例提供了一种计算机可读存储介质，存储有计算机程序，计算机程序被处理器执行时，实现上述实施例提及的告警因果关系挖掘方法。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本申请实施例中的告警因果关系挖掘方法的流程图；
图2是本申请实施例中的基于表1和表2生成的系统告警图；
图3是本申请实施例中的告警因果挖掘装置的结构示意图；
图4是本申请实施例中的另一系统告警图；
图5是本申请实施例中的系统告警环境的状态空间的示意图；
图6是本申请实施例中的A2C模型的结构示意图；
图7是本申请实施例中的告警因果挖掘装置20与告警因果挖掘目标系统30的交互示意图；
图8是本申请实施例中的告警因果挖掘装置20得到告警因果模型的过程示意图一；
图9是本申请另一实施例中的告警因果挖掘装置20与告警因果挖掘目标系统30的交互示意图；
图10是本申请实施例中的告警因果挖掘装置20得到告警因果模型的过程示意图二；
图11是本申请另一实施例中的告警因果挖掘装置的结构示意图；
图12是本申请又一实施例中的告警因果挖掘装置的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请的各实施例进行详细的阐述。然而，本领域的普通技术人员可以理解，在本申请各实施例中，为了使读者更好地理解本申请而提出了许多技术细节。但是，即使没有这些技术细节和基于以下各实施例的种种变化和修改，也可以实现本申请所要求保护的技术方案。以下各个实施例的划分是为了描述方便，不应对本申请的具体实现方式构成任何限定，各个实施例在不矛盾的前提下可以相互结合相互引用。
在本申请公开的描述中，需要理解的是，术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性。此外，在本申请公开的描述中，除非另有说明，“多个”的含义是两个或两个以上。
本申请实施方式的目的在于提供一种告警因果关系挖掘方法、告警因果挖掘装置及存储介质，能够基于深度强化学习得到告警因果模型，避免了手工规则的使用，实现了告警因果关系的准确挖掘和表征。
本申请实施例中，如图1所示的告警因果关系挖掘方法，由告警因果挖掘装置执行，至少包括但不限于如下步骤。
步骤101：基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境。
步骤102：通过深度强化学习的智能体与系统告警环境交互，学习并生成表征告警因果关系和结构的告警因果模型。
本申请实施例中，根据系统告警信息和系统告警信息的根因标签数据搭建深度强化学习的系统告警环境，深度强化学习的智能体通过与系统告警环境进行探索性交互，来不断的探索优化完成告警变量间因果关系挖掘和表征，从而得到告警因果模型，通过利用告警因果模型，实现了告警因果关系的准确挖掘和表征。通过深度强化学习得到告警因果模型，避免了手工规则的使用，进而避免了规则错误导致因果关系无效的问题，也避免了变量因果假设获取困难和 假设空间过大问题。
在一个实施例中，基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境，包括：获取指示系统告警信息和系统告警信息的根因标签数据的系统告警图；系统告警图包括系统的各节点的告警类别索引、告警特征向量和根因标记；根据系统告警图，搭建系统告警环境。具体地，告警因果挖掘装置包括系统告警环境模块，系统告警环境模块根据系统告警图，搭建系统告警环境。通过告警图构建系统告警环境，使得系统各节点关系更直观。
在一个例子中，系统告警图可以由系统告警图生成模块生成。以下对系统告警图生成模块基于系统告警信息生成系统告警图的过程进行举例说明。
例如，系统告警信息包括系统告警日志数据(如表1所示)和告警关键性能指标(Key Performance Indicator，KPI)信息(如表2所示)。表1和表2中，node表示节点。表1中，CPU Steal Time表示中央处理器偷盗时间。表2中，CPU：*％表示CPU占用*％，Mem：*％表示存储器占用*％，InputR：*％表示数据输入速率为*％，OutputR：*％表示数据输出速率为*％。
表1
Figure PCTCN2022098772-appb-000001
Figure PCTCN2022098772-appb-000002
表2
Figure PCTCN2022098772-appb-000003
系统告警图生成模块获取如表1和表2所示的系统告警信息，并对系统告警信息进行清洗和收敛等处理，以去除系统告警信息中数值字段、无效字符字段等。系统告警图生成模块对处理后的数据进行分类，并分配告警类别索引(Identity Document，ID)，得到系统的告警类别集合字典。例如，基于表1生成的告警类别集合字典为：
Figure PCTCN2022098772-appb-000004
其中，告警类别集合字典指示告警类别索引和实际的告警类别的对应关系。告警系统图生成模块在生成系统告警图的过程中，可根据各节点的系统告警信息对应的告警类别，查找该告警类别集合字典，确定该系统告警信息对应的告警类别索引，以便将系统告警信息对应的告警类别索引记录在系统告警图的对 应节点中，供系统告警环境根据各节点的告警类别索引，响应智能体下发的指令。
可选择的，在获取得到系统的告警类别集合字典和各节点的告警类别索引之前或之后，系统告警图生成模块可对系统中的每一个节点的告警日志数据和告警KPI信息进行数字化。例如，通过word2vec技术，将告警日志数据和告警KPI信息转化为向量。例如，基于表1和表2中的node_1至node_3的系统告警信息，生成的向量如表3所示。
表3
Figure PCTCN2022098772-appb-000005
在得到如表3所示的基于告警日志数据生成的向量(简称日志向量)和基于告警KPI信息生成的向量(简称KPI向量)后，针对每个节点，系统告警图生成模块将该节点的日志向量和KPI向量进行融合，得到该节点的告警特征向量。其中，日志向量和KPI向量融合的方式可例如是将日志向量和KPI向量合并为一个向量，如表4所示。
表4
Figure PCTCN2022098772-appb-000006
Figure PCTCN2022098772-appb-000007
在获取各节点的告警类别索引和告警特征向量后，系统告警图生成模块根据系统拓扑数据，生成系统拓扑图；结合系统拓扑图、根因标签数据、以及各节点的告警类别索引和告警特征向量，生成系统告警图。其中，拓扑图包括节点和边，节点在系统中可以为微服务、组件实例等，边则可以定义为微服务调用关系、微服务数据流向等关系。根因标签数据用于指示根因节点。具体地，在系统中，某一节点发生故障后迅速传播，引发其邻接节点故障告警，该节点可称为根因节点。系统告警图生成模块结合系统拓扑图、根因标签数据、以及各节点的告警特征向量和告警类别索引，得到系统告警图。系统告警图中，每个节点可例如为系统的实例化节点(例如微服务组件)，节点的特征值v为节点的告警特征向量，节点的特征值Wid为告警类别索引，节点的特征值rcn为根因标签。
例如，若表1和表2对应的系统拓扑数据为：{"node_1":["node_3"],"node_2":["node_3"],"node_3":["node_4"]}，则基于表1和表2生成的系统告警图如图2所示。其中，"node_1":["node_3"]表示"node_1"的出度节点为"node_3"，"node_2":["node_3"]表示"node_2"的出度节点为"node_3","node_3":["node_4"]表示"node_3"的出度节点为"node_4"。N1表示node_1，v1表示node_1的告警特征向量，N2表示node_2，v2表示node_2的告警特征向量，N3表示node_3，v3表示node_3的告警特征向量，N4表示node_4，v4表示node_4的告警特征向量。Wid表示告警类别索引，rcn表示根因节点标记。rcn＝0表示该节点不是根因节点，rcn＝1表示该节点是根因节点。
需要说明的是，本领域技术人员可以理解，上述例子为一个告警信息样本的处理过程，实际处理中，可以是批量的样本数据处理，得到系统告警图集合， 基于系统告警图集合搭建系统告警环境，本实施例不限制系统告警信息的数量。
需要说明的是，本领域技术人员可以理解，系统告警信息也可以包括系统告警日志数据和告警KPI信息中的一种告警信息，系统告警图生成模块将该告警信息的向量作为节点的告警特征向量。
需要说明的是，本领域技术人员可以理解，系统告警信息也可以包括其他告警信息，本实施例不做限制。
需要说明的是，本领域技术人员可以理解，本实施例中，系统告警图生成模块可以是告警因果挖掘装置内部的虚拟模块，也可以是独立于告警因果挖掘装置的其他装置的模块，本实施例不限制系统告警图生成模块和告警因果挖掘装置的关系。
在本申请的一个实施例中，系统告警环境可根据告警类别集合字典中的告警类别索引，建立告警类别索引和挖掘动作的索引的对应关系，以便在接收到智能体下发的挖掘动作的索引后，根据挖掘动作的索引和告警类别索引的对应关系，确定与挖掘动作索引对应的下一节点，并调整系统告警环境的环境状态。
在一个实施例中，通过深度强化学习的智能体与系统告警环境交互，学习并生成表征告警因果关系和结构的告警因果模型，包括：智能体根据系统告警环境当前的环境状态选择挖掘动作；系统告警环境根据挖掘动作、根因标记、状态转移机制和奖励机制，反馈奖励值和继承状态至智能体；其中，挖掘动作的索引与告警类别索引对应；智能体根据挖掘动作和奖励值，学习并生成表征告警因果关系和结构的告警因果模型。具体地，基于深度强化学习模型，构建告警因果挖掘装置。告警因果挖掘装置10如图3所示，包括智能体模块11和系统告警环境模块12。智能体模块11负责告警挖掘策略的生成和学习，完成告警变量因果关系的挖掘和因果结构表征。系统告警环境模块12根据系统告警图生成模块13生成的系统告警图集合和告警类别集合，构建系统告警环境，对状态转移机制和动作奖励机制进行管理，并根据状态转移机制和动作奖励机制响应智能体模块11下发的挖掘动作。系统告警环境模块12还可提供环境交互接口，从而为智能体模块11提供用于挖掘告警变量因果关系的系统告警环境。系统告警环境模块12存储有带有根因节点标签的系统告警图，而系统告警图由系统的节点和关系边组成。其中，系统的节点为微服务组件，节点特征为其告 警状态特征，即告警特征向量；关系边为微服务间的业务关系，可以为微服务间调用关系、数据流向关系等关系。根因节点标签则表示节点是否为系统告警图的根因。智能体模块11通过与系统告警环境模块12进行探索性交互，在交互过程中从初始节点为起点对目标节点(根因节点)进行探索游走，智能体模块11通过不断最大化挖掘动作奖励累积值来不断的探索优化完成告警变量间因果关系挖掘和表征，以得到告警因果模型。
在一个实施例中，系统告警环境模块12包括：环境状态表征及转移功能组件、环境动作空间功能组件、告警因果挖掘奖励功能组件和环境状态复位功能组件。
以下对各个功能组件的功能进行举例说明。
1、环境状态表征及转移功能组件
环境状态表征及转移功能组件负责系统告警环境的环境状态管理，存储有环境状态的表征机制和状态转移机制。智能体模块11与系统告警环境模块12交互操作包括：智能体模块11根据当前系统告警环境的环境状态选择相应的挖掘动作，系统告警环境模块12根据当前的环境状态和该挖掘动作将系统告警环境转移到相应继承状态中。
在一个例子中，环境状态的表征机制中，系统告警环境的环境状态根据当前节点的告警状态确定，当前节点的告警状态根据当前节点的告警特征向量，以及当前节点的邻接节点的告警特征向量生成。具体地，当前节点的告警状态由两部分组成：节点自身的告警状态和邻接节点的告警状态。当前节点的邻接节点为当前节点的出度邻接节点或入度邻接节点。其中，在系统告警图中，边的源节点为目标节点的入度邻接点，边的目标节点为源节点的出度邻接点。考虑到在不同出度邻接节点或入度邻接节点的情况下，具有同样告警特征向量的节点的期望继承节点可以是不一致的。本实施例中，可根据当前节点和当前的系统告警图获取当前节点的告警特征向量和邻接节点的告警特征向量，对这两个告警特征向量进行融合，作为当前节点的告警状态，将当前节点的告警状态作为系统告警环境的环境状态。系统告警环境的环境状态(State)采用当前节点的告警特征向量及其出度邻接节点(或者入度邻接节点)的告警特征向量融合进行表征，便于智能体模块11选择挖掘动作。
需要说明的是，本领域技术人员可以理解，邻接节点为出度邻接节点或入度邻接节点可以根据应用场景进行选择，本实施例不做限制。
在一个例子中，当前节点的告警特征向量和当前节点的邻接节点的告警特征向量融合得到当前节点的告警状态的过程如下：计算当前节点的邻接节点的告警特征向量的第一融合特征向量，第一融合特征向量为当前节点的邻接节点的告警特征向量的均值；将当前节点的告警特征向量和第一融合特征向量拼接，得到第二融合特征向量，第二融合特征向量为当前节点的告警状态。
以系统告警图如图4所示，邻接节点为出度邻接节点，当前节点为N3为例，对融合过程进行举例说明。N3的出度邻接节点为节点N4和节点N1，系统告警环境的环境状态为[v3,(v4+v1)/2]。例如，v3＝[1,2,3]，v1＝[2,3,4]，v4＝[2,5,6]，则通过计算，可以得到：v4+v2＝[4,8,10]，(v4+v1)/2＝[2,4,5]，[v3,(v4+v1)/2]＝[1,2,3,2,4,5])。
在一个例子中，状态转移机制负责根据当前系统告警环境的环境状态、挖掘动作、当前节点的局部动作空间决定系统告警环境的继承状态和继承节点。具体地，状态转移机制包括：在挖掘动作属于当前节点的局部动作空间的情况下，将当前节点的含有与挖掘动作的索引对应的告警类别索引的邻接节点作为继承节点；将继承节点的告警状态作为系统告警环境的继承状态；在挖掘动作不属于当前节点的局部动作空间的情况下，将系统告警环境的继承状态设置为指定状态，例如，零状态。具体地，系统告警图的节点转移动作的定义可以是从当前节点转移到期望节点所选择(或执行)的动作。节点转移动作分为全局动作空间和局部动作空间。全局动作空间是指整个系统告警图的样本环境中可选的转移动作空间。局部动作空间是指当前步骤所处的样本环境允许的转移动作空间(有效动作空间)。例如，如图5所示，某一系统告警环境的状态空间为ABCDE，对每一个状态分配动作ID：1:A，2:B，3:C，4:D，5:E。若当前状态为B，那么全局动作空间为{1,2,3,4,5}。如果当前步骤所处的节点为B节点，其局部动作空间为{1,3,4}，其可以选择动作3转移到C节点，但是B节点无法转移到E节点的。
例如，挖掘动作的索引和告警类别索引一致，以图4为例，对于节点N2，其局部动作空间为{2，3}，如果智能体模块11的挖掘动作的索引为2，对应的 是wid＝2的节点N4，则将节点N4作为继承节点，进行状态转移；并根据继承节点的告警状态，获取系统告警环境的继承状态(State_)。如果挖掘动作的索引为3，对应的是wid＝3的节点N3，则将节点N3作为继承节点，进行状态转移；并根据继承节点的告警状态，获取系统告警环境的继承状态(State_)。如果挖掘动作不是2和3，例如挖掘动作的索引为5，不在当前节点的局部动作空间中，则判断挖掘动作为非法动作，则不进行节点转移，直接将系统告警环境继承状态State_设置为指定状态，例如零状态。
本实施例中，通过环境状态表征机制及状态转移机制，保证了系统告警图的节点特征表征的准确性和转移的合理性。
需要说明的是，本领域技术人员可以理解，指定状态也可以是其他状态，本实施例不做限制。
2、环境动作空间功能组件
负责交互环境的动作空间的设计。具体地，在智能体模块11与系统告警环境模块12交互过程中，挖掘动作的含义为根据当前节点的告警(果告警)状态，选择相应的挖掘动作，转移到因告警节点中，所以挖掘动作定义可以为因告警的选择。通过系统图生成模块对系统告警信息进行采集(包括但不限于系统告警日志数据和KPI数据)、清洗、收敛，然后对系统告警信息进行分类和索引分配。本实施例中，告警类别空间即动作空间，环境状态表征及转移功能组件将每一个挖掘动作对应每一个告警类，挖掘动作的索引与告警类别索引一一对应。
3、告警因果挖掘奖励功能组件
告警因果挖掘奖励功能组件负责根据奖励机制、当前环境状态和执行动作，为智能体模块11反馈相应的奖励，该奖励表征当前状态下所执行的挖掘动作的好坏。因为同一个环境状态下，执行不同的挖掘动作获取的奖励是不一样的，奖励的目标是为了更好地引导智能体模块11从初始节点游走到目标节点(例如可以为系统告警图的根因节点)，快速的完成告警变量因果关系挖掘和表征。
可选择的，奖励机制包括：根据挖掘动作是否属于当前节点的局部动作空间，以及执行挖掘动作后，系统告警环境的继承节点的类型，确定奖励值。具体地，系统告警环境模块12在执行完挖掘动作后进行状态转移，将系统告警环 境的环境状态转移至继承状态中，在系统告警图中体现为从当前节点转移到继承节点中。
可选择的，将继承节点分为四类：根因节点、根因节点的入度邻接节点、零出度节点(或非根因节点)和正常节点。根因节点为当前系统告警图的根因节点，例如，图4中的节点N5。根因节点的入度邻接节点为根因节点入度边的源节点，例如图4中根因节点N5有一条入度边N4→N5，其源节点为N4，所以N4节点为根因节点N5的入度邻接节点。零出度节点为出度为0且不是根因节点的节点，例如图4中节点N1和节点N6，其出度边等于0。除以上类型节点外的节点归为正常节点，例如图4中的节点N2和节点N3。在交互过程中，系统告警环境模块12执行挖掘动作并将系统告警环境的从当前节点转移到继承节点，如果当前执行的挖掘动作为非法动作，则反馈负奖励；如果当前执行的挖掘动作属于当前节点的局部动作空间内的动作，且，继承节点为根因节点，则反馈正奖励；如果当前执行的挖掘动作属于当前节点的局部动作空间内的动作，且，继承节点为根因节点入度邻接点，则反馈正奖励；如果当前执行的挖掘动作属于当前节点的局部动作空间内的动作，且，继承节点为零出度节点，则反馈负奖励；如果当前执行的挖掘动作属于当前节点的局部动作空间内的动作，且，继承节点为正常节点，反馈0奖励。
例如，当前节点为节点N2，转移到节点N3时，N3为正常节点，所以反馈奖励为0；而从节点N3转移到节点N4时，因为节点N4为根因节点入度邻接节点，所以反馈正奖励；从节点N4转移到节点N5时，因为节点N5为根因节点，所以反馈正奖励；如果从节点N3转移到节点N1，因为节点N1为零出度节点，所以反馈负奖励；如果在节点N3执行非法动作则反馈负奖励。
需要说明的是，本领域技术人员可以理解，以上正奖励的具体奖励值和负奖励的具体奖励值可以根据应用场景特点进行设置，本实施例不做限制。
值得一提的是，通过设立奖励机制，实现了因果告警挖掘动作的正确引导，保证了挖掘动作的合法性。
4、环境状态复位功能组件
环境状态复位功能组件负责根据环境状态复位机制，对系统告警环境进行复位。具体地，在智能体与系统告警环境交互过程中，还包括：系统告警环境 在执行完智能体下发的挖掘动作后，若满足交互终止条件，触发环境状态复位机制。
可选择的，智能体与系统告警环境的交互终止条件包括：交互次数达到交互阈值、继承节点为根因节点或挖掘动作不属于当前节点的局部动作空间中的至少一个。具体地，智能体与系统告警环境交互终止条件包括但不限于：当前交互周期的挖掘探索次数(即交互次数)达到指定数值(即交互阈值)、挖掘探索达到系统告警图根因节点(即继承节点为根因节点)和智能体输出非法挖掘动作。非法挖掘动作是指不属于当前节点的局部动作空间的动作。当满足以上条件中的任意一条时，对系统告警环境进行复位，进行下一周期告警因果挖掘探索。
可选择的，环境状态复位机制包括：在基于系统告警信息和系统告警信息的根因标签数据生成的系统告警图集合中，随机选取一个系统告警图，作为系统告警环境的系统告警图；从系统告警环境的系统告警图中，选择一个出度或入度不为0的节点作为起始节点；将起始节点的告警状态作为系统告警环境的环境状态，反馈给智能体，以便智能体与系统告警环境重新进行交互。
在一个实施例中，智能体负责根据系统告警环境的环境状态，生成相应的挖掘动作(Action)与系统告警环境进行交互，实现告警因果结构挖掘和表征，其功能包括挖掘和表征。
可选择的，智能体可以采用深度强化学习模型中的演员-评论家架构的A2C模型，A2C模型的结构如图6所示，智能体11由一个演员(Actor)网络模型和一个评论家(Critic)网络模型组成。其中，Actor网络模型负责根据系统告警环境12的环境状态(State)生成挖掘动作(Action)，因此，Actor模型的输入维度与系统告警环境的环境状态(State)空间维度一致，而输出维度与系统告警环境的动作(Action)空间维度一致。Critic网络模型负责评估动作-状态Q值Q(State,Action)，用于Actor网络模型的优化，所以Critic网络模型的输入维度＝State空间维度+Action空间维度，输出维度＝1。
以下结合针对不同的系统告警信息和系统告警图中拓扑边的不同含义，对 本实施例提及的告警因果关系挖掘方法进行举例说明。
在一个实施例中，如图7所示，告警因果挖掘装置20与告警因果挖掘目标系统30通信连接。告警因果挖掘目标系统30对系统的告警日志和系统拓扑进行采集，打上根因标签，打包成系统告警样本，将系统告警样本上传到告警因果挖掘装置20。例如，如图8所示，告警因果挖掘装置20包括智能体模块21和系统告警环境模块22，得到告警因果模型的步骤如下：
步骤201：系统告警环境模块22对告警因果挖掘目标系统30上传的系统告警样本进行解析。
步骤202：系统告警环境模块22根据系统拓扑数据构建系统拓扑图，并创建图节点属性。节点属性包括告警特征向量v、告警类别索引wid和节点根因标记rcn。对所有系统告警样本的告警日志进行清洗、收敛、分类，获取告警类别集合，并对告警类别分配索引。
步骤203：系统告警环境模块22获取系统告警图。具体地，以单样本为例，系统告警环境模块22结合word2vec技术，将节点的告警日志进行向量化，获取节点的告警日志特征向量，并赋值给系统拓扑图对应的节点的属性v；根据步骤203中的告警类别索引的分配结果，获取节点的告警类别索引，并赋值给系统拓扑图对应的节点的属性wid；根据根因标签信息对系统拓扑图中的根因节点的属性rcn值置1，其余节点属性rcn值置0。
以上获取的系统告警图存放到系统告警图集合中。
步骤204：系统告警环境模块22根据告警类别集合构建系统告警环境动作空间。具体地，每一个告警类别对应一个挖掘动作，挖掘动作索引与告警类别索引保持一致。
步骤205：告警因果挖掘装置20创建智能体模块21，并设置告警因果挖掘周期数(episode_num)和周期长度(episode_len)，初始化周期和周期长度计数器episode_counter＝0，step_counter＝0。其中，智能体模块21可以包括Actor模型和Critic模型。
步骤206：系统告警环境模块22创建系统告警环境和交互经验池。
步骤207：系统告警环境模块22对系统告警环境进行复位。首先从系统告警图集合中随机获取一个系统告警图作为当前系统告警图A，然后从当前系统 告警图A中随机选取一个出度不为0的节点作为当前节点，最后根据当前节点及其邻接节点的告警特征向量v，融合得到系统告警环境的环境状态(State)，并将系统告警环境的环境状态反馈给智能体模块21。
步骤208：智能体模块21读取系统告警环境的环境状态，根据系统告警环境的环境状态生成挖掘动作(Action)，并发送到系统告警环境中。
步骤209：系统告警环境模块22判断Action的合法性，即判断挖掘动作是否在当前节点的局部动作空间中。如果Action非法，执行步骤210。如果合法，执行步骤211。
步骤210：系统告警环境模块22不进行继承节点转移，直接将告警环境继承状态(State_)设置为零状态，将动作奖励设置为-10.0，周期交互结束标记(done)设置为正确(True)。之后执行步骤212。
步骤211：系统告警环境模块22转移到包含挖掘动作索引所对应的告警类别索引(wid)的节点中，将该节点作为继承节点，并根据继承节点获取系统告警环境的继承状态(State_)。如果继承节点为rcn＝1，那么将动作奖励(reward)设置为50.0，周期交互结束标记(done)设置为True；如果继承节点的出度＝0，那么将动作奖励设置为-1.0，周期交互结束标记设置为True；如果继承节点为根因节点入度邻接节点，那么将动作奖励设置为1.0，周期交互结束标记设置为错误(False)；如果继承节点为正常节点，那么将动作奖励设置为0.0，周期交互结束标记设置为False。
步骤212：系统告警环境模块22将继承状态、动作奖励和周期交互结束标记反馈至智能体模块21。
步骤213：智能体模块21将当前状态、挖掘动作、动作奖励和继承状态，组成四元组(State，Action，reward，State_)，作为经验样本存放到交互经验池中，并将系统告警环境的环境状态(State)更新为继承状态State_。
步骤214：智能体模块21将step_counter累计加1。
步骤215：智能体模块21基于以下判断结果执行后续操作：
如果确定周期交互标记done＝False并且step_counter<episode_len，跳转到步骤208；
如果周期交互标记done＝True或者step_counter>＝episode_len，那么 episode_counter累计加1，并且利用交互经验池中经验样本对智能体模块21进行模型优化(包括Actor模型和Critic模型)，使得智能体模块21能最大化交互累积回报；
如果episode_counter<episode_num，跳转到步骤207，否则跳转到步骤208。
步骤216：完成告警因果挖掘，将智能体模块21的Actor模型导出，作为目标系统告警因果关系和结构表征模型。可为系统故障根因定位提供二次开发支撑等。
在一个实施例中，如图9所示，告警因果挖掘装置20与告警因果挖掘目标系统30通信连接。告警因果挖掘目标系统30对系统的告警日志、告警KPI信息、系统拓扑采集，打上根因标签，打包成系统告警样本，然后将数据上传到告警因果挖掘装置20。如图10所示，告警因果挖掘装置20包括智能体模块21和系统告警环境模块22，得到告警因果模型的步骤如下：
步骤301：系统告警环境模块22对告警因果挖掘目标系统上传的系统告警样本进行解析。
步骤302：系统告警环境模块22根据系统拓扑数据构建系统拓扑图，并创建图节点属性。节点属性包括告警特征向量v、告警类别索引wid和节点根因标记rcn。对所有系统告警样本的告警日志进行清洗、收敛、分类，获取告警类别集合，并对告警类别分配索引。
步骤303：系统告警环境模块22获取系统告警图。具体地，以单样本为例，系统告警环境模块22结合word2vec技术对节点的告警日志进行向量化，获取节点的日志向量；对节点告警KPI信息进行向量化，获取节点的KPI向量；对日志向量和KPI向量进行拼接融合，获取节点的告警特征向量；将告警特性向量赋值给系统拓扑图对应的节点的属性v；根据步骤203中的告警类别索引的分配结果，获取节点的告警类别索引，并赋值给系统拓扑图对应的节点的属性wid；根据根因标签信息对系统拓扑图中的根因节点的属性rcn值置1，其余节点属性rcn值置0。
以上获取的系统告警图存放到系统告警图集合中。
步骤304：系统告警环境模块22根据告警类别集合构建系统告警环境动作 空间。具体地，每一个告警类别对应一个挖掘动作，挖掘动作索引与告警类别索引保持一致。
步骤305：告警因果挖掘装置20创建智能体模块21，并设置告警因果挖掘周期数(episode_num)和周期长度(episode_len)，初始化周期和周期长度计数器episode_counter＝0，step_counter＝0。其中，智能体模块21可以包括Actor模型和Critic模型。
步骤306：系统告警环境模块22创建系统告警环境和交互经验池。
步骤307：系统告警环境模块22对系统告警环境进行复位。首先从系统告警图集合中随机获取一个系统告警图作为当前系统告警图A，然后从当前系统告警图A中随机选取一个出度不为0的节点作为当前节点，最后根据当前节点及其邻接节点的告警特征向量v，融合得到系统告警环境的环境状态(State)，并将系统告警环境的环境状态反馈给智能体模块21。
步骤308：智能体模块21读取系统告警环境的环境状态，根据系统告警环境的环境状态生成挖掘动作(Action)，并发送到系统告警环境中。
步骤309：系统告警环境模块22判断Action的合法性，即判断挖掘动作是否在当前节点的局部动作空间中。如果Action非法，执行步骤310。如果合法，执行步骤311。
步骤310：系统告警环境模块22不进行继承节点转移，直接将告警环境继承状态(State_)设置为零状态，将动作奖励设置为-10.0，周期交互结束标记(done)设置为正确(True)。之后执行步骤212。
步骤311：系统告警环境模块22转移到包含挖掘动作索引所对应的告警类别索引(wid)的节点中，将该节点作为继承节点，并根据继承节点获取系统告警环境的继承状态(State_)。如果继承节点为rcn＝1，那么将动作奖励(reward)设置为50.0，周期交互结束标记(done)设置为True；如果继承节点的出度＝0，那么将动作奖励设置为-1.0，周期交互结束标记设置为True；如果继承节点为根因节点入度邻接节点，那么将动作奖励设置为1.0，周期交互结束标记设置为错误(False)；如果继承节点为正常节点，那么将动作奖励设置为0.0，周期交互结束标记设置为False。
步骤312：系统告警环境模块22将继承状态、动作奖励和周期交互结束标 记反馈至智能体21。
步骤313：智能体模块21将当前状态、挖掘动作、动作奖励和继承状态，组成四元组(State，Action，reward，State_)，作为经验样本存放到交互经验池中，并将系统告警环境的环境状态(State)更新为继承状态State_。
步骤314：智能体模块21将step_counter累计加1。
步骤315：智能体模块21基于以下判断结果执行后续操作：
如果确定周期交互标记done＝False并且step_counter<episode_len，跳转到步骤308；
如果周期交互标记done＝True或者step_counter>＝episode_len，那么episode_counter累计加1，并且利用交互经验池中经验样本对智能体进行模型优化(包括Actor模型和Critic模型)，使得智能体能最大化交互累积回报；
如果episode_counter<episode_num，跳转到步骤307，否则跳转到步骤308。
步骤316：完成告警因果挖掘，将智能体的Actor模型导出，作为目标系统告警因果关系和结构表征模型。可为系统故障根因定位提供二次开发支撑等。
上面各种方法的步骤划分，只是为了描述清楚，实现时可以合并为一个步骤或者对某些步骤进行拆分，分解为多个步骤，只要包括相同的逻辑关系，都在本专利的保护范围内；对算法中或者流程中添加无关紧要的修改或者引入无关紧要的设计，但不改变其算法和流程的核心设计都在该专利的保护范围内。
本申请实施例还提供一种告警因果挖掘装置，如图11所示，告警因果挖掘装置20包括：智能体模块21和系统告警环境模块22。系统告警环境模块21可基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境；基于深度强化学习的智能体模块21与系统告警环境模块22交互，学习并生成表征告警因果关系和结构的告警因果模型。其中，智能体模块21和系统告警环境模块22的功能组件等及交互方式可参考上述实施例的相关描述，此处不再赘述。
不难发现，本实施例为与上述方法实施例相对应的装置实施方式，本实施例可与上述方法实施例互相配合实施。上述方法实施例中提到的相关技术细节 在本实施例中依然有效，为了减少重复，这里不再赘述。相应地，本实施例中提到的相关技术细节也可应用在上述方法实施例中。
值得一提的是，本实施例中所涉及到的各模块均为逻辑模块，在实际应用中，一个逻辑单元可以是一个物理单元，也可以是一个物理单元的一部分，还可以以多个物理单元的组合实现。此外，为了突出本申请的创新部分，本实施例中并没有将与解决本申请所提出的技术问题关系不太密切的单元引入，但这并不表明本实施例中不存在其它的单元。
本申请实施例还提供一种告警因果挖掘装置，如图12所示，包括：至少一个处理器31；以及与至少一个处理器31通信连接的存储器32；其中，存储器存储有可被至少一个处理器31执行的指令，指令被至少一个处理器31执行，以使至少一个处理器31能够执行上述方法实施例。
其中，存储器32和处理器31采用总线方式连接，总线可以包括任意数量的互联的总线和桥，总线将一个或多个处理器31和存储器32的各种电路连接在一起。总线还可以将诸如外围设备、稳压器和功率管理电路等之类的各种其他电路连接在一起，这些都是本领域所公知的，因此，本文不再对其进行进一步描述。总线接口在总线和收发机之间提供接口。收发机可以是一个元件，也可以是多个元件，比如多个接收器和发送器，提供用于在传输介质上与各种其他装置通信的单元。经处理器31处理的数据通过天线在无线介质上进行传输，进一步，天线还接收数据并将数据传送给处理器31。
处理器31负责管理总线和通常的处理，还可以提供各种功能，包括定时，外围接口，电压调节、电源管理以及其他控制功能。而存储器32可以被用于存储处理器31在执行操作时所使用的数据。
本申请实施例还提供一种计算机可读存储介质，存储有计算机程序。计算机程序被处理器执行时实现上述方法实施例。
即，本领域技术人员可以理解，实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor) 执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
本领域的普通技术人员可以理解，上述各实施方式是实现本申请的具体实施例，而在实际应用中，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (11)

  1. 一种告警因果关系挖掘方法，包括：
    基于系统告警信息和所述系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境；
    通过所述深度强化学习的智能体与所述系统告警环境的交互，学习并生成表征告警因果关系和结构的告警因果模型。
  2. 根据权利要求1所述的告警因果关系挖掘方法，其中，所述基于所述系统告警信息和所述系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境，包括：
    获取指示所述系统告警信息和所述系统告警信息的根因标签数据的系统告警图；所述系统告警图包括系统的各节点的告警类别索引、告警特征向量和根因标记；
    根据所述系统告警图，搭建所述系统告警环境。
  3. 根据权利要求2所述的告警因果关系挖掘方法，其中，所述通过所述深度强化学习的智能体与所述系统告警环境交互，学习并生成表征告警因果关系和结构的告警因果模型，包括：
    所述智能体根据所述系统告警环境当前的环境状态选择挖掘动作；
    所述系统告警环境根据所述挖掘动作、所述根因标记、状态转移机制和奖励机制，反馈奖励值和继承状态至所述智能体；其中，所述挖掘动作的索引与所述告警类别索引对应；
    所述智能体根据所述挖掘动作和所述奖励值，学习并生成表征告警因果关系和结构的告警因果模型。
  4. 根据权利要求3所述的告警因果关系挖掘方法，其中，所述系统告警环境的环境状态根据当前节点的告警状态确定，所述当前节点的告警状态根据所述当前节点的告警特征向量，以及所述当前节点的邻接节点的告警特征向量生成。
  5. 根据权利要求3所述的告警因果关系挖掘方法，其中，所述状态转移机制包括：
    在所述挖掘动作属于当前节点的局部动作空间的情况下，将所述当前节点 的含有与所述挖掘动作的索引对应的告警类别索引的邻接节点作为继承节点；将所述继承节点的告警状态作为所述系统告警环境的继承状态；
    在所述挖掘动作不属于所述当前节点的局部动作空间的情况下，将所述系统告警环境的继承状态设置为指定状态。
  6. 根据权利要求3所述的告警因果关系挖掘方法，其中，所述奖励机制包括：根据所述挖掘动作是否属于当前节点的局部动作空间，以及执行所述挖掘动作后，所述系统告警环境的继承节点的类型，确定所述奖励值。
  7. 根据权利要求3至6中任一项所述的告警因果关系挖掘方法，其中，在所述智能体与所述系统告警环境交互过程中，还包括：
    所述系统告警环境在执行完所述智能体下发的挖掘动作后，在满足交互终止条件的情况下，触发环境状态复位机制。
  8. 根据权利要求7所述的告警因果关系挖掘方法，其中，所述环境状态复位机制包括：
    在基于所述系统告警信息和所述系统告警信息的根因标签数据生成的系统告警图集合中，随机选取一个系统告警图，作为所述系统告警环境的系统告警图；
    从所述系统告警环境的系统告警图中，选择一个出度或入度不为0的节点作为起始节点；
    将所述起始节点的告警状态作为所述系统告警环境的环境状态，反馈给所述智能体，以便所述智能体与所述系统告警环境重新进行交互。
  9. 一种告警因果挖掘装置，包括：基于深度强化学习的智能体模块和系统告警环境模块；
    所述系统告警环境模块基于系统告警信息和系统告警信息的根因标签数据，搭建深度强化学习的系统告警环境；
    所述智能体模块与所述系统告警环境模块交互，学习并生成表征告警因果关系和结构的告警因果模型。
  10. 一种告警因果挖掘装置，包括：至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使 所述至少一个处理器能够执行如权利要求1至8中任一项所述的告警因果关系挖掘方法。
  11. 一种计算机可读存储介质，存储有计算机程序，所述计算机程序被处理器执行时，实现如权利要求1至8中任一项所述的告警因果关系挖掘方法。
PCT/CN2022/098772 2021-09-18 2022-06-14 告警因果关系挖掘方法、告警因果挖掘装置及存储介质 WO2023040381A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111113557.2 2021-09-18
CN202111113557.2A CN113901126A (zh) 2021-09-18 2021-09-18 告警因果关系挖掘方法、告警因果挖掘装置及存储介质

Publications (1)

Publication Number Publication Date
WO2023040381A1 true WO2023040381A1 (zh) 2023-03-23

Family

ID=79028951

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/098772 WO2023040381A1 (zh) 2021-09-18 2022-06-14 告警因果关系挖掘方法、告警因果挖掘装置及存储介质

Country Status (2)

Country Link
CN (1) CN113901126A (zh)
WO (1) WO2023040381A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116257363A (zh) * 2023-05-12 2023-06-13 中国科学技术大学先进技术研究院 资源调度方法、装置、设备及存储介质
CN116991683A (zh) * 2023-08-03 2023-11-03 北京优特捷信息技术有限公司 一种告警信息处理方法、装置、设备及介质
CN117170997A (zh) * 2023-11-03 2023-12-05 中保车服科技服务股份有限公司 用于告警信息的关联方法、系统、存储介质及设备
CN117709806A (zh) * 2024-02-05 2024-03-15 慧新全智工业互联科技(青岛)有限公司 协同工作的多设备异常自动化检测方法及检测系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113901126A (zh) * 2021-09-18 2022-01-07 中兴通讯股份有限公司 告警因果关系挖掘方法、告警因果挖掘装置及存储介质
CN114637649A (zh) * 2022-03-01 2022-06-17 清华大学 一种基于oltp数据库系统的告警根因分析方法及装置
CN114666204B (zh) * 2022-04-22 2024-04-16 广东工业大学 一种基于因果强化学习的故障根因定位方法及系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112003718A (zh) * 2020-09-25 2020-11-27 南京邮电大学 一种基于深度学习的网络告警定位方法
CN112100392A (zh) * 2020-07-28 2020-12-18 浙江大学 一种基于强化学习和知识图谱的设备故障溯因方法
CN112181758A (zh) * 2020-08-19 2021-01-05 南京邮电大学 一种基于网络拓扑及实时告警的故障根因定位方法
CN113220946A (zh) * 2021-05-25 2021-08-06 平安付科技服务有限公司 基于强化学习的故障链路搜索方法、装置、设备及介质
CN113240139A (zh) * 2021-06-03 2021-08-10 南京中兴新软件有限责任公司 告警因果评估方法、故障根因定位方法及电子设备
WO2021179643A1 (zh) * 2020-03-12 2021-09-16 华为技术有限公司 故障处理的方法、装置以及系统
CN113901126A (zh) * 2021-09-18 2022-01-07 中兴通讯股份有限公司 告警因果关系挖掘方法、告警因果挖掘装置及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021179643A1 (zh) * 2020-03-12 2021-09-16 华为技术有限公司 故障处理的方法、装置以及系统
CN112100392A (zh) * 2020-07-28 2020-12-18 浙江大学 一种基于强化学习和知识图谱的设备故障溯因方法
CN112181758A (zh) * 2020-08-19 2021-01-05 南京邮电大学 一种基于网络拓扑及实时告警的故障根因定位方法
CN112003718A (zh) * 2020-09-25 2020-11-27 南京邮电大学 一种基于深度学习的网络告警定位方法
CN113220946A (zh) * 2021-05-25 2021-08-06 平安付科技服务有限公司 基于强化学习的故障链路搜索方法、装置、设备及介质
CN113240139A (zh) * 2021-06-03 2021-08-10 南京中兴新软件有限责任公司 告警因果评估方法、故障根因定位方法及电子设备
CN113901126A (zh) * 2021-09-18 2022-01-07 中兴通讯股份有限公司 告警因果关系挖掘方法、告警因果挖掘装置及存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116257363A (zh) * 2023-05-12 2023-06-13 中国科学技术大学先进技术研究院 资源调度方法、装置、设备及存储介质
CN116257363B (zh) * 2023-05-12 2023-07-25 中国科学技术大学先进技术研究院 资源调度方法、装置、设备及存储介质
CN116991683A (zh) * 2023-08-03 2023-11-03 北京优特捷信息技术有限公司 一种告警信息处理方法、装置、设备及介质
CN116991683B (zh) * 2023-08-03 2024-01-30 北京优特捷信息技术有限公司 一种告警信息处理方法、装置、设备及介质
CN117170997A (zh) * 2023-11-03 2023-12-05 中保车服科技服务股份有限公司 用于告警信息的关联方法、系统、存储介质及设备
CN117709806A (zh) * 2024-02-05 2024-03-15 慧新全智工业互联科技(青岛)有限公司 协同工作的多设备异常自动化检测方法及检测系统
CN117709806B (zh) * 2024-02-05 2024-05-28 慧新全智工业互联科技(青岛)有限公司 协同工作的多设备异常自动化检测方法及检测系统

Also Published As

Publication number Publication date
CN113901126A (zh) 2022-01-07

Similar Documents

Publication Publication Date Title
WO2023040381A1 (zh) 告警因果关系挖掘方法、告警因果挖掘装置及存储介质
Neider et al. Learning linear temporal properties
Palomares et al. Consensus under a fuzzy context: Taxonomy, analysis framework AFRYCA and experimental case of study
Ripley et al. Manual for RSIENA
CN114090902B (zh) 一种基于异质网络的社交网络影响力预测方法、装置
WO2019185039A1 (zh) 数据处理方法和电子设备
CN110929047A (zh) 关注邻居实体的知识图谱推理方法和装置
WO2022022233A1 (zh) Ai模型更新的方法、装置、计算设备和存储介质
Rkhami et al. Learn to improve: A novel deep reinforcement learning approach for beyond 5G network slicing
JPWO2018083804A1 (ja) 分析プログラム、情報処理装置および分析方法
Gu et al. Learning big Gaussian Bayesian networks: Partition, estimation and fusion
Huang et al. Generating programmatic referring expressions via program synthesis
Huang et al. Partitioned hybrid learning of Bayesian network structures
US11487604B2 (en) Cognitive service request construction
CN116151384B (zh) 量子电路处理方法、装置及电子设备
CN114900435B (zh) 一种连接关系预测方法及相关设备
Shamsinejadbabaki et al. Causality-based cost-effective action mining
Trabelsi et al. Benchmarking dynamic Bayesian network structure learning algorithms
CN113342500B (zh) 任务执行方法、装置、设备及存储介质
CN115599990A (zh) 一种知识感知结合深度强化学习的跨域推荐方法及系统
WO2023091144A1 (en) Forecasting future events from current events detected by an event detection engine using a causal inference engine
CN114898184A (zh) 模型训练方法、数据处理方法、装置及电子设备
KR20080087571A (ko) 상황 예측 시스템 및 그 방법
CN111581812B (zh) 一种目标驱动、基于遗传算法的自适应系统规划生成方法
US11775265B2 (en) Method and system for library package management

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22868758

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2401001330

Country of ref document: TH

WWE Wipo information: entry into national phase

Ref document number: 2022868758

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022868758

Country of ref document: EP

Effective date: 20240320

NENP Non-entry into the national phase

Ref country code: DE