WO2018127304A1

WO2018127304A1 - Apparatus and method for network incident troubleshooting

Info

Publication number: WO2018127304A1
Application number: PCT/EP2017/051686
Authority: WO
Inventors: MingXue Wang; Vincent Huang
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2017-01-04
Filing date: 2017-01-26
Publication date: 2018-07-12
Anticipated expiration: 2019-07-04

Abstract

A method of providing network incident troubleshooting comprises collecting document data and building a signature database of stored signatures from terms in the document data, wherein the stored signatures comprise: a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or resolution for an incident. The method comprises building a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data, wherein the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document. The method comprises receiving a set of incident signatures, ic, and calculating knowledge recommendation scores, R_k (ac|ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph. The method comprises recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score.

Description

APPARATUS AND METHOD FOR NETWORK INCIDENT

TROUBLESHOOTING

Technical Field

The embodiments described herein relate to a method and apparatus for providing network incident troubleshooting, in particular, by utilising data collection.

Background

When a problem occurs in a mobile network, a support engineer needs to collect any system logs and analyse the information from the log in order to locate the source of the problem. In this current fault and incident management system, it can take a long time for the support engineer to locate the source of the problem and take any actions accordingly.

In this example, it is up to each individual engineer to master the knowledge surrounding each individual problem, and the resolution management is completely manual. There have been some existing proposals to automate the incident troubleshooting and recommendation process. For example, by using a Bayesian network approach to model the causes of the faults and their associated symptoms. However, this still requires a human expert to build a core knowledge base from which to determine the nature of the faults and their symptoms or causes.

Summary

According to one aspect, there is provided a method of providing network incident troubleshooting. The method comprises collecting document data, and building a signature database of stored signatures from terms in the document data. The stored signatures comprise a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or resolution for an incident. The method comprises building a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data. The conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document. The method comprises receiving a set of incident signatures, ic, and calculating knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph. The method comprises recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score.

According to another aspect, there is provided an apparatus for providing network incident troubleshooting. The apparatus comprises a processor and a memory, said memory containing instructions executable by said processor. The apparatus is operative to collect document data and build a signature database of stored signatures from terms in the document data, wherein the stored signatures comprise a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or resolution for an incident. The apparatus is operative to build a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data, wherein the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document. The apparatus is operative to receive a set of incident signatures, ic, and calculate knowledge recommendation scores, R_k (ac \ ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph. The apparatus is operative to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic) . According to another aspect, there is provided an apparatus for providing network incident troubleshooting. The apparatus comprises a collection module to collect document data, and a database module to build a signature database of stored signatures from terms in the document data, wherein the stored signatures comprise a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or resolution for an incident. The apparatus comprises a knowledge graph building module to build a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data, wherein the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document. The apparatus comprises a receiving module to receive a set of incident signatures, ic, and a calculation module to calculate knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph. The apparatus comprises a recommendation module to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score. Brief Description of the Drawings

Figure 1 illustrates a network system 100 according to some embodiments;

Figure 2 illustrates a flow chart of a method for network incident troubleshooting according to some embodiments; Figure 3 illustrates an example of documents containing signatures;

Figure 4 illustrates an example of a knowledge graph according to some embodiments;

Figure 5 illustrates a method according to some embodiments which incorporate user feedback information;

Figure 6 illustrates an apparatus 600 for providing network incident troubleshooting according to some embodiments; and

Figure 7 illustrates an apparatus 700 for providing network incident troubleshooting according to some embodiments. Description

The following sets forth specific details, such as particular embodiments for purposes of explanation and not limitation. But it will be appreciated by one skilled in the art that other embodiments may be employed apart from these specific details. In some instances, detailed descriptions of well known methods, nodes, interfaces, circuits, and devices are omitted so as not obscure the description with unnecessary detail.

The present embodiments relate to a method for providing network incident troubleshooting by extracting information from historical documents to build a knowledge graph from which to provide, for example, recommendations for root causes and potential resolutions for a particular incident. This makes use of previous knowledge efficiently, and automates the troubleshooting process. Figure 1 illustrates a network system 100 according to embodiments of the invention. An incident may be represented by a group of network events which are closely related, or related to the same network problem or root cause. An incident may therefore comprise a number of network alarms, Key Performance Indicators (KPI) anomalies, Configuration Management (CM) changes, etc. which have close relations, i.e. those network events or alarms that belong to a single network incident.

A community knowledge collector and extractor (CKCE) 101 collects document data from a variety of sources. The document data sources may include for example one or more of: online forums 102; engineer experience reports 103; product documents 104; troubleshooting guides 105; and user feedback 106. It will be appreciated that many other types of data may be utilized as the document data, according to the invention.

The CKCE 101 may collect free text documents from various data sources as a part of the process of collecting knowledge from different communities. These collected free text documents may be used as raw data to build structured knowledge graphs and signature databases. Text mining techniques can be used to convert the unstructured free text documents into structured data. The structured data, such as knowledge graphs can then be used as input for computing recommendations.

An expert knowledge base 107 can be used to build a signature database from the collected document data.

Useful words or phrases in the collected document data can be obtained based on the relationship between the frequency of a particular term in the document data, and the number of documents in which that particular term occurs. The expert knowledge base 107 can then be used to refine this collection of terms in order to provide a signature database of stored signatures, where the stored signatures are terms relevant to network incidents, the causes of such network incidents and/or resolutions for the incidents. For example, the term "radio" may occur at a high frequency in a high number of the collected documents. However, the expert knowledge base 107 may be used to specify that this term is only relevant to a network incident or incident cause when grouped with other terms, for example "radio base station". The signature database in this example would therefore only store the signature "radio base station" for further use, and not the isolated term "radio".

The resulting signature database will comprise stored signatures which include a plurality of incident signatures, each representing a characteristic of an incident, for example "power failure", and a plurality of assistance signatures, each representing a root cause of or resolution/recommendation for an incident, for example, "Power cable fault" (a root cause signature). In some cases, the signature database will also comprise resolution or recommendation signatures which are signatures representing how to overcome network incidents. For simplicity we will refer to root cause signatures and recommendation/resolution signatures as both being types of assistance signatures, ac.

Returning to Figure 1 , the signature database can then be utilised to build a knowledge graph in the Knowledge Graph Builder 108. This may comprise calculating conditional probabilities, between pairs of the stored signatures, in the document data. The conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document.

In response to a network incident, a recommender 109 can receive context information, which may be supplied for example by a user, comprising a set of incident signatures, ic. The context information may be, for example, a written query by a service engineer. Such a query may contain a number of terms, some of which will contain useful information, and some which will not. The signature database can be used to extract from the context information the set of incident signatures, ic, corresponding to the useful terms, which describe the characteristics of the incident.

The context information may comprise incident signatures of one or more of the following types: alarm information, Key Performance Indicators (KPI) anomalies, network configuration commands, and other incident related information.

The recommender 109 will then calculate knowledge recommendation scores, R_k (ac \ ic), which indicate the likelihood that the set of incident signatures, ic, represents respective different assistance signatures, ac, in the knowledge graph.

The recommender 109 may then recommend an assistance signature, ac, from the different root causes or recommendation/resolution signatures based on the knowledge recommendation score. In some cases, this recommended assistance signature, ac, will be output to a user or a help desk. In embodiments where the output assistance signature is a root cause signature, it may be output along with recommended solutions associated with the root cause.

In some embodiments, a plurality of assistance signatures, i.e. all or some of the possible assistance signatures, are recommended to the user along with their respective knowledge recommendation scores, or in some cases a total recommendation score as will be described below with reference to Figure 4. The user may then determine from the scores, and his/her own experience, which course of action to take.

Figure 2 illustrates an example of a flow chart of a method for network incident troubleshooting, according to an embodiment.

In step 201 , the method comprises collecting document data. As described above the document data sources may include one or more of: online forums 102; engineer experience reports 103; product documents 104; troubleshooting guides 105; and user feedback 106. It will be appreciated that many other types of data may be utilized as the document data according to the invention.

In step 202, the method comprises building a signature database of stored signatures from terms in the document data. The stored signatures may comprise a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or a resolution for, an incident.

With regard to the step 202 of building a signature database, Figure 3 illustrates an example free text document, highlighting the potential signatures.

The free text document 300 contains a plurality of signatures, for example the alarm incident signatures "Power Failure" 301 and "false power input" 302. An example of a network element incident signature is "MINI-LINK TN" 303. There is also an example root cause signature, "a big unexpected jump in FSINFO on several Basic Frames in Radio Unit after power cycle" 304.

An example of a signature database which may be produced from document data which includes the document 300 is shown in Table 1 below.

Table 1

As can be seen from the above table, each signature is associated with a signature type, and each signature produced is relevant to an incident, root cause or resolution characteristic of incidents.

As described above, an expert knowledge base may be used to determine which terms in the document data classify as signatures for the purpose of providing incident characteristics and root cause or resolution characteristics.

Returning to the method of Figure 2, in step 203 the method comprises building a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data.

When calculating the conditional probability between pairs of stored signaturesx,_i and x_j in the document data the following method may be used. Firstly the total number of documents in the document data, N, is obtained. Furthermore, the number of documents, d in the data set, D that each signature, x occurs in, \d e D:. x∈ d\ , is obtained. Both of these values can be obtained through analysis of the document data.

It is therefore clear that, p ( x_i), the marginal probability function of a signature, x_i occurring in any one document can be given as:

The joint probability function, p ( x_i. xj), i.e. the probability that bothx_i and x_j occur in any one document can be given by:

where is the number of documents containing both the

signatures x_i and χ, .

Therefore, by using for example Bayes' Theorem, for each pair of stored signatures x_i and xj , the Knowledge Graph Builder 108 can compute the conditional probability between the pair of stored signaturexs_i and xj in the document data as:

By applying these equations to the signatures in the signature database a knowledge graph can be produced.

Figure 4 illustrates an example of a knowledge graph according to embodiments of the invention. This knowledge graph is produced from the example signatures shown in Table 1 . Each signature, both incident signatures and assistance signatures, are represented by a vertex, 401 , in the graph.

The weights, w,, of the edges 402 between the vertices represent the conditional probability between the signatures, as calculated above. If the conditional probability between two signatures is zero, then there is no edge between the two signatures. In some embodiments, the knowledge graph is directed. In these cases, a weight, wi, of an edge, from x_i to x_j in the knowledge graph may

be calculated as:

In some embodiments the knowledge graph is undirected. In these cases, a weight, of an edge, E (x_t - x_j) betweenx_i and x_j in the knowledge graph

is calculated as:

the prior probability function and, the joint probability function

are calculated as described previously.

Returning to Figure 2, in step 204, the method comprises receiving a set of incident signatures. As described above, the incident signatures may be received in context information, which may be for example a written query by a service engineer. Such a query may contain a number of terms, some of which will contain useful information, and some which will not. A signature database may be used to extract from the context information the set of incident signatures, ic, corresponding to the useful terms, which describe the characteristics of the incident.

The set of incident signatures may comprise incident signatures of one or more of the following types: alarm information, Key Performance Indicators (KPI) anomalies, network configuration commands, and other incident related information. In step 205, the method comprises calculating knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph.

One implementation of the method is to calculate the knowledge recommendation scores as described below. However, it will be appreciated that other methods could be used, for example a Markov network. In this example, the knowledge recommendation score, R_k (ac |ic), for a particular assistance signature ac given the set of incident signatures ic = can be calculated as:

R_k (ac\ic) is the probability of the assistance signature, ac, to the set of incident signatures, ic, and G_k (ac\ic_i) is the largest conditional probability between ac and a specific incident signature ic_i considering all paths connecting the signature ac and ic_i in the knowledge graph.

The value of G_k (ac\ic_i) is used because in some cases there may not be a direct link between two signatures, however, there may be an indirect link via other signatures in the knowledge graph. Therefore, a shortest path algorithm can be used to calculate or estimate the greatest conditional probability between any two signatures. Therefore G_k (ac\ic_i) can be calculated as:

where (7)

PA are all available paths between a pair of signatures ac and

ic_i in the knowledge graph, and are weights of the edges of

each path pa between the signatures ac and ic_i in the knowledge graph. In step 206, the method comprises recommending an assistance signature, ac, from the different assistance signature based on the knowledge recommendation score R_k (ac\ic) . The assistance signature in the knowledge graph which has the highest knowledge recommendation score may be recommended. The assistance signature ac, may be recommended with the knowledge recommendation score, R_k (ac\ic) .

In other embodiments, a plurality of root causes may be recommended, and they may be ranked by knowledge recommendation score.

For example, the context information may contain two alarms (power failure and ima link reception unusable at far end) and one network element (base receiver station) signatures. Then, based on the knowledge graph of Figure 4, there are two potential root cause signatures which could be recommended (power cable fault or unexpected jump in FSINFO on several Basic Frames in Radio Unit after power cycle) based on the incident signatures, 401f, 401 a and 401 b, obtained from the context information. By using the above method the recommendation score for each root cause signature can be calculated. The resulting scores can be output to a user as shown in Table 2.

Table 2: A table showing an example of ranked root causes and recommendation scores.

In some embodiments, the step 206 of recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), comprises utilizing user feedback information. In these embodiments, the assistance signature, ac, may instead be recommended alongside a total recommendation score, R (ac\ic), for the assistance signature, ac, as will be described below.

Figure 5 illustrates a method according to embodiments of the invention incorporating user feedback information. In step 501 , the method comprises collecting documents, for example similarly to step 201 in Figure 2.

In step 502, the method comprises building a signature database of stored signatures, for example similarly to step 202 in Figure 2.

In step 503, the method comprises calculating conditional probabilities between pairs of the stored signatures in the document data.

In step 504, the method comprises building a knowledge graph as described above. A knowledge graph recommendation score, R_k (ac\ic), can be calculated from this knowledge graph as described above.

In step 505, the method comprises collecting user feedback information. In some embodiments, the user feedback information comprises a user rating, r_j, for example of 1 to 5 stars which represents a 0 to 1 probability the user associates with a relationship between an incident signature ic_i and an output assistance signature ac. It will be appreciated that other forms of user feedback may be used. In step 506, the method comprises calculating a user conditional probability, p_u (ac\ic_i) between the pair of signatures ic_i and ac. In this embodiment the user conditional probabilities are calculated as:

n is an integer, and N(ic) is the total number of incident signatures in the context information.

In step 507, a user feedback knowledge graph is built based on the user conditional probabilities. This user feedback knowledge graph can be built using the user conditional probabilities in the same manner that the knowledge graph is built from the conditional probabilities p (ac\ic_i) . In other words, the user feedback knowledge graph may be directed. Therefore, in this embodiment, the user feedback knowledge graph may be directed, and a weight, wi, of an edge, E (ac→ ic_i), from ac to ic_i in the user feedback knowledge graph is calculated as:

Alternatively, the user feedback knowledge graph may be undirected, in this case the weight of an edge E(ac - ic_i), from ac to ic_i in the user feedback knowledge graph is calculated as:

A user recommendation score, R_u (ac\ic), is calculated from this user feedback knowledge graph in the same manner as the knowledge recommendation score is calculated from the knowledge graph. It will be appreciated that other methods of calculating a user recommendation score, R_u (ac\ic), may be used and, in particular, different methods may be used to calculate the user recommendation score and the knowledge recommendation score.

In this embodiment, the user recommendation score, R_u (ac\ic) can be calculated as:

R_u (ac\ic) is the probability of the root cause, ac, to the set of incident signatures, ic, and G_u (ac\ic_i) is the largest user conditional probability between ac and a specific incident signature ic_i of all paths connecting the signature ac and ic_i in the user feedback knowledge graph.

The value of G_u (ac\ic_i) is used because in some cases there may not be a direct link between two signatures, however, there may be an indirect link via other signatures in the graph. Therefore, a shortest path algorithm can be used to calculate or estimate the greatest user conditional probability between any two signatures.

Therefore G_u (ac\ic_i) can be calculated as:

are all available paths between a pair of signatures rc and

ic_i in the user knowledge graph, and

are weights of the edges of each path pa between the signatures ac and ic_i in the user feedback knowledge graph.

Then in step 508, total recommendation scores, R(ac\ic) are calculated. In this embodiment, recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), comprises recommending the assistance signature, ac, with a total recommendation score, R (ac\ic), wherein R(ac\ic) is the average of the respective knowledge recommendation score, R_k (ac\ic), and the user recommendation score, R_u (ac\ic) . In step 509 a recommendation list of assistance signatures for an incident can be built based on the calculated total recommendation scores, R(ac\ic) . The assistance signatures may be ranked by total recommendation score,

R(ac\ic) .

In step 510 the ranked list of assistance signatures, may be output to a user or helpdesk as a list or a graph.

Figure 6 illustrates an apparatus 600 for providing network incident troubleshooting according to an embodiment. The apparatus 600 comprises a processor 602 and a memory 601 , said memory 601 containing instructions executable by said processor 602. The apparatus 600 is operative to collect document data; build a signature database of stored signatures from terms in the document data, and build a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data. The stored signatures comprise; a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or resolution for an incident. The apparatus 600 is also operative to receive a set of incident signatures, ic; calculate knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph; and recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k(ac\ic) .

The set of incident signatures, ic, may comprise one or more of: alarm information, Key Performance Indicators (KPI) anomalies, network configuration commands, and other incident related information. The apparatus 600 may be operative such that the document data is extracted from a plurality of data sources comprising one or more of: online forums; engineer experience reports; product documents and user feedback. The apparatus 600 may be operative to use an expert knowledge base to build the signature database from the collected document data.

The apparatus 600 may be operative to calculate the conditional probabilities between pairs of stored signatures; for each pair of stored signaturesx_i and x_j, compute the conditional probability between the pair of stored signatures wherein p(x_i) is a marginal probability function

of a signature, x_i in the document data and: and wherein

a joint probability function of the signatures, x_i and x_j, and

where

is the number of documents containing the signature x_i N is the total number of documents, and

is the number of documents containing both the signatures x_i and x_j .

The knowledge graph may be directed. In such an example, the apparatus 600 may be operative such that a weight, wi, of an edge, from x_i to x_j

in the knowledge graph is calculated as

In another example the knowledge graph may be undirected. In such an example the apparatus 600 may be operative such that a weight, w of an edge,

between x_i and x_j in the knowledge graph is calculated as:

The apparatus 600 may be operative to calculate knowledge recommendation scores, R_k(ac\ic), indicative that the set of incident signatures, ic, represent respective assistance signatures, ac, in the knowledge graph the processor is further configured to: for each different assistance signature, ac, calculate the knowledge recommendation score, where

_k is the probability of the assistance signature, ac, to the set of incident signatures, ic, and G_k (ac\ic_i) is the greatest conditional probability that ac is linked to a particular context signature ic_i by all paths connecting the signature ac and ic_i in the knowledge graph.

In one example, the greatest conditional probability, G_k (ac\ic_i) ,

is calculated by; where

are all available paths between a pair of signatures ac and

ic_i in the knowledge graph, and are weights of edges of a

each path pa between the signatures ac and ic_i in the knowledge graph. In one example, the apparatus 600 is operative such that, to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), the apparatus 600 is further operative to recommend the assistance signature, ac, with the knowledge recommendation score, R_k (ac\ic) . In one example, the apparatus 600 is operative such that, to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), the processor is further configured to utilize user feedback information. In such an example the user feedback information comprises a user rating, η, for example of 1 to 5 stars which represents a 0 to 1 probability the user associates with a relationship between an incident signature ic_i and an output assistance signature, ac. The apparatus 600 may be further operative to calculate a user conditional probability between the pair of signatures ic_i and ac. as: where

n is an integer, and N(ic) is the total number of incident signatures in the context information, build a user feedback knowledge graph based on the user conditional probabilities; and calculate user recommendation scores,

R_u (ac\ic) . The user feedback knowledge graph may be directed, and a weight, wi, of an edge, E (ac→ ic_i), from ac to ic_i in the user feedback knowledge graph is calculated as

The user feedback knowledge graph may be undirected, and a weight wi, of an edge, E(ac - icj, from ac to ic_i in the user feedback knowledge graph is calculated as E(ac - iq) = 1 - ((l - p (ac| iCj))(l - p (ac| iCj))) .

In one example, to calculate user recommendation scores, R_u (ac\ic), indicative that the set of incident signatures, ic, represent respective assistance signatures, ac, in the user feedback knowledge graph, the apparatus 600 is further operative to: for each different assistance signature, ac, calculate the user recommendation score, R_u (ac\ic), as: R_u (ac\ic) = 1 - where R_u (ac\ic) is the probability of the assistance

signature, ac, to the set of incident signatures, ic, and G_u (ac\ic_i) is the greatest conditional probability that the signature ac is linked to a particular context signature ic_i by all paths connecting the signature ac and ic_i in the user feedback knowledge graph.

In one example the greatest conditional probability, G_u (ac\ic_i),

is calculated by: where

(pa_1, pa₂, ... } = PA are all available paths between a pair of signatures ac and ic_i in the user feedback knowledge graph, and are weights

of the edges of each path pa between the signatures ac and ic_i in the user feedback knowledge graph.

In one example, the apparatus 600 is operative such that, to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), the apparatus 600 is further configured to recommend the assistance signature, ac, alongside a recommendation score, R (ac\ic), for the assistance signature, ac, wherein R(ac\ic) is the average of the respective knowledge recommendation score, R_k (ac\ic), and the user recommendation score, R_u (ac\ic) . In one example the apparatus 600 is further operative to: receive context information, wherein the context information comprises the set of incident signatures, and extract the set of incident signatures from the context information using the signature database. The apparatus 600 also comprises an interface 606. The interface 606 may, for example, be a simple Ethernet network card to connect to a network.

Figure 7 illustrates an apparatus 700 according to another embodiment. The apparatus 700 comprises a collection module 701 to collect document data. A database module 702 is provided to build a signature database of stored signatures from terms in the document data; wherein the stored signatures comprise; a plurality of incident signatures each representing a characteristic of an incident, and a plurality of assistance signatures each representing a root cause of or resolution for an incident.

The apparatus 700 further comprises a knowledge graph building module 703 to build a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data, wherein the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document.

A receiving module 704 is provided to receive a set of incident signatures, ic. In come embodiments the incident signatures are received as context information and the signatures are extracted from the context information using the signature database.

The apparatus 700 further comprises a calculation module 705 to calculate knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph.

The apparatus 700 further comprises a recommendation module 706 to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score.

In some embodiments the the set of incident signatures, ic, in the database module 702 comprises one or more of: alarm information, Key Performance Indicators (KPI) anomalies, network configuration commands, and other incident related information.

In some embodiment the collection module 701 extracts the document data from a plurality of data sources comprising one or more of: online forums; engineer experience reports; product documents and user feedback.

In some embodiments, an expert knowledge base is used to build the signature database, in the database module 702 from the collected document data.

The knowledge building graph module 703 may be configured to calculate the conditional probabilities between pairs of the stored signatures in the document data by; for each pair of stored signatures x_i and x_j, computing the conditional probability between the pair of stored signatures

wherein

is a marginal probability function of a signature, in the document data

and:

and wherein

is a joint probability function of the signatures, and

,

where

is the number of documents containing the signature

is the total number of documents, and

is the number of documents containing both the signatures x_i and x_j .

In some embodiments the knowledge graph built by the knowledge building graph module 703 is directed. In this case a weight, of an edge,

from x_i to x_j in the knowledge graph may be calculated as

In some embodiments the knowledge graph built by the knowledge building graph module 703 is undirected. In this case a weight, of an edge,

between x_i and x_j in the knowledge graph may be calculated as:

The calculation module 705 may calculate the knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective assistance signatures, ac, in the knowledge graph comprises: for each different assistance signature, ac, calculating the knowledge recommendation score, R_k (ac\ic), as:

where

R_k (ac\ic) is the probability of the assistance signature, ac, to the set of incident signatures, ic, and G_k (ac\ic_i) is the greatest conditional probability that ac is linked to a particular context signature ic_i by all paths connecting the signature ac and ic_i in the knowledge graph. The calculation module may calculate the greatest conditional probability,

are all available paths between a pair of signatures ac and ic_i in the knowledge graph, and are weights of edges of a

each path pa between the signatures ac and ic_i in the knowledge graph.

The recommendation module 706 may recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), by recommending the assistance signature, ac, with the knowledge recommendation score, R_k (ac\ic) .

The recommendation module 706 may recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), by utilizing user feedback information.

The user feedback information may a user rating, r_j , representing a probability the user associates with a relationship between an incident signature ic_i and an output assistance signature, ac.

The apparatus 700 may calculate a user conditional probability between the pair of signatures ic_i and ac. as: .

where

n is an integer, and N(ic) is the total number of incident signatures in the context information. A user feedback knowledge graph may be built based on the user conditional probabilities; and user recommendation scores may be calculated, R_u (ac\ic) . The user feedback knowledge graph may be directed, and a weight, wi, of an edge, E (ac→ ic_i), from ac to ic_i in the user feedback knowledge graph may be calculated as

The user feedback knowledge graph may be undirected, and a weight wi, of an edge, E(ac - ic , from ac to ic_i in the user feedback knowledge graph is calculated as

The calculation module 705 may calculate user recommendation scores, R_u (ac\ic), indicative that the set of incident signatures, ic, represent respective assistance signatures, ac, in the user feedback knowledge graph comprises: for each different assistance signature, ac, calculating the user recommendation score, R_u (ac\ic), as:

where

R_u (ac\ic) is the probability of the assistance signature, ac, to the set of incident signatures, ic, and

is the greatest conditional probability that the signature ac is linked

to a particular context signature ic_i by all paths connecting the signature ac and ici in the user feedback knowledge graph.

The calculation module 705 may calculate the greatest conditional probability, as:

where

are all available paths between a pair of signatures

ac and ic_i in the user feedback knowledge graph, and are

weights of the edges of each path pa between the signatures ac and ic_i in the user feedback knowledge graph.

The recommendation module 706 may recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), by recommending the assistance signature, ac, alongside a recommendation score, R(ac\ic) , for the assistance signature, ac, wherein R(ac\ic) is the average of the respective knowledge recommendation score, R_k (ac\ic), and the user recommendation score, R_u (ac\ic) .

The receiving module 703 may receive context information, wherein the context information comprises the set of incident signatures, and may extract the set of incident signatures from the context information using the signature database. The apparatus 700 also comprises an interface 707. The interface 707 may, for example, be a simple Ethernet network card to connect to a network.

From the embodiments described herein, there is therefore provided a method and apparatus for network incident troubleshooting which provides for fast incident resolution with recommendations which automatically provides root causes and recommendations for how to resolve particular network incidents without requiring any manual troubleshooting. It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim, "a" or "an" does not exclude a plurality, and a single processor or other unit may fulfil the functions of several units recited in the claims. Any reference signs in the claims shall not be construed so as to limit their scope.

Claims

1 . A method of providing network incident troubleshooting, the method comprising:

collecting document data;

building a signature database of stored signatures from terms in the document data; wherein the stored signatures comprise;

a plurality of incident signatures each representing a characteristic of an incident, and

a plurality of assistance signatures each representing a root cause of or resolution for an incident;

building a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data, wherein the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document;

receiving a set of incident signatures, ic;

calculating knowledge recommendation scores, R_k(ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph; and

recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score.

2. A method as claimed in claim 1 wherein the set of incident signatures, ic, comprises one or more of: alarm information, Key Performance Indicators, KPI, anomalies, network configuration commands.

3. A method as claimed in claim 1 or 2, wherein the document data is extracted from a plurality of data sources comprising one or more of: online forums; engineer experience reports; product documents and user feedback.

4. A method as claimed in any preceding claim wherein an expert knowledge base is used to build the signature database from the collected document data.

5. A method as claimed in any preceding claim wherein the step of calculating conditional probabilities between pairs of stored signatures comprises;

for each pair of stored signatures x_i and x_j,

computing the conditional probability between the pair of stored signaturesx_i and X_j-as: wherein

is a marginal probability function of a signature, x_i in the document

data and:

and wherein

is a joint probability function of the signatures, x_i and x_j, and

where

is the number of documents containing the signature x_i, N is the total number of documents, and is the

number of documents containing both the signatures x_i and x_j .

6. A method as claimed in any preceding claim wherein the knowledge graph is directed.

7. A method as claimed in claim 6 when dependent on claim 5, wherein a weight, wi, of an edge, from x_i to x_j in the knowledge graph is

calculated as

8. A method as claimed in any one of claims 1 to 5 wherein the knowledge graph is undirected.

9. A method as claimed in claim 8 when dependent on claim 5, wherein a weight, of an edge, betweenx_i and x_j in the knowledge graph

is calculated as:

10. A method as claimed in any one of claims 5 to 9, wherein the step of calculating knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective assistance signatures, ac, in the knowledge graph comprises: for each different assistance signature, ac, calculating the knowledge recommendation score, as:

where

is the probability of the assistance signature, ac, to the set of

incident signatures, ic, and

is the greatest conditional probability that ac is linked to a particular context signature ic_i by all paths connecting the signature ac and ici in the knowledge graph.

1 1 . A method as claimed in claim 10 wherein, the greatest conditional probability, G_k (ac\ic_i), is calculated by:

where

are all available paths between a pair of signatures

ac and ic_i in the knowledge graph, and are weights of edges

of each path pa between the signatures ac and ic_i in the knowledge graph.

12. A method as claimed in claim 1 1 wherein the step of recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), comprises recommending the assistance signature, ac, alongside the knowledge recommendation score, R_k (ac\ic) for the recommended assistance signature, ac.

13. A method as claimed in any one of claims 1 to 1 1 w, herein the step of recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), comprises utilizing user feedback information .

14. A method as claimed in claim 1 3 wherein the user feedback information comprises a user rating, r_j, representing a probability the user associates with a relationship between an incident signature ic_i and an output assistance signature, ac.

15. A method as claimed in claim 14 further comprising:

calculating a user conditional probability between the pair of signatures ic_i and ac as: where

n is an integer, and N(ic) is the total number of incident signatures in the context information,

building a user feedback knowledge graph based on the user conditional probabilities; and

calculating user recommendation scores, R_u (ac\ic) .

16. A method as claimed in claim 15 wherein the user feedback knowledge graph is directed, and a weight, wi, of an edge, E (ac→ ic_i), from ac to ic_i in the user feedback knowledge graph is calculated as E (ac→ ic_i) = p_u (ac\ ic_i) .

17. A method as claimed in claim 15 wherein, the user feedback knowledge graph is undirected, and a weight wi, of an edge, E(ac - ic_i), from ac to ic_i in the user feedback knowledge graph is calculated as

18. A method as claimed in one of claims 16 or 17 wherein the step of calculating user recommendation scores, R_u (ac\ic), indicative that the set of incident signatures, ic, represent respective assistance signatures, ac, in the user feedback knowledge graph comprises: for each different assistance

signature, ac, calculating the user recommendation score, as:

where

is the probability of the assistance signature, ac, to the set of incident signatures, ic, and

is the greatest conditional probability that the signature ac is linked to a particular context signature ic_i by all paths connecting the signature ac and ic_i in the user feedback knowledge graph.

19. A method as claimed in claim 18 wherein, the greatest conditional probability, G_u (ac\ic_i) , is calculated by: where

are all available paths between a pair of signatures

ac and ic_i in the user feedback knowledge graph, and are

20. A method as claimed in any one of claims 15 to 19, wherein the step of recommending an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic), comprises recommending the assistance signature, ac, alongside a recommendation score, R (ac\ic), for the assistance signature, ac, wherein R(ac\ic) is the average of the respective knowledge recommendation score, R_k (ac\ic), and the user recommendation score, R_u (ac\ic) .

21 . A method as claimed in any preceding claim further comprising;

receiving context information, wherein

the context information comprises the set of incident signatures, and

extracting the set of incident signatures from the context information using the signature database.

22. An apparatus for providing network incident troubleshooting, comprising a processor and a memory said memory containing instructions executable by said processor, wherein said apparatus is operative to:

collect document data;

build a signature database of stored signatures from terms in the document data; wherein the stored signatures comprise;

build a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data, wherein

the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document; receive a set of incident signatures, ic;

calculate knowledge recommendation scores, R_k(ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph; and

recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score, R_k (ac\ic) .

23. An apparatus as claimed in claim 22, wherein the apparatus is operative to perform the method as described in any one of claims 2 to 20.

24. An apparatus for providing network incident troubleshooting, the apparatus comprising:

a collection module to collect document data;

a database module to build a signature database of stored signatures from terms in the document data; wherein the stored signatures comprise;

a knowledge graph building module to build a knowledge graph by calculating conditional probabilities between pairs of the stored signatures in the document data,

wherein the conditional probabilities are the probabilities that a one of a pair of stored signatures will be in a document given that the other of the pair of stored signatures is in the document;

a receiving module to receive a set of incident signatures, ic; a calculation module to calculate knowledge recommendation scores, R_k (ac\ic), indicative that the set of incident signatures, ic, represent respective different assistance signatures, ac, in the knowledge graph; and a recommendation module to recommend an assistance signature, ac, from the different assistance signatures based on the knowledge recommendation score.