WO2013034448A1 - Method and system for optimizing and streamlining troubleshooting - Google Patents
Method and system for optimizing and streamlining troubleshooting Download PDFInfo
- Publication number
- WO2013034448A1 WO2013034448A1 PCT/EP2012/066423 EP2012066423W WO2013034448A1 WO 2013034448 A1 WO2013034448 A1 WO 2013034448A1 EP 2012066423 W EP2012066423 W EP 2012066423W WO 2013034448 A1 WO2013034448 A1 WO 2013034448A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- tickets
- management system
- ticket management
- predictive models
- trouble
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013024 troubleshooting Methods 0.000 title claims description 8
- 230000009471 action Effects 0.000 claims abstract description 47
- 238000007726 management method Methods 0.000 claims description 29
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 238000012360 testing method Methods 0.000 claims description 15
- 230000007246 mechanism Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 11
- 238000012937 correction Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 4
- 238000010200 validation analysis Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 claims description 3
- 238000012706 support-vector machine Methods 0.000 claims description 2
- 230000007257 malfunction Effects 0.000 description 14
- 238000007781 pre-processing Methods 0.000 description 7
- 238000009434 installation Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000003014 reinforcing effect Effects 0.000 description 5
- 230000006978 adaptation Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 206010003830 Automatism Diseases 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 239000012636 effector Substances 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 241000196324 Embryophyta Species 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 208000032368 Device malfunction Diseases 0.000 description 1
- MWRWFPQBGSZWNV-UHFFFAOYSA-N Dinitrosopentamethylenetetramine Chemical compound C1N2CN(N=O)CN1CN(N=O)C2 MWRWFPQBGSZWNV-UHFFFAOYSA-N 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
-
- G06Q50/60—
Definitions
- the present invention relates to a method for optimizing and streamlining troubleshooting, wherein said trouble is indicated in tickets generated by a ticket management system, and more specifically to a method which comprises, by means of using a module parallel to said ticket management system, collecting said tickets, solving the type of trouble in real time by means of predictive models built at least from historical data stored in said ticket management system and performing at least one of the following actions: launching an action suggestion, notifying the ticket management system of the action to be performed and/or automatically launching a task depending on a prediction obtained by means of said predictive models.
- a second aspect of the invention relates to a system provided for implementing the method of the first aspect.
- Trouble ticketing systems are used to improve efficiency of the management of these processes, and they are basically responsible for managing tickets from the time they are created until they can be solved and closed. Tickets are generally created manually by the operator or automatically by means of automatic alarm detection systems. Once the ticket is created, the technician is responsible for analyzing and diagnosing the problem, making a decision concerning the action or actions to be taken to solve it or forwarding it to another specialized group if he/she is not qualified to do it.
- the operator can perform actions such as forwarding the ticket to another area of operation, establishing ticket processing priorities, deciding whether or not to send a technician to the site of the malfunction to solve the problem in-situ, etc.
- Machine Learning and Artificial Intelligence techniques have begun to be implemented in recent years in a number of fields. Examples include device malfunction diagnostics, medical diagnostics, market analysis, voice and image recognition, industrial process parameter optimization, advertising applications, fraud detection, IT security, electric power demand prediction [1] [4] .
- Machine Learning techniques There are supervised learning algorithms within Machine Learning techniques which have desired inputs and outputs and the models are responsible for establishing correspondence between both.
- a clear example of such techniques are classification problems, where the system tries to label each of the samples or examples it receives in one of the values included among the possible categories or classes. The knowledge acquired by these systems is based on previously labeled and validated examples.
- AUC area under the ROC curve
- the algorithms which are more widely used for classification problem solving are based on decision trees, algorithms based on Bayes' theorem, neural networks, SVM, rule- based algorithms (JRip) , as well as the Bagging method [12] and Boosting method [13] to attain more robust and stable models.
- the present invention proposes a method for optimizing and streamlining troubleshooting, wherein said trouble is indicated in tickets generated by a ticket management system.
- the method of the invention characteristically comprises, by means of using a module parallel to said ticket management system:
- the present invention proposes a system for optimizing and streamlining troubleshooting provided for implementing the method of the first aspect.
- FIG. 1 illustrates the context in which the present invention is encompassed.
- Figure 2 shows the main sub-modules of the platform proposed in the present invention.
- Figure 3 shows a detailed diagram of the blocks integrating each sub-module of the platform proposed in the present invention.
- Figure 4 shows the sub-modules and blocks supported by the execution platform, those of the database and those of the web server according to a possible implementation of the architecture of the platform proposed in the present invention.
- Figure 5 shows the sub-modules and blocks involved in generating the predictive models in the platform of the present invention .
- Figure 6 shows the information flow that is exchanged between the sub-modules and blocks involved in generating the predictive models in the platform of the present invention.
- Figure 7 shows the sub-modules and blocks involved in classifying the ticket received by the platform of the present invention .
- Figure 8 shows the information flow exchanged between the sub-modules and blocks involved in classifying the ticket received by the platform of the present invention.
- Figure 9 shows the sub-modules and blocks involved in retrieving feedback both from skilled technicians and users of the platform as well as that derived from the actions performed on the tickets to try to know the actual value of response to the problem to be predicted in the platform of the present invention .
- Figure 10 shows the information flow exchanged between the sub-modules and blocks involved in retrieving feedback both from technicians as well as that derived from the actions performed on the tickets to try to know the actual value of response to the problem to be predicted in the platform of the present invention .
- the present invention relates to a method and system that works in real time for optimizing and streamlining ticket resolution of any type in principle, but specific to the area of telecommunications operator maintenance operations. To that end, it is based on action suggestions that can be integrated in the TT system or the automation of specific tasks, being supported by supervised learning techniques.
- the TT system can be easily integrated into the workflow of the TT system since it can collect data at the time the system receives it by means of several types of interface. It processes the information and calculates the predictions in a server outside the system generating the data. Finally, depending on the configuration, it can send a notification with the suggested action or automatically launch a specific task according to the value of the prediction.
- the method starts from the historical data stored in TT systems, being applicable to any problem associated with tickets that can be solved by means of classification techniques.
- Specific cases of application of the proposed mechanism include integrating this system with a complaint management system of a telecommunications operator to try to anticipate the repetition of a customer malfunction.
- the invention tries to predict if a specific complaint is not going to be solved satisfactorily, causing within a determined time period the reoccurrence of the malfunction.
- a similar case would be the detection of those installations of a service that are going to cause a malfunction in a relatively short time period after concluding the installation.
- the system When applied to another TT system responsible for managing trouble of elements of the installed plant, at the time of creating the ticket, the system is capable of deciding whether a technician should be sent to solve the problem or if the problem can be solved remotely, preventing the unnecessary travel of technicians. It is also possible to determine whether the ticket really needs attention or if it could be closed automatically because it was actually not a problem.
- the platform is capable of interacting with the TT system in real time and becoming enriched with the information of the ticket throughout its life cycle in the system.
- the ticket is received by both the TT system and by the platform, as shown in Figure 1.
- TT system is a platform outside the TT system made up of a set of modules collecting information of the TT systems, preparing it for applying the predictive models to them, launching the corresponding prediction about the problems to be dealt with and triggering a preconfigured action (launching a notification, performing a specific operation on the ticket,%)
- the platform is made up of a series of modules that communicate with one another. These modules automate data collection from the source system, prepare said data and launch a prediction with a determined probability of the problem in question.
- This prediction can be in the form of a notification, an action suggestion or the launch of an automatism of a specific task subject to prior configuration.
- Another group of modules are responsible for checking whether or not the predictive models are still valid. If they are not, they automatically rebuild them and render them available so that the administrator managing the platform can perform the relevant tests and include them in the tool. It is possible for the tool to automatically incorporate them into the classifying modules responsible for launching predictions based on established allowed error rate thresholds.
- the Knowledge Generator collects data of the TT system for generating the necessary training sets and test sets and builds the predictive models with them. It further uses the validations or corrections of cases retrieved from the Feedback Collector for correcting or reinforcing the previously generated models.
- the Online Engine works in real time receiving the tickets while they are created in the TT system, it prepares them and applies the predictive models to them to obtain the result of the prediction. This result could be translated into an action suggestion through a web console, a notification about the TT system or a preconfigured action on a specific ticket in the TT system. The last two cases need the TT system to support such operations through some type of standardized interface.
- the Feedback Collector is a module that retrieves information about the final decision made for each case either because the technicians enters it directly in the platform or because the platform itself is responsible for finding the action which the technician ultimately performed. This information will be used for correcting or reinforcing the models .
- Figure 3 shows the diagram of the sub-modules within the main modules:
- Data Collector Module that retrieves information from the TT System in order to start working with it in the platform.
- the mechanism for extracting information from the source system can be by means of probing the database, web services, file interface, according to the system requirements. It is possible to configure this information retrieval to work in real time as it is used in the Online Engine or to retrieve data to build models in the Knowledge Generator.
- Preprocessing Data Module that performs the operations necessary to prepare the algorithm input instances. It is used both in the Online Engine for generating the inputs necessary for the predictive models, and in the Knowledge Generator for preparing the training sets and test sets for building the models .
- Predictive Engine Classifying module embedded in the predictive model, it receives data from the preprocessed tickets, applies the model to them and obtains the result of the classification for the proposed problem.
- Effector Module responsible for performing the actions configured in the platform on the TT system, ranging from sending a notification with an action suggestion to a direct action on the system, automatically performing an operation on the ticket.
- Models Builder Predictive model generator which, starting from a series of training tickets, that can be configured by the administrator, automatically builds and tests a pre-established list of both classification and attribute selection algorithms. The best option is automatically established based on comparing the ROC curves of the different models generated. Both the models and the results of their statistics are stored in the Models Repository. The administrator can see these results by means of an output display tool and perform configurable tests with different sets of data or pre-configure the automatic insertion of the model in the Predictive Engine.
- Models Checker Module that automatically checks the goodness of fit of the models using the statistics thereof. When the model starts to produce results worse than those established in the threshold that can be configured by the administrator, an alarm is launched.
- Models Repository Repository storing the predictive models that have been built by means of the Models Builder. It also saves information about the statistics of the models and the comparisons calculated by means of the Models Checker.
- User Feedback Collector Module whereby the technician enters feedback into the platform, validating or correcting the value of the predictions, aiding in reinforcing or correcting the decisions to achieve an improvement in the models.
- System Feedback Collector Module whereby feedback from the TT System itself is retrieved, revealing the decisions finally made on the tickets, comparing them with the prediction of the platform and using them for reinforcing or correcting the knowledge stored in the models.
- This module selects the feedback to be sent to the Models Builder or to the Models Checker depending on what has previously been configured (if both feedback retrieval methods are active, which of the two has a higher weight in the event of an inconsistency can be decided by means of configuration) .
- a possible implementation software of the system could be as follows, though it is not subject to any restriction.
- Execution Platform Container where the different processes are run with a communication mechanism between them.
- Database It allows the persistence of information necessary for retrieval in the event of possible errors, as well as of the generic data model storing the information about the configuration and administration of the different modules. It stores the results of the predictions, the statistics of the models and the remaining information necessary for display. It also maintains the models repository if the retrieval of any of them at any time is necessary.
- Web Server The tools that can be accessed by the technician for entering inputs into the system and displaying outputs are deployed on the web server.
- Sub-module interconnection and operation within the main modules are described below according to the operating mode of the system. It must be taken into account that normal operation comprises the simultaneous running of all the main modules, but it is possible to active or deactivate those considered appropriate by means of configuration.
- the Knowledge Generator is responsible for generating the predictive models making up the learning module of the system.
- Figure 5 shows the modules involved and the steps detailing this functionality.
- Figure 6 shows a block diagram with information flow :
- the Data Collector module is connected to the TT System for retrieving the data with which the training set and test set subsequently used to generate the predictive models will be generated .
- the Feedback Collector If the Feedback Collector is activated and has feedback, it corrects the tickets retrieved from the TT System for improving the data for generating the models.
- the instances are passed along to the Preprocessing Data module for preparing them and generating the training set and the test sets.
- the Preprocessing Data passes the training set and the test set along to the Models Builder, which tests different combinations of attributes selected by means of InfoGain, GreedyStepwise or BestFirst methods, on different types of algorithms (Naive Bayes, Bayes Net, C.45, AdaBoost, Bagging, JRip, SVM) .
- the parameters of these algorithms are heuristically optimized until finding the one that provides the best behavior by means of comparing their statistics (Area ROC, accuracy and error percentage) .
- the Models Builder sends the generated models, their statistics and the comparisons to the Models Repository.
- the Models Checker performs tests on the chosen model and checks that the statistics comply with a predefined threshold which considers the model to be good.
- the new model is sent from the Models Repository to the Predictive Engine.
- the order to replace the module manually or automatically is allowed to be sent.
- the Predictive Engine is manually or automatically restarted, according to the configuration, resuming operation with the new model.
- the Data Collector retrieves the tickets from the TT system that are pending processing by the system. Depending on the system requirements, the retrieval will be implemented by means of different mechanisms (databases, web services or files) .
- the Data Collector keeps the tickets in a temporary repository to deal with possible errors of communication with the system.
- the Data Collector informs the Preprocessing Data of the following ticket to be processed and prepares it to pass it along to the Predictive Engine and adjust it to the format which the predictive model expects to receive.
- the Predictive Engine passes the instance through the model and launches the prediction (classification with YES/NO type values in response to the proposed problem) .
- the Effector can return an outgoing message or automatically perform an action on the TT System.
- the output generated by the Predictive Engine is passed along to the Models Checker.
- the manual or automatic feedback is collected and passed along to the Models Checker so that it compares it with the outputs of the Predictive Engine and checks the validity of the model .
- the Area ROC of the model is calculated by comparing the predictions launched by the Predictive Engine with the Feedback retrieved manually or automatically
- the Feedback Collector retrieves the feedback both from technicians and that derived from the actions performed on the tickets to know the actual value of response to the problem to be predicted. This information serves to provide feedback to both the Models Builder and the Models Checker.
- Figure 9 shows the modules involved and Figure 10 shows a block diagram with the information flow:
- the System Feedback Collector retrieves from the TT system the actions that were actually performed on each ticket.
- the System Feedback Collector sends the retrieved feedback to the Feedback Selector.
- the User Feedback Collector retrieves the validations or corrections of the predictions entered by the technician and sends them to the Feedback Selector.
- the Feedback Selector sends the final feedback to the Models Checker so that it can validate the precision of the models .
- the order is sent for the Models Builder to regenerate the model using the data received from the Preprocessing Data enriched with the data received from the Feedback Collector as the training and test data.
- An expert user with platform administration permits is capable of connecting to the system ticket log and retrieving a given range of data for training use.
- the model is generated with such data by means of the Models Builder module, which automatically builds the optimal model from among those proposed (Naive Bayes, BayesNet, Adaboost, Bagging, SVM, Comite, C.45, JRip) . It performs input attribute selection for those that work the best (InfoGain, GreedyStepwise, BestFirst) and a heuristic search for the parameters that optimize it.
- the administrator is capable of selecting different sets of test by connecting to the TT system and validating the outputs. Once the statistics of the model exceed the allowed thresholds, it is capable of replacing it in the Predictive Engine which, by means of restarting the module, would start to work with the new model.
- the new tickets retrieved by the Data Collector enter gradually. They are processed by the Preprocessing Data module and the Predictive Engine decides to generate the prediction for the particular problem based on the knowledge integrated in the predictive model .
- this prediction is sent to the output for a technician to analyze it. If the technician thinks it is appropriate, he/she can enter feedback, reinforcing the decision if the decision was correct and correcting it if the decision was incorrect in order to use it in the future the next time the model is rebuilt.
- Messages can further be sent directly to the TT System by means of a notification system suitable for the interface designed between both systems. It is also possible to launch an automatism simulating a specific action of a technician on the ticket .
- the platform is capable of anticipating whether or not each of the malfunctions will be repeated in the future. This prediction is shown to the operators and a set of actions to be performed on the ticket, such as for example marking the customer in question for special tracking or establishing control calls to check if he/ she has any problem with the ticket resolution, is recommended.
- a similar case would be that applied to detecting possible early malfunctions after installing a service. These malfunctions are those which occur very shortly after the installation and may be related with an incorrect installation of the service.
- the platform connects to the TT system which collects the installation requests.
- the platform receives the request and the prediction is launched in real time. According to whether or not it is decided that an early malfunction is going to occur, the possible actions to be performed can be similar to the previous case, marking customers for special tracking, marking the ticket for post-installation analytics or another type of operation defined by the administrator.
- the platform connects to an TT system responsible for managing trouble in the installed plant of a telecommunications company.
- the system proposed enables predicting the cases in which it is necessary for a technician to travel to the indicated site of the malfunction or if, in contrast, it is a problem that can be solved remotely.
- the expenses associated with both cases are very different.
- the Knowledge Generator allows automating process of searching for the optimal model for each specific problem. This module can be enlarged to contemplate a larger number of algorithms allowed. Models which are working in real time can be validated and their statistics stored from here, checking whether the deviation of the results is within the allowed range. If it is not within this range, the models are automatically rebuilt. Temporary deviations of the results from the predictions are corrected or reduced with this mechanism.
- the Feedback Collector facilitates a predictive model correction or reinforcement mechanism based on the experience of expert technicians who validate or correct the predictions and on the experience of technicians who, without explicitly validating or correcting the results, have recorded their actions in the TT system.
- the types of algorithms that are going to be launched for automatic model generation as well as the automatic attribute selection and the heuristic search for parameters optimizing the result of the algorithms can be established by means of configuration. It is not necessary to go through all the types of algorithms if those offering the best behavior are already known.
- the thresholds of the statistics of the models which decide whether or not they are still valid as new tickets enter the platform over time can be established. These values can be checked and it can be decided whether or not to replace the old model with the one that was just generated. This model can be tested with test sets that can be configured by the administrator. If everything is okay, the order to replace the models can be sent, stopping the modules, replacing the model and restarting them again.
- the action to be performed after launching the prediction, sending messages that are visible from the TT system, messages to the output, performing actions directly on the TT system can be configured by means of an interface for interacting as a user of the system that can perform actions .
- this Predictive Analytics platform provides the operator with benefits in terms of reducing OPEX, since efficiency is gained with it, optimizing resource management. It also translates into increased customer satisfaction since it reduces ticket resolution time in addition to helping find the best resolution to such tickets.
Abstract
In the method of the invention, the trouble is indicated in tickets generated by a ticket management system, and it comprises, by means of using a module parallel to said ticket management system: - collecting said tickets; - solving the type of trouble in real time by means of predictive models built at least from historical data stored in said ticket management system; and - performing at least one of the following actions: launching an action suggestion, notifying the ticket management system of the action to be performed and/or automatically launching a task depending on a prediction obtained by means of said predictive models. The system is provided for implementing the method of the present invention.
Description
METHOD AND SYSTEM FOR OPTIMIZING AND STREAMLINING
TROUBLESHOOTING
Field of the Art
In a first aspect, the present invention relates to a method for optimizing and streamlining troubleshooting, wherein said trouble is indicated in tickets generated by a ticket management system, and more specifically to a method which comprises, by means of using a module parallel to said ticket management system, collecting said tickets, solving the type of trouble in real time by means of predictive models built at least from historical data stored in said ticket management system and performing at least one of the following actions: launching an action suggestion, notifying the ticket management system of the action to be performed and/or automatically launching a task depending on a prediction obtained by means of said predictive models.
A second aspect of the invention relates to a system provided for implementing the method of the first aspect.
Prior State of the Art
For some time, large telecommunication companies have sought to streamline and automate maintenance processes, to the greatest extent possible, by implementing systems which detect problems in their network in order to solve them as quickly as possible or which manage their resources more efficiently. Both cost reduction and increased customer satisfaction are therefore achieved.
Trouble ticketing systems are used to improve efficiency of the management of these processes, and they are basically responsible for managing tickets from the time they are created until they can be solved and closed. Tickets are generally created manually by the operator or automatically by means of automatic alarm detection systems. Once the ticket is created, the technician is responsible for analyzing and diagnosing the problem, making a decision concerning the action or actions to be taken to solve it or forwarding it to another specialized
group if he/she is not qualified to do it.
Throughout the life cycle of the ticket, the operator can perform actions such as forwarding the ticket to another area of operation, establishing ticket processing priorities, deciding whether or not to send a technician to the site of the malfunction to solve the problem in-situ, etc.
Until now these jobs were being steered towards solutions based on proprietary systems and expert systems which needed to be supplied with the knowledge acquired by experienced technicians with a broad business perspective to make these systems intelligent and to take advantage of technicians' knowledge .
From the viewpoint of trouble ticketing (TT) systems, attempts are being made to automate ticket management, starting with the automatic creation of the tickets [11] and continuing with the automation of specific operations [7].
Machine Learning and Artificial Intelligence techniques have begun to be implemented in recent years in a number of fields. Examples include device malfunction diagnostics, medical diagnostics, market analysis, voice and image recognition, industrial process parameter optimization, advertising applications, fraud detection, IT security, electric power demand prediction [1] [4] .
More specific cases which are also more closely related to the area of telecommunications are the optimization of the calls of a Call Center [2], churn detection [3], network traffic prediction [4] and detection of hardware failures within a network, correlating the different alarms included [5].
There are supervised learning algorithms within Machine Learning techniques which have desired inputs and outputs and the models are responsible for establishing correspondence between both. A clear example of such techniques are classification problems, where the system tries to label each of the samples or examples it receives in one of the values included among the possible categories or classes. The knowledge acquired by these systems is based on previously labeled and
validated examples.
Work is being done on implementations and methods encouraging the application of predictive analytics to solving the largest number of problems [9], in the automatic update of predictive models used in a production setting [10] as well as in the reinforcement of learning modules [8] . These are examples of the problems arising when integrating all these techniques into production settings.
One of the methods currently being used to decide on the optimal model is the comparison of certain statistics of these models. The study of the area under the ROC curve, also referred to as AUC, is based on the ratio between the sensitivity and specificity of the model, which represent the fraction of true positives of one experiment with respect to the fraction of true negatives of the same experiment, respectively.
The algorithms which are more widely used for classification problem solving are based on decision trees, algorithms based on Bayes' theorem, neural networks, SVM, rule- based algorithms (JRip) , as well as the Bagging method [12] and Boosting method [13] to attain more robust and stable models.
When examining another type of proprietary solutions, difficulties can be encountered when they are integrated in the workflow of the TT system in real time and receive information, without needing to deal with adaptations on TT systems, which are fairly expensive in most cases.
The development and inclusion of a similar solution for each new similar problem that needs to be solved by means of predictive analytics techniques is likewise found to be expensive and slow. There is no mechanism that streamlines the building and selection of optimal models for each problem and that allows incorporating them into the workflow of the TT system, minimizing adaptations thereof.
Despite there being previously designed algorithms that could be used on the data of TT systems, the application of these algorithms on certain data is not so simple. This data can vary considerably due to business changes which make it
necessary to have a mechanism that allows checking the validity of the predictions over time. Model degradation effects readily appear in the area of application due to data variability and it should be possible to interact with the platform, automating the process as much as possible.
Since the solution is aimed at using the logs indicating the operations performed on tickets, it must be taken into account that not all the knowledge that is extracted is valid, since sometimes the models can even be forced to learn erroneous procedures for ticket resolution. For this reason the direct application of classification algorithms on the data of the TT system can lead to generating predictive models which, instead of streamlining and improving ticket management, slows it down due to the accumulation of errors. A mechanism is necessary in order to be capable of correcting those cases which, a priori, these algorithms do not provide, giving the technician the possibility to validate or correct the predictions of the models .
Summary of the Invention
It is necessary to find an alternative to the state of the art that allows making up for the detected deficiencies, particularly relating to the lack of proposals that can be integrated in the workflow of a TT system in real time and that receive information without needing to deal with adaptations as well as the lack of a mechanism that streamlines the building and selection of optimal models for each problem that presents.
To that end, in a first aspect the present invention proposes a method for optimizing and streamlining troubleshooting, wherein said trouble is indicated in tickets generated by a ticket management system.
Unlike the already known proposals, the method of the invention characteristically comprises, by means of using a module parallel to said ticket management system:
- collecting said tickets;
- solving the type of trouble in real time by means of predictive models built at least from historical data stored in
said ticket management system; and
performing at least one of the following actions: launching an action suggestion, notifying said ticket management system of the action to be performed and/or automatically launching a task depending on a prediction obtained by means of said predictive models.
Other aspects of the method of the first aspect of the invention are described according to the attached claims 2 to 13.
In a second aspect the present invention proposes a system for optimizing and streamlining troubleshooting provided for implementing the method of the first aspect.
Other features of the system of the second aspect of the invention are described according to the attached claims 14 and 15.
Brief Description of the Drawings
The foregoing and other advantages and features will be better understood from the following detailed description of several embodiments in reference to the attached illustrative and non-limiting drawings in which:
Figure 1 illustrates the context in which the present invention is encompassed.
Figure 2 shows the main sub-modules of the platform proposed in the present invention.
Figure 3 shows a detailed diagram of the blocks integrating each sub-module of the platform proposed in the present invention.
Figure 4 shows the sub-modules and blocks supported by the execution platform, those of the database and those of the web server according to a possible implementation of the architecture of the platform proposed in the present invention.
Figure 5 shows the sub-modules and blocks involved in generating the predictive models in the platform of the present invention .
Figure 6 shows the information flow that is exchanged between the sub-modules and blocks involved in generating the
predictive models in the platform of the present invention.
Figure 7 shows the sub-modules and blocks involved in classifying the ticket received by the platform of the present invention .
Figure 8 shows the information flow exchanged between the sub-modules and blocks involved in classifying the ticket received by the platform of the present invention.
Figure 9 shows the sub-modules and blocks involved in retrieving feedback both from skilled technicians and users of the platform as well as that derived from the actions performed on the tickets to try to know the actual value of response to the problem to be predicted in the platform of the present invention .
Figure 10 shows the information flow exchanged between the sub-modules and blocks involved in retrieving feedback both from technicians as well as that derived from the actions performed on the tickets to try to know the actual value of response to the problem to be predicted in the platform of the present invention .
Detailed Description of several Embodiments
The present invention relates to a method and system that works in real time for optimizing and streamlining ticket resolution of any type in principle, but specific to the area of telecommunications operator maintenance operations. To that end, it is based on action suggestions that can be integrated in the TT system or the automation of specific tasks, being supported by supervised learning techniques.
It is based on predictive models built from the historical data stored in the trouble ticketing system. These models can be rebuilt automatically in those cases in which the tool detects that the model is no longer reliable. It contains a functionality of obtaining feedback from both the technicians and from the trouble ticketing system itself to correct or reinforce the previously generated knowledge, whichever may be appropriate .
It can be easily integrated into the workflow of the TT
system since it can collect data at the time the system receives it by means of several types of interface. It processes the information and calculates the predictions in a server outside the system generating the data. Finally, depending on the configuration, it can send a notification with the suggested action or automatically launch a specific task according to the value of the prediction.
It is a method or mechanism for optimizing and streamlining ticket resolution, recommending the best use of resources in terms of cost reduction and increased customer satisfaction in real time. Supervised learning techniques that can be integrated in already running TT systems are used.
The method starts from the historical data stored in TT systems, being applicable to any problem associated with tickets that can be solved by means of classification techniques.
Specific cases of application of the proposed mechanism include integrating this system with a complaint management system of a telecommunications operator to try to anticipate the repetition of a customer malfunction. In other words, the invention tries to predict if a specific complaint is not going to be solved satisfactorily, causing within a determined time period the reoccurrence of the malfunction. A similar case would be the detection of those installations of a service that are going to cause a malfunction in a relatively short time period after concluding the installation.
When applied to another TT system responsible for managing trouble of elements of the installed plant, at the time of creating the ticket, the system is capable of deciding whether a technician should be sent to solve the problem or if the problem can be solved remotely, preventing the unnecessary travel of technicians. It is also possible to determine whether the ticket really needs attention or if it could be closed automatically because it was actually not a problem.
The platform is capable of interacting with the TT system in real time and becoming enriched with the information of the ticket throughout its life cycle in the system. At the time of
creating the ticket, the ticket is received by both the TT system and by the platform, as shown in Figure 1.
As discussed, it is a platform outside the TT system made up of a set of modules collecting information of the TT systems, preparing it for applying the predictive models to them, launching the corresponding prediction about the problems to be dealt with and triggering a preconfigured action (launching a notification, performing a specific operation on the ticket,...)
The platform is made up of a series of modules that communicate with one another. These modules automate data collection from the source system, prepare said data and launch a prediction with a determined probability of the problem in question. This prediction can be in the form of a notification, an action suggestion or the launch of an automatism of a specific task subject to prior configuration.
Another group of modules are responsible for checking whether or not the predictive models are still valid. If they are not, they automatically rebuild them and render them available so that the administrator managing the platform can perform the relevant tests and include them in the tool. It is possible for the tool to automatically incorporate them into the classifying modules responsible for launching predictions based on established allowed error rate thresholds.
There are three main modules: Knowledge Generator, Online Engine and Feedback Collector, as shown in Figure 2.
The Knowledge Generator collects data of the TT system for generating the necessary training sets and test sets and builds the predictive models with them. It further uses the validations or corrections of cases retrieved from the Feedback Collector for correcting or reinforcing the previously generated models.
The Online Engine works in real time receiving the tickets while they are created in the TT system, it prepares them and applies the predictive models to them to obtain the result of the prediction. This result could be translated into an action suggestion through a web console, a notification about the TT system or a preconfigured action on a specific ticket in the TT
system. The last two cases need the TT system to support such operations through some type of standardized interface.
The Feedback Collector is a module that retrieves information about the final decision made for each case either because the technicians enters it directly in the platform or because the platform itself is responsible for finding the action which the technician ultimately performed. This information will be used for correcting or reinforcing the models .
There is a mechanism for inputting information into the system to provide it with the capacities of stopping, starting and configuring all the modules, as well as more specific functionalities for monitoring the behavior of the predictive models. Examples of actions that can be performed using capacities of configuring the modules are:
Determining the types of algorithms from among those contemplated by the platform that will be used for rebuilding and comparing several models then selecting the models with optimal algorithms.
- Defining a range of tickets to be used in the training set and test set for rebuilding models both manually and automatically .
- Establishing the manual or automatic nature of replacing the old predictive models with the recently generated predictive models .
- Configuring the actions to be performed according to the result of the prediction (displaying the result in the web console, action suggestion, notification, performing an automatism, ... )
Figure 3 shows the diagram of the sub-modules within the main modules:
Data Collector: Module that retrieves information from the TT System in order to start working with it in the platform. The mechanism for extracting information from the source system can be by means of probing the database, web services, file interface, according to the system requirements. It is possible
to configure this information retrieval to work in real time as it is used in the Online Engine or to retrieve data to build models in the Knowledge Generator.
Preprocessing Data: Module that performs the operations necessary to prepare the algorithm input instances. It is used both in the Online Engine for generating the inputs necessary for the predictive models, and in the Knowledge Generator for preparing the training sets and test sets for building the models .
Predictive Engine: Classifying module embedded in the predictive model, it receives data from the preprocessed tickets, applies the model to them and obtains the result of the classification for the proposed problem.
Effector: Module responsible for performing the actions configured in the platform on the TT system, ranging from sending a notification with an action suggestion to a direct action on the system, automatically performing an operation on the ticket.
Models Builder: Predictive model generator which, starting from a series of training tickets, that can be configured by the administrator, automatically builds and tests a pre-established list of both classification and attribute selection algorithms. The best option is automatically established based on comparing the ROC curves of the different models generated. Both the models and the results of their statistics are stored in the Models Repository. The administrator can see these results by means of an output display tool and perform configurable tests with different sets of data or pre-configure the automatic insertion of the model in the Predictive Engine.
Models Checker: Module that automatically checks the goodness of fit of the models using the statistics thereof. When the model starts to produce results worse than those established in the threshold that can be configured by the administrator, an alarm is launched.
Models Repository: Repository storing the predictive models that have been built by means of the Models Builder. It also
saves information about the statistics of the models and the comparisons calculated by means of the Models Checker.
User Feedback Collector: Module whereby the technician enters feedback into the platform, validating or correcting the value of the predictions, aiding in reinforcing or correcting the decisions to achieve an improvement in the models.
System Feedback Collector: Module whereby feedback from the TT System itself is retrieved, revealing the decisions finally made on the tickets, comparing them with the prediction of the platform and using them for reinforcing or correcting the knowledge stored in the models.
Feedback Selector: This module selects the feedback to be sent to the Models Builder or to the Models Checker depending on what has previously been configured (if both feedback retrieval methods are active, which of the two has a higher weight in the event of an inconsistency can be decided by means of configuration) .
• Architecture
A possible implementation software of the system could be as follows, though it is not subject to any restriction.
Execution Platform: Container where the different processes are run with a communication mechanism between them.
Database: It allows the persistence of information necessary for retrieval in the event of possible errors, as well as of the generic data model storing the information about the configuration and administration of the different modules. It stores the results of the predictions, the statistics of the models and the remaining information necessary for display. It also maintains the models repository if the retrieval of any of them at any time is necessary.
Web Server: The tools that can be accessed by the technician for entering inputs into the system and displaying outputs are deployed on the web server.
The sub-modules that could be supported in the Execution Platform, those that could use the Database layer and those that could be deployed on a Web Server are shown for each of the main
modules in Figure 4.
• Module operation and interconnection
Sub-module interconnection and operation within the main modules are described below according to the operating mode of the system. It must be taken into account that normal operation comprises the simultaneous running of all the main modules, but it is possible to active or deactivate those considered appropriate by means of configuration.
Predictive model building operation
The Knowledge Generator is responsible for generating the predictive models making up the learning module of the system. Figure 5 shows the modules involved and the steps detailing this functionality. Figure 6 shows a block diagram with information flow :
1. The Data Collector module is connected to the TT System for retrieving the data with which the training set and test set subsequently used to generate the predictive models will be generated .
2. If the Feedback Collector is activated and has feedback, it corrects the tickets retrieved from the TT System for improving the data for generating the models.
3. The instances are passed along to the Preprocessing Data module for preparing them and generating the training set and the test sets.
4. The Preprocessing Data passes the training set and the test set along to the Models Builder, which tests different combinations of attributes selected by means of InfoGain, GreedyStepwise or BestFirst methods, on different types of algorithms (Naive Bayes, Bayes Net, C.45, AdaBoost, Bagging, JRip, SVM) . The parameters of these algorithms are heuristically optimized until finding the one that provides the best behavior by means of comparing their statistics (Area ROC, accuracy and error percentage) .
5. The Models Builder sends the generated models, their statistics and the comparisons to the Models Repository.
6. The Models Checker performs tests on the chosen model
and checks that the statistics comply with a predefined threshold which considers the model to be good.
7. The new model is sent from the Models Repository to the Predictive Engine. The order to replace the module manually or automatically is allowed to be sent.
8. The Predictive Engine is manually or automatically restarted, according to the configuration, resuming operation with the new model.
- Classification operation in real time
In the Online Engine, it is important to start from the fact that the Knowledge Generator has previously been used for building the model optimizing the problem and has been embedded in the Predictive Engine. Figure 7 shows the modules involved and Figure 8 shows a block diagram with the information flow:
1. The Data Collector retrieves the tickets from the TT system that are pending processing by the system. Depending on the system requirements, the retrieval will be implemented by means of different mechanisms (databases, web services or files) . The Data Collector keeps the tickets in a temporary repository to deal with possible errors of communication with the system.
2. The Data Collector informs the Preprocessing Data of the following ticket to be processed and prepares it to pass it along to the Predictive Engine and adjust it to the format which the predictive model expects to receive.
3. The Predictive Engine passes the instance through the model and launches the prediction (classification with YES/NO type values in response to the proposed problem) .
4. According to its configuration, the Effector can return an outgoing message or automatically perform an action on the TT System.
5. The output generated by the Predictive Engine is passed along to the Models Checker.
6. The manual or automatic feedback is collected and passed along to the Models Checker so that it compares it with the outputs of the Predictive Engine and checks the validity of the
model .
7. The Area ROC of the model is calculated by comparing the predictions launched by the Predictive Engine with the Feedback retrieved manually or automatically
8. If the Area ROC of the model is under the allowed value, the rebuilding of the models is triggered.
- Feedback retrieval operation
The Feedback Collector retrieves the feedback both from technicians and that derived from the actions performed on the tickets to know the actual value of response to the problem to be predicted. This information serves to provide feedback to both the Models Builder and the Models Checker.
Figure 9 shows the modules involved and Figure 10 shows a block diagram with the information flow:
1. It is possible to configure the feedback retrieval mode, being able to activate only the User Feedback Collector, only the System Feedback Collector or both, weighting the result by means of the Feedback Selector.
2. The System Feedback Collector retrieves from the TT system the actions that were actually performed on each ticket.
3. The System Feedback Collector sends the retrieved feedback to the Feedback Selector.
4. The User Feedback Collector retrieves the validations or corrections of the predictions entered by the technician and sends them to the Feedback Selector.
5. The Feedback Selector sends the final feedback to the Models Checker so that it can validate the precision of the models .
6. According to the results extracted from the Models Checker, the order is sent for the Models Builder to regenerate the model using the data received from the Preprocessing Data enriched with the data received from the Feedback Collector as the training and test data.
• Cases of use
The operation can be described as follows for any of the cases of use explained below.
An expert user with platform administration permits is capable of connecting to the system ticket log and retrieving a given range of data for training use. The model is generated with such data by means of the Models Builder module, which automatically builds the optimal model from among those proposed (Naive Bayes, BayesNet, Adaboost, Bagging, SVM, Comite, C.45, JRip) . It performs input attribute selection for those that work the best (InfoGain, GreedyStepwise, BestFirst) and a heuristic search for the parameters that optimize it.
Once the model is built, the administrator is capable of selecting different sets of test by connecting to the TT system and validating the outputs. Once the statistics of the model exceed the allowed thresholds, it is capable of replacing it in the Predictive Engine which, by means of restarting the module, would start to work with the new model.
Once the Predictive Engine is ready, the new tickets retrieved by the Data Collector enter gradually. They are processed by the Preprocessing Data module and the Predictive Engine decides to generate the prediction for the particular problem based on the knowledge integrated in the predictive model .
According to what has been configured, this prediction is sent to the output for a technician to analyze it. If the technician thinks it is appropriate, he/she can enter feedback, reinforcing the decision if the decision was correct and correcting it if the decision was incorrect in order to use it in the future the next time the model is rebuilt.
Messages can further be sent directly to the TT System by means of a notification system suitable for the interface designed between both systems. It is also possible to launch an automatism simulating a specific action of a technician on the ticket .
Several cases of use of the platform in the area of problems specific to TT systems are described below.
- Detecting the reoccurrence of one and the same malfunction in a customer complaint TT system
This is a case of being applied to a complaint TT system of a telecommunications operator. The problem to be solved is to determine whether a given complaint is going to occur again in the system after a given time period because it was not correctly solved in a first action of the technician.
The platform is capable of anticipating whether or not each of the malfunctions will be repeated in the future. This prediction is shown to the operators and a set of actions to be performed on the ticket, such as for example marking the customer in question for special tracking or establishing control calls to check if he/ she has any problem with the ticket resolution, is recommended.
Detecting the occurrence of early malfunctions after installing a service
A similar case would be that applied to detecting possible early malfunctions after installing a service. These malfunctions are those which occur very shortly after the installation and may be related with an incorrect installation of the service.
The platform connects to the TT system which collects the installation requests. The platform receives the request and the prediction is launched in real time. According to whether or not it is decided that an early malfunction is going to occur, the possible actions to be performed can be similar to the previous case, marking customers for special tracking, marking the ticket for post-installation analytics or another type of operation defined by the administrator.
Detecting the need to send a technician to solve a malfunction or possibility of solving it remotely.
In this case, the platform connects to an TT system responsible for managing trouble in the installed plant of a telecommunications company.
The system proposed enables predicting the cases in which it is necessary for a technician to travel to the indicated site of the malfunction or if, in contrast, it is a problem that can be solved remotely. The expenses associated with both cases are
very different.
- Detecting productivity of the malfunction resolution
In this case, by using this system it can be predicted whether or not the work that should theoretically be done to solve the malfunction associated with a ticket is going to be productive, i.e. whether it is necessary to associate it with a worker (local or remote service) or whether it is best to directly close the ticket because no error is going to occur in the system.
• Advantages of the invention
Adapting to new scenarios is easier by means of the system. It is a very flexible mechanism for such problems, solving communication with the TT system for retrieving tickets by means of prepared file reader interfaces, probing a database or using the necessary connector.
Only minimal adaptations would have to be made in the Data Collector and Preprocessing Data modules. The Effector would further have to be adapted in cases where direct action on the TT system is configured. The rest are generic modules that do not depend on the architecture of the TT system.
The Knowledge Generator allows automating process of searching for the optimal model for each specific problem. This module can be enlarged to contemplate a larger number of algorithms allowed. Models which are working in real time can be validated and their statistics stored from here, checking whether the deviation of the results is within the allowed range. If it is not within this range, the models are automatically rebuilt. Temporary deviations of the results from the predictions are corrected or reduced with this mechanism.
The Feedback Collector facilitates a predictive model correction or reinforcement mechanism based on the experience of expert technicians who validate or correct the predictions and on the experience of technicians who, without explicitly validating or correcting the results, have recorded their actions in the TT system.
The types of algorithms that are going to be launched for
automatic model generation as well as the automatic attribute selection and the heuristic search for parameters optimizing the result of the algorithms can be established by means of configuration. It is not necessary to go through all the types of algorithms if those offering the best behavior are already known. The thresholds of the statistics of the models which decide whether or not they are still valid as new tickets enter the platform over time can be established. These values can be checked and it can be decided whether or not to replace the old model with the one that was just generated. This model can be tested with test sets that can be configured by the administrator. If everything is okay, the order to replace the models can be sent, stopping the modules, replacing the model and restarting them again. The action to be performed after launching the prediction, sending messages that are visible from the TT system, messages to the output, performing actions directly on the TT system, can be configured by means of an interface for interacting as a user of the system that can perform actions .
From the commercial viewpoint, the application of this Predictive Analytics platform provides the operator with benefits in terms of reducing OPEX, since efficiency is gained with it, optimizing resource management. It also translates into increased customer satisfaction since it reduces ticket resolution time in addition to helping find the best resolution to such tickets.
A person skilled in the art would be able to introduce changes and modifications in the described embodiments without departing from the scope of the invention as defined in the attached claims .
ACRONYMS
AUC Area Under Curve
ROC Receiver Operating Characteristic
SVM Support Vector Machine
TT Trouble Ticket
LITERATURE
[1] Fabien Girardin, Josep Blat, Francesco Calabrese, Filippo Dal Fiore, and Carlo Ratti . Digital footprinting : Uncovering tourists with user-generated content. IEEE Pervasive Computing, 7 (4) : 36B43, 2008
[2] Tye Rattenbury, Nathaniel Good, Mor Naaman : Towards automatic extraction of event and place semantics from flickr tags. SIGIR 2007: 103-110
Claims
1. - A method for optimizing and streamlining troubleshooting, said trouble being indicated in tickets generated by a ticket management system, characterized in that it comprises, by means of using a module parallel to said ticket management system:
- collecting said tickets;
- solving the type of trouble in real time by means of predictive models built at least from historical data stored in said ticket management system; and
performing at least one of the following actions: launching an action suggestion, notifying said ticket management system of the action to be performed and/or automatically launching a task depending on a prediction obtained by means of said predictive models.
2. - The method according to claim 1, wherein said troubleshooting is encompassed within the maintenance operations of a telecommunications operator.
3. - The method according to claim 1 or 2, wherein said predictive systems comprise a classification of said tickets depending on the type of trouble they indicate.
4. - The method according to claim 3, which comprises generating said predictive models by testing a pre-established list of algorithms by means of configurable training tickets, choosing those predictive models having better characteristics depending on Receiver Operating Characteristic curves, or ROC curves .
5. - The method according to claim 4, which comprises manually or automatically rebuilding said predictive models depending on validations or corrections concerning the final decision made for each trouble indicated in said tickets, said validations or corrections from a technician or from said ticket management system.
6. - The method according to any of the preceding claims, which comprises extracting data from said ticket management system by means of: a database, web services or file interface.
7. - The method according to claim 6 when it depends on claim 5, which comprises using said data in real time to apply them to said predictive models or using said data for said rebuilding of said predictive models, said data comprising said tickets .
8. - The method according to any of the preceding claims, which comprises calculating the statistics of said predictive models and checking the goodness of fit of said predictive models depending on at least one configurable threshold.
9. - The method according to claim 8, wherein said generation of said predictive models comprises:
performing said extraction of data from said ticket management system;
- correcting tickets associated with said data by means of said corrections from technicians or from said ticket management system;
- generating said configurable training tickets by means of preparing said corrected data and/or tickets;
- combining said configurable tickets with said algorithms;
- heuristically optimizing parameters of said algorithms depending on statistics obtained from said combinations;
- choosing at least one predictive model depending on said statistics; and
- performing said goodness of fit check of said predictive model .
10. - The method according to claim 9, wherein said algorithms are Naive Bayes, Bayes Net, C.45, AdaBoost, Bagging, JRip or Support Vector Machine type algorithms.
11. - The method according to claim 8, 9 or 10, wherein said classification of said tickets comprises:
performing said extraction of data from said ticket management system;
- adjusting tickets associated with said data to the format required by said predictive models;
- launching a classification of a trouble associated with one of said tickets as the result of applying said predictive models on said ticket; and
sending an outgoing message or performing an action automatically depending on said classification.
12. - The method according to any of the preceding claims, which comprises predicting the repetition of a trouble associated with said tickets if a prior action for solving said trouble is unsatisfactory.
13. - A system for optimizing and streamlining troubleshooting, said trouble being indicated in tickets generated by a ticket management system, characterized in that it comprises a module parallel to said ticket management system containing the following sub-modules:
- generator sub-module, which collects said tickets from said ticket management system and creates predictive models;
- predictor sub-module, which resolves the type of trouble in real time by means of said predictive models built at least from historical data stored in said ticket management system and performs at least one of the following actions: launches an action suggestion, notifies said ticket management system of the action to be performed and/or automatically launches a task depending on a prediction obtained by means of said predictive models; and
feedback collector sub-module, which corrects or reinforces said predictive models depending on information entered by a technician or information extracted from said ticket management system, said information referring to the final decision made to solve said trouble.
14. - The system according to claim 13, wherein said sub- modules of said module parallel to said ticket management system have a communication mechanism for communicating with one another .
15. - The system wherein said sub-modules implement the method according to any of the preceding claims 1 to 12.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
ES201131470A ES2408112B1 (en) | 2011-09-07 | 2011-09-07 | Method and system for optimization and speeding up incident resolution |
ESP201131470 | 2011-09-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013034448A1 true WO2013034448A1 (en) | 2013-03-14 |
Family
ID=46801463
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2012/066423 WO2013034448A1 (en) | 2011-09-07 | 2012-08-23 | Method and system for optimizing and streamlining troubleshooting |
Country Status (2)
Country | Link |
---|---|
ES (1) | ES2408112B1 (en) |
WO (1) | WO2013034448A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110866790A (en) * | 2019-11-26 | 2020-03-06 | 上海景域文化传播股份有限公司 | Scenic spot ticket sales prediction system and method |
WO2020237535A1 (en) * | 2019-05-29 | 2020-12-03 | Nokia Solutions And Networks Oy | Systems, methods, and computer readable mediums for controlling federation of automated agents |
US11741065B2 (en) | 2020-02-04 | 2023-08-29 | International Business Machines Corporation | Hardware, firmware, and software anomaly handling based on machine learning |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10771314B2 (en) | 2017-09-15 | 2020-09-08 | Accenture Global Solutions Limited | Learning based incident or defect resolution, and test generation |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3268529B2 (en) * | 1990-03-14 | 2002-03-25 | 株式会社日立製作所 | Knowledge database processing system and expert system |
US5666481A (en) * | 1993-02-26 | 1997-09-09 | Cabletron Systems, Inc. | Method and apparatus for resolving faults in communications networks |
US20050169452A1 (en) * | 2004-02-03 | 2005-08-04 | Sigma Dynamics, Inc. | Method and apparatus for self-evaluation and randomization for predictive models |
GB0624024D0 (en) * | 2006-12-01 | 2007-01-10 | Ibm | Event correlation based trouble ticket resolution system incorporating adaptive rules optimization |
-
2011
- 2011-09-07 ES ES201131470A patent/ES2408112B1/en not_active Withdrawn - After Issue
-
2012
- 2012-08-23 WO PCT/EP2012/066423 patent/WO2013034448A1/en active Application Filing
Non-Patent Citations (4)
Title |
---|
FABIEN GIRARDIN; JOSEP BLAT; FRANCESCO CALABRESE; FILIPPO DAL FIORE; CARLO RATTI: "Digital footprinting: Uncovering tourists with user-generated content", IEEE PERVASIVE COMPUTING, vol. 7, no. 4, 2008, pages 36 - 43, XP011236560, DOI: doi:10.1109/MPRV.2008.71 |
JULIA GÓMEZ, YAIZA TEMPRADO, MARGARITA GALLARDO, CAROLINA GARCÍA, FRANCISCO JAVIER MOLINERO: "APPLICATION OF NEURAL NETWORK TO PREDICT ADVERSE SITUATIONS IN TROUBLE TICKETING REPORTS", 2009, pages 204 - 208, XP002686619, ISBN: 978-972-8924-87-4, Retrieved from the Internet <URL:http://www.iadisportal.org/digital-library/application-of-neural-network-to-predict-adverse-situations-in-trouble-ticketing-reports> [retrieved on 20121114] * |
TYE RATTENBURY; NATHANIEL GOOD; MOR NAAMAN: "Towards automatic extraction of event and place semantics from flickr tags", SIGIR, 2007, pages 103 - 110 |
YAIZA TEMPRADO ET AL: "Knowledge Discovery from Trouble Ticketing Reports in a Large Telecommunication Company", COMPUTATIONAL INTELLIGENCE FOR MODELLING CONTROL&AUTOMATION, 2008 INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 10 December 2008 (2008-12-10), pages 37 - 42, XP031495990, ISBN: 978-0-7695-3514-2 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020237535A1 (en) * | 2019-05-29 | 2020-12-03 | Nokia Solutions And Networks Oy | Systems, methods, and computer readable mediums for controlling federation of automated agents |
CN110866790A (en) * | 2019-11-26 | 2020-03-06 | 上海景域文化传播股份有限公司 | Scenic spot ticket sales prediction system and method |
US11741065B2 (en) | 2020-02-04 | 2023-08-29 | International Business Machines Corporation | Hardware, firmware, and software anomaly handling based on machine learning |
Also Published As
Publication number | Publication date |
---|---|
ES2408112A2 (en) | 2013-06-18 |
ES2408112B1 (en) | 2014-02-28 |
ES2408112R1 (en) | 2013-08-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10901727B2 (en) | Monitoring code sensitivity to cause software build breaks during software project development | |
EP3889777A1 (en) | System and method for automating fault detection in multi-tenant environments | |
US10877863B2 (en) | Automatic prediction system for server failure and method of automatically predicting server failure | |
US11645191B2 (en) | Review process for evaluating changes to target code for a software-based product | |
US10310968B2 (en) | Developing software project plans based on developer sensitivity ratings detected from monitoring developer error patterns | |
US20050144151A1 (en) | System and method for decision analysis and resolution | |
KR20190021560A (en) | Failure prediction system using big data and failure prediction method | |
EP3920042A1 (en) | Hybrid cloud integration deployment and management | |
Friederich et al. | Towards data-driven reliability modeling for cyber-physical production systems | |
US8794513B2 (en) | Self-service device servicing utilizing a hardware database | |
CN113516244B (en) | Intelligent operation and maintenance method and device, electronic equipment and storage medium | |
WO2013034448A1 (en) | Method and system for optimizing and streamlining troubleshooting | |
CN115664939B (en) | Comprehensive operation and maintenance method based on automation technology and storage medium | |
CN114880312B (en) | Flexibly-set application system service data auditing method | |
Pernici et al. | Automatic learning of repair strategies for web services | |
CN111062827B (en) | Engineering supervision method based on artificial intelligence mode | |
US20230072123A1 (en) | Method and system for automating analysis of log data files | |
KR102489119B1 (en) | Smart FMEA system and process management system and method using thereof | |
CN112069197A (en) | Abnormal work order method and device | |
US20210302954A1 (en) | System and method for increasing mean time between service visits in an industrial system | |
US20150156090A1 (en) | Systems and Methods for Monitoring Multiple Services | |
CN110689144B (en) | Management method and device of intelligent recovery device | |
CN115098739A (en) | Message comparison method and device of decision engine | |
CN111858291B (en) | Root cause determination method, equipment and system for data abnormity in charging system migration test | |
WO2024066346A1 (en) | Alarm processing method and apparatus, and storage medium and electronic apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12756142 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 12756142 Country of ref document: EP Kind code of ref document: A1 |