GB2614710A

GB2614710A - A method for monitoring an application by a monitoring system as well as a corresponding monitoring system

Info

Publication number: GB2614710A
Application number: GB2200307.3A
Authority: GB
Inventors: Mcgill Kyle
Original assignee: Mercedes Benz Group AG
Current assignee: Mercedes Benz Group AG
Priority date: 2022-01-12
Filing date: 2022-01-12
Publication date: 2023-07-19
Also published as: DE102022004161A1

Abstract

A method for monitoring an application wherein an application update is received from a user of the application, analysed depending on a mathematical model and an output provided depending on the result of the analysis. The update may comprise a metadata section and data comprising an M x N matrix. The monitoring system may use an artificial intelligence to carry out the analysis. The user may be asked to confirm the usefulness of the output. The output confirmation may be used to further train the monitoring system. Depending on the received update, a new mathematical model may be created for analysing the update. The mathematical model may be created by receiving an application update from a user, applying a mathematical model to the data to produce better data, evaluating the data and better data to return a value and adding the mathematical model to the monitoring system. The mathematical model may be applied to future application updates before evaluating for an alert.

Description

A METHOD FOR MONITORING AN APPLICATION BY A MONITORING SYSTEM AS

WELL AS A CORRESPONDING MONITORING SYSTEM

FIELD OF THE INVENTION

[0001] The invention relates to the field of application monitoring. More specifically, the invention relates to a method for automated monitoring of an application as well as to a corresponding monitoring system.

BACKGROUND INFORMATION

[0002] In the state of the art, cloud computing refers to one or more devices in a hardware fabric designed to support hardware-agnostic applications. Cloud-based application refers to an application configured to run in a cloud computing environment. The hardware fabric does not necessarily need to be provided by a vendor and can be a privately set up and run hardware fabric.

SUMMARY OF THE INVENTION

[0003] It is an object of the invention to provide a method as well as a monitoring system by which automated monitoring of an application is realized.

[0004] This object is solved by a method as well as a monitoring system according to the independent claims. Advantageous forms of configurations are presented in the dependent claims.

[0005] One aspect of the invention relates to a method for monitoring an application by a monitoring system. The monitoring system is receiving an update from an application. The received update is analyzed depending on a mathematical model of an electronic computing device of the monitoring system. Depending on the results of the analysis of the mathematical model on the received update, an output is provided to the user by the monitoring system.

[0006] A new way of monitoring cloud-based applications is presented by automating the configuration and execution of statistical models to determine the health of an application. The monitoring system automatically creates a model and monitors the application thus saving user's time and money spent on creating and configuring the monitoring system for the application. This monitoring system also has a loose coupling to the application that is being monitored. The application choses what, if any, information to be sent to the monitoring system. In the case of exceptions thrown by the application, it is up to the application to send that information to the monitoring system rather than having the monitoring system collect and aggregate itself.

[0007] According to the state of the art, software engineers spend large amounts of their time configuring alerts specific for their applications. This loss of productivity as well as the paid time leads to cost savings when an engineer uses this method. This process makes their entire workload faster and smoother.

[0008] In an embodiment, the electronic computing device uses artificial intelligence for analyzing an update from an application.

[0009] According to another embodiment depending on the received update, the electronic computing device creates a mathematical model for analyzing the received update.

[0010] In another embodiment, depending on the outcome of the analysis an alert-message is generated as an output by the monitoring system.

[0011] In another embodiment, if the alert-message is generated, the monitoring system asks a user of the monitoring system to confirm the alert-message.

[0012] The method is in particular a computer-implemented method. Therefore, another aspect of the invention relates to a computer program product comprising program code means for performing the method.

[0013] A still further aspect of the invention relates to a monitoring system for monitoring an application comprising at least one electronic computing device, wherein the device is configured for performing a method according to the preceding aspect. In particular, the monitoring system performs the method.

[0014] Advantageous forms of configurations of the method are to be regarded as advantageous forms of configurations of the monitoring system. The monitoring system therefore comprises means for performing the method.

[0015] Further advantages, features, and details of the invention derive from the following description of preferred embodiments as well as from the drawings. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figures and/or shown in the figures alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and/or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0017] The drawings show in: [0018] Fig. 1 a schematic block diagram according to an embodiment of the monitoring system; and [0019] Fig. 2 another schematic block diagram according to an embodiment of the monitoring system.

[0020] In the figures the same elements or elements having the same function are indicated by the same reference signs.

DETAILED DESCRIPTION

[0021] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0022] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0023] The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by "comprises" or "comprise" does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0024] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawings that form part hereof, and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0025] Fig. 1 shows a schematic block diagram according to an embodiment of a monitoring system 10. The monitoring system 10 comprises at least one electronic computing device 12, wherein the electronic computing device 12 may comprise in one embodiment an artificial intelligence for analyzing an application update 14. The update 14 is received from a user 16 of an application.

[0026] According to the embodiment shown in Fig. 1, the application update 14 may comprise metadata and data to be supplied to the monitoring system 10. In a first step Si, the monitoring system 10 decides to which mathematical model 18a, 18b, 18c the application update 14 should be routed to. According to this embodiment, the data set 14 is routed to a second mathematical model 18b. In a second step S2, the monitoring system 10 checks, if this model exists. If it does not exist, the monitoring system 10 will create a new mathematical model 18a, 18b, 18c in a third step S3. In particular, the monitoring system 10 uses a strategy engine 20. If the model exists, the monitoring system 10 will predict an output 22 in a fourth step S4. In a fifth step 55, the monitoring system 10 decides if an alert-message has to be generated. If not, a sixth step 56 is performed, wherein the application update 14 is added to a strategy repository 30 of the monitoring system 10. After the sixth step S6, the mathematical model 18a, 18b, 18c, in particular the second mathematical model 18b, may be updated.

[0027] Coming from the fifth step S5, if an alert-message is generated, an eighth step S8 is performed, wherein the root cause is determined. In a ninth step S9, the alert is appended with the root cause information. In a tenth step 510, the alert is published. Coming from the tenth step S10, an eleventh step 511 is performed, wherein the user 16 is asked to confirm the alert-message. If the user 16 confirms the alert-message, which is shown in block 26, the method proceeds to the sixth step 56. If the alert-message is not confirmed, which is shown with block 28, a twelfth step S12 is performed, wherein the method is truncated.

[0028] In particular, Fig. 1 shows that the monitoring system, which may also be referred to as an artificial intelligence system monitor, is a cloud-based application that alerts the user 16 when it detects outlying events given from the user's application. Currently, configuration of these alerts is a manual process with the user 16 making choices of what to alert on, defining an outlying event, what mathematical model 18a, 18b, 18c to use, or furthermore. The monitoring system 10 according to the invention solves this manual process by replacing it with an automated one. The application sends updates 14 to the monitoring system 10, and the monitoring system 10 automatically creates a model 18a, 18b, 18c and monitors the application immediately. The monitoring system 10 solves the problem of manual configuration and saves the user's time and money spent on creating and configuring the monitoring system 10 for the application.

[0029] The two key technologies of the monitoring system 10 used to accomplish its tasks is the monitoring system 10 itself and the strategy engine 20. The monitoring system 10 accepts the incoming application update 14 from the user 16 and predicts whether the user system is behaving correctly based on the mathematical model 18a, 18b, 18c that was constructed. The strategy engine 20 builds the mathematical model 18a, 18b, 18c by determining which strategy succeeds on the application update 14 and which does not. A strategy has two subtypes, it is either a statistical preprocessing technique or a statistical model. Every strategy has a unique evaluator associated with them. The evaluator determines whether the applied strategy is used in the mathematical model 18a, 18b, 18c. The mathematical model 18a, 18b, 18c used for prediction in the monitoring system 10 is a collection of strategies applied to the application update 14.

[0030] The monitoring system 10 starts by the user 16 sending the application update 14. The application update 14 contains two sections. The first is metadata needed to route the application update 14 to the correct model 18a, 18b, 18c, the second is an n-dimensional list of data supplied to the monitoring system 10. The user 16 does not need to configure the monitoring system 10 with anything other than the routing information to get started. The monitoring system 10 dynamically sizes the dimensions based on the input and adjusts the mathematical model 18a, 18b, 18c in real time to accommodate changes in the user input. The user 16 can change the dimensions at any time by providing the same routing information with different fields.

[0031] Once the monitoring system 10 routes the application update to the correct mathematical model 18a, 18b, 18c, creating one if no model is present, it makes a prediction. Predictions are generated from the mathematical model 18a, 18b, 18c, where the mathematical model 18a, 18b, 18c is a set of strategies to apply on the application update 14. If the application update 14 is of statistical significance, it generates the alert-message. This alert-message notifies the user 16 and asks for feedback on whether this was a false positive or a true positive. If the user 16 responds to the monitoring system 10 that a false positive has occurred, the monitoring system 10 updates the entry and learns from the supervision. At the end of this process, the mathematical model 18a, 18b, 18c is recurrently updated to include this new update.

[0032] Fig. 2 shows another schematic block diagram according to an embodiment of the monitoring system 10, in particular of the strategy engine 20. The strategy engine 20 is a component which trains the mathematical model 18a, 18b, 18c on a given application update 14. Its object is to apply the strategies on the application update 14 and decide, based on an associated evaluator, whether to include this strategy in the final mathematical model 18a, 18b, 18c or not. The strategy engine 20 automates the model discovery.

[0033] A strategy is defined as an operation on an application update 14 which transforms one or all of the columns of the data in an application update 14. An evaluator is defined as an operation on a transformed data in an application update 14 by the strategy which produces a Boolean, true or false, on whether the associated strategy is used in the final model 18a, 18b, 18c. The final model 18a, 18b, 18c is defined as a set of strategies which have had their evaluators returned after operating on the data in an application update 14.

[0034] According to Fig. 2, an example might be to consider the data in application update 14, which may also be referred to as D comprising an M x N Matrix. The data might be analyzed to know if a clustering algorithm might produce a better data D" than D by itself. For example, a hypothetical strategy, a k-means clustering algorithm with k = 3 is being tested. The programmer of the strategy engine 20 may write this strategy, which takes the D as the input and produces D', the clustered data, as the output. This is shown in the thirteenth step S13 and in the fourteenth step 314. This can also be thought of as adding a new column to the data D, the cluster ID, making the D" size M x (N+1). The strategy engine programmer may now write an associated evaluator, which compares D to D" in a meaningful way. In the context of this example, the evaluator could be a logistic regression written by the programmer; the evaluated process signals 1 for an alert and 0 for no alert. The evaluator would compute some metric for both D and D1, here we could use the r2-coefficient as a measure of performance between the two matrices as r2 and r2' respectively, this is shown in the fifteenth step S15. Assume r2 < r2' in this case and the strategy applied generates a better mathematical model. This is shown in the figure in the fifteenth step 515 as well as in the sixteenth step S16. The evaluator evaluates these results and returns "true", which puts the strategy into the final model 18a, 18b, 18c. From now on, until the data in the application update 14 is processed again in the strategy engine 20, the incoming application update 14 from the user 16 has the k-means strategy applied to it before it is evaluated for an alert. This is, in particular, shown in the seventeenth step 317.

[0035] After evaluating all strategies on the application update 14 in the manner described above, the final model 18a, 18b, 18c has a set of strategies, which act on the incoming application update 14 from the live application before it is evaluated for alerting. To update the alerting system with the final model 18a, 18b, 18c, the strategy engine 20 posts the update to the alerting system using the metadata and strategy set to inform the current model 18a, 18b, 18c.

[0036] The strategy engine 20 process can be parallelized across all active mathematical models 18a, 18b, 18c in the alerting system. Due to this feature, the strategy engine 20 itself does not decide what to work on, the alerting system or a scheduling component would be in charge of providing the application update 14 to be worked on. The scheduling component might be discern on time, set size or some other metric to queue/dispatch jobs.

[0037] According to Fig. 2, the strategy engine 20 decides in an eighteenth step S18, if there are more strategies needed. If there are no more strategies needed, the strategy engine 20 performs again the thirteenth step S13. If no further strategies are needed, the updated model 18a, 18b, 18c is used in the nineteenth step S19.

Reference Signs monitoring system 12 electronic computing device 14 application update 16 user 18 mathematical model strategy engine 22 output 24 update 26 block 28 block strategy repository Si to S19 steps of the method

Claims

CLAIMS1. A method for monitoring an application by a monitoring system (10), the method comprising the steps of: - receiving an application update (14) from a user (16) of the application; - analyzing the received application update (14) depending on a mathematical model (18a, 18b, 18c) of the monitoring system (10); and - depending on the analyzed application update (14) providing an output (22) to the application.
2. The method according to claim 1, wherein the application update (14) comprises of --a metadata section; and -a data (D) comprising an M x N Matrix.
3. The method according to claim 1, wherein the monitoring system (10) uses artificial intelligence for analyzing the application update (14).
4. The method according to claim 1, wherein the monitoring system (10) asks a user (16) of the monitoring system (10) to confirm the usefulness of the output (22).
5. The method according to claim 4, wherein the confirmation of the output (22) by the user (16) is further used to train the monitoring system (10).
6. The method according to claim 1 or 2, wherein depending on the received application update (14) the monitoring system (10) creates a new mathematical model (18a, 18b, 18c) for analyzing the received application update (14).
7. The method according to claim 2, wherein the mathematical model (18a, 18b, 18c) of the monitoring system (10) is created comprising the steps of: -receiving an application update (14) from a user (16) of the application; -applying a mathematical model (18a, 18b, 18c) on the data (D) to produce better data (D') -evaluating the data (D) and the better data (D') -returning a value based on the evaluation -adding the mathematical model (18a, 18b, 18c) to the monitoring system -applying the mathematical model (18a, 18b, 18c) on a future application update (14) before evaluating for an alert.
8. A monitoring system (10) for monitoring an application, comprising at least one electronic computing device (12), wherein the electronic computing device (10) is configured for perform a method according to claim 1.