WO2022015313A1

WO2022015313A1 - Generation of alerts of correlated time-series behavior of environments

Info

Publication number: WO2022015313A1
Application number: PCT/US2020/042350
Authority: WO
Inventors: Mauricio COUTINHO MORAES; Jhonny Marcos Acordi MERTZ; Nailson BOAZ COSTA LEITE
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2020-07-16
Filing date: 2020-07-16
Publication date: 2022-01-20

Abstract

An example system includes an environment engine to: execute an environment that includes an application or applications. The example system further includes a predefined alert engine to: monitor the environment being executed by the environment engine; and, in response to predefined behavior of the environment, generate a predefined alert. The example system further includes a correlation engine to: determine correlations between the predefined alert and time-series behavior of the environment. The example system further includes a correlation alert engine to: in response to a correlated time-series behavior of the environment occurring, generate an alert.

Description

Generation Of Alerts Of Correlated Time-series Behavior Of Environments

BACKGROUND

[0001] Alerts for behavior of environments, for example that implement applications, may be manually defined, however this relies on a knowledge of the behavior that require such predefined alerts.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Reference will now be made, by way of example only, to the accompanying drawings in which:

[0003] Figure 1 is a block diagram of an example system to generate alerts of correlated time-series behavior of environments.

[0004] Figure 2 is a block diagram of another example system to generate alerts of correlated time-series behavior of environments.

[0005] Figure 3 is a flow diagram of an example method to generate alerts of correlated time-series behavior of environments.

[0006] Figure 4 is a block diagram of an example computer-readable medium including instructions that causes a processor to generate alerts of correlated time-series behavior of environments.

[0007] Figure 5 is an example of a predefined behavior associated with a predefined alert, and time-series behavior correlated therewith.

[0008] Figure 6 is an example of the system of Figure 1 generating an alert in response to detecting a correlated time-series behavior.

DETAILED DESCRIPTION

[0009] Alerts for behavior of environments, for example that implement applications, may be manually defined, however this relies on a knowledge of the behavior that require such predefined alerts. However, such environments are generally implemented in the cloud, such as container-orchestrated environments (COEs), and the like. Applications in such environments generally undergo complex interactions with other applications, for example in cascading application dependencies (e.g. a first application depends on a second application which depends on a third application, etc.).

[0010] As such, there may be behavior which occur in such environments that cause applications to fail that may not be easily detectable and/or discernible, such that an engineer and/or system administrator may not have knowledge of the behavior that leads to failure. In particular, unexpected and/or unknown factors may impact the behavior of applications and/or infrastructure of the environment in unsuspected ways. After receiving a predefined alert, on-call engineers generally use monitoring and alerting tools in a reactive way to troubleshoot emerging and anomalous events in the environment, such as outages or malicious attacks. This process includes looking at system logs, dashboards, and/or correlation systems in order to identify and fix the root causes, which is time consuming and/or labor intensive. In addition to fixing the problem, the engineers may manually create or update alerts to later identify these known events in the future. However, this process again relies on an engineer identifying behavior after it occurs to set predefined alerts.

[0011] Hence, provided herein is a system which determines correlations between predefined alerts and time-series behavior of an environment that includes an application or applications; once such a correlation is determined, the system provides an alert when correlated time-series behavior of the environment occurs, prior to a predefined alert (e.g. a manually predefined alert). Such an alert provided prior to the predefined alert may inform an engineer that the predefined alert, for example for a predefined behavior, is about to occur (and/or that there is a chance that the predefined alert is about to occur and/or that there is a chance that the predefined behavior is about to occur), and the engineer may then proactively take steps to prevent the predefined behavior associated with the predefined alert (and which may also prevent the predefined alert). Alternatively, the system may proactively implement corrective actions to prevent the predefined behavior associated with the predefined alert (and which may also prevent the predefined alert). [0012] An aspect of the present specification provides a system comprising: an environment engine to: execute an environment that includes an application or applications; a predefined alert engine to: monitor the environment being executed by the environment engine; and, in response to predefined behavior of the environment, generate a predefined alert; a correlation engine to: determine correlations between the predefined alert and time-series behavior of the environment; and a correlation alert engine to: in response to a correlated time- series behavior of the environment occurring, generate an alert.

[0013] Another aspect of the present specification provides a method comprising: determining, at a computing device, correlations between: time- series behavior of an environment that includes an application or applications; and a predefined alert of predefined behavior of the environment; and in response to a correlated time-series behavior of the environment occurring, generating, at the computing device, an alert prior to the predefined alert.

[0014] Another aspect of the present specification provides a non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to: execute a correlation module to determine correlations between: time-series behaviors of an application environment; and respective predefined alerts for predefined behaviors occurring at the application environment; and execute a correlation alert module to: in response to correlated time-series behaviors of the application environment occurring, generate respective alerts.

[0015] Figure 1 is a block diagram of an example system 100 to generate alerts of correlated time-series behavior of environments. The system 100 includes an environment engine 111 , a predefined alert engine 113, a correlation engine 115, and a correlation alert engine 117. Communication between components and/or engines described herein is shown in the figures of the present specification as arrows therebetween.

[0016] As used herein, the term “engine” refers to hardware (e.g., a processor, such as a central processing unit (CPU) an integrated circuit or other circuitry) or a combination of hardware and software (e.g., programming such as machine- or processor-executable instructions, commands, or code such as firmware, a device driver, programming, object code, etc. as stored on hardware). Hardware includes a hardware element with no software elements such as an application specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA), etc. A combination of hardware and software includes software hosted at hardware (e.g., a software module that is stored at a processor-readable memory such as random access memory (RAM), a hard disk or solid-state drive, resistive memory, or optical media such as a digital versatile disc (DVD), and/or implemented or interpreted by a processor), or hardware and software hosted at hardware.

[0017] For example, the environment engine 111 may comprise hardware or a combination of software and hardware for implementing functionality to: execute an environment that includes an application or applications, such as websites and applications providing services to websites and the like, however the environment engine 111 may be to implement any suitable application or applications. The environment engine 111 may comprise a portion of a server and/or computing device which hosts the environment engine 111. However, the environment engine 111 may comprise hardware or a combination of software and hardware of any suitable server and/or computing device and/or more than one suitable server and/or computing device.

[0018] In particular, the environment engine 111 may execute an environment such as a cloud-based environment including, but not limited to, a container- orchestrated environment (COE) and/or any other suitable environment. In some examples, such an environment may alternatively be referred to as an application environment and the like.

[0019] Hence, in some examples, the environment engine 111 may include a container-orchestrated environment engine and/or the environment engine 111 may be a component of a container-orchestrated environment engine, and the like. In such examples, the environment engine 111 and/or a container- orchestrated environment engine may be to manage a number of instances of an application or applications as demand for an application or applications (e.g. of the environment being executed by the environment engine 111) changes. As such, the environment engine 111 may execute a plurality of instances of an application, which may change (e.g. increase or decrease) as demand changes, which is referred to hereafter as auto-scaling of an application.

[0020] Furthermore, an application of an environment, being executed by the environment engine 111 , may depend on other applications being executed by the environment engine 111. Such applications may experience behavior that causes the application and/or other applications and/or the environment to experience behavior that may lead to failures and/or problems in the environment.

[0021] Hence, the system 100 further comprises the predefined alert engine 113 to: monitor the environment being executed by the environment engine 111 ; and, in response to predefined behavior of the environment occurring, generate a predefined alert. For example, while not depicted, the system 100 may include a memory and/or database, and the like, storing definitions of predefined behavior of the environment being executed by the environment engine 111 , and associated predefined alerts. Such a predefined behavior definition may comprise a classifier of a machine learning model, and the like; in other words, the predefined alert engine 113 may implement a machine learning model, and the like, to detect predefined behaviors using predefined behavior definitions and/or classifiers.

[0022] Furthermore, the predefined alert engine 113 may detect a predefined behavior via the use of sensors or sensor engines, described in more detail below.

[0023] When the predefined alert engine 113 detects a predefined behavior, as defined by a definition of a predefined behavior, the definitions of predefined behavior may generate a predefined alert which may the cause predefined alert engine 113 to transmit the predefined alert to a notification device or notification devices (e.g. of an engineer and/or engineers and/or a system administrator and/or system administrators, and the like) to cause the notification device(s) to provide an alert of the predefined behavior(s) occurring. Such predefined behavior(s) and predefined alert(s) are understood to have been predetermined by the engineer and/or engineers and/or the system administrator and/or system administrators, and the like, and stored at the memory and/or database.

[0024] The correlation engine 115 is generally to: determine correlations between a predefined alert and time-series behavior of the environment executed by the environment engine 111. For example, the correlation engine 115 may be further to monitor behavior of the environment executed by the environment engine 111 and determine when time-series behavior thereof occurs prior to a predefined alert occurring. Such “time-series behavior” may be understood to include behavior of a parameter of an application and/or applications of the environment, and/or the environment, executed by the environment engine 111 that changes over time and which may be monitored via a “sensor” and the like which may be to monitor a particular parameter. Hence, “time-series behavior” may be understood to include behavior of a parameter of an application and/or applications of the environment, and/or the environment, as the parameter changes over time, such as numeric values of messages and/or status codes, memory usage, processing resource usage, and/or any other suitable parameter.

[0025] In a particular example, a sensor may monitor messages, such as status codes, and the like, generated by applications of the environment including, but not limited to, Hypertext Transfer Protocol (HTTP) status codes. In the particular example, the status codes may include: “200” series status codes, which generally indicate that a request to an HTTP application (e.g. a request from a computing device to a website) has been successfully received and/or accepted; “300” series status codes, which generally indicate further actions that to be taken to complete a request; “400” series status codes, which generally indicate problems and/or errors occurring in association with a request; and/or “500” series status codes, which generally indicate server failures occurring in association with a request.

[0026] Furthermore, the term “sensor” may be understood to mean instructions, and the like, being implemented by hardware of system 100 and/or the correlation engine 115 and/or the predefined alert engine 113, which are to (e.g. periodically and/or over time) query and/or monitor (corresponding code segments and/or hardware of the environment executed by the environment engine 111 to determine the parameter over time.

[0027] Alternatively, while not depicted, such sensors may be implemented by sensor engines, and/or such sensor engines may be components of the correlation engine 115 and/or the predefined alert engine 113. However, the correlation engine 115 may monitor time-series behavior of the environment in any suitable manner which may, or may not, include sensors and/or sensor engines as described above.

[0028] Furthermore, such sensors and/or sensor engines may be used by the predefined alert engine 113 to determine when predefined behavior occurs. Put another way, the predefined behavior detected by the predefined alert engine 113 may comprise predefined behavior (e.g. predefined time-series behavior) of parameters of the environment engine 211 monitored by the sensors or sensor engines, but such monitored parameters may be different from (or similar to) of parameters used to determine correlated time-series behavior by the correlation engine 115.

[0029] In some examples, the correlation engine 113 may comprise a machine learning model trained to determine correlations between time-series behavior and a predefined alert. For example, the machine learning model may include classifiers, and the like, which may be used to identify time-series behaviors including, but not limited, increases and/or decreases in parameters, periodic behavior of parameters, periodic-like behavior of parameters, and/or any other suitable behavior of parameters.

[0030] In general, the correlation engine 115 may store values of parameters generated by the sensors as a function of time, for example, in a memory and/or database. Furthermore, the correlation engine 115 may be in communication with the predefined alert engine 113 to determine when predefined alerts occur. When a predefined alert occurs, the correlation engine 115 may process and/or search the values of parameters generated by the sensors, stored as a function of time, to determine whether any time-series behaviors of the environment correlate with the predefined alert, for example, using the above described machine learning models. In general, a determination of such a correlation includes determining that a time-series behaviors of the environment, occurs prior to the predefined alert.

[0031] For example, generation of status codes 400 or greater may occur when problems are occurring at an application which may be due to problems with requests and/or processing requests due to issues with memory resources, processing resources, etc. Hence, for example, a numeric increase in status codes by a first application may occur prior to the predefined alert, but the predefined alert may occur when a numeric increase in status codes occurs at a second application; as such, the predefined alert may initially appear to be unrelated to the numeric increase in status codes by the first application. However, the correlation alert engine 117 may determine a correlation between: the predefined alert of the (e.g. time-series) numeric increase in status codes of the second application, and a time-series numeric increase of status codes of the first application prior to an occurrence of the predefined alert of the numeric increase in status codes at the second application.

[0032] The correlation engine 115 may store an indication, and the like, of such a correlation at a memory and/or database, and/or pass the indication to the correlation alert engine 117. Such an indication is generally indicative of time- series behavior of the environment that has been correlated with a predefined alert, and which may be referred to hereafter as correlated time-series behavior and which may be defined via a correlated behavior definition. In other words, the correlation engine 115 may store an indication as a correlated behavior definition at the memory and/or database. Such a correlated behavior definition may comprise a classifier of a machine learning model, and the like.

[0033] The correlation alert engine 117 is generally to: in response to a correlated time-series behavior of the environment occurring, generate an alert, for example prior to a correlated predefined alert. Hence, when correlated time- series behavior occurs, the correlation alert engine 117 generates an alert. [0034] The correlation alert engine 117 may be further to generate the alert by: controlling a notification device to provide the alert; or transmitting the alert to the notification device to cause the notification device to provide the alert. As already described, such a notification device may be operated by an engineer who may receive the alert and implement a corrective action to prevent the predefined behavior associated with the predefined alert from occurring.

[0035] In some examples, the correlation engine 115 may be further to determine an estimate (e.g. a probability expressed as a percentage, and the like) of the predefined alert occurring after the correlated time-series behavior. Such an estimate may be determined using the aforementioned machine learning models (e.g. trained, in these examples, to determine such estimates) and/or by determining numbers of occurrences correlated time-series behavior as compared to numbers of occurrences of the predefined alerts. In these examples, the alert provided by the correlation alert engine 117 may include the estimate of the predefined alert occurring. For example, the alert may include text, and the like, indicating that there is a given probability (e.g. an “X%” probability and/or chance) of a predefined alert occurring.

[0036] In further examples, the correlation engine 115 may be further to determine an estimated time period (e.g. a time period and the like) of the predefined alert occurring after the correlated time-series behavior. Such an estimate may be determined using the aforementioned machine learning models (e.g. trained, in these examples, to determine such estimated time periods) and/or by determining time periods between occurrences correlated time-series behavior the predefined alerts. In these examples, the alert provided by the correlation alert engine 117 may include the estimated time period. For example, the alert may include text, and the like, indicating that a predefined alert may occur after the estimated time period and/or at a given time (e.g. determined from the estimated time period and a time at which the correlated time-series behavior occurs).

[0037] In yet further examples, the alert provided by the correlation alert engine 117 may include an indication of a corrective action to be implemented in response to the predefined alert. For example, such a corrective action may be determined by the correlation engine 115 via the aforementioned machine learning models, and the like (e.g. trained, in these examples, to determine corrective actions) and/or based on predefined corrective actions that may be implemented when a particular predefined alert occurs (e.g. and which may be stored with an associated predefined behavior definition).

[0038] Alternatively, the correlation alert engine 117 (and/or another engine, such as a corrective response engine, and the like) may be further to, in response to a correlated time-series behavior of the environment occurring, implement a custom action and/or corrective action and/or an automation script associated with the correlated time-series behavior or the alert and/or the predefined behavior. For example, the custom action and/or corrective action and/or an automation script may include instructions that may be implemented by a server and/or computing device to prevent the predefined behavior associated with the predefined alert. Such a custom action and/or corrective action and/or automation script may include increasing (or decreasing) memory resources and/or increasing (or decreasing) processing resources available for use at the environment engine 111 to execute applications, and/or auto-scaling an application executed by the environment engine 111 (e.g. increase or decrease instances of the application in COE and the like). Such a custom action and/or corrective action and/or automation script may be stored with an associated predefined behavior definition and/or stored with an indication of correlated time-series behavior, and the like.

[0039] Attention is next directed to Figure 2 which is a block diagram of another example system 200 to generate alerts of correlated time-series behavior of environments. The system 200 is substantially similar to the system 100, with like components having like numbers, but in a “200” series rather than a “100” series. For example, the system 200 may include an environment engine 211 , a predefined alert engine 213, a correlation engine 215, and a correlation alert engine 217, which are respectively substantially similar to the environment engine 111 , the predefined alert engine 113, the correlation engine 115, and the correlation alert engine 117. [0040] However, system 200 further includes a plurality of applications 219-1 , 219-2, 219-3 being executed by the environment engine 211 . The applications 219-1 , 219-2, 219-3 are interchangeably referred to hereafter, collectively, as the applications 219 and, generically, as an application 219. While three applications 219 are in the depicted example, the system 200 may include more than three applications 219 or fewer than three applications 219. Furthermore, while one instance of an application 219 is depicted, the system 200 may include more than one instance of an application 219, which may change as auto-scaling of the applications 219 occurs.

[0041] As depicted, one way arrows are shown between the applications 219 which indicate that the application 219-2 is dependent on services and/or data of the application 219-1 , and the application 219-3 is dependent on services and/or data of the application 219-2; however, despite the arrows between the applications 219 being one way, two communication therebetween may occur; for example, the application 219-1 may comprise a log-in authentication application, the application 219-2 may comprise a website application, and the application 219-3 may comprise an analytics application. The application 219-2 may communicate with the application 219-1 to perform authentication of log-ins to a website, and the application 219-3 may communicate with the application 219-2 to collect and analyze data collected at the website. However, any suitable types of applications 219 and/or dependencies and/or services, and the like, are within the scope of the present specification.

[0042] As depicted, the system 200 further comprises sensor engine(s) 221 (e.g. a sensor engine 221 and/or sensor engines 221). While the sensor engine(s) 221 are depicted as separate from other components of the system 200, the sensor engine(s) 221 be implemented as components of the correlation engine 215 and/or as components of another engine of the system 200. For example, the sensor engine(s) 221 may also be used by the predefined alert engine 213 to monitor the behavior of the environment engine 211 to determine when a predefined behavior occurs.

[0043] The sensor engine(s) 221 generally implement the functionality of the sensors as described above.

[0044] Hence, in general, the sensor engines 221 may be to monitor a particular parameter and/or parameters of the environment engine 211 and/or the applications 219 including, but not limited to, status codes, memory usage, processing resource usage, numbers of auto-scaled applications, and the like.

In some examples the parameters monitored by the sensor engines 221 may be predetermined and/or such parameters may be determined using machine learning models, and the like, for example implemented by the correlation engine 215 and/or another engine (e.g. a parameter determination engine, not depicted) which may cause a corresponding sensor engine 221 for measuring a particular parameter to be generated.

[0045] As depicted, the system 200 further comprises a memory 222 in the form of a database which, as depicted, is separate from the other components of the system 200; however, the memory 222 may comprise a component and/or a shared component an engine and/or engines of the system 200 (e.g. the engines 215, 213) and/or the memory 222 may be a component of yet a further engine (e.g. a database engine, and the like, not depicted) of the system 200.

[0046] As depicted, the memory 222 stores predefined behavior definition(s) 223, used by the predefined alert engine 213 to determine when associated predefined behaviors occur at the environment implemented by the environment engine 211 such that a predefined alert may be generated. Similarly, the memory 222 stores correlated behavior definition(s) 225, which may be generated and stored by the correlation engine 215, and which may be used by the correlation alert engine 217 to determine when correlated time-series behaviors occur at the environment implemented by the environment engine 211 , such that an associated alert may be generated.

[0047] For example, also depicted in the system 200 is a notification device 227 which may comprise a mobile device, and the like, of an engineer and/or system administrator, and/or which may include a display screen and/or a speaker and/or haptic device, and the like, to provide alerts and/or notifications thereof. The predefined alert engine 213 may generate a predefined alert at the notification device 227, which may include, but is not limited to, transmitting the predefined alert to the notification device 22, to provide a notification thereof. Similarly, the correlation alert engine 217, in response to a correlated time- series behavior of the environment occurring, may generate an associated alert at the notification device 227, which may include, but is not limited to, transmitting the associated alert to the notification device 227, to provide a notification thereof.

[0048] As mentioned above, the alert may include, but is not limited to, an estimate of the predefined alert occurring after the correlated time-series behavior and/or an estimated time period of the predefined alert occurring after the correlated time-series behavior and/or an indication of a corrective action, and the like.

[0049] It is further understood that, in some examples, the system 100 and/or the system 200 may comprise the correlation engine 115 and/or the correlation engine 215 (with or without the sensor engines 221 ) and the correlation alert engine 117 and/or the correlation alert engine 217, and the remaining components of the system 100 and/or the system 200 be components of an existing system with which the correlation engine 115 and/or the correlation engine 215 (with or without the sensor engines 221 ) and the correlation alert engine 117 and/or the correlation alert engine 217, interact. In other words, the correlation engine 115 and/or the correlation engine 215 (with or without the sensor engines 221 ) and the correlation alert engine 117 and/or the correlation alert engine 217 may be components of a correlation alert system that are separate from an environment engine, a predefined alert engine, a notification device, and the like, of an existing environment and/or predefined alert system, such that the correlation alert system may interact with the existing system to determine and provide alerts of correlated time-series behavior occurring at the existing system. In these examples, the correlation alert system may rely on a respective memory and/or database to store correlated behavior definitions 225, and the like (e.g. different from a memory, such as the memory 222, of the existing system), and/or the correlation alert system may share such a memory with the existing system (e.g. the memory 222), amongst other possibilities. [0050] Referring to Figure 3, a flowchart of an example method 300 to generate alerts of correlated time-series behavior of environments is depicted. In order to assist in the explanation of method 300, it will be assumed that method 300 may be performed with the system 200, and at least partially by a computing device implementing the system 200 and/or a processor thereof. Indeed, the method 300 may be one way in which the system 200 may be configured. Furthermore, the following discussion of method 300 may lead to a further understanding of the system 200, and its various components. Furthermore, it is to be emphasized, that method 300 may not be performed in the exact sequence as shown, and various blocks may be performed in parallel rather than in sequence, or in a different sequence altogether. Furthermore, it is to be emphasized that the method 300 may alternatively be performed with the system 100, and at least partially by a computing device implementing the system 100 and/or a processor thereof.

[0051] Beginning at a block 301 , a computing device determines correlations between: time-series behavior of an environment that includes an application or applications; and a predefined alert of predefined behavior of the environment.

[0052] For example, the block 301 may be implemented by way of the computing device implementing the correlation engine 215. It is understood that, in some of these examples, the computing device may be relying on existing predefined behavior definitions and/or predefined alerts, as described above.

[0053] The block 301 may further include the computing device determining an estimate of the predefined alert occurring after the correlated time-series behavior. Similarly, the block 301 may further include the computing device determining an estimated time period of the predefined alert occurring after the correlated time-series behavior. Similarly, the block 301 may further include the computing device determining a corrective action to be implemented in response to the predefined alert.

[0054] At a block 303, in response to a correlated time-series behavior of the environment occurring, generating, at the computing device, an alert prior to the predefined alert. [0055] For example, the block 303 may be implemented by way of the computing device implementing the correlation alert engine 217.

[0056] Furthermore, the alert may be generated and transmitted to the notification device 227 which provides the alert, and/or a notification thereof, at a display screen of the notification device 227, for example in the form of text and/or graphics which indicate that the alert is occurring prior to an associated predefined alert. The alert and/or the notification thereof, may include an indication of the predefined behavior associated with the predefined alert, and/or the alert and/or the notification thereof, may include an indication of the correlated time-series behavior, for example to inform an engineer, and the like, of behavior of an environment and/or an application thereof, which may be causing problems.

[0057] In yet further examples, the alert and/or the notification thereof, may include a link to a memory at which an indication of the predefined behavior and/or the predefined alert and/or the correlated time-series behavior is stored such that the notification device 227 may be used to open the link (e.g. via a browser application) such that the indication may be reviewed. In other words, the correlation alert engine 217 may further store the indication at such a memory (e.g. the memory 222 and/or another memory) and generate a link to the indication. The engineer operating the notification device 227 may use the notification device 227 (and/or a browser application thereof and/or another device) to open the link and access the indication, and/or receipt of the link in the alert may cause the notification device 227 to automatically open a browser application to access the link and retrieve the indication.

[0058] In yet further examples, the text and/or graphics of the alert may be converted to speech (e.g. by a text-to-speech engine at the notification device 227 and/or of the system 200), and played at a speaker thereof. In yet further examples, the alert, and/or notification thereof, may comprise a haptic alert implemented by a haptic device of the notification device 227; while such a haptic alert may not include details of the predefined behavior and/or the predefined alert and/or the correlated time-series behavior, such a haptic alert may cause the engineer operating the notification device 227 to use the notification device 227, and/or another device, to log-in to a memory (e.g. via a server, and the like) to access the indication of the predefined behavior and/or the predefined alert and/or the correlated time-series behavior, as described above.

[0059] Similar to as described above, in some examples, the alert generated at the block 303 may comprise an estimate of the predefined alert occurring after the correlated time-series behavior. Furthermore, also similar to as described above, the alert may comprise an estimated time period of the predefined alert occurring after the correlated time-series behavior. Furthermore, also similar to as described above, the alert may comprise an indication of a corrective action to be implemented in response to the predefined alert. However, the alert may comprise any suitable combination of indications of such information, and/or any other suitable information.

[0060] At a block 305, which may be optional, the computing device, in response to the correlated time-series behavior of the environment occurring, implements a custom action or a corrective action or an automation script associated with the correlated time-series behavior or the alert, for example to prevent the predefined behavior associated with the predefined alert from occurring, as has already been described.

[0061] Figure 4 is a block diagram of an example device 400 that includes a computer-readable medium 401 and a processor 402. The computer-readable medium 401 includes instructions that, when implemented by the processor 402, cause the processor 402 to generate alerts of correlated time-series behavior of environments.

[0062] The computer-readable medium 401 may be a non-transitory computer- readable medium, such as a volatile computer-readable medium (e.g., volatile RAM, a processor cache, a processor register, etc.), a non-volatile computer- readable medium (e.g., a magnetic storage device, an optical storage device, a paper storage device, flash memory, read-only memory, non-volatile RAM, etc.), and/or the like. [0063] The processor 402 may be a general-purpose processor or special purpose logic, such as a microprocessor (e.g., a central processing unit, a graphics processing unit, etc.), a digital signal processor, a microcontroller, an ASIC, an FPGA, a PAL (programmable array logic), a PLA (programmable logic array), a PLD (programmable logic device), etc. The computer-readable medium 401 or the processor 402 may be distributed among a plurality of computer- readable media or a plurality of processors.

[0064] The computer-readable medium 401 includes modules. As used herein, a “module” (in some examples referred to as a “software module”) is a set of instructions that when implemented or interpreted by a processor or stored at a processor-readable medium realizes a component or performs a method.

[0065] As depicted, the computer-readable medium 401 includes various modules which correspond to functionality of the engines of the system 200.

[0066] In particular the computer-readable medium 401 includes an environment module 411 , which, when processed by the processor 402, may provide the processor 402 with functionality similar to the environment engine 211. For example, the environment module 411 may cause the processor 402 to execute an application environment that includes an application or applications.

[0067] As depicted, the computer-readable medium 401 includes a predefined alert module 413, which, when processed by the processor 402, may provide the processor 402 with functionality similar to the predefined alert engine 213. For example, the predefined alert module 413 may cause the processor 402 to: monitor an environment being executed via the environment module 411 ; and, in response to predefined behavior of the environment, generate a predefined alert.

[0068] As depicted, the computer-readable medium 401 includes the definitions 223, 225, as described above, and which may be stored in a database format.

[0069] In some examples, however, the modules 411 , 413, and the predefined behavior definitions 223, may be optional and rather the modules 411 , 413, and the predefined behavior definitions 223, may be implemented by, and/or stored at, another device that is dedicated to implementing an environment and providing predefined alerts therefor.

[0070] As depicted, the computer-readable medium 401 includes a correlation module 415, which, when processed by the processor 402, may provide the processor 402 with functionality similar to the correlation engine 215. For example, the correlation module 415 may cause the processor 402 to determine correlations between: time-series behaviors of an application environment (e.g. being executed by the environment module 411); and respective predefined alerts for predefined behaviors occurring at the application environment.

[0071] Similar to the correlation engine 215, the correlation module 415 may be further to cause the processor 402 to determine respective estimates of the respective predefined alerts occurring after a respective correlated time-series behavior, the respective alerts including a respective estimate of a respective predefined alert occurring.

[0072] Similar to the correlation engine 215, the correlation module 415 may be further to cause the processor 402 to determine respective estimated time periods of the respective predefined alerts occurring after a respective correlated time-series behavior, the respective alerts including a respective estimated time period.

[0073] Similar to the correlation engine 215, the correlation module 415 may be further to cause the processor 402 to determine respective estimated time periods of the respective predefined alerts occurring after a respective correlated time-series behavior, the respective alerts including a respective estimated time period.

[0074] As depicted, the computer-readable medium 401 includes a correlation alert module 417, which, when processed by the processor 402, may provide the processor 402 with functionality similar to the correlation alert engine 217. For example, the correlation alert module 417may cause the processor 402 to: in response to correlated time-series behaviors of the application environment occurring, generate respective alerts. [0075] As depicted, the computer-readable medium 401 includes sensor module(s) 421 (e.g. a sensor module 421 and/or sensor module 421 ), which, when processed by the processor 402, may provide the processor 402 with functionality similar to the sensor engine 221 . For example, includes sensor module(s) 421 may cause the processor 402 to monitor respective parameters of an application environment being executed by the environment module 411. In some examples, the functionality of the sensor module(s) 421 may be combined with the correlation module 415 and/or another suitable module, and/or the sensor module(s) 421 may be components of the correlation module 415 and/or another suitable module.

[0076] As depicted, the computer-readable medium 401 includes a corrective response module 431 which may cause the processor 402 to, in response to a respective correlated time-series behavior of an application environment occurring, the respective correlated time-series behavior correlated with a respective predefined alert, implement a corrective response to prevent an associated predefined behavior from occurring.

[0077] In a particular example that is specific to COEs, the corrective response module 431 which may be cause the processor 402 to: in response to a respective correlated time-series behavior of the application environment occurring, the respective correlated time-series behavior correlated with a respective predefined alert, auto-scaling an application executed at the application environment. However, such auto-scaling is but one examples of a corrective response that may occur and the corrective response module 431 may cause the processor 402 to implement any suitable corrective response, including, but not limited to, increasing memory resources and/or processing resources of applications of the application environment.

[0078] In some examples, the functionality of the corrective response module 431 may be combined with the correlation alert module 417 and/or another suitable module, and/or the corrective response module 431 may be a component of the correlation alert module 417 and/or another suitable module.

[0079] Attention is next directed to Figure 5, which depicts an example of a predefined behavior associated with a predefined alert, and time-series behavior correlated therewith, as well as an example of a portion of the method 300.

[0080] In particular, three graphs 501 , 502, 503 depicted in Figure 5 respectively show behavior of the applications 219-1 , 219-2, 219-2 being executed by the environment engine 211 , and in particular the three graphs 501 , 502 show generation of particular respective status codes of the applications 219-1 , 219-2, 219-2 as a function of time. The data of the graphs 501 , 502, 503 may be generated via the sensor engines 221.

[0081] For example, in the graph 501 , the status code of the application 219-1 starts at 200, jumps briefly to 300 before returning to 200, and then jumps to above 400 (e.g. to 404), eventually returning to 200).

[0082] Similarly, in the graph 502, the status code of the application 219-2 starts at 200 and then jumps to above 400 (e.g. to 404).

[0083] For example, in the graph 503, the status code of the application 219-3 starts at 200, jumps briefly to above 400 (e.g. to 404), and returns briefly to 200, jumps to above 300, again returns to 200, and then jumps to above 400.

[0084] Also depicted in the graph 502, is a time of a predefined alert 504 that may be generated when a status code generated by the application 219-2 is above 400 for a given period of time due, for example, to problems with memory usage and/or processing resource usage at the application 219-2. Put another way, a predefined behavior definition 223 may include data indicating that, when the status code generated by the application 219-2 is above 400 for a given period of time, the predefined alert 504 is to be generated, for example via the predefined alert engine 213.

[0085] However, the correlation engine 215 (e.g. at the block 301 of the method 300) may determine that a time-series behavior 511 of the application 219-1 correlates with the predefined alert 504, and/or that a time-series behavior 513 of the application 219-3 correlates with the predefined alert 504. For example, the time-series behavior 511 may comprise the status code of the application 219-1 jumping briefly to the 300 from the 200, before dropping back to 200, and then increasing to above 400. Similarly, the time-series behavior 513 may comprise the status code of the application 219-3 jumping briefly to above 400, returning briefly to 200, jumping to above 300, and again returning to 200. Furthermore, while the time-series behaviors 511 , 513 seem unrelated to the predefined alert 504, the correlation engine 215 may determine that over a period of time, the time-series behavior 511 occurs 5 minutes before the predefined alert 90% of the time, and that the time-series behavior 513 occurs 4 minutes before the predefined alert 80% of the time. As such, the correlation engine 215 may generate and store corresponding correlated behavior definitions 225. Put another way, in this example, the correlation engine 215 determines that the time-series behaviors 511 , 513 correlate with the predefined alert 504 and hence the time-series behaviors 511 , 513 comprise correlated time-series behaviors 511 , 513, and corresponding correlated behavior definitions 225 are generated accordingly.

[0086] When the correlated time-series behaviors 511 , 513 later occur, the correlation alert engine 217 (e.g. at the block 303 of the method 300) may generate an alert thereof.

[0087] For example, attention is next directed to Figure 6 which depicts the correlated time-series behavior 511 occurring at the application 219-1. Furthermore, a sensor engine(s) 221 is being used to detect the behavior 511 by monitoring status codes of the application 219-1 and generating data 611 indicative of such behavior (e.g. the data 611 may be similar to the graph 501 , but in a data format rather than a graph format).

[0088] As depicted, the correlation alert engine 217 receives the data 611 and compares the data 611 with a correlated time-series behavior definition 225-1 , which may comprise one of the correlated behavior definitions 225 retrieved from the memory 222. For example, correlated time-series behavior definition 225-1 may define the correlation between the time-series behavior 511 and the predefined alert 504.

[0089] Via the comparison (which may include using a machine learning classifier) of the correlated time-series behavior definition 225-1 and the data 611 , the correlation alert engine 217 determines that the correlated time-series behavior 511 occurred and generates an alert 699 thereof. As depicted, the alert 699 is transmitted to the notification device, which renders the alert 699 as text “Warning: There is a 90% chance the Website Of Application 219-2 may fail in 5 minutes; expect a predefined 404 failure alert”. Hence, for example, the alert 699 includes an estimate of of the website of the application 219-2 failing (e.g. a 90% chance), an estimated time period until such failure occurs (e.g. 5 minutes), and that a predefined alert of the failure may occur, for example unless a corrective action is taken.

[0090] The alert 699 hence alerts an engineer, and the like, operating the notification device 227 to take a corrective action to prevent the failure. Alternatively, the correlation alert engine 217 (and/or a corrective response engine and the like) may implement such a corrective action.

[0091] In other examples, the alert 699 may include an indication of the behavior 511 of the application 219-1 that resulted to the alert 699 (e.g. “The log-in application 219-1 experienced a jump to a 300 status code and then a jump to a 404 status code). In these examples the engineer may generate a predefined behavior definition 223 thereof such that a predefined alert thereof may be later generated. Such a new predefined behavior definition 223 and new predefined alert may cause the correlation engine 215 to determine correlations between the new predefined alert and other behavior of the environment being executed by the environment engine 211.

[0092] It should be recognized that features and aspects of the various examples provided above may be combined into further examples that also fall within the scope of the present disclosure.

Claims

1. A system comprising: an environment engine to: execute an environment that includes an application or applications; a predefined alert engine to: monitor the environment being executed by the environment engine; and, in response to predefined behavior of the environment, generate a predefined alert; a correlation engine to: determine correlations between the predefined alert and time-series behavior of the environment; and a correlation alert engine to: in response to a correlated time-series behavior of the environment occurring, generate an alert.

2. The system of claim 1 , wherein the correlation alert engine is further to generate the alert by: controlling a notification device to provide the alert; or transmitting the alert to the notification device to cause the notification device to provide the alert.

3. The system of claim 1 , wherein the correlation engine comprises a machine learning model trained to determine the correlations between the time- series behavior and the predefined alert.

4. The system of claim 1 , wherein the correlation engine is further to determine an estimate of the predefined alert occurring after the correlated time- series behavior, the alert including the estimate of the predefined alert occurring.

5. The system of claim 1 , wherein the correlation engine is further to determine an estimated time period of the predefined alert occurring after the correlated time-series behavior, the alert including the estimated time period.

6. A method comprising: determining, at a computing device, correlations between: time-series behavior of an environment that includes an application or applications; and a predefined alert of predefined behavior of the environment; and in response to a correlated time-series behavior of the environment occurring, generating, at the computing device, an alert prior to the predefined alert.

7. The method of claim 6, wherein the alert comprises an estimate of the predefined alert occurring after the correlated time-series behavior.

8. The method of claim 6, wherein the alert comprises an estimated time period of the predefined alert occurring after the correlated time-series behavior.

9. The method of claim 6, wherein the alert comprises an indication of a corrective action to be implemented in response to the predefined alert.

10. The method of claim 6, further comprising, in response to the correlated time-series behavior of the environment occurring, implementing, at the computing device, a custom action or corrective action or an automation script associated with the correlated time-series behavior or the alert.

11. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to: execute a correlation module to determine correlations between: time- series behaviors of an application environment; and respective predefined alerts for predefined behaviors occurring at the application environment; and execute a correlation alert module to: in response to correlated time- series behaviors of the application environment occurring, generate respective alerts.

12. The non-transitory computer-readable medium of claim 11 , wherein: the correlation module is further to cause the processor to determine respective estimates of the respective predefined alerts occurring after a respective correlated time-series behavior, the respective alerts including a respective estimate of a respective predefined alert occurring.

13. The non-transitory computer-readable medium of claim 11 , wherein: the correlation module is further to cause the processor to determine respective estimated time periods of the respective predefined alerts occurring after a respective correlated time-series behavior, the respective alerts including a respective estimated time period.

14. The non-transitory computer-readable medium of claim 11 , wherein the instructions, when executed by the processor, further cause the processor to: execute a response module to, in response to a respective correlated time-series behavior of the application environment occurring, the respective correlated time-series behavior correlated with a respective predefined alert, implement a corrective response to prevent an associated predefined behavior from occurring.

15. The non-transitory computer-readable medium of claim 11 , wherein: execute a response module to, in response to a respective correlated time-series behavior of the application environment occurring, the respective correlated time-series behavior correlated with a respective predefined alert, auto-scaling an application executed at the application environment.