JP2023547354A - Share insights between pre- and post-deployment to enhance cloud workload security - Google Patents

Abstract

Embodiments may provide techniques that may provide more accurate and actionable alerts by cloud workload security systems to improve overall cloud workload security. For example, in one embodiment, the method may be implemented in a computer system that includes a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor. The method includes generating performance and security information related to a software system during development of the software system and generating performance and security information related to the software system during operation after deployment of the software system. performance and security information that should be escalated by checking the performance and security information generated during the development of the software system with the performance and security information generated during the post-deployment operation of the software system. The methods may include determining warnings and reporting escalated performance and security warnings.

Description

The present invention relates to techniques that may provide more accurate and actionable alerts by cloud workload security systems to improve overall cloud workload security.

Cloud workload security is one of today's fastest growing security concerns. Alerts generated by run-time workload security systems often have a high rate of false positives. A high false positive rate may cause the system to ignore many positive alerts and miss some true positives. As a result, many attacks are able to evade security defenses. Furthermore, pre-deployment security testing suffers from the same issues, resulting in a high number of false positive warnings in pre-deployment security testing, where sometimes true positives (legitimate warnings) are ignored or unverified, or Sometimes it's both.

Thus, there is a need for techniques that can provide more accurate and actionable alerts by cloud workload security systems to improve overall cloud workload security.

Embodiments may provide techniques that may provide more accurate and actionable alerts by cloud workload security systems to improve overall cloud workload security.

Embodiments combine development testing (pre-deployment) and run-time (post-deployment) monitoring of cloud software systems to provide more accurate and actionable alerts, thereby providing a comprehensive overview of the pre-deployment software environment and the post-deployment software environment. You can share insights with your environment. For example, insights can be shared using processes such as application log analysis. Similarly, call graphs can be used to share local log sequences. Additionally, error handling can be used as an indication of security weaknesses.

For example, in one embodiment, the method can be implemented in a computer system that includes a processor, a memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method , generating performance and security information related to the software system during the development of the software system; and generating performance and security information related to the software system during post-deployment operation of the software system. and performance and security information that should be escalated by matching performance and security information generated during software system development with performance and security information generated during the software system's deployed operation. The methods may include determining warnings and reporting escalated performance and security warnings.

In embodiments, generating performance and security information related to a software system during development of the software system includes performing static and dynamic testing of code of the software system during development of the software system. By collecting a history of warnings generated during testing, and by performing dynamic testing of subsystems of the software system's code during the development of the software system, generating logs and error handling logs for warnings produced by performing dynamic testing of units of code of the software system during development of the software system; may include at least one. The method may further include associating each generated alert with the intermediate log sequence using the generated call graph representing calling relationships between routines in the software system. Generating performance and security information related to the software system during post-deployment operation of the software system may include monitoring the post-deployment operation of the software and collecting alerts generated by the monitoring. . Generating performance and security information related to a software system during post-deployment operation of the software system may include collecting information about configuration changes and/or alerts about configuration changes. By comparing the collected configuration change information and/or configuration change warnings with performance and security information generated during post-deployment operation of the software system, Including determining performance and security alerts to escalate. Matching performance and security information generated during software system development with performance and security information generated during post-deployment operation of the software system uses the history of alerts generated during testing. to determine the root cause of the alert generated by the monitoring and when the root cause of the alert generated by the monitoring corresponds to recent alerts in the history of alerts generated during the test; It may include escalating alerts generated by monitoring, or escalating both alerts generated by monitoring and alerts in a history of alerts generated during testing. Matching the root cause of an alert generated by monitoring to recent alerts in the alert history can be performed based on similar log sequences or log groups and using machine learning techniques. Matching performance and security information generated during the development of a software system with performance and security information generated during post-deployment operation of the software system is based on similar log sequences or log groups. , and is performed using machine learning techniques to match the root cause of alerts generated by monitoring with recent alerts in the alert history and application logs using recent time windows. By examining the Unit Test Error Handling log for warnings generated during testing with warnings produced by monitoring, searching for correspondences with known log signatures of the problem, and checking for at least one Escalating the severity of the monitoring-generated alert when a correspondence is discovered and correlating information related to the monitoring-generated alert with the source code that gave rise to the alert generated during testing. It may further include.

In one embodiment, a system includes a processor, a memory accessible by the processor, and computer program instructions stored in the memory that cause the processor to generate performance and security information related to a software system. Generating during the development of the system and performance and security information related to the software system during operation after deployment of the software system and performance and security information generated during the development of the software system. and determining performance and security alerts to escalate by matching the information with performance and security information generated during post-deployment operation of the software system. and the computer program instructions.

In one embodiment, a computer program product can include non-transitory computer readable storage in which program instructions are embodied, the program instructions are executable by a computer, and the program instructions are associated with a software system. generating performance and security information related to the software system during development of the software system; generating performance and security information related to the software system during operation after deployment of the software system; Determining performance and security alerts to escalate by matching performance and security information generated during development with performance and security information generated during post-deployment operation of the software system. A method including:

The details of the invention, both regarding its structure and operation, may be best understood by reference to the accompanying drawings, in which like reference numbers and designations refer to like elements.

1 is a diagram illustrating an example environment in which cloud workload applications may be developed, tested, and used in production deployment, according to embodiments of the present technology; FIG. FIG. 2 is an exemplary flow diagram of a process of operation according to embodiments of the present technology. 1 is an example block diagram of a computer system in which processes implicit in embodiments described herein may be implemented; FIG.

Embodiments may provide techniques that may provide more accurate and actionable alerts by cloud workload security systems to improve overall cloud workload security.

Embodiments combine development testing (pre-deployment) and run-time (post-deployment) monitoring of cloud software systems to provide more accurate and actionable alerts, thereby providing a comprehensive overview of the pre-deployment software environment and the post-deployment software environment. Share insights with your environment. For example, processes such as application log analysis may be used to share insights. Similarly, call graphs may be used for local log sequence sharing. Additionally, error handling may be used as an indication of security weaknesses.

Continuous integration (CI) is a coding technique that implements small changes to code and allows frequent check-ins to a version control repository of a version control system. CI thus allows for a consistent and automated way to build, package, and test applications. This allows for more frequent commitments to code changes, which leads to better collaboration and software quality. Continuous delivery (CD) automates the delivery of applications to selected infrastructure environments. Most teams work in multiple non-production environments, such as development and test environments, and CD ensures an automated way to push code changes to those environments. Continuous integration and continuous delivery (CI/CD) combines these two technologies into an integrated coding and testing workflow.

An example environment 100 in which cloud workload applications may be developed, tested, and used in production deployment is shown in FIG. In this example, environment 100 may include a CI/CD development environment 102 and a cloud production deployment 104. CI/CD development environment 102 may include processing blocks such as code merge block 106, static test block 108, build block 110, dynamic test block 112, and deployment block 114. Code merge block 106 performs code merging (also referred to as integration), which is the act of reconciling multiple changes made to software source code files under version control. It is possible. For example, if a file is modified on two independent branches, the different branches can be merged to form a single collection of files that contains the entire set of changes. In embodiments, automatic merging may be implemented by a version control system in the CI/CD development environment 102 to reconcile changes that occur at the same time (in a logical sense). If automatic merging is ineffective or inoperable, manual merging can be performed if necessary.

Static testing block 108 tests a merged version of computer software, e.g., source code and/or object code output from code merging block 106, without actually running the program. and analysis. In embodiments, such analysis may be performed by automated tools. Build block 110 performs building of a complete application or software system, including compiling source code to object code, linking object code and library object code and other software resources, etc. By doing so, you can create a set of executable code. Dynamic testing block 112 can monitor the actual behavior of the software during operation by running programs on real or virtual processors and performing analysis of the computer software. Deployment block 114 performs the deployment of computer software, including, for example, the generation of an installation package that allows the software to be easily installed, and the installation of the software and associated information required for operation. Includes software installation, etc.

A cloud production deployment 104 can provide an environment for live or production operation of computer software applications and systems with which users (people, client computers, other servers, etc.) can interact directly. environment can be provided. Cloud production deployment 104 may include cloud workload monitoring block 116 and CI/CD insights 118. Cloud workload monitoring block 116 may monitor the behavior of deployed software and workload parameters in cloud production deployment 104 and generate alerts based on the monitored behavior. CI/CD insights 118 can include information about the behavior of deployed software obtained from CI/CD development environment 102, such as alert history, which cloud workload monitoring block 116 uses to Utilization can provide enhanced information regarding the behavior of deployed software in cloud production deployment 104.

An exemplary process 200 of operation according to this embodiment is shown in FIG. That FIG. 2 is best considered in conjunction with FIG. Process 200 begins at 202, where CI/CD tests are performed and alert history from such tests may be stored, for example, within CI/CD Insight 118. Such CI/CD testing may include, for example, static tests such as configuration tests and code tests performed by static test block 108 and dynamic tests performed by dynamic test block 112. can. During static testing, a call graph can be constructed to represent the calling relationships between routines in the computer software. All generated warnings can be associated with possible intermediate log sequences (finite steps) as documented in the call graph. At 204, a CI/CD subsystem dynamic test may be performed to collect application logs for generated alerts. The collected application logs may be stored within CI/CD Insight 118, for example. At 206, CI/CD unit testing may be performed to collect error handling logs that may include sequences indicating unexpected control flow. Additionally, information about configuration changes and/or warnings about configuration changes can be collected. In addition, characterization of error handling unit test log signatures can be performed. The collected error handling logs and characterization may be stored within CI/CD Insight 118, for example.

After the software has been deployed to the cloud production deployment 104, ie, post-deployment, the cloud workload monitoring block 116 monitors the operation of the cloud production deployment 104 at 208 and reports any alerts generated by such monitoring. can be collected. When such workload warnings occur, recent log anomalies can be detected by examining application logs using recent time windows. At 210, root cause analysis of each workload alert is performed using the CI/CD alert history 118. At 212, if the workload alert root cause corresponds to a recent alert in the CI/CD alert history 118, the alert score of the workload alert or of both the workload alert and the alert in the CI/CD alert history 118. may be escalated and root cause analysis results may be reported, e.g., by posting, e.g., to a CI on github(R). For example, process 200 can connect to git commit based on code commit. That is, information related to the warning may be associated with source code determined to be causing the warning in the version control system of the CI/CD development environment 102.

Correspondence between workload alerts and alerts discovered during static analysis (CI/CD alert history 118) can be determined based on similar log sequences (or log groups). For example, matching techniques can be used to determine correspondences, which can include machine learning techniques and even techniques such as one-to-one matching, sequence alignment, DNA comparison, and the like. For example, the matcher is trained to correlate static test results, configuration changes, git logs, and configuration alert logs during the development phase with sequential application log data from the testing and initial deployment phases; It can be done by a DNN (deep neural network) classifier that attempts to do so. These can then be matched in time with warnings during runtime. At 214, unit test error handling logs may be checked against workload warnings. Embodiments may perform such a match by examining the application logs collected at 204 using a recent time window. Then, at 206, the process 200 can search for correspondences with known log signatures of the scenario in question. Once one or more such correspondences are discovered, the severity of the workload warning can be escalated or increased and can be reported, for example by being tied to a code. That is, information related to the warning may be associated with source code determined to be causing the warning in the version control system of the CI/CD development environment 102. Additionally, warnings discovered during the post-deployment stage may be matched with information about configuration changes and/or warnings about configuration changes collected during CI/CD testing.

An exemplary block diagram of a computer system 300 in which the processes implicit in the embodiments described herein may be implemented is shown in FIG. Computer system 300 includes one or more programmed general-purpose computers, such as embedded processors, systems on chips, personal computers, workstations, server systems, and minicomputers or mainframe computers. system or in a distributed, networked computing environment. Computer system 300 may include one or more processors, CPUs 302A-302N, input/output circuitry 304, network adapter 306, and memory 308. CPUs 302A-302N execute program instructions to implement the functionality of the present communication systems and methods. Typically, CPUs 302A-302N are one or more microprocessors. FIG. 3 illustrates computer system 300 as a single multi-processor computer system in which multiple processors CPUs 302A-302N share system resources such as memory 308, input/output circuitry 304, and network adapter 306. 1 illustrates one embodiment that may be implemented. However, the present communication systems and methods further include computer system 300 as a plurality of networked computer systems, which may be a single processor computer system, a multi-processor computer system, or a mixture thereof. Embodiments include embodiments in which the following is realized.

Input/output circuitry 304 provides the ability to input data to or output data from computer system 300 . For example, input/output circuits include input devices such as keyboards, mice, touch pads, trackballs, scanners, analog-to-digital converters, etc., output devices such as video adapters, monitors, printers, etc., and output devices such as modems, etc. It may include input/output devices such as. Network adapter 306 interfaces computer system 300 with network 310 . Network 310 may be any public or private LAN or WAN, including but not limited to the Internet.

Memory 308 stores program instructions executed by and data used and processed by CPUs 302A-302N to implement the functions of computer system 300. The memory 308 may include, for example, an integrated drive electronics (IDE) interface, or a variation or extension of the IDE interface, such as Enhanced IDE (EIDE) or Ultra Direct Memory Access (UDMA), or a small A computer system interface (SCSI)-based interface, or a variation or extension of that SCSI-based interface, such as Fast SCSI, Wide SCSI, Fast and Wide SCSI, etc., or Serial Advanced Technology. - Random Access Memory (RAM), read-only memory that may use an Attachment (SATA), or a variation or extension of SATA, or a Fiber Channel Arbitrated Loop (FC-AL) interface. electronic memory devices such as (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), flash memory, and others; and magnetic disk drives, tape drives, optical disk drives, etc. It may include electromechanical memory such as drives, etc.

The contents of memory 308 may vary depending on the functions that computer system 300 is programmed to perform. In the example shown in FIG. 3, exemplary memory contents are shown representing routines and data for the embodiments of the processes described above. However, these routines, along with their associated memory contents, may not be contained on a single system or device, but rather may be contained on multiple systems or devices based on well-known engineering considerations. Those skilled in the art will recognize that the information may be distributed among different systems or devices. The present systems and methods may include any such arrangement.

In the example shown in FIG. 3, memory 308 includes CI/CD routines 312, which may include code merge routines 314, static test routines 316, build routines 318, dynamic test routines 320, and deployment routines 322; A cloud production deployment 324 may include cloud workload monitoring routines 326 , CI/CD insight data 328 , and an operating system 330 . CI/CD routines 312 may include software routines for implementing CI/CD development environment 102 shown in FIG. 1, as described above. Code merge routine 314 performs code merging (as described above), which is an operation that reconciles multiple changes made to software source code files under version control. It may also include software routines for performing the integration (also referred to as integration). Static test routines 316 may include software routines for performing testing and analysis of computer software without actually executing a program, such as those described above. Build routine 318 performs the compilation of source code into object code, object code and library object code, and other components to form a set of code that can be executed, as described above. It may include software routines for implementing the building of a complete application or software system, including linking software resources, etc. Dynamic testing routine 320 performs an analysis of computer software, such as that described above, performed by running a program on a real or virtual processor to monitor the actual behavior of the software in operation. may include software routines to do so. Deployment routine 322 includes, for example, the generation of an installation package that allows the software to be easily installed, and the installation of associated software required for operation, as described above. may include software routines for implementing deployment of computer software, including deployment, etc. Cloud production deployment 324 may include software routines for providing an environment for active or production operation of computer software applications and systems, as described above, and for providing an environment for users (people, clients, etc.). computers, other servers, etc.) may directly interact with each other. Cloud workload monitoring routines 326 may include software routines for implementing monitoring the operation and workload parameters of deployed software in a cloud production deployment, as described above; Alerts may be generated based on monitored behavior. CI/CD insight data 328 may include information regarding the behavior of deployed software obtained from CI/CD development environment 102, such as warning history, as described above, and may include information about the behavior of deployed software, such as warning history. , can be utilized to provide increased information about the operation of cloud production deployments. Operating system 330 may provide overall system functionality.

As shown in FIG. 3, the present communication systems and methods operate on one or more systems that provide multi-processor, multi-tasking, multi-process, and/or multi-threaded computing. and implementations on systems that provide only single-processor, single-threaded computing. Multi-processor computing entails using two or more processors to perform computing. Multi-tasking computing entails performing computing using two or more operating system tasks. A task is an operating system concept that refers to a combination of programs being executed by the operating system and bookkeeping information used. Whenever a program is run, the operating system creates a new task for that program. A task is like an envelope for a program in that it identifies the program by task number and attaches other bookkeeping information to the program. Many operating systems have the ability to run many tasks simultaneously and are called multitasking operating systems. Multitasking is the ability of an operating system to run two or more executable files simultaneously. Each executable runs within its own address space, which means that the executables have no way of sharing any of their memory. do. This has an advantage, since it is not possible for any program to damage the execution of any other programs running on the system. However, programs have no way of exchanging any information except through the operating system (or by reading files stored on the file system). Multi-process computing is similar to multi-tasking computing, as the terms task and process are often used interchangeably, although some operating systems draw a distinction between the two. .

The invention may be a system, method, and/or computer program product at any possible level of technical detail of integration. A computer program product may include a computer readable storage medium (or media) having computer readable program instructions for causing a processor to implement aspects of the invention. A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.

The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Not done. A non-exhaustive enumeration of more specific examples of computer readable storage media follows: portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable Read Only Memory (EPROM or Flash Memory), Static Random Access Memory (SRAM), Portable Compact Disk Read Only Memory (CD-ROM), Digital Versatile Disk (DVD), Memory Stick ( R), floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with recorded instructions, and any suitable combinations of the foregoing. A computer-readable storage medium, as used herein, refers essentially to radio waves or other free-propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., fiber optic cables). It should not be construed to be a transient signal, such as a pulse of light traveling through a wire, or an electrical signal transmitted through a wire.

The computer-readable program instructions described herein may be transmitted from a computer-readable storage medium to a respective computing/processing device or over a network, such as the Internet, a local area network, a wide area network, or a wireless network. It may be downloaded to an external computer or external storage device over a network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or combinations thereof. A network adapter card or network interface within each computing/processing device receives computer readable program instructions from the network for storage within a computer readable storage medium within the respective computing/processing device. , transmitting computer readable program instructions.

Computer-readable program instructions for carrying out the operations of the present invention include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, and instructions for integrated circuits. Configuration data or one or more programming languages, including object-oriented programming languages such as Smalltalk®, C++, or the like, and procedural programming languages such as the "C" programming language or similar programming languages. can be either source code or object code, written in any combination of . The computer-readable program instructions may be provided entirely on a user's computer, partially on a user's computer, as a stand-alone software package, partially on a user's computer, and partially on a remote computer. or may be executed entirely on a remote computer or server. In the last scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be external It can be done to a computer (eg, over the Internet using an Internet service provider). In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), is used to personalize the electronic circuit to implement aspects of the invention. Computer-readable program instructions may be executed by utilizing state information of the computer-readable program instructions to do so.

Aspects of the invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions are such that the instructions executed by a processor of a computer or other programmable data processing device perform the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams. A processor of a general purpose computer, special purpose computer, or other programmable data processing device may be provided to produce a machine for producing the means for. These computer readable program instructions further include instructions for a computer readable storage medium having stored instructions to implement aspects of the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams. stored in a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a discrete manner so as to constitute a component of an article of manufacture containing can be done.

Computer-readable program instructions further include instructions for execution on a computer, other programmable apparatus, or other device to perform the functions/acts specified in one or more blocks of flowcharts and/or block diagrams. A computer, other programmable apparatus, or other device for causing a sequence of operational steps to be performed on a computer, other programmable apparatus, or other device to produce a computer-implemented process. programmable data processing apparatus or other device.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions noted in the blocks may appear out of the order noted in the figures. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. . Each block in the block diagrams and/or flowchart diagrams, and combinations of blocks in the block diagrams and/or flowchart diagrams, perform designated functions or acts, or implement a combination of specialized hardware and computer instructions. It is further pointed out that this can be realized by a dedicated hardware-based system.

Although particular embodiments of the invention have been described, it will be understood by those skilled in the art that there are other embodiments that are equivalent to the described embodiments. It is therefore to be understood that the invention is to be limited not by the particular illustrated embodiments, but only by the scope of the appended claims.

Claims (20)

A method implemented in a computer system including a processor, a memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising:
generating first performance and security information related to a software system during development of the software system;
generating second performance and security information related to the software system during post-deployment operation of the software system;
by matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system; , determining one or more performance and security alerts to escalate;
and reporting escalated performance and security alerts. Generating the first performance and security information related to the software system during the development of the software system comprises:
collecting a history of one or more generated warnings by performing static and dynamic testing of code of the software system during the development of the software system;
one about the one or more generated warnings by performing a first dynamic test of one or more subsystems of code of the software system during the development of the software system; or generate multiple application logs, and
one or more of the one or more generated warnings by performing a second dynamic test of one or more units of code of the software system during the development of the software system; 2. The method of claim 1, comprising at least one of generating a plurality of error handling logs. Intermediate each of the one or more generated alerts using one or more generated call graphs representing one or more calling relationships between routines in the software system. 3. The method of claim 2, further comprising associating with a log sequence. generating the second performance and security information related to the software system during the post-deployment operation of the software system includes monitoring the post-deployment operation of the software; 4. The method of claim 3, comprising collecting one or more monitor alerts generated by monitoring post-deployment operations. Generating the second performance and security information related to the software system during the post-deployment operation of the software system includes information regarding one or more configuration changes; collecting at least one of one or more generated alerts regarding a plurality of configuration changes, the collating includes collecting at least one of the one or more generated alerts regarding the one or more configuration changes; the collected information regarding at least one of the one or more generated alerts regarding configuration changes to the second performance and security information generated during the post-deployment operation of the software system; 5. The method of claim 4, comprising determining performance to escalate and the one or more performance and security warnings by comparing the performance and security warnings to the performance and security warnings. Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system. ,
using the history of alerts generated during the static and dynamic testing to determine the root cause of alerts generated by monitoring the post-deployment behavior of the software;
the root cause of the alert generated by monitoring the post-deployment behavior of the software corresponds to a recent alert in the history of alerts generated during the static and dynamic testing; in response, escalating the alert generated by monitoring the post-deployment behavior of the software, or escalating the alert generated by monitoring the post-deployment behavior of the software; 5. The method of claim 4, comprising: escalating both alerts in the alert history generated during static and dynamic testing. by matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system; , determining the one or more performance and security alerts to escalate comprises:
said software based on at least one of one or more similar log sequences and one or more log groups and implemented using one or more machine learning techniques; matching the root cause of the alert generated by monitoring the post-deployment behavior of the alert to the recent alert in the alert history;
one or more unit test error handling logs for warnings generated during the static and dynamic tests by examining the one or more application logs using a recent time window; to one or more monitor alerts produced by monitoring the deployed operation of the software, searching for one or more correspondences with known log signatures of the problem, and at least one escalating the severity of the alert generated by monitoring the deployed behavior of the software if one anaphora is discovered;
information related to the one or more alerts generated by monitoring the post-deployment behavior of the software that caused the one or more alerts generated during the static and dynamic testing; 7. The method of claim 6, further comprising: associating with source code that causes the process to occur. A system comprising a processor, a memory accessible by the processor, and computer program instructions stored in the memory and executed by the processor:
generating first performance and security information related to a software system during development of the software system;
generating second performance and security information related to the software system during post-deployment operation of the software system;
by matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system; , determining one or more performance and security alerts to escalate;
and reporting escalated performance and security alerts. Generating the first performance and security information related to the software system during the development of the software system comprises:
collecting a history of one or more generated warnings by performing static and dynamic testing of code of the software system during the development of the software system;
one about the one or more generated warnings by performing a first dynamic test of one or more subsystems of code of the software system during the development of the software system; or generate multiple application logs, and
one or more of the one or more generated warnings by performing a second dynamic test of one or more units of code of the software system during the development of the software system; 9. The system of claim 8, comprising at least one of generating a plurality of error handling logs. Intermediate each of the one or more generated alerts using one or more generated call graphs representing one or more calling relationships between routines in the software system. 10. The system of claim 9, further comprising associating with a log sequence. generating the second performance and security information related to the software system during the post-deployment operation of the software system includes monitoring the post-deployment operation of the software; 11. The system of claim 10, comprising collecting one or more monitor alerts generated by monitoring post-deployment operations. Generating the second performance and security information related to the software system during the post-deployment operation of the software system includes information regarding one or more configuration changes; collecting at least one of one or more generated alerts regarding a plurality of configuration changes, the collating includes collecting at least one of the one or more generated alerts regarding the one or more configuration changes; the collected information regarding at least one of the one or more generated alerts regarding configuration changes to the second performance and security information generated during the post-deployment operation of the software system; 12. The system of claim 11, comprising determining performance to escalate and the one or more performance and security alerts by comparing the performance and security warnings to the performance and security warnings. Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system. ,
using the history of alerts generated during the static and dynamic testing to determine the root cause of alerts generated by monitoring the post-deployment behavior of the software;
the root cause of the alert generated by monitoring the post-deployment behavior of the software corresponds to a recent alert in the history of alerts generated during the static and dynamic testing; in response, escalating the alert generated by monitoring the post-deployment behavior of the software, or escalating the alert generated by monitoring the post-deployment behavior of the software; and escalating both the alerts in the alert history generated during static and dynamic testing. by matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system; , determining the one or more performance and security alerts to escalate comprises:
said software based on at least one of one or more similar log sequences and one or more log groups and implemented using one or more machine learning techniques; matching the root cause of the alert generated by monitoring the post-deployment behavior of the alert to the recent alert in the alert history;
one or more unit test error handling logs for warnings generated during the static and dynamic tests by examining the one or more application logs using a recent time window; to one or more monitor alerts produced by monitoring the deployed operation of the software, searching for one or more correspondences with known log signatures of the problem, and at least one escalating the severity of the alert generated by monitoring the deployed behavior of the software if one anaphora is discovered;
information related to the one or more alerts generated by monitoring the post-deployment behavior of the software that caused the one or more alerts generated during the static and dynamic testing; 14. The system of claim 13, further comprising: associating with source code that causes A computer program product comprising a non-transitory computer readable storage embodying program instructions, the program instructions being executable by a computer, the program instructions being executable by a computer;
generating first performance and security information related to a software system during development of the software system;
generating second performance and security information related to the software system during post-deployment operation of the software system;
by matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system; , determining one or more performance and security alerts to escalate;
and reporting escalated performance and security alerts. Generating the first performance and security information related to the software system during the development of the software system comprises:
collecting a history of one or more generated warnings by performing static and dynamic testing of code of the software system during the development of the software system;
one about the one or more generated warnings by performing a first dynamic test of one or more subsystems of code of the software system during the development of the software system; or generate multiple application logs, and
one or more of the one or more generated warnings by performing a second dynamic test of one or more units of code of the software system during the development of the software system; 16. The computer program product of claim 15, comprising at least one of generating a plurality of error handling logs. each of the one or more generated alerts using one or more generated call graphs representing one or more calling relationships between routines in the software system; 17. The computer program product of claim 16, further comprising associating an alert with an intermediate log sequence. Generating the second performance and security information related to the software system during the post-deployment operation of the software system includes information regarding one or more configuration changes; collecting at least one of one or more generated alerts regarding a plurality of configuration changes, the collating includes collecting at least one of the one or more generated alerts regarding the one or more configuration changes; the collected information regarding at least one of the one or more generated alerts regarding configuration changes to the second performance and security information generated during the post-deployment operation of the software system; 18. The computer program product of claim 17, comprising determining performance to escalate and the one or more performance and security alerts by comparing the performance and security warnings to the performance and security warnings. Matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system. ,
using the history of alerts generated during the static and dynamic testing to determine the root cause of alerts generated by monitoring the post-deployment behavior of the software;
the root cause of the alert generated by monitoring the post-deployment behavior of the software corresponds to a recent alert in the history of alerts generated during the static and dynamic testing; in response, escalating the alert generated by monitoring the post-deployment behavior of the software, or escalating the alert generated by monitoring the post-deployment behavior of the software; escalating both warnings in the warning history generated during static and dynamic testing, and escalating the root causes of warnings generated by monitoring the post-deployment behavior of the software 19. The computer program product of claim 18, wherein matching against recent alerts in the alert log history is performed using machine learning techniques based on similar log sequences or log groups. by matching the first performance and security information generated during the development of the software system with the second performance and security information generated during the post-deployment operation of the software system; , determining the one or more performance and security alerts to escalate comprises:
said software based on at least one of one or more similar log sequences and one or more log groups and implemented using one or more machine learning techniques; matching the root cause of the alert generated by monitoring the post-deployment behavior of the alert to the recent alert in the alert history;
one or more unit test error handling logs for warnings generated during the static and dynamic tests by examining the one or more application logs using a recent time window; to one or more monitor alerts produced by monitoring the deployed operation of the software, searching for one or more correspondences with known log signatures of the problem, and at least one escalating the severity of the alert generated by monitoring the deployed behavior of the software if one anaphora is discovered;
information related to the one or more alerts generated by monitoring the post-deployment behavior of the software that caused the one or more alerts generated during the static and dynamic testing; 20. The computer program product of claim 19, further comprising: associating source code for causing the computer program product to perform a computer program.