US20060106702A1 - Method and apparatus for extracting and visualizing execution patterns from web services - Google Patents
Method and apparatus for extracting and visualizing execution patterns from web services Download PDFInfo
- Publication number
- US20060106702A1 US20060106702A1 US11/274,383 US27438305A US2006106702A1 US 20060106702 A1 US20060106702 A1 US 20060106702A1 US 27438305 A US27438305 A US 27438305A US 2006106702 A1 US2006106702 A1 US 2006106702A1
- Authority
- US
- United States
- Prior art keywords
- sub
- graphs
- web services
- patterns
- accordance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
Definitions
- the present invention relates generally to Web Services and relates more particularly to the performance analysis, debugging and program understanding of Web Services configurations.
- Web Services applications are web-based enterprise applications that use open, extensible markup language (XML)-based standards and transport protocols to exchange data with calling clients.
- XML extensible markup language
- Web Services developers can build distributed applications with an eye toward interoperability as the priority.
- Perhaps the most popular architectural approach to service-based infrastructures is organizing computational resources as a broad collection of individual services that can be flexibly combined into applications.
- One embodiment of the present method and apparatus for extracting and visualizing execution patterns from Web Services collects data relating to one or more transactions executed in accordance with one or more Web Services applications (e.g., a messaging system or other service). One or more patterns is then extracted from the collected data, said where the patterns are representative transactions commonly executed in accordance with the Web Services configuration under analysis.
- Web Services applications e.g., a messaging system or other service.
- FIG. 1 is a flow diagram illustrating one embodiment of a method for extracting and visualizing execution patterns from a Web Services configuration, according to the present invention
- FIG. 2 is a sequence diagram illustrating a plurality of exemplary transactions and a plurality of patterns extracted therefrom;
- FIG. 3 illustrates a group of exemplary transactions represented as graphs or trees
- FIG. 4 illustrates the sub-groups and of FIG. 3 partitioned into sub-sub-groups
- FIG. 5 is a flow diagram illustrating one embodiment of a method for defining a transaction pattern, according to the present invention.
- FIG. 6 is a diagram illustrating a manner in which first and second similar transaction patterns and may be compared, according to the present invention.
- FIG. 7 is a high level block diagram of the pattern extraction method that is implemented using a general purpose computing device.
- the present invention is a method and apparatus for extracting and visualizing execution patterns from Web Services configurations.
- Embodiments of the present invention provide for the efficient identification and definition of patterns in Web Services transactions, which facilitates operations including performance analysis, debugging and program understanding by allowing a user to analyze and understand system conditions without having to review hundreds or thousands of sets of individual data
- FIG. 1 is a flow diagram illustrating one embodiment of a method 100 for extracting and visualizing execution patterns from a Web Services configuration, according to the present invention.
- the method 100 is initialized at step 102 and proceeds to step 104 , where the method 100 analyzes the Web Services configuration. That is, the method 100 collects data relating to transactions (e.g., sets of units of work initiated by a user or a batch job) that are executed in accordance with the Web Service.
- transactions e.g., sets of units of work initiated by a user or a batch job
- the method 100 may, in step 104 , implement a tracing infrastructure in order to record a plurality of messages being sent, along with their associated identifications, timestamps, participants (e.g., service points, machine names, etc.) and attributes (e.g., content, style, etc.).
- the tracing infrastructure enables end-to-end tracking of a transaction.
- step 106 reduces repetitive sequences in the collected data. For example, if the collected data includes a plurality of recorded messages from a messaging system, a sequence of messages within an individual transaction can be very repetitive, and this repetition can be detected so that the transaction can be described in a more “canonical” form. For instance, the sequences of messages:
- the method 100 proceeds to step 108 and identifies isomorphic or similar graphs (e.g., trees) within the sequences. Since a Web Services transaction starts at one point (e.g., a client call) and goes through one or more services (possibly branching off), the structure of the transaction can be described or illustrated as a tree.
- isomorphic or similar graphs e.g., trees
- FIG. 3 illustrates a group 300 of exemplary transactions represented as graphs or trees.
- isomorphic or similar transactions are found by comparing the tree structures starting at the roots, where the roots are the transaction starting points (e.g., client calls). It can be seen by comparing the trees in this manner that several of the transactions illustrated in the group 300 are isomorphic with respect to other transactions in the group 300 .
- the group 300 may be partitioned into a plurality of subgroups—in this case a first subgroup 302 and a second sub-group 304 —where each sub-group comprises a sub-set of the original group of transactions that shares certain features or graphical elements.
- the method 100 proceeds to step 110 and identifies matching node/edge attributes in the graphs that are relevant to a desired level of abstraction of the analysis, where a node is a service point (e.g., a Web Service endpoint) and an edge is an operation (e.g., what was sent in the message).
- a service point e.g., a Web Service endpoint
- an edge is an operation (e.g., what was sent in the message).
- attributes are deemed to match based on predefined criteria that define when transactions may be automatically considered as similar and classified into a common pattern.
- matching of node or edge attributes may be performed in accordance with a matcher.
- sub-groups of graphs such as sub-groups 302 and 304 of FIG. 3
- FIG. 4 illustrates the sub-groups 302 and 304 of FIG. 3 partitioned into sub-sub-groups. Specifically, sub-group 302 is partitioned into sub-sub-group 402 and sub-sub-group 404 ; sub-group 304 has been partitioned into sub-sub-group 406 and sub-sub-group 408 .
- the sub-sub-groups 402 , 404 , 406 and 408 are defined based on graph isomorphism and on operation name as the matching criterion, where the operation names are illustrated by letters of the alphabet (e.g., a-b-c-d-e-f-g-h in the case of sub-sub-group 402 , p-q-r-s-t-u-v-w in the case of sub-sub-group 404 , etc.).
- Matching criterion for sub-sub-groups may be based on node attributes, edge attributes, or both.
- potentially relevant node attributes include, but are not limited to, service points, port types, machines, node performance data or any function thereof.
- potentially relevant edge attributes include, but are not limited to, operations (e.g., business processes), message names (e.g., unique identifiers such as names, parameter types or return types), message types (e.g., synchronous or asynchronous, operation style, etc.), error types, performance data (for what is sent), message content or any function thereof.
- operations e.g., business processes
- message names e.g., unique identifiers such as names, parameter types or return types
- message types e.g., synchronous or asynchronous, operation style, etc.
- error types e.g., performance data (for what is sent), message content or any function thereof.
- identical operations between two transactions may reveal business processes across different deployments of the same business
- identical operations and service points
- Identical PortTypes may reveal typical patterns across different deployments of the same business, identical machines may reveal patterns in traffic between and workloads on machines (which may help in repartitioning the application across the machines in a different way), and identical service points may reveal traffic patterns between service points (which may help in repartitioning an application on several machines).
- matching criteria may require only a defined degree of similarity, and not a perfect identical match.
- wildcards may be implemented.
- sendClientOrder calling sendRetailOrder calling sendWarehouseAOrder calling sendManufacturerXOrder calling and sendClientOrder calling sendRetailOrder calling sendWarehouseBOrder calling sendManufacturerYOrder calling is substantially equivalent to: sendClientOrder calling sendRetailOrder calling sendWarehouse*Order calling sendManufacturer*Order calling
- the content of two or more pieces of data is matched against each other according to different criteria.
- message content that is structured, meaning the message contents have parameter types (or tags) and parameter values.
- SOAP Simple Object Access Protocol
- message content is typically represented in Simple Object Access Protocol (SOAP), which contains tag and value fields.
- SOAP Simple Object Access Protocol
- a match between messages can be based on the parameter types (tag fields), on the value fields, or on both.
- the information that is used to match corresponding nodes could also be found outside the system. For example, two nodes could be matched based on port type, in accordance with the Web Services Description Language (WSDL) resource for the associated application.
- WSDL Web Services Description Language
- the matching criteria could be discovered automatically (e.g., instead of being based on pre-defined or user-defined criteria).
- automatic discovery of matching criteria might involve first identifying which nodes, edges or attributes share common values.
- step 112 represents (or “visualizes”) the partitioned sub-sub-groups as patterns.
- the method 100 then terminates in step 114 .
- the present invention allows patterns in Web Services transactions to be efficiently identified. Efficient identification of transaction patterns facilitates operations including performance analysis, debugging and program understanding by allowing a user to analyze and understand system conditions without having to review hundreds or thousands of sets of individual data. Common trends, abnormalities and outliers can thus be easily summarized in accordance with various different levels of abstraction.
- FIG. 2 is a sequence diagram illustrating a plurality of exemplary transactions 200 (i.e., on the left-hand side) and a plurality of patterns 202 extracted therefrom (i.e., on the right-hand side).
- the plurality of transactions 200 comprises a sequence of one hundred Web Services calls. Points along the x axis represent service points, while points along the y axis represent time (moving downward).
- the one hundred transactions 200 may be reduced to a group of transaction patterns 202 .
- one hundred transactions 200 are reduced to five transaction patterns 202 .
- Such capability may be especially useful in the eyes of business owners (who may analyze patterns to verify the correctness of a business process or to identify trends in business activities), information technology (IT) architects (who may analyze patterns to identify how different IT resources interact or to identify system bottlenecks), Web Services developers (who may analyze patterns to identify frequently occurring patterns and outliers relating to resource consumption) and Web Services testers (who may analyze patterns rather than test every transaction in a Web Services configuration).
- IT information technology
- Web Services developers who may analyze patterns to identify frequently occurring patterns and outliers relating to resource consumption
- Web Services testers who may analyze patterns rather than test every transaction in a Web Services configuration.
- Web Services transactions may be classified as belonging to a defined transaction pattern according to a plurality of criteria. This can help in quickly analyzing the Web Services configuration for performance analysis, debugging and program understanding. For example, in one embodiment, given a metric relating to either a node or an edge, all transactions more than x standard deviations from the norm may be classified as outliers. Thus, if resource utilization at a node is more than x standard deviations above or below the norm, this may indicate an abnormal execution of the service and may be chosen automatically for investigation. In another embodiment, given a metric relating to either a node or an edge, x transactions falling farthest from the norm may be selected for further investigation.
- a historical representative transaction may be used as a classification mechanism. For example, if a historical set of representative transactions is used as a collection of “seeds” for patterns, then completed transactions can be classified as having characteristics associated with the historical representative transaction. Thus, a current transaction could be classified as nominally “normal” if a metric associates the current transaction with a representative transaction that is considered normal (or, conversely, classified as “abnormal” with needed remediation if the metric associated the current transaction with a historical error representative transaction).
- correlation and statistical mechanisms are implemented to identify abnormal transaction patterns. For example, collections of transactions could be classified as probable errors if an identified pattern represents only one or a few executed transactions.
- correlation techniques employing data correlations, header correlations or temporal correlations are used to associate multiple singleton or low frequency patterns. For example, consider the instance in which an executing transaction triggers an error during a unit of work; the error is handled and processing continues. The error handling may break the correlation chain used for associating the parts of the complete transaction. In such cases, parts of the complete transaction chain may appear distributed between multiple singleton or low frequency patterns. If such patterns appear to be temporally close or share significant data, meta-data or attributes, the analysis system could suggest aggregation of the associated parts into another identified pattern.
- FIG. 5 is a flow diagram illustrating one embodiment of a method 500 for defining a transaction pattern (e.g., in accordance with step 112 of the method 100 ), according to the present invention.
- the method 500 is initialized in step 502 and proceeds to step 504 , where the method 500 identifies a suitable representation for all transactions in a sub-sub-group.
- the transaction pattern is defined by selecting one transaction from the sub-sub-group as a representative.
- a suitable representation is defined by producing a new set of nodes, edges and attributes that is as similar as possible to the transactions in the sub-sub-group.
- this new set of nodes, edges and attributes is produced by first using a node, an edge or an attribute that was used as a matching criterion for the sub-sub-group as a new representation. For example, if a matching criterion was that corresponding nodes should have identical PortTypes, the PortType value may be used to label the node. Then, for each node, edge or attribute that was not a matching criterion, a common denominator is found.
- a commonality may be extracted between two nodes, edges or attributes based on names (e.g., getOrderShoe+getOrderStock->getOrderS*), PortType, or symbolic variables (e.g., [a,a,a,b,a,b] and [c,c,c,d,c,d] produces [x,x,x,y,x,y]).
- names e.g., getOrderShoe+getOrderStock->getOrderS*
- PortType e.g., [a,a,a,b,a,b] and [c,c,c,d,c,d] produces [x,x,x,y,x,y]).
- step 506 the method 500 defines the representation as the pattern for the sub-sub-group.
- this step includes adding information to the representation that aggregates data (e.g., minima, maxima, averages, standard deviations, etc.) from the transactions that are represented.
- aggregates data e.g., minima, maxima, averages, standard deviations, etc.
- the method 500 then proceeds to step 508 and indicates all nodes, edges and attributes for which a similar representation can not be found as different.
- the method 500 then terminates in step 510 .
- different patterns produced in accordance with the present invention are visualized together, for example for the purposes of comparison. These different patterns are input into a method similar to the method 100 in order to identify the similarities and differences between the patterns. In such an instance, matching criteria for nodes and/or edges may be relaxed to facilitate comparison.
- FIG. 6 is a diagram illustrating a manner in which first and second similar transaction patterns 602 and 604 may be compared, according to the present invention.
- a first portion 608 of the resulting pattern 606 illustrates a common pattern; however, second and third portions 610 and 612 of the resulting pattern 606 illustrate the differences in the first and second transaction patterns 602 and 604 .
- Patterns extracted and visualized in accordance with the method 100 may be selected for comparison (of the representative transactions) in a system composed of a collection of services. For example, in one embodiment, code is generated to produce and execute a selected transaction pattern. In another embodiment, tools for performing re-play of a transaction pattern (and for verifying a match or equivalency of the result) are implemented. In another embodiment still, each message in a transaction pattern is “decorated” with data, meta-data or attributes such that the service invocations will not have undesirable side effects when called with the selected transaction pattern. In yet another embodiment, representative transactions are obtained historically to provide test data with known coverage for new or modified Web Services systems that are designed to be upward compatible with the historical Web Services system.
- a selected metric is applied to the entire collection of data (e.g., data and meta-data) derived from transactions and used to compute a representative transaction that has the shortest mean square distance from all transactions in the collection of data. Then, one or more individual transactions are classified based on their distances (using either the same metric or a different metric) from the representative transaction.
- the data and/or meta-data is collected from either the operation of a software system or synthesized to be representative of software systems.
- relevant metrics are related to resources and include, but are not limited to, resources that are consumed based on or are derived from metering data (e.g., as returned by a metering service or other metering system), resources that are consumed based on or are derived from monitoring data (e.g., as returned by a monitoring service or other monitoring system), resources that are consumed based on or are derived from analysis of communications traffic (e.g., message bodies, headers, trailers, attachments, added attributes or other message content) and resources that are computed from any combination of these sources.
- metering data e.g., as returned by a metering service or other metering system
- monitoring data e.g., as returned by a monitoring service or other monitoring system
- resources that are consumed based on or are derived from analysis of communications traffic e.g., message bodies, headers, trailers, attachments, added attributes or other message content
- relevant metrics are related to latency and include temporal latency (e.g., resulting from business logic execution, network execution of logic unrelated to the business logic such as in-line monitoring, or calls to services outside of the Web Services system under analysis).
- the primary source of latency data is computed based on time-stamps obtained during Web Services monitoring.
- the latency data may include, or even be limited to, latency data taken from other sources.
- relevant metrics are combined metrics that include architectural approaches frequently applied in Web Services designs.
- one approach is to provide equivalent choices for execution of an individual step in a sequence of steps comprising a Web Services transaction.
- a clearer representation of the overall logic flow can be obtained by applying a metric that treats all equivalent services as a single instance by collapsing the corresponding metrics into a single representation used for all equivalent services.
- FIG. 7 is a high level block diagram of the pattern extraction method that is implemented using a general purpose computing device 700 .
- a general purpose computing device 700 comprises a processor 702 , a memory 704 , a pattern extraction module 705 and various input/output (I/O) devices 706 such as a display, a keyboard, a mouse, a modem, and the like.
- I/O devices 706 such as a display, a keyboard, a mouse, a modem, and the like.
- at least one I/O device is a storage device (e.g., a disk drive, an optical disk drive, a floppy disk drive).
- the pattern extraction module 705 can be implemented as a physical device or subsystem that is coupled to a processor through a communication channel.
- the pattern extraction module 705 can be represented by one or more software applications (or even a combination of software and hardware, e.g., using Application Specific Integrated Circuits (ASIC)), where the software is loaded from a storage medium (e.g., I/O devices 706 ) and operated by the processor 702 in the memory 704 of the general purpose computing device 700 .
- a storage medium e.g., I/O devices 706
- the pattern extraction module 705 for extracting and visualizing execution patterns from Web Services described herein with reference to the preceding Figures can be stored on a computer readable medium or carrier (e.g., RAM, magnetic or optical drive or diskette, and the like).
- the present invention represents a significant advancement in the field of Web Services.
- a method and apparatus are provided that enable the efficient identification and definition of patterns in Web Services transactions. This capability facilitates operations including performance analysis, debugging and program understanding by allowing a user to analyze and understand system conditions without having to review hundreds or thousands of sets of individual data
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- General Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Technology Law (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
- This application claims priority under 35 U.S.C. 119(e) of U.S. Provisional Patent Application Ser. No. 60/628,302, filed Nov. 15, 2004, which is herein incorporated by reference in its entirety.
- The present invention relates generally to Web Services and relates more particularly to the performance analysis, debugging and program understanding of Web Services configurations.
- Web Services applications are web-based enterprise applications that use open, extensible markup language (XML)-based standards and transport protocols to exchange data with calling clients. Using Web Services, developers can build distributed applications with an eye toward interoperability as the priority. Perhaps the most popular architectural approach to service-based infrastructures is organizing computational resources as a broad collection of individual services that can be flexibly combined into applications.
- One of the challenges associated with such an approach is the management of horizontal complexity (e.g., because there are many nodes or computing devices in a Web Services framework, and many interconnections and overlapping applications). Tasks such as performance analysis, debugging and program understanding thus become more complex in a large Web Services configuration.
- Thus, there is a need in the art for a method and apparatus for extracting and visualizing execution patterns from Web Services configurations.
- One embodiment of the present method and apparatus for extracting and visualizing execution patterns from Web Services collects data relating to one or more transactions executed in accordance with one or more Web Services applications (e.g., a messaging system or other service). One or more patterns is then extracted from the collected data, said where the patterns are representative transactions commonly executed in accordance with the Web Services configuration under analysis.
- So that the manner in which the above recited embodiments of the invention are attained and can be understood in detail, a more particular description of the invention, briefly summarized above, may be obtained by reference to the embodiments thereof which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
-
FIG. 1 is a flow diagram illustrating one embodiment of a method for extracting and visualizing execution patterns from a Web Services configuration, according to the present invention; -
FIG. 2 is a sequence diagram illustrating a plurality of exemplary transactions and a plurality of patterns extracted therefrom; -
FIG. 3 illustrates a group of exemplary transactions represented as graphs or trees; -
FIG. 4 illustrates the sub-groups and ofFIG. 3 partitioned into sub-sub-groups; -
FIG. 5 is a flow diagram illustrating one embodiment of a method for defining a transaction pattern, according to the present invention; -
FIG. 6 is a diagram illustrating a manner in which first and second similar transaction patterns and may be compared, according to the present invention; and -
FIG. 7 is a high level block diagram of the pattern extraction method that is implemented using a general purpose computing device. - To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures.
- In one embodiment, the present invention is a method and apparatus for extracting and visualizing execution patterns from Web Services configurations. Embodiments of the present invention provide for the efficient identification and definition of patterns in Web Services transactions, which facilitates operations including performance analysis, debugging and program understanding by allowing a user to analyze and understand system conditions without having to review hundreds or thousands of sets of individual data
-
FIG. 1 is a flow diagram illustrating one embodiment of amethod 100 for extracting and visualizing execution patterns from a Web Services configuration, according to the present invention. Themethod 100 is initialized atstep 102 and proceeds tostep 104, where themethod 100 analyzes the Web Services configuration. That is, themethod 100 collects data relating to transactions (e.g., sets of units of work initiated by a user or a batch job) that are executed in accordance with the Web Service. For example, if the Web Services configuration is a messaging system, themethod 100 may, instep 104, implement a tracing infrastructure in order to record a plurality of messages being sent, along with their associated identifications, timestamps, participants (e.g., service points, machine names, etc.) and attributes (e.g., content, style, etc.). In one embodiment, the tracing infrastructure enables end-to-end tracking of a transaction. - Once sufficient data has been collected, the
method 100 proceeds tostep 106 and reduces repetitive sequences in the collected data. For example, if the collected data includes a plurality of recorded messages from a messaging system, a sequence of messages within an individual transaction can be very repetitive, and this repetition can be detected so that the transaction can be described in a more “canonical” form. For instance, the sequences of messages: -
- -A->B->C->D->B->C->D->B->C->D->B->C->D->B->C->D->E -A->B->C->D->B->C->D->B->C->D->E
can be summarized as: - -A->[->B->C->D]*->E
In one embodiment, the messages are not necessarily identical in all respects (e.g., the messages may only be identical as far as the operation names are concerned).
- -A->B->C->D->B->C->D->B->C->D->B->C->D->B->C->D->E -A->B->C->D->B->C->D->B->C->D->E
- Once the repetitive sequences have been reduced, the
method 100 proceeds tostep 108 and identifies isomorphic or similar graphs (e.g., trees) within the sequences. Since a Web Services transaction starts at one point (e.g., a client call) and goes through one or more services (possibly branching off), the structure of the transaction can be described or illustrated as a tree. - For example,
FIG. 3 illustrates agroup 300 of exemplary transactions represented as graphs or trees. In one embodiment, isomorphic or similar transactions are found by comparing the tree structures starting at the roots, where the roots are the transaction starting points (e.g., client calls). It can be seen by comparing the trees in this manner that several of the transactions illustrated in thegroup 300 are isomorphic with respect to other transactions in thegroup 300. Thus, thegroup 300 may be partitioned into a plurality of subgroups—in this case afirst subgroup 302 and asecond sub-group 304—where each sub-group comprises a sub-set of the original group of transactions that shares certain features or graphical elements. - Referring back to
FIG. 1 , once the isomorphic or similar graphs have been identified, themethod 100 proceeds tostep 110 and identifies matching node/edge attributes in the graphs that are relevant to a desired level of abstraction of the analysis, where a node is a service point (e.g., a Web Service endpoint) and an edge is an operation (e.g., what was sent in the message). - Most transactions will not be identical; however, many of the differences may seem overwhelming at first and may distract from more significant trends. Finding the commonalities may provide a better understanding of the typical behavior of the Web Services configuration under analysis. In one embodiment, attributes are deemed to match based on predefined criteria that define when transactions may be automatically considered as similar and classified into a common pattern. In another embodiment, matching of node or edge attributes may be performed in accordance with a matcher.
- Thus, sub-groups of graphs, such as
sub-groups FIG. 3 , may be further partitioned into sub-sub-groups based on criteria such as node and/or edge attributes. - For example,
FIG. 4 illustrates thesub-groups FIG. 3 partitioned into sub-sub-groups. Specifically,sub-group 302 is partitioned into sub-sub-group 402 and sub-sub-group 404;sub-group 304 has been partitioned into sub-sub-group 406 and sub-sub-group 408. In one embodiment, the sub-sub-groups group 402, p-q-r-s-t-u-v-w in the case of sub-sub-group 404, etc.). - Matching criterion for sub-sub-groups may be based on node attributes, edge attributes, or both. In one embodiment, potentially relevant node attributes include, but are not limited to, service points, port types, machines, node performance data or any function thereof. In one embodiment, potentially relevant edge attributes include, but are not limited to, operations (e.g., business processes), message names (e.g., unique identifiers such as names, parameter types or return types), message types (e.g., synchronous or asynchronous, operation style, etc.), error types, performance data (for what is sent), message content or any function thereof. Thus, for example, identical operations between two transactions may reveal business processes across different deployments of the same business, while identical operations and service points may reveal business processes that are specific to a given deployment. Identical PortTypes may reveal typical patterns across different deployments of the same business, identical machines may reveal patterns in traffic between and workloads on machines (which may help in repartitioning the application across the machines in a different way), and identical service points may reveal traffic patterns between service points (which may help in repartitioning an application on several machines).
- In further embodiments, matching criteria may require only a defined degree of similarity, and not a perfect identical match. In such a case, wildcards may be implemented. For example,
sendClientOrder calling sendRetailOrder calling sendWarehouseAOrder calling sendManufacturerXOrder calling and sendClientOrder calling sendRetailOrder calling sendWarehouseBOrder calling sendManufacturerYOrder calling is substantially equivalent to: sendClientOrder calling sendRetailOrder calling sendWarehouse*Order calling sendManufacturer*Order calling - In one embodiment, the content of two or more pieces of data (e.g., two or more messages) is matched against each other according to different criteria. For example, many messaging systems use message content that is structured, meaning the message contents have parameter types (or tags) and parameter values. In the case of Web Services, message content is typically represented in Simple Object Access Protocol (SOAP), which contains tag and value fields. Given this, a match between messages can be based on the parameter types (tag fields), on the value fields, or on both. The information that is used to match corresponding nodes could also be found outside the system. For example, two nodes could be matched based on port type, in accordance with the Web Services Description Language (WSDL) resource for the associated application.
- In yet another embodiment, the matching criteria could be discovered automatically (e.g., instead of being based on pre-defined or user-defined criteria). For example, automatic discovery of matching criteria might involve first identifying which nodes, edges or attributes share common values.
- Referring back to
FIG. 1 , once transactions are partitioned into sub-groups and sub-sub-groups of matching transactions (based on the same or similar attributes), themethod 100 proceeds to step 112 and represents (or “visualizes”) the partitioned sub-sub-groups as patterns. Themethod 100 then terminates instep 114. - Thus, the present invention allows patterns in Web Services transactions to be efficiently identified. Efficient identification of transaction patterns facilitates operations including performance analysis, debugging and program understanding by allowing a user to analyze and understand system conditions without having to review hundreds or thousands of sets of individual data. Common trends, abnormalities and outliers can thus be easily summarized in accordance with various different levels of abstraction.
- For example,
FIG. 2 is a sequence diagram illustrating a plurality of exemplary transactions 200 (i.e., on the left-hand side) and a plurality ofpatterns 202 extracted therefrom (i.e., on the right-hand side). In the embodiment illustrated, the plurality oftransactions 200 comprises a sequence of one hundred Web Services calls. Points along the x axis represent service points, while points along the y axis represent time (moving downward). As illustrated, the one hundredtransactions 200 may be reduced to a group oftransaction patterns 202. In the illustrated embodiment, one hundredtransactions 200 are reduced to fivetransaction patterns 202. - Such capability may be especially useful in the eyes of business owners (who may analyze patterns to verify the correctness of a business process or to identify trends in business activities), information technology (IT) architects (who may analyze patterns to identify how different IT resources interact or to identify system bottlenecks), Web Services developers (who may analyze patterns to identify frequently occurring patterns and outliers relating to resource consumption) and Web Services testers (who may analyze patterns rather than test every transaction in a Web Services configuration).
- As described above, Web Services transactions may be classified as belonging to a defined transaction pattern according to a plurality of criteria. This can help in quickly analyzing the Web Services configuration for performance analysis, debugging and program understanding. For example, in one embodiment, given a metric relating to either a node or an edge, all transactions more than x standard deviations from the norm may be classified as outliers. Thus, if resource utilization at a node is more than x standard deviations above or below the norm, this may indicate an abnormal execution of the service and may be chosen automatically for investigation. In another embodiment, given a metric relating to either a node or an edge, x transactions falling farthest from the norm may be selected for further investigation.
- In another embodiment, a historical representative transaction may be used as a classification mechanism. For example, if a historical set of representative transactions is used as a collection of “seeds” for patterns, then completed transactions can be classified as having characteristics associated with the historical representative transaction. Thus, a current transaction could be classified as nominally “normal” if a metric associates the current transaction with a representative transaction that is considered normal (or, conversely, classified as “abnormal” with needed remediation if the metric associated the current transaction with a historical error representative transaction).
- In another embodiment, correlation and statistical mechanisms are implemented to identify abnormal transaction patterns. For example, collections of transactions could be classified as probable errors if an identified pattern represents only one or a few executed transactions. In another embodiment, correlation techniques employing data correlations, header correlations or temporal correlations are used to associate multiple singleton or low frequency patterns. For example, consider the instance in which an executing transaction triggers an error during a unit of work; the error is handled and processing continues. The error handling may break the correlation chain used for associating the parts of the complete transaction. In such cases, parts of the complete transaction chain may appear distributed between multiple singleton or low frequency patterns. If such patterns appear to be temporally close or share significant data, meta-data or attributes, the analysis system could suggest aggregation of the associated parts into another identified pattern.
-
FIG. 5 is a flow diagram illustrating one embodiment of amethod 500 for defining a transaction pattern (e.g., in accordance withstep 112 of the method 100), according to the present invention. Themethod 500 is initialized instep 502 and proceeds to step 504, where themethod 500 identifies a suitable representation for all transactions in a sub-sub-group. In one embodiment (e.g., where all of the attributes of all of the transactions in the sub-sub-group are identical), the transaction pattern is defined by selecting one transaction from the sub-sub-group as a representative. - In another embodiment (e.g., where all of the attributes of all of the transactions in the sub-sub-group are not identical), a suitable representation is defined by producing a new set of nodes, edges and attributes that is as similar as possible to the transactions in the sub-sub-group. In one embodiment, this new set of nodes, edges and attributes is produced by first using a node, an edge or an attribute that was used as a matching criterion for the sub-sub-group as a new representation. For example, if a matching criterion was that corresponding nodes should have identical PortTypes, the PortType value may be used to label the node. Then, for each node, edge or attribute that was not a matching criterion, a common denominator is found. For example, a commonality may be extracted between two nodes, edges or attributes based on names (e.g., getOrderShoe+getOrderStock->getOrderS*), PortType, or symbolic variables (e.g., [a,a,a,b,a,b] and [c,c,c,d,c,d] produces [x,x,x,y,x,y]).
- In
step 506, themethod 500 defines the representation as the pattern for the sub-sub-group. In one embodiment, this step includes adding information to the representation that aggregates data (e.g., minima, maxima, averages, standard deviations, etc.) from the transactions that are represented. - The
method 500 then proceeds to step 508 and indicates all nodes, edges and attributes for which a similar representation can not be found as different. Themethod 500 then terminates instep 510. - In one embodiment, different patterns produced in accordance with the present invention are visualized together, for example for the purposes of comparison. These different patterns are input into a method similar to the
method 100 in order to identify the similarities and differences between the patterns. In such an instance, matching criteria for nodes and/or edges may be relaxed to facilitate comparison. -
FIG. 6 is a diagram illustrating a manner in which first and secondsimilar transaction patterns first portion 608 of the resultingpattern 606 illustrates a common pattern; however, second andthird portions pattern 606 illustrate the differences in the first andsecond transaction patterns - Patterns extracted and visualized in accordance with the
method 100 may be selected for comparison (of the representative transactions) in a system composed of a collection of services. For example, in one embodiment, code is generated to produce and execute a selected transaction pattern. In another embodiment, tools for performing re-play of a transaction pattern (and for verifying a match or equivalency of the result) are implemented. In another embodiment still, each message in a transaction pattern is “decorated” with data, meta-data or attributes such that the service invocations will not have undesirable side effects when called with the selected transaction pattern. In yet another embodiment, representative transactions are obtained historically to provide test data with known coverage for new or modified Web Services systems that are designed to be upward compatible with the historical Web Services system. - In one embodiment, in order to find a metric of similarity between two or more transaction patterns (e.g., where the goal may be to find trends and outliers in a group of similar transactions that belong to one or more patterns), one classifies individual transactions in the context of a group of transactions, using either user-supplied metrics or pre-defined metrics for classification. In one embodiment, a selected metric is applied to the entire collection of data (e.g., data and meta-data) derived from transactions and used to compute a representative transaction that has the shortest mean square distance from all transactions in the collection of data. Then, one or more individual transactions are classified based on their distances (using either the same metric or a different metric) from the representative transaction. In one embodiment, the data and/or meta-data is collected from either the operation of a software system or synthesized to be representative of software systems.
- In one embodiment, relevant metrics are related to resources and include, but are not limited to, resources that are consumed based on or are derived from metering data (e.g., as returned by a metering service or other metering system), resources that are consumed based on or are derived from monitoring data (e.g., as returned by a monitoring service or other monitoring system), resources that are consumed based on or are derived from analysis of communications traffic (e.g., message bodies, headers, trailers, attachments, added attributes or other message content) and resources that are computed from any combination of these sources.
- In another embodiment, relevant metrics are related to latency and include temporal latency (e.g., resulting from business logic execution, network execution of logic unrelated to the business logic such as in-line monitoring, or calls to services outside of the Web Services system under analysis). In one embodiment, the primary source of latency data is computed based on time-stamps obtained during Web Services monitoring. In another embodiment, the latency data may include, or even be limited to, latency data taken from other sources.
- In yet another embodiment, relevant metrics are combined metrics that include architectural approaches frequently applied in Web Services designs. For example, one approach is to provide equivalent choices for execution of an individual step in a sequence of steps comprising a Web Services transaction. In such cases, a clearer representation of the overall logic flow can be obtained by applying a metric that treats all equivalent services as a single instance by collapsing the corresponding metrics into a single representation used for all equivalent services. For example, if there are three tax computation services, T1, T2 and T3, and there is a derivable set of equivalent resource metrics R1, R2 and R3, then a valuable representation can be obtained by collapsing [T1, R1], [T2, R2] and [T3, R3] into [T, R].
-
FIG. 7 is a high level block diagram of the pattern extraction method that is implemented using a generalpurpose computing device 700. In one embodiment, a generalpurpose computing device 700 comprises a processor 702, amemory 704, apattern extraction module 705 and various input/output (I/O)devices 706 such as a display, a keyboard, a mouse, a modem, and the like. In one embodiment, at least one I/O device is a storage device (e.g., a disk drive, an optical disk drive, a floppy disk drive). It should be understood that thepattern extraction module 705 can be implemented as a physical device or subsystem that is coupled to a processor through a communication channel. - Alternatively, the
pattern extraction module 705 can be represented by one or more software applications (or even a combination of software and hardware, e.g., using Application Specific Integrated Circuits (ASIC)), where the software is loaded from a storage medium (e.g., I/O devices 706) and operated by the processor 702 in thememory 704 of the generalpurpose computing device 700. Thus, in one embodiment, thepattern extraction module 705 for extracting and visualizing execution patterns from Web Services described herein with reference to the preceding Figures can be stored on a computer readable medium or carrier (e.g., RAM, magnetic or optical drive or diskette, and the like). - Thus, the present invention represents a significant advancement in the field of Web Services. A method and apparatus are provided that enable the efficient identification and definition of patterns in Web Services transactions. This capability facilitates operations including performance analysis, debugging and program understanding by allowing a user to analyze and understand system conditions without having to review hundreds or thousands of sets of individual data
- While foregoing is directed to the preferred embodiment of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.
Claims (20)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/274,383 US20060106702A1 (en) | 2004-11-15 | 2005-11-15 | Method and apparatus for extracting and visualizing execution patterns from web services |
US12/164,726 US7873728B2 (en) | 2004-11-15 | 2008-06-30 | Method and apparatus for extracting and visualizing execution patterns from web services |
US12/983,551 US8326982B2 (en) | 2004-11-15 | 2011-01-03 | Method and apparatus for extracting and visualizing execution patterns from web services |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US62830204P | 2004-11-15 | 2004-11-15 | |
US11/274,383 US20060106702A1 (en) | 2004-11-15 | 2005-11-15 | Method and apparatus for extracting and visualizing execution patterns from web services |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/164,726 Continuation US7873728B2 (en) | 2004-11-15 | 2008-06-30 | Method and apparatus for extracting and visualizing execution patterns from web services |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060106702A1 true US20060106702A1 (en) | 2006-05-18 |
Family
ID=36387583
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/274,383 Abandoned US20060106702A1 (en) | 2004-11-15 | 2005-11-15 | Method and apparatus for extracting and visualizing execution patterns from web services |
US12/164,726 Expired - Fee Related US7873728B2 (en) | 2004-11-15 | 2008-06-30 | Method and apparatus for extracting and visualizing execution patterns from web services |
US12/983,551 Expired - Fee Related US8326982B2 (en) | 2004-11-15 | 2011-01-03 | Method and apparatus for extracting and visualizing execution patterns from web services |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/164,726 Expired - Fee Related US7873728B2 (en) | 2004-11-15 | 2008-06-30 | Method and apparatus for extracting and visualizing execution patterns from web services |
US12/983,551 Expired - Fee Related US8326982B2 (en) | 2004-11-15 | 2011-01-03 | Method and apparatus for extracting and visualizing execution patterns from web services |
Country Status (1)
Country | Link |
---|---|
US (3) | US20060106702A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080086348A1 (en) * | 2006-10-09 | 2008-04-10 | Rajagopa Rao | Fast business process test case composition |
US20110227925A1 (en) * | 2010-03-16 | 2011-09-22 | Imb Corporation | Displaying a visualization of event instances and common event sequences |
US8468129B2 (en) * | 2011-09-23 | 2013-06-18 | Loyal3 Holdings, Inc. | Asynchronous replication of databases of peer networks |
US20130185643A1 (en) * | 2011-12-28 | 2013-07-18 | Compuware Corporation | Method And System For Tracing End-To-End Transaction, Including Browser Side Processing And End User Performance Experience |
EP3493063A1 (en) * | 2017-11-17 | 2019-06-05 | Valtech UK Ltd. | A monitoring system and method |
US11182537B2 (en) * | 2017-12-01 | 2021-11-23 | Dynatrace Llc | Method and system for real-user capable detecting of the visual completeness of browser rendering processes |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060106702A1 (en) * | 2004-11-15 | 2006-05-18 | Pauw Wim D | Method and apparatus for extracting and visualizing execution patterns from web services |
US8612573B2 (en) * | 2008-08-28 | 2013-12-17 | Ca, Inc. | Automatic and dynamic detection of anomalous transactions |
US11016867B2 (en) | 2015-12-18 | 2021-05-25 | Micro Focus Llc | Test execution comparisons |
Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5694548A (en) * | 1993-06-29 | 1997-12-02 | International Business Machines Corporation | System and method for providing multimedia quality of service sessions in a communications network |
US6279113B1 (en) * | 1998-03-16 | 2001-08-21 | Internet Tools, Inc. | Dynamic signature inspection-based network intrusion detection |
US20020062368A1 (en) * | 2000-10-11 | 2002-05-23 | David Holtzman | System and method for establishing and evaluating cross community identities in electronic forums |
US6430617B1 (en) * | 1999-03-22 | 2002-08-06 | Hewlett-Packard Co. | Methods and systems for dynamic measurement of a system's ability to support data collection by network management system applications |
US6438579B1 (en) * | 1999-07-16 | 2002-08-20 | Agent Arts, Inc. | Automated content and collaboration-based system and methods for determining and providing content recommendations |
US20030131097A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Interactive path analysis |
US6643613B2 (en) * | 2001-07-03 | 2003-11-04 | Altaworks Corporation | System and method for monitoring performance metrics |
US20040059807A1 (en) * | 2002-09-16 | 2004-03-25 | Finisar Corporation | Network analysis topology detection |
US20040103371A1 (en) * | 2002-11-27 | 2004-05-27 | Yu Chen | Small form factor web browsing |
US20050102392A1 (en) * | 2003-11-12 | 2005-05-12 | International Business Machines Corporation | Pattern based web services using caching |
US20050125271A1 (en) * | 2003-12-09 | 2005-06-09 | Peltz Christopher J. | Web services project planning |
US20050154627A1 (en) * | 2003-12-31 | 2005-07-14 | Bojan Zuzek | Transactional data collection, compression, and processing information management system |
US20050223021A1 (en) * | 2004-03-30 | 2005-10-06 | Alok Batra | Providing enterprise information |
US7406436B1 (en) * | 2001-03-22 | 2008-07-29 | Richard Reisman | Method and apparatus for collecting, aggregating and providing post-sale market data for an item |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3282470B2 (en) * | 1995-11-08 | 2002-05-13 | 三菱電機株式会社 | Numerical control device using personal computer and control method thereof |
US6914893B2 (en) * | 1998-06-22 | 2005-07-05 | Statsignal Ipc, Llc | System and method for monitoring and controlling remote devices |
US6891838B1 (en) * | 1998-06-22 | 2005-05-10 | Statsignal Ipc, Llc | System and method for monitoring and controlling residential devices |
US7650425B2 (en) * | 1999-03-18 | 2010-01-19 | Sipco, Llc | System and method for controlling communication between a host computer and communication devices associated with remote devices in an automated monitoring system |
US7039639B2 (en) * | 1999-03-31 | 2006-05-02 | International Business Machines Corporation | Optimization of system performance based on communication relationship |
US7185286B2 (en) * | 2001-08-28 | 2007-02-27 | Nvidia International, Inc. | Interface for mobilizing content and transactions on multiple classes of devices |
US20030061340A1 (en) * | 2001-09-25 | 2003-03-27 | Mingqiu Sun | Network health monitoring through real-time analysis of heartbeat patterns from distributed agents |
US20040015719A1 (en) * | 2002-07-16 | 2004-01-22 | Dae-Hyung Lee | Intelligent security engine and intelligent and integrated security system using the same |
US20060106702A1 (en) | 2004-11-15 | 2006-05-18 | Pauw Wim D | Method and apparatus for extracting and visualizing execution patterns from web services |
-
2005
- 2005-11-15 US US11/274,383 patent/US20060106702A1/en not_active Abandoned
-
2008
- 2008-06-30 US US12/164,726 patent/US7873728B2/en not_active Expired - Fee Related
-
2011
- 2011-01-03 US US12/983,551 patent/US8326982B2/en not_active Expired - Fee Related
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5694548A (en) * | 1993-06-29 | 1997-12-02 | International Business Machines Corporation | System and method for providing multimedia quality of service sessions in a communications network |
US6279113B1 (en) * | 1998-03-16 | 2001-08-21 | Internet Tools, Inc. | Dynamic signature inspection-based network intrusion detection |
US6430617B1 (en) * | 1999-03-22 | 2002-08-06 | Hewlett-Packard Co. | Methods and systems for dynamic measurement of a system's ability to support data collection by network management system applications |
US6438579B1 (en) * | 1999-07-16 | 2002-08-20 | Agent Arts, Inc. | Automated content and collaboration-based system and methods for determining and providing content recommendations |
US20020062368A1 (en) * | 2000-10-11 | 2002-05-23 | David Holtzman | System and method for establishing and evaluating cross community identities in electronic forums |
US7406436B1 (en) * | 2001-03-22 | 2008-07-29 | Richard Reisman | Method and apparatus for collecting, aggregating and providing post-sale market data for an item |
US6643613B2 (en) * | 2001-07-03 | 2003-11-04 | Altaworks Corporation | System and method for monitoring performance metrics |
US20030131097A1 (en) * | 2002-01-09 | 2003-07-10 | Stephane Kasriel | Interactive path analysis |
US20040059807A1 (en) * | 2002-09-16 | 2004-03-25 | Finisar Corporation | Network analysis topology detection |
US20040103371A1 (en) * | 2002-11-27 | 2004-05-27 | Yu Chen | Small form factor web browsing |
US20050102392A1 (en) * | 2003-11-12 | 2005-05-12 | International Business Machines Corporation | Pattern based web services using caching |
US20050125271A1 (en) * | 2003-12-09 | 2005-06-09 | Peltz Christopher J. | Web services project planning |
US20050154627A1 (en) * | 2003-12-31 | 2005-07-14 | Bojan Zuzek | Transactional data collection, compression, and processing information management system |
US20050223021A1 (en) * | 2004-03-30 | 2005-10-06 | Alok Batra | Providing enterprise information |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080086348A1 (en) * | 2006-10-09 | 2008-04-10 | Rajagopa Rao | Fast business process test case composition |
US8893089B2 (en) * | 2006-10-09 | 2014-11-18 | Sap Se | Fast business process test case composition |
US20110227925A1 (en) * | 2010-03-16 | 2011-09-22 | Imb Corporation | Displaying a visualization of event instances and common event sequences |
US8468129B2 (en) * | 2011-09-23 | 2013-06-18 | Loyal3 Holdings, Inc. | Asynchronous replication of databases of peer networks |
US20130185643A1 (en) * | 2011-12-28 | 2013-07-18 | Compuware Corporation | Method And System For Tracing End-To-End Transaction, Including Browser Side Processing And End User Performance Experience |
US9571591B2 (en) * | 2011-12-28 | 2017-02-14 | Dynatrace Llc | Method and system for tracing end-to-end transaction which accounts for content update requests |
US10516747B2 (en) | 2011-12-28 | 2019-12-24 | Dynatrace Llc | Method and system for tracing end-to-end transaction, including browser side processing and end user performance experience |
US11240325B2 (en) * | 2011-12-28 | 2022-02-01 | Dynatrace Llc | Method and system for tracing end-to-end transaction, including browser side processing and end user performance experience |
US11659056B2 (en) | 2011-12-28 | 2023-05-23 | Dynatrace Llc | Method and system for tracing end-to-end transaction, including browser side processing and capturing of end user performance experience |
EP3493063A1 (en) * | 2017-11-17 | 2019-06-05 | Valtech UK Ltd. | A monitoring system and method |
US11182537B2 (en) * | 2017-12-01 | 2021-11-23 | Dynatrace Llc | Method and system for real-user capable detecting of the visual completeness of browser rendering processes |
Also Published As
Publication number | Publication date |
---|---|
US8326982B2 (en) | 2012-12-04 |
US20110106944A1 (en) | 2011-05-05 |
US7873728B2 (en) | 2011-01-18 |
US20090006615A1 (en) | 2009-01-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8326982B2 (en) | Method and apparatus for extracting and visualizing execution patterns from web services | |
US10860618B2 (en) | Low-latency streaming analytics | |
Xiong et al. | A Petri net approach to analysis and composition of web services | |
US8583779B2 (en) | Root cause analysis approach with candidate elimination using network virtualization | |
US7290048B1 (en) | Method of semi-automatic data collection, data analysis, and model generation for the performance analysis of enterprise applications | |
CN104426768A (en) | Data message forwarding method and device | |
US20140223555A1 (en) | Method and system for improving security threats detection in communication networks | |
EP1394984A1 (en) | Method and Apparatus for Network Resource Utilization Assessment | |
WO2019223062A1 (en) | Method and system for processing system exceptions | |
CN114637989B (en) | APT attack tracing method, system and storage medium based on distributed system | |
US8533279B2 (en) | Method and system for reconstructing transactions in a communication network | |
US11960545B1 (en) | Retrieving event records from a field searchable data store using references values in inverted indexes | |
US11429608B2 (en) | Optimizing search of an accelerated data model by enabling emitting of structured and unstructured fields from the data model | |
US20030074439A1 (en) | Systems and methods for providing off-line decision support for correlation analysis | |
CN110489317B (en) | Cloud system task operation fault diagnosis method and system based on workflow | |
Basu et al. | Toward web service dependency discovery for SOA management | |
CN102457578A (en) | Distributed network monitoring method based on event mechanism | |
US8392577B2 (en) | Reduction of message flow between bus-connected consumers and producers | |
CN111431733A (en) | Service alarm coverage information evaluation method and device | |
US20110191778A1 (en) | Computer program, method, and apparatus for grouping tasks into series | |
US20040111706A1 (en) | Analysis of latencies in a multi-node system | |
CN112422349B (en) | Network management system, method, equipment and medium for NFV | |
CN116112399B (en) | Industrial control network flow analysis system | |
Anchuri et al. | Hotspot detection in a service-oriented architecture | |
Horovitz et al. | Online Automatic Characteristics Discovery of Faulty Application Transactions in the Cloud. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DE PAUW, WIM;KRASIKOV, SOPHIA;LEI, MICHELLE;AND OTHERS;REEL/FRAME:016877/0799 Effective date: 20051115 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: KYNDRYL, INC., NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:061706/0202 Effective date: 20221010 |