WO2015033235A1 - Suivi spécifique à un module dans un environnement de module partagé - Google Patents

Suivi spécifique à un module dans un environnement de module partagé Download PDF

Info

Publication number
WO2015033235A1
WO2015033235A1 PCT/IB2014/060233 IB2014060233W WO2015033235A1 WO 2015033235 A1 WO2015033235 A1 WO 2015033235A1 IB 2014060233 W IB2014060233 W IB 2014060233W WO 2015033235 A1 WO2015033235 A1 WO 2015033235A1
Authority
WO
WIPO (PCT)
Prior art keywords
module
data
application
tracing
request
Prior art date
Application number
PCT/IB2014/060233
Other languages
English (en)
Inventor
Tetsuo Seto
Alexander G. Gounares
Russell KRAJEC
Original Assignee
Concurix Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Concurix Corporation filed Critical Concurix Corporation
Priority to CN201480049023.XA priority Critical patent/CN105723357B/zh
Priority to KR1020167005793A priority patent/KR102202923B1/ko
Priority to EP14843127.3A priority patent/EP3042314B1/fr
Publication of WO2015033235A1 publication Critical patent/WO2015033235A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/865—Monitoring of software
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/88—Monitoring involving counting

Definitions

  • Application tracing is one mechanism to understand and monitor an application. Tracing is a mechanism to collect data while the application executes. In some uses, application tracing may be used for monitoring the ongoing performance of an application. In other uses, application tracing may be used by a developer to understand an application, identify any problems, and improve the application.
  • modules may be distributed as modules, libraries, or other reusable components. These modules may be distributed as source code, intermediate code, executable code, or some other form, but may all share the characteristic that the modules may be reused by other programmers in many different applications.
  • a module-specific tracing mechanism may trace the usage of a module on behalf of the module developer.
  • the module may be used by multiple application developers, and the tracing system may collect and summarize data for the module in each of the different applications.
  • the data may include usage data as well as performance data.
  • Usage data may include anonymized data for each time the module may be invoked and called, and performance data may include the processing time, memory consumption, and other metrics.
  • the module-specific tracing may be enabled or disabled by an application developer.
  • a tracing system may trace applications and their modules, and may make module-specific data available through various interfaces.
  • the tracing system may collect tracer data while an application executes, and may preprocess the data into application-specific and module-specific databases.
  • An analysis engine may further analyze and process these databases to create application-specific views and module- specific views into the data.
  • the application-specific views may be intended for a developer of the application, while the module-specific views may have a public version accessible to everybody and a module developer version that may contain additional details that may be useful to the module developer.
  • a database of module performance may be generated by adding tracing components to applications, as well as by adding tracing components to modules themselves.
  • Modules may be reusable code that may be made available for reuse across multiple applications.
  • the module-specific databases may be public databases that may assist application developers in selecting modules for various tasks.
  • the module-specific databases may include usage and performance data, as well as stability and robustness metrics, error logs, and analyses of similar modules.
  • the database may be accessed through links in module description pages and repositories, as well as through a website or other repository.
  • FIGURE 1 is a diagram illustration of an embodiment showing a system for tracing applications and modules.
  • FIGURE 2 is a diagram illustration of an embodiment showing a network environment with devices that may collect and view application and module trace data.
  • FIGURE 3 is a diagram illustration of an example embodiment showing a user interface for module trace data.
  • FIGURE 4 is a diagram illustration of an embodiment showing an example trace coverage graph.
  • FIGURE 5 is a diagram illustration of an embodiment showing an example module topology graph.
  • FIGURE 6 is a flowchart illustration of an embodiment showing a method for creating applications.
  • FIGURE 7 is a flowchart illustration of an embodiment showing a method for application execution with module tracing.
  • FIGURE 8 is a flowchart illustration of an embodiment showing a method for application execution with application tracing.
  • FIGURE 9 is a flowchart illustration of an embodiment showing a method for preprocessing tracer data.
  • FIGURE 10 is a flowchart illustration of an embodiment showing a method for processing module trace data.
  • FIGURE 11 is a flowchart illustration of an embodiment showing a method for processing requests for module data.
  • a tracing system may collect data about modules that may be
  • the modules may be shared sets of code that may be distributed among developers, and the developers may select various modules to incorporate into their applications.
  • Some of the modules may incorporate a tracing mechanism, which may trace the operations of the module and store tracer data.
  • the tracer data may include usage data, which may describe the number of uses, timestamps for uses, conditions under which the module was used, and other usage data.
  • the tracer data may also include performance data, such as the amount of time taken to execute, amount of computational resources, memory resources, network resources, or other resources consumed during execution.
  • the module specific tracing system may consolidate the raw data for the module developer and for other users. Some embodiments may include a detailed view of the data for module developers and a less detailed view for other users. Module developers may use the tracer data to identify portions of the module that may be executing poorly or have some other issue. The other users may examine the module tracing data to determine a general notion of performance of the module and may use the tracing data as part of the criteria for comparing and selecting one module over another.
  • a module developer may incorporate a tracing mechanism in the module.
  • the tracing mechanism may operate within the confines of the module and only trace code within the module. In many cases, the tracing mechanism may be able to gather some metadata about the environment in which the module was executed.
  • the tracing mechanism may gather tracing data while the module executes in an application.
  • the tracing mechanism may transmit the tracing data to a database for analysis.
  • the application developer may have an option to turn off the tracing mechanism or set various options for the tracing mechanism, even though the tracing mechanism may have been initially incorporated and configured by the module developer.
  • the tracing mechanism may gather usage and performance data that the module developer may use to improve the module. These tracer data may help the module developer understand which portions of the module are used more frequently than others, which may help the module developer prioritize improving the most used portions. The tracer data may also help identify code that is less reliable than other code, and the data may be used to generate robustness or fragility measurements of individual functions.
  • an application developer may access the module specific data to gauge whether or not to use the module in a particular application.
  • the developer may have identified several modules that may serve a particular purpose, and then may use the tracer data as one metric to select between the modules.
  • the application developer may investigate the module's reliability and robustness by viewing the performance and usage data.
  • a tracing system may provide tracing for applications and modules using similar techniques and mechanisms yet with some differences.
  • the tracing system may gather tracing data while an application executes, and that data may be shared with the application developer, the module developer, and a wider audience of potential module users. In some cases, the wider audience may be public at large.
  • Each of the three audiences may have different uses for the tracer data and different security concerns.
  • the application developer may view the application as a trade secret, and may not wish certain tracer information be shared outside of the team developing the application.
  • the module developer may wish to collect data on how the module performed, but may not wish for some details of the operations be disclosed to the general public.
  • the public at large may include developers who may be building their own applications, and these developers may wish to view the module specific data to determine whether or not the module is suited for their use.
  • the application developer may request tracing be performed on their application.
  • Such tracing data may include tracing information that may be proprietary, such as the values of data elements handled by the application, the application architecture and function, source code of the application, and other information. Because the application developer may consider this as secret or proprietary, such information may be processed and stored in a database that may be separate from the data that may be shared with the module developer and the public at large.
  • the data collected for each module may be collected when the application is executed. As such, module-specific data collection may be a subset of the available data because the module-specific data may be shared with a module developer who may be another party other than the application development team. In some cases, the module developer may be a third party who may create and disseminate a module without knowing who may use the module in their application.
  • the module-specific data may be collected as part of executing an application, but only those subsets of data that the application developer may permit to be collected may actually be collected.
  • the application developer may have a set of configuration settings that may enable or disable certain types of data to be collected.
  • certain data elements may not be collected at all for module- specific tracing.
  • an application developer may disable or not install application-level tracing but may permit a module developer to collect tracer data as a module executes within the application.
  • an application may execute without tracing, but when the module is executed, the tracing may occur only within the module.
  • Such module-specific tracing may be processed and made available to the module developer and, in some cases, a wider audience. In such cases, the module- specific tracing may be much more limited in scope than if the application developer had enabled tracing for the entire application.
  • the application developer When an application developer enables tracing for an entire application and permits tracing for individual modules, the application developer may be able to view a complete set of the data relating to each module, with a subset of the data being transmitted and processed in the module-specific manner. In such a situation, the application developer may have access to a superset of data for a specific module than the module developer would be able to access.
  • a module database may use tracing data to decorate descriptions of modules.
  • the module database may list various modules that may be incorporated into applications.
  • the decorations may include performance and usage data, as well as summaries and other data that may be useful for evaluating modules and comparing modules against each other.
  • the module database may be constructed by analyzing tracer data gathered while an application executes a module.
  • a tracer may gather performance and usage data for the module during execution, and these data may be aggregated, summarized, and displayed for a user to browse.
  • the tracer data may include actual usage of the module by third parties, as well as the manners in which the module was incorporated into various applications.
  • the application developers may select and use a module but may only exercise a subset of the module's functionality. In many cases, a module may have many different functions, methods, objects, or other components, yet an application developer may use one a small subset of the components.
  • the third party usage may be gathered when the application is used by an end user.
  • an application may consist of an app that runs on a mobile device along with a backend component that executes on a server in a datacenter.
  • the end user may exercise the application in many different manners, some of which may exercise the module and some which may not.
  • the usage data may reflect the popularity and usefulness of the various components of the module. When these data may be presented to the module developer or to other application developers, the data may be arranged as a popularity score or percentage.
  • the usage data may be tracked over time to determine which applications continue to use the module and which modules are being included and removed from various applications.
  • an application developer may select a module, use the module for a short period of time, then switch to another module.
  • the application developer made a conscious decision to switch from one module to another, indicating the application developer's preference for the second module over the first. This preference may be valuable to another application developer who may be considering the use of the first module.
  • the performance data for the various functions or components within the module may be used to develop a reliability or robustness metric for each function.
  • the reliability or robustness metric may be an indicator of how fragile a function may be, and may be useful for an application developer when selecting specific functions for incorporation in their application.
  • the reliability or robustness metric may be based on the variance of performance metrics or other factors.
  • the module database may include graphical or other indicators of the architecture of the module.
  • a module may include several other modules, each of which may be invoked when an application executes. Such complex interactions may not be readily apparent from reading the source code or from other sources.
  • the graphical representation of the module may give an application developer a visual indication of the complexity of the module and the various dependencies.
  • the module database may roll up or aggregate various metrics about the dependencies of a module to generate data for a given module.
  • the various use and performance data of the modules may be apportioned to the various modules that actually perform the underlying functions. For example, a module may call a second module to perform certain tasks, and one of those tasks may be performed by a third module. In such a case, the first module's function may be displayed along with the second and third module's functions and the data collected from each of the dependencies.
  • module is used to define a group of reusable code that may be incorporated into an application.
  • a module may be known as a 'library', 'subroutine', or some other notion. For the purposes of this specification and claims, these terms are considered synonymous.
  • the "module” may be code that is arranged in a way that multiple applications may access the code, even though the applications may have no connection with each other.
  • a “module” may be code that is configured to be reused.
  • a module may be reused within the scope of a large application, while in other cases, the module may be shared to other application developers who may use the module in disparate and unconnected applications.
  • module or library, where the module may have a defined interface through which an application may invoke and use the module.
  • Some paradigms may allow a programmer to incorporate a module in a static manner, such that the module code does not further change after the application is written and deployed.
  • Some paradigms may allow for dynamic libraries, which may be loaded and invoked at runtime or even after execution has begun. The dynamic libraries may be updated and changed after the application may have been distributed, yet the manner of invoking the libraries or modules may remain the same.
  • Modules may be distributed in source code, intermediate code, executable code, or in some other form.
  • modules may be services that may be invoked through an application programming interface.
  • instrumentation may refer to stubs, hooks, or other data collection mechanisms that may be inserted into executable code and thereby change the executable code
  • profiler or “tracer” may classically refer to data collection mechanisms that may not change the executable code.
  • the use of any of these terms and their derivatives may implicate or imply the other.
  • data collection using a “tracer” may be performed using non-contact data collection in the classic sense of a “tracer” as well as data collection using the classic definition of "instrumentation” where the executable code may be changed.
  • data collected through “instrumentation” may include data collection using non-contact data collection mechanisms.
  • instrumentation may include any type of data that may be collected, including performance related data such as processing times, throughput, performance counters, and the like.
  • the collected data may include function names, parameters passed, memory object names and contents, messages passed, message contents, registry settings, register contents, error flags, interrupts, or any other parameter or other collectable data regarding an application being traced.
  • execution environment may be used to refer to any type of supporting software used to execute an application.
  • An example of an execution environment is an operating system.
  • an "execution environment” may be shown separately from an operating system. This may be to illustrate a virtual machine, such as a process virtual machine, that provides various support functions for an application.
  • a virtual machine may be a system virtual machine that may include its own internal operating system and may simulate an entire computer system.
  • execution environment includes operating systems and other systems that may or may not have readily identifiable "virtual machines” or other supporting software.
  • an application is used to refer to any combination of software and hardware products that may perform a desired function.
  • an application may be a single software program that operates with a hardware platform.
  • Some applications may use multiple software components, each of which may be written in a different language or may execute within different hardware or software execution environments. In some cases, such applications may be dispersed across multiple devices and may use software and hardware components that may be connected by a network or other communications system.
  • references to "a processor” include multiple processors. In some cases, a process that may be performed by “a processor” may be actually performed by multiple processors on the same device or on different devices. For the purposes of this specification and claims, any reference to “a processor” shall include multiple processors which may be on the same device or different devices, unless expressly specified otherwise.
  • the subject matter may be embodied as devices, systems, methods, and/or computer program products. Accordingly, some or all of the subject matter may be embodied in hardware and/or in software (including firmware, resident software, microcode, state machines, gate arrays, etc.) Furthermore, the subject matter may take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system.
  • a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • the computer-usable or computer-readable medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.
  • computer readable media may comprise computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by an instruction execution system.
  • the computer-usable or computer-readable medium could be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, of otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
  • FIG. 1 is an illustration of an example embodiment 100 showing a tracer data collection system.
  • Embodiment 100 may be an overview of a process that collects tracer data from an application. The tracer data may fall into application-specific or module-specific classifications, and may be handled differently based on the
  • a tracer may be incorporated into individual modules or an application as a whole.
  • the tracer output may be used to populate a module database, which may be used by application developers to evaluate, compare, and select modules for their application.
  • the module database may include records for each module for which the tracing system has gathered data.
  • a module developer may incorporate a tracing mechanism into a module.
  • the embedded tracer may collect data for that module.
  • the tracer may gather data for that module but not for the remainder of the application.
  • the tracer data may be accessed in multiple manners. Module developers may access tracer data for their modules and view more detailed tracer data than the general public, which may have access to a subset of the tracer data for the module.
  • Application developers may access application-specific data, which may be more detailed than the data available to the module developers or the general public.
  • the application may be a proprietary project that may include trade secrets or other information that the application developer may not wish to share.
  • This application- specific data may include, for example, the control and sequence of the application, data types handled by the application, the raw data processed by the application, and other information that may be proprietary.
  • the application-specific data may be stored in a separate database than module-specific data and access to the application-specific data may be limited to authorized users.
  • module developer may have created and distributed a module so that application developers may reuse the module.
  • Module developers may be commercial software companies as well as open source software developers. Such developers may desire to see their modules in use, either for commercial purposes or for the satisfaction of contributing to the community.
  • the tracer data that may be collected from an application but made available to the module developers may be sanitized, anonymized, or otherwise scrubbed to remove proprietary information from the data. Such operations may limit the application-specific information in the module traces, but may enable the module developer to have access to the module specific data.
  • a module developer may access module-specific data to monitor the deployment and use of the module, as well as to identify performance issues with the module.
  • the module-specific data may also be made available to a wider audience, such as the general public.
  • the general public may make use of the module-specific data to compare and select modules.
  • a module developer 102 may contribute modules 104, which may be used by an application developer 106 to build an application 108.
  • a tracing module 110 may be incorporated into individual modules 104 or into the application 108. When a tracing module 110 is incorporated into one or more modules 104, those modules may be traced. When a tracing module 110 is incorporated into the application 108, all of the application 108 may be traced, including any modules included in the application 108.
  • the application 108 may be executed in an execution environment 112. During execution, a tracer 114 may gather data, which may be passed to a preprocessor 116. In many cases, the tracer 114 may gather data and transmit those data to the preprocessor 116 on a periodic basis.
  • the preprocessor 116 may perform lightweight analyses, formatting, or other processing, then store application-specific data in an application database 118 and module-specific data in various module databases 120.
  • the module databases 120 may be configured with a separate database for each module that may be traced.
  • An analysis engine 122 may perform further analysis of the stored data to produce analyzed application data 124 or analyzed module data 126, respectively.
  • the analysis engine 122 may perform many different types of analyses, including analyzing historical data, summarizing usage and performance statistics, graphing and charting data, and other analyses.
  • the analysis engine 122 may perform analyses on demand, meaning that some analyses may be performed when the analyzed data may be requested.
  • the analysis engine 122 may perform analyses ahead of time so that the analyzed data may be readily available when requested.
  • a module developer 102 may have private access 130 to the analyzed module data 128.
  • the module developer's private access of the module-specific data may include details about performance and usage.
  • an application developer 106 may have public access 132 to the analyzed module data 128, which may contain fewer details and only a subset of the data available through the private access 130 of the module developer 102.
  • the public access 132 may include summaries of the tracer data collected for the module, including performance and usage statistics. An example of such a user interface may be found later in this specification.
  • An application developer 106 may have private access 126 to the analyzed application data 124. This access may include extensive data regarding the performance of the application as a whole, including the performance of the various modules. In some cases, the application developer 106 may be able to access more data or a different set of data than a module developer 104. For example, an application developer 106 may be able to access parameter values passed to a module, where the parameter values may be proprietary and not available to the module developer 104.
  • the application developer 106 may have control over which types of data may be made available to the module databases 120. For example, the application developer 106 may fully turn off any sharing of the module-specific data, but such data may still be collected, stored, and made available through the private access 126 of the application developer 106.
  • the application developer 106 may place various limits on the data that may be shared in the module databases. For example, the application developer 106 may permit usage statistics to be collected, but may not permit values of variables to be collected. The application developer 106 may establish that the data may be obfuscated or anonymized prior to being included in the module databases 120.
  • Figure 2 is a diagram of an embodiment 200 showing components that may collect data when an application executes and present various user interfaces showing the collected data.
  • the example of embodiment 200 is merely one example of a multi-device system that may generate and view tracer data. Other architectures may include single device and multiple device architectures.
  • the architecture of embodiment 200 includes a device 202 on which the tracer data may be collected, as well as several other devices for storing and processing different elements of the collected data.
  • a client device may present and view the collected data.
  • some or all of the functions illustrated may be combined into one or more devices.
  • the diagram of Figure 2 illustrates functional components of a system.
  • the component may be a hardware component, a software component, or a combination of hardware and software.
  • Some of the components may be application level software, while other components may be execution environment level components.
  • the connection of one component to another may be a close connection where two or more components are operating on a single hardware platform. In other cases, the connections may be made over network connections spanning long distances.
  • Each embodiment may use different hardware, software, and interconnection
  • Embodiment 200 illustrates a device 202 that may have a hardware platform 204 and various software components.
  • the device 202 as illustrated represents a conventional computing device, although other embodiments may have different configurations, architectures, or components.
  • the device 202 may be a server computer. In some embodiments, the device 202 may still also be a desktop computer, laptop computer, netbook computer, tablet or slate computer, wireless handset, cellular telephone, game console or any other type of computing device.
  • the hardware platform 204 may include a processor 208, random access memory 210, and nonvolatile storage 212.
  • the hardware platform 204 may also include a user interface 214 and network interface 216.
  • the random access memory 210 may be storage that contains data objects and executable code that can be quickly accessed by the processors 208. In many embodiments, the random access memory 210 may have a high-speed bus connecting the memory 210 to the processors 208.
  • the nonvolatile storage 212 may be storage that persists after the device 202 is shut down.
  • the nonvolatile storage 212 may be any type of storage device, including hard disk, solid state memory devices, magnetic tape, optical storage, or other type of storage.
  • the nonvolatile storage 212 may be read only or read/write capable.
  • the nonvolatile storage 212 may be cloud based, network storage, or other storage that may be accessed over a network connection.
  • the user interface 214 may be any type of hardware capable of displaying output and receiving input from a user.
  • the output display may be a graphical display monitor, although output devices may include lights and other visual output, audio output, kinetic actuator output, as well as other output devices.
  • Conventional input devices may include keyboards and pointing devices such as a mouse, stylus, trackball, or other pointing device.
  • Other input devices may include various sensors, including biometric input devices, audio and video input devices, and other sensors.
  • the network interface 216 may be any type of connection to another computer.
  • the network interface 216 may be a wired Ethernet connection.
  • Other embodiments may include wired or wireless connections over various communication protocols.
  • the software components 206 may include an operating system 218 on which various software components and services may operate.
  • the application 222 may be executed in an operating system 218 or in an execution environment 220.
  • An execution environment 220 may have memory management, process scheduling, and other components that may manage application execution in a similar manner to an operating system 218.
  • a tracing gatherer 224 may work with either the operating system 218 or execution environment 220.
  • the tracing gatherer 224 may include a tracer 226 and a communications manager 228.
  • the tracer 226 may monitor the operations of the application 222, while the communications manager 228 may transmit the tracer data to a preprocessor system 240.
  • the tracer 226 and communications manager 228 may be components of a tracer that may be included in the application 222.
  • the application 222 may have a tracer 230 which may trace the entire application 222, including all of the modules 234.
  • a tracer 236 may be included in the specific module 234 to be traced.
  • the application 222 may include a tracer configuration 232 which may define different parameters for the tracer.
  • the tracer configuration 232 may define which data elements may be collected, the precision of the data being collected, which data elements may be shared with module developers, and other items.
  • the tracer configuration 232 may define one configuration for one module and a different configuration for another module.
  • the communications manager 228 may package and transmit tracer data to a preprocessor system 240, which may be accessed over a network 238.
  • the preprocessor system 240 may have a hardware platform 242, which may be similar to the hardware platform 204, and on which a preprocessor 244 may operate.
  • the preprocessor 244 may receive tracer data and perform some preliminary processing prior to storing the data in the application database server 246 or the module database server 254. In many cases, the preprocessor 244 may be designed to handle a high volume of tracer data.
  • the application database server 246 may have a hardware platform 248, which may be similar to the hardware platform 204, on which two databases may operate.
  • An application database 250 may contain application-specific tracer data in raw or preprocessed form.
  • An analyzed application database 252 may contain analyzed application data that may be ready for viewing by an application developer.
  • the module database server 254 may have a hardware platform 256, which may be similar to the hardware platform 204, on which two databases may operate.
  • a module database 258 may contain module-specific tracer data in raw or preprocessed form.
  • An analyzed module database 260 may contain analyzed module data that may be ready for viewing by a module developer or a third party.
  • An analysis system 262 may have a hardware platform 264, which may be similar to the hardware platform 204, on which an analysis engine 266 may execute.
  • the analysis engine 266 may perform various analyses of the application tracer data or module tracer data. The analyses may include summarizing the data, combining the tracer data with other data sources, visualizing the data, or other operations on the data.
  • An access portal system 268 may have a hardware platform 270, which may be similar to the hardware platform 204, on which an access portal 272 may execute.
  • the access portal 272 may be a web service or other application that may gather data from the analyzed application database 252 or the analyzed module database 260 for display on a client system 274.
  • the access portal 272 may include authentication systems, user account and login systems, billing and accounting systems, and other functions.
  • the client system 274 may have a hardware platform 276, which may be similar to the hardware platform 204, on which a browser 278 may execute.
  • the browser 278 may be used to access the access portal 272 and generate a user interface 280.
  • the user interface 280 may be different based on the user and the user's credentials.
  • application developers may be able to view application data for their applications, as well as the module database for third party or general consumption.
  • a module developer may be able to see detailed module-specific data for their modules but not for other modules or for applications.
  • a third party may be able to view module information permitted for general consumption but not be able to access application data or detailed module-specific data.
  • Figure 3 is an example embodiment 300 showing a user interface for module trace data.
  • Embodiment 300 is a user interface 302 that may be an example of a publically available module-specific user interface for a module named CONFIG.
  • the user interface 302 may represent the type of data that may be publically available after being gathered from a tracer.
  • the tracer may be a module- specific tracer or may be an application-level tracer.
  • the type of data illustrated in the example of embodiment 300 may be merely illustrative as possible types of data and possible methods for aggregating and displaying the data. Other embodiments may have different types of data and mechanisms for communicating the data.
  • a name 304 may identify the module as CONFIG.
  • a set of summarized ratings 306 may give a user a high level summary of the module's reliability, popularity, and how the module is trending. Reliability may be a metric derived from usage and performance data that may reflect the robustness or fragility of the module as a whole.
  • Popularity may be a metric that reflects the community's usage of the module. In some cases, the popularity may reflect the module's popularity in comparison to the community as a whole, in comparison to comparable modules, or in some other context.
  • a trending indicator may indicate if the module is increasing or decreasing in overall popularity and robustness. If the module is being used less and less or if the subsequent releases of the module are poorer performing than previous releases, the trend indicator may be down. Conversely, if the module is gaining users and each release of the module increases reliability, the trend may be upwards.
  • the reliability, popularity, and trending indicators are merely three examples of high level summary indicators that may be useful for a user interface describing a particular module.
  • a set of dataset information 308 may display the quantity of data that may underlie the displayed data.
  • the number of datasets analyzed may be 252,000 and the number of applications using the module may be 15,000. These numbers may lend credibility to the overall data, giving the views confidence that the performance and usage data are based on a statistically significant population of data.
  • a set of function-specific data 310 may show observations for individual functions within a module. Many modules may include multiple functions, objects, or other components, each of which may be called or invoked individually.
  • lines 314, 316, 318, and 320 may illustrate summary data for config.foo, config.bar, config.baz, and config.qux, respectively.
  • the type of function-specific data may include a use percentage, which may indicate which of the functions are used the most. In the case of config.qux, the use percentage may be 0, which may occur when no trace data exists for the function.
  • the source code for the config module may be read to identify each of the available functions. The list of functions may be compared with the tracer data to generate some of the function specific data 310.
  • An error rate may be determined for each function, as well as the CPU consumption and memory consumption.
  • the resource consumption of CPU and memory may be given as a mean with a standard deviation.
  • the standard deviation may be one metric of a function's stability or reliance.
  • a reliability score for the function may also be included. The reliability score may be determined using an algorithm or heuristic that may capture the variance in resource consumption.
  • a graph of usage trends 320 may be one mechanism that shows usage of the function over time.
  • the top portion 322 of the graph may show new applications that add the module, while the bottom portion 324 may show applications that no longer use the module.
  • a module may be added to an application during an initial phase, then removed later when an application developer elects to change out the module for another one.
  • This usage pattern is one mechanism that may indicate that the second module may be better suited for the application that the current module.
  • the desirability of the second module may be strongly indicated and the undesirability of the first module may also be strongly indicated.
  • the graph may be interactive, and an example interactive box 326 may be placed on the user interface when a user hovers or clicks on one of the bars in the graph.
  • the interactive box 326 may show underlying data for the selected bar.
  • a coverage graph 328 may visually illustrate the components of the module for which trace data exists. An example of a coverage graph may be found later in this specification.
  • a module topology graph 330 may visually illustrate the links between the current module and other modules that the current module may call.
  • An example of a module topology graph may be found later in this specification.
  • a competing modules area 332 may list similar or competitive modules to the current module. The modules listed may have hot links, buttons, or other mechanisms that may cause the user interface to change to that module. The competing modules may include indicators showing the relative strength of the other modules, the module's trends, or some other indicators.
  • FIG. 4 is an example diagram of an embodiment 400 showing a trace coverage graph.
  • the graph 402 may show various functions or components of a module as the nodes of the graph.
  • the edges of the graph may reflect the connections or sequences of execution of the nodes, and may be drawn to reflect amount of data that were used to generate the coverage graph.
  • each of the nodes of graph 402 may be labeled with references to the executable code represented by each of the nodes. For the sake of simplicity in the figure, such labels have been removed.
  • nodes 404, 406, 408, and 410 may be connected with thick, heavy lines. Such lines may indicate that a large amount of trace data may be present for that sequence of execution. In contrast, the sequence of node 404, 412, 414, and 416 may have much less supporting data. In the case of nodes 418, 420, 422, and 424, the dashed lines may indicate that no trace data may be available. In such a case, the code associated with nodes 418, 420, 422, and 424 may never have been exercised by an application.
  • the graph 402 may be an interactive graph. As an example of an interaction, a user may hover, click, select, or otherwise indicate node 404 and an interactive component 426 may be displayed. The interactive component 426 may display additional underlying data about the node.
  • FIG. 5 is an example diagram of an embodiment 500 showing a module topology graph.
  • the graph 502 may show a module and its dependencies, which may be other modules that may be included or called from the base module.
  • the nodes of the graph may reflect the base module and its dependencies.
  • the edges of the graph may reflect the connections or function calls to the dependent modules.
  • the graph 502 may be a visual image of the call structure of a module, and may be used to give a user a graphical view of the complexity and dependencies of a module.
  • a module config 504 may be illustrated as a shaded or highlighted node. This node may represent the base node for the graph.
  • the nodes 506, 508, 510, 510, 512, 514, 516, and 518 may represent modules alpha, beta, gamma, delta, epsilon, zeta, and eta, respectively.
  • the interconnections illustrate the function calls or other dependencies between modules.
  • the module config 504 is shown to call node 510, module gamma, which in turn calls node 514, module epsilon.
  • Module epsilon, node 514 calls modules zeta and eta, as represented by nodes 516 and 518. This structure may communicate to a viewer how module eta on node 518 relates to module config 514.
  • FIG. 6 is a flowchart illustration of an embodiment 600 showing a method for creating applications.
  • Embodiment 600 illustrates a general method that an application developer may use to create an application that includes one or more modules or libraries.
  • a developer may begin coding an application block 602. While coding, the developer may identify a function in block 604 that may prompt a search in block 606 for modules that may perform the function. From the list of candidate modules in block 606, the developer may evaluate each candidate in block 608.
  • the developer may examine the module-specific trace data in block 610 for each of the candidate modules.
  • An example of such data may be found in the user interface of embodiment 300. From these data, the developer may be able to select an appropriate module in block 612 and incorporate the module into the application in block 614.
  • the application developer may be able to configure various tracing parameters for the module in block 618.
  • the tracing parameters may allow the application developer to select different options for the tracer.
  • the tracing parameters may be configured in many different manners to allow the application developer to control how the module may be traced.
  • the module tracing may be requested by a module developer to address specific goals that the module developer may have, yet the application developer may have the final approval and control over how the module tracing may occur. In many cases, the application developer may be able to completely disable tracing for the module, as well as to limit or expand some of the parameters that a tracer may collect.
  • the tracing frequency may be part of the tracer configuration.
  • tracing may consume processing and memory resources.
  • the tracing may be performed on a sampling basis or may have other architectures that limit the amount of resources consumed by tracing.
  • the application developer may be incented to permit tracing for the module because the module tracing data may be fed back to the module developer to help improve the module, as well as to further populate a public database for the module.
  • the application developer may have already accessed the public database in block 610 and may wish to give back to the community by permitting the module tracing.
  • a tracing module may be added in block 624 and the application specific tracing may be configured in block 626.
  • the application developer may compile the code in block 628 and execute the code in block 630.
  • Figure 7 is a flowchart illustration of an embodiment 700 showing a method for executing applications with module tracing.
  • Embodiment 700 illustrates how an application may be executed with module-specific tracing. The module-specific tracing may occur only when the module executes and may not operate when other portions of the application execute.
  • An application may be received in block 702 and begin execution in block 704.
  • a module may be encountered in block 706.
  • the module may be loaded in block 708 and begin execution in block 710.
  • tracing may be turned on in block 714.
  • the tracing may be performed by a separate thread or process, or may be incorporated into a single thread with the module itself. If the tracing is not included in the module, the tracing may not be turned on.
  • the module tracer operations in block 718 may be performed.
  • the module tracer may collect tracing data in block 720 and send the tracer data to a preprocessor in block 722.
  • the tracer data may be sent to the preprocessor on a periodic basis, such as every second, every several seconds, every minute, or some other frequency.
  • the module processing may continue in block 724 by looping back to block 716.
  • processing may continue to block 726.
  • the process may loop back to block 706.
  • the application may end in block 728.
  • Figure 8 is a flowchart illustration of an embodiment 800 showing a method for executing applications with both application and module tracing.
  • Embodiment 800 illustrates how an application may be executed with application-specific and module-specific tracing.
  • Application-specific tracing may occur while the application executes, and module-specific tracing may occur while various modules execute.
  • Embodiment 800 may be compared to embodiment 700 where module-specific tracing may occur without application-specific tracing.
  • An application may be received in block 802 and begin execution in block 804. When the application includes tracing in block 806, application tracing may begin in block 808. The operations of the tracer may be illustrated in block 810.
  • the application may be executed in block 814. While the application executes in block 814, the tracer may collect application-specific tracer data in block 812.
  • the module may be executed in block 818. While the module executes in block 818, the tracer may collect tracer data in block 820.
  • the tracer may send tracer data to a preprocessor in block 822.
  • the tracer data may be transmitted on a periodic basis, for example.
  • Figure 9 is a flowchart illustration of an embodiment 900 showing a method for preprocessing tracer data.
  • Embodiment 900 may be performed to gather tracer data and dispatch the data to the appropriate databases.
  • the data may be further processed and analyzed by an analysis engine once the data are in the databases.
  • Embodiment 900 is one example of a preprocessor. In many
  • the preprocessor may handle large volumes of data. Consequently, the preprocessor may perform a limited amount of analysis and may operate in a lightweight fashion.
  • the operations of embodiment 900 may be performed on each packet or message sent from a tracer.
  • the trace data may be received in block 902.
  • the trace data may come in a packet, message, or other form that may contain a group of observations, metadata, and other information gathered by a tracer.
  • the module- specific data may be extracted in block 906, anonymized in block 908, and sent to a module preprocessor in block 910. If the trace data is module trace data in block 904, the trace data is sent to the module preprocessor.
  • module-specific data in blocks 906 and 908 may remove data that may identify the application, data handled by the application, or other information that may relate to the application. These data may, in some cases, be considered proprietary and therefore are removed prior to being added to the module database.
  • the operations of a module preprocessor are illustrated in block 912.
  • An initial analysis of the module-specific data may be performed in block 914.
  • the new data may be aggregated into existing module data in block 916, and the module database may be updated in block 918.
  • the data in the module database may be further processed by an analysis engine to generate data viewable by the module developer as well as a wider audience, which may include the general public.
  • the application-specific data may be processed an application preprocessor as illustrated in block 920.
  • An application preprocessor may perform initial analysis on the application data in block 922, aggregate the new data into existing application data in block 924, and update the application database in block 926.
  • FIG 10 is a flowchart illustration of an embodiment 1000 showing a method for analyzing tracer data.
  • Embodiment 1000 may be performed by an analysis engine to incorporate module trace data into an analyzed module database. From the analyzed module database, the data may be presented to a user with a user interface such as the user interface of embodiment 300.
  • Preprocessed trace data may be received in block 1002.
  • the module metadata may be extracted from the data in block 1004.
  • a process may be executed to add the module to the database beginning in block 1008.
  • an entry in the analyzed module database may be created.
  • the source code for the module may be retrieved in block 1010 and parsed in block 1012 to locate the exported functions and other objects.
  • the function or object may be added to the analyzed trace database in block 1016. The process may continue at block 1018.
  • module level data elements may be extracted from the data in block 1018 and the analyzed module database may be updated in block 1020.
  • the functions or other objects in the data may be identified in block 1022.
  • the statistics relating to the function may be updated in block 1026 and the statistics used to update the analyzed module database in block 1028.
  • Any statistics for the module as a whole may be updated in block 1030 and the updates may be published in the analyzed module database in block 1032.
  • Figure 11 is a flowchart illustration of an embodiment 1100 showing a method for servicing requests for data from analyzed trace data.
  • Embodiment 1100 may be performed by a portal server in response to a request from a client device.
  • a request may be received in block 1102 for summary data for a particular module. If the user is not an authenticated user in block 1104, the general data for the module may be retrieved in block 1106 and transmitted to the user in block 1108. If the user is an authenticated user in block 1104, the module developer data may be retrieved in block 1110 and transmitted in block 1108. [00165] In the example of embodiment 1100, the notion of the data being delivered as a 'page' may refer to an example the delivery of the data in the form of a web page. Some embodiments may transmit the data in other manners to be rendered or presented to a user in a user interface.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

L'invention concerne un mécanisme de suivi spécifique à un module qui permet de suivre l'utilisation d'un module pour le compte d'un développeur de module. Le module peut être utilisé par de multiples développeurs d'applications, et le système de suivi peut recueillir et compiler des données pour le module dans chacune des applications différentes. Les données peuvent comprendre des données d'utilisation ainsi que des données de performances. Les données d'utilisation peuvent comprendre des données rendues anonymes pour chaque moment où le module peut être invoqué et appelé, et les données de performances peuvent comprendre le temps de traitement, la consommation de mémoire et d'autres mesures. Le suivi spécifique à un module peut être activé ou désactivé par un développeur d'applications.
PCT/IB2014/060233 2013-09-04 2014-03-27 Suivi spécifique à un module dans un environnement de module partagé WO2015033235A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201480049023.XA CN105723357B (zh) 2013-09-04 2014-03-27 共享模块环境中的因模块而异的跟踪
KR1020167005793A KR102202923B1 (ko) 2013-09-04 2014-03-27 공유 모듈 환경 내의 모듈 특정 트레이싱 기법
EP14843127.3A EP3042314B1 (fr) 2013-09-04 2014-03-27 Suivi spécifique à un module dans un environnement de module partagé

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201361873773P 2013-09-04 2013-09-04
US61/873,773 2013-09-04
US201361874931P 2013-09-06 2013-09-06
US201361874927P 2013-09-06 2013-09-06
US61/874,927 2013-09-06
US61/874,931 2013-09-06

Publications (1)

Publication Number Publication Date
WO2015033235A1 true WO2015033235A1 (fr) 2015-03-12

Family

ID=52627860

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2014/060233 WO2015033235A1 (fr) 2013-09-04 2014-03-27 Suivi spécifique à un module dans un environnement de module partagé

Country Status (4)

Country Link
EP (1) EP3042314B1 (fr)
KR (1) KR102202923B1 (fr)
CN (1) CN105723357B (fr)
WO (1) WO2015033235A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110831030A (zh) * 2018-08-13 2020-02-21 华为技术有限公司 一种信号覆盖效果图的获取方法及网络设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080120400A1 (en) * 2006-11-16 2008-05-22 Alexander Keller Systems and Methods for Constructing Relationship Specifications from Component Interactions
US20100281468A1 (en) * 2008-04-01 2010-11-04 Kaspersky Lab, Zao Method and system for monitoring execution performance of software program product
US20110314543A1 (en) * 2010-06-16 2011-12-22 Microsoft Corporation System state based diagnostic scan
US20120023475A1 (en) * 2010-05-19 2012-01-26 Google Inc. Bug Clearing House
US20120079456A1 (en) * 2010-09-23 2012-03-29 International Business Machines Corporation Systems and methods for identifying software performance influencers
US20130145350A1 (en) * 2011-12-05 2013-06-06 Microsoft Corporation Efficient, large scale trace storage system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020087949A1 (en) * 2000-03-03 2002-07-04 Valery Golender System and method for software diagnostics using a combination of visual and dynamic tracing
US7640459B2 (en) * 2006-09-30 2009-12-29 Sap Ag Performing computer application trace with other operations
US9043788B2 (en) * 2012-08-10 2015-05-26 Concurix Corporation Experiment manager for manycore systems

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080120400A1 (en) * 2006-11-16 2008-05-22 Alexander Keller Systems and Methods for Constructing Relationship Specifications from Component Interactions
US20100281468A1 (en) * 2008-04-01 2010-11-04 Kaspersky Lab, Zao Method and system for monitoring execution performance of software program product
US20120023475A1 (en) * 2010-05-19 2012-01-26 Google Inc. Bug Clearing House
US20110314543A1 (en) * 2010-06-16 2011-12-22 Microsoft Corporation System state based diagnostic scan
US20120079456A1 (en) * 2010-09-23 2012-03-29 International Business Machines Corporation Systems and methods for identifying software performance influencers
US20130145350A1 (en) * 2011-12-05 2013-06-06 Microsoft Corporation Efficient, large scale trace storage system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110831030A (zh) * 2018-08-13 2020-02-21 华为技术有限公司 一种信号覆盖效果图的获取方法及网络设备
CN110831030B (zh) * 2018-08-13 2021-09-14 华为技术有限公司 一种信号覆盖效果图的获取方法及网络设备

Also Published As

Publication number Publication date
EP3042314A1 (fr) 2016-07-13
KR102202923B1 (ko) 2021-01-13
CN105723357A (zh) 2016-06-29
CN105723357B (zh) 2019-06-28
EP3042314B1 (fr) 2018-06-27
EP3042314A4 (fr) 2017-05-17
KR20160058768A (ko) 2016-05-25

Similar Documents

Publication Publication Date Title
US9864672B2 (en) Module specific tracing in a shared module environment
US9298588B2 (en) Tracing system for application and module tracing
US9311213B2 (en) Module database with tracing options
Beyer et al. Reliable benchmarking: requirements and solutions
US11880270B2 (en) Pruning and prioritizing event data for analysis
Yadwadkar et al. Selecting the best vm across multiple public clouds: A data-driven performance modeling approach
Sambasivan et al. Principled workflow-centric tracing of distributed systems
US10346292B2 (en) Software component recommendation based on multiple trace runs
US20150143180A1 (en) Validating software characteristics
Garcia Pinto et al. A visual performance analysis framework for task‐based parallel applications running on hybrid clusters
US12093389B2 (en) Data traffic characterization prioritization
US11449409B2 (en) Schema inference and log data validation system
US8412744B2 (en) Visualization of runtime analysis across dynamic boundaries
Ochei et al. Degrees of tenant isolation for cloud-hosted software services: a cross-case analysis
Sfaxi et al. Babel: a generic benchmarking platform for Big Data architectures
EP3042314B1 (fr) Suivi spécifique à un module dans un environnement de module partagé
US11409769B2 (en) Computer-implemented method and system for attribute discovery for operation objects from operation data
Silva et al. Adding domain data to code profiling tools to debug workflow parallel execution
Sfaxi et al. Designing and implementing a Big Data benchmark in a financial context: application to a cash management use case
Iosup et al. Towards benchmarking IaaS and PaaS clouds for graph analytics
Fernando Implementing Observability for Enterprise Software Systems
US20240202100A1 (en) System, method, and graphical user interface for temporal presentation and navigation of code path data
US12093670B2 (en) System, method, and graphical user interface for temporal presentation of stack trace and associated data
Ferreira A javascript framework comparison based on benchmarking software metrics and environment configuration
Palm et al. A Bayesian system for cloud performance diagnosis and prediction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14843127

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20167005793

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2014843127

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2014843127

Country of ref document: EP