WO2015156808A1 - Partial snapshots for creating generalized snapshots - Google Patents

Partial snapshots for creating generalized snapshots Download PDF

Info

Publication number
WO2015156808A1
WO2015156808A1 PCT/US2014/033597 US2014033597W WO2015156808A1 WO 2015156808 A1 WO2015156808 A1 WO 2015156808A1 US 2014033597 W US2014033597 W US 2014033597W WO 2015156808 A1 WO2015156808 A1 WO 2015156808A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen
pixels
partial
snapshot
view
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2014/033597
Other languages
French (fr)
Inventor
Nitsan Amichai
Haim SHUVALI
Michael Gopshtein
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hewlett Packard Development Co LP
Original Assignee
Hewlett Packard Development Co LP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett Packard Development Co LP filed Critical Hewlett Packard Development Co LP
Priority to PCT/US2014/033597 priority Critical patent/WO2015156808A1/en
Priority to US15/115,365 priority patent/US9992379B2/en
Publication of WO2015156808A1 publication Critical patent/WO2015156808A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/44—Secrecy systems
    • H04N1/448—Rendering the image unintelligible, e.g. scrambling
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/362—Debugging of software
    • G06F11/366—Debugging of software using diagnostics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/44—Secrecy systems
    • H04N1/4446—Hiding of documents or document information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H04N25/441—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by reading contiguous pixels from selected rows or columns of the array, e.g. interlaced scanning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H04N25/443—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H04N25/445—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by skipping some contiguous pixels within the read portion of the array
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/86—Event-based monitoring

Definitions

  • Applications e.g., computer software
  • Ut user interface
  • a snapshot of an application refers to a screen capture of pari or ail of the screen presented to the user. Snapshots may be captured by various tools (e.g., testing tools for mobile or web applications) at various times and/or in response to various events (e.g., user actions).
  • a snapshot may be an image file, digital file or some other type of binary data that includes a number of color pixels.
  • FIG. 1 is a block diagram of an example computing environment in which partial snapshots for creating generalized snapshots may be useful
  • FIG. 2 is a block diagram of an example partial snapshot generator for partial snapshots for creating generalized snapshots
  • FIG. 3 is a flowchart of an example method for partial snapshots for creating generalized snapshots
  • FIG. 4 is a flowchart of an example method for partial snapshots for creating generalized snapshots
  • FIG. 5 is a block diagram of an example computing device for partial snapshots for creating generalized snapshots.
  • FIG. 6 is a biock diagram of an example computing device for partial snapshots for creating generalized snapshots.
  • Snapshots may be useful for testing an application.
  • a testing tool may capture or record a certain flow of an application ⁇ e.g., a series of user actions such as button clicks, etc.) such that the flow can be analyzed, it may be useful for various steps of such a flow to include at least one snapshot, for example, to add application context to a textual description of the step or action.
  • Adding appiication context via snapshots may allow a user (e.g., a developer or debugger) to better and more quickly understand a performance incident or error.
  • snapshots originate from applications being used by reai users, it may be desirable to maintain the users' privacy.
  • Some pre-production testing tools may capture screen shots and attach them to a report of the text execution. Because, in these situations, the appiication is not being used by reai users, the appiication does not contain any persona! information that needs to be protected. Such applications may use fake or test information instead of real user information. As such, for each test step or simulated user action, a single screen shot may be attached. These tools do not generate partial snapshots for creating generalized snapshots.
  • Some production testing tools, or monitoring toois may also capture screen shots and attach them to a report of the text execution.
  • synthetic monitoring toots may periodically check the status an appiication ⁇ e.g., whether the application is online, available, running, etc.).
  • some synthetic monitoring tools may check general functionalities of an application that do not require a user login.
  • some synthetic monitoring tools may check functionalities of an application that do require a user login by using a test account instead of a real user account. In ait of these situations, no real user information is needed for the testing. As such, for each test step or simulated user action, a single screen shot may be attached. These toois do not generate partial snapshots for creating generalized snapshots.
  • Some other production testing tools may capture screen shots that are associated with reai user actions as real users use the application, in these situations, the snapshots may contain sensitive or private information of the users.
  • a snapshot couid include a user's mailing address, account number, credit card information or the like. This may pose a privacy concern for many users, as the users may not trust the developers or debuggers that are using the snapshots. Thus, it may be desirable for real user monitoring tools to use snapshot information while maintaining users' privacy. It may be desirable to create a generalized or scrubbed snapshot that testers or debuggers can use without seeing any sensitive reai user information.
  • a client e.g., a computing device
  • the screen or view may include multiple pixels arranged over an area.
  • the client may sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. This sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area.
  • the client may then transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots.
  • the resulting generalized snapshot may be a "generic" or “most representative” ⁇ but scrubbed) snapshot based on the screen of multiple real users.
  • the generalized snapshot may preserve the users' privacy by not sending confidential/persona! information over a network to the monitoring system.
  • FIG. 1 is a block diagram of an example computing environment 100 in which partial snapshots for creating generalized snapshots may be useful.
  • Computing environment 100 may include a monitoring system 110 and a number of clients (e.g., 120, 130, 140, etc.). Each client may provide, e.g., at a particular time or in response to a particular event, a partial snapshot to monitoring system 110.
  • Each partial snapshot may foe an image fiie, digital fiie or some other type of binary data that includes a number of color pixels.
  • the clients may provide the partial snapshots to the monitoring system 1 10 over at ieast one network.
  • Such a network may be any wired or wireless network, and may include any number of hubs, routers, switches, cell towers or the like.
  • Such a network may be, for example, part of a cellular network, part of the internet, part of an intranet and/or other type of network.
  • Monitoring system 1 10 may receive, e.g., at particular times or in response to particular events, the multiple partial snapshots from the various clients (e.g., 120, 130, 140, etc.) and may create a generalized snapshot based on these multiple partial snapshots.
  • Monitoring system 1 10 may include at Ieast one computing device that is capable of receiving multiple partial snapshots from the various clients (e.g., 120, 30, 140, etc.) and creating a generalized snapshot.
  • the term "system” may be used to refer to a single computing device or multiple computing devices that communicate with each other ⁇ e.g., via a network) and operate together to provide a service.
  • Monitoring system 110 may include a monitoring tool 112 that runs on monitoring system 1 10.
  • Monitoring too! 1 12 may be any type of application testing toot (e.g., a real user monitoring too! that receives execution information (e.g., execution reports, errors, performance issues, etc.) from various applications ⁇ e.g., 122) running on various clients (e.g., 120, 130, 140, etc.). Monitoring tool 1 12 may also be capable of receiving snapshots and/or partial snapshots that may be attached to or included as part of the received execution information. Monitoring tool 1 12 may include instructions ⁇ e.g., stored on a machine-readable storage medium of system 110) that, when executed (e.g., by a processor of system 1 10), implement the functionality of the monitoring tool 1 12. Alternatively or in addition, monitoring too!
  • execution information e.g., execution reports, errors, performance issues, etc.
  • Monitoring tool 1 12 may also be capable of receiving snapshots and/or partial snapshots that may be attached to or included as part of the received execution information.
  • Monitoring tool 1 12 may include instructions ⁇ e.g., stored on a machine-
  • Monitoring system 110 may include a generalized snapshot generator 114.
  • Generalized snapshot generator 114 may be included as part of monitoring tool 112 or may be independent of (e.g., in communication with) monitoring tool 1 12.
  • Generalized snapshot generator 1 14 may receive multiple partial snapshots from the various clients (e.g., 120, 130, 140, etc.). in particular, generalized snapshot generator 114 may receive partial snapshots from partial snapshot generators (e.g., 124) running on the various clients. Generalized snapshot generator 114 may create a generalized snapshot based on these multiple partial snapshots. More specifically, generalized snapshot generator 114 may group received partial snapshots according to their classification (e.g., a particular U! screen or view, or timestamp, etc). Then, for each classification, generalized snapshot generator 1 14 may determine (i.e., resolve), for each target pixel of the generalized snapshot, a final target pixel (i.e., a color value).
  • classification e.g., a particular U! screen or view, or timestamp, etc.
  • Generalized snapshot generator 114 may determine each target pixel ⁇ e.g., for a particular pixel location of the generalized snapshot) by considering the set of source pixels from the partial snapshots. For a particular target pixel of the generalized snapshot, generalized snapshot generator 14 may determine a color value based on the color values of the relevant set of source pixels from the partial snapshots.
  • Client 120 may be any type of computing device that is capable of creating partial snapshots and transmitting them, e.g., over a network, to monitoring system 110.
  • Client 120 may be, for example, a smartphone, cell phone, tablet, ph ablet, laptop, desktop, server, application-specific computing device or any other type of computing device.
  • Client 120 may be described specifically herein, but it should be understood that clients 130, 140 and any other clients in computing environment 100 may be similar to client 120 as described herein.
  • Client 120 may include an application 122 that runs on the client 120.
  • Application 122 may include instructions (e.g., stored on a machine-readable storage medium of client 120) that, when executed (e.g., by a processor of client 120), implement the functionality of the application.
  • application 122 may include electronic circuitry (i.e., hardware) that implements the functionality of the application.
  • Application 122 may be any type of application, for example, a software program, web browser, web application, mobile application or the like.
  • Client 120 may include a partial snapshot generator 124.
  • Partial snapshot generator 124 may be included as part of application 122 or may be independent of (e.g., in communication with) application 122.
  • Partial snapshot generator 124 may be part of a larger application execution reporting too! that sends various pieces of execution information regarding applications (e.g., 122) to a monitoring tool (e.g., 112).
  • Partial snapshot generator 124 and/or the larger application execution reporting tool may be provided to users (e.g., creators of applications such as 122) in the form of an SDK (Software Development Kit) library that the users can embed inside their applications.
  • SDK Software Development Kit
  • the partial snapshot generator 124 and/or the larger application execution reporting tool may be running in the background, for example, to monitor events that happen with regard to the client 120, for example, user actions, timing events, network events, sensor data events (e.g., camera events such as photographs taken) and the like.
  • users of client 120 may be notified that their actions may be monitored and may be given the option to opt-in or opt-out of such monitoring.
  • Partial snapshot generator 124 may, at various times and/or in response to various events ⁇ e.g., user actions), create a partial snapshot.
  • the partial snapshot may capture a portion of a screen or view displayed by a user interface (Ul) component of application 122.
  • the partial snapshot may be generated in a way such that personal information of the user of the client is protected, as is explained in more detail below (e.g., with regard to pixel sampler 206 of FIG. 2).
  • Partial snapshot generator 124 may provide the partial snapshots to the monitoring system 110 over at least one network.
  • Other clients e.g., 130, 140, etc.
  • FIG. 2 is a block diagram of an example partial snapshot generator 200, which may be similar to partial snapshot generator 124 of F!G. 1.
  • Partial snapshot generator 200 may include an event detector 202, a snapshot classifier 204, a pixel sampler 206, a partial snapshot assembler 208 and a partial snapshot transmitter 210.
  • Each of these components and partial snapshot generator 200 overall may each include instructions ⁇ e.g., stored on a machine-readable storage medium of client 120 ⁇ that, when executed ⁇ e.g., by a processor of client 120), implement the functionality of the particular component.
  • each of these components and partial snapshot generator 200 overall may include electronic circuitry ⁇ i.e., hardware) that implements the functionality of the particular component.
  • Event defector 202 may detect events that cause partial snapshot generator 200 to create partial snapshots.
  • Partiai snapshot generator 200 may (e.g., depending on its configuration) create partial snapshots in response to various events.
  • partial snapshot generator may create snapshots when users interact with certain portions of an application user interface, !n this example, event detector 202 may defect such user interactions.
  • partial snapshot generator may create snapshots when particuiar timing events occur, e.g., clock events, video or audio playback events or other timing events. In these examples, event detector 202 may detect such timing events.
  • Snapshot classifier 204 may classify each partiai snapshot that is created or about to be created according to at least one classification scheme. For example, partiai snapshots may be classified according to particular Ul screens, screen views (e.g., particular buttons, windows, etc.), timestamps or time events, user actions, sensor data, other events or the like. Partial snapshots may be classified, for example, by the particular screen or view they capture and/or by the event that caused the partial snapshot to be created. Such classification information for a particular partial snapshot may be referred to as metadata, and may be sent to the monitoring system ⁇ e.g., 1 10) along with the partial snapshot. Snapshot classifier 204 may indicate the classification and/or metadata in various ways. For example, each classification may be associated with a particular unique identifier (ID).
  • ID unique identifier
  • a partia! snapshot may include a sampling of pixels dispersed over the image (e.g., screen or view) being captured, and different partia! snapshots may include different samp!ings of pixels even though they are capturing the same screen or view.
  • these different partiaf snapshots may still be classified according to the same screen or view, e.g., such that these snapshots can be used to create a generalized snapshot (e.g., by monitoring system 1 10).
  • all partial snapshots related to a particular button of an application e.g., a "Checkout” button
  • multiple partial snapshots may each be classified according to a consistent time stamping scheme.
  • a system or web-based dock may be used.
  • a timestamp from a timer that starts on the client at some time may be used.
  • partia! screen shots may be generated by partial snapshot generator 200 in conjunction with a video playing on the related client or in conjunction with video being captured on the client using a camera device, in each of these cases, when the video or recording starts, a timer may start and may create timestamps that can be used to categorize partiai snapshots.
  • Various other manners of categorizing partial snapshots are contemplated by this disclosure as well.
  • Pixel sampler 206 may, for a particular partiai snapshot, seiect which pixels of the total available Ul screen or view are to be included in the partiai snapshot, and may capture these pixels.
  • the captured pixels are a portion of the total pixels included in the screen or view.
  • pixel sampler 206 may capture k (e.g., some percentage or other portion of the total pixels) pixels.
  • Pixel sampier 206 may maintain a dispersion of the captured pixels over the total area of the screen or view and may prevent clustering of captured pixels within sub-areas of the total area.
  • Pixel sampler 206 may capture enough pixels over the total area of the screen or view such thai a generalized version of the screen or view can be reconstructed ⁇ e.g., by monitoring system 110) in conjunction with multiple other similarly categorized partial snapshots. Pixel sampler 206 may also prevent clustering of pixels in areas which may allow personal user information or confidential information in the screen or view to be deciphered by looking at the particular partial snapshot. A goal may be to include a small enough amount of data such that the included image data cannot be extrapolated. At the same time, including more data in each partial snapshot may allow for quicker creating of generalized snapshots (e.g., in the monitoring system).
  • Pixel sampler 206 may maintain a dispersion of the captured pixels and prevent clustering of captured pixels by using a "grid" sampling approach.
  • An example grid sampling approach may include dividing the Ul screen or view into a number of cells, e.g. , arranged in a grid orientation.
  • the approach may divide the Ul screen or view into a grid of n*m cells (i.e., a grid of n horizontal rectangles and m vertical rectangles). The number of horizontal and vertical rectangles and the size of each rectangle may vary depending on the amount of pixe! dispersion desired across the screen or view.
  • the approach may sample a number of pixels (e.g., k (n * m) pixels) from each ce!L
  • pixels may be selected from each ceil in various ways. For example, pixel sampler 206 may select a number of random pixels from each ceil. Alternatively, pixel sampler 206 may select a number of random pixels from one celi and then select the same pixel locations (e.g., within each celi) for the rest of the ceil in the grid. Various other ways of selecting pixeis from grids may be used.
  • Pixei sampler 206 may select pixels from the US screen or view in manners other than a grid approach. For example, pixei sampler 206 may sample pixels using a screen or view wide random approach. Specifically, pixei sampler 206 may randomly sample k pixels out of a screen or view with dimensions of W" * H pixels. As another example, pixei sampler 206 may use a scan line sampling approach. This approach includes selecting scan lines (e.g., horizontal and/or vertical pixel lines across the screen or view) from the screen or view. The vertical (or horizontal for vertical scan lines) position and spacing of the scan lines may be randomly selected or otherwise selected. The number of scan iines for a particular screen or view may be determined.
  • scan lines e.g., horizontal and/or vertical pixel lines across the screen or view
  • the vertical (or horizontal for vertical scan lines) position and spacing of the scan lines may be randomly selected or otherwise selected. The number of scan iines for a particular screen or view may be determined.
  • an example goaf of the pixel sampler 206 may be to achieve a good coverage of pixels across the screen or view while ensuring variation of pixei selections from other partial snapshots. Then, when a monitoring system (e.g., 1 10) receives this partial snapshot and other partial snapshots, the goat is to have a number of pixeis samples for each pixel location in the general snapshot image. The monitoring system may then determine each pixei of the general snapshot image by considering the samples for each pixel from the partial snapshots.
  • a monitoring system e.g. 1 10
  • a programmer, user or administrator of the partial snapshot generator could modify various parameters or setting to achieve a desired coverage of pixels samples and number of pixel samples for each pixei in the generalized snapshot. For example, if the rough number of users of the application (e.g., 122) were known and/or the rough number of users that work with a particular Ul screen or view, then appropriate parameters (e.g., how many horizontal and vertical ceils in the sampling grids, the number of pixeis selected from each grid, etc.) could be chosen.
  • Partial snapshot assembler 208 may create a partial snapshot from the sampiing of pixels determined by pixel sampler 206. More particularly, partial snapshot assembler 208 may assemble these pixels into an image file, digital file or some other type of binary data that includes a number of color pixels. Partial snapshot assembler 208 may, for example, use location information ⁇ e.g., X and Y coordinates) of the various sampled pixels to place the sampled pixeis in the right location in the partial snapshot, or to record the right location such that these pixels can be used in the right location when they are used to create a generalized snapshot.
  • location information ⁇ e.g., X and Y coordinates
  • the pixel sampling and assembling approaches described herein may consider the screen resolution of the U! screens displayed to users. For example, for different screen resolutions, pixel sampler 106 may select more or less pixels per ceil (for the grid approach) or different pixels per celt. As another example, partial snapshot assembler 208 may assemble differently classified partial snapshots for different screen resolutions, even if the pixels are sampled from the same type of U! screen or view, !n these examples, the classification or metadata information described above may account for screen resolution. Specifically, when clients send partial snapshots to the monitoring system, the accompanying metadata may include the screen resolution of the client device ⁇ e.g., 120), and/or of the application (e.g., 122).
  • Partiai snapshot transmitter 210 may send partial snapshots created b partial snapshot assembler 208 to at least one monitoring system (e.g., 11 ⁇ ). As described above, each partiai snapshot may only include a portion of the tola! pixels included in a full Ui screen or view. Partial snapshot transmitter 210 may send only these pixels (e.g., in a condensed or compressed format) for each partiai snapshot). This may result in various bandwidth benefits because full UI screens or views may not need to be sent over a network for each partial snapshot.
  • Partiai snapshot transmitter 210 may also, for each partiai snapshot, send reiated metadata to the monitoring system. Such metadata may reiate to at least one classification of the partial snapshot, as described in more detail above. Then, this metadata may be used by the monitoring system to group partiai snapshots that relate to the same classification (e.g., a particular screen, view, timestamp, etc.). Partial snapshot transmitter 2 0 may also send screen resolution information of the client to the monitoring system.
  • Partiai snapshot transmitter 210 may continuously or regularly send partiai snapshots related to at least one application.
  • the partial screen shots may change, and thus the monitoring system may update its generalized snapshots automaticaiiy and dynamically.
  • various other components of partial snapshot generator 200 may function continuously or regularly.
  • event detector may continue to detect events of a particular type, even though a similar partial snapshot may have been sent for that type of event in the past.
  • components 204, 206 and 208 may continuously or regularly perform their functions as described herein.
  • FIG. 3 is a flowchart of an example method 300 for partial snapshots for creating generalized snapshots.
  • Method 300 may be described below as being executed or performed by a computing device, for exampie, ciients 120, 130, 140, etc. of FIG. 1. Other suitable systems and/or computing devices may be used as weli.
  • Method 300 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of the computing device and executed by at ieast one processor of the computing device.
  • method 300 may be implemented in the form of electronic circuitry (e.g., hardware).
  • one or more steps of method 300 may be executed substantiaily concurrently or in a different order than shown in FIG. 3.
  • method 300 may include more or less steps than are shown in FIG. 3. in some embodiments, one or more of the steps of method 300 may, at certain times, be ongoing and/or may repeat.
  • Method 300 may start at step 302 and continue to step 304, where a computing device (e.g., 120) may detect an event that should cause a partial snapshot to be generated. More details of this event defection are provided above with regard to event detector 202.
  • the computing device may classify the snapshot, for example, as relating to a particular U! screen, view, timestamp, etc. More details of this ciassification are provided above with regard to snapshot classifier 204.
  • the computing device may capture a portion of the pixels (i.e., sampling) from the entire U! screen or view. More details of this sampling are provided above with regard to pixe! sampler 206.
  • the computing device may assemble the captured pixels into a partial snapshot. More details of this assembiy are provided above with regard to partial snapshot assembler 208.
  • the computing device may transmit the partial snapshot to a monitoring system (e.g., 1 10 of FIG. 1 ). More details of this transmission are provided above with regard to partial snapshot transmitter 210.
  • Method 300 may eventually continue to step 314, where method 300 may stop.
  • FIG. 4 is a flowchart of an example method 400 for partial snapshots for creating generalized snapshots.
  • Method 400 may be described below as being executed or performed by a computing device, for example, computing device 500 of FIG. 5, or computing device 600 of FIG. 6. Other suitable systems and/or computing devices may be used as we!!.
  • Method 400 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of the computing device and executed by at least one processor of the computing device.
  • method 400 may be implemented in the form of electron ic circuitry (e.g., hardware), in alternate embodiments of the present disclosure, one or more steps of method 400 may be executed substantially concurrently or in a different order than shown in FiG. 4. in alternate embodiments of the present disclosure, method 400 may include more or less steps than are shown in F!G. 4.
  • one or more of the steps of method 400 may, at certain times, be ongoing and/or may repeat.
  • Method 400 may start at step 402 and continue to step 404, where a computing device ⁇ e.g., 500 or 600 ⁇ may, in response to an event, access a user interface screen or view associated with an application.
  • the screen or view includes multiple pixels arranged over an area.
  • the computing device may sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area.
  • the computing device may transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots. Method 400 may eventually continue to ste 410, where method 400 may stop.
  • FiG. 5 is a block diagram of an example computing device 500 for partial snapshots for creating generalized snapshots.
  • Computing device 500 may be any computing device capable of creating partial snapshots and transmitting them, e.g., over a network, to monitoring system ⁇ e.g., 110 of FIG. 1).
  • Computing device 500 may be similar to clients 120, 130, 140, etc. of FiG. 1.
  • computing device 500 includes an event detector engine 510, a pixel sampler engine 520 and a snapshot transmission engine 530.
  • Event detector engine 510 may detect an event, for example, in a manner similar to event detector 202 of FiG. 2.
  • Event detector engine 510 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of computing device 500 and executed by at least one processor of computing device 500. Alternatively or in addition, event detector engine 510 may be implemented in the form of one or more hardware devices including eiectronic circuitry for implementing the functionality of event detector engine 510.
  • Pixel sampler engine 520 may access, in response to the event, a user interface screen or view associated with an application.
  • the screen or view includes multiple pixeis arranged over an area.
  • Pixel sampler engine 520 may sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area.
  • Pixel sampler engine 520 may function in a manner similar to pixel sampler 206 and partial snapshot assembler 208 of FIG. 2 and perhaps other components of partial snapshot generator 200.
  • Pixel sampler engine 520 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of computing device 500 and executed by at least one processor of computing device 500. Alternativeiy or in addition, pixel sampler engine 520 may be implemented in the form of one or more hardware devices including electronic circuitry for implementing the functionality of pixel sampler engine 520.
  • Snapshot transmission engine 530 may transmit the captured portion of the total pixels as a partial snapshot to a system (e.g., monitoring system 1 0 of FIG. 1 ) to create a generalized snapshot from the partial snapshot and other partial snapshots ⁇ e.g., from other clients such as 130, 140, etc.). Snapshot transmission engine 530 may function in a manner similar to partial snapshot transmitter 210 of FIG. 2. Snapshot transmission engine 530 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of computing device 500 and executed by at least one processor of computing device 500. Alternatively or in addition, snapshot transmission engine 530 may be implemented in the form of one or more hardware devices including electronic circuitry for implementing the functionality of snapshot transmission engine 530.
  • FIG. 6 is a block diagram of an example computing device 600 for partial snapshots for creating generalized snapshots.
  • Computing device 600 may be any computing device capable of creating partial snapshots and transmitting them, e.g., over a network, to monitoring system ⁇ e.g., 110 of FIG. 1).
  • Computing device 600 may be similar to clients 120, 130, 140, etc. of FIG. 1.
  • computing device 600 includes a processor 610 and a machine-readable storage medium 620.
  • the following descriptions refer to a single processor and a single machine-readable storage medium, the descriptions may also apply to a computing device with multiple processors and multiple machine-readable storage mediums.
  • the instructions may be distributed ⁇ e.g., stored) across multiple machine-readable storage mediums and the instructions may be distributed (e.g., executed by) across multiple processors.
  • Processor 610 may be one or more central processing units (CPUs), microprocessors, and/or other hardware devices suitable for retrieval and execution of instructions stored in machine-readable storage medium 620.
  • processor 610 may fetch, decode, and execute instructions 622, 624, 626 to create partial snapshots for creating generalized snapshots.
  • processor 610 may include one or more electronic circuits comprising a number of electronic components for performing the functionafity of one or more of the instructions in machine-readable storage medium 620.
  • Machine-readable storage medium 620 may be any electronic, magnetic, optica!, or other physical storage device that stores executable instructions.
  • machine-readable storage medium 620 may be, for example. Random Access Memory (RAM), an EieciricaHy-ErasabSe Programmable Read-On !y Memory (EEPROEvl), a storage drive, an optical disc, and the like.
  • Machine-readabie storage medium 620 may be disposed within computing device 600, as shown in FIG. 6. In this situation, the executable instructions may be "installed" on the computing device 600. Alternatively, machine-readabie storage medium 620 may be a portable, external or remote storage medium, for example, that allows computing device 600 to download the instructions from the portabSe externai/remote storage medium, !n this situation, the executable instructions may be part of an "installation package". As described herein, machine-readable storage medium 620 may be encoded with executable instructions for partial snapshots for creating generalized snapshots.
  • screen or view accessing instructions 622 when executed by a processor (e.g., 610), may cause computing device 600 to access, in response to an event, a user interface screen or view associated with an application.
  • the screen or view includes multiple pixels arranged over an area.
  • Pixel sampling instructions 624 when executed by a processor (e.g., 610), may cause computing device 600 to sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area.
  • Partial snapshot transmission instructions 626 when executed by a processor (e.g., 610), may cause computing device 600 to transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partiai snapshot and other partial snapshots.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

Example embodiments relate to partial snapshots for creating generalized snapshots. An example method may include, in response to an event, accessing a user interface screen or view associated with an application. The screen or view includes multiple pixels arranged over an area. The method may include sampling the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area. The method may include transmitting the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots.

Description

PARTIAL SNAPSHOTS FOR CREATING GENERALIZED SNAPSHOTS
BACKGROUND
[0001] Applications (e.g., computer software) often include a user interface (Ut) component thai displays screens to users of the application. A snapshot of an application refers to a screen capture of pari or ail of the screen presented to the user. Snapshots may be captured by various tools (e.g., testing tools for mobile or web applications) at various times and/or in response to various events (e.g., user actions). A snapshot may be an image file, digital file or some other type of binary data that includes a number of color pixels.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] The following detailed description references the drawings, wherein;
[0003] FIG. 1 is a block diagram of an example computing environment in which partial snapshots for creating generalized snapshots may be useful;
[0004] FIG. 2 is a block diagram of an example partial snapshot generator for partial snapshots for creating generalized snapshots;
[0005] FIG. 3 is a flowchart of an example method for partial snapshots for creating generalized snapshots;
[0006] FIG. 4 is a flowchart of an example method for partial snapshots for creating generalized snapshots;
[0007] FIG. 5 is a block diagram of an example computing device for partial snapshots for creating generalized snapshots; and
[0008] FIG. 6 is a biock diagram of an example computing device for partial snapshots for creating generalized snapshots. DETAILED DESCRIPTION
[0009] Snapshots may be useful for testing an application. For example, a testing tool may capture or record a certain flow of an application {e.g., a series of user actions such as button clicks, etc.) such that the flow can be analyzed, it may be useful for various steps of such a flow to include at least one snapshot, for example, to add application context to a textual description of the step or action. Adding appiication context via snapshots may allow a user (e.g., a developer or debugger) to better and more quickly understand a performance incident or error. However, when snapshots originate from applications being used by reai users, it may be desirable to maintain the users' privacy.
[0010] Some pre-production testing tools {e.g., toots that test an application before it is reieased to reai users) may capture screen shots and attach them to a report of the text execution. Because, in these situations, the appiication is not being used by reai users, the appiication does not contain any persona! information that needs to be protected. Such applications may use fake or test information instead of real user information. As such, for each test step or simulated user action, a single screen shot may be attached. These tools do not generate partial snapshots for creating generalized snapshots.
[001 1] Some production testing tools, or monitoring toois, may also capture screen shots and attach them to a report of the text execution. For example, synthetic monitoring toots may periodically check the status an appiication {e.g., whether the application is online, available, running, etc.). Additionally, some synthetic monitoring tools may check general functionalities of an application that do not require a user login. Additionally, some synthetic monitoring tools may check functionalities of an application that do require a user login by using a test account instead of a real user account. In ait of these situations, no real user information is needed for the testing. As such, for each test step or simulated user action, a single screen shot may be attached. These toois do not generate partial snapshots for creating generalized snapshots.
[0012] Some other production testing tools, such as real user monitoring tools, may capture screen shots that are associated with reai user actions as real users use the application, in these situations, the snapshots may contain sensitive or private information of the users. For example, a snapshot couid include a user's mailing address, account number, credit card information or the like. This may pose a privacy concern for many users, as the users may not trust the developers or debuggers that are using the snapshots. Thus, it may be desirable for real user monitoring tools to use snapshot information while maintaining users' privacy. It may be desirable to create a generalized or scrubbed snapshot that testers or debuggers can use without seeing any sensitive reai user information.
[0013] The present disciosure describes partial snapshots for creating generalized snapshots. In some examples, a client {e.g., a computing device) may, in response to an event, access a user interface screen or view associated with an application. The screen or view may include multiple pixels arranged over an area. The client may sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. This sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area. The client may then transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots. The resulting generalized snapshot may be a "generic" or "most representative" {but scrubbed) snapshot based on the screen of multiple real users. The generalized snapshot may preserve the users' privacy by not sending confidential/persona! information over a network to the monitoring system.
[0014] FIG. 1 is a block diagram of an example computing environment 100 in which partial snapshots for creating generalized snapshots may be useful. Computing environment 100 may include a monitoring system 110 and a number of clients (e.g., 120, 130, 140, etc.). Each client may provide, e.g., at a particular time or in response to a particular event, a partial snapshot to monitoring system 110. Each partial snapshot may foe an image fiie, digital fiie or some other type of binary data that includes a number of color pixels. The clients may provide the partial snapshots to the monitoring system 1 10 over at ieast one network. Such a network may be any wired or wireless network, and may include any number of hubs, routers, switches, cell towers or the like. Such a network may be, for example, part of a cellular network, part of the internet, part of an intranet and/or other type of network. Monitoring system 1 10 may receive, e.g., at particular times or in response to particular events, the multiple partial snapshots from the various clients (e.g., 120, 130, 140, etc.) and may create a generalized snapshot based on these multiple partial snapshots.
[0015] Monitoring system 1 10 may include at Ieast one computing device that is capable of receiving multiple partial snapshots from the various clients (e.g., 120, 30, 140, etc.) and creating a generalized snapshot. The term "system" may be used to refer to a single computing device or multiple computing devices that communicate with each other {e.g., via a network) and operate together to provide a service. Monitoring system 110 may include a monitoring tool 112 that runs on monitoring system 1 10.
[0016] Monitoring too! 1 12 may be any type of application testing toot (e.g., a real user monitoring too!) that receives execution information (e.g., execution reports, errors, performance issues, etc.) from various applications {e.g., 122) running on various clients (e.g., 120, 130, 140, etc.). Monitoring tool 1 12 may also be capable of receiving snapshots and/or partial snapshots that may be attached to or included as part of the received execution information. Monitoring tool 1 12 may include instructions {e.g., stored on a machine-readable storage medium of system 110) that, when executed (e.g., by a processor of system 1 10), implement the functionality of the monitoring tool 1 12. Alternatively or in addition, monitoring too! 112 may include electronic circuitry {i.e., hardware) that implements the functionality of the monitoring tool 1 12. Monitoring system 110 may include a generalized snapshot generator 114. Generalized snapshot generator 114 may be included as part of monitoring tool 112 or may be independent of (e.g., in communication with) monitoring tool 1 12.
[0017] Generalized snapshot generator 1 14 may receive multiple partial snapshots from the various clients (e.g., 120, 130, 140, etc.). in particular, generalized snapshot generator 114 may receive partial snapshots from partial snapshot generators (e.g., 124) running on the various clients. Generalized snapshot generator 114 may create a generalized snapshot based on these multiple partial snapshots. More specifically, generalized snapshot generator 114 may group received partial snapshots according to their classification (e.g., a particular U! screen or view, or timestamp, etc). Then, for each classification, generalized snapshot generator 1 14 may determine (i.e., resolve), for each target pixel of the generalized snapshot, a final target pixel (i.e., a color value). Generalized snapshot generator 114 may determine each target pixel {e.g., for a particular pixel location of the generalized snapshot) by considering the set of source pixels from the partial snapshots. For a particular target pixel of the generalized snapshot, generalized snapshot generator 14 may determine a color value based on the color values of the relevant set of source pixels from the partial snapshots.
[0018] Client 120 may be any type of computing device that is capable of creating partial snapshots and transmitting them, e.g., over a network, to monitoring system 110. Client 120 may be, for example, a smartphone, cell phone, tablet, ph ablet, laptop, desktop, server, application-specific computing device or any other type of computing device. Client 120 may be described specifically herein, but it should be understood that clients 130, 140 and any other clients in computing environment 100 may be similar to client 120 as described herein. Client 120 may include an application 122 that runs on the client 120. Application 122 may include instructions (e.g., stored on a machine-readable storage medium of client 120) that, when executed (e.g., by a processor of client 120), implement the functionality of the application. Alternatively or in addition, application 122 may include electronic circuitry (i.e., hardware) that implements the functionality of the application. Application 122 may be any type of application, for example, a software program, web browser, web application, mobile application or the like.
[0019] Client 120 may include a partial snapshot generator 124. Partial snapshot generator 124 may be included as part of application 122 or may be independent of (e.g., in communication with) application 122. Partial snapshot generator 124 may be part of a larger application execution reporting too! that sends various pieces of execution information regarding applications (e.g., 122) to a monitoring tool (e.g., 112). Partial snapshot generator 124 and/or the larger application execution reporting tool may be provided to users (e.g., creators of applications such as 122) in the form of an SDK (Software Development Kit) library that the users can embed inside their applications. Then when users of the client {e.g., 120) run the application (e.g., 122), the partial snapshot generator 124 and/or the larger application execution reporting tool may be running in the background, for example, to monitor events that happen with regard to the client 120, for example, user actions, timing events, network events, sensor data events (e.g., camera events such as photographs taken) and the like. In some situations, users of client 120 may be notified that their actions may be monitored and may be given the option to opt-in or opt-out of such monitoring.
[0020] Partial snapshot generator 124 may, at various times and/or in response to various events {e.g., user actions), create a partial snapshot. The partial snapshot may capture a portion of a screen or view displayed by a user interface (Ul) component of application 122. The partial snapshot may be generated in a way such that personal information of the user of the client is protected, as is explained in more detail below (e.g., with regard to pixel sampler 206 of FIG. 2). Partial snapshot generator 124 may provide the partial snapshots to the monitoring system 110 over at least one network. Other clients (e.g., 130, 140, etc.) may also provide partial snapshots to the monitoring system 110 such that the monitoring system 110 may use multiple partial snapshots to create a generalized snapshot. More details regarding the partial snapshot generator may be provided below with regard to the description of partial snapshot generator 200 of FIG. 2.
[0021] FIG. 2 is a block diagram of an example partial snapshot generator 200, which may be similar to partial snapshot generator 124 of F!G. 1. Partial snapshot generator 200 may include an event detector 202, a snapshot classifier 204, a pixel sampler 206, a partial snapshot assembler 208 and a partial snapshot transmitter 210. Each of these components and partial snapshot generator 200 overall may each include instructions {e.g., stored on a machine-readable storage medium of client 120} that, when executed {e.g., by a processor of client 120), implement the functionality of the particular component. Alternatively or in addition, each of these components and partial snapshot generator 200 overall may include electronic circuitry {i.e., hardware) that implements the functionality of the particular component.
[0022] Event defector 202 may detect events that cause partial snapshot generator 200 to create partial snapshots. Partiai snapshot generator 200 may (e.g., depending on its configuration) create partial snapshots in response to various events. For example, partial snapshot generator may create snapshots when users interact with certain portions of an application user interface, !n this example, event detector 202 may defect such user interactions. As another example, partial snapshot generator may create snapshots when particuiar timing events occur, e.g., clock events, video or audio playback events or other timing events. In these examples, event detector 202 may detect such timing events.
[0023] Several of the descriptions herein may describe various routines that occur with respect to generating a single partial snapshot. However, it should be understood that the foiiowing descriptions may apply to generating muitiple other partial snapshots, for example, in response to various other events.
[0024] Snapshot classifier 204 may classify each partiai snapshot that is created or about to be created according to at least one classification scheme. For example, partiai snapshots may be classified according to particular Ul screens, screen views (e.g., particular buttons, windows, etc.), timestamps or time events, user actions, sensor data, other events or the like. Partial snapshots may be classified, for example, by the particular screen or view they capture and/or by the event that caused the partial snapshot to be created. Such classification information for a particular partial snapshot may be referred to as metadata, and may be sent to the monitoring system {e.g., 1 10) along with the partial snapshot. Snapshot classifier 204 may indicate the classification and/or metadata in various ways. For example, each classification may be associated with a particular unique identifier (ID).
[0025] in the example of partia! snapshots being classified by the particular screen or view they capture, multiple partial snapshots (e.g. , from multiple clients) may each be classified according to the same particular screen or view even though each of these partia! snapshots may sample different pixels of the screen or view. As described below {e.g., with regard to pixel sampler 206), a partia! snapshot may include a sampling of pixels dispersed over the image (e.g., screen or view) being captured, and different partia! snapshots may include different samp!ings of pixels even though they are capturing the same screen or view. In this case, these different partiaf snapshots may still be classified according to the same screen or view, e.g., such that these snapshots can be used to create a generalized snapshot (e.g., by monitoring system 1 10). As one particular example, all partial snapshots related to a particular button of an application (e.g., a "Checkout" button) may be classified the same, even though each of these partia! snapshots may sample different pixels of the button image.
[0026] In the example of partial snapshots being classified by timestamps or time events, multiple partial snapshots (e.g., from multiple clients) may each be classified according to a consistent time stamping scheme. For example, a system or web-based dock may be used. Alternately, a timestamp from a timer that starts on the client at some time may be used. As a specific example, partia! screen shots may be generated by partial snapshot generator 200 in conjunction with a video playing on the related client or in conjunction with video being captured on the client using a camera device, in each of these cases, when the video or recording starts, a timer may start and may create timestamps that can be used to categorize partiai snapshots. Various other manners of categorizing partial snapshots are contemplated by this disclosure as well.
[0027] Pixel sampler 206 may, for a particular partiai snapshot, seiect which pixels of the total available Ul screen or view are to be included in the partiai snapshot, and may capture these pixels. The captured pixels are a portion of the total pixels included in the screen or view. For example, for a screen of size W*H, pixel sampler 206 may capture k (e.g., some percentage or other portion of the total pixels) pixels. Pixel sampier 206 may maintain a dispersion of the captured pixels over the total area of the screen or view and may prevent clustering of captured pixels within sub-areas of the total area. Pixel sampler 206 may capture enough pixels over the total area of the screen or view such thai a generalized version of the screen or view can be reconstructed {e.g., by monitoring system 110) in conjunction with multiple other similarly categorized partial snapshots. Pixel sampler 206 may also prevent clustering of pixels in areas which may allow personal user information or confidential information in the screen or view to be deciphered by looking at the particular partial snapshot. A goal may be to include a small enough amount of data such that the included image data cannot be extrapolated. At the same time, including more data in each partial snapshot may allow for quicker creating of generalized snapshots (e.g., in the monitoring system).
[0028] Pixel sampler 206 may maintain a dispersion of the captured pixels and prevent clustering of captured pixels by using a "grid" sampling approach. An example grid sampling approach may include dividing the Ul screen or view into a number of cells, e.g. , arranged in a grid orientation. As a specific example, the approach may divide the Ul screen or view into a grid of n*m cells (i.e., a grid of n horizontal rectangles and m vertical rectangles). The number of horizontal and vertical rectangles and the size of each rectangle may vary depending on the amount of pixe! dispersion desired across the screen or view. Then, the approach may sample a number of pixels (e.g., k (n*m) pixels) from each ce!L
[0029] For the grid sampling approach, pixels may be selected from each ceil in various ways. For example, pixel sampler 206 may select a number of random pixels from each ceil. Alternatively, pixel sampler 206 may select a number of random pixels from one celi and then select the same pixel locations (e.g., within each celi) for the rest of the ceil in the grid. Various other ways of selecting pixeis from grids may be used.
[0030] Pixei sampler 206 may select pixels from the US screen or view in manners other than a grid approach. For example, pixei sampler 206 may sample pixels using a screen or view wide random approach. Specifically, pixei sampler 206 may randomly sample k pixels out of a screen or view with dimensions of W"*H pixels. As another example, pixei sampler 206 may use a scan line sampling approach. This approach includes selecting scan lines (e.g., horizontal and/or vertical pixel lines across the screen or view) from the screen or view. The vertical (or horizontal for vertical scan lines) position and spacing of the scan lines may be randomly selected or otherwise selected. The number of scan iines for a particular screen or view may be determined.
[0031] Regardless of the way pixel sampler 206 selects pixels from the screen or view, an example goaf of the pixel sampler 206 may be to achieve a good coverage of pixels across the screen or view while ensuring variation of pixei selections from other partial snapshots. Then, when a monitoring system (e.g., 1 10) receives this partial snapshot and other partial snapshots, the goat is to have a number of pixeis samples for each pixel location in the general snapshot image. The monitoring system may then determine each pixei of the general snapshot image by considering the samples for each pixel from the partial snapshots. A programmer, user or administrator of the partial snapshot generator could modify various parameters or setting to achieve a desired coverage of pixels samples and number of pixel samples for each pixei in the generalized snapshot. For example, if the rough number of users of the application (e.g., 122) were known and/or the rough number of users that work with a particular Ul screen or view, then appropriate parameters (e.g., how many horizontal and vertical ceils in the sampling grids, the number of pixeis selected from each grid, etc.) could be chosen.
[0032] Partial snapshot assembler 208 may create a partial snapshot from the sampiing of pixels determined by pixel sampler 206. More particularly, partial snapshot assembler 208 may assemble these pixels into an image file, digital file or some other type of binary data that includes a number of color pixels. Partial snapshot assembler 208 may, for example, use location information {e.g., X and Y coordinates) of the various sampled pixels to place the sampled pixeis in the right location in the partial snapshot, or to record the right location such that these pixels can be used in the right location when they are used to create a generalized snapshot.
[0033] In some examples, the pixel sampling and assembling approaches described herein may consider the screen resolution of the U! screens displayed to users. For example, for different screen resolutions, pixel sampler 106 may select more or less pixels per ceil (for the grid approach) or different pixels per celt. As another example, partial snapshot assembler 208 may assemble differently classified partial snapshots for different screen resolutions, even if the pixels are sampled from the same type of U! screen or view, !n these examples, the classification or metadata information described above may account for screen resolution. Specifically, when clients send partial snapshots to the monitoring system, the accompanying metadata may include the screen resolution of the client device {e.g., 120), and/or of the application (e.g., 122). In other examples, the pixel sampiing and assembling approaches described herein may accommodate different screen resolutions and may not classify partial snapshots as being different just because of different screen resolutions. [0034] Partiai snapshot transmitter 210 may send partial snapshots created b partial snapshot assembler 208 to at least one monitoring system (e.g., 11Ό). As described above, each partiai snapshot may only include a portion of the tola! pixels included in a full Ui screen or view. Partial snapshot transmitter 210 may send only these pixels (e.g., in a condensed or compressed format) for each partiai snapshot). This may result in various bandwidth benefits because full UI screens or views may not need to be sent over a network for each partial snapshot.
[0035] Partiai snapshot transmitter 210 may also, for each partiai snapshot, send reiated metadata to the monitoring system. Such metadata may reiate to at least one classification of the partial snapshot, as described in more detail above. Then, this metadata may be used by the monitoring system to group partiai snapshots that relate to the same classification (e.g., a particular screen, view, timestamp, etc.). Partial snapshot transmitter 2 0 may also send screen resolution information of the client to the monitoring system.
[0036] Partiai snapshot transmitter 210 may continuously or regularly send partiai snapshots related to at least one application. Thus, for exampie, when new versions of an application are reieased, the partial screen shots may change, and thus the monitoring system may update its generalized snapshots automaticaiiy and dynamically. To faciiitate partiai snapshot transmitter 210 to continuously or regularly send partial snapshots, various other components of partial snapshot generator 200 may function continuously or regularly. For example, event detector may continue to detect events of a particular type, even though a similar partial snapshot may have been sent for that type of event in the past. As another example, components 204, 206 and 208 may continuously or regularly perform their functions as described herein.
[0037] FIG. 3 is a flowchart of an example method 300 for partial snapshots for creating generalized snapshots. Method 300 may be described below as being executed or performed by a computing device, for exampie, ciients 120, 130, 140, etc. of FIG. 1. Other suitable systems and/or computing devices may be used as weli. Method 300 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of the computing device and executed by at ieast one processor of the computing device. Alternatively or in addition, method 300 may be implemented in the form of electronic circuitry (e.g., hardware). In alternate embodiments of the present disclosure, one or more steps of method 300 may be executed substantiaily concurrently or in a different order than shown in FIG. 3. In alternate embodiments of the present disclosure, method 300 may include more or less steps than are shown in FIG. 3. in some embodiments, one or more of the steps of method 300 may, at certain times, be ongoing and/or may repeat.
[0038] Method 300 may start at step 302 and continue to step 304, where a computing device (e.g., 120) may detect an event that should cause a partial snapshot to be generated. More details of this event defection are provided above with regard to event detector 202. At step 306, the computing device may classify the snapshot, for example, as relating to a particular U! screen, view, timestamp, etc. More details of this ciassification are provided above with regard to snapshot classifier 204. At step 308, the computing device may capture a portion of the pixels (i.e., sampling) from the entire U! screen or view. More details of this sampling are provided above with regard to pixe! sampler 206. At step 310, the computing device may assemble the captured pixels into a partial snapshot. More details of this assembiy are provided above with regard to partial snapshot assembler 208. At step 312, the computing device may transmit the partial snapshot to a monitoring system (e.g., 1 10 of FIG. 1 ). More details of this transmission are provided above with regard to partial snapshot transmitter 210. Method 300 may eventually continue to step 314, where method 300 may stop.
[0039] FIG. 4 is a flowchart of an example method 400 for partial snapshots for creating generalized snapshots. Method 400 may be described below as being executed or performed by a computing device, for example, computing device 500 of FIG. 5, or computing device 600 of FIG. 6. Other suitable systems and/or computing devices may be used as we!!. Method 400 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of the computing device and executed by at least one processor of the computing device. Alternatively or in addition, method 400 may be implemented in the form of electron ic circuitry (e.g., hardware), in alternate embodiments of the present disclosure, one or more steps of method 400 may be executed substantially concurrently or in a different order than shown in FiG. 4. in alternate embodiments of the present disclosure, method 400 may include more or less steps than are shown in F!G. 4. In some embodiments, one or more of the steps of method 400 may, at certain times, be ongoing and/or may repeat.
[0040] Method 400 may start at step 402 and continue to step 404, where a computing device {e.g., 500 or 600} may, in response to an event, access a user interface screen or view associated with an application. The screen or view includes multiple pixels arranged over an area. At step 406, the computing device may sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area. At step 408, the computing device may transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots. Method 400 may eventually continue to ste 410, where method 400 may stop.
[0041] FiG. 5 is a block diagram of an example computing device 500 for partial snapshots for creating generalized snapshots. Computing device 500 may be any computing device capable of creating partial snapshots and transmitting them, e.g., over a network, to monitoring system {e.g., 110 of FIG. 1). Computing device 500 may be similar to clients 120, 130, 140, etc. of FiG. 1. In the embodiment of FIG. 5, computing device 500 includes an event detector engine 510, a pixel sampler engine 520 and a snapshot transmission engine 530. Event detector engine 510 may detect an event, for example, in a manner similar to event detector 202 of FiG. 2. Event detector engine 510 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of computing device 500 and executed by at least one processor of computing device 500. Alternatively or in addition, event detector engine 510 may be implemented in the form of one or more hardware devices including eiectronic circuitry for implementing the functionality of event detector engine 510.
[0042] Pixel sampler engine 520 may access, in response to the event, a user interface screen or view associated with an application. The screen or view includes multiple pixeis arranged over an area. Pixel sampler engine 520 may sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area. Pixel sampler engine 520 may function in a manner similar to pixel sampler 206 and partial snapshot assembler 208 of FIG. 2 and perhaps other components of partial snapshot generator 200. Pixel sampler engine 520 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of computing device 500 and executed by at feast one processor of computing device 500. Alternativeiy or in addition, pixel sampler engine 520 may be implemented in the form of one or more hardware devices including electronic circuitry for implementing the functionality of pixel sampler engine 520.
[0043] Snapshot transmission engine 530 may transmit the captured portion of the total pixels as a partial snapshot to a system (e.g., monitoring system 1 0 of FIG. 1 ) to create a generalized snapshot from the partial snapshot and other partial snapshots {e.g., from other clients such as 130, 140, etc.). Snapshot transmission engine 530 may function in a manner similar to partial snapshot transmitter 210 of FIG. 2. Snapshot transmission engine 530 may be implemented in the form of executable instructions stored on at least one machine-readable storage medium of computing device 500 and executed by at least one processor of computing device 500. Alternatively or in addition, snapshot transmission engine 530 may be implemented in the form of one or more hardware devices including electronic circuitry for implementing the functionality of snapshot transmission engine 530.
[0001] FIG. 6 is a block diagram of an example computing device 600 for partial snapshots for creating generalized snapshots. Computing device 600 may be any computing device capable of creating partial snapshots and transmitting them, e.g., over a network, to monitoring system {e.g., 110 of FIG. 1). Computing device 600 may be similar to clients 120, 130, 140, etc. of FIG. 1. In the embodiment of FIG. 6, computing device 600 includes a processor 610 and a machine-readable storage medium 620. Although the following descriptions refer to a single processor and a single machine-readable storage medium, the descriptions may also apply to a computing device with multiple processors and multiple machine-readable storage mediums. In such examples, the instructions may be distributed {e.g., stored) across multiple machine-readable storage mediums and the instructions may be distributed (e.g., executed by) across multiple processors.
[0002] Processor 610 may be one or more central processing units (CPUs), microprocessors, and/or other hardware devices suitable for retrieval and execution of instructions stored in machine-readable storage medium 620. In the particular embodiment shown in FIG. 6, processor 610 may fetch, decode, and execute instructions 622, 624, 626 to create partial snapshots for creating generalized snapshots. As an alternative or in addition to retrieving and executing instructions, processor 610 may include one or more electronic circuits comprising a number of electronic components for performing the functionafity of one or more of the instructions in machine-readable storage medium 620. With respect to the executable instruction representations (e.g., boxes) described and shown herein, it should be understood that part or all of the executable instructions and/or electronic circuits included within one box may, in alternate embodiments, be included in a different box shown in the figures or in a different box not shown. [0003] Machine-readable storage medium 620 may be any electronic, magnetic, optica!, or other physical storage device that stores executable instructions. Thus, machine-readable storage medium 620 may be, for example. Random Access Memory (RAM), an EieciricaHy-ErasabSe Programmable Read-On !y Memory (EEPROEvl), a storage drive, an optical disc, and the like. Machine-readabie storage medium 620 may be disposed within computing device 600, as shown in FIG. 6. In this situation, the executable instructions may be "installed" on the computing device 600. Alternatively, machine-readabie storage medium 620 may be a portable, external or remote storage medium, for example, that allows computing device 600 to download the instructions from the portabSe externai/remote storage medium, !n this situation, the executable instructions may be part of an "installation package". As described herein, machine-readable storage medium 620 may be encoded with executable instructions for partial snapshots for creating generalized snapshots.
[0004] Referring to FIG. 6, screen or view accessing instructions 622, when executed by a processor (e.g., 610), may cause computing device 600 to access, in response to an event, a user interface screen or view associated with an application. The screen or view includes multiple pixels arranged over an area. Pixel sampling instructions 624, when executed by a processor (e.g., 610), may cause computing device 600 to sample the multiple pixels by capturing a portion of the total pixels included in the screen or view. The sampling may include maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area. Partial snapshot transmission instructions 626, when executed by a processor (e.g., 610), may cause computing device 600 to transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partiai snapshot and other partial snapshots.

Claims

1. A method for creating a partial snapshot, the method comprising:
in response to an event, accessing a user interface screen or view associated with an application, wherein the screen or view includes multiple pixels arranged over an area;
sampling the multiple pixels by capturing a portion of the total pixels included In the screen or view, wherein the sampling includes maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area; and
transmitting the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots.
2. The method of claim 1 , wherein the sampling includes dividing the screen or view into multiple cells, and wherein the captured portion of the multiple pixels includes an equal number of pixels from each of the cells.
3. The method of claim 2, wherein the multiple ceils are arranged in a grid arrangement over the area of the screen or view.
4. The method of claim 2, wherein, for each cell, the captured pixels are chosen at random locations within the cell.
5. The method of claim 2, wherein, for each ceil, the captured pixels are chosen at locations within the cell that correspond to relative locations of captured pixels in the other ceils.
8. The method of ciaim 1. wherein the sampling inciudes selecting scan lines across the screen or view, wherein each scan line is a horizontal or vertical line of pixels.
7. The method of ciaim 1 , further comprising assigning a classification to the partial screenshot according to one or more of the fo!iowing:
a particular user interface screen:
a particular user interface screen view;
a particular dock time or timestamp; and
a particular user action related to an application user interface.
8. The method of claim 7, wherein the transmission includes transmitting the classification to the system.
9. A computing device for creating a partiaf snapshot, the computing device comprising:
an event detector engine to detect an event;
a pixei sampler engine to:
access, in response to the event, a user interface screen or view associated with an application, wherein the screen or view includes multiple pixels arranged over an area, and
sample the multiple pixels by capturing a portion of the total pixels included in the screen or view, wherein the sampling inciudes maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area; and a snapshot transmission engine to transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots.
10. The computing device of claim 9, the event is one or more of the following: a particular user action related to the user interface of the application; and a particular clock time or timestamp.
11. The computing device of claim 9, wherein to sample, the pixel sampler divides the screen or view into multiple cells, and wherein the captured portion of the multiple pixels includes an equal number of pixels from each of the cells.
12. The computing device of claim 11 , wherein the multiple cells are arranged in a grid arrangement over the area of the screen or view.
13. A machine-readable storage medium encoded with instructions for creating a partial snapshot, the instructions executable by a processor of a computing device to cause the computing device to:
access, in response to an event, a user interface screen or view associated with an application, wherein the screen or view includes multiple pixels arranged over an area;
sample the multiple pixels by capturing a portion of the total pixels included in the screen or view, wherein the sampling includes maintaining a dispersion of the captured pixels over the area of the screen or view and preventing clustering of captured pixels within sub-areas of the area; and
transmit the captured portion as a partial snapshot to a system to create a generalized snapshot from the partial snapshot and other partial snapshots.
14. The machine-readable storage medium of claim 13, wherein the instructions ar further executable by a processor of the computing device to cause the CDmputing device to assign a classification to the partial screenshot according to one or more of the foi lowing:
a particular user interface screen,
a particular user interface screen view
a particular clock time or timestamp. and
a particular user action related to an application user interface;
wherein the transmission includes transmitting the classification to the system.
15. The machine-readable storage medium of claim 13, wherein the sampling includes selecting scan lines across the screen or view, wherein each scan Sine is a horizontal or vertical line of pixels, and the spacing between the scan lines is selected randomly.
PCT/US2014/033597 2014-04-10 2014-04-10 Partial snapshots for creating generalized snapshots Ceased WO2015156808A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/US2014/033597 WO2015156808A1 (en) 2014-04-10 2014-04-10 Partial snapshots for creating generalized snapshots
US15/115,365 US9992379B2 (en) 2014-04-10 2014-04-10 Partial snapshots for creating generalized snapshots

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2014/033597 WO2015156808A1 (en) 2014-04-10 2014-04-10 Partial snapshots for creating generalized snapshots

Publications (1)

Publication Number Publication Date
WO2015156808A1 true WO2015156808A1 (en) 2015-10-15

Family

ID=54288227

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/033597 Ceased WO2015156808A1 (en) 2014-04-10 2014-04-10 Partial snapshots for creating generalized snapshots

Country Status (2)

Country Link
US (1) US9992379B2 (en)
WO (1) WO2015156808A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019501446A (en) * 2016-01-16 2019-01-17 平安科技（深▲せん▼）有限公司 Test method, test system, test apparatus, and readable recording medium

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10146667B2 (en) * 2014-04-10 2018-12-04 Entit Software Llc Generalized snapshots based on multiple partial snapshots
US10445166B2 (en) * 2014-06-24 2019-10-15 International Business Machines Corporation System verification of interactive screenshots and log files between client systems and server systems within a network computing environment
KR102592955B1 (en) * 2016-06-17 2023-10-24 삼성디스플레이 주식회사 Display panel, and electronic apparatus including the same
US11316896B2 (en) * 2016-07-20 2022-04-26 International Business Machines Corporation Privacy-preserving user-experience monitoring
US10425445B2 (en) * 2016-12-15 2019-09-24 Interwise Ltd Deception using screen capture
US11531644B2 (en) * 2020-10-14 2022-12-20 EMC IP Holding Company LLC Fractional consistent global snapshots of a distributed namespace
WO2025213427A1 (en) * 2024-04-11 2025-10-16 Lemon Inc. Purpose limit room for limiting purpose of data usage

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100079596A1 (en) * 2008-09-26 2010-04-01 Hong Fu Jin Precision Industry (Shenzhen)Co., Ltd. Device and method for automatically testing display device
WO2011073759A1 (en) * 2009-12-01 2011-06-23 Cinnober Financial Technology Ab Methods and systems for automatic testing of a graphical user interface
US20110214107A1 (en) * 2010-03-01 2011-09-01 Experitest, Ltd. Method and system for testing graphical user interfaces
US20130219365A1 (en) * 2011-05-05 2013-08-22 Carlo RAGO Method and system for visual feedback
WO2014041366A2 (en) * 2012-09-14 2014-03-20 Tangentix Limited Method and apparatus for delivery of interactive multimedia content over a network

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5670935A (en) 1993-02-26 1997-09-23 Donnelly Corporation Rearview vision system for vehicle including panoramic view
AU5896498A (en) 1996-12-06 1998-07-15 Stereographics Corporation Synthetic panoramagram
US6646680B1 (en) * 1998-09-30 2003-11-11 Foveon, Inc. Focusing method and apparatus for high resolution digital cameras
US7019773B1 (en) 2000-05-25 2006-03-28 Prc Inc. Video mosaic
JP4166585B2 (en) 2003-01-20 2008-10-15 株式会社サキコーポレーション Appearance inspection apparatus and appearance inspection method
JP4082318B2 (en) 2003-09-04 2008-04-30 カシオ計算機株式会社 Imaging apparatus, image processing method, and program
US20090100096A1 (en) * 2005-08-01 2009-04-16 Phanfare, Inc. Systems, Devices, and Methods for Transferring Digital Information
EP1821483A1 (en) 2006-02-21 2007-08-22 BrainLAB AG Computer network system and method for operating the network system screenshot and sourceshot control
ATE458231T1 (en) * 2007-07-08 2010-03-15 Univ Liege VISUAL BACKGROUND EXTRACTOR
TW200939763A (en) 2008-03-07 2009-09-16 Sunplus Technology Co Ltd Image processing system and method for processing divisional TV pictures
US9256510B2 (en) 2011-06-09 2016-02-09 International Business Machines Corporation Automatic rules based capturing of graphical objects for specified applications
DE102011078039A1 (en) 2011-06-24 2012-12-27 Siemens Aktiengesellschaft Generation of scan data and sequence control commands
US8452814B1 (en) 2011-10-24 2013-05-28 Google Inc. Gathering context in action to support in-context localization
US20130183951A1 (en) 2012-01-12 2013-07-18 Shih-Wei Chien Dynamic mobile application classification
US9608881B2 (en) 2012-04-13 2017-03-28 International Business Machines Corporation Service compliance enforcement using user activity monitoring and work request verification
US20140218385A1 (en) 2012-09-10 2014-08-07 Applitools Ltd. System and method for visual segmentation of application screenshots
US20140095600A1 (en) 2012-09-28 2014-04-03 Bradford H. Needham Multiple-device screen capture
US9429425B2 (en) 2013-03-05 2016-08-30 Here Global B.V. Aerial image collection
KR102172354B1 (en) 2013-06-28 2020-10-30 삼성전자주식회사 Image file generating method and apparatus thereof
CA2931216A1 (en) * 2013-12-31 2015-07-09 Ventana Medical Systems, Inc. Systems and methods for spectral unmixing of microscopic images using pixel grouping
US10146667B2 (en) 2014-04-10 2018-12-04 Entit Software Llc Generalized snapshots based on multiple partial snapshots

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100079596A1 (en) * 2008-09-26 2010-04-01 Hong Fu Jin Precision Industry (Shenzhen)Co., Ltd. Device and method for automatically testing display device
WO2011073759A1 (en) * 2009-12-01 2011-06-23 Cinnober Financial Technology Ab Methods and systems for automatic testing of a graphical user interface
US20110214107A1 (en) * 2010-03-01 2011-09-01 Experitest, Ltd. Method and system for testing graphical user interfaces
US20130219365A1 (en) * 2011-05-05 2013-08-22 Carlo RAGO Method and system for visual feedback
WO2014041366A2 (en) * 2012-09-14 2014-03-20 Tangentix Limited Method and apparatus for delivery of interactive multimedia content over a network

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019501446A (en) * 2016-01-16 2019-01-17 平安科技（深▲せん▼）有限公司 Test method, test system, test apparatus, and readable recording medium

Also Published As

Publication number Publication date
US9992379B2 (en) 2018-06-05
US20170006188A1 (en) 2017-01-05

Similar Documents

Publication Publication Date Title
US9992379B2 (en) Partial snapshots for creating generalized snapshots
AU2022203527B2 (en) Methods and systems for ranking, filtering and patching detected vulnerabilities in a networked system
CN113687974B (en) Client log processing method and device and computer equipment
US10146667B2 (en) Generalized snapshots based on multiple partial snapshots
AU2011340789B2 (en) End-user performance monitoring for mobile applications
US10521322B2 (en) Modeling and testing of interactions between components of a software system
CN111937006B (en) Systems that determine performance based on entropy values
US10706488B2 (en) Embedding debugging information via watermarks
US9317416B2 (en) Merging automated testing reports
US20160274997A1 (en) End user monitoring to automate issue tracking
AlGhamdi et al. Towards reducing the time needed for load testing
CN110737639A (en) Audit log method, device, computer equipment and storage medium
CN112799925A (en) Data acquisition method, apparatus, electronic device and readable storage medium
US11968095B2 (en) Capturing and correlating multiple sources of debugging information relating to a network resource via a browser extension
CN111198797B (en) Operation monitoring method and device and operation analysis method and device
Böhm et al. Visual decision-support for live digital forensics
CN107465652B (en) Operation behavior detection method, server and system
US20260057579A1 (en) Generating video streams to depict bot performance during an automation run
CN108595957A (en) Main browser page altering detecting method, device and storage medium
US20250363038A1 (en) Detecting funtional anomalies associated with software services in a distributed computing environment
US10432490B2 (en) Monitoring single content page application transitions
CN109582406B (en) Script-based security survey using a card system framework
CN120356131A (en) Method, device, equipment, storage medium and program product for determining abnormal data
CN107861867A (en) Page fault monitoring method, device, system and electronic equipment
CN105339974B (en) analog sensor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14888594

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15115365

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14888594

Country of ref document: EP

Kind code of ref document: A1