US20230374095A1 - Inter-application workflow performance analytics - Google Patents
Inter-application workflow performance analytics Download PDFInfo
- Publication number
- US20230374095A1 US20230374095A1 US18/309,982 US202318309982A US2023374095A1 US 20230374095 A1 US20230374095 A1 US 20230374095A1 US 202318309982 A US202318309982 A US 202318309982A US 2023374095 A1 US2023374095 A1 US 2023374095A1
- Authority
- US
- United States
- Prior art keywords
- application
- inter
- content management
- applications
- interaction events
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 claims abstract description 216
- 238000000034 method Methods 0.000 claims abstract description 96
- 238000005259 measurement Methods 0.000 claims abstract description 95
- 238000004891 communication Methods 0.000 claims abstract description 45
- 238000004458 analytical method Methods 0.000 claims description 10
- 230000004048 modification Effects 0.000 abstract description 9
- 238000012986 modification Methods 0.000 abstract description 9
- 238000004590 computer program Methods 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 abstract 2
- 238000007726 management method Methods 0.000 description 55
- 238000013523 data management Methods 0.000 description 23
- 238000012545 processing Methods 0.000 description 18
- 238000005192 partition Methods 0.000 description 16
- 230000009471 action Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 239000008186 active pharmaceutical agent Substances 0.000 description 8
- 238000013459 approach Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 7
- 238000000691 measurement method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 238000000638 solvent extraction Methods 0.000 description 4
- 239000000470 constituent Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000006855 networking Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K14/00—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof
- C07K14/435—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from animals; from humans
- C07K14/575—Hormones
- C07K14/59—Follicle-stimulating hormone [FSH]; Chorionic gonadotropins, e.g.hCG [human chorionic gonadotropin]; Luteinising hormone [LH]; Thyroid-stimulating hormone [TSH]
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K1/00—General methods for the preparation of peptides, i.e. processes for the organic chemical preparation of peptides or proteins of any length
- C07K1/14—Extraction; Separation; Purification
- C07K1/34—Extraction; Separation; Purification by filtration, ultrafiltration or reverse osmosis
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K14/00—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof
- C07K14/435—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from animals; from humans
- C07K14/475—Growth factors; Growth regulators
- C07K14/505—Erythropoietin [EPO]
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K14/00—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof
- C07K14/435—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from animals; from humans
- C07K14/52—Cytokines; Lymphokines; Interferons
- C07K14/555—Interferons [IFN]
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K14/00—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof
- C07K14/435—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from animals; from humans
- C07K14/575—Hormones
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K14/00—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof
- C07K14/435—Peptides having more than 20 amino acids; Gastrins; Somatostatins; Melanotropins; Derivatives thereof from animals; from humans
- C07K14/575—Hormones
- C07K14/61—Growth hormone [GH], i.e. somatotropin
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12N—MICROORGANISMS OR ENZYMES; COMPOSITIONS THEREOF; PROPAGATING, PRESERVING, OR MAINTAINING MICROORGANISMS; MUTATION OR GENETIC ENGINEERING; CULTURE MEDIA
- C12N15/00—Mutation or genetic engineering; DNA or RNA concerning genetic engineering, vectors, e.g. plasmids, or their isolation, preparation or purification; Use of hosts therefor
- C12N15/09—Recombinant DNA-technology
- C12N15/63—Introduction of foreign genetic material using vectors; Vectors; Use of hosts therefor; Regulation of expression
- C12N15/79—Vectors or expression systems specially adapted for eukaryotic hosts
- C12N15/85—Vectors or expression systems specially adapted for eukaryotic hosts for animal cells
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12N—MICROORGANISMS OR ENZYMES; COMPOSITIONS THEREOF; PROPAGATING, PRESERVING, OR MAINTAINING MICROORGANISMS; MUTATION OR GENETIC ENGINEERING; CULTURE MEDIA
- C12N9/00—Enzymes; Proenzymes; Compositions thereof; Processes for preparing, activating, inhibiting, separating or purifying enzymes
- C12N9/14—Hydrolases (3)
- C12N9/48—Hydrolases (3) acting on peptide bonds (3.4)
- C12N9/50—Proteinases, e.g. Endopeptidases (3.4.21-3.4.25)
- C12N9/64—Proteinases, e.g. Endopeptidases (3.4.21-3.4.25) derived from animal tissue
- C12N9/6421—Proteinases, e.g. Endopeptidases (3.4.21-3.4.25) derived from animal tissue from mammals
- C12N9/6424—Serine endopeptidases (3.4.21)
- C12N9/6437—Coagulation factor VIIa (3.4.21.21)
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12N—MICROORGANISMS OR ENZYMES; COMPOSITIONS THEREOF; PROPAGATING, PRESERVING, OR MAINTAINING MICROORGANISMS; MUTATION OR GENETIC ENGINEERING; CULTURE MEDIA
- C12N9/00—Enzymes; Proenzymes; Compositions thereof; Processes for preparing, activating, inhibiting, separating or purifying enzymes
- C12N9/14—Hydrolases (3)
- C12N9/48—Hydrolases (3) acting on peptide bonds (3.4)
- C12N9/50—Proteinases, e.g. Endopeptidases (3.4.21-3.4.25)
- C12N9/64—Proteinases, e.g. Endopeptidases (3.4.21-3.4.25) derived from animal tissue
- C12N9/6421—Proteinases, e.g. Endopeptidases (3.4.21-3.4.25) derived from animal tissue from mammals
- C12N9/6424—Serine endopeptidases (3.4.21)
- C12N9/644—Coagulation factor IXa (3.4.21.22)
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61K—PREPARATIONS FOR MEDICAL, DENTAL OR TOILETRY PURPOSES
- A61K38/00—Medicinal preparations containing peptides
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K2319/00—Fusion polypeptide
-
- C—CHEMISTRY; METALLURGY
- C07—ORGANIC CHEMISTRY
- C07K—PEPTIDES
- C07K2319/00—Fusion polypeptide
- C07K2319/31—Fusion polypeptide fusions, other than Fc, for prolonged plasma life, e.g. albumin
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12Y—ENZYMES
- C12Y304/00—Hydrolases acting on peptide bonds, i.e. peptidases (3.4)
- C12Y304/21—Serine endopeptidases (3.4.21)
- C12Y304/21021—Coagulation factor VIIa (3.4.21.21)
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12Y—ENZYMES
- C12Y304/00—Hydrolases acting on peptide bonds, i.e. peptidases (3.4)
- C12Y304/21—Serine endopeptidases (3.4.21)
- C12Y304/21022—Coagulation factor IXa (3.4.21.22)
Definitions
- This disclosure relates to shared content management systems, and more particularly to techniques for generating and presenting inter-application workflow performance analytics.
- Modern computing environments facilitate the proliferation and use of numerous applications that are accessed by the users to carry out the foregoing interactions over content objects. These applications (e.g., application-specific tools, point tools, etc.) are often used to improve the efficiency of the interactions performed over the content objects.
- a particular application may be selected from the hundreds of applications available in a computing environment for a variety of reasons such as: (1) familiarity and/or proficiency of users with the application, (2) popularity and/or functionality of the application as pertains to a particular content object type and/or a particular interaction (e.g., operation) over that content object type, and/or (3) for other reasons.
- CCM customer relationship management
- real-time messaging tool to perform various interactions and/or workflows over certain content objects (e.g., a proposal slide deck, an ROI analysis worksheet, etc.) that are associated with a particular sales opportunity.
- content objects e.g., a proposal slide deck, an ROI analysis worksheet, etc.
- a draft contract might be created and sent to outside counsel for their review and possible modification, which interactions might involve the use of a document editing tool, possibly in conjunction with a document revisions tracking tool.
- a successful contract execution event will spawn new interactions and/or workflows over new content objects (e.g., onboarding forms, training materials, etc.).
- new content objects e.g., onboarding forms, training materials, etc.
- the present disclosure describes techniques used in systems, methods, and in computer program products for inter-application workflow performance analytics, which techniques advance the relevant technologies to address technological issues with legacy approaches. More specifically, the present disclosure describes techniques used in systems, methods, and in computer program products for measuring the performance of workflows performed over multiple applications that are hosted on multiple heterogeneous computing systems. Certain embodiments are directed to technological solutions for linking content object interactions that are performed across multiple applications so as to generate sales engagement performance measurements of workflows that comprise the interactions.
- the disclosed embodiments modify and improve over legacy approaches.
- the herein-disclosed techniques provide technical solutions that address the technical problems attendant to measuring the performance of workflows performed over multiple applications.
- Such technical solutions involve specific implementations (i.e., data organization, data communication paths, module-to-module interrelationships, etc.) that relate to the software arts for improving computer functionality.
- the ordered combination of steps of the embodiments serve in the context of practical applications that perform steps for linking content object interactions that are performed across multiple applications so as to generate workflow performance measurements.
- these techniques for linking content object interactions that are performed across multiple applications overcome long standing yet heretofore unsolved technological problems associated with measuring the performance of workflows performed over multiple applications.
- FIG. 1 illustrates a computing environment in which embodiments of the present disclosure can be implemented.
- FIG. 2 is a flowchart depicting an inter-application workflow performance analysis technique as implemented in systems that measure the performance of workflows performed over multiple applications, according to an embodiment.
- FIG. 3 is a block diagram of a system that measures the performance of workflows performed over multiple applications, according to an embodiment.
- FIG. 4 illustrates an inter-application data management technique as used to measure the performance of workflows performed over multiple applications, according to an embodiment.
- FIG. 5 presents an interaction event recording technique as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications, according to an embodiment.
- FIG. 6 presents a workflow performance measurement technique as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications, according to an embodiment.
- FIG. 7 presents a workflow performance analytics presentation technique as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications, according to an embodiment.
- FIG. 8 A and FIG. 8 B depict system components as arrangements of computing modules that are interconnected so as to implement certain of the herein-disclosed embodiments.
- FIG. 9 A and FIG. 9 B present block diagrams of computer system architectures having components suitable for implementing embodiments of the present disclosure, and/or for use in the herein-described environments.
- aspects of the present disclosure solve problems associated with using computer systems for measuring the performance of workflows performed over multiple applications. These problems are unique to, and may have been created by, various computer-implemented methods for measuring the performance of workflows performed over multiple applications in the context of content management systems. Some embodiments are directed to approaches for linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions.
- the accompanying figures and discussions herein present example environments, systems, methods, and computer program products for measuring the performance of workflows performed over multiple applications.
- the techniques are implemented in a computing environment comprising a content management system that facilitates interactions over a plurality of content objects that are created by, or modified by, or accessed by a plurality of applications.
- the applications available in the computing environment can include a native application (e.g., browser portal, mobile application, etc.) that is provided by the content management system and is accessed by its users (e.g., managed users) to facilitate certain interactions (e.g., authoring, editing, viewing, etc.) with and/or modifications to the content objects.
- a native application e.g., browser portal, mobile application, etc.
- users e.g., managed users
- Such modification to the content objects are modifications that arise from operations performed by an application during the lifecycle of a project.
- a contract might be signed many times using a document signing application.
- an onboarding form might be populated from a signed contract using a data import application.
- the applications may comprise third-party applications that are available in the overall computing environment.
- Such third-party applications are applications that are not provided and/or maintained by the provider of the content management system, but rather are applications that are integrated with the content management system to facilitate certain interactions with at least some of the types of content objects managed at the content management system.
- the interaction events associated with the applications are processed to select specific subsets of interaction events. For example, a request might be issued from a native application (e.g., at direction of a user) so as to select a set of interaction events that correspond to a particular project (e.g., sales engagement) lifecycle. Performance measurements associated with various workflow performance metrics are generated from the selected interaction events. As an example, the workflow performance metrics might pertain to various aspects of a sales engagement lifecycle. Visual representations of the performance measurements are then formed and presented to one or more of the users.
- a dashboard of workflow performance metrics might be populated with the performance measurements and presented to a user at a user interface.
- workflow performance metrics for a particular project lifecycle might pertain to the entire project lifecycle, or might pertain to particular workflows of the project, or might pertain to particular operations of the workflows, and/or other aspects of the project lifecycle.
- At least one of A or B means at least one of A, or at least one of B, or at least one of both A and B. In other words, this phrase is disjunctive.
- the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or is clear from the context to be directed to a singular form.
- FIG. 1 illustrates a computing environment 100 in which embodiments of the present disclosure can be implemented.
- computing environment 100 may be implemented in the context of the architecture and functionality of the embodiments described herein.
- FIG. 1 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure presents a logical depiction of how certain of the herein disclosed techniques can be used to generate inter-application performance measurements from interactions performed over multiple applications. A representative scenario and set of high order operations are also presented to illustrate how the herein disclosed techniques are applied in computing environment 100 .
- FIG. 1 depicts a representative set of users 102 who interact with various instances of content objects 106 managed at a content management system 108 .
- Users 102 may be users (e.g., managed users) of content management system 108 that facilitates interactions (e.g., authoring, editing, viewing, etc.) over content objects 106 for sharing, collaboration, and/or other purposes.
- interactions e.g., authoring, editing, viewing, etc.
- interactions are organized into workflows.
- Interactions and/or workflows over a particular content object may be performed with some human interaction by one or more of users 102 and/or with autonomous interaction by one or more computing entities (e.g., processes, agents, applications, etc.).
- applications 104 might comprise instances of a native application provided by content management system 108 to facilitate the interactions of users 102 with content objects 106 and/or interactions with one another.
- the native application might be a browser portal or mobile application accessed by a user to manage a set of content objects that the user has authority to access.
- Applications 104 might also comprise various third-party applications.
- a third-party application might be a web application, mobile application, or another type of application that is served by a third-party server and accessed by some or all of users 102 to perform the aforementioned interactions.
- the applications are referred to as “third-party applications” as they are provided (e.g., developed, delivered, served, maintained, etc.) by an entity (e.g., the owner of the third-party server) other than the entity providing the content management system 108 .
- entity e.g., the owner of the third-party server
- third-party applications are applications that are not provided and/or maintained by content management system 108
- applications 104 may still be integrated with content management system 108 to facilitate certain interactions with at least some of the types of content objects 106 managed at content management system 108 .
- Such integration may include registration of applications 104 with content management system 108 , establishment of APIs to facilitate communication between the applications and system, and/or other integration capabilities.
- applications 104 can improve the efficiency of the interactions performed over content objects 106 .
- a particular application may be selected from the hundreds of applications available in computing environment 100 for a variety of reasons, such as the familiarity and/or proficiency of users with the application, the popularity and/or functionality of the application as pertains to a particular content object type, and/or a particular interaction (e.g., operation) over that content object type, and/or for other reasons.
- workflows over a particular content object might be performed entirely at one or more of the applications 104 , entirely at content management system 108 , or at some combination of one or more of applications 104 and content management system 108 .
- project lifecycle 150 might pertain to a project associated with a particular customer and comprise a “Proposal” stage, a “Contract” stage, an “Onboarding” stage, and a “Support” stage.
- Project lifecycle 150 might pertain to a project associated with a particular customer and comprise a “Proposal” stage, a “Contract” stage, an “Onboarding” stage, and a “Support” stage.
- Project lifecycle 150 might pertain to a project associated with a particular customer and comprise a “Proposal” stage, a “Contract” stage, an “Onboarding” stage, and a “Support” stage.
- Project lifecycle 150 might pertain to a project associated with a particular customer and comprise a “Proposal” stage, a “Contract” stage, an “Onboarding” stage, and a “Support” stage.
- the users involved over project lifecycle 150 include a sales engineer (e.g., user “SE”), an applications engineer (e.g., user “AE”), a program manager (e.g., user “PM”), a contract attorney (e.g., user “CA”), a customer representative (e.g., user “CR”), a service manager (e.g., user “SM”), a service technician (e.g., user “ST”), and/or other users.
- a sales engineer e.g., user “SE”
- an applications engineer e.g., user “AE”
- PM program manager
- CA contract attorney
- CA customer representative
- service manager e.g., user “SM”
- ST service technician
- the foregoing users might access a SalesForce application (e.g., app “SF”), a Slack application (e.g., app “SL”), a Gmail application (e.g., app “GM”), a Microsoft application (e.g., app “MS”), a DocuSign application (e.g., app “DS”), a ServiceNow application (e.g., app “SN”), a ZenDesk application (e.g., app “ZD”), and/or other applications to interact with various content objects (e.g., a “Proposal.pptx” file, a “Contract.docx” file, an “OnBoarding” folder and underlying files, a “Support” folder and underlying files, etc.) over project lifecycle 150 .
- a SalesForce application e.g., app “SF”
- Slack application e.g., app “SL”
- Gmail application e.g., app “GM”
- one or more of the aforementioned users or another user may desire to know the time to complete certain workflows (e.g., complete proposal, draft and sign contract, etc.).
- a different user e.g., a sales VP
- a still different user or different type of user e.g., user “PM”
- PM might desire to know how much time is spent using the applications (e.g., point tools) and how such usage relates to achievement of various specific objectives that are accomplished over the duration of the project.
- applications e.g., point tools
- the herein disclosed techniques address such challenges pertaining to measuring the performance of workflows performed over multiple applications at least in part by linking interactions associated with content objects 106 performed over multiple heterogeneous instances of applications 104 to generate performance measurements of workflows that comprise the interactions.
- an inter-application data management framework 112 is also established at content management system 108 to facilitate management of any data pertaining to interactions performed over content objects at multiple heterogeneous applications (operation 1). More specifically, inter-application data management framework 112 comprises various specialized data structures that, at least in part, link the interactions performed over content objects at multiple heterogeneous applications.
- an inter-application data management framework is any operational element that receives and stores information from a plurality of applications so as to facilitate subsequent querying and retrievals by another operational element.
- inter-application data management framework 112 are used to organize and store application-specific information 132 associated with applications 104 (e.g., application attributes, workflow states, task states, etc.) and/or enterprise-specific information 134 (e.g., workflow definitions, interaction definitions, metric definitions, etc.) associated with users 102 .
- application-specific information 132 associated with applications 104
- enterprise-specific information 134 e.g., workflow definitions, interaction definitions, metric definitions, etc.
- event attributes may comprise attributes that describe the interaction and other attributes that describe certain entities (e.g., users, content objects, workflows, etc.) associated with the interaction events.
- the event attributes might also comprise one or more link attributes that facilitate the aforementioned links between the applications.
- inter-application analytics engine 110 analyzes the foregoing event attributes associated with inter-application interactions to generate one or more inter-application performance measurements 124 (operation 4).
- data organized and stored in accordance with inter-application data management framework 112 is often accessed by inter-application analytics engine 110 to generate one or more of the inter-application performance measurements 124 .
- certain enterprise-specific metric definitions organized according to the inter-application data management framework 112 might be queried by inter-application analytics engine 110 to retrieve a performance measurement for a particular metric.
- the inter-application performance measurements determined by the herein disclosed techniques are then presented to one or more of users 102 (operation 5).
- the set of inter-application performance measurements 124 may be presented to a user at the user interface of a native application.
- an inter-application performance dashboard view 152 presented at the user interface graphically depicts performance measurements associated with “ProjectX”, which performance measurements include a measurement associated with a first metric “Metric1”, a measurement associated with a second metric “Metric2”, as well as additional performance measurements associated with applications “SF”, “MS”, and “GM”, which are shown as “AppSF”, “AppMS”, and “AppGM”, respectively.
- the automatic inter-application performance analytics capability facilitated by the herein disclosed techniques serves to address the problems attendant to measuring the performance of workflows performed over multiple applications.
- application of the techniques disclosed herein facilitate improvements in computer functionality that serve to reduce the demand for computer memory, reduce the demand for computer processing power, reduce network bandwidth use, and reduce the demand for intercomponent communication. Specifically, consumption of computing resources that might be needed to at least facilitate manual recording and aggregation of interactions performed over a large number of users, content objects, and applications is eliminated.
- FIG. 2 is a flowchart depicting an inter-application workflow performance analysis technique 200 as implemented in systems that measure the performance of workflows performed over multiple applications.
- inter-application workflow performance analysis technique 200 may be implemented in the context of the architecture and functionality of the embodiments described herein.
- the inter-application workflow performance analysis technique 200 or any aspect thereof may be implemented in any environment.
- FIG. 2 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions.
- the figure is presented to illustrate one embodiment of certain steps and/or operations performed over a network of devices (e.g., user devices, computing systems, etc.) to record and analyze interactions over content objects performed at multiple heterogeneous applications to generate inter-application performance measurements.
- the steps and/or operations can be grouped into a set of setup operations 210 and a set of ongoing operations 220 .
- Setup operations 210 of inter-application workflow performance analysis technique 200 commences by identifying or configuring a content management system that facilitates interactions over a plurality of users and a plurality of content objects (step 212 ). Such interactions can involve both user-to-user interactions and user-to-content interactions.
- a plurality of applications e.g., apps
- apps are integrated with the content management system to facilitate interactions over the users and/or content objects performed by the apps (step 214 ).
- a sales contract document managed by the content management system might be shared using a first application (e.g., SalesForce) to facilitate the development of the contract, after which development the contract might be submitted to a second application (e.g., DocuSign) to facilitate execution (e.g., signing) of the contract.
- the SalesForce and DocuSign applications might be registered with the content management system to facilitate authorized access to the sales contract document managed (e.g., stored, updated, etc.) at the content management system.
- An inter-application data management framework is established at the content management system to interrelate certain sets of interactions performed over the applications (step 216 ).
- Frameworks such as the inter-application data management framework or any other framework discussed herein, are sets of computing entities that cooperatively facilitate various data collection and data distribution functions.
- the computing entities of a framework might comprise various data structures, application programming interfaces (APIs), communication protocols, authentication protocols, and/or other entities. At least some of the entities of a framework are often strictly designed to facilitate a certain function or certain set of functions.
- APIs application programming interfaces
- an embodiment of the inter-application data management framework described herein might specify that one or more link attributes are to be included in API calls and/or certain data records to facilitate the aforementioned linking between applications.
- Other entities of a framework might be designed to offer flexibility.
- an embodiment of the inter-application data management framework described herein might comprise specialized data structures that facilitate enterprise-specific definitions of workflows, interactions, or metrics.
- any interaction events corresponding to interactions over the content objects performed by the applications are recorded (step 222 ).
- certain APIs and/or data structures of the inter-application data management framework might be used to record event attributes associated with interaction events invoked at the SalesForce and DocuSign applications.
- sets of interaction events are selected, which selected sets comprise events from two or more applications (step 224 ).
- a set of interaction events associated with one or more link attributes e.g., project name, customer name, workflow name, document name, etc.
- One or more inter-application performance measurements pertaining to the sets of interaction events are generated (step 226 ).
- inter-application performance measurements that pertain to interaction events from respective workflows are generated.
- inter-application performance measurements are then presented to one or more users (step 228 ). For example, at least some of the measurements might be visually presented in text and/or graphics at user interfaces accessible by the users.
- FIG. 3 is a block diagram of a system 300 that measures the performance of workflows performed over multiple applications.
- system 300 may be implemented in the context of the architecture and functionality of the embodiments described herein.
- the system 300 or any aspect thereof may be implemented in any environment.
- FIG. 3 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions.
- the figure is being presented to show one embodiment of certain representative components and associated data structures and data flows implemented in a computing environment to facilitate the herein disclosed techniques.
- the components, data flows, and data structures are associated with a set of users (e.g., user 102 1 , . . . , user 102 N ) that interact with each other and a set of content objects 106 managed at a content management system 108 .
- the components, data flows, and data structures shown in FIG. 3 present one partitioning and associated data manipulation approach.
- the specific example shown is purely exemplary, and other subsystems, data structures, and/or partitionings are reasonable.
- system 300 comprises an instance of content management server 310 operating at content management system 108 .
- Content management server 310 comprises a message processor 312 and an instance of an inter-application analytics engine 110 , which comprises a selection service 314 , an event analyzer 316 , and a display generator 318 .
- a plurality of instances of the foregoing components might operate at a plurality of instances of servers (e.g., content management server 310 ) at content management system 108 and/or any portion of system 300 .
- Such instances can interact with a communications layer 320 to access each other and/or a set of storage devices 330 that store various information to support the operation of the components of system 300 and/or any implementations of the herein disclosed techniques.
- the servers and/or storage devices of content management system 108 might facilitate interactions over content objects 106 by the users (e.g., user 102 1 , . . . , user 102 N ) from a respective set of user devices (e.g., user device 302 1 , . . . , user device 302 N ).
- a content management system “manages” a plurality of content objects at least in part by maintaining (e.g., storing, updating, resolving interaction conflicts, etc.) the content objects subject to the various interactions performed over the content objects by users of the content objects at their respective user devices.
- the content objects (e.g., files, folders, etc.) in content objects 106 are characterized at least in part by a set of object attributes 340 (e.g., content object metadata) stored at storage devices 330 .
- the users are characterized at least in part by a set of user attributes 342 stored in a set of user profiles 332 at storage devices 330 .
- the users access instances of applications 104 at their respective user devices to interact with content objects 106 managed by content management system 108 .
- the applications can comprise instances of native applications (e.g., native application 304 1 , . . . , native application 304 N ) or instances of third-party applications (e.g., a SalesForce or “SF” application, a DocuSign or “DS” application, etc.).
- native applications e.g., native application 304 1 , . . . , native application 304 N
- third-party applications e.g., a SalesForce or “SF” application, a DocuSign or “DS” application, etc.
- Various information pertaining to integrations of such third-party applications with content management system 108 are normalized and then codified for storage in an app registry 336 , which in turn is maintained as data in storage devices 330 . At least some information of app registry 336 is organized according to an inter-application data management framework as earlier described.
- app registry 336 certain portions of the information in app registry 336 might be locally accessible at the user devices by the third-party applications. For example, a first local app registry might be accessible by app “SF” at user device 302 1 and a second local app registry might be accessible by app “DS” at user device 302 N .
- the instances of the applications operating at the user devices send or receive various instances of messages 322 that are received or sent by message processor 312 at content management server 310 .
- messages 322 are sent to or received from content management server 310 without human interaction.
- One class of messages 322 corresponds to application-specific information received at content management system 108 in response to executing application integration operations.
- instances of application attributes 346 that correspond to a particular application might be issued by an enterprise and stored in app registry 336 when the application is registered with content management system 108 .
- Select users (e.g., system administrators) at enterprises may also submit certain policy information and/or other enterprise-specific information in instances of messages 322 .
- Such enterprise-specific policy information is stored in a set of enterprise policies 338 at storage devices 330 .
- enterprise policies 338 might comprise instances of workflow definitions 347 , interaction definitions 348 , metric definitions 349 , and/or other information that can be accessed to facilitate the herein disclosed techniques.
- an enterprise-specific policy is any information that identifies and/or governs the use of workflows, and/or information that identifies or characterized tasks of the workflows, and/or information that pertains to metrics to be considered when generating performance measurements.
- Another class of messages 322 corresponds to interaction events that are invoked by the users when they interact with one another (e.g., user-to-user interactions) and/or when users interact with various content objects (e.g., user-to-content interactions).
- the interactions can be performed at any of the instances of applications 104 (e.g., native applications, third-party applications, etc.).
- the users might log in to their respective instances of a native application or any third-party applications to interact with content objects that they own or that are shared with them to invite other users to collaborate on content objects and/or to perform other collaboration activities. Any of the foregoing interactions or collaboration activities can be characterized by one or more interaction events.
- the message processor 312 at content management server 310 monitors the messages 322 to detect interaction event messages that correspond to interaction events performed over the plurality of content objects at the plurality of applications.
- Message processor 312 codifies certain event attributes 344 pertaining to the interaction events in a set of event records 334 stored in storage devices 330 .
- message processor 312 will retrieve other attributes (e.g., user attributes 342 , object attributes 340 , application attributes 346 , interaction definition attributes, etc.) to facilitate populating the event records 334 .
- instances of messages 322 comprising measurement requests are received at message processor 312 .
- Such requests are issued to retrieve one or more inter-application performance measurements from content management system 108 .
- user 102 1 might load a performance dashboard view in native application 304 1 that invokes one or more measurement requests to facilitate population of the dashboard view.
- inter-application analytics engine 110 accesses the selection service 314 to select respective sets of selected interaction events 326 from event records 334 in accordance with request attributes associated with the requests.
- request attributes associated with the requests For example, consider a measurement request that includes a link attribute that identifies a particular customer. In this case, the customer identifier and/or other attributes might be included in a query of event records 334 to select a set of interaction events that are associated with the customer.
- the event analyzer 316 is then called to analyze the event attributes that constitute the selected interaction events 326 to generate one or more instances of inter-application performance measurements 124 .
- the inter-application performance measurements 124 generated by event analyzer 316 will often be based at least in part on other data available at content management system 108 , such as certain data stored in enterprise policies 338 and/or app registry 336 .
- the inter-application performance measurements 124 are accessed by the display generator 318 to prepare the measurements for presentation.
- display generator 318 generates the user interface display elements that comprise an active, visual presentation of the inter-application performance measurements 124 and/or other information.
- Such user interface display elements might comprise data tables, charts, graphs, and/or other human-readable elements.
- the user interface display elements and other information associated with the inter-application performance measurements 124 are codified into instances of messages 322 which are in turn delivered to the users.
- instances of user interface display elements that describe various inter-application performance measurements are dynamically generated and presented to respective users in their corresponding instances of a native application.
- FIG. 4 illustrates an inter-application data management technique 400 as used to measure the performance of workflows performed over multiple applications.
- inter-application data management technique 400 may be implemented in the context of the architecture and functionality of the embodiments described herein.
- the inter-application data management technique 400 or any aspect thereof may be implemented in any environment.
- FIG. 4 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions.
- the figure presents certain specialized data structures for organizing and/or storing various data to facilitate the aforementioned inter-application linking and other techniques disclosed herein.
- the data structures are associated with the inter-application data management framework 112 earlier described.
- data structures and other data structures discussed herein are configured to improve the way a computer stores and retrieves certain data in memory when performing the herein disclosed techniques.
- Data can be organized and/or stored in accordance with these data structures using various techniques.
- the representative data structures associated with inter-application data management framework 112 shown in FIG. 4 indicate that the constituent data of the data structures might be organized and/or stored in a tabular structure (e.g., relational database table) that has rows that relate various attributes with a particular data entity.
- the underlying data might be organized and/or stored in a programming code object that has instances corresponding to a particular data entity and properties corresponding to the various attributes associated with the data entity.
- a representative set of select data structure relationships 420 between certain data entities are also shown.
- app registry 336 When instances of applications 104 (e.g., third-party applications) are integrated (e.g., registered) with a content management system 108 , respective sets of application-specific information are populated in an app registry 336 . Moreover, instances of enterprises 404 that have access to content management system 108 populate respective instances of enterprise policies 338 with enterprise-specific information.
- the data structures of app registry 336 and enterprise policies 338 are defined by the inter-application data management framework 112 to facilitate the herein disclosed techniques.
- the system supports a plurality of applications 104 as well as a plurality of enterprises 404 .
- data that is stored in the app registry 336 and/or data that is stored in enterprise policies may be normalized so as to facilitate comparisons and other processing in spite of a plurality of origins and formats of incoming data.
- a first application of the applications 104 might refer to its “application ID” as a number
- a second application of the applications 104 might refer to its “application ID” as a character string.
- a normalizer e.g., 405 1
- a first app might refer to an event raised at or by particular enterprise using an app-specific ID (e.g., “entID” of the select event attributes 428 ) whereas a second app might refer to a different event raised at or by the same particular enterprise using a different app-specific ID (e.g., “entID” of the select event attributes 428 ).
- an app-specific ID e.g., “entID” of the select event attributes 428
- a second app might refer to a different event raised at or by the same particular enterprise using a different app-specific ID (e.g., “entID” of the select event attributes 428 ).
- the normalization processing of normalizer 405 1 creates and manages a plurality of maps, each of which maps relate incoming data (e.g., from the applications 104 ) to an internal representation. For example, in a first map or map entry, the attribute value “Enterprise1” might be mapped to “CustomerA”, and in a second map or map entry, the attribute value “Enterprise2” might be also mapped to “CustomerA”. As such, variations that may occur in the representation of any attribute in any particular app can be normalized into a comment representation.
- data that is stored in the enterprise policies 338 may be normalized so as to facilitate comparisons and other processing in spite of a plurality of origins and formats of incoming data.
- a first enterprise of the enterprises 404 might refer to its “workflow ID” as a number
- a second enterprise of the enterprises 404 might refer to its “workflow ID” as a character string.
- a normalizer e.g., 405 2
- normalization processing of normalizer 405 2 creates and manages a plurality of maps, each of which maps relate incoming data (e.g., from the enterprises 404 ) to an internal representation which is in turned used by operational elements of the system.
- the content management system 108 also stores instances of event records 334 in accordance with the data structures of the inter-application data management framework 112 .
- Data stored in the data structures of the inter-application data management framework 112 may be normalized so as to facilitate comparisons.
- one data item might refer to units of time or durations as a number of days, whereas another data item might refer units of time or durations as a number of hours.
- days can be normalized to a number and/or fraction of hours, or, for comparison purposes, hours can be normalized to a number and/or fraction of days.
- certain portions of the aforementioned data structures and/or data stores are populated in response to various inputs (e.g., selections, entered text, etc.) received from system administrators and/or application developers in response to interactions with various user interfaces (e.g., admin and/or developer console).
- various inputs e.g., selections, entered text, etc.
- system administrators and/or application developers in response to interactions with various user interfaces (e.g., admin and/or developer console).
- user interfaces e.g., admin and/or developer console.
- an application developer might first register a set of applications, then a system administrator might define certain interactions that are performed at the applications.
- the system administrator might also define various workflows and/or workflow performance metrics that pertain to combinations of the foregoing interactions.
- each of the applications that are registered with the content management system is identified by an application identifier (e.g., stored in an “appID” field), an application name (e.g., stored in an “appName” field), an enterprise identifier (e.g., stored in an “entID” field), an endpoint URL (e.g., stored in an “endpoint” field), a set of OAuth2 credentials (e.g., stored in an “OAuth2[ ]” object), and/or other attributes.
- the application identifier or “appID” is referenced by other data structures to associate the data underlying those structures with a particular application.
- Certain attributes (e.g., “entID”, “endpoint”, etc.) from select application attributes 422 might be included in interaction event messages issued from the applications to facilitate identification of the particular instances of the application or applications that issued the messages.
- Such interaction event messages may also identify one or more interactions as defined by a set of select interaction definition attributes 426 .
- an interaction associated with an application identified in an “appID” field is defined by an interaction identifier (e.g., stored in an “actionID” field), an interaction type description (e.g., stored in an “action” field), a set of interaction parameters specific to the interaction (e.g., stored in an “iParams[ ]” object), and/or other attributes.
- an interaction identifier e.g., “SF.create” might be constructed from the application identifier (e.g., “SF”) and the interaction type description (e.g., “create”).
- a particular workflow is defined by a workflow identifier (e.g., stored in a “workflowID” field), a workflow name (e.g., stored in a “name” field), a workflow description (e.g., stored in a “description” field), a set of tasks associated with the workflow (e.g., stored in a “tasks[ ]” object), a set of metrics associated with the workflow (e.g., stored in a “metrics[ ]” object), and/or other attributes.
- a workflow identifier e.g., stored in a “workflowID” field
- a workflow name e.g., stored in a “name” field
- a workflow description e.g., stored in a “description” field
- a set of tasks associated with the workflow e.g., stored in a “tasks[ ]” object
- a set of metrics associated with the workflow e.g., stored in a “metrics[ ]” object
- each task of the workflow is described by an interaction identifier (e.g., stored in an “actionID” field), a task sequence index (e.g., stored in an “index” field), a task state description (e.g., stored in a “state” field), and/or other attributes.
- each metric associated with the workflow is described by a metric identifier (e.g., stored in a “metricID” field), an interaction identifier that signals the start event of the metric (e.g., stored in an “actionID_s” field), an interaction identifier that signals the end event of the metric (e.g., stored in an “actionID_e” field), and/or other attributes.
- each event record stored in event records 334 comprises an object identifier (e.g., stored in an “objID” field), an application identifier (e.g., stored in an “appID” field), an interaction type description (e.g., stored in an “action” field), an enterprise identifier (e.g., stored in an “entID” field), a link identifier (e.g., stored in a “linkID” field), a link name (e.g., stored in a “linkName” field), a timestamp (e.g., stored in a “time” field), one or more interaction parameters specific to the interaction event (e.g., stored in an “iParams[ ]” object), and/or other attributes.
- the link identifier and/or the link name and/or other attributes can constitute a set of link attributes 430 that facilitate the linking of interactions over content objects performed over multiple heterogeneous applications.
- FIG. 5 presents an interaction event recording technique 500 as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications.
- one or more variations of interaction event recording technique 500 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein.
- the interaction event recording technique 500 or any aspect thereof may be implemented in any environment.
- FIG. 5 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure is presented to illustrate one embodiment of certain steps and/or operations that facilitate recording interaction events performed over content objects at various heterogeneous applications. As depicted in the figure, the steps and/or operations are associated with step 222 of FIG. 2 . A representative scenario is also shown in the figure to illustrate an example application of interaction event recording technique 500 .
- the interaction event recording technique 500 commences by monitoring a plurality of applications for interaction events (step 502 ).
- an instance of message processor 312 may continuously listen or poll for interaction events performed at a plurality of applications that includes an instance of app “SF” accessed by user “SE” and an instance of app “DS” accessed by user “CR”.
- interaction event messages are received (step 504 )
- the interaction event messages are parsed to retrieve respective sets of interaction attributes from the messages (step 506 ).
- message processor 312 receives an interaction event message 522 1 in response to user “SE” interacting with file “F3” at app “SF” and receives an interaction event message 522 K in response to user “CR” interacting with file “F3” at app “DS”.
- the interaction attributes associated with the interaction event messages include an application identifier (e.g., stored in an “appID” field), an interaction type description (e.g., stored in an “action” field), a timestamp (e.g., stored in a “time” field), a user identifier (e.g., stored in a “userID” field), an enterprise identifier (e.g., stored in an “entID” field), a link identifier (e.g., stored in a “linkID” field), a content object identifier (e.g., stored in an “objID” field), and/or other attributes.
- an application identifier e.g., stored in an “appID” field
- an interaction type description e.g., stored in an “action” field
- a timestamp e.g., stored in a “time” field
- a user identifier e.g., stored in a “userID” field
- an enterprise identifier
- message processor 312 might access the datastores of content objects 106 , user profiles 332 , app registry 336 , enterprise policies 338 , and/or other data sources to retrieve certain attributes associated with the interaction attributes of the interaction event messages.
- All retrieved attributes are then recorded as event attributes associated with the interaction event message (step 512 ).
- the event attributes comprise some or all of the retrieved interaction attributes and the retrieved other attributes. If merely the interaction attributes are considered (“No” path of decision 508 ), the event attributes comprise some or all of the retrieved interaction attributes.
- message processor 312 stores in event records 334 sets of event attributes that correspond to a respective set of subject interaction events 544 described by interaction event message 522 1 and interaction event message 522 K .
- FIG. 6 presents a workflow performance measurement technique 600 as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications.
- workflow performance measurement technique 600 may be implemented in the context of the architecture and functionality of the embodiments described herein.
- the workflow performance measurement technique 600 or any aspect thereof may be implemented in any environment.
- FIG. 6 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure is presented to illustrate one embodiment of certain steps and/or operations that facilitate selecting and analyzing sets of interaction events associated with two or more applications to generate inter-application performance measurements. As depicted in the figure, the steps and/or operations are associated with step 224 and step 226 of FIG. 2 . A representative scenario is also shown in the figure to illustrate an example application of workflow performance measurement technique 600 .
- the workflow performance measurement technique 600 commences by monitoring a plurality of applications for measurement requests (step 602 ). As illustrated, an instance of message processor 312 may continuously listen or poll for measurement requests issued from applications to retrieve various inter-application workflow performance measurements. When a measurement request is received (step 604 ), the request is parsed to retrieve its request attributes (step 606 ). For example, message processor 312 might receive a measurement request 622 issued from native application 304 1 by user 102 1 (e.g., a system administrator or “SA”).
- SA system administrator
- the request attributes associated with measurement request 622 or any measurement request include a user identifier (e.g., stored in a “userID” field), an enterprise identifier (e.g., stored in an “entID” field), a content object identifier (e.g., stored in an “objID” field), a workflow identifier (e.g., stored in a “workflowID” field), a metric identifier (e.g., stored in a “metricID” field), a link identifier (e.g., stored in a “linkID” field), and/or other attributes.
- a user identifier e.g., stored in a “userID” field
- an enterprise identifier e.g., stored in an “entID” field
- a content object identifier e.g., stored in an “objID” field
- a workflow identifier e.g., stored in a “workflowID” field
- metric identifier
- a set of interaction events are selected based at least in part on the request attributes (step 608 ).
- message processor 312 forwards the request attributes extracted from measurement request 622 to an instance of a selection service 314 .
- the selection service 314 uses one or more of the request attributes (e.g., the link attribute) to query the event records 334 to select a set of subject selected interaction events 626 that pertain to the measurement request.
- any enterprise policy information associated with the measurement request are accessed (step 612 ).
- an instance of an event analyzer 316 might use certain request attributes (e.g., enterprise identifier, workflow identifier, metric identifier, etc.) to retrieve any relevant enterprise policy information from a store of enterprise policies 338 .
- the enterprise policy information is applied to the event attributes associated with the selected interaction events to determine one or more inter-application performance measurements (step 614 ).
- the enterprise policy information includes content object access information.
- enterprise policy information might allow or deny access to a content object by a particular user.
- such enterprise policy information might allow or deny access to any actions taken over a particular content object by a particular user. As such it can happen that a particular user many not be able to see measurements that derive from events that are not accessible (e.g., access denied) to that particular user.
- event analyzer 316 applies the enterprise policy information retrieved from enterprise policies 338 to the event attributes of subject selected interaction events 626 to generate a particular inter-application performance measurement.
- event analyzer 316 generates a performance measurement for metric “Metric1” from interaction events performed over two or more applications (e.g., app “SF” and app “DS”) that are linked by a “ProjX” link identifier.
- the events are linked by the “ProjX” link identifier that appears in multiple timestamped instances of the select subject event attributes 632 .
- the action “create” and the action “sign” are interrelated by the data structure shown as select subject metric attributes 634 .
- a measurement for “Metric1” might be equal to 10 (e.g., “t123” ⁇ “t113”) time units (e.g., days).
- the metrics are not generic hardware-centric metrics such as CPU utilization or memory usage, but rather are metrics that pertain to the usage of a particular application and/or that pertain to operations that are performed cooperatively between two or more heterogeneous applications.
- FIG. 7 presents a workflow performance analytics presentation technique 700 as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications.
- workflow performance analytics presentation technique 700 may be implemented in the context of the architecture and functionality of the embodiments described herein.
- the workflow performance analytics presentation technique 700 or any aspect thereof may be implemented in any environment.
- FIG. 7 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions.
- the figures are presented to illustrate one embodiment of certain steps and/or operations that facilitate presenting inter-application performance measurements to a user. As depicted in the figure, the steps and/or operations are associated with step 228 of FIG. 2 .
- a representative scenario is also shown in the figures to illustrate an example application of workflow performance analytics presentation technique 700 .
- the workflow performance analytics presentation technique 700 commences by retrieving a set of inter-application performance measurements to present to a user (step 702 ). As illustrated, an instance of display generator 318 might retrieve a set of inter-application performance measurements 124 . A set of measurement display components associated with the inter-application performance measurements is accessed (step 704 ). As shown, display generator 318 accesses a set of measurement display components 722 to identify certain components (e.g., chart types, style sheets, etc.) associated with inter-application performance measurements 124 . For example, horizontal bar charts might be identified as the chart type for the measurements.
- components e.g., chart types, style sheets, etc.
- Certain user interface display elements are constructed from the display components and the inter-application performance measurements (step 706 ).
- display generator 318 might access a user interface library (e.g., React library) to generate the code for rendering the display components of inter-application performance measurements 124 and other related information at a target user interface.
- the user interface display elements are then presented to the user at the target user interface (step 708 ).
- the user interface display elements associated with inter-application performance measurements 124 are presented in an inter-application performance dashboard view 152 at native application 304 1 associated with user 102 1 .
- inter-application performance dashboard view 152 is a graphical visualization of a specific inter-application performance measurement value 724 that corresponds to “Metric1”.
- FIG. 8 A depicts a system 8 A 00 as an arrangement of computing modules that are interconnected so as to operate cooperatively to implement certain of the herein-disclosed embodiments.
- This and other embodiments present particular arrangements of elements that, individually or as combined, serve to form improved technological processes that address measuring the performance of workflows performed over multiple applications.
- the partitioning of system 8 A 00 is merely illustrative and other partitions are possible.
- the system 8 A 00 may be implemented in the context of the architecture and functionality of the embodiments described herein. Of course, however, the system 8 A 00 or any operation therein may be carried out in any desired environment.
- the system 8 A 00 comprises at least one processor and at least one memory, the memory serving to store program instructions corresponding to the operations of the system.
- an operation can be implemented in whole or in part using program instructions accessible by a module.
- the modules are connected to a communication path 8 A 05 , and any operation can communicate with any other operations over communication path 8 A 05 .
- the modules of the system can, individually or in combination, perform method operations within system 8 A 00 . Any operations performed within system 8 A 00 may be performed in any order unless as may be specified in the claims.
- the shown embodiment implements a portion of a computer system, presented as system 8 A 00 , comprising one or more computer processors to execute a set of program code instructions (module 8 A 10 ) and modules for accessing memory to hold program code instructions to perform: establishing one or more network communication links between a content management system that manages a plurality of shared content objects and a plurality of applications that are registered with the content management system (module 8 A 20 ); recording interaction events that correspond to interactions over one or more of the plurality of shared content objects, the interactions performed at the plurality of applications (module 8 A 30 ); selecting at least two of the interaction events associated with at least two of the plurality of applications, the at least two of the interaction events being selected based at least in part on event attributes associated with the interaction events (module 8 A 40 ); and generating one or more performance measurements, the one or more performance measurements being generated based at least in part on the at least two of the interaction events associated with at least two of the plurality of applications (module 8 A 50 ).
- Variations of the foregoing may include more or fewer of the shown modules. Certain variations may perform more or fewer (or different) steps and/or certain variations may use data elements in more, or in fewer, or in different operations. Still further, some embodiments include variations in the operations performed, and some embodiments include variations of aspects of the data elements used in the operations.
- FIG. 8 B depicts a system 8 B 00 as an arrangement of computing modules that are interconnected so as to operate cooperatively to implement certain of the herein-disclosed embodiments.
- the partitioning of system 8 B 00 is merely illustrative and other partitions are possible.
- the system 8 B 00 may be implemented in the context of the architecture and functionality of the embodiments described herein. Of course, however, the system 8 B 00 or any operation therein may be carried out in any desired environment.
- the system 8 B 00 comprises at least one processor and at least one memory, the memory serving to store program instructions corresponding to the operations of the system. As shown, an operation can be implemented in whole or in part using program instructions accessible by a module.
- the modules are connected to a communication path 8 B 05 , and any operation can communicate with any other operations over communication path 8 B 05 .
- the modules of the system can, individually or in combination, perform method operations within system 8 B 00 . Any operations performed within system 8 B 00 may be performed in any order unless as may be specified in the claims.
- the shown embodiment implements a portion of a computer system, presented as system 8 B 00 , comprising one or more computer processors to execute a set of program code instructions (module 8 B 10 ) and modules for accessing memory to hold program code instructions to perform: establishing one or more network communication links between a content management system that manages a plurality of shared content objects and a plurality of applications that cause modifications to the shared content objects in accordance with workflows of the project (module 8 B 20 ); recording interaction events that correspond to the modifications to the shared content objects (module 8 B 30 ); selecting at least two of the interaction events associated with at least two of the plurality of applications, the at least two of the interaction events being selected based at least in part on event attributes associated with the interaction events (module 8 B 40 ); and generating one or more project performance measurements, the one or more project performance measurements being generated based at least in part on the at least two of the interaction events associated with at least two of the plurality of applications (module 8 B 50 ).
- FIG. 9 A depicts a block diagram of an instance of a computer system 9 A 00 suitable for implementing embodiments of the present disclosure.
- Computer system 9 A 00 includes a bus 906 or other communication mechanism for communicating information.
- the bus interconnects subsystems and devices such as a central processing unit (CPU), or a multi-core CPU (e.g., data processor 907 ), a system memory (e.g., main memory 908 , or an area of random access memory (RAM)), a non-volatile storage device or non-volatile storage area (e.g., read-only memory 909 ), an internal storage device 910 or external storage device 913 (e.g., magnetic or optical), a data interface 933 , a communications interface 914 (e.g., PHY, MAC, Ethernet interface, modem, etc.).
- CPU central processing unit
- a multi-core CPU e.g., data processor 907
- system memory e.g., main memory 908 , or an
- Computer system 9 A 00 further comprises a display 911 (e.g., CRT or LCD), various input devices 912 (e.g., keyboard, cursor control), and an external data repository 931 .
- display 911 e.g., CRT or LCD
- input devices 912 e.g., keyboard, cursor control
- external data repository 931 e.g., external data repository
- computer system 9 A 00 performs specific operations by data processor 907 executing one or more sequences of one or more program instructions contained in a memory.
- Such instructions e.g., program instructions 902 1 , program instructions 902 2 , program instructions 902 3 , etc.
- program instructions 902 1 , program instructions 902 2 , program instructions 902 3 , etc. can be contained in or can be read into a storage location or memory from any computer readable/usable storage medium such as a static storage device or a disk drive.
- the sequences can be organized to be accessed by one or more processing entities configured to execute a single process or configured to execute multiple concurrent processes to perform work.
- a processing entity can be hardware-based (e.g., involving one or more cores) or software-based, and/or can be formed using a combination of hardware and software that implements logic, and/or can carry out computations and/or processing steps using one or more processes and/or one or more tasks and/or one or more threads or any combination thereof.
- computer system 9 A 00 performs specific networking operations using one or more instances of communications interface 914 .
- Instances of communications interface 914 may comprise one or more networking ports that are configurable (e.g., pertaining to speed, protocol, physical layer characteristics, media access characteristics, etc.) and any particular instance of communications interface 914 or port thereto can be configured differently from any other particular instance.
- Portions of a communication protocol can be carried out in whole or in part by any instance of communications interface 914 , and data (e.g., packets, data structures, bit fields, etc.) can be positioned in storage locations within communications interface 914 , or within system memory, and such data can be accessed (e.g., using random access addressing, or using direct memory access DMA, etc.) by devices such as data processor 907 .
- data e.g., packets, data structures, bit fields, etc.
- DMA direct memory access
- Communications link 915 can be configured to transmit (e.g., send, receive, signal, etc.) any types of communications packets (e.g., communication packet 938 1 , communication packet 938 N ) comprising any organization of data items.
- the data items can comprise a payload data area 937 , a destination address 936 (e.g., a destination IP address), a source address 935 (e.g., a source IP address), and can include various encodings or formatting of bit fields to populate packet characteristics 934 .
- the packet characteristics include a version identifier, a packet or payload length, a traffic class, a flow label, etc.
- payload data area 937 comprises a data structure that is encoded and/or formatted to fit into byte or word boundaries of the packet.
- hard-wired circuitry may be used in place of or in combination with software instructions to implement aspects of the disclosure.
- embodiments of the disclosure are not limited to any specific combination of hardware circuitry and/or software.
- the term “logic” shall mean any combination of software or hardware that is used to implement all or part of the disclosure.
- Non-volatile media includes, for example, optical or magnetic disks such as disk drives or tape drives.
- Volatile media includes dynamic memory such as RAM.
- Computer readable media include, for example, floppy disk, flexible disk, hard disk, magnetic tape, or any other magnetic medium; CD-ROM or any other optical medium; punch cards, paper tape, or any other physical medium with patterns of holes; RAM, PROM, EPROM, FLASH-EPROM, or any other memory chip or cartridge, or any other non-transitory computer readable medium.
- Such data can be stored, for example, in any form of external data repository 931 , which in turn can be formatted into any one or more storage areas, and which can comprise parameterized storage 939 accessible by a key (e.g., filename, table name, block address, offset address, etc.).
- Execution of the sequences of instructions to practice certain embodiments of the disclosure are performed by a single instance of a computer system 9 A 00 .
- two or more instances of computer system 9 A 00 coupled by a communications link 915 may perform the sequence of instructions required to practice embodiments of the disclosure using two or more instances of components of computer system 9 A 00 .
- Computer system 9 A 00 may transmit and receive messages such as data and/or instructions organized into a data structure (e.g., communications packets).
- the data structure can include program instructions (e.g., application code 903 ), communicated through communications link 915 and communications interface 914 .
- Received program instructions may be executed by data processor 907 as it is received and/or stored in the shown storage device or in or upon any other non-volatile storage for later execution.
- Computer system 9 A 00 may communicate through a data interface 933 to a database 932 on an external data repository 931 . Data items in a database can be accessed using a primary key (e.g., a relational database primary key).
- a primary key e.g., a relational database primary key
- Processing element partition 901 is merely one sample partition.
- Other partitions can include multiple data processors, and/or multiple communications interfaces, and/or multiple storage devices, etc. within a partition.
- a partition can bound a multi-core processor (e.g., possibly including embedded or co-located memory), or a partition can bound a computing cluster having plurality of computing elements, any of which computing elements are connected directly or indirectly to a communications link.
- a first partition can be configured to communicate to a second partition.
- a particular first partition and particular second partition can be congruent (e.g., in a processing element array) or can be different (e.g., comprising disjoint sets of components).
- a module as used herein can be implemented using any mix of any portions of the system memory and any extent of hard-wired circuitry including hard-wired circuitry embodied as a data processor 907 .
- Some embodiments include one or more special-purpose hardware components (e.g., power control, logic, sensors, transducers, etc.).
- Some embodiments of a module include instructions that are stored in a memory for execution so as to facilitate operational and/or performance characteristics pertaining to measuring the performance of workflows performed over multiple applications.
- a module may include one or more state machines and/or combinational logic used to implement or facilitate the operational and/or performance characteristics pertaining to measuring the performance of workflows performed over multiple applications.
- database 932 comprise storage media organized to hold a series of records or files such that individual records or files are accessed using a name or key (e.g., a primary key or a combination of keys and/or query clauses).
- Such files or records can be organized into one or more data structures (e.g., data structures used to implement or facilitate aspects of measuring the performance of workflows performed over multiple applications).
- Such files, records, or data structures can be brought into and/or stored in volatile or non-volatile memory.
- the occurrence and organization of the foregoing files, records, and data structures improve the way that the computer stores and retrieves data in memory, for example, to improve the way data is accessed when the computer is performing operations pertaining to measuring the performance of workflows performed over multiple applications, and/or for improving the way data is manipulated when performing computerized operations pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions.
- FIG. 9 B depicts a block diagram of an instance of a cloud-based environment 9 B 00 .
- a cloud-based environment supports access to workspaces through the execution of workspace access code (e.g., workspace access code 942 0 , workspace access code 942 1 , and workspace access code 942 2 ).
- Workspace access code can be executed on any of access devices 952 (e.g., laptop device 952 4 , workstation device 952 5 , IP phone device 952 3 , tablet device 952 2 , smart phone device 952 1 , etc.), and can be configured to access any type of object. Strictly as examples, such objects can be folders or directories or can be files of any filetype.
- a group of users can form a collaborator group 958 , and a collaborator group can be composed of any types or roles of users.
- a collaborator group can comprise a user collaborator, an administrator collaborator, a creator collaborator, etc. Any user can use any one or more of the access devices, and such access devices can be operated concurrently to provide multiple concurrent sessions and/or other techniques to access workspaces through the workspace access code.
- a portion of workspace access code can reside in and be executed on any access device. Any portion of the workspace access code can reside in and be executed on any computing platform 951 , including in a middleware setting. As shown, a portion of the workspace access code resides in and can be executed on one or more processing elements (e.g., processing element 905 1 ).
- the workspace access code can interface with storage devices such as networked storage 955 . Storage of workspaces and/or any constituent files or objects, and/or any other code or scripts or data can be stored in any one or more storage partitions (e.g., storage partition 904 1 ).
- a processing element includes forms of storage, such as RAM and/or ROM and/or FLASH, and/or other forms of volatile and non-volatile storage.
- a stored workspace can be populated via an upload (e.g., an upload from an access device to a processing element over an upload network path 957 ).
- a stored workspace can be delivered to a particular user and/or shared with other particular users via a download (e.g., a download from a processing element to an access device over a download network path 959 ).
Landscapes
- Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Organic Chemistry (AREA)
- Genetics & Genomics (AREA)
- Business, Economics & Management (AREA)
- Zoology (AREA)
- General Health & Medical Sciences (AREA)
- Biochemistry (AREA)
- Human Resources & Organizations (AREA)
- Molecular Biology (AREA)
- Medicinal Chemistry (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Wood Science & Technology (AREA)
- Endocrinology (AREA)
- Biophysics (AREA)
- Strategic Management (AREA)
- Biomedical Technology (AREA)
- Proteomics, Peptides & Aminoacids (AREA)
- Entrepreneurship & Innovation (AREA)
- Gastroenterology & Hepatology (AREA)
- Toxicology (AREA)
- General Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Biotechnology (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Microbiology (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Marketing (AREA)
- Theoretical Computer Science (AREA)
- Game Theory and Decision Science (AREA)
- Accounting & Taxation (AREA)
- Tourism & Hospitality (AREA)
- Finance (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
Abstract
Methods, systems and computer program products for shared content management systems that provide performance analytics pertaining to a project. Embodiments include establishing one or more network communication links between a content management system that manages a plurality of shared content objects and a plurality of applications that cause modifications to the shared content objects in accordance with workflows of the project. Iteraction events that correspond to modifications over the shared content objects are recorded such that interaction events associated with the plurality of applications are selected based at least in part on attributes associated with the interaction events. Relationships between the recorded interaction events such as time durations between certain of the interaction events are calculated. Project performance measurements are generated based on the calculations and/or based on other relationships between the interaction events. The calculations may span across many different applications and/or many different departments and/or many different enterprises.
Description
- The present application is a continuation of U.S. application Ser. No. 16/726,081 filed on Dec. 23, 2019, which claims the benefit of priority to U.S. Provisional Patent Application Ser. No. 62/909,116 titled “INTER-APPLICATION WORKFLOW PERFORMANCE ANALYTICS”, filed on Oct. 1, 2019, which are hereby incorporated by reference in their entirety.
- This disclosure relates to shared content management systems, and more particularly to techniques for generating and presenting inter-application workflow performance analytics.
- High-performance computing, storage, and networking capabilities characteristic of today's computing environments have impacted the way personal and corporate computer-readable content objects (e.g., documents, spreadsheets, images, programming code files, etc.) are created, stored, and shared. The capabilities of these environments facilitate interactions (e.g., authoring, editing, viewing, etc.) that are performed over content objects by trusted users (e.g., collaborators) on a variety of user devices such as mobile phones, tablets, laptop computers, desktop computers, and/or other devices. In many cases, interactions over the content objects are organized into workflows. Such workflows often specify an ordered sequence or “flow” of operations to be performed on the content objects, which operations are often invoked by user interactions with the content objects. Interactions with a particular content object may be performed by one user, multiple users, and/or even autonomously by one or more computing entities (e.g., processes, agents, applications, etc.). Moreover, such interactions may span across multiple departments and/or multiple enterprises.
- Modern computing environments facilitate the proliferation and use of numerous applications that are accessed by the users to carry out the foregoing interactions over content objects. These applications (e.g., application-specific tools, point tools, etc.) are often used to improve the efficiency of the interactions performed over the content objects. A particular application may be selected from the hundreds of applications available in a computing environment for a variety of reasons such as: (1) familiarity and/or proficiency of users with the application, (2) popularity and/or functionality of the application as pertains to a particular content object type and/or a particular interaction (e.g., operation) over that content object type, and/or (3) for other reasons.
- As merely one example, several members of the sales department might use a customer relationship management (CRM) tool and a real-time messaging tool to perform various interactions and/or workflows over certain content objects (e.g., a proposal slide deck, an ROI analysis worksheet, etc.) that are associated with a particular sales opportunity. At some point in the sales engagement, a draft contract might be created and sent to outside counsel for their review and possible modification, which interactions might involve the use of a document editing tool, possibly in conjunction with a document revisions tracking tool. When the contract is finalized and signed by the customer, a successful contract execution event will spawn new interactions and/or workflows over new content objects (e.g., onboarding forms, training materials, etc.). Over the project lifecycle of the foregoing sales deal, many users over multiple departments and enterprises may interact with many content objects using numerous applications to carry out the workflows and achieve the objectives of the deal.
- Unfortunately, there is no mechanism for measuring the performance of workflows that involve interactions over content objects that are performed at multiple applications. Consider, for example, the aforementioned sales deal scenario. In this scenario, one or more of the users (e.g., a sales director) may desire to know the time to complete certain workflows (e.g., complete proposal, draft contract, sign contract, etc.) that are performed over the project lifecycle of the sales deal. Another one of the users (e.g., a sales program manager) might desire to know how much time is spent using the applications (e.g., point tools) over the project lifecycle of the sales deal. One approach to measuring such performance metrics is to have the users manually log all of the inter-application interactions performed over the content objects. In highly collaborative environments that span multiple users and enterprises and use many applications, such an approach is a burden to the users that detracts from the collaborative efficiency of the users. What is needed is a way to automatically monitor inter-application interactions with content objects so as to be able to measure various key metrics over the lifecycle of the interactions.
- The present disclosure describes techniques used in systems, methods, and in computer program products for inter-application workflow performance analytics, which techniques advance the relevant technologies to address technological issues with legacy approaches. More specifically, the present disclosure describes techniques used in systems, methods, and in computer program products for measuring the performance of workflows performed over multiple applications that are hosted on multiple heterogeneous computing systems. Certain embodiments are directed to technological solutions for linking content object interactions that are performed across multiple applications so as to generate sales engagement performance measurements of workflows that comprise the interactions.
- The disclosed embodiments modify and improve over legacy approaches. In particular, the herein-disclosed techniques provide technical solutions that address the technical problems attendant to measuring the performance of workflows performed over multiple applications. Such technical solutions involve specific implementations (i.e., data organization, data communication paths, module-to-module interrelationships, etc.) that relate to the software arts for improving computer functionality.
- The ordered combination of steps of the embodiments serve in the context of practical applications that perform steps for linking content object interactions that are performed across multiple applications so as to generate workflow performance measurements. As such, these techniques for linking content object interactions that are performed across multiple applications overcome long standing yet heretofore unsolved technological problems associated with measuring the performance of workflows performed over multiple applications.
- Many of the herein-disclosed embodiments for linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions are technological solutions pertaining to technological problems that arise in the hardware and software arts that underlie deployment and use of workflows in content management systems. Aspects of the present disclosure achieve performance and other improvements in peripheral technical fields including (but not limited to) human-machine interfaces and distributed storage systems.
- Further details of aspects, objectives, and advantages of the technological embodiments are described herein, and in the drawings and claims.
- The drawings described below are for illustration purposes only. The drawings are not intended to limit the scope of the present disclosure.
-
FIG. 1 illustrates a computing environment in which embodiments of the present disclosure can be implemented. -
FIG. 2 is a flowchart depicting an inter-application workflow performance analysis technique as implemented in systems that measure the performance of workflows performed over multiple applications, according to an embodiment. -
FIG. 3 is a block diagram of a system that measures the performance of workflows performed over multiple applications, according to an embodiment. -
FIG. 4 illustrates an inter-application data management technique as used to measure the performance of workflows performed over multiple applications, according to an embodiment. -
FIG. 5 presents an interaction event recording technique as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications, according to an embodiment. -
FIG. 6 presents a workflow performance measurement technique as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications, according to an embodiment. -
FIG. 7 presents a workflow performance analytics presentation technique as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications, according to an embodiment. -
FIG. 8A andFIG. 8B depict system components as arrangements of computing modules that are interconnected so as to implement certain of the herein-disclosed embodiments. -
FIG. 9A andFIG. 9B present block diagrams of computer system architectures having components suitable for implementing embodiments of the present disclosure, and/or for use in the herein-described environments. - Aspects of the present disclosure solve problems associated with using computer systems for measuring the performance of workflows performed over multiple applications. These problems are unique to, and may have been created by, various computer-implemented methods for measuring the performance of workflows performed over multiple applications in the context of content management systems. Some embodiments are directed to approaches for linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. The accompanying figures and discussions herein present example environments, systems, methods, and computer program products for measuring the performance of workflows performed over multiple applications.
- Disclosed herein are techniques for linking interactions associated with content objects performed over multiple applications to generate performance measurements of workflows that comprise the linked interactions. In certain embodiments, the techniques are implemented in a computing environment comprising a content management system that facilitates interactions over a plurality of content objects that are created by, or modified by, or accessed by a plurality of applications. The applications available in the computing environment can include a native application (e.g., browser portal, mobile application, etc.) that is provided by the content management system and is accessed by its users (e.g., managed users) to facilitate certain interactions (e.g., authoring, editing, viewing, etc.) with and/or modifications to the content objects. Such modification to the content objects are modifications that arise from operations performed by an application during the lifecycle of a project. For example, at several moments during the prosecution of a sales engagement, a contract might be signed many times using a document signing application. As another example, at some point during the course of a sales engagement (e.g., after signing the contract), an onboarding form might be populated from a signed contract using a data import application.
- The applications may comprise third-party applications that are available in the overall computing environment. Such third-party applications are applications that are not provided and/or maintained by the provider of the content management system, but rather are applications that are integrated with the content management system to facilitate certain interactions with at least some of the types of content objects managed at the content management system.
- When users interact with content objects using the applications, the corresponding interaction events are recorded at the content management system. Certain specialized data structures are implemented in the content management system to facilitate recording of interaction events that arise at multiple heterogeneous applications. At certain moments in time, the interaction events associated with the applications are processed to select specific subsets of interaction events. For example, a request might be issued from a native application (e.g., at direction of a user) so as to select a set of interaction events that correspond to a particular project (e.g., sales engagement) lifecycle. Performance measurements associated with various workflow performance metrics are generated from the selected interaction events. As an example, the workflow performance metrics might pertain to various aspects of a sales engagement lifecycle. Visual representations of the performance measurements are then formed and presented to one or more of the users. In certain embodiments, a dashboard of workflow performance metrics might be populated with the performance measurements and presented to a user at a user interface. In certain embodiments, workflow performance metrics for a particular project lifecycle might pertain to the entire project lifecycle, or might pertain to particular workflows of the project, or might pertain to particular operations of the workflows, and/or other aspects of the project lifecycle.
- Some of the terms used in this description are defined below for easy reference. The presented terms and their respective definitions are not rigidly restricted to these definitions—a term may be further defined by the term's use within this disclosure. The term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete fashion. As used in this application and the appended claims, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise, or is clear from the context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A, X employs B, or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. As used herein, at least one of A or B means at least one of A, or at least one of B, or at least one of both A and B. In other words, this phrase is disjunctive. The articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or is clear from the context to be directed to a singular form.
- Various embodiments are described herein with reference to the figures. It should be noted that the figures are not necessarily drawn to scale, and that elements of similar structures or functions are sometimes represented by like reference characters throughout the figures. It should also be noted that the figures are only intended to facilitate the description of the disclosed embodiments—they are not representative of an exhaustive treatment of all possible embodiments, and they are not intended to impute any limitation as to the scope of the claims. In addition, an illustrated embodiment need not portray all aspects or advantages of usage in any particular environment.
- An aspect or an advantage described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiments even if not so illustrated. References throughout this specification to “some embodiments” or “other embodiments” refer to a particular feature, structure, material or characteristic described in connection with the embodiments as being included in at least one embodiment. Thus, the appearance of the phrases “in some embodiments” or “in other embodiments” in various places throughout this specification are not necessarily referring to the same embodiment or embodiments. The disclosed embodiments are not intended to be limiting of the claims.
-
FIG. 1 illustrates acomputing environment 100 in which embodiments of the present disclosure can be implemented. As an option, one or more variations ofcomputing environment 100 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. -
FIG. 1 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure presents a logical depiction of how certain of the herein disclosed techniques can be used to generate inter-application performance measurements from interactions performed over multiple applications. A representative scenario and set of high order operations are also presented to illustrate how the herein disclosed techniques are applied incomputing environment 100. - The logical depiction of
FIG. 1 depicts a representative set ofusers 102 who interact with various instances ofcontent objects 106 managed at acontent management system 108.Users 102 may be users (e.g., managed users) ofcontent management system 108 that facilitates interactions (e.g., authoring, editing, viewing, etc.) over content objects 106 for sharing, collaboration, and/or other purposes. In some cases, such interactions are organized into workflows. Interactions and/or workflows over a particular content object may be performed with some human interaction by one or more ofusers 102 and/or with autonomous interaction by one or more computing entities (e.g., processes, agents, applications, etc.). - As can be observed,
users 102 interact withcontent objects 106 and/or other users by accessing a plurality ofapplications 104. As merely one example,applications 104 might comprise instances of a native application provided bycontent management system 108 to facilitate the interactions ofusers 102 withcontent objects 106 and/or interactions with one another. As an example, the native application might be a browser portal or mobile application accessed by a user to manage a set of content objects that the user has authority to access.Applications 104 might also comprise various third-party applications. A third-party application might be a web application, mobile application, or another type of application that is served by a third-party server and accessed by some or all ofusers 102 to perform the aforementioned interactions. In this case, the applications are referred to as “third-party applications” as they are provided (e.g., developed, delivered, served, maintained, etc.) by an entity (e.g., the owner of the third-party server) other than the entity providing thecontent management system 108. - While such third-party applications are applications that are not provided and/or maintained by
content management system 108,applications 104 may still be integrated withcontent management system 108 to facilitate certain interactions with at least some of the types of content objects 106 managed atcontent management system 108. Such integration may include registration ofapplications 104 withcontent management system 108, establishment of APIs to facilitate communication between the applications and system, and/or other integration capabilities. - In many cases,
applications 104 can improve the efficiency of the interactions performed over content objects 106. A particular application may be selected from the hundreds of applications available incomputing environment 100 for a variety of reasons, such as the familiarity and/or proficiency of users with the application, the popularity and/or functionality of the application as pertains to a particular content object type, and/or a particular interaction (e.g., operation) over that content object type, and/or for other reasons. As such, workflows over a particular content object might be performed entirely at one or more of theapplications 104, entirely atcontent management system 108, or at some combination of one or more ofapplications 104 andcontent management system 108. - Referring to the example scenario illustrated in
FIG. 1 , consider that certain users access various applications to interact with a set of content objects so as to carry out constituent stages of aproject lifecycle 150. Specifically,project lifecycle 150 might pertain to a project associated with a particular customer and comprise a “Proposal” stage, a “Contract” stage, an “Onboarding” stage, and a “Support” stage. To achieve the objectives of each stage and the overall project, many users over multiple departments and enterprises interact with many content objects using numerous applications to carry out the various workflows associated with the project. In the shown scenario, the users involved overproject lifecycle 150 include a sales engineer (e.g., user “SE”), an applications engineer (e.g., user “AE”), a program manager (e.g., user “PM”), a contract attorney (e.g., user “CA”), a customer representative (e.g., user “CR”), a service manager (e.g., user “SM”), a service technician (e.g., user “ST”), and/or other users. As merely examples, the foregoing users might access a SalesForce application (e.g., app “SF”), a Slack application (e.g., app “SL”), a Gmail application (e.g., app “GM”), a Microsoft application (e.g., app “MS”), a DocuSign application (e.g., app “DS”), a ServiceNow application (e.g., app “SN”), a ZenDesk application (e.g., app “ZD”), and/or other applications to interact with various content objects (e.g., a “Proposal.pptx” file, a “Contract.docx” file, an “OnBoarding” folder and underlying files, a “Support” folder and underlying files, etc.) overproject lifecycle 150. - In this scenario, one or more of the aforementioned users or another user (e.g., a sales director) may desire to know the time to complete certain workflows (e.g., complete proposal, draft and sign contract, etc.). Moreover, a different user (e.g., a sales VP) may want to analyze end-to-end performance measurements that pertain to the various actions that are carried out during the course of a sales engagement. A still different user or different type of user (e.g., user “PM”) might desire to know how much time is spent using the applications (e.g., point tools) and how such usage relates to achievement of various specific objectives that are accomplished over the duration of the project. However, unfortunately, there are no mechanisms for measuring the performance of workflows that involve interactions over
content objects 106 that are performed over heterogeneous instances ofapplications 104. - The herein disclosed techniques address such challenges pertaining to measuring the performance of workflows performed over multiple applications at least in part by linking interactions associated with
content objects 106 performed over multiple heterogeneous instances ofapplications 104 to generate performance measurements of workflows that comprise the interactions. - In the embodiment of
FIG. 1 , the techniques are facilitated at least in part by aninter-application analytics engine 110 implemented atcontent management system 108. An inter-applicationdata management framework 112 is also established atcontent management system 108 to facilitate management of any data pertaining to interactions performed over content objects at multiple heterogeneous applications (operation 1). More specifically, inter-applicationdata management framework 112 comprises various specialized data structures that, at least in part, link the interactions performed over content objects at multiple heterogeneous applications. As used herein, an inter-application data management framework is any operational element that receives and stores information from a plurality of applications so as to facilitate subsequent querying and retrievals by another operational element. - As indicated in the figure, at least some of the data structures of inter-application
data management framework 112 are used to organize and store application-specific information 132 associated with applications 104 (e.g., application attributes, workflow states, task states, etc.) and/or enterprise-specific information 134 (e.g., workflow definitions, interaction definitions, metric definitions, etc.) associated withusers 102. Whenusers 102 interact withcontent objects 106 using any of the applications 104 (operation 2), certain event attributes that correspond to respective instances ofinteraction events 122 are recorded byinter-application analytics engine 110 at content management system 108 (operation 3). Such event attributes may comprise attributes that describe the interaction and other attributes that describe certain entities (e.g., users, content objects, workflows, etc.) associated with the interaction events. According to the herein disclosed techniques, the event attributes might also comprise one or more link attributes that facilitate the aforementioned links between the applications. - The foregoing event attributes associated with inter-application interactions are analyzed by
inter-application analytics engine 110 to generate one or more inter-application performance measurements 124 (operation 4). As can be observed, data organized and stored in accordance with inter-applicationdata management framework 112 is often accessed byinter-application analytics engine 110 to generate one or more of theinter-application performance measurements 124. For example, certain enterprise-specific metric definitions organized according to the inter-applicationdata management framework 112 might be queried byinter-application analytics engine 110 to retrieve a performance measurement for a particular metric. - The inter-application performance measurements determined by the herein disclosed techniques are then presented to one or more of users 102 (operation 5). As an example, the set of
inter-application performance measurements 124 may be presented to a user at the user interface of a native application. In one example, as shown, an inter-applicationperformance dashboard view 152 presented at the user interface graphically depicts performance measurements associated with “ProjectX”, which performance measurements include a measurement associated with a first metric “Metric1”, a measurement associated with a second metric “Metric2”, as well as additional performance measurements associated with applications “SF”, “MS”, and “GM”, which are shown as “AppSF”, “AppMS”, and “AppGM”, respectively. - The automatic inter-application performance analytics capability facilitated by the herein disclosed techniques serves to address the problems attendant to measuring the performance of workflows performed over multiple applications. As such, application of the techniques disclosed herein facilitate improvements in computer functionality that serve to reduce the demand for computer memory, reduce the demand for computer processing power, reduce network bandwidth use, and reduce the demand for intercomponent communication. Specifically, consumption of computing resources that might be needed to at least facilitate manual recording and aggregation of interactions performed over a large number of users, content objects, and applications is eliminated.
- One embodiment of techniques for such automatic inter-application performance analytics is disclosed in further detail as follows.
-
FIG. 2 is a flowchart depicting an inter-application workflowperformance analysis technique 200 as implemented in systems that measure the performance of workflows performed over multiple applications. As an option, one or more variations of inter-application workflowperformance analysis technique 200 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. The inter-application workflowperformance analysis technique 200 or any aspect thereof may be implemented in any environment. -
FIG. 2 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure is presented to illustrate one embodiment of certain steps and/or operations performed over a network of devices (e.g., user devices, computing systems, etc.) to record and analyze interactions over content objects performed at multiple heterogeneous applications to generate inter-application performance measurements. As can be observed, the steps and/or operations can be grouped into a set ofsetup operations 210 and a set ofongoing operations 220. -
Setup operations 210 of inter-application workflowperformance analysis technique 200 commences by identifying or configuring a content management system that facilitates interactions over a plurality of users and a plurality of content objects (step 212). Such interactions can involve both user-to-user interactions and user-to-content interactions. A plurality of applications (e.g., apps) are integrated with the content management system to facilitate interactions over the users and/or content objects performed by the apps (step 214). As an example, a sales contract document managed by the content management system might be shared using a first application (e.g., SalesForce) to facilitate the development of the contract, after which development the contract might be submitted to a second application (e.g., DocuSign) to facilitate execution (e.g., signing) of the contract. In this case, the SalesForce and DocuSign applications might be registered with the content management system to facilitate authorized access to the sales contract document managed (e.g., stored, updated, etc.) at the content management system. - An inter-application data management framework is established at the content management system to interrelate certain sets of interactions performed over the applications (step 216). Frameworks, such as the inter-application data management framework or any other framework discussed herein, are sets of computing entities that cooperatively facilitate various data collection and data distribution functions. The computing entities of a framework might comprise various data structures, application programming interfaces (APIs), communication protocols, authentication protocols, and/or other entities. At least some of the entities of a framework are often strictly designed to facilitate a certain function or certain set of functions. For example, an embodiment of the inter-application data management framework described herein might specify that one or more link attributes are to be included in API calls and/or certain data records to facilitate the aforementioned linking between applications. Other entities of a framework might be designed to offer flexibility. For example, an embodiment of the inter-application data management framework described herein might comprise specialized data structures that facilitate enterprise-specific definitions of workflows, interactions, or metrics.
- As depicted in
ongoing operations 220, any interaction events corresponding to interactions over the content objects performed by the applications are recorded (step 222). Referring to the aforementioned example, certain APIs and/or data structures of the inter-application data management framework might be used to record event attributes associated with interaction events invoked at the SalesForce and DocuSign applications. At certain moments in time, sets of interaction events are selected, which selected sets comprise events from two or more applications (step 224). For example, a set of interaction events associated with one or more link attributes (e.g., project name, customer name, workflow name, document name, etc.) might be selected in response to a request for performance measurements associated with those link attributes. One or more inter-application performance measurements pertaining to the sets of interaction events are generated (step 226). As an example, inter-application performance measurements that pertain to interaction events from respective workflows are generated. - Various event attributes and/or other attributes (e.g., metric definition attributes, etc.) associated with the sets of interaction events might be analyzed to generate the inter-application performance measurements. The inter-application performance measurements are then presented to one or more users (step 228). For example, at least some of the measurements might be visually presented in text and/or graphics at user interfaces accessible by the users.
- One embodiment of a system, data flows, and data structures for implementing the inter-application workflow
performance analysis technique 200 and/or other herein disclosed techniques is disclosed as follows. -
FIG. 3 is a block diagram of asystem 300 that measures the performance of workflows performed over multiple applications. As an option, one or more variations ofsystem 300 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. Thesystem 300 or any aspect thereof may be implemented in any environment. -
FIG. 3 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure is being presented to show one embodiment of certain representative components and associated data structures and data flows implemented in a computing environment to facilitate the herein disclosed techniques. As shown, the components, data flows, and data structures are associated with a set of users (e.g.,user 102 1, . . . , user 102 N) that interact with each other and a set of content objects 106 managed at acontent management system 108. The components, data flows, and data structures shown inFIG. 3 present one partitioning and associated data manipulation approach. The specific example shown is purely exemplary, and other subsystems, data structures, and/or partitionings are reasonable. - As shown,
system 300 comprises an instance ofcontent management server 310 operating atcontent management system 108.Content management server 310 comprises amessage processor 312 and an instance of aninter-application analytics engine 110, which comprises aselection service 314, anevent analyzer 316, and adisplay generator 318. A plurality of instances of the foregoing components might operate at a plurality of instances of servers (e.g., content management server 310) atcontent management system 108 and/or any portion ofsystem 300. Such instances can interact with acommunications layer 320 to access each other and/or a set ofstorage devices 330 that store various information to support the operation of the components ofsystem 300 and/or any implementations of the herein disclosed techniques. - For example, the servers and/or storage devices of
content management system 108 might facilitate interactions overcontent objects 106 by the users (e.g.,user 102 1, . . . , user 102 N) from a respective set of user devices (e.g., user device 302 1, . . . , user device 302 N). A content management system “manages” a plurality of content objects at least in part by maintaining (e.g., storing, updating, resolving interaction conflicts, etc.) the content objects subject to the various interactions performed over the content objects by users of the content objects at their respective user devices. The content objects (e.g., files, folders, etc.) in content objects 106 are characterized at least in part by a set of object attributes 340 (e.g., content object metadata) stored atstorage devices 330. Furthermore, the users are characterized at least in part by a set of user attributes 342 stored in a set ofuser profiles 332 atstorage devices 330. - The users access instances of
applications 104 at their respective user devices to interact withcontent objects 106 managed bycontent management system 108. As shown, the applications can comprise instances of native applications (e.g., native application 304 1, . . . , native application 304 N) or instances of third-party applications (e.g., a SalesForce or “SF” application, a DocuSign or “DS” application, etc.). Various information pertaining to integrations of such third-party applications withcontent management system 108 are normalized and then codified for storage in anapp registry 336, which in turn is maintained as data instorage devices 330. At least some information ofapp registry 336 is organized according to an inter-application data management framework as earlier described. In some cases, certain portions of the information inapp registry 336 might be locally accessible at the user devices by the third-party applications. For example, a first local app registry might be accessible by app “SF” at user device 302 1 and a second local app registry might be accessible by app “DS” at user device 302 N. - The instances of the applications operating at the user devices send or receive various instances of
messages 322 that are received or sent bymessage processor 312 atcontent management server 310. In some cases,messages 322 are sent to or received fromcontent management server 310 without human interaction. One class ofmessages 322 corresponds to application-specific information received atcontent management system 108 in response to executing application integration operations. For example, instances of application attributes 346 that correspond to a particular application might be issued by an enterprise and stored inapp registry 336 when the application is registered withcontent management system 108. Select users (e.g., system administrators) at enterprises may also submit certain policy information and/or other enterprise-specific information in instances ofmessages 322. Such enterprise-specific policy information is stored in a set ofenterprise policies 338 atstorage devices 330. As shown,enterprise policies 338 might comprise instances ofworkflow definitions 347,interaction definitions 348,metric definitions 349, and/or other information that can be accessed to facilitate the herein disclosed techniques. As used herein, an enterprise-specific policy is any information that identifies and/or governs the use of workflows, and/or information that identifies or characterized tasks of the workflows, and/or information that pertains to metrics to be considered when generating performance measurements. - Another class of
messages 322 corresponds to interaction events that are invoked by the users when they interact with one another (e.g., user-to-user interactions) and/or when users interact with various content objects (e.g., user-to-content interactions). As described herein, the interactions can be performed at any of the instances of applications 104 (e.g., native applications, third-party applications, etc.). As examples, the users might log in to their respective instances of a native application or any third-party applications to interact with content objects that they own or that are shared with them to invite other users to collaborate on content objects and/or to perform other collaboration activities. Any of the foregoing interactions or collaboration activities can be characterized by one or more interaction events. - The
message processor 312 atcontent management server 310 monitors themessages 322 to detect interaction event messages that correspond to interaction events performed over the plurality of content objects at the plurality of applications.Message processor 312 codifies certain event attributes 344 pertaining to the interaction events in a set ofevent records 334 stored instorage devices 330. In some cases,message processor 312 will retrieve other attributes (e.g., user attributes 342, object attributes 340, application attributes 346, interaction definition attributes, etc.) to facilitate populating the event records 334. - At certain moments in time, instances of
messages 322 comprising measurement requests are received atmessage processor 312. Such requests are issued to retrieve one or more inter-application performance measurements fromcontent management system 108. For example,user 102 1 might load a performance dashboard view in native application 304 1 that invokes one or more measurement requests to facilitate population of the dashboard view. In response to receiving measurement requests,inter-application analytics engine 110 accesses theselection service 314 to select respective sets of selectedinteraction events 326 fromevent records 334 in accordance with request attributes associated with the requests. As an example, consider a measurement request that includes a link attribute that identifies a particular customer. In this case, the customer identifier and/or other attributes might be included in a query ofevent records 334 to select a set of interaction events that are associated with the customer. - The
event analyzer 316 is then called to analyze the event attributes that constitute the selectedinteraction events 326 to generate one or more instances ofinter-application performance measurements 124. Theinter-application performance measurements 124 generated byevent analyzer 316 will often be based at least in part on other data available atcontent management system 108, such as certain data stored inenterprise policies 338 and/orapp registry 336. - The
inter-application performance measurements 124 are accessed by thedisplay generator 318 to prepare the measurements for presentation. Specifically,display generator 318 generates the user interface display elements that comprise an active, visual presentation of theinter-application performance measurements 124 and/or other information. Such user interface display elements might comprise data tables, charts, graphs, and/or other human-readable elements. The user interface display elements and other information associated with theinter-application performance measurements 124 are codified into instances ofmessages 322 which are in turn delivered to the users. As an example, instances of user interface display elements that describe various inter-application performance measurements are dynamically generated and presented to respective users in their corresponding instances of a native application. - A detailed embodiment of the inter-application data management framework described herein is disclosed as follows.
-
FIG. 4 illustrates an inter-applicationdata management technique 400 as used to measure the performance of workflows performed over multiple applications. As an option, one or more variations of inter-applicationdata management technique 400 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. The inter-applicationdata management technique 400 or any aspect thereof may be implemented in any environment. -
FIG. 4 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure presents certain specialized data structures for organizing and/or storing various data to facilitate the aforementioned inter-application linking and other techniques disclosed herein. As shown, the data structures are associated with the inter-applicationdata management framework 112 earlier described. - These data structures and other data structures discussed herein are configured to improve the way a computer stores and retrieves certain data in memory when performing the herein disclosed techniques. Data can be organized and/or stored in accordance with these data structures using various techniques. For example, the representative data structures associated with inter-application
data management framework 112 shown inFIG. 4 indicate that the constituent data of the data structures might be organized and/or stored in a tabular structure (e.g., relational database table) that has rows that relate various attributes with a particular data entity. As another example, the underlying data might be organized and/or stored in a programming code object that has instances corresponding to a particular data entity and properties corresponding to the various attributes associated with the data entity. A representative set of selectdata structure relationships 420 between certain data entities are also shown. - When instances of applications 104 (e.g., third-party applications) are integrated (e.g., registered) with a
content management system 108, respective sets of application-specific information are populated in anapp registry 336. Moreover, instances ofenterprises 404 that have access tocontent management system 108 populate respective instances ofenterprise policies 338 with enterprise-specific information. The data structures ofapp registry 336 andenterprise policies 338 are defined by the inter-applicationdata management framework 112 to facilitate the herein disclosed techniques. - As shown, the system supports a plurality of
applications 104 as well as a plurality ofenterprises 404. As such, data that is stored in theapp registry 336 and/or data that is stored in enterprise policies may be normalized so as to facilitate comparisons and other processing in spite of a plurality of origins and formats of incoming data. For example, a first application of theapplications 104 might refer to its “application ID” as a number, whereas a second application of theapplications 104 might refer to its “application ID” as a character string. A normalizer (e.g., 405 1) might recode these different representations into a common representation. As another example, a first app might refer to an event raised at or by particular enterprise using an app-specific ID (e.g., “entID” of the select event attributes 428) whereas a second app might refer to a different event raised at or by the same particular enterprise using a different app-specific ID (e.g., “entID” of the select event attributes 428). In this and other cases, a correspondence between a first occurrence of an “entID” and a second of an “entID” is maintained. As such, when “entID”s or other metadata entries that are different as between apps, but the different “entID”s or other metadata nevertheless refer to the same attribute the occurrence of events can be normalized or otherwise mapped so as to reflect that different events processed by different apps refer to the same attribute. In some cases, the normalization processing of normalizer 405 1 creates and manages a plurality of maps, each of which maps relate incoming data (e.g., from the applications 104) to an internal representation. For example, in a first map or map entry, the attribute value “Enterprise1” might be mapped to “CustomerA”, and in a second map or map entry, the attribute value “Enterprise2” might be also mapped to “CustomerA”. As such, variations that may occur in the representation of any attribute in any particular app can be normalized into a comment representation. - Similarly, data that is stored in the
enterprise policies 338 may be normalized so as to facilitate comparisons and other processing in spite of a plurality of origins and formats of incoming data. For example, a first enterprise of theenterprises 404 might refer to its “workflow ID” as a number, whereas a second enterprise of theenterprises 404 might refer to its “workflow ID” as a character string. A normalizer (e.g., 405 2) might recode these different representations into a common representation. In some cases, the normalization processing of normalizer 405 2 creates and manages a plurality of maps, each of which maps relate incoming data (e.g., from the enterprises 404) to an internal representation which is in turned used by operational elements of the system. - Further details regarding general approaches to handling content object metadata and/or other types of metadata are described in U.S. application Ser. No. 16/553,144 titled “EXTENSIBLE CONTENT OBJECT METADATA”, filed on Aug. 27, 2019, which is hereby incorporated by reference in its entirety.
- The
content management system 108 also stores instances ofevent records 334 in accordance with the data structures of the inter-applicationdata management framework 112. Data stored in the data structures of the inter-applicationdata management framework 112 may be normalized so as to facilitate comparisons. For example, one data item might refer to units of time or durations as a number of days, whereas another data item might refer units of time or durations as a number of hours. For comparison purposes, days can be normalized to a number and/or fraction of hours, or, for comparison purposes, hours can be normalized to a number and/or fraction of days. - In some cases, certain portions of the aforementioned data structures and/or data stores are populated in response to various inputs (e.g., selections, entered text, etc.) received from system administrators and/or application developers in response to interactions with various user interfaces (e.g., admin and/or developer console). For example, an application developer might first register a set of applications, then a system administrator might define certain interactions that are performed at the applications. The system administrator might also define various workflows and/or workflow performance metrics that pertain to combinations of the foregoing interactions.
- Additionally or alternatively, various techniques may be employed to automatically determine various workflows that might pertain to combinations of the foregoing interactions.
- Further details regarding general approaches to automatically determining workflows are described in U.S. application Ser. No. 16/553,161 titled “WORKFLOW SELECTION”, filed on Aug. 27, 2019, which is hereby incorporated by reference in its entirety.
- As indicated in a set of select application attributes 422, each of the applications that are registered with the content management system is identified by an application identifier (e.g., stored in an “appID” field), an application name (e.g., stored in an “appName” field), an enterprise identifier (e.g., stored in an “entID” field), an endpoint URL (e.g., stored in an “endpoint” field), a set of OAuth2 credentials (e.g., stored in an “OAuth2[ ]” object), and/or other attributes. As can be observed, the application identifier or “appID” is referenced by other data structures to associate the data underlying those structures with a particular application. Certain attributes (e.g., “entID”, “endpoint”, etc.) from select application attributes 422 might be included in interaction event messages issued from the applications to facilitate identification of the particular instances of the application or applications that issued the messages.
- Such interaction event messages may also identify one or more interactions as defined by a set of select interaction definition attributes 426. As can be observed, an interaction associated with an application identified in an “appID” field is defined by an interaction identifier (e.g., stored in an “actionID” field), an interaction type description (e.g., stored in an “action” field), a set of interaction parameters specific to the interaction (e.g., stored in an “iParams[ ]” object), and/or other attributes. In some embodiments, an interaction identifier (e.g., “SF.create”) might be constructed from the application identifier (e.g., “SF”) and the interaction type description (e.g., “create”).
- Various workflows are also defined in the inter-application
data management framework 112 in accordance with a data structure described by a set of select workflow definition attributes 424. Specifically, a particular workflow is defined by a workflow identifier (e.g., stored in a “workflowID” field), a workflow name (e.g., stored in a “name” field), a workflow description (e.g., stored in a “description” field), a set of tasks associated with the workflow (e.g., stored in a “tasks[ ]” object), a set of metrics associated with the workflow (e.g., stored in a “metrics[ ]” object), and/or other attributes. As shown, each task of the workflow is described by an interaction identifier (e.g., stored in an “actionID” field), a task sequence index (e.g., stored in an “index” field), a task state description (e.g., stored in a “state” field), and/or other attributes. Also as shown each metric associated with the workflow is described by a metric identifier (e.g., stored in a “metricID” field), an interaction identifier that signals the start event of the metric (e.g., stored in an “actionID_s” field), an interaction identifier that signals the end event of the metric (e.g., stored in an “actionID_e” field), and/or other attributes. - As indicated in a set of select event attributes 428, each event record stored in
event records 334 comprises an object identifier (e.g., stored in an “objID” field), an application identifier (e.g., stored in an “appID” field), an interaction type description (e.g., stored in an “action” field), an enterprise identifier (e.g., stored in an “entID” field), a link identifier (e.g., stored in a “linkID” field), a link name (e.g., stored in a “linkName” field), a timestamp (e.g., stored in a “time” field), one or more interaction parameters specific to the interaction event (e.g., stored in an “iParams[ ]” object), and/or other attributes. As can be observed, the link identifier and/or the link name and/or other attributes can constitute a set of link attributes 430 that facilitate the linking of interactions over content objects performed over multiple heterogeneous applications. - The foregoing discussion includes techniques for recording interaction events associated with interactions over content objects performed at multiple heterogeneous applications (e.g., step 222 of
FIG. 2 ), which techniques are disclosed in further detail as follows. -
FIG. 5 presents an interactionevent recording technique 500 as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications. As an option, one or more variations of interactionevent recording technique 500 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. The interactionevent recording technique 500 or any aspect thereof may be implemented in any environment. -
FIG. 5 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure is presented to illustrate one embodiment of certain steps and/or operations that facilitate recording interaction events performed over content objects at various heterogeneous applications. As depicted in the figure, the steps and/or operations are associated withstep 222 ofFIG. 2 . A representative scenario is also shown in the figure to illustrate an example application of interactionevent recording technique 500. - The interaction
event recording technique 500 commences by monitoring a plurality of applications for interaction events (step 502). As illustrated, an instance ofmessage processor 312 may continuously listen or poll for interaction events performed at a plurality of applications that includes an instance of app “SF” accessed by user “SE” and an instance of app “DS” accessed by user “CR”. When interaction event messages are received (step 504), the interaction event messages are parsed to retrieve respective sets of interaction attributes from the messages (step 506). As shown,message processor 312 receives an interaction event message 522 1 in response to user “SE” interacting with file “F3” at app “SF” and receives an interaction event message 522 K in response to user “CR” interacting with file “F3” at app “DS”. As indicated by a set of select interaction attributes 524, the interaction attributes associated with the interaction event messages include an application identifier (e.g., stored in an “appID” field), an interaction type description (e.g., stored in an “action” field), a timestamp (e.g., stored in a “time” field), a user identifier (e.g., stored in a “userID” field), an enterprise identifier (e.g., stored in an “entID” field), a link identifier (e.g., stored in a “linkID” field), a content object identifier (e.g., stored in an “objID” field), and/or other attributes. - If other attributes are to be considered (“Yes” path of decision 508), then various other attributes associated with the interaction event messages are retrieved (step 510). In this case,
message processor 312 might access the datastores of content objects 106, user profiles 332,app registry 336,enterprise policies 338, and/or other data sources to retrieve certain attributes associated with the interaction attributes of the interaction event messages. - All retrieved attributes are then recorded as event attributes associated with the interaction event message (step 512). As stated, if other attributes are to be considered (“Yes” path of decision 508), the event attributes comprise some or all of the retrieved interaction attributes and the retrieved other attributes. If merely the interaction attributes are considered (“No” path of decision 508), the event attributes comprise some or all of the retrieved interaction attributes. In the shown scenario,
message processor 312 stores inevent records 334 sets of event attributes that correspond to a respective set ofsubject interaction events 544 described by interaction event message 522 1 and interaction event message 522 K. - The foregoing discussion includes techniques for selecting and analyzing such sets of interaction events associated with two or more applications to generate inter-application performance measurements (e.g.,
step 224 and step 226 ofFIG. 2 ), which techniques are disclosed in further detail as follows. -
FIG. 6 presents a workflowperformance measurement technique 600 as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications. As an option, one or more variations of workflowperformance measurement technique 600 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. The workflowperformance measurement technique 600 or any aspect thereof may be implemented in any environment. -
FIG. 6 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figure is presented to illustrate one embodiment of certain steps and/or operations that facilitate selecting and analyzing sets of interaction events associated with two or more applications to generate inter-application performance measurements. As depicted in the figure, the steps and/or operations are associated withstep 224 and step 226 ofFIG. 2 . A representative scenario is also shown in the figure to illustrate an example application of workflowperformance measurement technique 600. - The workflow
performance measurement technique 600 commences by monitoring a plurality of applications for measurement requests (step 602). As illustrated, an instance ofmessage processor 312 may continuously listen or poll for measurement requests issued from applications to retrieve various inter-application workflow performance measurements. When a measurement request is received (step 604), the request is parsed to retrieve its request attributes (step 606). For example,message processor 312 might receive ameasurement request 622 issued from native application 304 1 by user 102 1 (e.g., a system administrator or “SA”). As indicated by a set of select request attributes 624, the request attributes associated withmeasurement request 622 or any measurement request include a user identifier (e.g., stored in a “userID” field), an enterprise identifier (e.g., stored in an “entID” field), a content object identifier (e.g., stored in an “objID” field), a workflow identifier (e.g., stored in a “workflowID” field), a metric identifier (e.g., stored in a “metricID” field), a link identifier (e.g., stored in a “linkID” field), and/or other attributes. - A set of interaction events are selected based at least in part on the request attributes (step 608). As an example,
message processor 312 forwards the request attributes extracted frommeasurement request 622 to an instance of aselection service 314. Theselection service 314 uses one or more of the request attributes (e.g., the link attribute) to query the event records 334 to select a set of subject selectedinteraction events 626 that pertain to the measurement request. - To facilitate processing of the selected interaction events, any enterprise policy information associated with the measurement request are accessed (step 612). As shown, an instance of an
event analyzer 316 might use certain request attributes (e.g., enterprise identifier, workflow identifier, metric identifier, etc.) to retrieve any relevant enterprise policy information from a store ofenterprise policies 338. The enterprise policy information is applied to the event attributes associated with the selected interaction events to determine one or more inter-application performance measurements (step 614). In some cases, the enterprise policy information includes content object access information. For example, enterprise policy information might allow or deny access to a content object by a particular user. Furthermore, such enterprise policy information might allow or deny access to any actions taken over a particular content object by a particular user. As such it can happen that a particular user many not be able to see measurements that derive from events that are not accessible (e.g., access denied) to that particular user. - In the illustrated scenario,
event analyzer 316 applies the enterprise policy information retrieved fromenterprise policies 338 to the event attributes of subject selectedinteraction events 626 to generate a particular inter-application performance measurement. In the specific example shown,event analyzer 316 generates a performance measurement for metric “Metric1” from interaction events performed over two or more applications (e.g., app “SF” and app “DS”) that are linked by a “ProjX” link identifier. As shown, the events are linked by the “ProjX” link identifier that appears in multiple timestamped instances of the select subject event attributes 632. Still further, the action “create” and the action “sign” are interrelated by the data structure shown as select subject metric attributes 634. - Considering that the “Metric1” start event is indicated by a “create” action at app “SF” and that the “Metric1” end event is indicated by a “sign” action at app “DS”, a measurement for “Metric1” might be equal to 10 (e.g., “t123”−“t113”) time units (e.g., days).
- As can be understood from the foregoing discussion, the metrics are not generic hardware-centric metrics such as CPU utilization or memory usage, but rather are metrics that pertain to the usage of a particular application and/or that pertain to operations that are performed cooperatively between two or more heterogeneous applications.
- The foregoing discussions include techniques for presenting the inter-application performance measurements to various users (e.g., step 228 of
FIG. 2 ), which techniques are disclosed in further detail as follows. -
FIG. 7 presents a workflow performanceanalytics presentation technique 700 as implemented in systems that facilitate measuring the performance of workflows performed over multiple applications. As an option, one or more variations of workflow performanceanalytics presentation technique 700 or any aspect thereof may be implemented in the context of the architecture and functionality of the embodiments described herein. The workflow performanceanalytics presentation technique 700 or any aspect thereof may be implemented in any environment. -
FIG. 7 illustrates aspects pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. Specifically, the figures are presented to illustrate one embodiment of certain steps and/or operations that facilitate presenting inter-application performance measurements to a user. As depicted in the figure, the steps and/or operations are associated withstep 228 ofFIG. 2 . A representative scenario is also shown in the figures to illustrate an example application of workflow performanceanalytics presentation technique 700. - The workflow performance
analytics presentation technique 700 commences by retrieving a set of inter-application performance measurements to present to a user (step 702). As illustrated, an instance ofdisplay generator 318 might retrieve a set ofinter-application performance measurements 124. A set of measurement display components associated with the inter-application performance measurements is accessed (step 704). As shown,display generator 318 accesses a set ofmeasurement display components 722 to identify certain components (e.g., chart types, style sheets, etc.) associated withinter-application performance measurements 124. For example, horizontal bar charts might be identified as the chart type for the measurements. - Certain user interface display elements are constructed from the display components and the inter-application performance measurements (step 706). As an example,
display generator 318 might access a user interface library (e.g., React library) to generate the code for rendering the display components ofinter-application performance measurements 124 and other related information at a target user interface. The user interface display elements are then presented to the user at the target user interface (step 708). As illustrated, the user interface display elements associated withinter-application performance measurements 124 are presented in an inter-applicationperformance dashboard view 152 at native application 304 1 associated withuser 102 1. Specifically shown in inter-applicationperformance dashboard view 152 is a graphical visualization of a specific inter-application performance measurement value 724 that corresponds to “Metric1”. -
FIG. 8A depicts a system 8A00 as an arrangement of computing modules that are interconnected so as to operate cooperatively to implement certain of the herein-disclosed embodiments. This and other embodiments present particular arrangements of elements that, individually or as combined, serve to form improved technological processes that address measuring the performance of workflows performed over multiple applications. The partitioning of system 8A00 is merely illustrative and other partitions are possible. As an option, the system 8A00 may be implemented in the context of the architecture and functionality of the embodiments described herein. Of course, however, the system 8A00 or any operation therein may be carried out in any desired environment. The system 8A00 comprises at least one processor and at least one memory, the memory serving to store program instructions corresponding to the operations of the system. As shown, an operation can be implemented in whole or in part using program instructions accessible by a module. The modules are connected to a communication path 8A05, and any operation can communicate with any other operations over communication path 8A05. The modules of the system can, individually or in combination, perform method operations within system 8A00. Any operations performed within system 8A00 may be performed in any order unless as may be specified in the claims. The shown embodiment implements a portion of a computer system, presented as system 8A00, comprising one or more computer processors to execute a set of program code instructions (module 8A10) and modules for accessing memory to hold program code instructions to perform: establishing one or more network communication links between a content management system that manages a plurality of shared content objects and a plurality of applications that are registered with the content management system (module 8A20); recording interaction events that correspond to interactions over one or more of the plurality of shared content objects, the interactions performed at the plurality of applications (module 8A30); selecting at least two of the interaction events associated with at least two of the plurality of applications, the at least two of the interaction events being selected based at least in part on event attributes associated with the interaction events (module 8A40); and generating one or more performance measurements, the one or more performance measurements being generated based at least in part on the at least two of the interaction events associated with at least two of the plurality of applications (module 8A50). - Variations of the foregoing may include more or fewer of the shown modules. Certain variations may perform more or fewer (or different) steps and/or certain variations may use data elements in more, or in fewer, or in different operations. Still further, some embodiments include variations in the operations performed, and some embodiments include variations of aspects of the data elements used in the operations.
-
FIG. 8B depicts a system 8B00 as an arrangement of computing modules that are interconnected so as to operate cooperatively to implement certain of the herein-disclosed embodiments. The partitioning of system 8B00 is merely illustrative and other partitions are possible. As an option, the system 8B00 may be implemented in the context of the architecture and functionality of the embodiments described herein. Of course, however, the system 8B00 or any operation therein may be carried out in any desired environment. The system 8B00 comprises at least one processor and at least one memory, the memory serving to store program instructions corresponding to the operations of the system. As shown, an operation can be implemented in whole or in part using program instructions accessible by a module. The modules are connected to a communication path 8B05, and any operation can communicate with any other operations over communication path 8B05. The modules of the system can, individually or in combination, perform method operations within system 8B00. Any operations performed within system 8B00 may be performed in any order unless as may be specified in the claims. The shown embodiment implements a portion of a computer system, presented as system 8B00, comprising one or more computer processors to execute a set of program code instructions (module 8B10) and modules for accessing memory to hold program code instructions to perform: establishing one or more network communication links between a content management system that manages a plurality of shared content objects and a plurality of applications that cause modifications to the shared content objects in accordance with workflows of the project (module 8B20); recording interaction events that correspond to the modifications to the shared content objects (module 8B30); selecting at least two of the interaction events associated with at least two of the plurality of applications, the at least two of the interaction events being selected based at least in part on event attributes associated with the interaction events (module 8B40); and generating one or more project performance measurements, the one or more project performance measurements being generated based at least in part on the at least two of the interaction events associated with at least two of the plurality of applications (module 8B50). -
FIG. 9A depicts a block diagram of an instance of a computer system 9A00 suitable for implementing embodiments of the present disclosure. Computer system 9A00 includes a bus 906 or other communication mechanism for communicating information. The bus interconnects subsystems and devices such as a central processing unit (CPU), or a multi-core CPU (e.g., data processor 907), a system memory (e.g.,main memory 908, or an area of random access memory (RAM)), a non-volatile storage device or non-volatile storage area (e.g., read-only memory 909), aninternal storage device 910 or external storage device 913 (e.g., magnetic or optical), adata interface 933, a communications interface 914 (e.g., PHY, MAC, Ethernet interface, modem, etc.). The aforementioned components are shown withinprocessing element partition 901, however other partitions are possible. Computer system 9A00 further comprises a display 911 (e.g., CRT or LCD), various input devices 912 (e.g., keyboard, cursor control), and anexternal data repository 931. - According to an embodiment of the disclosure, computer system 9A00 performs specific operations by
data processor 907 executing one or more sequences of one or more program instructions contained in a memory. Such instructions (e.g., program instructions 902 1, program instructions 902 2, program instructions 902 3, etc.) can be contained in or can be read into a storage location or memory from any computer readable/usable storage medium such as a static storage device or a disk drive. The sequences can be organized to be accessed by one or more processing entities configured to execute a single process or configured to execute multiple concurrent processes to perform work. A processing entity can be hardware-based (e.g., involving one or more cores) or software-based, and/or can be formed using a combination of hardware and software that implements logic, and/or can carry out computations and/or processing steps using one or more processes and/or one or more tasks and/or one or more threads or any combination thereof. - According to an embodiment of the disclosure, computer system 9A00 performs specific networking operations using one or more instances of
communications interface 914. Instances ofcommunications interface 914 may comprise one or more networking ports that are configurable (e.g., pertaining to speed, protocol, physical layer characteristics, media access characteristics, etc.) and any particular instance ofcommunications interface 914 or port thereto can be configured differently from any other particular instance. Portions of a communication protocol can be carried out in whole or in part by any instance ofcommunications interface 914, and data (e.g., packets, data structures, bit fields, etc.) can be positioned in storage locations withincommunications interface 914, or within system memory, and such data can be accessed (e.g., using random access addressing, or using direct memory access DMA, etc.) by devices such asdata processor 907. - Communications link 915 can be configured to transmit (e.g., send, receive, signal, etc.) any types of communications packets (e.g., communication packet 938 1, communication packet 938 N) comprising any organization of data items. The data items can comprise a
payload data area 937, a destination address 936 (e.g., a destination IP address), a source address 935 (e.g., a source IP address), and can include various encodings or formatting of bit fields to populatepacket characteristics 934. In some cases, the packet characteristics include a version identifier, a packet or payload length, a traffic class, a flow label, etc. In some cases,payload data area 937 comprises a data structure that is encoded and/or formatted to fit into byte or word boundaries of the packet. - In some embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement aspects of the disclosure. Thus, embodiments of the disclosure are not limited to any specific combination of hardware circuitry and/or software. In embodiments, the term “logic” shall mean any combination of software or hardware that is used to implement all or part of the disclosure.
- The term “computer readable medium” or “computer usable medium” as used herein refers to any medium that participates in providing instructions to
data processor 907 for execution. Such a medium may take many forms including, but not limited to, non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks such as disk drives or tape drives. Volatile media includes dynamic memory such as RAM. - Common forms of computer readable media include, for example, floppy disk, flexible disk, hard disk, magnetic tape, or any other magnetic medium; CD-ROM or any other optical medium; punch cards, paper tape, or any other physical medium with patterns of holes; RAM, PROM, EPROM, FLASH-EPROM, or any other memory chip or cartridge, or any other non-transitory computer readable medium. Such data can be stored, for example, in any form of
external data repository 931, which in turn can be formatted into any one or more storage areas, and which can comprise parameterizedstorage 939 accessible by a key (e.g., filename, table name, block address, offset address, etc.). - Execution of the sequences of instructions to practice certain embodiments of the disclosure are performed by a single instance of a computer system 9A00. According to certain embodiments of the disclosure, two or more instances of computer system 9A00 coupled by a communications link 915 (e.g., LAN, public switched telephone network, or wireless network) may perform the sequence of instructions required to practice embodiments of the disclosure using two or more instances of components of computer system 9A00.
- Computer system 9A00 may transmit and receive messages such as data and/or instructions organized into a data structure (e.g., communications packets). The data structure can include program instructions (e.g., application code 903), communicated through communications link 915 and
communications interface 914. Received program instructions may be executed bydata processor 907 as it is received and/or stored in the shown storage device or in or upon any other non-volatile storage for later execution. Computer system 9A00 may communicate through adata interface 933 to adatabase 932 on anexternal data repository 931. Data items in a database can be accessed using a primary key (e.g., a relational database primary key). - Processing
element partition 901 is merely one sample partition. Other partitions can include multiple data processors, and/or multiple communications interfaces, and/or multiple storage devices, etc. within a partition. For example, a partition can bound a multi-core processor (e.g., possibly including embedded or co-located memory), or a partition can bound a computing cluster having plurality of computing elements, any of which computing elements are connected directly or indirectly to a communications link. A first partition can be configured to communicate to a second partition. A particular first partition and particular second partition can be congruent (e.g., in a processing element array) or can be different (e.g., comprising disjoint sets of components). - A module as used herein can be implemented using any mix of any portions of the system memory and any extent of hard-wired circuitry including hard-wired circuitry embodied as a
data processor 907. Some embodiments include one or more special-purpose hardware components (e.g., power control, logic, sensors, transducers, etc.). Some embodiments of a module include instructions that are stored in a memory for execution so as to facilitate operational and/or performance characteristics pertaining to measuring the performance of workflows performed over multiple applications. A module may include one or more state machines and/or combinational logic used to implement or facilitate the operational and/or performance characteristics pertaining to measuring the performance of workflows performed over multiple applications. - Various implementations of
database 932 comprise storage media organized to hold a series of records or files such that individual records or files are accessed using a name or key (e.g., a primary key or a combination of keys and/or query clauses). Such files or records can be organized into one or more data structures (e.g., data structures used to implement or facilitate aspects of measuring the performance of workflows performed over multiple applications). Such files, records, or data structures can be brought into and/or stored in volatile or non-volatile memory. More specifically, the occurrence and organization of the foregoing files, records, and data structures improve the way that the computer stores and retrieves data in memory, for example, to improve the way data is accessed when the computer is performing operations pertaining to measuring the performance of workflows performed over multiple applications, and/or for improving the way data is manipulated when performing computerized operations pertaining to linking content object interactions that are performed across multiple applications so as to generate performance measurements of workflows that comprise the interactions. -
FIG. 9B depicts a block diagram of an instance of a cloud-based environment 9B00. Such a cloud-based environment supports access to workspaces through the execution of workspace access code (e.g., workspace access code 942 0, workspace access code 942 1, and workspace access code 942 2). Workspace access code can be executed on any of access devices 952 (e.g.,laptop device 952 4,workstation device 952 5,IP phone device 952 3,tablet device 952 2,smart phone device 952 1, etc.), and can be configured to access any type of object. Strictly as examples, such objects can be folders or directories or can be files of any filetype. A group of users can form acollaborator group 958, and a collaborator group can be composed of any types or roles of users. For example, and as shown, a collaborator group can comprise a user collaborator, an administrator collaborator, a creator collaborator, etc. Any user can use any one or more of the access devices, and such access devices can be operated concurrently to provide multiple concurrent sessions and/or other techniques to access workspaces through the workspace access code. - A portion of workspace access code can reside in and be executed on any access device. Any portion of the workspace access code can reside in and be executed on any
computing platform 951, including in a middleware setting. As shown, a portion of the workspace access code resides in and can be executed on one or more processing elements (e.g., processing element 905 1). The workspace access code can interface with storage devices such asnetworked storage 955. Storage of workspaces and/or any constituent files or objects, and/or any other code or scripts or data can be stored in any one or more storage partitions (e.g., storage partition 904 1). In some environments, a processing element includes forms of storage, such as RAM and/or ROM and/or FLASH, and/or other forms of volatile and non-volatile storage. - A stored workspace can be populated via an upload (e.g., an upload from an access device to a processing element over an upload network path 957). A stored workspace can be delivered to a particular user and/or shared with other particular users via a download (e.g., a download from a processing element to an access device over a download network path 959).
- In the foregoing specification, the disclosure has been described with reference to specific embodiments thereof. It will however be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure. For example, the above-described process flows are described with reference to a particular ordering of process actions. However, the ordering of many of the described process actions may be changed without affecting the scope or operation of the disclosure. The specification and drawings are to be regarded in an illustrative sense rather than in a restrictive sense.
Claims (20)
1. A method, comprising:
capturing, at a content management computing system, first signals representing inter-process communications between a content management system and a plurality of heterogeneous applications;
capturing, from a first API (application programing interface) for the first application and a second API of the second application and based at least in part upon the inter-process communications, second signals representing collaboration between a first application and a second application of the plurality of heterogeneous applications on a shared content object that is hosted by the content management computing system;
selecting, by a software service instance, a smaller subset of interaction events from a set of interaction events in the collaboration based at least in part upon a request attribute of a first inter-process communication from the first application; and
generating an inter-application performance measurement based at least in part upon an event attribute associated with the smaller subset of interaction events.
2. The method of claim 1 , further comprising;
analyzing at least the event attribute associated with the smaller subset of interaction events, wherein the inter-application performance measurement is generated based at least in part upon an analysis result of analyzing at least the event attribute associated with the smaller subset of interaction events.
3. The method of claim 2 , further comprising;
selecting, by the software service instance or a separate software service instance, policy information from a plurality of policies stored in a central store based at least in part upon the request attribute, wherein
the smaller subset of interaction events is selected at least by applying the policy information to the event attribute associated with the smaller subset of interaction events.
4. The method of claim 3 , wherein the policy information comprises at least one of a plurality of workflow definitions of a plurality of work flows for the collaboration on the shared content object, a definition of an interaction between at least two applications of the plurality of heterogeneous applications, or a metric definition pertaining to the inter-application performance measurement.
5. The method of claim 3 , further comprising:
identifying the plurality of policies from the central store that is stored on the content management system; and
normalizing application information pertaining to the plurality of heterogeneous applications and the content management system into normalized application information that is stored in a application registry.
6. The method of claim 5 , further comprising:
normalizing policy data in at least one policy of the plurality of policies into normalized policy data for selecting the smaller subset of the interaction events, wherein
the event attribute comprises a link attribute and is stored in a storage device accessible by the content management system and remote from the plurality of heterogeneous applications and the central store,
the policy information is stored in the central store remote from remote from the plurality of heterogeneous applications and the content management system, and
the application registry is remote from the central store and the content management system.
7. The method of claim 1 , wherein the request attribute pertains to a request for the inter-application performance measurement, and the subset of interaction events is performed across multiple applications including the first application and the second applications of the plurality of heterogeneous applications that collaborate on the shared content object. ¶48, 52;
8. The method of claim 1 , wherein the request attribute pertains to a request for the inter-application performance measurement and comprises at least one of a link attribute that comprises information linking the shared content object to a user or a workflow for the first application or the second application.
9. The method of claim 1 , wherein the inter-process communications are transmitted to and from the content management system without human interaction, and the smaller subset of interaction events comprises a user-to-user interaction event.
10. The method of claim 1 , further comprising:
detecting at least one inter-process communication from the inter-process communications, wherein the at least one inter-process communication codifies at least the event attribute;
retrieving at least the event attribute from the at least one inter-process communication and one or more additional attributes, wherein
the one or more additional attributes comprise at least one of a user attribute pertaining to a user of the first application or the second application, an object attribute pertaining to the shared content object, an application attribute pertaining to the first or the second application, or an interaction definition attribute pertaining to the collaboration on the shared content object; and
populating an event record based at least in part upon at least the event attribute retrieved from the at least one inter-process communication and the one or more additional attributes.
11. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to execute a set of acts, the set of acts comprising:
capturing, at a content management computing system, first signals representing inter-process communications between a content management system and a plurality of heterogeneous applications;
capturing, from a first API for the first application and a second API of the second application and based at least in part upon the inter-process communications, second signals representing collaboration between a first application and a second application of the plurality of heterogeneous applications on a shared content object that is hosted by the content management computing system;
selecting, by a software service instance, a smaller subset of interaction events from a set of interaction events in the collaboration based at least in part upon a request attribute of a first inter-process communication from the first application; and
generating an inter-application performance measurement based at least in part upon an event attribute associated with the smaller subset of interaction events.
12. The non-transitory computer readable medium of claim 11 , the set of acts further comprising;
analyzing at least the event attribute associated with the smaller subset of interaction events, wherein the inter-application performance measurement is generated based at least in part upon an analysis result of analyzing at least the event attribute associated with the smaller subset of interaction events.
13. The non-transitory computer readable medium of claim 12 , the set of acts further comprising;
selecting, by the software service instance or a separate software service instance, policy information from a plurality of policies stored in a central store based at least in part upon the request attribute, wherein
the smaller subset of interaction events is selected at least by applying the policy information to the event attribute associated with the smaller subset of interaction events.
14. The non-transitory computer readable medium of claim 13 , wherein the policy information comprises at least one of a plurality of workflow definitions of a plurality of work flows for the collaboration on the shared content object, a definition of an interaction between at least two applications of the plurality of heterogeneous applications, or a metric definition pertaining to the inter-application performance measurement.
15. The non-transitory computer readable medium of claim 13 , the set of acts further comprising:
identifying the plurality of policies from the central store that is stored on the content management system;
normalizing application information pertaining to the plurality of heterogeneous applications and the content management system into normalized application information that is stored in a application registry;
normalizing policy data in at least one policy of the plurality of policies into normalized policy data for selecting the smaller subset of the interaction events, wherein
the event attribute comprises a link attribute and is stored in a storage device accessible by the content management system and remote from the plurality of heterogeneous applications and the central store,
the policy information is stored in the central store remote from remote from the plurality of heterogeneous applications and the content management system, and
the application registry is remote from the central store and the content management system.
16. A system, comprising:
a non-transitory storage medium having stored thereon a sequence of instructions; and
one or more processors that execute the sequence of instructions, execution of the sequence of instructions causes the one or more processors to perform a set of acts, the set of acts comprising,
capturing, at a content management computing system, first signals representing inter-process communications between a content management system and a plurality of heterogeneous applications;
capturing, from a first API (application programing interface) for the first application and a second API of the second application and based at least in part upon the inter-process communications, second signals representing collaboration between a first application and a second application of the plurality of heterogeneous applications on a shared content object that is hosted by the content management computing system;
selecting, by a software service instance, a smaller subset of interaction events from a set of interaction events in the collaboration based at least in part upon a request attribute of a first inter-process communication from the first application; and
generating an inter-application performance measurement based at least in part upon an event attribute associated with the smaller subset of interaction events.
17. The system of claim 16 , the set of acts further comprising;
analyzing at least the event attribute associated with the smaller subset of interaction events, wherein the inter-application performance measurement is generated based at least in part upon an analysis result of analyzing at least the event attribute associated with the smaller subset of interaction events.
18. The system of claim 17 , the set of acts further comprising;
selecting, by the software service instance or a separate software service instance, policy information from a plurality of policies stored in a central store based at least in part upon the request attribute, wherein
the smaller subset of interaction events is selected at least by applying the policy information to the event attribute associated with the smaller subset of interaction events.
19. The system of claim 18 , wherein the policy information comprises at least one of a plurality of workflow definitions of a plurality of work flows for the collaboration on the shared content object, a definition of an interaction between at least two applications of the plurality of heterogeneous applications, or a metric definition pertaining to the inter-application performance measurement.
20. The system of claim 18 , the set of acts further comprising:
identifying the plurality of policies from the central store that is stored on the content management system;
normalizing application information pertaining to the plurality of heterogeneous applications and the content management system into normalized application information that is stored in a application registry;
normalizing policy data in at least one policy of the plurality of policies into normalized policy data for selecting the smaller subset of the interaction events, wherein
the event attribute comprises a link attribute and is stored in a storage device accessible by the content management system and remote from the plurality of heterogeneous applications and the central store,
the policy information is stored in the central store remote from remote from the plurality of heterogeneous applications and the content management system, and
the application registry is remote from the central store and the content management system.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/309,982 US20230374095A1 (en) | 2019-10-01 | 2023-05-01 | Inter-application workflow performance analytics |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962909116P | 2019-10-01 | 2019-10-01 | |
US16/726,081 US11669793B2 (en) | 2019-10-01 | 2019-12-23 | Inter-application workflow performance analytics |
US18/309,982 US20230374095A1 (en) | 2019-10-01 | 2023-05-01 | Inter-application workflow performance analytics |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/726,081 Continuation US11669793B2 (en) | 2019-10-01 | 2019-12-23 | Inter-application workflow performance analytics |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230374095A1 true US20230374095A1 (en) | 2023-11-23 |
Family
ID=87553503
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/726,081 Active 2040-01-14 US11669793B2 (en) | 2019-10-01 | 2019-12-23 | Inter-application workflow performance analytics |
US18/309,982 Pending US20230374095A1 (en) | 2019-10-01 | 2023-05-01 | Inter-application workflow performance analytics |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/726,081 Active 2040-01-14 US11669793B2 (en) | 2019-10-01 | 2019-12-23 | Inter-application workflow performance analytics |
Country Status (1)
Country | Link |
---|---|
US (2) | US11669793B2 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11412049B2 (en) | 2018-08-27 | 2022-08-09 | Box, Inc. | Activity-based application recommendations |
Family Cites Families (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7424717B2 (en) | 2002-05-01 | 2008-09-09 | Bea Systems, Inc. | Systems and methods for business process plug-in development |
US20040260593A1 (en) * | 2003-05-20 | 2004-12-23 | Klaus Abraham-Fuchs | System and user interface supporting workflow operation improvement |
CN100465900C (en) * | 2003-10-29 | 2009-03-04 | 国际商业机器公司 | information system, load control method, load control program, and recording medium |
US20070203589A1 (en) | 2005-04-08 | 2007-08-30 | Manyworlds, Inc. | Adaptive Recombinant Process Methods |
US20060074703A1 (en) | 2004-10-04 | 2006-04-06 | Grand Central Communications, Inc. | Providing and managing business processes |
US8050952B2 (en) * | 2005-07-01 | 2011-11-01 | Sap Ag | Documenting occurrence of event |
US20070265900A1 (en) | 2006-05-09 | 2007-11-15 | Moore Dennis B | Business process evolution |
US7953758B2 (en) | 2006-11-10 | 2011-05-31 | Ricoh Company, Ltd. | Workflow management method and workflow management apparatus |
US7853039B2 (en) * | 2007-01-29 | 2010-12-14 | Kabushiki Kaisha Toshiba | Workflow management system and workflow management method |
US7917584B2 (en) * | 2007-10-22 | 2011-03-29 | Xcerion Aktiebolag | Gesture-based collaboration |
EP2203877A4 (en) | 2007-10-22 | 2012-08-01 | Open Text SA | Method and system for managing enterprise content |
JP5012628B2 (en) * | 2008-04-04 | 2012-08-29 | 日本電気株式会社 | Memory database, memory database system, and memory database update method |
US10678602B2 (en) * | 2011-02-09 | 2020-06-09 | Cisco Technology, Inc. | Apparatus, systems and methods for dynamic adaptive metrics based application deployment on distributed infrastructures |
US9940597B2 (en) | 2011-04-12 | 2018-04-10 | International Business Machines Corporation | Executing a business process by a standard business process engine |
US8782103B2 (en) | 2012-04-13 | 2014-07-15 | Dell Products, Lp | Monitoring system for optimizing integrated business processes to work flow |
US9148425B2 (en) | 2013-08-23 | 2015-09-29 | Oracle International Corporation | Second screen mediation |
US20150067028A1 (en) | 2013-08-30 | 2015-03-05 | Indian Space Research Organisation | Message driven method and system for optimal management of dynamic production workflows in a distributed environment |
US9846849B2 (en) | 2013-09-19 | 2017-12-19 | Oracle International Corporation | System and method for providing an editor for use with a business process design environment |
US9471711B2 (en) | 2013-09-23 | 2016-10-18 | Teradata Us, Inc. | Schema-less access to stored data |
US20150149535A1 (en) | 2013-11-27 | 2015-05-28 | Massively Parallel Technologies, Inc. | Systems And Methods For Application Execution And Deployment Cloud |
US10038731B2 (en) | 2014-08-29 | 2018-07-31 | Box, Inc. | Managing flow-based interactions with cloud-based shared content |
US10176435B1 (en) * | 2015-08-01 | 2019-01-08 | Shyam Sundar Sarkar | Method and apparatus for combining techniques of calculus, statistics and data normalization in machine learning for analyzing large volumes of data |
US10402399B2 (en) | 2015-09-04 | 2019-09-03 | Nuwafin Holdings Ltd | Computer implemented system and method for dynamically optimizing business processes |
US9753686B2 (en) | 2015-12-10 | 2017-09-05 | Ricoh Company, Ltd. | Document processing system, document workflow application and document processing method, configured for flexible workflow |
US10044636B2 (en) | 2016-03-11 | 2018-08-07 | Sap Se | Flow extension controller |
US20170316363A1 (en) | 2016-04-28 | 2017-11-02 | Microsoft Technology Licensing, Llc | Tailored recommendations for a workflow development system |
US10331416B2 (en) | 2016-04-28 | 2019-06-25 | Microsoft Technology Licensing, Llc | Application with embedded workflow designer |
US10540624B2 (en) | 2016-07-20 | 2020-01-21 | International Business Machines Corporation | System and method to automate provenance-aware application execution |
US20190026663A1 (en) | 2017-07-20 | 2019-01-24 | Ca, Inc. | Inferring time estimates in workflow tracking systems |
US10922282B2 (en) | 2017-10-09 | 2021-02-16 | Box, Inc. | On-demand collaboration user interfaces |
CN108256788A (en) * | 2018-02-13 | 2018-07-06 | 北京天元创新科技有限公司 | The method of end-to-end process performance analysis |
US11176105B2 (en) | 2018-04-27 | 2021-11-16 | Sap Se | System and methods for providing a schema-less columnar data store |
US10775974B2 (en) | 2018-08-10 | 2020-09-15 | Vignet Incorporated | User responsive dynamic architecture |
US20200065313A1 (en) | 2018-08-27 | 2020-02-27 | Box, Inc. | Extensible content object metadata |
US10521195B1 (en) | 2019-03-19 | 2019-12-31 | Servicenow, Inc. | Guided definition of an application programming interface action for a workflow |
US20200380432A1 (en) * | 2019-06-03 | 2020-12-03 | Sap Se | Predictive workflow control powered by machine learning in digital workplace |
US11240344B2 (en) | 2019-06-06 | 2022-02-01 | Servicenow, Inc. | Integration of remote software applications into a workflow |
US11256546B2 (en) | 2019-07-02 | 2022-02-22 | Nokia Technologies Oy | Methods, apparatuses and computer readable mediums for network based media processing |
US11861029B2 (en) | 2020-09-14 | 2024-01-02 | Box Inc. | Workflow execution state variables |
-
2019
- 2019-12-23 US US16/726,081 patent/US11669793B2/en active Active
-
2023
- 2023-05-01 US US18/309,982 patent/US20230374095A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US20200134518A1 (en) | 2020-04-30 |
US11669793B2 (en) | 2023-06-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11799969B2 (en) | Forming activity streams across heterogeneous applications | |
US10038731B2 (en) | Managing flow-based interactions with cloud-based shared content | |
US11727132B2 (en) | Activity-based content object access permissions | |
US11768707B2 (en) | Workflow selection | |
US20090271762A1 (en) | Business software application system and method | |
US20210286827A1 (en) | Collaboration activity summaries | |
US11861029B2 (en) | Workflow execution state variables | |
US10922282B2 (en) | On-demand collaboration user interfaces | |
WO2014071189A1 (en) | An interactive organizational decision-making and compliance facilitation portal | |
US20210103863A1 (en) | Cross-enterprise workflow adaptation | |
US11163834B2 (en) | Filtering collaboration activity | |
US20200092337A1 (en) | Context-aware content object security | |
US10747786B2 (en) | Spontaneous networking | |
US20160188762A1 (en) | Method and system for implementing intelligent system diagrams | |
US20230374095A1 (en) | Inter-application workflow performance analytics | |
US20170180476A1 (en) | Managing collaboration of shared content using collaborator indexing | |
US10757208B2 (en) | Curating collaboration activity | |
US11645187B2 (en) | Application curation | |
US11681572B2 (en) | Extensible workflow access | |
Lorincz et al. | Implementation of the Application Layer in the Business Process Management System |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BOX, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BAKSHI, ROHIT;MORKOVINE, DANIEL WAYNE;BUZDAR, FAIZAN N.;SIGNING DATES FROM 20191212 TO 20191219;REEL/FRAME:063493/0048 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |