US20190095261A1 - Software resource management - Google Patents

Software resource management Download PDF

Info

Publication number
US20190095261A1
US20190095261A1 US15/858,513 US201715858513A US2019095261A1 US 20190095261 A1 US20190095261 A1 US 20190095261A1 US 201715858513 A US201715858513 A US 201715858513A US 2019095261 A1 US2019095261 A1 US 2019095261A1
Authority
US
United States
Prior art keywords
multiapplication
workflow
applications
operations
interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/858,513
Inventor
Bin Wang
Dejun Zhang
Robert Yu Zhu
Lei Zhang
Pengxiang Zhao
Ying N. Chin
Satyendra Bahadur
Daryl A. Welsh
Carlos Picoto
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Priority to US15/858,513 priority Critical patent/US20190095261A1/en
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC reassignment MICROSOFT TECHNOLOGY LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PICOTO, CARLOS, WELSH, DARYL A., BAHADUR, SATYENDRA, CHIN, YING N., WANG, BIN, ZHANG, DEJUN, ZHANG, LEI, ZHAO, PENGXIANG, ZHU, ROBERT YU
Priority to PCT/US2018/039645 priority patent/WO2019067048A1/en
Publication of US20190095261A1 publication Critical patent/US20190095261A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/544Buffers; Shared memory; Pipes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/52Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
    • G06F21/53Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow by executing in a restricted environment, e.g. sandbox or secure virtual machine
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485Task life-cycle, e.g. stopping, restarting, resuming execution
    • G06F9/4856Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5072Grid computing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/541Interprogram communication via adapters, e.g. between incompatible applications
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/10Office automation; Time management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/148Migration or transfer of sessions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/34Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters 
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W8/00Network data management
    • H04W8/22Processing or transfer of terminal data, e.g. status or physical capabilities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/033Test or assess software
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482Interaction with lists of selectable items, e.g. menus

Definitions

  • Sandboxing allows an application to run on a computing device without allowing the application to have access to the resources of the computing device without explicit permission. Additionally, other applications on the computing device do not have access to the resources of the first application without explicit permission. Accordingly, the computing device is protected from damage caused by an untrusted application. However, when applications run in their own sandboxes, it is difficult for the applications to effectively work together without significant user involvement.
  • the disclosed technology provides a multiapplication workflow engine including one or more computing processors and a request processing engine configured to receive instruction to execute a multiapplication workflow via the one or more computing processors.
  • the multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations.
  • the interface definition corresponds to each operation being defined such that operations that are executed successively have compatible interface definitions.
  • a multiapplication workflow configuration manager is configured to instantiate one or more applications executed by the one or more computing processors. The one or more applications satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment.
  • the one or more applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow.
  • a multiapplication workflow execution engine is configured to execute the multiapplication workflow via the one or more computing processors in the protected execution environment. Data being passed among the one or more applications through the compatible interfaces.
  • FIG. 1 illustrates example computing environments for launching a multiapplication workflow.
  • FIG. 2 illustrates an example computing environment for executing a multiapplication workflow.
  • FIG. 3 illustrates an example computing environment for selecting applications to include in a chosen multiapplication workflow.
  • FIG. 4 illustrates an example computing environment for executing a multiapplication workflow on a computing device.
  • FIG. 5 illustrates example operations to initiate a multiapplication workflow.
  • FIG. 6 illustrates example operations for the execution of an example multiapplication workflow.
  • FIG. 7 illustrates an example system that may be useful in implementing the described technology.
  • each application When each application is sandboxed, the user acts as an intermediary between the applications, e.g., by manually launching an OCR application and selecting the photo from the photo browser and by manually inserting text output from the OCR application into an e-mail application.
  • a multiapplication workflow allows applications to communicate with each other through a shared memory.
  • the applications still cannot access the system resources or the user data of the computing device, so the multiapplication workflow does not jeopardize the security of the computing device.
  • the multiapplication workflow allows for a more efficient use of resources because the applications in the multiapplication workflow can interact more easily. For example, when the multiapplication workflow uses a shared memory, the applications use memory more efficiently and memory fragmentation may be improved.
  • FIG. 1 illustrates example computing environments 102 , 104 for launching a multiapplication workflow.
  • the multiapplication workflow is launched from a photo browsing application 106 .
  • a user may be viewing a photo 108 with text and want to send the text in the photo 108 as an e-mail.
  • the user would need to launch (e.g., instantiate or spawn a thread for) an OCR application, select the photo from the memory of the computing device, and manually input text into an e-mail application (e.g., by copying and pasting text from the OCR application into a new e-mail.)
  • the user may enter a query 110 into the photo browsing application 106 or some other search service or catalog to search for a multiapplication workflow that could convert the text in the photo to a text file and send the text in the body of an e-mail.
  • the user enters the query 110 “photo text to e-mail” into the photo browsing application 106 .
  • the photo browsing application 106 may display a list 112 of available multiapplication workflows that may fulfill the user's needs.
  • a description 114 may be available to provide the user with additional information to choose the correct multiapplication workflow.
  • description 114 states that the multiapplication workflow “OCR to e-mail” uses optical character recognition (OCR) to convert the text in a photo to text and send the text in an e-mail.
  • OCR optical character recognition
  • the user may select the OCR to e-mail multiapplication workflow to send the text in the photo 108 as an e-mail.
  • OCR to e-mail multiapplication workflow may also include, without limitation, a cost for applications not owned by the user, installation status of certain applications, or updates available for applications already owned by the user.
  • this example provides a multiapplication workflow for sending text in a photo as an e-mail
  • a variety of multiapplication workflows may exist for accomplishing a variety of tasks.
  • other multiapplication workflows may allow a user to take a photo of text with a camera built into a computing device and convert the text to a word processing document.
  • Another example multiapplication workflow may allow a user to check traffic and send an e-mail with their estimated time of arrival at a specific destination.
  • Many other multiapplication workflows may be created, deployed, and/or executed.
  • the example environments 102 and 104 show one method of launching a multiapplication workflow through an application.
  • the multiapplication workflow may be launched in a different way.
  • one implementation provides an index of multiapplication workflows available in a store or catalog that the user may search to find a multiapplication workflow with the user's desired functionality.
  • the user may store favorite multiapplication workflows on the computing device. The stored multiapplication workflows may then be launched directly by, for example, menus on the computing device or icons on the computing device's home screen or as an extension in another application.
  • FIG. 2 illustrates an example computing environment 200 for executing a multiapplication workflow.
  • the example computing environment 200 shows a user interface after a user has executed a multiapplication workflow to convert the text of a photo 208 to an e-mail.
  • the computing environment 200 includes both a photo browsing application 206 and an e-mail application 216 .
  • the user may not need to interact with each application individually to produce the desired result.
  • the multiapplication workflow used to convert the text in the photo 208 to the text 218 in the e-mail application 216 may use other applications with OCR to parse the text in the photo 208 to text 218 in the e-mail application 216 .
  • the user may interact with the photo browsing application 206 and the e-mail application 216 but need not interact with an additional OCR application.
  • the user may interact with additional applications or interfaces either before the execution of the multiapplication workflow or during execution of the multiapplication workflow.
  • the multiapplication workflow may receive permission from the user to install additional applications on the user's computing device.
  • the user may make selections in applications in the multiapplication workflow before receiving the final product of the multiapplication workflow.
  • FIG. 3 illustrates an example computing environment 300 for selecting applications to include in a chosen multiapplication workflow.
  • an OCR to e-mail multiapplication workflow 320 is launched from a photo browsing application 306 .
  • a list 312 populates with several multiapplication workflows that may satisfy the needs of the user.
  • the list 312 populates with the OCR to e-mail multiapplication workflow 320 , an e-mail to photo multiapplication workflow 322 , and a text to e-mail multiapplication workflow 324 .
  • the user may choose the multiapplication workflow that best fits the user's needs.
  • the user would choose the OCR to e-mail multiapplication workflow 320 .
  • the user may launch a multiapplication workflow from a different location.
  • the user may access and search a database including a catalog of multiapplication workflows instead of launching a multiapplication workflow from an already opened application.
  • the user may save multiapplication workflows on the memory of a computing device and may access a multiapplication workflow directly from the computing device.
  • a multiapplication workflow may launch automatically from the computing device without any input from the user.
  • the multiapplication workflow is populated with applications to perform the operations defined by the multiapplication workflow.
  • a multiapplication workflow includes several operations, with each operation being a sequential (or possibly, parallel) step in the overall process performed by the multiapplication workflow.
  • the OCR to e-mail multiapplication workflow 320 includes a photo viewing operation 326 , an OCR operation 328 , and an e-mail operation 330 . Each operation is fulfilled by at least one application.
  • the multiapplication workflow defines each operation by an interface definition and a functionality definition.
  • the interface definition specifies the input and/or output characteristics of an operation, including, without limitation, any one of Application Programming Interface (API), file type, data type, etc.
  • API Application Programming Interface
  • the OCR operation 328 may have an interface definition that defines the input capability by the ability to receive files in at least a .pdf file format and defines the output capability by the ability to output in plain text.
  • the interface definition of each application in the multiapplication workflow is defined partially based on the interface definitions of the other applications in the multiapplication workflow to ensure compatible interface definitions.
  • Compatibility means that applications that execute successively in the multiapplication workflow can communicate with each other through shared memory, shared file types, data types, API, or other interapplication communication means.
  • the term “successively” is used with reference to the sequential execution of the multiapplication workflow—an application and the application that follows it in the sequence of the multiapplication workflow are executed successively.
  • the OCR to e-mail multiapplication workflow 320 if the OCR operation 328 has an interface definition defining the output of the OCR operation 328 as plain text, the e-mail operation 330 has a compatible interface definition when the interface definition of the e-mail operation 330 defines the input of the e-mail operation 330 as plain text.
  • the functionality definition specifies what functionality the application should have to perform the function designated by an operation.
  • the OCR operation 328 may have a functionality definition for an application that has OCR capabilities.
  • the functionality definition may include both mandatory functionalities and optional functionalities.
  • Mandatory functionality specifies functionality that is necessary to execute the multiapplication workflow.
  • Optional functionality defines functionality that is desirable in an application, but not necessary to execute the multiapplication workflow as it is designed. Optional functionality is discussed in more detail with respect to FIG. 5 .
  • the OCR to e-mail multiapplication workflow 320 searches for applications to fulfill the interface definition and functionality definition for each operation defined by the OCR to e-mail multiapplication workflow 320 .
  • Each operation may have one or multiple applications that meet the interface definition and functionality definition of the operation.
  • the OCR to e-mail multiapplication workflow 320 may rank applications that fulfill both the interface definition and functionality definition to determine which application to include in each operation.
  • the photo viewing operation 326 may have a functionality definition for a user interface that allows for viewing photos and an interface definition for the ability to output photos in a .pdf format.
  • the photo browsing application 306 that the user is using to launch the OCR to e-mail multiapplication workflow 320 may meet both the functionality definition and the interface definition for the photo viewing operation 326 .
  • the computing device may have a second photo browsing application 332 stored in memory that meets both the functionality definition and the interface definition for the photo viewing operation 326 .
  • the photo browsing application 306 may be ranked ahead of the second photo browsing application 332 because the user launched the OCR to e-mail multiapplication workflow 320 from the photo browsing application 306 . Accordingly, the photo browsing application 306 is included in the OCR to e-mail multiapplication workflow 320 .
  • a variety of criteria may be used to rank applications to determine which application to include in a multiapplication workflow when multiple applications meet the functionality definition and the interface definition for a given operation.
  • Applications that include optional functionality may be included over applications that meet only the functionality definition for a given operation.
  • Other factors for example, without limitation, the price of an application, the file size of an application, or the developer identity, may be used to decide which applications to include in a multiapplication workflow.
  • the user may be prompted to decide which application to include when several applications meet both the functionality definition and the interface definition for a given operation within a multiapplication workflow.
  • a catalog of indexed applications may be searched when an application fitting the functionality definition and the interface definition is not already installed on the computing device.
  • an OCR application 334 fitting the functionality definition and the interface definition of the OCR operation 328 may be already downloaded to the computing device. Therefore, the indexed catalog of applications need not be searched for more applications that fit the functionality definition and the interface definition of the OCR operation 328 and the OCR application 334 is included in the OCR to e-mail multiapplication workflow 320 .
  • the catalog of indexed applications may still be searched to determine whether another application is available that may better fit the multiapplication workflow by, for example, having desired optional functionality or more recent software updates.
  • the computing device may search the catalog of indexed applications, there may be an e-mail application 336 , a second e-mail application 338 , and a third e-mail application 340 that all fulfill the functionality definition and the interface definition for the e-mail operation 330 .
  • the user may be prompted to choose which application to include and may choose the e-mail application 336 to include in the OCR to e-mail multiapplication workflow 320 . Because the e-mail application 336 is not already installed on the computing device, the e-mail application 336 may be downloaded and installed on the computing device prior to the execution of the OCR to e-mail multiapplication workflow 320 .
  • While each operation in the OCR to e-mail multiapplication workflow 320 is executed by one application, in other implementations, multiple applications may be used to obtain the functionality definition and the interface definition for a single operation. Multiple applications may be used to fulfill one operation when one application does not fulfill the functionality definition and the interface definition for the operation, but a combination of applications can be used to fulfill the functionality definition and the interface definition.
  • FIG. 4 illustrates an example computing environment 400 for executing a multiapplication workflow on a computing device 402 .
  • An OCR to e-mail multiapplication workflow 420 includes a shared memory 442 that can be accessed by all of the applications in the OCR to e-mail multiapplication workflow 420 .
  • the OCR to e-mail multiapplication workflow 420 includes a photo browsing application 406 , an OCR application 434 , and an e-mail application 436 .
  • the photo browsing application 406 , the OCR application 434 , and the e-mail application 436 are all able to access the shared memory 442 to share information without having access to system resources 444 and user data 446 of the computing device 402 .
  • the system resources 444 may include a processor, main memory, or other resources.
  • the photo browsing application 406 , the OCR application 434 , and the e-mail application 436 are located within a protected execution environment, meaning that the applications do not have access to the system resources 444 and user data 446 . Further, in the protected execution environment, other applications running outside of the protected execution environment do not have access to the resources and data of the applications located within the protected execution environment. Therefore, the OCR to e-mail multiapplication workflow 420 maintains the security of the computing device 402 while still allowing for collaboration between applications using the shared memory 442 .
  • the procedure performed by the multiapplication workflow is executed faster and with less memory use than executing the same procedure with individually sandboxed applications.
  • the applications may communicate using compatible interfaces via an API accessible to the applications executing within the protected execution environment.
  • the photo browsing application 406 , the OCR application 434 , and the e-mail application 436 are instantiated within the protected execution environment (e.g., launched, spawned in a thread, created) and ordered in accordance with execution order positions provided by the OCR to e-mail multiapplication workflow 420 .
  • the applications are ordered so that first, the photo browsing application 406 executes and passes data to the shared memory 442 .
  • the OCR application 434 receives data from the shared memory 442 , executes, and passes additional data to the shared memory 442 .
  • the e-mail application 436 receives data from the shared memory 442 and executes.
  • some applications in a multiapplication workflow may execute in parallel instead of sequentially. For example, if the user wanted to send a screenshot of the photo from the photo browsing application 406 , a screenshot application may execute at the same time as the OCR application 434 . The screenshot application and the OCR application 434 would both receive input from the photo browsing application 406 and would both send output to the e-mail application 436 . As such, the screenshot application and the OCR application 434 would execute in parallel and would be ordered to execute in parallel.
  • the shared memory 442 provides an access point for each application but does not convert data types.
  • the execution order positions provided by the OCR to e-mail application workflow provide compatibility between the applications.
  • the photo browsing application 406 passes data 448 in a .pdf format to the shared memory 442 .
  • the OCR application 434 receives the data 448 from the shared memory 442 in a .pdf format. Therefore, the photo browsing application 406 and the OCR application 434 are compatible.
  • Each of the photo browsing application 406 , the OCR application 434 , and the e-mail application 436 may have a data input interface and a data output interface that may be associated with the shared memory 442 before execution of OCR to e-mail multiapplication workflow 420 .
  • the applications may communicate by means other than being directly communicatively connected to a shared memory.
  • each application could execute in a workflow wrapper that could take advantage of a supervisory application within a protected execution environment.
  • FIG. 5 illustrates example operations 500 to initiate a multiapplication workflow.
  • a new multiapplication workflow is created by a query from the computing device 502 to a catalog of indexed applications 504 .
  • the query may be:
  • the computing device 502 requests a list of locally stored photo viewing, OCR, and e-mail applications from the catalog of indexed applications 504 .
  • the request may be communicated by a query such as:
  • the catalog of indexed applications 504 returns a list of locally stored photo viewing, OCR, and e-mail applications.
  • the returning operation 508 may communicate:
  • the computing device 502 After receiving a list of local applications, the computing device 502 communicates with the catalog of indexed applications 504 to determine which apps to include in the multiapplication workflow. In one example, there are two photo browsing applications and one e-mail application stored locally on the computing device 502 , but no locally stored OCR application. To determine which of the two locally stored photo browsing applications to include in the multiapplication workflow, the computing device 502 first queries the catalog of indexed applications 504 for photo browsing applications with a functionality definition in a query operation 510 . For example, if the functionality definition includes mandatory functionality of photo viewing through a user interface and optional functionality of photo editing, the query from the computing device 502 to the catalog of indexed applications 504 may be, for example:
  • the catalog of indexed applications 504 returns a list of locally stored photo viewing applications that meet the functionality definition in a second returning operation 512 .
  • the return list may be, for example:
  • the computing device 502 communicates which photo browsing app to include in the multiapplication workflow.
  • two photo browsing apps meet the functionality definition for a photo browsing app.
  • the computing device 502 may receive input from a user to determine which application to include.
  • the communication may be:
  • a second selection operation 516 the computing device 502 communicates which e-mail application to include in the multiapplication workflow.
  • the second selection operation may communicate:
  • preference is given to applications that are stored locally on the computing device 502 .
  • the user may be given a choice to download additional applications that may also fulfill the functionality definition.
  • the computing device 502 queries the catalog of indexed applications 504 for applications that fulfill the functionality definition for an OCR application that are available for download from the catalog of indexed applications 504 .
  • the functionality definition includes mandatory OCR functionality and an optional UI
  • the query may be:
  • the catalog of indexed applications 504 returns a list of applications with the mandatory functionality and returns indication of whether the applications listed also include the optional functionality.
  • the application database 504 would give three returns:
  • the computing device 502 selects the OCR application that has both the mandatory and optional functionality.
  • the computing device 502 would select the application by sending the following communication:
  • FIG. 5 illustrates operations for selecting applications to include in a multiapplication workflow based on a functionality definition.
  • An interface definition may also be used to select applications to include in a multiapplication workflow.
  • the query operations 510 and 518 include an additional query to search for applications that meet the interface definition.
  • the return operation 520 would then include applications that meet the functionality definition and the interface definition.
  • FIG. 6 illustrates example operations 600 for the execution of an example multiapplication workflow.
  • a user 602 communicates a request to launch a photo browsing application 606 .
  • the user may then launch the OCR to e-mail multiapplication workflow from the photo browsing application 606 .
  • the photo browsing application 606 sends a photo to the shared memory 642 in a sending operation 646 .
  • the shared memory 642 then sends the photo to an OCR application 634 .
  • the OCR application 634 converts the received photo to text.
  • the OCR application 634 then sends the text to the shared memory 642 in a third sending operation 652 .
  • the shared memory 642 sends the text to an e-mail application 636 in a fourth sending operation 654 .
  • FIG. 7 illustrates an example computing system 700 that may be useful in implementing the described technology.
  • the computing system 700 may be a client device such as a laptop, mobile device, desktop, tablet, or a server/cloud device.
  • the computing system 700 includes one or more processor(s) 702 , and a memory 704 .
  • the memory 704 generally includes both volatile memory (e.g., RAM) and non-volatile memory (e.g., flash memory).
  • An operating system 710 resides in the memory 704 and is executed by the processor(s) 702 .
  • One or more application programs 712 modules or segments, such as a multiapplication workflow engine 740 are loaded into the memory 704 and/or storage 720 and executed by the processor(s) 702 .
  • the application programs 712 or modules may include the multiapplication workflow engine 740 implemented by a request processing engine 744 , application selection engine 746 , multiapplication workflow configuration manager 748 , multiapplication workflow execution engine 750 etc. which may be embodied in instructions stored in the memory 704 and/or storage 720 and executed by the processor(s) 702 .
  • Data such as user preferences, hardware configurations, hardware responses, etc.
  • the storage 720 may be local to the computing system 700 or may be remote and communicatively connected to the computing system 700 and may include another server.
  • the storage 720 may store resources that are requestable by client devices (not shown).
  • the application programs 712 , the multiapplication workflow engine 740 , the request processing engine 744 , the application selection engine 746 , the multiapplication workflow configuration manage 748 , and/or the multiapplication workflow execution engine 750 may reside in or execute as part of the operating system 710 .
  • the multiapplication workflow engine 740 may involve printer drivers or other resources that also reside in or execute within the operating system 710 .
  • the computing system 700 includes a power supply 716 , which is powered by one or more batteries or other power sources and which provides power to other components of the computing system 700 .
  • the power supply 716 may also be connected to an external power source that overrides or recharges the built-in batteries or other power sources.
  • the computing system 700 may include one or more communication transceivers 730 which may be connected to one or more antenna(s) 732 to provide network connectivity (e.g., mobile phone network, Wi-Fi®, Bluetooth®, etc.) to one or more other servers and/or client devices (e.g., mobile devices, desktop computers, or laptop computers).
  • the computing system 700 may further include a network adapter 736 , which is a type of communication device.
  • the computing system 700 may use the adapter and any other types of communication devices for establishing connections over a wide-area network (WAN) or local-area network (LAN). It should be appreciated that the network connections shown are exemplary and that other communications devices and means for establishing a communications link between the computing system 700 and other devices may be used.
  • the computing system 700 may include one or more input devices 734 such that a user may enter commands and information (e.g., a keyboard or mouse). These and other input devices may be coupled to the server by one or more interfaces 738 such as a serial port interface, parallel port, universal serial bus (USB), etc.
  • the processing system 700 may further include a display 722 such as a touch screen display.
  • the computing system 700 may include a variety of tangible processor-readable storage media and intangible processor-readable communication signals.
  • Tangible processor-readable storage can be embodied by any available media that can be accessed by the computing system 700 and includes both volatile and nonvolatile storage media, removable and non-removable storage media.
  • Tangible processor-readable storage media excludes intangible communications signals and includes volatile and nonvolatile, removable and non-removable storage media implemented in any method or technology for storage of information such as processor-readable instructions, data structures, program modules or other data.
  • Tangible processor-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other tangible medium which can be used to store the desired information and which can be accessed by the processing system 700 .
  • intangible processor-readable communication signals may embody processor-readable instructions, data structures, program modules or other data resident in a modulated data signal, such as a carrier wave or other signal transport mechanism.
  • modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • intangible communication signals include signals traveling through wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.
  • the multiapplication workflow engine includes one or more computing processors, a request processing engine, a multiapplication workflow configuration manager, and a multiapplication workflow execution engine.
  • the request processing engine is configured to receive an instruction to execute a multiapplication workflow via the one or more computing processors.
  • the multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations.
  • the interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions.
  • the multiapplication workflow configuration manager is configured to instantiate one or more applications executed by the one or more computing processors, the one or more applications satisfying the interface definition and functionality definition for each of the two or more operations in a protected execution environment, the one or more applications each being configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow.
  • the multiapplication workflow execution engine is configured to execute the multiapplication workflow via the one or more computing processors in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, further including a shared memory inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • the compatible interfaces are configured for communication through the shared memory accessible to the applications executing in the protected execution environment.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the multiapplication workflow configuration manager associates a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the multiapplication workflow engine includes an application programming interface inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • the compatible interfaces are configured for communication through the application programming interface accessible to the one or more applications executing in the protected execution environment.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the multiapplication workflow engine includes an application selection engine configured to search a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the application selection engine selects one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the request processing engine receives an instruction to execute the multiapplication workflow and each functionality definition provided by the multiapplication workflow includes one or more mandatory functionalities and one or more optional functionalities.
  • An example method includes receiving instruction to execute a multiapplication workflow.
  • the multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations.
  • the interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions.
  • the example method further includes instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment.
  • the applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and are ordered in accordance with execution order positions provided by the multiapplication workflow.
  • the example method further includes executing the multiapplication workflow in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • An example method of any previous method includes associating a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
  • compatible interfaces are configured for communication through an application programming interface accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • the method of any previous method further includes searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • the method of any previous method further includes selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
  • An example system includes means for receiving instruction to execute a multiapplication workflow.
  • the multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations.
  • the interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions.
  • the example system further includes means for instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment.
  • the applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and are ordered in accordance with execution order positions provided by the multiapplication workflow.
  • the example system further includes means for executing the multiapplication workflow in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • compatible interfaces are configured for communication through a shared memory accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • An example system of any preceding system includes means for associating a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
  • compatible interfaces are configured for communication through an application programming interface accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • An example system of any preceding system includes means for searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • An example system of any preceding system includes means for selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
  • Example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a computing device a process for receiving instruction to execute a multiapplication workflow.
  • the multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations.
  • the interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions.
  • the process further includes instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment.
  • the applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and are ordered in accordance with execution order positions provided by the multiapplication workflow.
  • the process further includes executing the multiapplication workflow in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process wherein the compatible interfaces are configured for communication through a shared memory accessible to the one or more applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • FIG. 1 Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process including associating a data input interface and a data output interface of each of the selected applications with the shared memory.
  • processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process wherein each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
  • FIG. 1 Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process including searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • FIG. 1 Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process including selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • An article of manufacture may comprise a tangible storage medium to store logic.
  • Examples of a storage medium may include one or more types of computer-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and so forth.
  • Examples of the logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, operation segments, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.
  • an article of manufacture may store executable computer program instructions that, when executed by a computer, cause the computer to perform methods and/or operations in accordance with the described embodiments.
  • the executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like.
  • the executable computer program instructions may be implemented according to a predefined computer language, manner or syntax, for instructing a computer to perform a certain operation segment.
  • the instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.
  • the implementations described herein are implemented as logical steps in one or more computer systems.
  • the logical operations may be implemented (1) as a sequence of processor-implemented steps executing in one or more computer systems and (2) as interconnected machine or circuit modules within one or more computer systems.
  • the implementation is a matter of choice, dependent on the performance requirements of the computer system being utilized. Accordingly, the logical operations making up the implementations described herein are referred to variously as operations, steps, objects, or modules.
  • logical operations may be performed in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computer Hardware Design (AREA)
  • Marketing (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Economics (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Stored Programmes (AREA)
  • Computer And Data Communications (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The disclosed technology provides a multiapplication workflow engine. The multiapplication workflow engine includes a request processing engine, a multiapplication workflow configuration manager, and a multiapplication workflow execution engine. The request processing engine is configured to receive instructions to execute a multiapplication workflow. The multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the operations and an interface definition corresponding to each of the operations. The interface defintions of each operation are defined so that operations that execute successively have compatible interface definitions. The multiapplication workflow configuration manager is configured to instatiate one or more applications that satisfy the interface definition and the functionality definition for each operation in a protected execution environment, where the applications are configured to communicate via compatible interfaces. The multiapplication workflow engine is configured to execute the multiapplication workflow in the protected execution environment.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • The present application claims benefit of priority to U.S. Provisional Patent Application No. 62/564,888, entitled “Vertical and Horizontal Solutions for Hardware/Software Resources Management” and filed on Sep. 28, 2017, which is specifically incorporated by reference herein for all that it discloses and teaches.
  • BACKGROUND
  • Individual applications running on computing devices are often “sandboxed” separately for security reasons. Sandboxing allows an application to run on a computing device without allowing the application to have access to the resources of the computing device without explicit permission. Additionally, other applications on the computing device do not have access to the resources of the first application without explicit permission. Accordingly, the computing device is protected from damage caused by an untrusted application. However, when applications run in their own sandboxes, it is difficult for the applications to effectively work together without significant user involvement.
  • SUMMARY
  • In at least one implementation, the disclosed technology provides a multiapplication workflow engine including one or more computing processors and a request processing engine configured to receive instruction to execute a multiapplication workflow via the one or more computing processors. The multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations. The interface definition corresponds to each operation being defined such that operations that are executed successively have compatible interface definitions. A multiapplication workflow configuration manager is configured to instantiate one or more applications executed by the one or more computing processors. The one or more applications satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment. The one or more applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow. A multiapplication workflow execution engine is configured to execute the multiapplication workflow via the one or more computing processors in the protected execution environment. Data being passed among the one or more applications through the compatible interfaces.
  • This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
  • Other implementations are also described and recited herein.
  • BRIEF DESCRIPTIONS OF THE DRAWINGS
  • FIG. 1 illustrates example computing environments for launching a multiapplication workflow.
  • FIG. 2 illustrates an example computing environment for executing a multiapplication workflow.
  • FIG. 3 illustrates an example computing environment for selecting applications to include in a chosen multiapplication workflow.
  • FIG. 4 illustrates an example computing environment for executing a multiapplication workflow on a computing device.
  • FIG. 5 illustrates example operations to initiate a multiapplication workflow.
  • FIG. 6 illustrates example operations for the execution of an example multiapplication workflow.
  • FIG. 7 illustrates an example system that may be useful in implementing the described technology.
  • DETAILED DESCRIPTIONS
  • Currently, individual applications on some computing devices are executed in a “sandbox,” meaning that each individual application has its own temporary memory. Additionally, an application executed in a sandbox cannot directly access the system resources and user data of the computing device. Further, other applications running on the same computing device cannot access the first application running in the sandbox. While sandboxed execution is beneficial for security, it can be inefficient when a user wants to execute a procedure that requires input from multiple applications. For example, a user may wish to use optical character recognition (OCR) to convert the text in a photo to text format, and then to send that text via e-mail. When each application is sandboxed, the user acts as an intermediary between the applications, e.g., by manually launching an OCR application and selecting the photo from the photo browser and by manually inserting text output from the OCR application into an e-mail application.
  • A multiapplication workflow allows applications to communicate with each other through a shared memory. The applications still cannot access the system resources or the user data of the computing device, so the multiapplication workflow does not jeopardize the security of the computing device. However, the multiapplication workflow allows for a more efficient use of resources because the applications in the multiapplication workflow can interact more easily. For example, when the multiapplication workflow uses a shared memory, the applications use memory more efficiently and memory fragmentation may be improved.
  • FIG. 1 illustrates example computing environments 102, 104 for launching a multiapplication workflow. In this example environment, the multiapplication workflow is launched from a photo browsing application 106. A user may be viewing a photo 108 with text and want to send the text in the photo 108 as an e-mail. Without a multiapplication workflow, the user would need to launch (e.g., instantiate or spawn a thread for) an OCR application, select the photo from the memory of the computing device, and manually input text into an e-mail application (e.g., by copying and pasting text from the OCR application into a new e-mail.) In an example implementation with a multiapplication workflow, the user may enter a query 110 into the photo browsing application 106 or some other search service or catalog to search for a multiapplication workflow that could convert the text in the photo to a text file and send the text in the body of an e-mail. For example, in computing environment 102, the user enters the query 110 “photo text to e-mail” into the photo browsing application 106. In response to the query 110, the photo browsing application 106 may display a list 112 of available multiapplication workflows that may fulfill the user's needs.
  • Once the list 112 of available multiapplication workflows populates, the user may choose which multiapplication workflow would best accomplish the task that the user wants to accomplish. A description 114 may be available to provide the user with additional information to choose the correct multiapplication workflow. For example, description 114 states that the multiapplication workflow “OCR to e-mail” uses optical character recognition (OCR) to convert the text in a photo to text and send the text in an e-mail. After reading the description 114, the user may select the OCR to e-mail multiapplication workflow to send the text in the photo 108 as an e-mail. Such description may also include, without limitation, a cost for applications not owned by the user, installation status of certain applications, or updates available for applications already owned by the user.
  • Though this example provides a multiapplication workflow for sending text in a photo as an e-mail, a variety of multiapplication workflows may exist for accomplishing a variety of tasks. For example, other multiapplication workflows may allow a user to take a photo of text with a camera built into a computing device and convert the text to a word processing document. Another example multiapplication workflow may allow a user to check traffic and send an e-mail with their estimated time of arrival at a specific destination. Many other multiapplication workflows may be created, deployed, and/or executed.
  • The example environments 102 and 104 show one method of launching a multiapplication workflow through an application. In other implementations, the multiapplication workflow may be launched in a different way. For example, one implementation provides an index of multiapplication workflows available in a store or catalog that the user may search to find a multiapplication workflow with the user's desired functionality. In another implementation, the user may store favorite multiapplication workflows on the computing device. The stored multiapplication workflows may then be launched directly by, for example, menus on the computing device or icons on the computing device's home screen or as an extension in another application.
  • FIG. 2 illustrates an example computing environment 200 for executing a multiapplication workflow. The example computing environment 200 shows a user interface after a user has executed a multiapplication workflow to convert the text of a photo 208 to an e-mail. The computing environment 200 includes both a photo browsing application 206 and an e-mail application 216. Though several applications are included in the multiapplication workflow, the user may not need to interact with each application individually to produce the desired result. For example, the multiapplication workflow used to convert the text in the photo 208 to the text 218 in the e-mail application 216 may use other applications with OCR to parse the text in the photo 208 to text 218 in the e-mail application 216. However, the user may interact with the photo browsing application 206 and the e-mail application 216 but need not interact with an additional OCR application.
  • In some implementations, the user may interact with additional applications or interfaces either before the execution of the multiapplication workflow or during execution of the multiapplication workflow. For example, the multiapplication workflow may receive permission from the user to install additional applications on the user's computing device. In other implementations, the user may make selections in applications in the multiapplication workflow before receiving the final product of the multiapplication workflow.
  • FIG. 3 illustrates an example computing environment 300 for selecting applications to include in a chosen multiapplication workflow. In this implementation, an OCR to e-mail multiapplication workflow 320 is launched from a photo browsing application 306. In response to a user inquiry, a list 312 populates with several multiapplication workflows that may satisfy the needs of the user. In FIG. 3, the list 312 populates with the OCR to e-mail multiapplication workflow 320, an e-mail to photo multiapplication workflow 322, and a text to e-mail multiapplication workflow 324. After the list 312 is populated with options, the user may choose the multiapplication workflow that best fits the user's needs. In FIG. 3, the user would choose the OCR to e-mail multiapplication workflow 320.
  • In other implementations, the user may launch a multiapplication workflow from a different location. For example, in one implementation, the user may access and search a database including a catalog of multiapplication workflows instead of launching a multiapplication workflow from an already opened application. In another implementation, the user may save multiapplication workflows on the memory of a computing device and may access a multiapplication workflow directly from the computing device. In yet another implementation, a multiapplication workflow may launch automatically from the computing device without any input from the user.
  • Once a multiapplication workflow has been selected the multiapplication workflow is populated with applications to perform the operations defined by the multiapplication workflow. A multiapplication workflow includes several operations, with each operation being a sequential (or possibly, parallel) step in the overall process performed by the multiapplication workflow. For example, the OCR to e-mail multiapplication workflow 320 includes a photo viewing operation 326, an OCR operation 328, and an e-mail operation 330. Each operation is fulfilled by at least one application.
  • To determine which application or applications to place in each operation, the multiapplication workflow defines each operation by an interface definition and a functionality definition. The interface definition specifies the input and/or output characteristics of an operation, including, without limitation, any one of Application Programming Interface (API), file type, data type, etc. For example, the OCR operation 328 may have an interface definition that defines the input capability by the ability to receive files in at least a .pdf file format and defines the output capability by the ability to output in plain text. The interface definition of each application in the multiapplication workflow is defined partially based on the interface definitions of the other applications in the multiapplication workflow to ensure compatible interface definitions. Compatibility means that applications that execute successively in the multiapplication workflow can communicate with each other through shared memory, shared file types, data types, API, or other interapplication communication means. In the described technology, the term “successively” is used with reference to the sequential execution of the multiapplication workflow—an application and the application that follows it in the sequence of the multiapplication workflow are executed successively. For example, in the OCR to e-mail multiapplication workflow 320, if the OCR operation 328 has an interface definition defining the output of the OCR operation 328 as plain text, the e-mail operation 330 has a compatible interface definition when the interface definition of the e-mail operation 330 defines the input of the e-mail operation 330 as plain text.
  • The functionality definition specifies what functionality the application should have to perform the function designated by an operation. For example, the OCR operation 328 may have a functionality definition for an application that has OCR capabilities. The functionality definition may include both mandatory functionalities and optional functionalities. Mandatory functionality specifies functionality that is necessary to execute the multiapplication workflow. Optional functionality defines functionality that is desirable in an application, but not necessary to execute the multiapplication workflow as it is designed. Optional functionality is discussed in more detail with respect to FIG. 5.
  • The OCR to e-mail multiapplication workflow 320 searches for applications to fulfill the interface definition and functionality definition for each operation defined by the OCR to e-mail multiapplication workflow 320. Each operation may have one or multiple applications that meet the interface definition and functionality definition of the operation. The OCR to e-mail multiapplication workflow 320 may rank applications that fulfill both the interface definition and functionality definition to determine which application to include in each operation. For example, the photo viewing operation 326 may have a functionality definition for a user interface that allows for viewing photos and an interface definition for the ability to output photos in a .pdf format. The photo browsing application 306 that the user is using to launch the OCR to e-mail multiapplication workflow 320 may meet both the functionality definition and the interface definition for the photo viewing operation 326. The computing device may have a second photo browsing application 332 stored in memory that meets both the functionality definition and the interface definition for the photo viewing operation 326. The photo browsing application 306 may be ranked ahead of the second photo browsing application 332 because the user launched the OCR to e-mail multiapplication workflow 320 from the photo browsing application 306. Accordingly, the photo browsing application 306 is included in the OCR to e-mail multiapplication workflow 320.
  • In other implementations, a variety of criteria may be used to rank applications to determine which application to include in a multiapplication workflow when multiple applications meet the functionality definition and the interface definition for a given operation. Applications that include optional functionality may be included over applications that meet only the functionality definition for a given operation. Other factors, for example, without limitation, the price of an application, the file size of an application, or the developer identity, may be used to decide which applications to include in a multiapplication workflow. In some implementations, the user may be prompted to decide which application to include when several applications meet both the functionality definition and the interface definition for a given operation within a multiapplication workflow.
  • When searching for applications to fulfill the functionality definition and the interface definition for a given operation, a catalog of indexed applications may be searched when an application fitting the functionality definition and the interface definition is not already installed on the computing device. For example, an OCR application 334 fitting the functionality definition and the interface definition of the OCR operation 328 may be already downloaded to the computing device. Therefore, the indexed catalog of applications need not be searched for more applications that fit the functionality definition and the interface definition of the OCR operation 328 and the OCR application 334 is included in the OCR to e-mail multiapplication workflow 320. However, the catalog of indexed applications may still be searched to determine whether another application is available that may better fit the multiapplication workflow by, for example, having desired optional functionality or more recent software updates.
  • In contrast, there may be no applications on the computing device that fulfill the functionality definition and the interface definition for the e-mail operation 330. After searching the catalog of indexed applications, there may be an e-mail application 336, a second e-mail application 338, and a third e-mail application 340 that all fulfill the functionality definition and the interface definition for the e-mail operation 330. In one implementation, the user may be prompted to choose which application to include and may choose the e-mail application 336 to include in the OCR to e-mail multiapplication workflow 320. Because the e-mail application 336 is not already installed on the computing device, the e-mail application 336 may be downloaded and installed on the computing device prior to the execution of the OCR to e-mail multiapplication workflow 320.
  • While each operation in the OCR to e-mail multiapplication workflow 320 is executed by one application, in other implementations, multiple applications may be used to obtain the functionality definition and the interface definition for a single operation. Multiple applications may be used to fulfill one operation when one application does not fulfill the functionality definition and the interface definition for the operation, but a combination of applications can be used to fulfill the functionality definition and the interface definition.
  • FIG. 4 illustrates an example computing environment 400 for executing a multiapplication workflow on a computing device 402. An OCR to e-mail multiapplication workflow 420 includes a shared memory 442 that can be accessed by all of the applications in the OCR to e-mail multiapplication workflow 420. In this implementation, the OCR to e-mail multiapplication workflow 420 includes a photo browsing application 406, an OCR application 434, and an e-mail application 436. The photo browsing application 406, the OCR application 434, and the e-mail application 436 are all able to access the shared memory 442 to share information without having access to system resources 444 and user data 446 of the computing device 402. The system resources 444 may include a processor, main memory, or other resources. The photo browsing application 406, the OCR application 434, and the e-mail application 436 are located within a protected execution environment, meaning that the applications do not have access to the system resources 444 and user data 446. Further, in the protected execution environment, other applications running outside of the protected execution environment do not have access to the resources and data of the applications located within the protected execution environment. Therefore, the OCR to e-mail multiapplication workflow 420 maintains the security of the computing device 402 while still allowing for collaboration between applications using the shared memory 442. Further, because many redundant steps may be eliminated by a multiapplication workflow, the procedure performed by the multiapplication workflow is executed faster and with less memory use than executing the same procedure with individually sandboxed applications. In other implementations, the applications may communicate using compatible interfaces via an API accessible to the applications executing within the protected execution environment.
  • Before execution of the OCR to e-mail multiapplication workflow 420, the photo browsing application 406, the OCR application 434, and the e-mail application 436 are instantiated within the protected execution environment (e.g., launched, spawned in a thread, created) and ordered in accordance with execution order positions provided by the OCR to e-mail multiapplication workflow 420. For example, the applications are ordered so that first, the photo browsing application 406 executes and passes data to the shared memory 442. Second, the OCR application 434 receives data from the shared memory 442, executes, and passes additional data to the shared memory 442. Third, the e-mail application 436 receives data from the shared memory 442 and executes. In some implementations, some applications in a multiapplication workflow may execute in parallel instead of sequentially. For example, if the user wanted to send a screenshot of the photo from the photo browsing application 406, a screenshot application may execute at the same time as the OCR application 434. The screenshot application and the OCR application 434 would both receive input from the photo browsing application 406 and would both send output to the e-mail application 436. As such, the screenshot application and the OCR application 434 would execute in parallel and would be ordered to execute in parallel.
  • The shared memory 442 provides an access point for each application but does not convert data types. As such, the execution order positions provided by the OCR to e-mail application workflow provide compatibility between the applications. For example, the photo browsing application 406 passes data 448 in a .pdf format to the shared memory 442. The OCR application 434 receives the data 448 from the shared memory 442 in a .pdf format. Therefore, the photo browsing application 406 and the OCR application 434 are compatible. Each of the photo browsing application 406, the OCR application 434, and the e-mail application 436 may have a data input interface and a data output interface that may be associated with the shared memory 442 before execution of OCR to e-mail multiapplication workflow 420.
  • In some implementations, the applications may communicate by means other than being directly communicatively connected to a shared memory. For example, each application could execute in a workflow wrapper that could take advantage of a supervisory application within a protected execution environment.
  • FIG. 5 illustrates example operations 500 to initiate a multiapplication workflow. A new multiapplication workflow is created by a query from the computing device 502 to a catalog of indexed applications 504. For example, the query may be:

  • CreateWorkflow(“DigArc”)
  • After the multiapplication workflow is initiated, in a requesting operation 506, the computing device 502 requests a list of locally stored photo viewing, OCR, and e-mail applications from the catalog of indexed applications 504. The request may be communicated by a query such as:

  • DigArc.Features(photo, ocr, email)
  • In a returning operation 508, the catalog of indexed applications 504 returns a list of locally stored photo viewing, OCR, and e-mail applications. When, for example, the computing device has two photo browsing applications and one e-mail application stored locally, but does not have an OCR application stored locally, the returning operation 508 may communicate:

  • photo.Applist=[photos, picbrowse], email.Applist=[Mail], OCR.Applist=Empty
  • After receiving a list of local applications, the computing device 502 communicates with the catalog of indexed applications 504 to determine which apps to include in the multiapplication workflow. In one example, there are two photo browsing applications and one e-mail application stored locally on the computing device 502, but no locally stored OCR application. To determine which of the two locally stored photo browsing applications to include in the multiapplication workflow, the computing device 502 first queries the catalog of indexed applications 504 for photo browsing applications with a functionality definition in a query operation 510. For example, if the functionality definition includes mandatory functionality of photo viewing through a user interface and optional functionality of photo editing, the query from the computing device 502 to the catalog of indexed applications 504 may be, for example:

  • photo. QueryInterface(Mandatory=UI, Optional=Photo Edit)
  • In response to the query operation 510, the catalog of indexed applications 504 returns a list of locally stored photo viewing applications that meet the functionality definition in a second returning operation 512. Where both of the locally stored applications meet both the mandatory and optional functionality, the return list may be, for example:

  • photo.Applist=[photos, picbrowse]
  • In a selection operation 514, the computing device 502 communicates which photo browsing app to include in the multiapplication workflow. In this example, two photo browsing apps meet the functionality definition for a photo browsing app. To determine which of the two photo browsing applications to include, in one implementation, the computing device 502 may receive input from a user to determine which application to include. In other implementations, there may be additional criteria to assist the computing device in the determination. For example, the computing device may give preference to applications from specific developers, applications that the user opens more often, or applications with higher user ratings in the application database. Here, if the computing device 502 determines that it will include the “photos” application in the multiapplication workflow, the communication may be:

  • photo.AppSelect=photos
  • In a second selection operation 516, the computing device 502 communicates which e-mail application to include in the multiapplication workflow. Here, there is no additional determination because there is only one e-mail application stored locally on the computing device 502. The second selection operation may communicate:

  • email.AppSelect=mail
  • In this implementation, preference is given to applications that are stored locally on the computing device 502. In other implementations, the user may be given a choice to download additional applications that may also fulfill the functionality definition.
  • In a second query operation 518, the computing device 502 queries the catalog of indexed applications 504 for applications that fulfill the functionality definition for an OCR application that are available for download from the catalog of indexed applications 504. Where the functionality definition includes mandatory OCR functionality and an optional UI, the query may be:

  • ocr.FindStoreApp(Mandatory, Optional), Mandatory=OCR, Optional=UI
  • In a third returning operation 520, the catalog of indexed applications 504 returns a list of applications with the mandatory functionality and returns indication of whether the applications listed also include the optional functionality. Here, there may be two applications with OCR capability, where only one application also has a UI. The application database 504 would give three returns:

  • ocr.StoreAppList=[eRealOCR, PaperMaster]

  • eRealOCR.Mandatory=OCR, eRealOCR.Optional=No UI

  • PaperMaster.Madatory=OCR, PaperMaster.Optional=Has UI
  • In a third selection operation 522, the computing device 502 selects the OCR application that has both the mandatory and optional functionality. Here, the computing device 502 would select the application by sending the following communication:

  • ocr.AppSelect=PaperMaster
  • After the computing device 502 communicates its selection, the catalog of indexed applications 504 sends the selected application to the computing device 502 in a downloading operation 524. FIG. 5 illustrates operations for selecting applications to include in a multiapplication workflow based on a functionality definition. An interface definition may also be used to select applications to include in a multiapplication workflow. When the interface definition is used, the query operations 510 and 518 include an additional query to search for applications that meet the interface definition. The return operation 520 would then include applications that meet the functionality definition and the interface definition.
  • FIG. 6 illustrates example operations 600 for the execution of an example multiapplication workflow. In a launching operation 644, a user 602 communicates a request to launch a photo browsing application 606. The user may then launch the OCR to e-mail multiapplication workflow from the photo browsing application 606. Once the OCR to e-mail multiapplication workflow has been launched, the photo browsing application 606 sends a photo to the shared memory 642 in a sending operation 646. In a second sending operation 648, the shared memory 642 then sends the photo to an OCR application 634. In a converting operation 650, the OCR application 634 converts the received photo to text. The OCR application 634 then sends the text to the shared memory 642 in a third sending operation 652. Next, the shared memory 642 sends the text to an e-mail application 636 in a fourth sending operation 654.
  • FIG. 7 illustrates an example computing system 700 that may be useful in implementing the described technology. The computing system 700 may be a client device such as a laptop, mobile device, desktop, tablet, or a server/cloud device. The computing system 700 includes one or more processor(s) 702, and a memory 704. The memory 704 generally includes both volatile memory (e.g., RAM) and non-volatile memory (e.g., flash memory). An operating system 710 resides in the memory 704 and is executed by the processor(s) 702.
  • One or more application programs 712 modules or segments, such as a multiapplication workflow engine 740 are loaded into the memory 704 and/or storage 720 and executed by the processor(s) 702. The application programs 712 or modules may include the multiapplication workflow engine 740 implemented by a request processing engine 744, application selection engine 746, multiapplication workflow configuration manager 748, multiapplication workflow execution engine 750 etc. which may be embodied in instructions stored in the memory 704 and/or storage 720 and executed by the processor(s) 702. Data such as user preferences, hardware configurations, hardware responses, etc. may be stored in the memory 704 or storage 720 and may be retrievable by the processor(s) 702 for use by the multiapplication workflow engine 740, the request processing engine 744, the application selection engine 746, the multiapplication workflow configuration manager 748, the multiapplication workflow execution engine 750, etc. The storage 720 may be local to the computing system 700 or may be remote and communicatively connected to the computing system 700 and may include another server. The storage 720 may store resources that are requestable by client devices (not shown). In some implementations, the application programs 712, the multiapplication workflow engine 740, the request processing engine 744, the application selection engine 746, the multiapplication workflow configuration manage 748, and/or the multiapplication workflow execution engine 750 may reside in or execute as part of the operating system 710. For example, the multiapplication workflow engine 740 may involve printer drivers or other resources that also reside in or execute within the operating system 710.
  • The computing system 700 includes a power supply 716, which is powered by one or more batteries or other power sources and which provides power to other components of the computing system 700. The power supply 716 may also be connected to an external power source that overrides or recharges the built-in batteries or other power sources.
  • The computing system 700 may include one or more communication transceivers 730 which may be connected to one or more antenna(s) 732 to provide network connectivity (e.g., mobile phone network, Wi-Fi®, Bluetooth®, etc.) to one or more other servers and/or client devices (e.g., mobile devices, desktop computers, or laptop computers). The computing system 700 may further include a network adapter 736, which is a type of communication device. The computing system 700 may use the adapter and any other types of communication devices for establishing connections over a wide-area network (WAN) or local-area network (LAN). It should be appreciated that the network connections shown are exemplary and that other communications devices and means for establishing a communications link between the computing system 700 and other devices may be used.
  • The computing system 700 may include one or more input devices 734 such that a user may enter commands and information (e.g., a keyboard or mouse). These and other input devices may be coupled to the server by one or more interfaces 738 such as a serial port interface, parallel port, universal serial bus (USB), etc. The processing system 700 may further include a display 722 such as a touch screen display.
  • The computing system 700 may include a variety of tangible processor-readable storage media and intangible processor-readable communication signals. Tangible processor-readable storage can be embodied by any available media that can be accessed by the computing system 700 and includes both volatile and nonvolatile storage media, removable and non-removable storage media. Tangible processor-readable storage media excludes intangible communications signals and includes volatile and nonvolatile, removable and non-removable storage media implemented in any method or technology for storage of information such as processor-readable instructions, data structures, program modules or other data. Tangible processor-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other tangible medium which can be used to store the desired information and which can be accessed by the processing system 700. In contrast to tangible processor-readable storage media, intangible processor-readable communication signals may embody processor-readable instructions, data structures, program modules or other data resident in a modulated data signal, such as a carrier wave or other signal transport mechanism. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, intangible communication signals include signals traveling through wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.
  • An example multiapplication workflow engine is provided. The multiapplication workflow engine includes one or more computing processors, a request processing engine, a multiapplication workflow configuration manager, and a multiapplication workflow execution engine. The request processing engine is configured to receive an instruction to execute a multiapplication workflow via the one or more computing processors. The multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations. The interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions. The multiapplication workflow configuration manager is configured to instantiate one or more applications executed by the one or more computing processors, the one or more applications satisfying the interface definition and functionality definition for each of the two or more operations in a protected execution environment, the one or more applications each being configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow. The multiapplication workflow execution engine is configured to execute the multiapplication workflow via the one or more computing processors in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, further including a shared memory inaccessible to applications not executing within the protected execution environment of the multiapplication workflow. The compatible interfaces are configured for communication through the shared memory accessible to the applications executing in the protected execution environment.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the multiapplication workflow configuration manager associates a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the multiapplication workflow engine includes an application programming interface inaccessible to applications not executing within the protected execution environment of the multiapplication workflow. The compatible interfaces are configured for communication through the application programming interface accessible to the one or more applications executing in the protected execution environment.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the multiapplication workflow engine includes an application selection engine configured to search a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the application selection engine selects one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • An example multiapplication workflow engine of any previous multiapplication workflow engine is provided, wherein the request processing engine receives an instruction to execute the multiapplication workflow and each functionality definition provided by the multiapplication workflow includes one or more mandatory functionalities and one or more optional functionalities.
  • An example method includes receiving instruction to execute a multiapplication workflow. The multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations. The interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions. The example method further includes instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment. The applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and are ordered in accordance with execution order positions provided by the multiapplication workflow. The example method further includes executing the multiapplication workflow in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • An example method of any previous method is provided, wherein the compatible interfaces are configured for communication through a shared memory accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • An example method of any previous method includes associating a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
  • An example method of any previous method is provided, wherein the compatible interfaces are configured for communication through an application programming interface accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • The method of any previous method further includes searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • The method of any previous method further includes selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • The method of any previous method is provided, wherein each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
  • An example system includes means for receiving instruction to execute a multiapplication workflow. The multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations. The interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions. The example system further includes means for instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment. The applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and are ordered in accordance with execution order positions provided by the multiapplication workflow. The example system further includes means for executing the multiapplication workflow in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • An example system of any preceding system is provided, wherein the compatible interfaces are configured for communication through a shared memory accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • An example system of any preceding system includes means for associating a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
  • An example system of any preceding system is provided, wherein the compatible interfaces are configured for communication through an application programming interface accessible to the applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • An example system of any preceding system includes means for searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • An example system of any preceding system includes means for selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • An example system of any preceding system is provided, wherein each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
  • Example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a computing device a process for receiving instruction to execute a multiapplication workflow. The multiapplication workflow defines two or more operations and provides a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations. The interface definition corresponding to each operation is defined such that operations that are executed successively have compatible interface definitions. The process further includes instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment. The applications are each configured to communicate via compatible interfaces defined by the compatible interface definitions and are ordered in accordance with execution order positions provided by the multiapplication workflow. The process further includes executing the multiapplication workflow in the protected execution environment. Data is passed between the one or more applications through the compatible interfaces.
  • Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process wherein the compatible interfaces are configured for communication through a shared memory accessible to the one or more applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
  • Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process including associating a data input interface and a data output interface of each of the selected applications with the shared memory.
  • Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process wherein each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
  • Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process including searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
  • Another example one or more tangible processor-readable storage media are embodied with instructions for executing on one or more processors and circuits of a device a process of any preceding process including selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
  • Some implementations may comprise an article of manufacture. An article of manufacture may comprise a tangible storage medium to store logic. Examples of a storage medium may include one or more types of computer-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and so forth. Examples of the logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, operation segments, methods, procedures, software interfaces, application program interfaces (API), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. In one implementation, for example, an article of manufacture may store executable computer program instructions that, when executed by a computer, cause the computer to perform methods and/or operations in accordance with the described embodiments. The executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. The executable computer program instructions may be implemented according to a predefined computer language, manner or syntax, for instructing a computer to perform a certain operation segment. The instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.
  • The implementations described herein are implemented as logical steps in one or more computer systems. The logical operations may be implemented (1) as a sequence of processor-implemented steps executing in one or more computer systems and (2) as interconnected machine or circuit modules within one or more computer systems. The implementation is a matter of choice, dependent on the performance requirements of the computer system being utilized. Accordingly, the logical operations making up the implementations described herein are referred to variously as operations, steps, objects, or modules. Furthermore, it should be understood that logical operations may be performed in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.

Claims (20)

What is claimed is:
1. A multiapplication workflow engine comprising:
one or more computing processors;
a request processing engine configured to receive instruction to execute a multiapplication workflow via the one or more computing processors, the multiapplication workflow defining two or more operations and providing a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations, the interface definition corresponding to each operation being defined such that operations that are executed successively have compatible interface definitions;
a multiapplication workflow configuration manager configured to instantiate one or more applications executed by the one or more computing processors, the one or more applications satisfying the interface definition and functionality definition for each of the two or more operations in a protected execution environment, the one or more applications each being configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow; and
a multiapplication workflow execution engine configured to execute the multiapplication workflow via the one or more computing processors in the protected execution environment, data being passed between the one or more applications through the compatible interfaces.
2. The multiapplication workflow engine of claim 1 further comprising:
a shared memory inaccessible to applications not executing within the protected execution environment of the multiapplication workflow, the compatible interfaces being configured for communication through the shared memory accessible to the one or more applications executing in the protected execution environment.
3. The multiapplication workflow engine of claim 2, wherein the multiapplication workflow configuration manager is further configured to associate a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
4. The multiapplication workflow engine of claim 1, further comprising:
an application programming interface inaccessible to applications not executing within the protected execution environment of the multiapplication workflow, the compatible interfaces being configured for communication through the application programming interface accessible to the one or more applications executing in the protected execution environment.
5. The multiapplication workflow engine of claim 1, further comprising:
an application selection engine configured to search a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
6. The multiapplication workflow engine of claim 5, wherein the application selection engine is further configured to select one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
7. The multiapplication workflow engine of claim 1, wherein the request processing engine is further configured to receive an instruction to execute the multiapplication workflow, each functionality definition being provided by the multiapplication workflow including one or more mandatory functionalities and one or more optional functionalities.
8. A method comprising:
receiving instruction to execute a multiapplication workflow, the multiapplication workflow defining two or more operations and providing a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations, the interface definition corresponding to each operation being defined such that operations that are executed successively have compatible interface definitions;
instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment, the one or more applications each configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow; and
executing the multiapplication workflow in the protected execution environment, data being passed between the one or more applications through the compatible interfaces.
9. The method of claim 8, wherein the compatible interfaces are configured for communication through a shared memory accessible to the one or more applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
10. The method of claim 9, further comprising:
associating a data input interface and a data output interface of each of the instantiated applications with the shared memory provided by the multiapplication workflow.
11. The method of claim 8, wherein the compatible interfaces are configured for communication through an application programming interface accessible to the one or more applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
12. The method of claim 8, further comprising:
searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
13. The method of claim 12, further comprising:
selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
14. The method of claim 8, wherein each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
15. One or more tangible processor-readable storage media embodied with instructions for executing on one or more processors and circuits of a computing device a process comprising:
receiving instruction to execute a multiapplication workflow, the multiapplication workflow defining two or more operations and providing a functionality definition corresponding to each of the two or more operations and an interface definition corresponding to each of the two or more operations, the interface definition corresponding to each operation being defined such that operations that are executed successively have compatible interface definitions;
instantiating one or more applications that satisfy the interface definition and functionality definition for each of the two or more operations in a protected execution environment, the one or more applications each configured to communicate via compatible interfaces defined by the compatible interface definitions and being ordered in accordance with execution order positions provided by the multiapplication workflow; and
executing the multiapplication workflow in the protected execution environment, data being passed between the one or more applications through the compatible interfaces.
16. The one or more tangible processor-readable storage media of claim 15, wherein the compatible interfaces are configured for communication through a shared memory accessible to the one or more applications executing in the protected execution environment and inaccessible to applications not executing within the protected execution environment of the multiapplication workflow.
17. The one or more tangible processor-readable storage media of claim 16, wherein the process further comprises:
associating a data input interface and a data output interface of each of the selected applications with the shared memory.
18. The one or more tangible processor-readable storage media of claim 15, wherein each functionality definition includes one or more mandatory functionalities and one or more optional functionalities.
19. The one or more tangible processor-readable storage media of claim 15, wherein the process further comprises:
searching a catalog of indexed applications for one or more applications that satisfy the interface definition and the functionality definition for each of the two or more operations.
20. The one or more tangible processor-readable storage media of claim 15, wherein the process further comprises:
selecting one or more applications that satisfy the interface definition and the functionality definition corresponding to each of the two or more operations from the catalog of indexed applications.
US15/858,513 2017-09-28 2017-12-29 Software resource management Abandoned US20190095261A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/858,513 US20190095261A1 (en) 2017-09-28 2017-12-29 Software resource management
PCT/US2018/039645 WO2019067048A1 (en) 2017-09-28 2018-06-27 Software resource management

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762564888P 2017-09-28 2017-09-28
US15/858,513 US20190095261A1 (en) 2017-09-28 2017-12-29 Software resource management

Publications (1)

Publication Number Publication Date
US20190095261A1 true US20190095261A1 (en) 2019-03-28

Family

ID=65807530

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/828,050 Active 2038-03-03 US10592311B2 (en) 2017-09-28 2017-11-30 Computing device resource management
US15/858,513 Abandoned US20190095261A1 (en) 2017-09-28 2017-12-29 Software resource management

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US15/828,050 Active 2038-03-03 US10592311B2 (en) 2017-09-28 2017-11-30 Computing device resource management

Country Status (4)

Country Link
US (2) US10592311B2 (en)
EP (1) EP3688966B1 (en)
CN (1) CN111226428B (en)
WO (2) WO2019067045A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10592311B2 (en) 2017-09-28 2020-03-17 Microsoft Technology Licensing, Llc Computing device resource management
US11431817B2 (en) * 2018-12-04 2022-08-30 Samsung Electronics Co., Ltd. Method and apparatus for management of network based media processing functions

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11470162B2 (en) * 2021-01-30 2022-10-11 Zoom Video Communications, Inc. Intelligent configuration of personal endpoint devices
US11838333B2 (en) * 2022-05-02 2023-12-05 Dell Products L.P. Continuity of audio and/or video presets across different contexts

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030017718A1 (en) * 1999-07-23 2003-01-23 Matsushita Electric Industrial Co., Ltd. Method for forming interlayer dielectric film
US20120011222A1 (en) * 2009-05-01 2012-01-12 Kenta Yasukawa information processing system and method providing a composed service

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030177182A1 (en) 1999-06-14 2003-09-18 International Business Machines Corporation Ensuring a given transactional unit of work arrives at an appropriate server instance
US7680957B1 (en) * 2003-05-09 2010-03-16 Symantec Operating Corporation Computer system configuration representation and transfer
US20070011334A1 (en) 2003-11-03 2007-01-11 Steven Higgins Methods and apparatuses to provide composite applications
JP4820073B2 (en) * 2004-09-10 2011-11-24 ソニー株式会社 Information processing system, electronic device, information processing method, computer-processable program, and recording medium
US7694305B2 (en) * 2004-10-21 2010-04-06 Hewlett-Packard Development Company, L.P. Method of controlling access to computing resource within shared computing environment
US7831325B1 (en) * 2005-04-18 2010-11-09 Hewlett-Packard Development Company, L.P. Computing estimated performance of a software application in a target system
US8255883B2 (en) 2007-04-20 2012-08-28 Microsoft Corporation Translating late bound LINQ expressions into database queries
US8346974B2 (en) 2007-07-27 2013-01-01 Microsoft Corporation Hardware control interface for IEEE standard 802.11
US20090125977A1 (en) 2007-10-31 2009-05-14 Docomo Communications Laboratories Usa, Inc. Language framework and infrastructure for safe and composable applications
US8175863B1 (en) * 2008-02-13 2012-05-08 Quest Software, Inc. Systems and methods for analyzing performance of virtual environments
US8593476B2 (en) * 2008-02-13 2013-11-26 Gary Demos System for accurately and precisely representing image color information
US8271014B2 (en) * 2008-08-11 2012-09-18 Qualcomm Incorporated Automated parameter adjustment to compensate self adjusting transmit power and sensitivity level at the node B
US20100064357A1 (en) 2008-09-09 2010-03-11 Kerstin Baird Business Processing System Combining Human Workflow, Distributed Events, And Automated Processes
US20100153336A1 (en) * 2008-12-15 2010-06-17 Sony Ericsson Mobile Communications Ab Method and system for synchronizing information
US20120056910A1 (en) * 2010-08-30 2012-03-08 Qualcomm Incorporated Calibration of display for color response shifts at different luminance settings and for cross-talk between channels
JP2012150149A (en) * 2011-01-17 2012-08-09 Mitsubishi Electric Corp Multiscreen display device
US9110674B1 (en) * 2011-06-23 2015-08-18 Marvell International Ltd. Systems and methods for dynamic power management and performance adjustment of media playing
EP2590349B1 (en) * 2011-11-03 2014-10-15 NTT DoCoMo, Inc. A method for scheduling a scalable video coding stream in a wireless mobile network
US8531474B2 (en) * 2011-11-11 2013-09-10 Sharp Laboratories Of America, Inc. Methods, systems and apparatus for jointly calibrating multiple displays in a display ensemble
US20130339980A1 (en) 2012-06-14 2013-12-19 Clicksoftware Technologies Ltd Composite application enabling workflow between unmodified constituent applications
US10244021B2 (en) * 2013-12-06 2019-03-26 Harman International Industries, Inc. Media content and user experience delivery system
US9811393B2 (en) 2014-05-29 2017-11-07 Apple Inc. Consistent extension points to allow an extension to extend functionality of an application to another application
US10097902B2 (en) * 2015-05-06 2018-10-09 Blackfire Research Corporation System and method for using multiple audio input devices for synchronized and position-based audio
US20170331904A1 (en) 2016-05-12 2017-11-16 Avaya Inc. Maintenance of feature settings across multiple user systems on a communication
CN106506646B (en) * 2016-11-10 2020-01-14 Oppo广东移动通信有限公司 Control method and device of playing equipment, mobile terminal and playing system
US10592311B2 (en) 2017-09-28 2020-03-17 Microsoft Technology Licensing, Llc Computing device resource management

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030017718A1 (en) * 1999-07-23 2003-01-23 Matsushita Electric Industrial Co., Ltd. Method for forming interlayer dielectric film
US20120011222A1 (en) * 2009-05-01 2012-01-12 Kenta Yasukawa information processing system and method providing a composed service

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10592311B2 (en) 2017-09-28 2020-03-17 Microsoft Technology Licensing, Llc Computing device resource management
US11431817B2 (en) * 2018-12-04 2022-08-30 Samsung Electronics Co., Ltd. Method and apparatus for management of network based media processing functions

Also Published As

Publication number Publication date
CN111226428B (en) 2022-09-02
CN111226428A (en) 2020-06-02
WO2019067048A1 (en) 2019-04-04
EP3688966A1 (en) 2020-08-05
EP3688966B1 (en) 2022-07-13
US20190095251A1 (en) 2019-03-28
WO2019067045A1 (en) 2019-04-04
US10592311B2 (en) 2020-03-17

Similar Documents

Publication Publication Date Title
US11003630B2 (en) Remote access of metadata for collaborative documents
US10445083B2 (en) Application containers with dynamic sub-package loading
EP3584696B1 (en) Managing delivery of code and dependent data using application containers
US20190095261A1 (en) Software resource management
KR102146592B1 (en) Task completion through inter-application communication
US9513938B2 (en) Virtual appliance integration with cloud management software
US20180196665A1 (en) Managing, using, and updating application resources
US20180316772A1 (en) Brokering services from partner cloud platforms
US11019156B1 (en) Automatic discovery and registration of service applications for files introduced to a user interface
US20150331923A1 (en) Crm-based data migration system and method
US10572449B2 (en) Systems, devices, and methods for software discovery using application ID tags
US10778617B2 (en) Electronic device and method of transferring data in an application to another application
EP2779582A2 (en) Automatic mobile application discovery and integration in a mobile telecommunication device
US20130226975A1 (en) Method for file management and mobile device using the same
CN108776592B (en) Page construction method, device, equipment and storage medium
US8756254B2 (en) Integration of CRM applications to ECS application user interface
CN112416425B (en) APP construction method and device, computer equipment and storage medium
US11113097B2 (en) System and method for provisioning integration infrastructure at runtime indifferent to hybrid nature of endpoint applications
EP2972818B1 (en) Application registration and interaction
KR101907837B1 (en) Application activation framework
US20220035773A1 (en) Form field creation systems and methods
CN114817146A (en) Method and device for processing data
US8971212B2 (en) Identifying a resource geographically associated with one or more transceivers
CN106066874A (en) Object processing method and terminal
CN115422277A (en) Data source connection pool control method and device and server

Legal Events

Date Code Title Description
AS Assignment

Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WANG, BIN;ZHANG, DEJUN;ZHU, ROBERT YU;AND OTHERS;SIGNING DATES FROM 20171130 TO 20171201;REEL/FRAME:044507/0317

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE