WO2023224682A1 - Setting simultaneous focus across multiple operating systems - Google Patents

Setting simultaneous focus across multiple operating systems Download PDF

Info

Publication number
WO2023224682A1
WO2023224682A1 PCT/US2023/012979 US2023012979W WO2023224682A1 WO 2023224682 A1 WO2023224682 A1 WO 2023224682A1 US 2023012979 W US2023012979 W US 2023012979W WO 2023224682 A1 WO2023224682 A1 WO 2023224682A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
focus
operating system
guest
commanding
Prior art date
Application number
PCT/US2023/012979
Other languages
French (fr)
Inventor
Nick Eubanks
Philip John NACHREINER
Original Assignee
Microsoft Technology Licensing, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/852,145 external-priority patent/US20230376337A1/en
Application filed by Microsoft Technology Licensing, Llc filed Critical Microsoft Technology Licensing, Llc
Publication of WO2023224682A1 publication Critical patent/WO2023224682A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • G06F9/452Remote windowing, e.g. X-Window System, desktop virtualisation

Definitions

  • Native software applications are written for a single operating system (OS) and do not automatically run on other operating systems.
  • OS operating system
  • ANDROID does not automatically run on WINDOWS.
  • Running applications native to different operating systems on the same OS allows interactions between applications that would otherwise be impossible. For example, a user may be able to “drag and drop” content between a WINDOWS application and an ANDROID application.
  • Another advantage is increasing the number of available applications that can run on a particular operating system.
  • a native WINDOWS version of a legacy ANDROID application may not be available, so it would be desirable to seamlessly and naturally run the legacy ANDROID application on a WINDOWS device.
  • Virtual machines (VMs) and emulators allow a guest operating system to execute on the same computing device as a host operating system.
  • applications native to the host operating system and applications native to the guest operating system typically do not interoperate as well as two applications running on the same OS. For example, even if cut and paste is enabled between native applications and guest applications running in a VM, drag and drop may not be.
  • Traditional use of a VM also degrades the user experience by bringing the guest OS’s entire desktop into view when switching to a guest application.
  • a guest application running on the guest OS in a proxy application running on the host OS.
  • the proxy application looks and feels like an application that is native to the host OS.
  • User interface (UI) commands are forwarded from the proxy application to the guest application, allowing a user to control the guest application via the proxy application.
  • UI User interface
  • a user may not know that a proxied application is actually running on a guest OS.
  • an application running on the host OS may not be able to automate, debug, extract data from, or otherwise control a guest application that is being proxied.
  • This challenge is not limited to virtualization scenarios in which a guest application running in a guest OS is controlled by a debugger running on a host OS, but applies in any scenario where simultaneous focus between some number of applications, containers, virtual machines and/or other entities running on a system is desired.
  • the techniques disclosed herein enable a commanding application running on a primary OS to control a virtualized application running on a secondary OS when doing so requires that both applications have the focus of their respective operating systems.
  • an operating system provides focus to one application at a time.
  • a proxy application running on a primary OS virtualizes a guest application running on a secondary OS, the proxy application and the guest application gain and lose focus synchronously.
  • the guest application loses the focus of the secondary OS, the guest application can no longer receive commands, and so a commanding application running on the primary OS is unable to send commands to the guest application.
  • a fixed-focus system allows the commanding application to have the focus of the primary OS while the proxied guest application retains the focus of the secondary OS.
  • the secondary OS is a guest OS running on a hypervisor while the primary OS is the host OS of the hypervisor.
  • an individual application running on the secondary OS may be virtualized, appearing on the desktop of the primary OS.
  • a proxy application running on the primary OS renders the user interface of a guest application running on the secondary OS.
  • User interface events received by the proxy application are forwarded to the guest application.
  • the guest application appears and operates as a native application within the desktop of the primary OS.
  • a commanding application e.g., a debugger
  • a commanding application is an application running on the primary OS that performs an operation on the virtualized guest application.
  • the commanding application is not the proxy application used to virtualize the guest application.
  • an application hosting engine instantiates the proxy application, configures the proxy application to render the UI content of the guest application, and provides UI commands received by the proxy application to the guest application.
  • the application hosting engine may translate or reformat UI commands received by the proxy application to the equivalent UI commands of the secondary OS. For example, when the proxy application receives a primary OS mouse click event, the application hosting engine will translate or reformat the primary OS mouse click event as necessary to create a secondary OS mouse click event.
  • the application hosting engine then forwards the secondary OS mouse click event to the secondary OS.
  • the secondary OS forwards the secondary OS mouse click event to the application with the focus of the secondary OS.
  • a UI event refers to an OS generated response to user input, such as a mouse click event that is generated in response to a user clicking a mouse button.
  • a UI command refers to a message sent to an application, e.g., a key press event, or a function invocation that affects an application, e.g., a SetFocus() function that sets focus to a particular application.
  • Forwarding UI commands from the proxy application to the guest application may affect the focus of the guest application. Merely forwarding all UI commands, without otherwise modifying the focus of either OS, is often enough to maintain focus synchronicity. For example, if neither the proxy application or the guest application are in focus, then a primary OS mouse click received by the proxy application will bring it into focus. The application hosting engine will observe the primary OS mouse click, translate it into a secondary OS mouse click, and forward the secondary OS mouse click to the secondary OS. If the location of the guest application is synchronized with the location of the proxy application, then the secondary OS mouse click alone will give the guest application the focus of the secondary OS.
  • the proxy application when the proxy application receives a primary OS UI command that causes the proxy application to lose focus, the corresponding secondary OS UI command may also cause the guest application to lose focus.
  • Some application hosting engines go further than merely forwarding UI commands and hoping that focus remains synchronized by actively monitoring and correcting focus synchronicity.
  • UI events are handled in a focus-consistent way. For example, when the proxy application and the guest application are both in focus, any UI commands received by the proxy application are appropriately forwarded to the guest application. At the same time, when the proxy application loses focus, the guest application loses focus, and miscellaneous UI events on the secondary OS are prevented from affecting the guest application.
  • focus synchronicity ensures that the guest application has focus if and only if the host application has focus.
  • focus synchronicity renders certain scenarios impossible.
  • focus synchronicity may make it impossible for a debugger running on the primary OS to debug a virtualized guest application running on the secondary OS.
  • the debugger in order for a user to interact with the debugger, the debugger must have the focus of the primary OS.
  • the guest application in order to receive commands from the debugger, the guest application must have the focus of the secondary OS.
  • focus when focus is synchronized, the debugger and the guest application cannot simultaneously have focus of their respective operating systems.
  • the application hosting engine may enter a fixed focus mode in which the debugger maintains the focus of the primary OS while the target guest application maintains the focus of the secondary OS.
  • the application hosting engine may enter this fixed focus mode in response to manual user configuration, focus of the primary OS switching from a proxy application to the debugger while the debugger is debugging the corresponding guest application, the debugger otherwise gaining focus of the primary OS, or upon detecting a use of an application programming interface (API) that requires sending commands to the target guest application.
  • API application programming interface
  • Debugging is just one example of an operation that sends commands to a target guest application.
  • Other such operations include automating the target guest application, capturing a screen shot of the target guest application, controlling the target guest application with a usability API such as by a screen reader, or any other operation that is initiated on the primary OS and performed at least in part with the target guest application having the focus of the secondary OS.
  • fixed focus mode explicitly maintains focus on the commanding application in the primary OS while the target guest application maintains the focus of the secondary OS.
  • the application hosting engine may identify UI commands that could change the focus of either OS. The application hosting engine then counteracts, mitigates, and/or intercepts these commands.
  • a mouse click event received by the debugger may be identified as a UI command that, if forwarded to the secondary OS, could take the focus of the secondary OS away from the target guest application.
  • the fixed focus mode is applied by forwarding these commands to the secondary OS and then counteracting the change of focus they cause with a synthetic UI command.
  • the mouse click event may be forwarded to the secondary OS, causing the target guest application to lose focus, followed by a synthetic UI command that explicitly returns focus to the targeted guest application.
  • the UI command that affects focus is modified to perform some of the same effects while not changing the focus of the secondary OS. In other configurations the UI command that affects focus is not forwarded at all.
  • FIG. 1A illustrates an application hosting engine rendering a proxy application that contains content generated by a guest application.
  • FIG. IB illustrates an input command received by the proxy application.
  • FIG. 1C illustrates the input command of FIG. IB as forwarded to the guest application by the application hosting engine.
  • FIG. 2A illustrates a guest application and corresponding proxy application with synchronized focus.
  • FIG. 2B illustrates shifting focus on the primary OS to an application that is native to the primary OS while focus on the secondary OS remains unchanged.
  • FIG. 2C illustrates an application hosting engine setting the focus of the secondary OS to a placeholder application.
  • FIG. 3A illustrates a commanding application receiving the focus of the primary OS.
  • FIG. 3B illustrates the application hosting engine setting the focus of the secondary OS to a placeholder application.
  • FIG. 3C illustrates the application hosting engine setting focus of the secondary OS back to the target guest application in accordance with a fixed focus mode.
  • FIG. 3D illustrates the commanding application controlling the target guest application.
  • FIG. 3E illustrates resuming focus synchronization upon turning off the fixed focus mode.
  • FIG. 4 is a flow diagram of an example method for setting simultaneous focus across multiple operating systems.
  • FIG. 5 is a computer architecture diagram illustrating an illustrative computer hardware and software architecture for a computing system capable of implementing aspects of the techniques and technologies presented herein.
  • FIG. 6 is a diagram illustrating a distributed computing environment capable of implementing aspects of the techniques and technologies presented herein.
  • an application hosting engine may enter a fixed focus mode in which the focus of the primary OS is a commanding application while the focus of the secondary OS remains on a target guest application.
  • the application hosting engine may enter this fixed focus mode in response to the focus of the primary OS switching from a proxy application to a commanding application that controls, interacts with, or otherwise commands the guest application virtualized by the proxy application.
  • Debugging is one example of an operation that controls a target guest application, and therefore requires fixed focus mode.
  • Other controlling operations include automating the guest application, capturing a screen shot of the guest application, or any other operation that is initiated by a commanding application with the focus of the primary OS and performed at least in part while the guest application has the focus of the secondary OS.
  • Operating systems typically only support direct user interaction with one application at a time. Specifically, user input from a keyboard, mouse, finger, voice or other input mechanism is sent to a “focused application".
  • the focused application may also be referred to as a foreground application or an active application, and is typically distinguished visually from background applications that do not receive user input. For example, the operating system may render a focused application with a different title bar color than non-focused applications. Setting the focus of an operating system refers to selecting which application receives user input.
  • references to a focused application also refer to a focused window, or whatever aspect of a GUI the operating system directs user input to.
  • FIG. 1A illustrates an application hosting engine 150 rendering a proxy application 114 that contains content generated by a guest application 124.
  • Computing device 102 executes host operating system (OS) 110, which displays host operating desktop 112 on a display 160.
  • OS host operating system
  • Guest operating system 120 is concurrently executing on computing device 102.
  • Guest OS 120 may be running within a hypervisor, virtual machine, emulator, or the like.
  • Guest OS 120 renders content to guest OS desktop 122, which is also configured to render content to display 160.
  • guest OS desktop 122 and host OS desktop 112 often have the same dimensions, resolution, and other properties. However, desktop dimensions, resolutions, and scales do not have to match in order to implement focus synchronization.
  • Guest application 124 is running on guest OS 120. Guest application 124 renders content that would ordinarily be displayed by guest OS 120 on guest OS desktop 122. However, in order to create the impression that guest application 124 is a native application running on host OS 110 and allow increased functionality between guest application 124 and native applications running on host OS 110, application hosting engine 150 renders the content generated by guest application 124 in proxy application 114.
  • Proxy application 114 appears on host OS desktop 112 and operates and appears as a true native application of host OS 110.
  • application hosting engine 150 only renders content generated by guest OS 120 that is associated with a guest application that is proxied in host OS 110. That guest OS 120 and guest application 124 do not actually render to display 160 is indicated by dashed lines of guest OS desktop 122 and guest application 124.
  • guest OS 120 renders content to a pseudo display device that does not actually render content to a screen.
  • application hosting engine 150 provides guest application 124 with a graphics buffer into which content from guest application 124 is rendered. Proxy application 114 may then use this buffer when rendering content to host OS desktop 112.
  • guest OS 120 and host OS 110 are configured to use the same display 160, screen coordinates 126 and 116 are the same.
  • application hosting engine 150 will move guest application 124 to the corresponding location within guest OS desktop 122.
  • application hosting engine 150 will resize guest application 124 to the same width and height.
  • application hosting engine 150 may keep guest application 124 a constant size while allowing proxy application 114 to be resized. Updating the size and location of guest application 124 in this way is referred to herein as synchronizing size and location of proxy application 114 and guest application 124.
  • application hosting engine 150 synchronizes the size and location of proxy application 114 and guest application 124 by forwarding UI commands received by proxy application 114, such as mouse click and mouse move events, to guest application 124.
  • application hosting engine 150 maintains focus synchronicity by invoking an application program interface (API) to directly set the size and/or location of guest application 124.
  • API application program interface
  • FIG. IB illustrates an input command 146 received by the proxy application 114.
  • Input command 146 may be a mouse click, keyboard input, touch input, voice command, automation command, the invocation of an operating system user interface API, or any other technique for controlling the proxy application 114.
  • Input command 146 may be the beginning of a sequence of input commands, such as a double click, drag and drop, hover, or other user interface idiom.
  • FIG. 1C illustrates the input command 146 as provided to the guest application 124 by the application hosting engine 150.
  • Application hosting engine 150 provides UI command 146 to guest application 124 as input command 156.
  • UI command 156 may be a direct copy of UI command 146 or a translation to one or more corresponding UI commands recognized by applications running on guest OS 120.
  • application hosting engine 150 may create a mouse click event with coordinates (200, 250) but that is formatted for applications running on guest OS 120. In some configurations, application hosting engine 150 may then forward the newly created event to guest application 124. In other configurations, application hosting engine 150 forwards the newly created event to guest OS 120, which forwards it to guest application 124. Guest OS 120 may forward the newly created event to guest application 124 because guest application 124 happens to have the focus of guest OS 120, or because the newly created event specifically targets guest application 124. Instead of forwarding the mouse click event, application hosting engine 150 may invoke an automation API, OS API, or an application-specific API to cause guest application 124 to perform a function equivalent to the function that mouse click was expected to perform.
  • FIG. 2 A illustrates a guest photo application 124 and a corresponding proxy photo application 114 with synchronized focus. As illustrated, focus is indicated by the thick borders around proxy photo application 114 and guest photo application 124.
  • Proxy photo application 114 executes on primary operating system desktop 212 of primary operating system 210.
  • Primary operating system 210 may be a host operating system similar to host operating system 110 described above in conjunction with FIG 1A.
  • Native spreadsheet application 218 is also executing on primary OS desktop 212.
  • Native spreadsheet application 218 executes on primary OS 210 directly, and is not a proxy application to a corresponding guest application.
  • a taskbar 219 located at the bottom of primary OS desktop 212 contains icons of running applications.
  • OS 210 is referred to as a primary OS because when focus synchronicity is applied, the focus of the primary OS 210 is used to set the focus of secondary OS 220. Specifically, when focus synchronicity is applied - i.e., fixed focus mode is off- switching focus of the primary OS 210 to a proxy application 114 will cause the focus of the secondary OS 220 to switch to corresponding guest application 124. Similarly, when focus of the primary OS 210 switches away from a proxy application 114, the focus of the secondary OS 220 is switched away from the corresponding guest application 124.
  • Chat application icon 216 may be activated to launch a chat application. Chat application icon 216 may be a desktop icon or a taskbar 219 icon; or the chat application may be accessed in another manner (e.g., start menu, voice command, automation, other programmatic manipulation). If chat application icon 216 is associated with a native chat application, then activating it would simply launch the associated application. However, chat application icon 216 may be associated with a guest application that executes within secondary operating system 220. In this scenario, activating chat application icon 216 causes application hosting engine 150 to send an application launch command to secondary operating system 220. The application launch command causes secondary operating system 220 to launch a guest chat application.
  • Secondary operating system 220 renders secondary operating system desktop 222.
  • Guest photo application 124 has the focus of secondary operating system 220, as indicated by the thick border.
  • the dashed lines around guest photo application 124 indicate that the content drawn by guest photo application 124 is not rendered to the display by the guest photo application 124 directly. Instead, content drawn by the guest photo application 124 is rendered by corresponding proxy photo application 114.
  • Placeholder application 226 is a stub application launched by application hosting engine 150 to hold the focus of secondary operating system 220 when primary operating system 210 has given focus to a native application. This enables focus synchronization by allowing a guest application on secondary operating system 220 to have focus if and only if a corresponding proxy application on primary operating system 210 has focus. Placeholder application 226 does not render any content.
  • Application hosting engine 150 includes values associated with currently focused primary operating system application 252 and currently focused guest operating system application 254. These values hold identifiers of the applications that are currently in focus in their respective operating systems. As illustrated, photo application 114 currently has the focus of primary operating system 210, and photo application 124 currently has the focus of secondary operating system 220. These values are updated in response to events generated by the operating systems 210 and 220, as discussed in more detail below.
  • Application hosting engine 150 also includes fixed focus mode 202, which is set to off in FIG. 2A.
  • Application hosting engine 150 uses fixed focus mode 202 to track whether fixed focus mode is on or off.
  • Fixed focus mode prevents focus synchronicity between primary OS 210 and secondary OS 220, allowing different applications to have focus of primary OS 210 and secondary OS 220.
  • Application hosting engine 150 may prevent focus synchronicity by selectively filtering out UI commands that would synchronize focus, such as a mouse click event received by a commanding application running on primary OS 210. Instead of forwarding the mouse click event, application hosting engine 150 may instead do nothing.
  • Application hosting engine 150 may also prevent focus synchronicity by counter-acting a change of focus caused by a UI command.
  • application hosting engine 150 may forward a mouse click event to secondary OS 220, allowing the focus of secondary OS 220 to be synchronized, only to undo the change of focus by sending a synthetic UI command to secondary OS 220.
  • Application hosting engine 150 may also prevent focus synchronicity by altering a UI command so that it provides the intended effect except for changing focus.
  • FIG. 2B illustrates shifting focus in the primary OS 210 to an application 218 that is native to the primary OS 210 while focus of the secondary OS 220 remains unchanged.
  • focus of primary OS 210 has shifted to native spreadsheet application 218.
  • the focus may have shifted in response to a user input command such as a mouse click, keyboard command, voice command, touch gesture, or the like.
  • the focus also may have shifted as a result of automation or other programmatic manipulation.
  • the primary operating system 210 generates set focus event 260, which application hosting engine 150 receives and uses to determine that native spreadsheet application 218 now has the focus of primary operating system 210.
  • the value of currently focused primary OS app 252 is updated accordingly.
  • the focus of secondary operating system 220 remains on guest photo application 124.
  • set focus event 260 is processed synchronously by application hosting engine 150, ensuring that changes made to the focus of secondary operating system 220 are made before a user is able to change the focus of primary OS 210 again.
  • set focus event 260 is processed asynchronously by application hosting engine 150.
  • FIG. 2C illustrates application hosting engine 150 setting the focus of the secondary OS to a placeholder application.
  • application hosting engine 150 sends set focus command 270 to secondary operating system 220.
  • secondary operating system 220 gives the focus to placeholder application 226, as illustrated by the thick border around placeholder application 226.
  • Application hosting engine 150 has also updated the value of currently focused guest OS app 254 to store an identifier of placeholder 226.
  • Application hosting engine 150 sets the focus of secondary operating system 220 to placeholder application 226 so that the focus of the operating systems remains synchronized.
  • FIG. 3 A illustrates a commanding application 380 receiving the focus of the primary OS 210.
  • commanding application 380 and/or primary OS 210 send SetFocus event 361 to application hosting engine 150.
  • SetFocus event 361 may be a specific message or callback that application hosting engine 150 has registered with primary OS 210 to receive.
  • application hosting engine 150 may infer that debugger 380 has gained focus indirectly. For example, by monitoring window messages sent to debugger 380, such as WM MOUSECLICK and WM SETFOCUS, application hosting engine 150 may infer that debugger 380 is receiving focus.
  • commanding application 380 is a debugger, a tool used to inspect and control the execution of a target application.
  • debugger 380 In order to inspect and/or control the execution of a target application, debugger 380 must be able to interact with it - e.g., send UI commands to the target application, invoke operating system functions that inspect data and control execution of the target application, etc.
  • the commanding application was an automation driver it would need to be able to send UI commands to the target application in order to automate it
  • the commanding application was a screen capture application it would need to be able to send UI commands to the target application in order to obtain a screen shot.
  • Commanding applications need the target application to have the focus of the secondary OS because some secondary operating systems are only able to forward UI commands to the focused application.
  • the ANDROID operating system maintains a stack of running applications, and only the application at the top of the stack - i.e., the focused application - receives UI commands. If a target application does not have the focus of the secondary OS 220, any UI commands sent to the secondary OS 220 will not reach their intended target.
  • debugger 380 is only able to debug guest photo application 124 if debugger 380 has the focus of primary OS 210 and guest photo application 124 has the focus of secondary OS 220.
  • Fixed focus mode 202 may be entered into manually. For example, a user may manually change a setting exposed by application hosting engine 150 that enables fixed focus mode 202.
  • Fixed focus mode may be enabled for all applications running on primary OS 210, for applications of a particular type, for particular applications, or for specific instances of an application. Fixed focus mode may also be enabled based on whether an application is a native application of the primary OS 210 or a proxy application that is virtualizing a guest application of the secondary OS 220.
  • the commanding application may enter fixed focus mode when it receives focus. In some configurations, the commanding application enters fixed focus mode when it receives focus from a proxy application that corresponds to a guest application that the commanding application sends commands to.
  • fixed focus mode When an application not meeting the fixed focus criteria gains focus, fixed focus mode is turned off and focus synchronization resumes. In some configurations, when fixed focus mode is turned off, focus is allowed to synchronize naturally as UI commands are forwarded to the secondary OS 220. In other configurations, application hosting engine 150 re-synchronizes focus. For example, if the focus of primary OS 210 is set to a native application that does not require fixed focus, application hosting engine 150 may generate and send a synthetic command to secondary OS 220 that sets the focus of secondary OS 220 to placeholder application 226. If the focus of primary OS 210 is set to a proxy application 114 that does not require fixed focus, application hosting engine 150 may generate and send a synthetic command to secondary OS 220 that sets the focus of secondary OS 220 to a corresponding guest application 124.
  • Fixed focus mode 202 may also be entered automatically.
  • fixed focus mode 202 may be automatically entered if an application that gains focus is contained in a list of fixed focus applications 350.
  • List of fixed focus applications 350 is a predefined list of commanding applications that, when focused by primary OS 210, cause fixed focus mode 202 to be turned on automatically.
  • List of fixed focus applications 350 may identify applications by file name or by a unique identifier associated with each application, e.g., a globally unique identifier (GUID).
  • GUID globally unique identifier
  • List of fixed focus applications 350 may also identify applications by application type.
  • list 350 may include an entry indicating that fixed focus mode 202 should be turned on when any application of type “debugger” gains the focus of primary OS 210.
  • fixed focus mode 202 is automatically entered when the application gaining focus of the primary OS is included or otherwise indicated by list of fixed focus application 350 and any other fixed focus criteria apply.
  • fixed focus mode is entered automatically in response to an application invoking an API that indicates it is a commanding application.
  • application hosting engine 150 may detect that an application running on primary OS 210 is invoking a screen shot API.
  • application hosting engine 150 may turn on fixed focus mode 202 for the duration of the API call.
  • fixed focus mode 202 may be turned on for as long as the commanding application maintains focus.
  • the application in response to detecting that the application invoked an API that indicates it is a commanding application, the application is added to the list of fixed focus applications 350 such that the next time it receives focus, fixed focus mode 202 will be turned on and the focus of the secondary OS 220 will be preserved.
  • application hosting engine 150 stores an indication of which guest application had the focus of secondary OS 220 when the focus of the primary OS 210 switches to an application that may or may not be a commanding application. If the application that may or may not be a commanding application does not perform any operations that send commands to the secondary OS 220 or invoke APIs indicative of a commanding application, then focus synchronicity is allowed to continue and fixed-focus mode remains disabled. However, if the application attempts to send a command to the secondary OS or invoke an API indicative of a commanding application, then application hosting engine 150 may enable fixed focus mode in a “just in time” manner, switching the focus of the secondary OS 220 back to the previously focused guest application. This allows the commanding application 380 to send UI commands to the target application sooner than if commanding application 380 were added to list 350, in which case the commanding application 380 would have to lose and regain focus before fixed focus mode was entered.
  • application hosting engine 150 may set the focus of secondary OS 220 to placeholder application 226 since there is no corresponding guest application to debugger 380.
  • Application hosting engine 150 may also store an indication that guest photo application 124 had the focus before placeholder application 226.
  • debugger 380 may enter fixed focus mode 202 and bring the most recent guest application, i.e., guest photo application 124, into focus of secondary OS 220.
  • the API indicative of being a commanding application includes an identifier of a target guest application.
  • application hosting engine 150 may switch the focus of the secondary OS to the guest application identified by the API invocation.
  • fixed focus mode 202 may be automatically disabled when commanding application 380 loses focus.
  • fixed focus may remain on for the different application.
  • FIG. 3B illustrates the application hosting engine 150 setting the focus of the secondary OS 220 to a placeholder application 226.
  • application hosting engine 150 sends focus command 370 to secondary OS 220, targeting placeholder application 226.
  • Set focus command 370 may be a synthetic UI command, as there is no guest application that corresponds to debugger 380, and so there is no guest application to forward a WM SETFOCUS message or other UI command that focused debugger 380.
  • setting focus to a placeholder application is the default behavior in a focus synchronized system when primary OS 210 switches focus from a virtualized application such as proxy photo application 114 to a native application.
  • Switching the focus of secondary OS 220 to placeholder application 226 is allowed to happen if fixed focus mode 202 has not been enabled for debugger 380.
  • Switching the focus of secondary OS 220 to placeholder application 226 may also be allowed if fixed-focus mode is implemented by allowing focus-changing UI commands to be forwarded to secondary OS 220 followed by synthetic UI commands that undo the focus change.
  • application hosting engine 150 may allow a WM KILLFOCUS command received by proxy application 114 to be forwarded to guest application 124, causing guest application 124 to lose focus, followed by a synthetic UI command the restores the focus of secondary OS 220 to guest application 124.
  • a synthetic UI command is a UI command that originates from application hosting engine 150, and which is not forwarded or otherwise based on a UI command received by primary OS 210 or an application running on primary OS 210.
  • FIG. 3C illustrates the application hosting engine 150 setting focus of the secondary OS 220 to a target guest application 124 in accordance with fixed focus mode 202. Specifically, application hosting engine 150 sends fixed focus command 371 to secondary operating system 220, a synthetic command, setting focus back to guest application 124.
  • application hosting engine 150 may instead have filtered out any UI commands that would cause guest photo application 124 to lose focus. Additionally, or alternatively, when application hosting engine 150 implements focus synchronicity by explicitly setting the focus of secondary OS 220 - i.e., using synthetic commands to fix discrepancies in focus synchronicity - application hosting engine 150 may not perform these fixes. In either case, the focus of secondary OS 220 remains on guest photo application 124.
  • fixed focus command 371 is sent in response to debugger 380 invoking an API indicative of a commanding application.
  • attach to application command 381 represents the invocation of the attach API, which is indicative of a commanding application.
  • application hosting engine 150 enables fixed focus mode 202 and sends fixed focus command 371 to secondary OS 220.
  • attach to application command 381 may be sent, allowing a debug session to commence.
  • Other APIs indicative of a commanding application are APIs that take a screen shot, APIs that attach an automation driver, or any other API indicative of sending commands to a target guest application.
  • APIs indicative of a commanding application specify a target application, e.g., with a process name, process identifier, GUID, or the like.
  • application hosting engine 150 may set the focus of secondary OS 220 to the target application referenced in the API. For example, if the create screenshot API is invoked with a parameter of an application named “chat_application.exe”, application hosting engine 150 may send a fixed focus command 371 that brings “chat_application.exe” into focus on secondary OS 220.
  • FIG. 3D illustrates the commanding application 380 controlling the target guest application 124.
  • Debugger command 382 is sent from debugger 380 to secondary OS 220, which forwards it to the focused guest photo application 124. Because guest photo application 124 is focused, command 382 will be received by the intended recipient. In this way, a debugger running on primary OS 210 is enabled to debug a virtualized application running on secondary OS 220.
  • FIG. 3E illustrates resuming focus synchronization upon turning off the fixed focus mode 202.
  • Fixed focus mode 202 may have been turned off for any reason - a user may have manually turned it off, the commanding application 380 may have lost focus to a non-commanding application 218, the commanding application 380 may have been closed, or the commanding application 380 may have been determined to no longer be a commanding application.
  • proxy chat application 314 is focused in primary OS 210 while guest chat application 324 is focused in secondary OS 220.
  • Guest chat application 324 may have been launched while debugger 380 had the focus of primary OS 210 and guest photo application 124 had the focus of secondary OS 220.
  • a user may have activated chat app icon 216 while debugger 380 had the focus of primary OS 210.
  • debugger 380 would continue to have focus of primary OS 210, and fixed focus mode 202 would still be enabled.
  • application hosting engine 150 would display proxy chat application 314 in primary OS 210, taking focus away from debugger 380, and thereby ending fixed focus mode.
  • proxy chat application 314 would be displayed but not receive focus, and fixed focus mode 202 would continue until a user manually switched focus to proxy chat application 314.
  • Fixed focus mode may also be disabled in response to a user focusing a different native application 218 of primary OS 210. Once fixed focus mode 202 is disabled, focus synchronicity would resume between primary OS 210 and secondary OS 220.
  • FIG. 4 aspects of a routine for synchronizing focus across incompatible operating systems is shown and described.
  • the processes discussed in this disclosure are delineated as separate operations represented as independent blocks. However, these separately delineated operations should not be construed as necessarily order dependent in their performance.
  • the order in which the process is described is not intended to be construed as a limitation, and any number of the described process blocks may be combined in any order to implement the process or an alternate process. Moreover, it is also possible that one or more of the provided operations is modified or omitted.
  • routine 400 begins at operation 402 the focus of the secondary OS 220 is set to guest application 124 in response to the focus of the primary OS 210 being set to a corresponding proxy application 114.
  • a fixed focus mode 202 is entered that allows the guest application 124 to retain the focus 254 of the secondary OS 220 while the commanding application 380 retains the focus 252 of the primary OS 210.
  • a command 382 is forwarded from the commanding application 380 to the guest application 124.
  • the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system.
  • the implementation is a matter of choice dependent on the performance and other requirements of the computing system.
  • the logical operations described herein are referred to variously as states, operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof.
  • routine 400 is described herein as being implemented, at least in part, by modules running the features disclosed herein can be a dynamically linked library (DLL), a statically linked library, functionality produced by an application programing interface (API), a compiled program, an interpreted program, a script or any other executable set of instructions.
  • DLL dynamically linked library
  • API application programing interface
  • Data can be stored in a data structure in one or more memory components. Data can be retrieved from the data structure by addressing links or references to the data structure.
  • routine 400 may be also implemented in many other ways.
  • routine 400 may be implemented, at least in part, by a processor of another remote computer or a local circuit.
  • one or more of the operations of the routine 400 may alternatively or additionally be implemented, at least in part, by a chipset working alone or in conjunction with other software modules.
  • one or more modules of a computing system can receive and/or process the data disclosed herein. Any service, circuit or application suitable for providing the techniques disclosed herein can be used in operations described herein. FIG.
  • FIG. 5 shows additional details of an example computer architecture 500 for a device, such as a computer or a server configured as part of the systems described herein, capable of executing computer instructions (e.g., a module or a program component described herein).
  • the computer architecture 500 illustrated in FIG. 5 includes processing unit(s) 502, a system memory 504, including a random-access memory 506 (“RAM”) and a read-only memory (“ROM”) 508, and a system bus 510 that couples the memory 504 to the processing unit(s) 502.
  • RAM random-access memory
  • ROM read-only memory
  • Processing unit(s), such as processing unit(s) 502 can represent, for example, a CPU-type processing unit, a GPU-type processing unit, a field-programmable gate array (FPGA), another class of digital signal processor (DSP), or other hardware logic components that may, in some instances, be driven by a CPU.
  • FPGA field-programmable gate array
  • DSP digital signal processor
  • illustrative types of hardware logic components that can be used include Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASSPs), System-on-a-Chip Systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
  • ASICs Application-Specific Integrated Circuits
  • ASSPs Application-Specific Standard Products
  • SOCs System-on-a-Chip Systems
  • CPLDs Complex Programmable Logic Devices
  • the computer architecture 500 further includes a mass storage device 512 for storing an operating system 514, application(s) 516, modules 518, and other data described herein.
  • the mass storage device 512 is connected to processing unit(s) 502 through a mass storage controller connected to the bus 510.
  • the mass storage device 512 and its associated computer- readable media provide non-volatile storage for the computer architecture 500.
  • computer-readable media can be any available computer-readable storage media or communication media that can be accessed by the computer architecture 500.
  • Computer-readable media can include computer-readable storage media and/or communication media.
  • Computer-readable storage media can include one or more of volatile memory, nonvolatile memory, and/or other persistent and/or auxiliary computer storage media, removable and nonremovable computer storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.
  • computer storage media includes tangible and/or physical forms of media included in a device and/or hardware component that is part of a device or external to a device, including but not limited to random access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), phase change memory (PCM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable readonly memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), optical cards or other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage, magnetic cards or other magnetic storage devices or media, solid- state memory devices, storage arrays, network attached storage, storage area networks, hosted computer storage or any other storage memory, storage device, and/or storage medium that can be used to store and maintain information for access by a computing device.
  • RAM random access memory
  • SRAM static random-access memory
  • DRAM dynamic random-access memory
  • PCM phase change memory
  • ROM read-only memory
  • communication media can embody computer- readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism.
  • computer storage media does not include communication media. That is, computer-readable storage media does not include communications media consisting solely of a modulated data signal, a carrier wave, or a propagated signal, per se.
  • the computer architecture 500 may operate in a networked environment using logical connections to remote computers through the network 520.
  • the computer architecture 500 may connect to the network 520 through a network interface unit 522 connected to the bus 510.
  • the computer architecture 500 also may include an input/output controller 524 for receiving and processing input from a number of other devices, including a keyboard, mouse, touch, or electronic stylus or pen. Similarly, the input/output controller 524 may provide output to a display screen, a printer, or other type of output device.
  • the processing unit(s) 502 may be constructed from any number of transistors or other discrete circuit elements, which may individually or collectively assume any number of states. More specifically, the processing unit(s) 502 may operate as a finite-state machine, in response to executable instructions contained within the software modules disclosed herein. These computer-executable instructions may transform the processing unit(s) 502 by specifying how the processing unit(s) 502 transition between states, thereby transforming the transistors or other discrete hardware elements constituting the processing unit(s) 502.
  • FIG. 6 depicts an illustrative distributed computing environment 600 capable of executing the software components described herein.
  • the distributed computing environment 600 illustrated in FIG. 6 can be utilized to execute any aspects of the software components presented herein.
  • the distributed computing environment 600 can be utilized to execute aspects of the software components described herein.
  • the distributed computing environment 600 can include a computing environment 602 operating on, in communication with, or as part of the network 604.
  • the network 604 can include various access networks.
  • One or more client devices 606A-606N (hereinafter referred to collectively and/or generically as “clients 606” and also referred to herein as computing devices 606) can communicate with the computing environment 602 via the network 604.
  • the clients 606 include a computing device 606A such as a laptop computer, a desktop computer, or other computing device; a slate or tablet computing device (“tablet computing device”) 606B; a mobile computing device 606C such as a mobile telephone, a smart phone, or other mobile computing device; a server computer 606D; and/or other devices 606N. It should be understood that any number of clients 606 can communicate with the computing environment 602.
  • the computing environment 602 includes servers 608, data storage 610, and one or more network interfaces 612.
  • the servers 608 can host various services, virtual machines, portals, and/or other resources.
  • the servers 608 host virtual machines 614, Web portals 616, mailbox services 618, storage services 620, and/or, social networking services 622.
  • the servers 608 also can host other services, applications, portals, and/or other resources (“other resources”) 624.
  • the computing environment 602 can include the data storage 610.
  • the functionality of the data storage 610 is provided by one or more databases operating on, or in communication with, the network 604.
  • the functionality of the data storage 610 also can be provided by one or more servers configured to host data for the computing environment 602.
  • the data storage 610 can include, host, or provide one or more real or virtual datastores 626A-626N (hereinafter referred to collectively and/or generically as “datastores 626”).
  • the datastores 626 are configured to host data used or created by the servers 608 and/or other data.
  • the datastores 626 also can host or store web page documents, word documents, presentation documents, data structures, algorithms for execution by a recommendation engine, and/or other data utilized by any application program. Aspects of the datastores 626 may be associated with a service for storing files.
  • the computing environment 602 can communicate with, or be accessed by, the network interfaces 612.
  • the network interfaces 612 can include various types of network hardware and software for supporting communications between two or more computing devices including, but not limited to, the computing devices and the servers. It should be appreciated that the network interfaces 612 also may be utilized to connect to other types of networks and/or computer systems.
  • the distributed computing environment 600 described herein can provide any aspects of the software elements described herein with any number of virtual computing resources and/or other distributed computing functionality that can be configured to execute any aspects of the software components disclosed herein.
  • the distributed computing environment 600 provides the software functionality described herein as a service to the computing devices.
  • the computing devices can include real or virtual machines including, but not limited to, server computers, web servers, personal computers, mobile computing devices, smart phones, and/or other devices.
  • various configurations of the concepts and technologies disclosed herein enable any device configured to access the distributed computing environment 600 to utilize the functionality described herein for providing the techniques disclosed herein, among other aspects.
  • Example 1 A method comprising: setting a focus (254) of a secondary operating system (220) to a guest application (124) in response to a focus (252) of a primary operating system (210) being set to a corresponding proxy application (114); determining that the focus (252) of the primary operating system (210) has switched from the proxy application (114) to a commanding application (380); determining that the commanding application (380) satisfies a fixed focus mode criteria; entering a fixed focus mode (202) that allows the guest application (124) to retain the focus (254) of the secondary operating system (220) while the commanding application (380) retains the focus (252) of the primary operating system (210); and providing a command (382) from the commanding application (380) to the guest application (124).
  • Example 2 The method of example 1, wherein the proxy application virtualizes the guest application by rendering a graphical user interface generated by the guest application in a desktop of the primary operating system, and by forwarding user interface commands received by the proxy application to the guest application.
  • Example 3 The method of example 1, further comprising: switching the focus of the secondary operating system away from the guest application when the focus of the primary operating system is switched away from the proxy application.
  • Example 4 The method of example 3, wherein the focus of the secondary operating system is switched to a placeholder application.
  • Example 5 The method of example 1, wherein forwarding the command to the guest application comprises forwarding the command to the secondary operating system, wherein the secondary operating system forwards the command to the focused application.
  • Example 6 The method of example 1, wherein the commanding application comprises a debugger, an automation driver, or a screen capture application.
  • Example 7 The method of example 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether a name of the commanding application is included in a list of commanding applications.
  • Example 8 The method of example 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether an application type of the commanding application is included in a list of commanding application types.
  • Example 9 The method of example 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether the commanding application invokes one of a predefined list of application program interfaces associated with commanding applications.
  • Example 10 A computing device comprising: one or more processors; a memory in communication with the one or more processors, the memory having computer-readable instructions stored thereupon which, when executed by the one or more processors, cause the computing device to: set a focus (254) of a secondary operating system (220) to a guest application (124) in response to a focus (252) of a primary operating system (210) being set to a corresponding proxy application (114); determine that the focus (252) of the primary operating system (210) has switched from the proxy application (114) to a commanding application (380); synchronize focus of the primary operating system (210) and the secondary operating system (220) by switching the focus of the secondary operating system (220) away from the guest application (124); determine that the commanding application (380) invokes an application program interface (381) indicative of commanding applications; enter a fixed focus mode (202) that changes the focus of the secondary operating system (220) to the guest application (124) and that allows the guest application (124) to retain the focus (254) of the secondary operating system (220) while the
  • Example 12 The computing device of example 10, wherein fixed focus mode changes the focus of the secondary operating system to the guest application based on the guest application having the focus of the secondary operating system when the commanding application gained the focus of the primary operating system.
  • Example 13 The computing device of example 10, wherein fixed focus mode changes the focus of the secondary operating system to the guest application based on the guest application being targeted by the application program interface that is indicative of commanding applications.
  • Example 14 The computing device of example 10, wherein the instructions further cause the processor to: maintain focus synchronicity by setting the focus of the secondary operating system to an individual guest application if and only if the focus of the primary operating system is set to a corresponding individual proxy application, wherein entering the fixed focus mode suspends focus synchronicity.
  • Example 15 A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to: set a focus (254) of a secondary operating system (220) to a guest application (124) in response to a focus (252) of a primary operating system (210) being set to a corresponding proxy application (114); forward user interface commands (146) to the guest application (124); determine that the focus (252) of the primary operating system (210) has switched from the proxy application (114) to a commanding application (380); enter a fixed focus mode (202) that prevents user interface commands that would set the focus (254) of the secondary operating system (220) to the guest application (124) from being forwarded to the secondary operating system (220); and provide a command (382) from the commanding application (380) to the guest application (124).
  • Example 16 The computer-readable storage medium of example 15, wherein the primary operating system comprises a host operating system running on a hypervisor and the secondary operating system comprises a guest operating system running on a hypervisor.
  • Example 17 The computer-readable storage medium of example 15, wherein the instructions further cause the processor to: restore the focus of the secondary operating system to the guest application when the focus of the secondary operating system changes unexpectedly.
  • Example 18 The computer-readable storage medium of example 17, wherein focus is restored by sending a synthetic command that focuses the guest application to the secondary operating system.
  • Example 19 The computer-readable storage medium of example 15, wherein the fixed focus mode is exited in response to the focus of the primary operating system switching away from the commanding application.
  • Example 20 The computer-readable storage medium of example 19, wherein focus synchronicity resumes upon exiting fixed focus mode.
  • any reference to “first,” “second,” etc. elements within the Summary and/or Detailed Description is not intended to and should not be construed to necessarily correspond to any reference of “first,” “second,” etc. elements of the claims. Rather, any use of “first” and “second” within the Summary, Detailed Description, and/or claims may be used to distinguish between two different instances of the same element.

Abstract

The techniques disclosed herein enable a commanding application running on a primary OS to control a virtualized application running on a secondary OS when doing so requires that both applications have the focus of their respective operating systems. Ordinarily, an operating system focuses one application at a time. When a proxy application running on a primary OS virtualizes a guest application running on a secondary OS, the proxy application and the guest application gain and lose focus synchronously. However, when the guest application loses the focus of the secondary OS, the guest application can no longer receive commands, and so a commanding application running on the primary OS is unable to send commands to the guest application. In some configurations, a fixed-focus system allows the commanding application to have the focus of the primary OS while the proxied guest application retains the focus of the secondary OS.

Description

SETTING SIMULTANEOUS FOCUS ACROSS MULTIPLE OPERATING SYSTEMS
BACKGROUND
Native software applications are written for a single operating system (OS) and do not automatically run on other operating systems. For example, an application written for ANDROID does not automatically run on WINDOWS. However, there are many advantages to running applications native to different operating systems simultaneously on the same operating system. Running applications native to different operating systems on the same OS allows interactions between applications that would otherwise be impossible. For example, a user may be able to “drag and drop” content between a WINDOWS application and an ANDROID application. Another advantage is increasing the number of available applications that can run on a particular operating system. For example, when migrating a business from an ANDROID platform to a WINDOWS platform, a native WINDOWS version of a legacy ANDROID application may not be available, so it would be desirable to seamlessly and naturally run the legacy ANDROID application on a WINDOWS device.
Virtual machines (VMs) and emulators allow a guest operating system to execute on the same computing device as a host operating system. However, applications native to the host operating system and applications native to the guest operating system typically do not interoperate as well as two applications running on the same OS. For example, even if cut and paste is enabled between native applications and guest applications running in a VM, drag and drop may not be. Traditional use of a VM also degrades the user experience by bringing the guest OS’s entire desktop into view when switching to a guest application.
Some of these challenges are addressed by rendering a guest application running on the guest OS in a proxy application running on the host OS. The proxy application looks and feels like an application that is native to the host OS. User interface (UI) commands are forwarded from the proxy application to the guest application, allowing a user to control the guest application via the proxy application. In some cases, a user may not know that a proxied application is actually running on a guest OS.
However, this solution introduces additional challenges. For example, an application running on the host OS may not be able to automate, debug, extract data from, or otherwise control a guest application that is being proxied. This challenge is not limited to virtualization scenarios in which a guest application running in a guest OS is controlled by a debugger running on a host OS, but applies in any scenario where simultaneous focus between some number of applications, containers, virtual machines and/or other entities running on a system is desired.
It is with respect to these and other considerations that the disclosure made herein is presented. SUMMARY
The techniques disclosed herein enable a commanding application running on a primary OS to control a virtualized application running on a secondary OS when doing so requires that both applications have the focus of their respective operating systems. Ordinarily, an operating system provides focus to one application at a time. When a proxy application running on a primary OS virtualizes a guest application running on a secondary OS, the proxy application and the guest application gain and lose focus synchronously. However, when the guest application loses the focus of the secondary OS, the guest application can no longer receive commands, and so a commanding application running on the primary OS is unable to send commands to the guest application. In some configurations, a fixed-focus system allows the commanding application to have the focus of the primary OS while the proxied guest application retains the focus of the secondary OS.
In some configurations, the secondary OS is a guest OS running on a hypervisor while the primary OS is the host OS of the hypervisor. Instead of displaying the entire desktop of the secondary OS, an individual application running on the secondary OS may be virtualized, appearing on the desktop of the primary OS. Specifically, a proxy application running on the primary OS renders the user interface of a guest application running on the secondary OS. User interface events received by the proxy application are forwarded to the guest application. In this way, the guest application appears and operates as a native application within the desktop of the primary OS. While the example of a hypervisor concurrently running two operating systems on the same device is used throughout this disclosure, any other configuration in which two applications both have exclusive focus of the OS they run on are also contemplated. As referred to herein, a commanding application, e.g., a debugger, is an application running on the primary OS that performs an operation on the virtualized guest application. The commanding application is not the proxy application used to virtualize the guest application.
In some configurations, an application hosting engine instantiates the proxy application, configures the proxy application to render the UI content of the guest application, and provides UI commands received by the proxy application to the guest application. The application hosting engine may translate or reformat UI commands received by the proxy application to the equivalent UI commands of the secondary OS. For example, when the proxy application receives a primary OS mouse click event, the application hosting engine will translate or reformat the primary OS mouse click event as necessary to create a secondary OS mouse click event. The application hosting engine then forwards the secondary OS mouse click event to the secondary OS. The secondary OS forwards the secondary OS mouse click event to the application with the focus of the secondary OS. As referred to herein, a UI event refers to an OS generated response to user input, such as a mouse click event that is generated in response to a user clicking a mouse button. A UI command refers to a message sent to an application, e.g., a key press event, or a function invocation that affects an application, e.g., a SetFocus() function that sets focus to a particular application.
Forwarding UI commands from the proxy application to the guest application may affect the focus of the guest application. Merely forwarding all UI commands, without otherwise modifying the focus of either OS, is often enough to maintain focus synchronicity. For example, if neither the proxy application or the guest application are in focus, then a primary OS mouse click received by the proxy application will bring it into focus. The application hosting engine will observe the primary OS mouse click, translate it into a secondary OS mouse click, and forward the secondary OS mouse click to the secondary OS. If the location of the guest application is synchronized with the location of the proxy application, then the secondary OS mouse click alone will give the guest application the focus of the secondary OS. Similarly, when the proxy application receives a primary OS UI command that causes the proxy application to lose focus, the corresponding secondary OS UI command may also cause the guest application to lose focus. Some application hosting engines go further than merely forwarding UI commands and hoping that focus remains synchronized by actively monitoring and correcting focus synchronicity.
When focus of the proxy application and the guest application is synchronized in this way, UI events are handled in a focus-consistent way. For example, when the proxy application and the guest application are both in focus, any UI commands received by the proxy application are appropriately forwarded to the guest application. At the same time, when the proxy application loses focus, the guest application loses focus, and miscellaneous UI events on the secondary OS are prevented from affecting the guest application.
As discussed above, focus synchronicity, whether incidental or explicitly enforced, ensures that the guest application has focus if and only if the host application has focus. However, focus synchronicity renders certain scenarios impossible. For example, focus synchronicity may make it impossible for a debugger running on the primary OS to debug a virtualized guest application running on the secondary OS. Specifically, in order for a user to interact with the debugger, the debugger must have the focus of the primary OS. At the same time, in order to receive commands from the debugger, the guest application must have the focus of the secondary OS. However, when focus is synchronized, the debugger and the guest application cannot simultaneously have focus of their respective operating systems.
This challenge arises due to the nature of sending commands to a target application running on a secondary OS. Instead of being able to specify which application should receive a command, commands sent to a secondary OS are sent to whichever application has focus. As a result, when the debugger running on the primary OS gains focus, and focus synchronicity takes the focus away from the target application, debugging commands sent to the secondary OS will not be received by the target application.
To overcome this technological challenge and allow the debugger to debug the target guest application, the application hosting engine may enter a fixed focus mode in which the debugger maintains the focus of the primary OS while the target guest application maintains the focus of the secondary OS. The application hosting engine may enter this fixed focus mode in response to manual user configuration, focus of the primary OS switching from a proxy application to the debugger while the debugger is debugging the corresponding guest application, the debugger otherwise gaining focus of the primary OS, or upon detecting a use of an application programming interface (API) that requires sending commands to the target guest application. Debugging is just one example of an operation that sends commands to a target guest application. Other such operations include automating the target guest application, capturing a screen shot of the target guest application, controlling the target guest application with a usability API such as by a screen reader, or any other operation that is initiated on the primary OS and performed at least in part with the target guest application having the focus of the secondary OS.
In some configurations, fixed focus mode explicitly maintains focus on the commanding application in the primary OS while the target guest application maintains the focus of the secondary OS. While in fixed focus mode, the application hosting engine may identify UI commands that could change the focus of either OS. The application hosting engine then counteracts, mitigates, and/or intercepts these commands. Continuing the example of a debugger attached to a target guest application, a mouse click event received by the debugger may be identified as a UI command that, if forwarded to the secondary OS, could take the focus of the secondary OS away from the target guest application. In some configurations the fixed focus mode is applied by forwarding these commands to the secondary OS and then counteracting the change of focus they cause with a synthetic UI command. Continuing the example, the mouse click event may be forwarded to the secondary OS, causing the target guest application to lose focus, followed by a synthetic UI command that explicitly returns focus to the targeted guest application. In other configurations, the UI command that affects focus is modified to perform some of the same effects while not changing the focus of the secondary OS. In other configurations the UI command that affects focus is not forwarded at all.
By applying these techniques, the benefits of maintaining focus synchronicity can continue to be enjoyed when it does not interfere with operations such as debugging, automating, or screenshotting a target guest application. At the same time, fixed focus mode conserves computer processing and other resources as debugging, automating, screen capture, and other operations may be performed on the same device that is virtualizing the target guest application. This avoids the need for a costly second device, which would otherwise be required to perform these operations.
Features and technical benefits other than those explicitly described above will be apparent from a reading of the following Detailed Description and a review of the associated drawings. This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The term “techniques,” for instance, may refer to system(s), method(s), computer-readable instructions, module(s), algorithms, hardware logic, and/or operation(s) as permitted by the context described above and throughout the document.
BRIEF DESCRIPTION OF THE DRAWINGS
The Detailed Description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same reference numbers in different figures indicate similar or identical items. References made to individual items of a plurality of items can use a reference number with a letter of a sequence of letters to refer to each individual item. Generic references to the items may use the specific reference number without the sequence of letters.
FIG. 1A illustrates an application hosting engine rendering a proxy application that contains content generated by a guest application.
FIG. IB illustrates an input command received by the proxy application.
FIG. 1C illustrates the input command of FIG. IB as forwarded to the guest application by the application hosting engine.
FIG. 2A illustrates a guest application and corresponding proxy application with synchronized focus.
FIG. 2B illustrates shifting focus on the primary OS to an application that is native to the primary OS while focus on the secondary OS remains unchanged.
FIG. 2C illustrates an application hosting engine setting the focus of the secondary OS to a placeholder application.
FIG. 3A illustrates a commanding application receiving the focus of the primary OS.
FIG. 3B illustrates the application hosting engine setting the focus of the secondary OS to a placeholder application.
FIG. 3C illustrates the application hosting engine setting focus of the secondary OS back to the target guest application in accordance with a fixed focus mode. FIG. 3D illustrates the commanding application controlling the target guest application.
FIG. 3E illustrates resuming focus synchronization upon turning off the fixed focus mode.
FIG. 4 is a flow diagram of an example method for setting simultaneous focus across multiple operating systems.
FIG. 5 is a computer architecture diagram illustrating an illustrative computer hardware and software architecture for a computing system capable of implementing aspects of the techniques and technologies presented herein.
FIG. 6 is a diagram illustrating a distributed computing environment capable of implementing aspects of the techniques and technologies presented herein.
DETAILED DESCRIPTION
To address the above and other issues, techniques have been developed to selectively enter a fixed focus mode. While in the fixed focus mode, applications running on different operating systems but displayed concurrently in a single desktop may each have the focus of their respective operating systems. Specifically, an application hosting engine may enter a fixed focus mode in which the focus of the primary OS is a commanding application while the focus of the secondary OS remains on a target guest application. The application hosting engine may enter this fixed focus mode in response to the focus of the primary OS switching from a proxy application to a commanding application that controls, interacts with, or otherwise commands the guest application virtualized by the proxy application. Debugging is one example of an operation that controls a target guest application, and therefore requires fixed focus mode. Other controlling operations include automating the guest application, capturing a screen shot of the guest application, or any other operation that is initiated by a commanding application with the focus of the primary OS and performed at least in part while the guest application has the focus of the secondary OS.
Operating systems typically only support direct user interaction with one application at a time. Specifically, user input from a keyboard, mouse, finger, voice or other input mechanism is sent to a “focused application". The focused application may also be referred to as a foreground application or an active application, and is typically distinguished visually from background applications that do not receive user input. For example, the operating system may render a focused application with a different title bar color than non-focused applications. Setting the focus of an operating system refers to selecting which application receives user input. Throughout this document, references to a focused application also refer to a focused window, or whatever aspect of a GUI the operating system directs user input to.
FIG. 1A illustrates an application hosting engine 150 rendering a proxy application 114 that contains content generated by a guest application 124. Computing device 102 executes host operating system (OS) 110, which displays host operating desktop 112 on a display 160. Guest operating system 120 is concurrently executing on computing device 102. Guest OS 120 may be running within a hypervisor, virtual machine, emulator, or the like.
Guest OS 120 renders content to guest OS desktop 122, which is also configured to render content to display 160. As such, guest OS desktop 122 and host OS desktop 112 often have the same dimensions, resolution, and other properties. However, desktop dimensions, resolutions, and scales do not have to match in order to implement focus synchronization. Guest application 124 is running on guest OS 120. Guest application 124 renders content that would ordinarily be displayed by guest OS 120 on guest OS desktop 122. However, in order to create the impression that guest application 124 is a native application running on host OS 110 and allow increased functionality between guest application 124 and native applications running on host OS 110, application hosting engine 150 renders the content generated by guest application 124 in proxy application 114. Proxy application 114 appears on host OS desktop 112 and operates and appears as a true native application of host OS 110. In some configurations, application hosting engine 150 only renders content generated by guest OS 120 that is associated with a guest application that is proxied in host OS 110. That guest OS 120 and guest application 124 do not actually render to display 160 is indicated by dashed lines of guest OS desktop 122 and guest application 124. In some configurations, guest OS 120 renders content to a pseudo display device that does not actually render content to a screen. Specifically, application hosting engine 150 provides guest application 124 with a graphics buffer into which content from guest application 124 is rendered. Proxy application 114 may then use this buffer when rendering content to host OS desktop 112. Since, as illustrated, guest OS 120 and host OS 110 are configured to use the same display 160, screen coordinates 126 and 116 are the same. As proxy application 114 is moved around host OS desktop 112, application hosting engine 150 will move guest application 124 to the corresponding location within guest OS desktop 122. Similarly, in some embodiments, as proxy application 114 is resized within host desktop 112, application hosting engine 150 will resize guest application 124 to the same width and height. In other embodiments, application hosting engine 150 may keep guest application 124 a constant size while allowing proxy application 114 to be resized. Updating the size and location of guest application 124 in this way is referred to herein as synchronizing size and location of proxy application 114 and guest application 124.
In some configurations, application hosting engine 150 synchronizes the size and location of proxy application 114 and guest application 124 by forwarding UI commands received by proxy application 114, such as mouse click and mouse move events, to guest application 124. In other configurations, application hosting engine 150 maintains focus synchronicity by invoking an application program interface (API) to directly set the size and/or location of guest application 124.
FIG. IB illustrates an input command 146 received by the proxy application 114. Input command 146 may be a mouse click, keyboard input, touch input, voice command, automation command, the invocation of an operating system user interface API, or any other technique for controlling the proxy application 114. Input command 146 may be the beginning of a sequence of input commands, such as a double click, drag and drop, hover, or other user interface idiom.
FIG. 1C illustrates the input command 146 as provided to the guest application 124 by the application hosting engine 150. Application hosting engine 150 provides UI command 146 to guest application 124 as input command 156. UI command 156 may be a direct copy of UI command 146 or a translation to one or more corresponding UI commands recognized by applications running on guest OS 120.
For example, if proxy application 114 receives a mouse click event at coordinates (200, 250), application hosting engine 150 may create a mouse click event with coordinates (200, 250) but that is formatted for applications running on guest OS 120. In some configurations, application hosting engine 150 may then forward the newly created event to guest application 124. In other configurations, application hosting engine 150 forwards the newly created event to guest OS 120, which forwards it to guest application 124. Guest OS 120 may forward the newly created event to guest application 124 because guest application 124 happens to have the focus of guest OS 120, or because the newly created event specifically targets guest application 124. Instead of forwarding the mouse click event, application hosting engine 150 may invoke an automation API, OS API, or an application-specific API to cause guest application 124 to perform a function equivalent to the function that mouse click was expected to perform.
FIG. 2 A illustrates a guest photo application 124 and a corresponding proxy photo application 114 with synchronized focus. As illustrated, focus is indicated by the thick borders around proxy photo application 114 and guest photo application 124. Proxy photo application 114 executes on primary operating system desktop 212 of primary operating system 210. Primary operating system 210 may be a host operating system similar to host operating system 110 described above in conjunction with FIG 1A. Native spreadsheet application 218 is also executing on primary OS desktop 212. Native spreadsheet application 218 executes on primary OS 210 directly, and is not a proxy application to a corresponding guest application. A taskbar 219 located at the bottom of primary OS desktop 212 contains icons of running applications.
OS 210 is referred to as a primary OS because when focus synchronicity is applied, the focus of the primary OS 210 is used to set the focus of secondary OS 220. Specifically, when focus synchronicity is applied - i.e., fixed focus mode is off- switching focus of the primary OS 210 to a proxy application 114 will cause the focus of the secondary OS 220 to switch to corresponding guest application 124. Similarly, when focus of the primary OS 210 switches away from a proxy application 114, the focus of the secondary OS 220 is switched away from the corresponding guest application 124.
Chat application icon 216 may be activated to launch a chat application. Chat application icon 216 may be a desktop icon or a taskbar 219 icon; or the chat application may be accessed in another manner (e.g., start menu, voice command, automation, other programmatic manipulation). If chat application icon 216 is associated with a native chat application, then activating it would simply launch the associated application. However, chat application icon 216 may be associated with a guest application that executes within secondary operating system 220. In this scenario, activating chat application icon 216 causes application hosting engine 150 to send an application launch command to secondary operating system 220. The application launch command causes secondary operating system 220 to launch a guest chat application.
Secondary operating system 220 renders secondary operating system desktop 222. Guest photo application 124 has the focus of secondary operating system 220, as indicated by the thick border. The dashed lines around guest photo application 124 indicate that the content drawn by guest photo application 124 is not rendered to the display by the guest photo application 124 directly. Instead, content drawn by the guest photo application 124 is rendered by corresponding proxy photo application 114.
Placeholder application 226 is a stub application launched by application hosting engine 150 to hold the focus of secondary operating system 220 when primary operating system 210 has given focus to a native application. This enables focus synchronization by allowing a guest application on secondary operating system 220 to have focus if and only if a corresponding proxy application on primary operating system 210 has focus. Placeholder application 226 does not render any content.
Application hosting engine 150 includes values associated with currently focused primary operating system application 252 and currently focused guest operating system application 254. These values hold identifiers of the applications that are currently in focus in their respective operating systems. As illustrated, photo application 114 currently has the focus of primary operating system 210, and photo application 124 currently has the focus of secondary operating system 220. These values are updated in response to events generated by the operating systems 210 and 220, as discussed in more detail below.
Application hosting engine 150 also includes fixed focus mode 202, which is set to off in FIG. 2A. Application hosting engine 150 uses fixed focus mode 202 to track whether fixed focus mode is on or off. Fixed focus mode prevents focus synchronicity between primary OS 210 and secondary OS 220, allowing different applications to have focus of primary OS 210 and secondary OS 220. Application hosting engine 150 may prevent focus synchronicity by selectively filtering out UI commands that would synchronize focus, such as a mouse click event received by a commanding application running on primary OS 210. Instead of forwarding the mouse click event, application hosting engine 150 may instead do nothing. Application hosting engine 150 may also prevent focus synchronicity by counter-acting a change of focus caused by a UI command. For example, application hosting engine 150 may forward a mouse click event to secondary OS 220, allowing the focus of secondary OS 220 to be synchronized, only to undo the change of focus by sending a synthetic UI command to secondary OS 220. Application hosting engine 150 may also prevent focus synchronicity by altering a UI command so that it provides the intended effect except for changing focus.
FIG. 2B illustrates shifting focus in the primary OS 210 to an application 218 that is native to the primary OS 210 while focus of the secondary OS 220 remains unchanged. Specifically, focus of primary OS 210 has shifted to native spreadsheet application 218. The focus may have shifted in response to a user input command such as a mouse click, keyboard command, voice command, touch gesture, or the like. The focus also may have shifted as a result of automation or other programmatic manipulation.
The primary operating system 210 generates set focus event 260, which application hosting engine 150 receives and uses to determine that native spreadsheet application 218 now has the focus of primary operating system 210. The value of currently focused primary OS app 252 is updated accordingly. At the time depicted by FIG. 2B, the focus of secondary operating system 220 remains on guest photo application 124. In some configurations, set focus event 260 is processed synchronously by application hosting engine 150, ensuring that changes made to the focus of secondary operating system 220 are made before a user is able to change the focus of primary OS 210 again. In other configurations, set focus event 260 is processed asynchronously by application hosting engine 150.
FIG. 2C illustrates application hosting engine 150 setting the focus of the secondary OS to a placeholder application. Specifically, application hosting engine 150 sends set focus command 270 to secondary operating system 220. In response, secondary operating system 220 gives the focus to placeholder application 226, as illustrated by the thick border around placeholder application 226. Application hosting engine 150 has also updated the value of currently focused guest OS app 254 to store an identifier of placeholder 226. Application hosting engine 150 sets the focus of secondary operating system 220 to placeholder application 226 so that the focus of the operating systems remains synchronized.
FIG. 3 A illustrates a commanding application 380 receiving the focus of the primary OS 210. In some configurations, upon receiving the focus of primary OS 210, commanding application 380 and/or primary OS 210 send SetFocus event 361 to application hosting engine 150. SetFocus event 361 may be a specific message or callback that application hosting engine 150 has registered with primary OS 210 to receive. Additionally, or alternatively, application hosting engine 150 may infer that debugger 380 has gained focus indirectly. For example, by monitoring window messages sent to debugger 380, such as WM MOUSECLICK and WM SETFOCUS, application hosting engine 150 may infer that debugger 380 is receiving focus.
As illustrated, commanding application 380 is a debugger, a tool used to inspect and control the execution of a target application. In order to inspect and/or control the execution of a target application, debugger 380 must be able to interact with it - e.g., send UI commands to the target application, invoke operating system functions that inspect data and control execution of the target application, etc. Similarly, if the commanding application was an automation driver it would need to be able to send UI commands to the target application in order to automate it, and if the commanding application was a screen capture application it would need to be able to send UI commands to the target application in order to obtain a screen shot.
Commanding applications need the target application to have the focus of the secondary OS because some secondary operating systems are only able to forward UI commands to the focused application. For example, the ANDROID operating system maintains a stack of running applications, and only the application at the top of the stack - i.e., the focused application - receives UI commands. If a target application does not have the focus of the secondary OS 220, any UI commands sent to the secondary OS 220 will not reach their intended target. For example, debugger 380 is only able to debug guest photo application 124 if debugger 380 has the focus of primary OS 210 and guest photo application 124 has the focus of secondary OS 220.
Fixed focus mode 202 may be entered into manually. For example, a user may manually change a setting exposed by application hosting engine 150 that enables fixed focus mode 202. Fixed focus mode may be enabled for all applications running on primary OS 210, for applications of a particular type, for particular applications, or for specific instances of an application. Fixed focus mode may also be enabled based on whether an application is a native application of the primary OS 210 or a proxy application that is virtualizing a guest application of the secondary OS 220. Once fixed focus mode has been enabled for a commanding application, the commanding application may enter fixed focus mode when it receives focus. In some configurations, the commanding application enters fixed focus mode when it receives focus from a proxy application that corresponds to a guest application that the commanding application sends commands to.
When an application not meeting the fixed focus criteria gains focus, fixed focus mode is turned off and focus synchronization resumes. In some configurations, when fixed focus mode is turned off, focus is allowed to synchronize naturally as UI commands are forwarded to the secondary OS 220. In other configurations, application hosting engine 150 re-synchronizes focus. For example, if the focus of primary OS 210 is set to a native application that does not require fixed focus, application hosting engine 150 may generate and send a synthetic command to secondary OS 220 that sets the focus of secondary OS 220 to placeholder application 226. If the focus of primary OS 210 is set to a proxy application 114 that does not require fixed focus, application hosting engine 150 may generate and send a synthetic command to secondary OS 220 that sets the focus of secondary OS 220 to a corresponding guest application 124.
Fixed focus mode 202 may also be entered automatically. For example, fixed focus mode 202 may be automatically entered if an application that gains focus is contained in a list of fixed focus applications 350. List of fixed focus applications 350 is a predefined list of commanding applications that, when focused by primary OS 210, cause fixed focus mode 202 to be turned on automatically. List of fixed focus applications 350 may identify applications by file name or by a unique identifier associated with each application, e.g., a globally unique identifier (GUID). List of fixed focus applications 350 may also identify applications by application type. For example, list 350 may include an entry indicating that fixed focus mode 202 should be turned on when any application of type “debugger” gains the focus of primary OS 210. In some configurations, fixed focus mode 202 is automatically entered when the application gaining focus of the primary OS is included or otherwise indicated by list of fixed focus application 350 and any other fixed focus criteria apply.
In some configurations, fixed focus mode is entered automatically in response to an application invoking an API that indicates it is a commanding application. For example, application hosting engine 150 may detect that an application running on primary OS 210 is invoking a screen shot API. In response, application hosting engine 150 may turn on fixed focus mode 202 for the duration of the API call. Alternatively, fixed focus mode 202 may be turned on for as long as the commanding application maintains focus. In some configurations, in response to detecting that the application invoked an API that indicates it is a commanding application, the application is added to the list of fixed focus applications 350 such that the next time it receives focus, fixed focus mode 202 will be turned on and the focus of the secondary OS 220 will be preserved.
In some configurations, application hosting engine 150 stores an indication of which guest application had the focus of secondary OS 220 when the focus of the primary OS 210 switches to an application that may or may not be a commanding application. If the application that may or may not be a commanding application does not perform any operations that send commands to the secondary OS 220 or invoke APIs indicative of a commanding application, then focus synchronicity is allowed to continue and fixed-focus mode remains disabled. However, if the application attempts to send a command to the secondary OS or invoke an API indicative of a commanding application, then application hosting engine 150 may enable fixed focus mode in a “just in time” manner, switching the focus of the secondary OS 220 back to the previously focused guest application. This allows the commanding application 380 to send UI commands to the target application sooner than if commanding application 380 were added to list 350, in which case the commanding application 380 would have to lose and regain focus before fixed focus mode was entered.
For example, if guest photo application 124 had the focus of secondary OS 220 when debugger 380 gained the focus of primary OS 210, and if debugger 380 was not known to be a commanding application, application hosting engine 150 may set the focus of secondary OS 220 to placeholder application 226 since there is no corresponding guest application to debugger 380. Application hosting engine 150 may also store an indication that guest photo application 124 had the focus before placeholder application 226. Then, if debugger 380 is determined to be a commanding application, e.g., by detecting an attempt to send a command to secondary OS 220, or by detecting the invoking of an API indicative of commanding applications, application hosting engine 150 may enter fixed focus mode 202 and bring the most recent guest application, i.e., guest photo application 124, into focus of secondary OS 220.
In some configurations, the API indicative of being a commanding application includes an identifier of a target guest application. In this scenario, application hosting engine 150 may switch the focus of the secondary OS to the guest application identified by the API invocation.
If fixed focus mode 202 was enabled automatically, then fixed focus mode 202 may be automatically disabled when commanding application 380 loses focus. Alternatively, if the focus of primary OS 210 changes from commanding application 380 to a different application that requires fixed focus, fixed focus may remain on for the different application.
FIG. 3B illustrates the application hosting engine 150 setting the focus of the secondary OS 220 to a placeholder application 226. Specifically, application hosting engine 150 sends focus command 370 to secondary OS 220, targeting placeholder application 226. Set focus command 370 may be a synthetic UI command, as there is no guest application that corresponds to debugger 380, and so there is no guest application to forward a WM SETFOCUS message or other UI command that focused debugger 380.
In some configurations, setting focus to a placeholder application is the default behavior in a focus synchronized system when primary OS 210 switches focus from a virtualized application such as proxy photo application 114 to a native application. Switching the focus of secondary OS 220 to placeholder application 226 is allowed to happen if fixed focus mode 202 has not been enabled for debugger 380. Switching the focus of secondary OS 220 to placeholder application 226 may also be allowed if fixed-focus mode is implemented by allowing focus-changing UI commands to be forwarded to secondary OS 220 followed by synthetic UI commands that undo the focus change. For example, application hosting engine 150 may allow a WM KILLFOCUS command received by proxy application 114 to be forwarded to guest application 124, causing guest application 124 to lose focus, followed by a synthetic UI command the restores the focus of secondary OS 220 to guest application 124. As referred to herein, a synthetic UI command is a UI command that originates from application hosting engine 150, and which is not forwarded or otherwise based on a UI command received by primary OS 210 or an application running on primary OS 210.
FIG. 3C illustrates the application hosting engine 150 setting focus of the secondary OS 220 to a target guest application 124 in accordance with fixed focus mode 202. Specifically, application hosting engine 150 sends fixed focus command 371 to secondary operating system 220, a synthetic command, setting focus back to guest application 124.
In some configurations, instead of allowing the focus of secondary OS 220 to switch to placeholder application 226 when the focus of primary OS 210 was changed to debugger 380, application hosting engine 150 may instead have filtered out any UI commands that would cause guest photo application 124 to lose focus. Additionally, or alternatively, when application hosting engine 150 implements focus synchronicity by explicitly setting the focus of secondary OS 220 - i.e., using synthetic commands to fix discrepancies in focus synchronicity - application hosting engine 150 may not perform these fixes. In either case, the focus of secondary OS 220 remains on guest photo application 124.
In some configurations, fixed focus command 371 is sent in response to debugger 380 invoking an API indicative of a commanding application. As illustrated, attach to application command 381 represents the invocation of the attach API, which is indicative of a commanding application. In response to invoking the attach API, but before the attach API completes, application hosting engine 150 enables fixed focus mode 202 and sends fixed focus command 371 to secondary OS 220. Once fixed focus command 371 has completed, attach to application command 381 may be sent, allowing a debug session to commence. Other APIs indicative of a commanding application are APIs that take a screen shot, APIs that attach an automation driver, or any other API indicative of sending commands to a target guest application.
In some configurations, APIs indicative of a commanding application specify a target application, e.g., with a process name, process identifier, GUID, or the like. When application hosting engine 150 determines that commanding application 380 has invoked an API indicative of a commanding application, application hosting engine 150 may set the focus of secondary OS 220 to the target application referenced in the API. For example, if the create screenshot API is invoked with a parameter of an application named “chat_application.exe”, application hosting engine 150 may send a fixed focus command 371 that brings “chat_application.exe” into focus on secondary OS 220.
FIG. 3D illustrates the commanding application 380 controlling the target guest application 124. Debugger command 382 is sent from debugger 380 to secondary OS 220, which forwards it to the focused guest photo application 124. Because guest photo application 124 is focused, command 382 will be received by the intended recipient. In this way, a debugger running on primary OS 210 is enabled to debug a virtualized application running on secondary OS 220.
FIG. 3E illustrates resuming focus synchronization upon turning off the fixed focus mode 202. Fixed focus mode 202 may have been turned off for any reason - a user may have manually turned it off, the commanding application 380 may have lost focus to a non-commanding application 218, the commanding application 380 may have been closed, or the commanding application 380 may have been determined to no longer be a commanding application.
As illustrated, proxy chat application 314 is focused in primary OS 210 while guest chat application 324 is focused in secondary OS 220. Guest chat application 324 may have been launched while debugger 380 had the focus of primary OS 210 and guest photo application 124 had the focus of secondary OS 220. For example, a user may have activated chat app icon 216 while debugger 380 had the focus of primary OS 210. While guest chat application 324 is launched, debugger 380 would continue to have focus of primary OS 210, and fixed focus mode 202 would still be enabled. In some configurations, when guest chat application 324 finished loading, application hosting engine 150 would display proxy chat application 314 in primary OS 210, taking focus away from debugger 380, and thereby ending fixed focus mode. In other configurations, proxy chat application 314 would be displayed but not receive focus, and fixed focus mode 202 would continue until a user manually switched focus to proxy chat application 314. Fixed focus mode may also be disabled in response to a user focusing a different native application 218 of primary OS 210. Once fixed focus mode 202 is disabled, focus synchronicity would resume between primary OS 210 and secondary OS 220.
Turning now to FIG. 4, aspects of a routine for synchronizing focus across incompatible operating systems is shown and described. For ease of understanding, the processes discussed in this disclosure are delineated as separate operations represented as independent blocks. However, these separately delineated operations should not be construed as necessarily order dependent in their performance. The order in which the process is described is not intended to be construed as a limitation, and any number of the described process blocks may be combined in any order to implement the process or an alternate process. Moreover, it is also possible that one or more of the provided operations is modified or omitted.
With reference to FIG. 4, routine 400 begins at operation 402 the focus of the secondary OS 220 is set to guest application 124 in response to the focus of the primary OS 210 being set to a corresponding proxy application 114.
Next at operation 404, a determination is made that the focus 252 of the primary OS 210 has switched from the proxy application 114 to a commanding application 380.
Next at operation 406, a determination is made that the commanding application 380 satisfies a fixed focus mode criteria.
Next at operation 408, a fixed focus mode 202 is entered that allows the guest application 124 to retain the focus 254 of the secondary OS 220 while the commanding application 380 retains the focus 252 of the primary OS 210.
Proceeding to operation 410, a command 382 is forwarded from the commanding application 380 to the guest application 124.
The particular implementation of the technologies disclosed herein is a matter of choice dependent on the performance and other requirements of a computing device. Accordingly, the logical operations described herein are referred to variously as states, operations, structural devices, acts, or modules. These states, operations, structural devices, acts, and modules can be implemented in hardware, software, firmware, in special-purpose digital logic, and any combination thereof. It should be appreciated that more or fewer operations can be performed than shown in the figures and described herein. These operations can also be performed in a different order than those described herein.
It also should be understood that the illustrated methods can end at any time and need not be performed in their entireties. Some or all operations of the methods, and/or substantially equivalent operations, can be performed by execution of computer-readable instructions included on a computer-storage media, as defined below. The term “computer-readable instructions,” and variants thereof, as used in the description and claims, is used expansively herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
Thus, it should be appreciated that the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations described herein are referred to variously as states, operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof.
For example, the operations of the routine 400 are described herein as being implemented, at least in part, by modules running the features disclosed herein can be a dynamically linked library (DLL), a statically linked library, functionality produced by an application programing interface (API), a compiled program, an interpreted program, a script or any other executable set of instructions. Data can be stored in a data structure in one or more memory components. Data can be retrieved from the data structure by addressing links or references to the data structure.
Although the following illustration refers to the components of the figures, it should be appreciated that the operations of the routine 400 may be also implemented in many other ways. For example, the routine 400 may be implemented, at least in part, by a processor of another remote computer or a local circuit. In addition, one or more of the operations of the routine 400 may alternatively or additionally be implemented, at least in part, by a chipset working alone or in conjunction with other software modules. In the example described below, one or more modules of a computing system can receive and/or process the data disclosed herein. Any service, circuit or application suitable for providing the techniques disclosed herein can be used in operations described herein. FIG. 5 shows additional details of an example computer architecture 500 for a device, such as a computer or a server configured as part of the systems described herein, capable of executing computer instructions (e.g., a module or a program component described herein). The computer architecture 500 illustrated in FIG. 5 includes processing unit(s) 502, a system memory 504, including a random-access memory 506 (“RAM”) and a read-only memory (“ROM”) 508, and a system bus 510 that couples the memory 504 to the processing unit(s) 502.
Processing unit(s), such as processing unit(s) 502, can represent, for example, a CPU-type processing unit, a GPU-type processing unit, a field-programmable gate array (FPGA), another class of digital signal processor (DSP), or other hardware logic components that may, in some instances, be driven by a CPU. For example, and without limitation, illustrative types of hardware logic components that can be used include Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASSPs), System-on-a-Chip Systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
A basic input/output system containing the basic routines that help to transfer information between elements within the computer architecture 500, such as during startup, is stored in the ROM 508. The computer architecture 500 further includes a mass storage device 512 for storing an operating system 514, application(s) 516, modules 518, and other data described herein.
The mass storage device 512 is connected to processing unit(s) 502 through a mass storage controller connected to the bus 510. The mass storage device 512 and its associated computer- readable media provide non-volatile storage for the computer architecture 500. Although the description of computer-readable media contained herein refers to a mass storage device, it should be appreciated by those skilled in the art that computer-readable media can be any available computer-readable storage media or communication media that can be accessed by the computer architecture 500.
Computer-readable media can include computer-readable storage media and/or communication media. Computer-readable storage media can include one or more of volatile memory, nonvolatile memory, and/or other persistent and/or auxiliary computer storage media, removable and nonremovable computer storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Thus, computer storage media includes tangible and/or physical forms of media included in a device and/or hardware component that is part of a device or external to a device, including but not limited to random access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), phase change memory (PCM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable readonly memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), optical cards or other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage, magnetic cards or other magnetic storage devices or media, solid- state memory devices, storage arrays, network attached storage, storage area networks, hosted computer storage or any other storage memory, storage device, and/or storage medium that can be used to store and maintain information for access by a computing device.
In contrast to computer-readable storage media, communication media can embody computer- readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer storage media does not include communication media. That is, computer-readable storage media does not include communications media consisting solely of a modulated data signal, a carrier wave, or a propagated signal, per se.
According to various configurations, the computer architecture 500 may operate in a networked environment using logical connections to remote computers through the network 520. The computer architecture 500 may connect to the network 520 through a network interface unit 522 connected to the bus 510. The computer architecture 500 also may include an input/output controller 524 for receiving and processing input from a number of other devices, including a keyboard, mouse, touch, or electronic stylus or pen. Similarly, the input/output controller 524 may provide output to a display screen, a printer, or other type of output device.
It should be appreciated that the software components described herein may, when loaded into the processing unit(s) 502 and executed, transform the processing unit(s) 502 and the overall computer architecture 500 from a general-purpose computing system into a special-purpose computing system customized to facilitate the functionality presented herein. The processing unit(s) 502 may be constructed from any number of transistors or other discrete circuit elements, which may individually or collectively assume any number of states. More specifically, the processing unit(s) 502 may operate as a finite-state machine, in response to executable instructions contained within the software modules disclosed herein. These computer-executable instructions may transform the processing unit(s) 502 by specifying how the processing unit(s) 502 transition between states, thereby transforming the transistors or other discrete hardware elements constituting the processing unit(s) 502.
FIG. 6 depicts an illustrative distributed computing environment 600 capable of executing the software components described herein. Thus, the distributed computing environment 600 illustrated in FIG. 6 can be utilized to execute any aspects of the software components presented herein. For example, the distributed computing environment 600 can be utilized to execute aspects of the software components described herein.
Accordingly, the distributed computing environment 600 can include a computing environment 602 operating on, in communication with, or as part of the network 604. The network 604 can include various access networks. One or more client devices 606A-606N (hereinafter referred to collectively and/or generically as “clients 606” and also referred to herein as computing devices 606) can communicate with the computing environment 602 via the network 604. In one illustrated configuration, the clients 606 include a computing device 606A such as a laptop computer, a desktop computer, or other computing device; a slate or tablet computing device (“tablet computing device”) 606B; a mobile computing device 606C such as a mobile telephone, a smart phone, or other mobile computing device; a server computer 606D; and/or other devices 606N. It should be understood that any number of clients 606 can communicate with the computing environment 602.
In various examples, the computing environment 602 includes servers 608, data storage 610, and one or more network interfaces 612. The servers 608 can host various services, virtual machines, portals, and/or other resources. In the illustrated configuration, the servers 608 host virtual machines 614, Web portals 616, mailbox services 618, storage services 620, and/or, social networking services 622. As shown in FIG. 6 the servers 608 also can host other services, applications, portals, and/or other resources (“other resources”) 624.
As mentioned above, the computing environment 602 can include the data storage 610. According to various implementations, the functionality of the data storage 610 is provided by one or more databases operating on, or in communication with, the network 604. The functionality of the data storage 610 also can be provided by one or more servers configured to host data for the computing environment 602. The data storage 610 can include, host, or provide one or more real or virtual datastores 626A-626N (hereinafter referred to collectively and/or generically as “datastores 626”). The datastores 626 are configured to host data used or created by the servers 608 and/or other data. That is, the datastores 626 also can host or store web page documents, word documents, presentation documents, data structures, algorithms for execution by a recommendation engine, and/or other data utilized by any application program. Aspects of the datastores 626 may be associated with a service for storing files.
The computing environment 602 can communicate with, or be accessed by, the network interfaces 612. The network interfaces 612 can include various types of network hardware and software for supporting communications between two or more computing devices including, but not limited to, the computing devices and the servers. It should be appreciated that the network interfaces 612 also may be utilized to connect to other types of networks and/or computer systems.
It should be understood that the distributed computing environment 600 described herein can provide any aspects of the software elements described herein with any number of virtual computing resources and/or other distributed computing functionality that can be configured to execute any aspects of the software components disclosed herein. According to various implementations of the concepts and technologies disclosed herein, the distributed computing environment 600 provides the software functionality described herein as a service to the computing devices. It should be understood that the computing devices can include real or virtual machines including, but not limited to, server computers, web servers, personal computers, mobile computing devices, smart phones, and/or other devices. As such, various configurations of the concepts and technologies disclosed herein enable any device configured to access the distributed computing environment 600 to utilize the functionality described herein for providing the techniques disclosed herein, among other aspects.
The present disclosure is supplemented by the following example clauses.
Example 1 : A method comprising: setting a focus (254) of a secondary operating system (220) to a guest application (124) in response to a focus (252) of a primary operating system (210) being set to a corresponding proxy application (114); determining that the focus (252) of the primary operating system (210) has switched from the proxy application (114) to a commanding application (380); determining that the commanding application (380) satisfies a fixed focus mode criteria; entering a fixed focus mode (202) that allows the guest application (124) to retain the focus (254) of the secondary operating system (220) while the commanding application (380) retains the focus (252) of the primary operating system (210); and providing a command (382) from the commanding application (380) to the guest application (124). Example 2: The method of example 1, wherein the proxy application virtualizes the guest application by rendering a graphical user interface generated by the guest application in a desktop of the primary operating system, and by forwarding user interface commands received by the proxy application to the guest application.
Example 3: The method of example 1, further comprising: switching the focus of the secondary operating system away from the guest application when the focus of the primary operating system is switched away from the proxy application.
Example 4: The method of example 3, wherein the focus of the secondary operating system is switched to a placeholder application.
Example 5: The method of example 1, wherein forwarding the command to the guest application comprises forwarding the command to the secondary operating system, wherein the secondary operating system forwards the command to the focused application.
Example 6: The method of example 1, wherein the commanding application comprises a debugger, an automation driver, or a screen capture application.
Example 7: The method of example 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether a name of the commanding application is included in a list of commanding applications.
Example 8: The method of example 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether an application type of the commanding application is included in a list of commanding application types.
Example 9: The method of example 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether the commanding application invokes one of a predefined list of application program interfaces associated with commanding applications.
Example 10: A computing device comprising: one or more processors; a memory in communication with the one or more processors, the memory having computer-readable instructions stored thereupon which, when executed by the one or more processors, cause the computing device to: set a focus (254) of a secondary operating system (220) to a guest application (124) in response to a focus (252) of a primary operating system (210) being set to a corresponding proxy application (114); determine that the focus (252) of the primary operating system (210) has switched from the proxy application (114) to a commanding application (380); synchronize focus of the primary operating system (210) and the secondary operating system (220) by switching the focus of the secondary operating system (220) away from the guest application (124); determine that the commanding application (380) invokes an application program interface (381) indicative of commanding applications; enter a fixed focus mode (202) that changes the focus of the secondary operating system (220) to the guest application (124) and that allows the guest application (124) to retain the focus (254) of the secondary operating system (220) while the commanding application (380) retains the focus (252) of the primary operating system (210); and provide a command (382) from the commanding application (380) to the guest application (124). Example 11 : The computing device of example 10, wherein focus of the primary operating system and the secondary operating system is synchronized by switching the focus of the secondary operating system to a placeholder application.
Example 12: The computing device of example 10, wherein fixed focus mode changes the focus of the secondary operating system to the guest application based on the guest application having the focus of the secondary operating system when the commanding application gained the focus of the primary operating system.
Example 13: The computing device of example 10, wherein fixed focus mode changes the focus of the secondary operating system to the guest application based on the guest application being targeted by the application program interface that is indicative of commanding applications.
Example 14: The computing device of example 10, wherein the instructions further cause the processor to: maintain focus synchronicity by setting the focus of the secondary operating system to an individual guest application if and only if the focus of the primary operating system is set to a corresponding individual proxy application, wherein entering the fixed focus mode suspends focus synchronicity.
Example 15: A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to: set a focus (254) of a secondary operating system (220) to a guest application (124) in response to a focus (252) of a primary operating system (210) being set to a corresponding proxy application (114); forward user interface commands (146) to the guest application (124); determine that the focus (252) of the primary operating system (210) has switched from the proxy application (114) to a commanding application (380); enter a fixed focus mode (202) that prevents user interface commands that would set the focus (254) of the secondary operating system (220) to the guest application (124) from being forwarded to the secondary operating system (220); and provide a command (382) from the commanding application (380) to the guest application (124).
Example 16: The computer-readable storage medium of example 15, wherein the primary operating system comprises a host operating system running on a hypervisor and the secondary operating system comprises a guest operating system running on a hypervisor.
Example 17: The computer-readable storage medium of example 15, wherein the instructions further cause the processor to: restore the focus of the secondary operating system to the guest application when the focus of the secondary operating system changes unexpectedly. Example 18: The computer-readable storage medium of example 17, wherein focus is restored by sending a synthetic command that focuses the guest application to the secondary operating system. Example 19: The computer-readable storage medium of example 15, wherein the fixed focus mode is exited in response to the focus of the primary operating system switching away from the commanding application.
Example 20: The computer-readable storage medium of example 19, wherein focus synchronicity resumes upon exiting fixed focus mode.
While certain example embodiments have been described, these embodiments have been presented by way of example only and are not intended to limit the scope of the inventions disclosed herein. Thus, nothing in the foregoing description is intended to imply that any particular feature, characteristic, step, module, or block is necessary or indispensable. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions disclosed herein. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of certain of the inventions disclosed herein.
It should be appreciated that any reference to “first,” “second,” etc. elements within the Summary and/or Detailed Description is not intended to and should not be construed to necessarily correspond to any reference of “first,” “second,” etc. elements of the claims. Rather, any use of “first” and “second” within the Summary, Detailed Description, and/or claims may be used to distinguish between two different instances of the same element.
In closing, although the various techniques have been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended representations is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed subject matter.

Claims

1. A method comprising: setting a focus of a secondary operating system to a guest application in response to a focus of a primary operating system being set to a corresponding proxy application; determining that the focus of the primary operating system has switched from the proxy application to a commanding application; determining that the commanding application satisfies a fixed focus mode criteria; entering a fixed focus mode that allows the guest application to retain the focus of the secondary operating system while the commanding application retains the focus of the primary operating system; and providing a command from the commanding application to the guest application.
2. The method of claim 1, wherein the proxy application virtualizes the guest application by rendering a graphical user interface generated by the guest application in a desktop of the primary operating system, and by forwarding user interface commands received by the proxy application to the guest application.
3. The method of claim 1, further comprising: switching the focus of the secondary operating system away from the guest application when the focus of the primary operating system is switched away from the proxy application.
4. The method of claim 1, wherein forwarding the command to the guest application comprises forwarding the command to the secondary operating system, wherein the secondary operating system forwards the command to the focused application.
5. The method of claim 1, wherein the commanding application comprises a debugger, an automation driver, or a screen capture application.
6. The method of claim 1, wherein determining that the commanding application satisfies the fixed focus mode criteria comprises determining whether a name of the commanding application is included in a list of commanding applications, whether an application type of the commanding application is included in a list of commanding application types, or whether the commanding application invokes one of a predefined list of application program interfaces associated with commanding applications.
7. A computing device comprising: one or more processors; a memory in communication with the one or more processors, the memory having computer-readable instructions stored thereupon which, when executed by the one or more processors, cause the computing device to: set a focus of a secondary operating system to a guest application in response to a focus of a primary operating system being set to a corresponding proxy application; determine that the focus of the primary operating system has switched from the proxy application to a commanding application; synchronize focus of the primary operating system and the secondary operating system by switching the focus of the secondary operating system away from the guest application; determine that the commanding application invokes an application program interface indicative of commanding applications; enter a fixed focus mode that changes the focus of the secondary operating system to the guest application and that allows the guest application to retain the focus of the secondary operating system while the commanding application retains the focus of the primary operating system; and provide a command from the commanding application to the guest application.
8. The computing device of claim 7, wherein focus of the primary operating system and the secondary operating system is synchronized by switching the focus of the secondary operating system to a placeholder application.
9. The computing device of claim 7, wherein fixed focus mode changes the focus of the secondary operating system to the guest application based on the guest application having the focus of the secondary operating system when the commanding application gained the focus of the primary operating system or based on the guest application being targeted by the application program interface that is indicative of commanding applications.
10. The computing device of claim 7, wherein the instructions further cause the processor to: maintain focus synchronicity by setting the focus of the secondary operating system to an individual guest application if and only if the focus of the primary operating system is set to a corresponding individual proxy application, wherein entering the fixed focus mode suspends focus synchronicity.
11. A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to: set a focus of a secondary operating system to a guest application in response to a focus of a primary operating system being set to a corresponding proxy application; forward user interface commands to the guest application; determine that the focus of the primary operating system has switched from the proxy application to a commanding application; enter a fixed focus mode that prevents user interface commands that would set the focus of the secondary operating system to the guest application from being forwarded to the secondary operating system; and provide a command from the commanding application to the guest application.
12. The computer-readable storage medium of claim 11, wherein the primary operating system comprises a host operating system running on a hypervisor and the secondary operating system comprises a guest operating system running on a hypervisor.
13. The computer-readable storage medium of claim 11, wherein the instructions further cause the processor to: restore the focus of the secondary operating system to the guest application when the focus of the secondary operating system changes unexpectedly.
14. The computer-readable storage medium of claim 11, wherein focus is restored by sending a synthetic command that focuses the guest application to the secondary operating system.
15. The computer-readable storage medium of claim 11, wherein the fixed focus mode is exited in response to the focus of the primary operating system switching away from the commanding application, and , wherein focus synchronicity resumes upon exiting fixed focus mode.
PCT/US2023/012979 2022-05-20 2023-02-14 Setting simultaneous focus across multiple operating systems WO2023224682A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202263344505P 2022-05-20 2022-05-20
US63/344,505 2022-05-20
US17/852,145 2022-06-28
US17/852,145 US20230376337A1 (en) 2022-05-20 2022-06-28 Setting simultaneous focus across multiple operating systems

Publications (1)

Publication Number Publication Date
WO2023224682A1 true WO2023224682A1 (en) 2023-11-23

Family

ID=85685499

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/012979 WO2023224682A1 (en) 2022-05-20 2023-02-14 Setting simultaneous focus across multiple operating systems

Country Status (1)

Country Link
WO (1) WO2023224682A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019217196A2 (en) * 2018-05-07 2019-11-14 Apple Inc. Devices, methods, and graphical user interfaces for navigating between user interfaces, displaying a dock, and displaying system user interface elements
US20200275048A1 (en) * 2017-12-20 2020-08-27 Hisense Visual Technology Co., Ltd. Smart television and method for displaying graphical user interface of television screen shot

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200275048A1 (en) * 2017-12-20 2020-08-27 Hisense Visual Technology Co., Ltd. Smart television and method for displaying graphical user interface of television screen shot
WO2019217196A2 (en) * 2018-05-07 2019-11-14 Apple Inc. Devices, methods, and graphical user interfaces for navigating between user interfaces, displaying a dock, and displaying system user interface elements

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN XIANG 'ANTHONY' XIANGCHEN@ACM ORG ET AL: "Duet exploring joint interactions on a smart phone and a smart watch", PROCEEDINGS OF THE SIGCHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, ACM, 2 PENN PLAZA, SUITE 701 NEW YORK NY 10121-0701 USA, 26 April 2014 (2014-04-26), pages 159 - 168, XP058618513, ISBN: 978-1-4503-2473-1, DOI: 10.1145/2556288.2556955 *

Similar Documents

Publication Publication Date Title
US9588657B1 (en) Seamless integration of non-native windows with dynamically scalable resolution into host operating system
US9317195B1 (en) Seamless integration of non-native windows with dynamically scalable resolution into host operating system
US10200453B2 (en) Reverse seamless integration between local and remote computing environments
US8387048B1 (en) Seamless integration, migration and installation of non-native application into native operating system
US8819180B2 (en) Integrated handset browser settings
US7975236B1 (en) Seamless integration of non-native application into host operating system
US9400801B1 (en) System and method for local file management and editing utilizing remote applications
US9842091B2 (en) Switching to and from native web applications
US20140047360A1 (en) Background application page architecture for web applications
WO2014035936A2 (en) Reverse seamless integration between local and remote computing environments
US10291701B2 (en) Virtualizing a shortcut to a file in a cloud desktop
US11106380B2 (en) Migration of storage for workloads between desktop and cloud environments
US9804872B1 (en) Method for emulation of a virtual OS bookmark on a host desktop
US20230376337A1 (en) Setting simultaneous focus across multiple operating systems
WO2023224682A1 (en) Setting simultaneous focus across multiple operating systems
WO2022228236A1 (en) Method and apparatus for controlling android app, and terminal device
US10579373B2 (en) Seamless extension porting
US11010212B2 (en) System and method for supporting and negotiating multiple API versions across multiple products
US20240053996A1 (en) Bridging ui elements across multiple operating systems
WO2023224705A1 (en) Focus synchronization across incompatible operating systems
US11947860B2 (en) Mapping incompatible windowing topographies across operating systems
WO2024039477A1 (en) Bridging ui elements across multiple operating systems
WO2023224681A1 (en) Mapping incompatible windowing topographies across operating systems
US20230350532A1 (en) System and method for on-screen graphical user interface encapsulation and application history reproduction
US20210240552A1 (en) Supporting universal windows platform and win32 applications in kiosk mode

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23711590

Country of ref document: EP

Kind code of ref document: A1