EP3347805A1 - Schémas de commande dynamique pour applications actives simultanément - Google Patents

Schémas de commande dynamique pour applications actives simultanément

Info

Publication number
EP3347805A1
EP3347805A1 EP16738321.5A EP16738321A EP3347805A1 EP 3347805 A1 EP3347805 A1 EP 3347805A1 EP 16738321 A EP16738321 A EP 16738321A EP 3347805 A1 EP3347805 A1 EP 3347805A1
Authority
EP
European Patent Office
Prior art keywords
processor
control element
computing device
application
overlapping
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP16738321.5A
Other languages
German (de)
English (en)
Inventor
Theodore John BORROMEO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of EP3347805A1 publication Critical patent/EP3347805A1/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Definitions

  • GUI graphical user interface
  • the functions of the second app may not be fully accessible on a computing device. This can impact the user experience, especially when prompts requiring user acknowledgement are unpredictably overlaid on an already in-use application. Additionally, such situations may waste data and time, such as when streaming content is obscured from view due to overlaid GUI elements of an unexpected prompt.
  • Various embodiments provide methods, devices, systems, and non-transitory process-readable storage media for automatically configuring input mechanisms supported by a computing device to receive user inputs for simultaneously-active applications.
  • Various embodiment methods performed by a processor of a computing device may include determining whether an overlapping control condition exists among two (or more) simultaneously-active applications.
  • An overlapping control conditions may exist when a control element (referred to as a first control element) of a first application and a control element (referred to as a second control element) of a second application are configured to receive user inputs via a common input mechanism, such as the same button or overlapping input icons on a touch screen display.
  • the processor may identify available alternative input mechanisms supported by the computing device, such as alternative buttons available on the computing device or space on the touch screen display for rendering alternative input icons. Using this information, the processor may reconfigure one of the available alternative input mechanisms to support one of the applications by accepting user inputs for one of the first control element or the second control element while the overlapping control condition exists.
  • a determination that an overlapping control condition exists may be arrived at in response to determining that touch screen display coordinates corresponding to the first control element and the second control element overlap.
  • identifying available alternative input mechanisms on the computing device may include identifying input mechanisms supported by the computing device that are not currently configured to receive inputs for any control elements of the simultaneously-active applications.
  • the input mechanisms supported by the computing device may include a portion of a touch screen display, a hardware button on the computing device, a peripheral coupled to the computing device, a sound/voice recognition module, and a sensor coupled to the computing device.
  • reconfiguring one of the identified available alternative input mechanisms to support one of the applications may include identifying one of the first application or the second application as a losing application based on input profiles of the first application and the second application, identifying an alternative input mechanism based on an input profile of the losing application, and reconfiguring the identified alternative input mechanism to receive user inputs for an overlapping control element of the identified losing application while the overlapping control condition exists.
  • the input profiles of the first application and the second application may include priority information and preferred alternative input mechanisms.
  • reconfiguring one of the available alternative input mechanisms to support one of the applications may include rendering an additional visual element while the overlapping control condition exists, and reconfiguring a portion of a touch screen display corresponding to the additional visual element to receive user inputs for either the first control element or the second control element while the overlapping control condition exists.
  • the additional visual element may be a graphical user interface (GUI) button.
  • the method may further include rendering a notification identifying the available alternative input mechanism that is reconfigured to receive the user inputs for one of the first control element or the second control element while the overlapping control condition exists.
  • the rendered notification may include an interactive selection list, and the method may further include receiving a user selection on the interactive selection list, and reconfiguring another of the identified available alternative input mechanisms to receive the user inputs for one of the first control element or the second control element while the overlapping control condition exists based on the received user selection.
  • Further embodiments include a computing device configured with processor- executable instructions for performing operations of the methods described above. Further embodiments include a non-transitory processor-readable medium on which is stored processor-executable instructions configured to cause a computing device to perform operations of the methods described above.
  • FIGS. 1A-1C are diagrams of a mobile computing device display illustrating examples of overlapping control conditions.
  • FIGS. 2A-2C are diagrams of a mobile computing device display illustrating automatic remapping of input mechanisms to control elements according to some embodiments.
  • FIG. 3 is a diagram of a mobile computing device illustrating an input mechanism reconfiguration that includes rendering an additional visual element to provide user inputs to a control element according to some embodiments.
  • FIG. 4 is a diagram of a mobile computing device illustrating a visual indicator rendered in association with an application corresponding to a remapped input mechanism according to some embodiments.
  • FIG. 5 is a process flow diagram illustrating an embodiment method for a computing device to automatically configure input mechanisms for enabling user inputs to simultaneously-active applications with overlapping control inputs.
  • FIG. 6 is a process flow diagram illustrating an embodiment method for a computing device to render notifications and configure input mechanisms.
  • FIG.7 is a process flow diagram illustrating an embodiment method for automatically generating and configuring a new input mechanism for use with an application during an overlapping control condition.
  • FIG. 8 is a component block diagram of a mobile computing device suitable for use in an embodiment.
  • control inputs e.g., icons on a touch screen display
  • computing device is used herein to refer to an electronic device equipped with at least a processor.
  • Examples of computing devices may include computing devices (e.g., cellular telephones, wearable devices, smart-phones, web- pads, tablet computers, Internet enabled cellular telephones, Wi-Fi® enabled electronic devices, personal data assistants (PDA's), laptop computers, etc.), personal computers, and server computing devices.
  • computing devices e.g., cellular telephones, wearable devices, smart-phones, web- pads, tablet computers, Internet enabled cellular telephones, Wi-Fi® enabled electronic devices, personal data assistants (PDA's), laptop computers, etc.
  • PDA's personal data assistants
  • laptop computers etc.
  • computing devices may be configured with memory and/or storage as well as networking capabilities, such as network transceiver(s) and antenna(s) configured to establish a wide area network (WAN) connection (e.g., a cellular network connection, etc.) and/or a local area network (LAN) connection (e.g., a wired/wireless connection to the Internet via a Wi-Fi® router, etc.).
  • WAN wide area network
  • LAN local area network
  • FIG. 8 An exemplary smartphone-type mobile computing device is illustrated in FIG. 8.
  • input mechanism is used herein to refer to various hardware and/or software-generated elements supported by a computing device that enable a user to provide inputs for controlling application functionalities.
  • Examples of input mechanisms may include an icon positioned on a touch screen display or a portion of a touch screen display (e.g., a subset of the touch screen display that is designated for use by a function of an application, a GUI button, etc.), a mechanical, hardware, tactile, or physical component (e.g., a hardware button, a switch, a dial, etc.), peripherals (e.g., a mouse, a stylus, etc.), a sound/voice recognition module, and/or sensors (e.g., a camera, an accelerometer, a gyroscope, a light sensor, a microphone, motion sensor, temperature sensor, barometric pressure sensor, magnetic field sensor, etc.).
  • sensors e.g., a camera, an accelerometer, a gyroscope, a light sensor, a microphone,
  • Computing devices may identify user inputs via various supported input mechanisms, such as touch inputs on a touch screen display or hardware button depressions, and to adjust and/or route signals reporting such identified inputs to various functionalities of the computing device, particularly functions or features of software, instructions, routines, and/or operations executing on the computing device (e.g., operating system services, applications, etc.).
  • various supported input mechanisms such as touch inputs on a touch screen display or hardware button depressions
  • various functionalities of the computing device particularly functions or features of software, instructions, routines, and/or operations executing on the computing device (e.g., operating system services, applications, etc.).
  • the various embodiments may be executed to improve the user experience when applications executing on modern computing devices compete for the same space on a display and/or other input mechanism as a result of dynamic operations and/or scheduled operations, such as prompts or notifications.
  • An "overlapping control condition" may occur when applications simultaneously running on a computing device utilize common input mechanisms that cause an overlap. For example, this condition may occur when an active application is interrupted by prompts or alerts generated by a second application that may have been running in the background.
  • an overlapping control condition may occur when the use of one input mechanism of a first application (e.g., tactile button, graphical user interface (GUI) elements displayed on a touch input, etc.) is blocked by another input mechanism of a second application.
  • GUI graphical user interface
  • an overlapping control condition may exist when a play GUI button for a video application executing on a mobile device is rendered on top of a volume GUI button of a phone application executing on the mobile device.
  • an overlapping control condition may exist when two active applications attempt to configure the same tactile volume button to receive user inputs to two active applications.
  • Such overlapping control conditions may prevent the user from easily accessing all the functions or features of the competing applications. This may interrupt or otherwise hinder a user's ability to respond to one or more of the competing applications. For example, when interacting with a first application, a user may be prevented from continuing to interact with the first application when a prompt from a second application is suddenly rendered over top of the first application.
  • FIGS. 1A-1C illustrate an example scenario in which a computing device 100 experiences an overlapping control condition.
  • FIG. 1A shows a first application display 102 of a first application (e.g., a video application or "app") executing on the computing device 100.
  • the first application display 102 includes a progress slider control element 104 that may be controlled via touch inputs on the touch screen display 120.
  • a user may click and draw the progress slider control element 104 to change the current time of the video.
  • the user may place a finger on the slider control element 104 and move the finger in a left or right direction to "scrub" through a timeline of a playing video.
  • FIG. IB shows a second application display 150 (e.g., a battery popup app or dialog box) generated by a second application that includes a settings button control element 152a and a confirmation button control element 152b that may be controlled via touch inputs on the touch screen display 120.
  • the illustrated example second application display 150 might be rendered when the computing device 100
  • FIG. 1C illustrates the overlapping control condition that occurs when both application displays 102 and 150 are simultaneously active and rendered in common portions of the touch screen display 120.
  • the overlapping control condition one or more control elements of the applications are rendered in the same location on the touch screen display.
  • the settings button control element 152a of the second application display 150 and the progress slider control element 104 of the first application display 102 share the same area on a touch screen display, and the user interacts with both control elements by touching the touch screen display 120 at the position of the control elements 152a and 104.
  • the topmost control element rendered on a touch screen display is active.
  • the computing device will interpret that touch as an input on the settings button control element 152a of the second application display 150.
  • the overlapping control condition prevents the user from accessing the progress slider control element 104 to manipulate the video application.
  • the various embodiments provide methods, devices, and non-transitory process-readable storage media that automatically reassigns input mechanisms for one or more control elements of simultaneously-active applications during overlapping control conditions.
  • a processor of a computing device e.g., a smartphone computing device
  • Such evaluations may include identifying the currently used or assigned input mechanisms for each of the applications executing on the computing device and determining whether any of the input mechanisms of the applications overlap.
  • the processor may evaluate the touch screen display coordinates of all GUI buttons of all applications rendered on the touch screen display to determine whether any of the rendered GUI buttons overlap or otherwise conflict with one another.
  • the processor may identify alternative input mechanisms (e.g., hardware volume buttons, power button, sensors, etc.) that are not being used and thus are available for reconfiguring to provide a control element for one (or more) of the conflicting applications.
  • alternative input mechanisms e.g., hardware volume buttons, power button, sensors, etc.
  • the processor may identify portions of the touch screen display that are not being used, and thus represents real estate that could be used for repositioning a conflicting GUI button.
  • the processor may identify a physical button (e.g., a volume button on a smartphone) that is not associated with an input for an active application, and thus is available for reconfiguring to support a conflicting application.
  • the processor may automatically reconfigure (or remap) an available input mechanism to serve as a new input mechanism for one of the overlapping control elements.
  • Repositioning or remapping an application input to an available input mechanism may enable the user of the computing device to interact with both of the simultaneously running applications.
  • the computing device may automatically reassign an available volume button, home button, new GUI button, voice command, peripheral, and/or sensor to be used to receive user input for a functionality of an application (i.e., a "losing" application) in place of the
  • methods according to various embodiments may maintain simultaneous display and control of applications by providing dynamic, non-conflicting control schemes that enable users to continue to provide inputs to applications in case of overlapping control conditions.
  • the processor may identify input mechanisms assigned to each control element of applications executing on the computing device.
  • the processor may utilize input profiles associated with the various active applications in order to identify assigned or otherwise used input mechanisms for the applications.
  • the active application input profiles may include information about each application and/or individual control elements of each application. Such information may include data that may be used to identify priority information or rankings for the application and/or input mechanisms and/or control elements of applications, touch screen display coordinates of control elements, minimal or required input mechanisms, predefined alternative input mechanism mappings, and/or input characteristics associated with GUI elements of the
  • input profiles may include data that indicates the native, preferred, and/or alternative touch input type(s) for various control elements of applications.
  • the processor may utilize the input profiles to determine whether overlapping control conditions exist. For example, the processor may compare input mechanisms indicated in the input profiles of simultaneously-running applications to identify whether one or more overlaps of control elements exist. In one example, predefined GUI elements and respective screen locations that are indicated within the input profiles may be evaluated by the processor to detect whether sliders, buttons, etc. from a first application share the same area on the touch screen display as active GUI elements of a second application.
  • the processor may perform run-time evaluations of the input mechanisms used by various applications to identify overlapping control conditions. Such run-time operations may identify currently rendered (or "popped- up") GUI elements of various applications and may compare the screen space occupied by input icons of active applications and the popped up GUI elements to detect overlaps among the GUI elements. For example, the processor may compare the screen coordinates of each GUI button rendered by a first application to the coordinates of any GUI elements rendered by a second application in order to detect overlaps.
  • Such a run-time technique may be useful to application developers because it may enable use of dynamic, friendlier approaches to layout interfaces, such as by using XML-based approaches to design webpage layouts with fluid positions for user interface elements instead of specifying absolute positions or touch screen display coordinates and/or controls used by applications.
  • the processor may use input profiles to identify appropriate and available input mechanisms for remapping. For example, the processor may select a particular available button to receive user inputs to an application functionality by selecting from a stored list of preferred alternative input mechanisms.
  • the processor may use data of the input profiles to identify a priority between applications having overlapping control elements. For example, based on ranking data within the input profiles of a first application and a second application, the processor may determine that the second application has a lower priority, and therefore reconfigure an alternative input mechanism (e.g., a volume button) to support the conflicting control element of the second application.
  • rankings or priorities of applications may be based on occupied screen space (e.g., the application that is rendered on top may have the highest ranking, etc.).
  • the processor may resolve overlapping control conditions based on a first-come, first-served basis, allowing an application having an earlier active time (e.g., first to be launched, first to display, etc.) to maintain native (or preferred) input mechanism mappings while the second application to start is assigned alternative input mechanisms for conflicting control elements.
  • an application having an earlier active time e.g., first to be launched, first to display, etc.
  • the processor may resolve overlapping control conditions based on various other ranking or priority schemes, such as a last-come, first-served basis (e.g., latest to launch or be rendered may be allowed to maintain default input mechanisms), ranking data within input profiles of applications, user inputs or selections, or any combination of schemes.
  • the processor may perform remapping operations that include creating new, additional GUI elements on-the-fly for receiving user inputs for an application with conflicting control elements. For example, in response to determining that the input mechanisms for receiving inputs for a fast-forward functionality of a video player app should be reassigned due to an overlapping control condition, the processor may create a new GUI control element (e.g., a virtual button) that was not previously defined by the video player app. Such a new GUI control element may be touched by the user in order to provide an input to provide an input to the application that previously would have been provided by the conflicting control element.
  • a new GUI control element e.g., a virtual button
  • the remapping of input mechanisms may include remapping to different types of inputs.
  • the processor may map certain types of touch inputs (e.g., swipes, directional swipes, taps, tap/holds, etc.) to particular control elements of applications during overlapping control conditions. For example, when two GUI buttons overlap, the processor may not change tap input mapping a first one of the two GUI buttons, and reassign swipe touch inputs to the second GUI button.
  • the computing device processor may interpret or scrutinize the characteristics of user inputs to distinguish between intended inputs for applications when there are overlapping control conditions. For example, the processor may identify a touch input as a swipe (vs. a tap/hold) and apply that swipe input to whichever of the overlapping application GUI element of the conflicting applications that is mapped to only swipe inputs. Such a method may effectively disable unwanted interactions with either application based on how the user provides inputs.
  • the processor of the computing device may adjust various other characteristics of applications (and/or control elements) during overlapping control conditions. For example, the processor may adjust a transparency setting (e.g., an alpha channel) or other visual attributes of an application or individual control elements associated with remapped input mechanisms in order to promote visibility of an overlapped application. For example, while both a video app and a camera app are simultaneously-active and the camera app is determined by the processor to have a lower priority (or is otherwise considered a secondary app), the processor may render the camera app (e.g., a viewfinder screen of the camera app) in a semi-transparent manner over the video app. The use of semi-transparent rendering may enable a user continue watching a video via the video app while using the viewfinder for taking a picture via the camera app.
  • a transparency setting e.g., an alpha channel
  • other visual attributes of an application or individual control elements associated with remapped input mechanisms in order to promote visibility of an overlapped application.
  • the processor may render the camera app (e.g.,
  • the computing device may be configured to provide notifications and/or other informational elements to inform a user about overlapping control conditions and/or remapped input mechanisms.
  • the computing device processor may render over a control element of a losing application an icon or visual element that graphically conveys how the user can now control the function of the application.
  • the processor may render a symbol that depicts or points to a volume button currently configured to receive user inputs when a GUI button of a second application is in conflict with a GUI button of a first application.
  • the computing device may be configured to prompt a user to provide selections or other guidance to be used by the processor to adjust automatic remappings of input mechanisms for overlapping control elements. For example, when inputs for changing a volume for an application are remapped from the volume buttons to a voice recognition module, the computing device processor may render a selection list indicating that a shake movement (sensed via accelerometer) may be used as an alternative input mechanism for providing volume inputs.
  • the computing device processor may perform operations of the various embodiments via an input manager functionality.
  • an input manager functionality may be a dedicated process, routine, application, logic, and/or circuitry that is configured to identify overlapping control conditions and
  • a mobile computing device processor may be configured with processor-executable instructions to provide an input manager that performs operations of the various embodiment methods. Via the input manager, the computing device may evaluate various data in real-time to detect an overlapping control condition. Data that may be evaluated by the input manager may include comparing predefined data from input profiles of currently executing applications. The input manager may determine that an overlapping control condition exists when a first GUI button of a browser app and a second GUI button of a music player app share the same area on a touch screen display and are rendered
  • the input manager may determine that music player app is the "losing" application, and therefore the second GUI button needs to be moved or remapped.
  • the input manager may make this determination, for example, based on predefined rankings from the input profiles of the applications or another priority scheme (e.g., first-come, first-serve, etc.).
  • the input manager may identify currently available (i.e., not in use or reserved) input mechanisms on the tablet mobile computing device that can be configured to operate as a substitute input mechanism for the music player app. For example, the input manager may remap the input mechanism for the music player app to an accelerometer.
  • This remapping may enable the user to control the function of the music player app by moving the tablet mobile computing device (e.g., shake, bump, swing, etc.) instead of touching the second GUI button while the overlapping control condition persists.
  • This remapping may enable the user to control the browser app using the first GUI button while also manipulating an audio clip rendered via the music player app by shaking the computing device.
  • the input manager may configure the second GUI button to be used again for providing user inputs for the music player app.
  • the input mechanism may not be switched until the app is closed, or the user has expressly requested returning to the default input mechanism.
  • the embodiment methods avoid tedious pauses typically required for a user to adjust the applications, avoid providing inputs to unintended applications due to unexpected display, and avoid the need to reposition or resize GUI elements due to overlapping control conditions.
  • the embodiment methods may also improve the efficiency of computing devices and the user experience, as redundant operations to correct or overcome unintended actions may be avoided. For example, instead of re-rendering data missed due to an overlapping control condition that prevented the user from pausing a video playback, the computing device
  • implementing embodiment methods may make simultaneous controls available at all times so that the user may immediately pause the video as needed and on demand, regardless of overlapping of applications.
  • the various embodiments provide methods that improve the way a user interacts with applications executing on a computing device when control elements of a plurality of the applications overlap (e.g., GUI buttons rendered at least partially over top of each other).
  • control elements of a plurality of the applications overlap (e.g., GUI buttons rendered at least partially over top of each other).
  • the manner in which applications with conflicting control elements are displayed relative to other applications does not need to be modified. Instead, the input mechanisms assigned to applications may be reconfigured to support functionalities of the applications while there is an overlapping control condition.
  • the control elements of a "winning" application may be maintained (i.e., not readjusted) while the control elements of a "losing" application are reassigned or remapped to alternative input mechanisms.
  • the embodiment methods may be used with screen sharing schemes that involve movement or adjustment of windows on a display, but do not require movement of display elements. This is because the remapping of control elements to available input mechanisms does not involve or require visual movements of the applications.
  • the embodiment methods may be used to remap input icons for applications that naturally use all of a computing device display to other device sensors (e.g., buttons, accelerometers, gyroscopes, etc.), thereby allowing both applications full use of all functions.
  • both a computing device system device volume setting and a video player app volume setting may be natively controlled via hardware volume buttons. When the video player app is active, this may cause the use of the volume buttons to adjust only the video player app volume setting without affecting the system volume setting.
  • a processor of a computing device may reconfigure another input mechanism (e.g., a sensor input) to enable the user to control the system volume setting simultaneously with controlling the video player app volume setting with the volume buttons.
  • a computing device configured to implement embodiment techniques may also execute a video application for receiving and rendering a video stream (a first app) and a camera application for taking images/ videos (a second app). Both the first and second apps may use the touch screen display as the native (or default) input mechanism for receiving user inputs.
  • the first app may natively be configured to pause a video stream in response to a user tap on the touch screen display
  • the second app may natively be configured to zoom-in and/or zoom-out in response to a user pinch on the touch screen display.
  • the second app may also be natively configured to take a snapshot with the camera sensor in response to an input on software GUI button (e.g., a "snap" GUI button) and switch active cameras (e.g., front, rear) via a different software GUI button.
  • both apps may be natively configured to use the volume buttons.
  • the first app may adjust a playback volume of a video in response to a press on the hardware volume buttons and the second app may take a snapshot with the camera in response to a press of one of the hardware volume buttons.
  • Both the first app and the second app may be active simultaneously, such as when both are executing and rendered on the touch screen display in response to the user launching both apps.
  • the computing device experiences an overlapping control condition, because, as discussed above, the first app and the second app both use the same portion of the touch screen display for receiving inputs to pause and zoom/take pictures/switch active cameras, respectively.
  • the first app and the second app both use the volume buttons for user inputs; the first app using the buttons for volume control and the second app using a button as a shutter actuator.
  • the user when attempting to watch a video via the first app and take a picture with the second app, the user will not be able to both adjust the volume and take a snapshot with the volume buttons and will not be able to use the touch screen display to both zoom/take pictures/switch active cameras and continue watching a video.
  • the user To use the camera, the user must pause the video (e.g., the first app may interpret the zoom touch inputs for the second app to be a pause input for the first app). Otherwise, if the user presses a volume button in an attempt to take a snapshot, the camera app may not receive the input and instead the video app may change the volume.
  • a processor of the computing device may perform operations to remap either the volume adjustment functionality of the first app or the camera control (e.g., take a picture) function of the second app to another available input mechanism. Similarly, the processor may remap either the pause of the first app or the zoom in/out of the second app to another available input mechanism.
  • the computing device may support seamless picture-taking with the second app while simultaneously enabling a user to watch a video via the first app, as zooming, taking a snapshot, and switching active cameras may not pause the video and snapshots may be taken without affecting the video volume (or causing a pause).
  • the computing device may determine that the first app is the "winning" app (or has priority over the second app).
  • the device processor may reconfigure a hardware 'home' button to receive user inputs for zooming control of the second app, a voice command via the microphone to switch the active cameras, and a proximity sensor typically used to turn a screen off to receive user inputs to the shutter (or snapshot) control of the second app (e.g., when the user puts the computing device up close to a face, a snapshot may be taken).
  • the computing device processor may determine that the first app is the "losing" app (or has lower priority than the second app). In response to this determination, the device processor may configure a voice recognition functionality to receive user inputs for the volume function of the first app may configure an accelerometer unit to receive user inputs for the pause function of the first app.
  • the computing device processor may also render indicia of any input remappings, such as icons that indicate the new hardware, sensor, or other input mechanisms that are temporarily configured to receive user inputs for the control elements of the losing app. For example, a rendered message or sound may communicate that "pinch to zoom is active because the first app's pause is now associated with accelerometer input.”
  • FIGS. 2A-2C, 3, and 4 are diagrams illustrating scenarios in which a
  • FIGS. 2A-2C, 3, 4 illustrate different example input mechanism reconfigurations that may be performed by the computing device 100 to overcome overlapping control conditions in which the second application is the losing application (e.g., an intruding or lower priority application).
  • the second application is the losing application for illustration purposes in FIGS. 2A-2C, 3, 4, the input mechanisms may be similarly reconfigured to provide inputs to the progress slider control element 104 of the first application display 102 (e.g., in response to the second application having higher priority than the first application).
  • a processor of the computing device 100 may execute an input manager that is configured to identify the overlapping control condition and enact reconfigurations of input mechanisms with regard to the simultaneously active applications.
  • the overlapping control condition illustrated in FIGS. 2A-2C, 3, and 4 may be similar to the condition described above with reference to FIG. 1C.
  • a first application and a second application are illustrated as having simultaneously active (i.e., rendered) displays 102, 150 in common portions (i.e., overlapping) of the touch screen display 120 of the computing device 100.
  • FIG. 2A and IB involves a settings button control element 152a of the second application displays 150 and a progress slider control element 104 of the first application display 102 that overlap (i.e., appear at least partially in the same area of the touch screen display).
  • the illustrated overlapping control condition in FIG. 2C involves a confirmation button control element 152b of the second application displays 150 and the progress slider control element 104 of the first application display 102.
  • FIG. 2A illustrates an example reconfiguration in which the computing device 100 has reconfigured the first volume button 110a (e.g., a volume "down” hardware button) to be used to provide user inputs to the settings button control element 152a while the overlapping control condition exists.
  • the user can depress the volume "down” button in order to control the settings functionality of the second application.
  • FIG. IB illustrates an example reconfiguration in which the computing device 100 has reconfigured the second volume button 110b (e.g., a volume "up” hardware button) to be used to provide user inputs to the settings button control element 152a while the overlapping control condition exists.
  • the user can depress the volume "up” button in order to control the settings functionality of the second application.
  • FIG. 2C illustrates an embodiment reconfiguration in which the computing device 100 has reconfigured the first volume button 110a (e.g., a volume "down" hardware button) to be used to provide user inputs to the confirmation button control element 152b while the overlapping control condition exists.
  • the computing device 100 may automatically perform a remapping to enable the user to depress the volume "down" button in order to control the confirmation functionality of the second application.
  • the computing device 100 may also perform operations to enable the settings button control element 152a to receive touch inputs when the progress slider control element 104 no longer overlaps on the touch screen display. For example, the computing device 100 may reapply default or native mappings for the settings button control element 152a when there is no longer an overlapping control condition related to the settings button control element 152a.
  • FIG. 3 illustrates an example reconfiguration in which the computing device 100 has rendered an additional visual element 302 (e.g., a GUI button) that can be used to provide user inputs to the settings button control element 152a while the overlapping control condition exists.
  • an additional visual element 302 may be displayed on a portion of the touch screen display 120 that is not currently associated with any other control element of the application displays 102, 150.
  • visual information may be rendered to inform the user of the computing device 100 regarding when and how the computing device 100 has automatically remapped an input mechanism during an overlapping control condition.
  • FIG. 4 illustrates an example visual indicator 402 (e.g., a speaker image) that may be rendered to show that the first volume button 110a has been remapped to provide user inputs to the settings button control element 152a during an overlapping control condition.
  • the computing device 100 may also render other informative graphics, such as an arrow 404 or other directional information that may further inform the user about to the new input mechanism mapping.
  • FIG. 4 illustrates an example visual indicator 402 (e.g., a speaker image) that may be rendered to show that the first volume button 110a has been remapped to provide user inputs to the settings button control element 152a during an overlapping control condition.
  • the computing device 100 may also render other informative graphics, such as an arrow 404 or other directional information that may further inform the user about to the new input mechanism mapping.
  • FIG. 5 illustrates an embodiment method 500 for a processor of a computing device to automatically reconfigure input mechanisms supported by the computing device for providing user inputs to simultaneously-active applications during overlapping control conditions.
  • the method 500 may be performed by processors of various computing devices, such as the computing device 100 as described with reference to FIGS. 1A-4. Further, although the descriptions may refer to a first and second application for simplicity purposes, the method 500 may also be used to manage overlapping control conditions when any number of applications are active (and rendered) on the computing device.
  • the method 500 may be performed continuously whenever two or more applications are executing simultaneously. Further, the method 500 may be performed to reconfigure input mechanisms for one or more overlapping control conditions between two or more simultaneously-running applications. For example, the computing device may perform the method 500 to overcome an overlapping control condition regarding a first set of GUI buttons of a first and second application as well as an overlapping control condition regarding a second set of GUI buttons of the first and second application.
  • the processor of the computing device may determine whether an overlapping control condition exists corresponding to a first application and a second application executing on the computing device at a given time. In other words, the processor may determine whether there are one or more overlapping control elements for the simultaneously-active applications (e.g., a video app rendering on a display while a battery warning application runs as a background process for evaluating battery state, etc.).
  • the processor may determine whether there are one or more overlapping control elements for the simultaneously-active applications (e.g., a video app rendering on a display while a battery warning application runs as a background process for evaluating battery state, etc.).
  • the processor may determine whether a first control element of the first application (e.g., a first GUI button for controlling a video rendering functionality of the first application, etc.) and a second control element of the second application (e.g., a second GUI button for controlling a settings functionality of the second application, etc.) are configured to receive respective user inputs via a common input mechanism, such as the same volume button.
  • the processor may determine whether touch screen display coordinates corresponding to the first control element of the first application and the second control element of the second application overlap, and determine that the overlapping control condition exists in response to determining that the touch screen display coordinates overlap.
  • the processor may analyze the current coordinates of GUI buttons of the currently rendered applications to identify two or more GUI buttons that are rendered on the same or overlapping portions of the touch screen display of the computing device.
  • the processor may also determine that overlapping control conditions exist when the simultaneously-active applications are using commonly- assigned hardware input mechanisms (e.g., physical buttons) for one or more control elements.
  • an overlapping control condition may be recognized when a video app is configured to receive inputs for adjusting volume settings for a video playback via hardware volume buttons on the housing of the computing device, and a camera app is configured to receive inputs to activate a shutter of the camera in response to pressing of one of the hardware volume buttons.
  • the processor may identify overlapping control conditions by evaluating a data table that includes a record associated with each of the input mechanisms supported by the computing device.
  • the data table may include data that indicates any current assignments or mappings of the various input mechanisms to control elements of active applications.
  • the data table may include a record that indicates the volume "up" button is currently assigned to a volume adjustment functionality of the first application and/or a fast-forward functionality of the second application.
  • the processor may continually or periodically evaluate the records for the various supported input mechanisms to identify when a particular input mechanism is associated with more than one control element of more than one simultaneously-active application.
  • the processor may end the method 500.
  • the computing device may continue monitoring for overlapping control conditions with the operations of determination block 502 in response to determining that there are no overlapping control conditions.
  • the processor of the computing device may identify available alternative input mechanisms in block 504. For example, out of a complete list of a plurality of input mechanisms that are supported for the computing device, the processor may identify all input mechanisms (e.g., software and/or hardware buttons, portions of the touch screen display, voice recognition units, sensors, etc.) that are not currently reserved or otherwise being used by the two or more simultaneously-active applications. In some embodiments, the identification may be made using the data table of all supported input mechanisms as described above. In some embodiments, the processor may utilize input profiles associated with the applications as described below with reference to FIG. 6.
  • the processor of the computing device may identify a losing application between the first and second applications.
  • the term "losing application” refers to the application for which an overlapping or overlapped input mechanism will be reassigned or moved in order to resolve the overlapping control condition detected in determination block 502.
  • the last application to be launched or the application that is rendered below another application i.e., an application that is not fully visible on the display due to the rendering of another application
  • the losing application may receive user inputs via one or more alternative input mechanisms while the overlapping control condition exists.
  • the processor may identify the losing application by evaluating data within predefined input profiles for each of the active applications.
  • Such input profiles may include data that is predetermined by application developers and that includes information that lists characteristics of the application.
  • input profiles may include ranking, importance, or priority values of applications as well as required, frequently used, and/or preferred input mechanisms that may be used by the processor in the event of an overlapping control condition.
  • the processor may compare ranking values from each of the input profiles to determine the application having the lower ranking value (i.e., the "losing" application).
  • a lower priority application may be designated as the losing application to have an input mechanism reconfigured to an alternative input mechanism while the overlapping control condition exists.
  • the last application to be rendered on the screen may be determined to be the losing application in a first-come, first-served method of selecting the winning and losing applications.
  • preferred alternative input mechanisms listed in input profiles of the simultaneously-active applications may be used to identify the losing application.
  • the processor may determine that the first application is the losing application when none of the second application's preferred alternative input mechanisms are available but at least one of the first application's preferred alternative input mechanisms is available for remapping.
  • the computing device may identify the losing application as the application best able to accommodate alternative user inputs.
  • the processor of the computing device may identify an alternative input mechanism from the identified available alternative input mechanisms to be used with the identified losing application.
  • the alternative input mechanism may be used for providing user inputs to a control element of the losing application.
  • the identification of an appropriate alternative input mechanism for the losing application may be based on the input profile of the losing application.
  • the processor may compare a list of ordered alternative input mechanisms in the input profile to the list of currently available input mechanisms and select the first matching input mechanism to be reconfigured as an alternative input mechanism.
  • the input profile of the losing application may or may not list any preferred or otherwise predefined input mechanisms. In such embodiments, the computing device may simply select any available input mechanism as the alternative input mechanism.
  • the processor of the computing device may reconfigure the identified alternative input mechanism of the losing application to receive user inputs for a control element of the identified losing application.
  • the processor of the computing device may reconfigure one of the identified available input mechanisms to provide input for the control element of the losing application while the overlapping control condition persists (e.g., the GUI button of the losing application that shares the same display coordinates as another GUI button of the winning application).
  • the processor may reconfigure a volume button to receive user inputs for an overlapping control element of the second application (i.e., the non-priority or "losing" app).
  • the processor may reconfigure a second hardware button to provide inputs for a control element of the losing app in response to determining that both the winning app and the losing app are natively configured to utilize a first hardware button to receive inputs for different control elements.
  • the processor may reconfigure a plurality of the available input mechanisms and/or a sequence of input mechanisms to receive user inputs for the overlapping control element of a losing application in block 510.
  • the computing device may reconfigure both a voice recognition module and a volume button to be the combined input mechanisms for receiving user inputs to adjust the progress of a video played via the second application.
  • the processor of the computing device may determine whether the overlapping control condition has ended. For example, the computing device may continually or periodically evaluate the screen coordinates of currently rendered GUI elements to determine whether any previously-identified overlaps in GUI elements no longer exist.
  • overlapping control conditions may no longer exist when one or more of the applications being ended, closed, and/or minimized.
  • the computing device may continue monitor the overlapping control condition in determination block 512.
  • the processor of the computing device may configure input mechanisms such that the control elements of the first application and/or the second application (if either or both applications are still executing) receive user inputs via default (or native) input mechanisms in block 514.
  • the losing application is still executing, any control elements of the losing application that were configured to receive user inputs from alternative input mechanisms with the operations in block 510 may be returned to using original or default input
  • FIG. 6 illustrates an embodiment method 600 for a processor of a computing device to render notifications and configure input mechanisms.
  • the method 600 may be similar to the method 500 described with reference to FIG. 5, except that the method 600 may include explicit operations for rendering notifications that may provide the user of the computing device with a prompt or opportunity to reconfigure automatic configurations of control elements due to overlapping control conditions. For example, in response to configuring the power button to control a function of an app, the computing device may render other alternative input mechanisms that the user may select to provide inputs for the function instead of the power button.
  • the method 600 may be performed by processors of various computing devices, such as the computing device 100 as described with reference to FIGS. 1A-4. In some embodiments, the method 600 may be performed continuously whenever two or more applications are executing simultaneously.
  • the processor of the computing device may perform operations in blocks 502-510 that are similar to operations of like numbered blocks of the method 500 described above with reference to FIG. 5.
  • the processor of the computing device may render a notification indicating the reconfiguration of the identified alternative input mechanism in block 602.
  • the processor may render a visual icon, text, and/or an arrow graphic that provides a cue for the user to begin using the now- assigned input mechanism to provide particular inputs for the losing application.
  • the processor may play a sound (e.g., a beep, etc.) and/or emit a buzz/ vibration.
  • the notification rendered in block 602 may include interactive elements.
  • the rendered notification may be an interactive selection list GUI element that may present an indication of the re-configured input mechanism, as well as the currently available input mechanisms of the computing device.
  • the user may manually override the automatic assignment made by the processor by selecting another input mechanism.
  • the user may prefer to have a volume "up” button receive user inputs for a settings control element of a warning application instead of a volume "down” button.
  • the processor of the computing device may receive a user selection responding to rendered visual information (e.g., the rendered selection list).
  • the processor of the computing device may configure (or reconfigure) another one of the available alternative input mechanisms to receive user input for the overlapping control element of the losing application based on the selection received from the user.
  • the computing device may then continue with the operations in determination block 512 as described.
  • FIG. 7 illustrates an embodiment method 700 for a processor of a computing device to automatically generate and configure a new input mechanism for use with an application during an overlapping control condition.
  • the method 700 may be similar to the method 500 described with reference to FIG. 5, except that the method 700 may include operations for rendering additional, new touch input elements on a touch screen display to be used for receiving inputs for an overlapping control element.
  • the processor may render a new GUI button on the touch screen display that is not native to the active applications, but that is configured to receive user inputs for a control element of a low-priority or losing application during an overlapping control condition.
  • the method 700 may be performed by a processor of various computing devices, such as the computing device 100 as described with reference to FIGS. 1A-4. In some embodiments, the method 700 may be performed continuously whenever two or more applications are executing simultaneously.
  • the processor may perform operations in blocks 502-514 that are similar to the operations of like numbered blocks of the method 500 described above with reference to FIG. 5.
  • the processor of the computing device may determine whether an available alternative input mechanism may be mapped to the overlapping control element of one of the simultaneously-active applications (e.g., to the losing application's overlapping control element). For example, the processor may determine whether any pre-existing input mechanism is available for use in re-configuration operations or whether any available input mechanism is compatible with the overlapping control element of the losing application.
  • the computing device may evaluate an input profile of the losing application to determine whether available alternative input mechanisms or a new additional input mechanism should be used for providing inputs to the overlapping control element of the losing application.
  • an input profile of the losing application may include a flag or other data that indicates whether the processor should or should not utilize new visual elements when reconfiguring input mechanisms during an overlapping control condition involving the losing application and/or the particular overlapping control element.
  • the processor may reconfigure the identified alternative available input mechanism in block 510 as described with reference to FIG. 5.
  • the processor of the computing device may render an additional visual element in block 704, such as a new GUI button, a slider, or another element.
  • the new visual element may be rendered on a particular portion of the touch screen display of the computing device.
  • the characteristics of the visual element may be defined by data within an input profile of the losing application.
  • the input profile may indicate the language, position, color, and/or size of the new visual element.
  • the processor of the computing device may configure the additional visual control element to receive input for the overlapping control element of the losing application. For example, the computing device may configure a portion of the touch screen
  • the processor of the computing device may perform the determination operations in determination block 512 as described.
  • the processor of the computing device may perform the configuration operations to reconfigure user inputs to default input mechanisms in block 514 and may delete any unused visual elements rendered due to overlapping control conditions in optional block 708. For example, once the overlapping control condition ends (e.g., one of the simultaneously- active applications is closed, hidden, and/or minimized), the additional visual element may no longer be rendered and the default input mechanism configurations may be reinstated in block 502.
  • FIG. 8 illustrates an example smartphone-type computing device 100.
  • Such a computing device 100 may include a processor 801 coupled to a touch screen display controller 804 and an internal memory 802.
  • the processor 801 may be one or more multicore ICs designated for general or specific processing tasks.
  • the internal memory 802 may be volatile or non-volatile memory, and may also be secure and/or encrypted memory, or unsecure and/or unencrypted memory, or any combination thereof.
  • the touch screen display controller 804 and the processor 801 may also be coupled to a touch screen display 120 (or touch sensitive panel), such as a resistive-sensing touch screen display, capacitive-sensing touch screen display, infrared sensing touch screen display, etc.
  • the computing device 100 may have one or more radio signal transceivers 808 (e.g., Bluetooth®, ZigBee®, Wi-Fi®, RF radio) and antennae 810, for sending and receiving, coupled to each other and/or to the processor 801.
  • the transceivers 808 and antennae 810 may be used with the above-mentioned circuitry to implement the various wireless transmission protocol stacks and interfaces.
  • the computing device 100 may include a cellular network wireless modem chip 816 that enables communication via a cellular network and is coupled to the processor.
  • the computing device 100 may include a peripheral device connection interface 818 coupled to the processor 801.
  • the peripheral device connection interface 818 may be singularly configured to accept one type of connection, or multiply configured to accept various types of physical and communication connections, common or proprietary, such as USB, Fire Wire, Thunderbolt, or PCIe.
  • the peripheral device connection interface 818 may also be coupled to a similarly configured peripheral device connection port (not shown).
  • the computing device 100 may also include speakers 814 for providing audio outputs.
  • the computing device 100 may also include a housing 820, constructed of a plastic, metal, or a combination of materials, for containing all or some of the components discussed herein.
  • the computing device 100 may include a power source 822 coupled to the processor 801, such as a disposable or rechargeable battery.
  • the rechargeable battery may also be coupled to the peripheral device connection port to receive a charging current from a source external to the computing device 100.
  • the computing device 100 may further include various buttons coupled to the processor 801 and that are configured to receive interactions with a user, such as volume buttons 1 lOa-110b and a power button 850.
  • microprocessor microcomputer or multiple processor chip or chips that can be configured by software instructions (applications) to perform a variety of functions, including the functions of the various embodiments described herein.
  • multiple processors may be provided, such as one processor dedicated to wireless communication functions and one processor dedicated to running other applications.
  • software applications may be stored in internal memory before they are accessed and loaded into the processors.
  • the processors may include internal memory sufficient to store the application software instructions.
  • the internal memory may be a volatile or nonvolatile memory, such as flash memory, or a mixture of both.
  • a general reference to memory refers to memory accessible by the processors including internal memory or removable memory plugged into the various devices and memory within the processors.
  • a general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine.
  • a processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry that is specific to a given function.
  • the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a non-transitory processor-readable, computer-readable, or server-readable medium or a non-transitory processor-readable storage medium.
  • the operations of a method or algorithm disclosed herein may be embodied in a processor-executable software module or processor-executable software instructions which may reside on a non-transitory computer-readable storage medium, a non- transitory server-readable storage medium, and/or a non-transitory processor-readable storage medium.
  • such instructions may be stored processor- executable instructions or stored processor-executable software instructions.
  • Tangible, non-transitory computer-readable storage media may be any available media that may be accessed by a computer.
  • such non- transitory computer-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer.
  • Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of non-transitory computer-readable media.
  • the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a tangible, non-transitory processor-readable storage medium and/or computer-readable medium, which may be incorporated into a computer program product.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne des procédés, des dispositifs et des supports de stockage lisibles par processus non transitoires qui configurent ou reconfigurent automatiquement des mécanismes d'entrée pour fournir des entrées d'utilisateur à des applications actives simultanément avec des mécanismes d'entrée qui sont en conflit. Un mode de réalisation de procédé mis en œuvre par un processeur d'un dispositif informatique peut comprendre la reconnaissance de l'existence d'une situation de chevauchement de commandes entre des applications actives simultanément lorsqu'un premier élément de commande d'une première application et un deuxième élément de commande d'une deuxième application sont configurés pour recevoir des entrées d'utilisateur respectives par l'intermédiaire d'un mécanisme d'entrée commun. Lorsque le processeur détermine l'existence d'une situation de chevauchement de commandes, les mécanismes d'entrée disponibles du dispositif informatique peuvent être identifiés et l'un des mécanismes d'entrée disponibles identifiés peut être reconfiguré de manière à fournir les entrées d'utilisateur pour l'un parmi le premier élément de commande ou le deuxième élément de commande pendant que la situation de chevauchement de commandes existe. Lorsque la situation de chevauchement de commandes n'existe plus, le processeur peut ramener les entrées de commande aux configurations normales.
EP16738321.5A 2015-09-10 2016-06-28 Schémas de commande dynamique pour applications actives simultanément Withdrawn EP3347805A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14/849,901 US9639234B2 (en) 2015-09-10 2015-09-10 Dynamic control schemes for simultaneously-active applications
PCT/US2016/039770 WO2017044176A1 (fr) 2015-09-10 2016-06-28 Schémas de commande dynamique pour applications actives simultanément

Publications (1)

Publication Number Publication Date
EP3347805A1 true EP3347805A1 (fr) 2018-07-18

Family

ID=56409215

Family Applications (1)

Application Number Title Priority Date Filing Date
EP16738321.5A Withdrawn EP3347805A1 (fr) 2015-09-10 2016-06-28 Schémas de commande dynamique pour applications actives simultanément

Country Status (4)

Country Link
US (1) US9639234B2 (fr)
EP (1) EP3347805A1 (fr)
CN (1) CN107924282A (fr)
WO (1) WO2017044176A1 (fr)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6848591B2 (ja) * 2017-03-28 2021-03-24 ブラザー工業株式会社 画像処理装置
US10831513B2 (en) * 2017-11-06 2020-11-10 International Business Machines Corporation Control transparency of a top layer provided by an additional transparent layer on top of the top layer based on relevance
CN108379839B (zh) * 2018-03-23 2019-12-10 网易（杭州）网络有限公司 控件的响应方法、装置和终端
CN110389799B (zh) * 2018-04-20 2022-06-07 中兴通讯股份有限公司 元素显示方法及装置、终端、存储介质、电子装置
CN109165013A (zh) * 2018-07-04 2019-01-08 惠州市德赛西威汽车电子股份有限公司 应用程序中多图层叠加显示方法、装置、存储介质及终端
CN109640164A (zh) * 2018-11-02 2019-04-16 重庆爱奇艺智能科技有限公司 一种用于多个虚拟现实设备间的播放方法与装置
US11361760B2 (en) 2018-12-13 2022-06-14 Learning Squared, Inc. Variable-speed phonetic pronunciation machine
CN109697007A (zh) * 2018-12-27 2019-04-30 珠海格力电器股份有限公司 组态软件的工程文件编制方法、组态软件及工业触摸屏
JP7070713B2 (ja) * 2018-12-28 2022-05-18 富士通株式会社 情報処理装置、段取り作業修正方法、および段取り作業修正プログラム
CN114174972B (zh) * 2019-07-19 2024-05-17 谷歌有限责任公司 用于复杂应用gui的自动化助理控制的压缩口头话语
CN111026482B (zh) * 2019-11-29 2022-02-22 维沃移动通信有限公司 一种应用程序控制方法及电子设备
US20220397985A1 (en) * 2021-06-14 2022-12-15 Citrix Systems, Inc. Transparent Application Window
US11617017B2 (en) 2021-06-30 2023-03-28 Rovi Guides, Inc. Systems and methods of presenting video overlays
US20230007335A1 (en) * 2021-06-30 2023-01-05 Rovi Guides, Inc. Systems and methods of presenting video overlays

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8475273B2 (en) 2005-09-07 2013-07-02 Bally Gaming, Inc. Video switcher and touch router system for a gaming machine
US7490296B2 (en) * 2003-01-31 2009-02-10 Microsoft Corporation Utility object for specialized data entry
US7429993B2 (en) 2004-09-17 2008-09-30 Microsoft Corporation Method and system for presenting functionally-transparent, unobtrusive on-screen windows
US7549130B2 (en) 2004-11-30 2009-06-16 Sap Ag Pattern-based keyboard controls
US20070293148A1 (en) 2004-12-23 2007-12-20 Chiang Kuo C Portable video communication device with multi-illumination source
PL1748354T3 (pl) 2005-07-29 2012-09-28 Advanced Digital Broadcast Sa Sposób zarządzania i wyświetlania wiadomości i urządzenie do zarządzania i wyświetlania
US7712039B2 (en) 2006-03-31 2010-05-04 Microsoft Corporation Setting control using edges of a user interface
US9575655B2 (en) * 2006-12-29 2017-02-21 Nokia Technologies Oy Transparent layer application
US9690474B2 (en) * 2007-12-21 2017-06-27 Nokia Technologies Oy User interface, device and method for providing an improved text input
US8769427B2 (en) * 2008-09-19 2014-07-01 Google Inc. Quick gesture input
EP2312427B1 (fr) 2009-10-13 2013-09-18 BlackBerry Limited Interface d'utilisateur pour affichage tactile
US8310351B2 (en) 2010-04-08 2012-11-13 Motorola Mobility Llc Apparatuses, methods, and systems for an electronic device with a detachable user input attachment
CN102243889B (zh) 2010-05-13 2014-12-17 北京三星通信技术研究有限公司 基于用户输入来执行应用程序的移动终端及其方法
EP2523083A1 (fr) 2011-05-13 2012-11-14 Harman Becker Automotive Systems GmbH Système et procédé de fonctionnement d'un écran tactile et unité de traitement
US8881170B2 (en) 2012-04-30 2014-11-04 Genesys Telecommunications Laboratories, Inc Method for simulating screen sharing for multiple applications running concurrently on a mobile platform
US9606725B2 (en) 2012-11-30 2017-03-28 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Collaborative overlay of user interface elements rendered on the display of a computing device
US20140282272A1 (en) 2013-03-15 2014-09-18 Qualcomm Incorporated Interactive Inputs for a Background Task
CN103279264A (zh) * 2013-05-03 2013-09-04 富泰华工业（深圳）有限公司 电子装置及其输入操作管理方法
KR102187255B1 (ko) 2013-09-30 2020-12-04 삼성전자주식회사 전자 장치의 디스플레이 방법 및 그 전자 장치

Also Published As

Publication number Publication date
CN107924282A (zh) 2018-04-17
WO2017044176A1 (fr) 2017-03-16
US9639234B2 (en) 2017-05-02
US20170075539A1 (en) 2017-03-16

Similar Documents

Publication Publication Date Title
US9639234B2 (en) Dynamic control schemes for simultaneously-active applications
US10928985B2 (en) User terminal device and displaying method thereof
US10366602B2 (en) Interactive multi-touch remote control
US20200034018A1 (en) Electronic device using auxiliary input device and operating method thereof
EP3680770B1 (fr) Procédé de modification d'un écran principal, interface graphique d'utilisateur et dispositif électronique
AU2014201856B2 (en) APP operating method and device and app output device supporting the same
US20140282272A1 (en) Interactive Inputs for a Background Task
EP3617861A1 (fr) Procédé d'affichage d'interface utilisateur graphique, et dispositif électronique
US20130241829A1 (en) User interface method of touch screen terminal and apparatus therefor
US10175880B2 (en) Display apparatus and displaying method thereof
US8994678B2 (en) Techniques for programmable button on bezel of mobile terminal
JP2013528304A (ja) ジャンプ、チェックマーク、および取消し線のジェスチャー
US11216065B2 (en) Input control display based on eye gaze
US10430071B2 (en) Operation of a computing device functionality based on a determination of input means
US20130120293A1 (en) Touchscreen-enabled terminal and application control method thereof
US20140191943A1 (en) Electronic apparatus and method for controlling electronic apparatus thereof
EP3026550B1 (fr) Appareil et procédé pour commander un écran d'affichage dans un dispositif électronique
US10719147B2 (en) Display apparatus and control method thereof
JP4765893B2 (ja) タッチパネル搭載装置、外部装置、及び外部装置の操作方法
KR20200066602A (ko) 사용자 단말 장치 및 이의 디스플레이 방법

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180328

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20200529

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20220104