WO2012118626A1 - System and method for touchscreen knob control - Google Patents
System and method for touchscreen knob control Download PDFInfo
- Publication number
- WO2012118626A1 WO2012118626A1 PCT/US2012/025519 US2012025519W WO2012118626A1 WO 2012118626 A1 WO2012118626 A1 WO 2012118626A1 US 2012025519 W US2012025519 W US 2012025519W WO 2012118626 A1 WO2012118626 A1 WO 2012118626A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- input
- user
- user interface
- category
- interface element
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Definitions
- the present disclosure relates to user interfaces and more specifically to making touch-based interfaces more user friendly.
- a system configured to practice the method first displays a user interface element, such as a virtual on-screen knob, on a touch-sensitive display and receives, from a user, input associated with the user interface element via the touch- sensitive display.
- the input can be a touch, a tap, a swipe, a multi-finger gesture, a flick, a drag, and/or a press-and-hold, for example.
- the system matches the input to an input category selected from a group of predefined input categories for the user interface element.
- the input can also be multi-modal, such as a combination of a touch gesture and a speech command.
- the system can match only a portion, such as the touch gesture, of the multi-modal input to the input category. Then the system updates the user interface element based on the input and based on the input category.
- the system can optionally translate the input to a standardized input based on the input category and update the user interface element based on the standardized input.
- This approach is flexible and can accept multiple different types of input.
- the user can provide a first user input of a first input category, such as a circular or rotational gesture about the knob, and a second user input of a second input category, such as a tap in a particular position relative to the knob.
- a first input category such as a circular or rotational gesture about the knob
- a second user input of a second input category such as a tap in a particular position relative to the knob.
- the system can match input to an input category based on a similarity threshold to one of the group of predefined input categories. If the input does not match any input category based on the similarity threshold, the system can prompt the user for additional input defining a new predefined input category, receive the additional input from the user, and create the new predefined input category based on the additional input.
- the user and/or an automated process can toggle availability of a certain input category in the group of predefined input categories.
- each individual input category can include its own sensitivity and/or scale which can be user customizable. For example, a horizontal tap and drag gesture 3 centimeters in length may be equivalent to a vertical tap and drag gesture 5 centimeters in length.
- the system can alter not only the user interface element, but also one or more variable represented by the user interface element and/or other aspects of a running software application based on the input and the input category.
- the system can update the user interface element in substantially real time as the input is received. Real time or substantially real time updating can serve as feedback to the user to demonstrate the effect of the particular gesture.
- FIG. 1 illustrates an example system embodiment
- FIG. 2 illustrates an example mobile device with a touch-screen interface and exemplary virtual knobs on the touch- screen interface
- FIG. 3 illustrates a rotational gesture in relation to a virtual knob
- FIG. 4 illustrates horizontal and vertical gestures in relation to a virtual knob
- FIG. 5 illustrates point gestures in relation to a virtual knob
- FIG. 6 illustrates a multi-finger rotational gesture in relation to a virtual knob
- FIG. 7 illustrates a multi-finger directional gesture in relation to a virtual knob
- FIG. 8 illustrates pinch and zoom gestures in relation to a virtual knob
- FIG. 9 illustrates a menu gesture in relation to a virtual knob
- FIG. 10 illustrates an exemplary block diagram for processing input associated with a virtual knob
- FIG. 11 illustrates an example method embodiment. DETAILED DESCRIPTION
- the present disclosure addresses the need in the art for improved user interfaces in touch-screen enabled devices such that the user interfaces are more intuitive and easily approachable.
- a system, method and non-transitory computer-readable media are disclosed which facilitate multiple input categories for a single user interface element, such as a knob.
- a brief introductory description of a basic general purpose system or computing device in FIG. 1 which can be employed to practice the concepts is disclosed herein.
- a more detailed description of the various input categories, user interface approaches, and exemplary methods will then follow.
- an exemplary system 100 includes a general-purpose computing device 100, including a processing unit (CPU or processor) 120 and a system bus 110 that couples various system components including the system memory 130 such as read only memory (ROM) 140 and random access memory (RAM) 150 to the processor 120.
- the system 100 can include a cache 122 of high speed memory connected directly with, in close proximity to, or integrated as part of the processor 120.
- the system 100 copies data from the memory 130 and/or the storage device 160 to the cache 122 for quick access by the processor 120. In this way, the cache 122 provides a performance boost that avoids processor 120 delays while waiting for data.
- These and other modules can control or be configured to control the processor 120 to perform various actions.
- Other system memory 130 may be available for use as well.
- the memory 130 can include multiple different types of memory with different performance characteristics. It can be appreciated that the disclosure may operate on a computing device 100 with more than one processor 120 or on a group or cluster of computing devices networked together to provide greater processing capability.
- the processor 120 can include any general purpose processor and a hardware module or software module, such as module 1 162, module 2 164, and module 3 166 stored in storage device 160, configured to control the processor 120 as well as a special-purpose processor where software instructions are incorporated into the actual processor design.
- the processor 120 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc.
- a multi-core processor may be symmetric or asymmetric.
- the system bus 110 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- a basic input/output (BIOS) stored in ROM 140 or the like may provide the basic routine that helps to transfer information between elements within the computing device 100, such as during start-up.
- the computing device 100 further includes storage devices 160 such as a hard disk drive, a magnetic disk drive, an optical disk drive, tape drive or the like.
- the storage device 160 can include software modules 162, 164, 166 for controlling the processor 120. Other hardware or software modules are contemplated.
- the storage device 160 is connected to the system bus 110 by a drive interface.
- the drives and the associated computer readable storage media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the computing device 100.
- a hardware module that performs a particular function includes the software component stored in a non-transitory computer-readable medium in connection with the necessary hardware components, such as the processor 120, bus 110, display 170, and so forth, to carry out the function.
- the basic components are known to those of skill in the art and appropriate variations are contemplated depending on the type of device, such as whether the device 100 is a small, handheld computing device, a desktop computer, or a computer server.
- Non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.
- an input device 190 represents any number of input mechanisms, such as a microphone for speech, a touch- sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth.
- An output device 170 can also be one or more of a number of output mechanisms known to those of skill in the art.
- multimodal systems enable a user to provide multiple types of input to communicate with the computing device 100.
- the communications interface 180 generally governs and manages the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.
- the illustrative system embodiment is presented as including individual functional blocks including functional blocks labeled as a "processor" or processor 120.
- the functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software and hardware, such as a processor 120, that is purpose-built to operate as an equivalent to software executing on a general purpose processor.
- the functions of one or more processors presented in FIG. 1 may be provided by a single shared processor or multiple processors.
- Illustrative embodiments may include microprocessor and/or digital signal processor (DSP) hardware, read-only memory (ROM) 140 for storing software performing the operations discussed below, and random access memory (RAM) 150 for storing results.
- DSP digital signal processor
- ROM read-only memory
- RAM random access memory
- VLSI Very large scale integration
- the logical operations of the various embodiments are implemented as: (1) a sequence of computer implemented steps, operations, or procedures running on a programmable circuit within a general use computer, (2) a sequence of computer implemented steps, operations, or procedures running on a specific -use programmable circuit; and/or (3) interconnected machine modules or program engines within the programmable circuits.
- the system 100 shown in FIG. 1 can practice all or part of the recited methods, can be a part of the recited systems, and/or can operate according to instructions in the recited non-transitory computer-readable storage media.
- Such logical operations can be implemented as modules configured to control the processor 120 to perform particular functions according to the programming of the module. For example, FIG.
- Modi 162, Mod2 164 and Mod3 166 which are modules configured to control the processor 120. These modules may be stored on the storage device 160 and loaded into RAM 150 or memory 130 at runtime or may be stored as would be known in the art in other computer-readable memory locations.
- FIG. 2 illustrates an example mobile device 200, such as a smartphone, tablet, or computer, with a touch-screen interface 202 and exemplary virtual knobs 204, 206, 208, 210, 212 on the touch-screen interface 202.
- the knobs can be images, three-dimensional representations or approximations of real knobs, simple geometric shapes, and/or any other graphical representation.
- the user interface 202 can also include a status indicator 214 for some or all of the values represented by the knobs.
- knobs When a user is presented with these knobs, the user may not know how to manipulate the knobs without some kind of tutorial.
- One goal of this disclosure is to allow users to manipulate the knobs (or other user applicable interface elements) in any of a set of predefined ways. The user does not need to read a manual or learn any new behaviors to interact with the application via the knobs. In this way, the user can simply choose whatever input approach makes sense and use that approach. If one user wants to tap, hold, and dial around the knob, the system can understand and appropriately process that input with a suitable hardware and/or software mechanism. If another user wants to tap at particular points around the knob, the system can understand and appropriately process that input using a different hardware and/or software mechanism. If yet another user wants to tap and drag across an imaginary horizontal area, the system can understand and appropriately process that input using yet a different hardware and/or software mechanism.
- FIG. 3 illustrates a rotational gesture 302 in relation to a virtual knob 300.
- the user taps and drags around the edge of the virtual knob in a circular motion.
- the system updates the knob and the value the knob represents.
- This input category can apply to virtually any user interface object, even ones that are not round.
- the user can tap and drag within the space of the virtual knob, on the edge or perimeter of the virtual knob, or outside the virtual knob.
- the system can determine to which knob to direct the input based on the user's tap and drag motion.
- the system can calculate the intended target of a tap and drag motion by treating the dragging motion as an arc of a circle or ellipse and determining/estimating the center of the circle or ellipse. Then the system can compare the determined/estimated center with user interface elements on the display and identify a best match.
- FIG. 4 illustrates horizontal 404 and vertical gestures 402 in relation to a virtual knob 400.
- the user taps on the touch screen and drags in a straight line.
- the system treats this input as if the user were dragging a slider bar across an imaginary slider.
- the value for the knob increases or decreases.
- the user can tap and drag in any direction, not simply across the horizontal or vertical axes.
- FIG. 5 illustrates point gestures 502, 504, 506 in relation to a virtual knob 500.
- the user taps at various points on the border or edge of the virtual knob.
- the system can translate the taps at the border of the virtual knob into direct input. For instance, if the knob represents a value from 1 to 12, and the user taps at position 502, then the system can adjust the knob and the value the knob represents to a value of 4. If the user taps at position 504, then the system can adjust the knob and the value of the knob represents to a value of 9.
- the system can provide different functionality to specialized taps, such as a tap in the center or approximately in the center of the knob, such as tap 506.
- the system can set the knob value to 0. If the user taps in the center again, then the system can restore the previous value of 7.
- a double tap anywhere on the knob can accomplish a similar functionality as the single tap in the center, for example.
- FIG. 6 illustrates a multi-finger rotational gesture 602, 604 in relation to a virtual knob 600.
- the user simultaneously taps two fingers 602, 604 (or other points of contact) on the touch screen and drags each about the virtual knob in a circular or rotational manner. This motion roughly approximates the motion necessary to manipulate an actual knob and not a virtual knob on a touch-screen.
- the same general principles apply in this input category as in FIG. 3, but the system can receive or infer additional information from the multiple points of contact, how fast each moves relative to the other, and so forth.
- FIG. 7 illustrates a multi-finger directional gesture 702 in relation to a virtual knob 700.
- the user taps and drags with two or more fingers.
- the system can treat this gesture in a same, similar, or different manner than the input category illustrated in FIG. 4.
- the system can treat a single finger directional gesture as a more sensitive motion, while the multi-finger directional gesture is a less sensitive, gross motion.
- the knob represents values from 1 to 1000
- a single finger directional gesture can increment the knob value by one for each unit of distance.
- a two finger directional gesture can increment the knob value by ten for the same unit of distance
- a three finger directional gesture can increment the knob value by hundred for the same unit of distance.
- the same overall gesture category can include subcategories of similar type, differing in the scale or sensitivity and the number of points of contact on the touch-screen.
- FIG. 8 illustrates a pinch and zoom gesture 802, 804 in relation to a virtual knob 800.
- the user taps two or more fingers and drags at least one away from the other and can drag each away from the other.
- the scale of the adjustment to the value of the knob is based on the increasing or decreasing distance between the points of finger contact.
- FIG. 9 illustrates a menu gesture 902 in relation to a virtual knob 900.
- the user taps and holds in a center region of the knob (or elsewhere) without dragging.
- the system opens a menu 904 with various options, such as sensitivity 906, a list of gesture types 908, etc.
- the user can enable or disable certain gesture types with respect to the entire interface or with respect to a particular user interface element. For instance, the user can disable a rotate gesture 910 by unchecking the checkbox.
- the categories of input disclosed herein are exemplary.
- the approaches set forth herein can include additional categories of input.
- the exact positioning of the taps, drags, and other touch gestures are exemplary of multiple different categories and should not be construed as limiting the principles disclosed herein.
- New input categories and variations of the disclosed input categories can be incorporated using the principles of this disclosure.
- FIG. 10 illustrates an exemplary block diagram 1000 for processing input associated with a virtual knob.
- a user input device 1002 receives the user input such as touch-screen input.
- the gesture processor 1004 receives the user input from the user input device 1002. Based on a current screen state representing which user interface elements are currently displayed on the touch-screen, the gesture processor analyzes the user input and compares the user input with entries in a gesture category database 1006. After finding a sufficient match in the gesture category database 1006, the gesture processor 1004 provides some kind of output 1008.
- the output 1008 can be a value. Alternatively, the output 1008 can be an action taken by the gesture processor 1004 to manipulate a value represented by a user interface element directly or indirectly.
- This approach is flexible and provides for a single user interface element to accept multiple different types of input.
- the user can provide a first user input of a first input category, such as a circular or rotational gesture about the knob, and a second user input of a second input category, such as a tap in a particular position relative to the knob.
- a first input category such as a circular or rotational gesture about the knob
- a second user input of a second input category such as a tap in a particular position relative to the knob.
- the system can match input to an input category based on a similarity threshold to one of the group of predefined input categories. If the input does not match any input category based on the similarity threshold, the system can prompt the user to for additional input defining a new predefined input category, receive the additional input from the user, and create the new predefined input category based on the additional input.
- FIG. 11 illustrates an example method embodiment.
- a system configured to practice the method first displays a user interface element, such as a virtual on-screen knob representing one or more values, on a touch-sensitive display (1102) and receives, from a user, input associated with the user interface element via the touch-sensitive display (1104).
- the user interface element can be capable of accepting any of a group of different categories of predefined user input.
- the input can be a touch, a tap, a swipe, a multi-finger gesture, a flick, a drag, and/or a press-and-hold, for example.
- the system matches the input to an input category selected from a group of predefined input categories for the user interface element (1106).
- the system can match the input to the input category based on a similarity threshold to one of the group of predefined input categories.
- the input can also be multi-modal, such as a combination of a touch gesture and a speech command. In this scenario, the system can match only a portion, such as the touch gesture, of the multi-modal input to the input category.
- the system updates the user interface element based on the input and based on the input category (1108).
- the system can optionally translate the input to a standardized input based on the input category and update the user interface element based on the standardized input.
- the system can prompt the user to for additional input defining a new predefined input category, receive the additional input from the user, and create the new predefined input category based on the additional input.
- the system can modify existing input categories instead of creating new predefined categories.
- the user and/or an automated process can toggle availability of a certain input category in the group of predefined input categories.
- each individual input category can include its own sensitivity and/or scale which can be user customizable. For example, a horizontal tap and drag gesture 3 centimeters in length may be equivalent to a vertical tap and drag gesture 5 centimeters in length.
- a first user input can update the user interface based on a first input category in an equivalent manner to a second user input based on a second input category.
- the system can also alter one or more aspect of a running software application based on the input and the input category.
- the system can also update the user interface element in real time or substantially real time as the input is received, effectively animating the knob as the user is manipulating the knob via touch input based on the detected input category.
- An exemplary system for controlling a software application can include a processor and a touch-sensitive display.
- the exemplary system can further include a first module configured to control the processor to display a group of user-manipulable user interface elements on the touch-sensitive display, a second module configured to control the processor to receive a group of simultaneous user inputs associated with a subset of the group of user-manipulable user interface elements, a third module configured to control the processor to match each of the group of simultaneous user inputs to a respective input category selected from a group of predefined input categories for the subset of the group of user-manipulable user interface elements, and a fourth module configured to control the processor to control the software application based on each of the group of simultaneous user inputs and the respective input category.
- Such an exemplary system can be a mobile device.
- the software application can be a media creation and/or consumption application.
- the group of simultaneous user inputs can be of a same input category or of different input categories.
- the system can further include a fifth module configured to control the processor to update the group of user-manipulable user interface elements based on the group of simultaneous user inputs and the respective input category for each simultaneous user input.
- the system can provide for a cancel gesture as an input category separate from other inputs so that a user can cancel a previously submitted input. Further, the system can provide for a cancel gesture added as a continuous part of a pending gesture so that the user can cancel the entire gesture.
- the user can also change gesture types in the middle of a gesture. For example, the user can start manipulating a knob with a circular motion dragging gesture in a first category, then change categories to a vertical dragging motion without lifting the dragging finger from the touch-screen. This can be advantageous if the circular motion has course-grain control over the knob value and then the user wishes to smoothly change to a more fine-grain control over the knob value afforded by the vertical dragging motion gesture category.
- the user is free to use/manipulate the knob or other user interface elements on a touch-enabled device in whichever way is most natural for the user.
- the user does not need to read some complicated manual or learn any new behavior to effectively use the knob to control the application.
- the usability and approachability are much improved for software applications incorporating the principles disclosed herein.
- Embodiments within the scope of the present disclosure may also include tangible and/or non-transitory computer-readable storage media for carrying or having computer- executable instructions or data structures stored thereon.
- Such non-transitory computer- readable storage media can be any available media that can be accessed by a general purpose or special purpose computer, including the functional design of any special purpose processor as discussed above.
- non- transitory computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions, data structures, or processor chip design.
- Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions.
- Computer- executable instructions also include program modules that are executed by computers in stand-alone or network environments.
- program modules include routines, programs, components, data structures, objects, and the functions inherent in the design of special-purpose processors, etc. that perform particular tasks or implement particular abstract data types.
- Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.
- Embodiments of the disclosure may be practiced in network computing environments with many types of computer system configurations, including personal computers, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Embodiments may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination thereof) through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Disclosed herein are systems, methods, and non-transitory computer-readable storage media for controlling a user interface. A system configured to practice the method displays a user interface element, such as a knob, on a touch-sensitive display. The system receives input from a user associated with the user interface element via the touch-sensitive display. For example, the user can provide input to the knob via a tap, twist, flick, press-and-hold, drag, slide, or other touch-based input with a single or multiple fingers or other points of contact. The system matches the input to an input category selected from a group of predefined input categories for the user interface element, and updates the user interface element based on the input and based on the input category. The system can also update a value represented by the user interface element, such as a value that affects output or other settings of a software application.
Description
SYSTEM AND METHOD FOR TOUCHSCREEN KNOB CONTROL
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. Patent Application Serial No. 13/038,276, entitled, "SYSTEM AND METHOD FOR TOUCHSCREEN KNOB CONTROL", filed on 1 March 2011, which is hereby incorporated herein by reference in its entirety.
BACKGROUND
1. Technical Field
[0002] The present disclosure relates to user interfaces and more specifically to making touch-based interfaces more user friendly.
2. Introduction
[0003] With the advent of capacitive touch screens and other touch- sensitive technology on devices such as smartphones, tablet computers, and desktop computers, software and hardware developers have focused on adapting user interfaces to take more effective advantage of unique features of this technology. While some user interface elements, such as a button, map very easily to a touch-based interface, other user interface elements, such as a scroll bar on an edge of a scrollable region, can be replaced completely. However, certain user interface elements, such as a knob, are not typical to traditional graphical user interfaces and can present problems in training new users. New or inexperienced users may be unsure how exactly to interact with this type of user interface elements, causing apprehension, fear, and/or uncertainty for a certain segment of the user population. This, in turn, can reduce the approachability of a particular hardware platform as well as individual software applications running on the hardware platform.
SUMMARY
[0004] Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or can be learned by practice of the herein disclosed principles. The features and advantages of the disclosure can be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the
disclosure will become more fully apparent from the following description and appended claims, or can be learned by the practice of the principles set forth herein.
[0005] Disclosed are systems, methods, and non-transitory computer-readable storage media for controlling a user interface in a flexible, intuitive way that is approachable and readily usable without detailed explanation or documentation. A system configured to practice the method first displays a user interface element, such as a virtual on-screen knob, on a touch-sensitive display and receives, from a user, input associated with the user interface element via the touch- sensitive display. The input can be a touch, a tap, a swipe, a multi-finger gesture, a flick, a drag, and/or a press-and-hold, for example. The system matches the input to an input category selected from a group of predefined input categories for the user interface element. The input can also be multi-modal, such as a combination of a touch gesture and a speech command. In this scenario, the system can match only a portion, such as the touch gesture, of the multi-modal input to the input category. Then the system updates the user interface element based on the input and based on the input category. The system can optionally translate the input to a standardized input based on the input category and update the user interface element based on the standardized input.
[0006] This approach is flexible and can accept multiple different types of input. For example, the user can provide a first user input of a first input category, such as a circular or rotational gesture about the knob, and a second user input of a second input category, such as a tap in a particular position relative to the knob. These two different types of input can cause the knob to adjust to the same position or value.
[0007] The system can match input to an input category based on a similarity threshold to one of the group of predefined input categories. If the input does not match any input category based on the similarity threshold, the system can prompt the user for additional input defining a new predefined input category, receive the additional input from the user, and create the new predefined input category based on the additional input.
[0008] The user and/or an automated process can toggle availability of a certain input category in the group of predefined input categories. Further, each individual input category can include its own sensitivity and/or scale which can be user customizable. For example, a horizontal tap and drag gesture 3 centimeters in length may be equivalent to a vertical tap and drag gesture 5 centimeters in length.
[0009] The system can alter not only the user interface element, but also one or more variable represented by the user interface element and/or other aspects of a running software application based on the input and the input category. The system can update the user interface element in substantially real time as the input is received. Real time or substantially real time updating can serve as feedback to the user to demonstrate the effect of the particular gesture.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only exemplary embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail through the use of the accompanying drawings in which:
[0011] FIG. 1 illustrates an example system embodiment;
[0012] FIG. 2 illustrates an example mobile device with a touch-screen interface and exemplary virtual knobs on the touch- screen interface;
[0013] FIG. 3 illustrates a rotational gesture in relation to a virtual knob;
[0014] FIG. 4 illustrates horizontal and vertical gestures in relation to a virtual knob;
[0015] FIG. 5 illustrates point gestures in relation to a virtual knob;
[0016] FIG. 6 illustrates a multi-finger rotational gesture in relation to a virtual knob;
[0017] FIG. 7 illustrates a multi-finger directional gesture in relation to a virtual knob;
[0018] FIG. 8 illustrates pinch and zoom gestures in relation to a virtual knob;
[0019] FIG. 9 illustrates a menu gesture in relation to a virtual knob;
[0020] FIG. 10 illustrates an exemplary block diagram for processing input associated with a virtual knob; and
[0021] FIG. 11 illustrates an example method embodiment.
DETAILED DESCRIPTION
[0022] Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the disclosure.
[0023] The present disclosure addresses the need in the art for improved user interfaces in touch-screen enabled devices such that the user interfaces are more intuitive and easily approachable. A system, method and non-transitory computer-readable media are disclosed which facilitate multiple input categories for a single user interface element, such as a knob. A brief introductory description of a basic general purpose system or computing device in FIG. 1 which can be employed to practice the concepts is disclosed herein. A more detailed description of the various input categories, user interface approaches, and exemplary methods will then follow. These variations shall be discussed herein as the various embodiments are set forth. The disclosure now turns to FIG. 1.
[0024] With reference to FIG. 1, an exemplary system 100 includes a general-purpose computing device 100, including a processing unit (CPU or processor) 120 and a system bus 110 that couples various system components including the system memory 130 such as read only memory (ROM) 140 and random access memory (RAM) 150 to the processor 120. The system 100 can include a cache 122 of high speed memory connected directly with, in close proximity to, or integrated as part of the processor 120. The system 100 copies data from the memory 130 and/or the storage device 160 to the cache 122 for quick access by the processor 120. In this way, the cache 122 provides a performance boost that avoids processor 120 delays while waiting for data. These and other modules can control or be configured to control the processor 120 to perform various actions. Other system memory 130 may be available for use as well. The memory 130 can include multiple different types of memory with different performance characteristics. It can be appreciated that the disclosure may operate on a computing device 100 with more than one processor 120 or on a group or cluster of computing devices networked together to provide greater processing capability. The processor 120 can include any general purpose processor and a hardware module or software module, such as module 1 162, module 2 164, and module 3 166 stored in storage device 160, configured to control the
processor 120 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. The processor 120 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.
[0025] The system bus 110 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. A basic input/output (BIOS) stored in ROM 140 or the like, may provide the basic routine that helps to transfer information between elements within the computing device 100, such as during start-up. The computing device 100 further includes storage devices 160 such as a hard disk drive, a magnetic disk drive, an optical disk drive, tape drive or the like. The storage device 160 can include software modules 162, 164, 166 for controlling the processor 120. Other hardware or software modules are contemplated. The storage device 160 is connected to the system bus 110 by a drive interface. The drives and the associated computer readable storage media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the computing device 100. In one aspect, a hardware module that performs a particular function includes the software component stored in a non-transitory computer-readable medium in connection with the necessary hardware components, such as the processor 120, bus 110, display 170, and so forth, to carry out the function. The basic components are known to those of skill in the art and appropriate variations are contemplated depending on the type of device, such as whether the device 100 is a small, handheld computing device, a desktop computer, or a computer server.
[0026] Although the exemplary embodiment described herein employs the hard disk 160, it should be appreciated by those skilled in the art that other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital versatile disks, cartridges, random access memories (RAMs) 150, read only memory (ROM) 140, a cable or wireless signal containing a bit stream and the like, may also be used in the exemplary operating environment. Non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.
[0027] To enable user interaction with the computing device 100, an input device 190 represents any number of input mechanisms, such as a microphone for speech, a touch- sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth. An output device 170 can also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems enable a user to provide multiple types of input to communicate with the computing device 100. The communications interface 180 generally governs and manages the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.
[0028] For clarity of explanation, the illustrative system embodiment is presented as including individual functional blocks including functional blocks labeled as a "processor" or processor 120. The functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software and hardware, such as a processor 120, that is purpose-built to operate as an equivalent to software executing on a general purpose processor. For example the functions of one or more processors presented in FIG. 1 may be provided by a single shared processor or multiple processors. (Use of the term "processor" should not be construed to refer exclusively to hardware capable of executing software.) Illustrative embodiments may include microprocessor and/or digital signal processor (DSP) hardware, read-only memory (ROM) 140 for storing software performing the operations discussed below, and random access memory (RAM) 150 for storing results. Very large scale integration (VLSI) hardware embodiments, as well as custom VLSI circuitry in combination with a general purpose DSP circuit, may also be provided.
[0029] The logical operations of the various embodiments are implemented as: (1) a sequence of computer implemented steps, operations, or procedures running on a programmable circuit within a general use computer, (2) a sequence of computer implemented steps, operations, or procedures running on a specific -use programmable circuit; and/or (3) interconnected machine modules or program engines within the programmable circuits. The system 100 shown in FIG. 1 can practice all or part of the recited methods, can be a part of the recited systems, and/or can operate according to
instructions in the recited non-transitory computer-readable storage media. Such logical operations can be implemented as modules configured to control the processor 120 to perform particular functions according to the programming of the module. For example, FIG. 1 illustrates three modules Modi 162, Mod2 164 and Mod3 166 which are modules configured to control the processor 120. These modules may be stored on the storage device 160 and loaded into RAM 150 or memory 130 at runtime or may be stored as would be known in the art in other computer-readable memory locations.
[0030] Having disclosed some basic computing device components and concepts, the disclosure returns to a discussion of controlling a user interface in a flexible, intuitive way that is approachable and readily usable without detailed explanation or documentation. FIG. 2 illustrates an example mobile device 200, such as a smartphone, tablet, or computer, with a touch-screen interface 202 and exemplary virtual knobs 204, 206, 208, 210, 212 on the touch-screen interface 202. The knobs can be images, three-dimensional representations or approximations of real knobs, simple geometric shapes, and/or any other graphical representation. While the disclosure here is primarily discussed in terms of virtual knobs on a touch-screen interface, the same principles apply equally to other user interface elements that a user may wish to interact with using different categories, types, or classes of user input. The user interface 202 can also include a status indicator 214 for some or all of the values represented by the knobs.
[0031] When a user is presented with these knobs, the user may not know how to manipulate the knobs without some kind of tutorial. One goal of this disclosure is to allow users to manipulate the knobs (or other user applicable interface elements) in any of a set of predefined ways. The user does not need to read a manual or learn any new behaviors to interact with the application via the knobs. In this way, the user can simply choose whatever input approach makes sense and use that approach. If one user wants to tap, hold, and dial around the knob, the system can understand and appropriately process that input with a suitable hardware and/or software mechanism. If another user wants to tap at particular points around the knob, the system can understand and appropriately process that input using a different hardware and/or software mechanism. If yet another user wants to tap and drag across an imaginary horizontal area, the system can understand and appropriately process that input using yet a different hardware and/or software
mechanism. Some exemplary categories of touch-screen input for manipulating a knob are discussed in FIGs. 3-9 below.
[0032] FIG. 3 illustrates a rotational gesture 302 in relation to a virtual knob 300. In this input category, the user taps and drags around the edge of the virtual knob in a circular motion. As the user sweeps around the perimeter of the virtual knob, the system updates the knob and the value the knob represents. This input category can apply to virtually any user interface object, even ones that are not round. The user can tap and drag within the space of the virtual knob, on the edge or perimeter of the virtual knob, or outside the virtual knob. In one aspect, if the user input is a tap and drag on a region of the screen outside of the virtual knob, the system can determine to which knob to direct the input based on the user's tap and drag motion. For instance, the system can calculate the intended target of a tap and drag motion by treating the dragging motion as an arc of a circle or ellipse and determining/estimating the center of the circle or ellipse. Then the system can compare the determined/estimated center with user interface elements on the display and identify a best match.
[0033] FIG. 4 illustrates horizontal 404 and vertical gestures 402 in relation to a virtual knob 400. In this input category, the user taps on the touch screen and drags in a straight line. The system treats this input as if the user were dragging a slider bar across an imaginary slider. As the dragging motion goes further in one direction or another, the value for the knob increases or decreases. In one embodiment, the user can tap and drag in any direction, not simply across the horizontal or vertical axes.
[0034] FIG. 5 illustrates point gestures 502, 504, 506 in relation to a virtual knob 500. In this input category, the user taps at various points on the border or edge of the virtual knob. The system can translate the taps at the border of the virtual knob into direct input. For instance, if the knob represents a value from 1 to 12, and the user taps at position 502, then the system can adjust the knob and the value the knob represents to a value of 4. If the user taps at position 504, then the system can adjust the knob and the value of the knob represents to a value of 9. In some instances, the system can provide different functionality to specialized taps, such as a tap in the center or approximately in the center of the knob, such as tap 506. If the knob is currently assigned a particular value, such as 7, and the user taps in the center, the system can set the knob value to 0. If the user taps in the center again, then the system can restore the previous value of 7. A double tap
anywhere on the knob can accomplish a similar functionality as the single tap in the center, for example.
[0035] FIG. 6 illustrates a multi-finger rotational gesture 602, 604 in relation to a virtual knob 600. In this input category, the user simultaneously taps two fingers 602, 604 (or other points of contact) on the touch screen and drags each about the virtual knob in a circular or rotational manner. This motion roughly approximates the motion necessary to manipulate an actual knob and not a virtual knob on a touch-screen. The same general principles apply in this input category as in FIG. 3, but the system can receive or infer additional information from the multiple points of contact, how fast each moves relative to the other, and so forth.
[0036] FIG. 7 illustrates a multi-finger directional gesture 702 in relation to a virtual knob 700. In this input category, the user taps and drags with two or more fingers. The system can treat this gesture in a same, similar, or different manner than the input category illustrated in FIG. 4. For example, the system can treat a single finger directional gesture as a more sensitive motion, while the multi-finger directional gesture is a less sensitive, gross motion. For example, if the knob represents values from 1 to 1000, a single finger directional gesture can increment the knob value by one for each unit of distance. A two finger directional gesture can increment the knob value by ten for the same unit of distance, and a three finger directional gesture can increment the knob value by hundred for the same unit of distance. Thus, the same overall gesture category can include subcategories of similar type, differing in the scale or sensitivity and the number of points of contact on the touch-screen.
[0037] FIG. 8 illustrates a pinch and zoom gesture 802, 804 in relation to a virtual knob 800. In this input category, the user taps two or more fingers and drags at least one away from the other and can drag each away from the other. The scale of the adjustment to the value of the knob is based on the increasing or decreasing distance between the points of finger contact.
[0038] The same user interface element can be configured to accept other types of input. For example, FIG. 9 illustrates a menu gesture 902 in relation to a virtual knob 900. In this input category, the user taps and holds in a center region of the knob (or elsewhere) without dragging. After a delay, the system opens a menu 904 with various options, such as sensitivity 906, a list of gesture types 908, etc. The user can enable or disable certain
gesture types with respect to the entire interface or with respect to a particular user interface element. For instance, the user can disable a rotate gesture 910 by unchecking the checkbox.
[0039] The categories of input disclosed herein are exemplary. The approaches set forth herein can include additional categories of input. The exact positioning of the taps, drags, and other touch gestures are exemplary of multiple different categories and should not be construed as limiting the principles disclosed herein. New input categories and variations of the disclosed input categories can be incorporated using the principles of this disclosure.
[0040] FIG. 10 illustrates an exemplary block diagram 1000 for processing input associated with a virtual knob. A user input device 1002 receives the user input such as touch-screen input. The gesture processor 1004 receives the user input from the user input device 1002. Based on a current screen state representing which user interface elements are currently displayed on the touch-screen, the gesture processor analyzes the user input and compares the user input with entries in a gesture category database 1006. After finding a sufficient match in the gesture category database 1006, the gesture processor 1004 provides some kind of output 1008. The output 1008 can be a value. Alternatively, the output 1008 can be an action taken by the gesture processor 1004 to manipulate a value represented by a user interface element directly or indirectly.
[0041] This approach is flexible and provides for a single user interface element to accept multiple different types of input. For example, the user can provide a first user input of a first input category, such as a circular or rotational gesture about the knob, and a second user input of a second input category, such as a tap in a particular position relative to the knob. These two different types of input can cause the knob to adjust to the same position or value.
[0042] The system can match input to an input category based on a similarity threshold to one of the group of predefined input categories. If the input does not match any input category based on the similarity threshold, the system can prompt the user to for additional input defining a new predefined input category, receive the additional input from the user, and create the new predefined input category based on the additional input.
[0043] Having disclosed some various gesture categories, the disclosure now turns to the exemplary method embodiment shown in FIG. 11. For the sake of clarity, the method is
discussed in terms of an exemplary system 100 as shown in FIG. 1 configured to practice the method. The steps outlined herein are exemplary and can be implemented in any combination thereof, including combinations that exclude, add, or modify certain steps. FIG. 11 illustrates an example method embodiment. A system configured to practice the method first displays a user interface element, such as a virtual on-screen knob representing one or more values, on a touch- sensitive display (1102) and receives, from a user, input associated with the user interface element via the touch-sensitive display (1104). The user interface element can be capable of accepting any of a group of different categories of predefined user input. The input can be a touch, a tap, a swipe, a multi-finger gesture, a flick, a drag, and/or a press-and-hold, for example. The system matches the input to an input category selected from a group of predefined input categories for the user interface element (1106). The system can match the input to the input category based on a similarity threshold to one of the group of predefined input categories. The input can also be multi-modal, such as a combination of a touch gesture and a speech command. In this scenario, the system can match only a portion, such as the touch gesture, of the multi-modal input to the input category. Then the system updates the user interface element based on the input and based on the input category (1108). The system can optionally translate the input to a standardized input based on the input category and update the user interface element based on the standardized input.
[0044] If the input does not match any input category based on the similarity threshold, the system can prompt the user to for additional input defining a new predefined input category, receive the additional input from the user, and create the new predefined input category based on the additional input. In some variations, the system can modify existing input categories instead of creating new predefined categories.
[0045] The user and/or an automated process can toggle availability of a certain input category in the group of predefined input categories. Further, each individual input category can include its own sensitivity and/or scale which can be user customizable. For example, a horizontal tap and drag gesture 3 centimeters in length may be equivalent to a vertical tap and drag gesture 5 centimeters in length. A first user input can update the user interface based on a first input category in an equivalent manner to a second user input based on a second input category. The system can also alter one or more aspect of a running software application based on the input and the input category. The system can
also update the user interface element in real time or substantially real time as the input is received, effectively animating the knob as the user is manipulating the knob via touch input based on the detected input category.
[0046] An exemplary system for controlling a software application according to the principles disclosed herein can include a processor and a touch-sensitive display. The exemplary system can further include a first module configured to control the processor to display a group of user-manipulable user interface elements on the touch-sensitive display, a second module configured to control the processor to receive a group of simultaneous user inputs associated with a subset of the group of user-manipulable user interface elements, a third module configured to control the processor to match each of the group of simultaneous user inputs to a respective input category selected from a group of predefined input categories for the subset of the group of user-manipulable user interface elements, and a fourth module configured to control the processor to control the software application based on each of the group of simultaneous user inputs and the respective input category. Such an exemplary system can be a mobile device. The software application can be a media creation and/or consumption application. The group of simultaneous user inputs can be of a same input category or of different input categories. The system can further include a fifth module configured to control the processor to update the group of user-manipulable user interface elements based on the group of simultaneous user inputs and the respective input category for each simultaneous user input.
[0047] The system can provide for a cancel gesture as an input category separate from other inputs so that a user can cancel a previously submitted input. Further, the system can provide for a cancel gesture added as a continuous part of a pending gesture so that the user can cancel the entire gesture. The user can also change gesture types in the middle of a gesture. For example, the user can start manipulating a knob with a circular motion dragging gesture in a first category, then change categories to a vertical dragging motion without lifting the dragging finger from the touch-screen. This can be advantageous if the circular motion has course-grain control over the knob value and then the user wishes to smoothly change to a more fine-grain control over the knob value afforded by the vertical dragging motion gesture category.
[0048] Using the principles disclosed herein, the user is free to use/manipulate the knob or other user interface elements on a touch-enabled device in whichever way is most natural for the user. The user does not need to read some complicated manual or learn any new behavior to effectively use the knob to control the application. The usability and approachability are much improved for software applications incorporating the principles disclosed herein.
[0049] Embodiments within the scope of the present disclosure may also include tangible and/or non-transitory computer-readable storage media for carrying or having computer- executable instructions or data structures stored thereon. Such non-transitory computer- readable storage media can be any available media that can be accessed by a general purpose or special purpose computer, including the functional design of any special purpose processor as discussed above. By way of example, and not limitation, such non- transitory computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions, data structures, or processor chip design. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or combination thereof) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of the computer-readable media.
[0050] Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Computer- executable instructions also include program modules that are executed by computers in stand-alone or network environments. Generally, program modules include routines, programs, components, data structures, objects, and the functions inherent in the design of special-purpose processors, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or
associated data structures represents examples of corresponding acts for implementing the functions described in such steps.
[0051] Those of skill in the art will appreciate that other embodiments of the disclosure may be practiced in network computing environments with many types of computer system configurations, including personal computers, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Embodiments may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination thereof) through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
[0052] The various embodiments described above are provided by way of illustration only and should not be construed to limit the scope of the disclosure. For example, the principles herein can be incorporated in audio creation/consumption programs, touch- enabled games, artistic/creative applications, and virtually any other touch-aware software application. Those skilled in the art will readily recognize various modifications and changes that may be made to the principles described herein without following the example embodiments and applications illustrated and described herein, and without departing from the spirit and scope of the disclosure.
Claims
1. A method comprising:
displaying a user interface element on a touch- sensitive display;
receiving, from a user, input associated with the user interface element via the touch-sensitive display;
based on an analysis of the input, matching the input to an input category selected from a plurality of predefined input categories for the user interface element; and
updating the user interface element based on the input and the input category.
2. The method of claim 1, wherein the input is at least one of a touch, a tap, a swipe, a multi-finger gesture, a flick, a drag, and a press-and-hold.
3. The method of any of claims 1-2, wherein a first user input can update the user interface based on a first input category in an equivalent manner to a second user input based on a second input category.
4. The method of any of claims 1-3, wherein matching the input to the input category is based on a similarity threshold to one of the plurality of predefined input categories.
5. The method of any of claims 1-4, further comprising, if the input does not match any input category based on the similarity threshold:
prompting the user to for additional input defining a new predefined input category;
receiving the additional input from the user; and
creating the new predefined input category based on the additional input.
6. The method of any of claims 1-5, further comprising:
translating the input to a standardized input based on the input category; and updating the user interface element based on the standardized input.
7. The method of any of claims 1-6, wherein the user can toggle availability of a certain input category in the plurality of predefined input categories.
8. The method of any of claims 1-7, wherein the user interface element is a virtual knob.
9. The method of any of claims 1-8, wherein a sensitivity for at least one input category in the plurality of input categories is user customizable.
10. The method of any of claims 1-9, wherein the input is multi-modal input, and wherein a portion of the multi-modal input is matched to the input category.
11. The method of any of claims 1-10, further comprising altering at least one aspect of a running software application based on the input and the input category.
12. The method of any of claims 1-11, wherein the user interface element is updated in substantially real time as the input is received.
13. A system comprising:
a processor;
a touch- sensitive display;
a computer-readable storage medium storing instructions which, when executed by the processor, cause the processor to perform steps comprising:
displaying a plurality of user-manipulable user interface elements on the touch-sensitive display;
receiving a plurality of simultaneous user inputs associated with a subset of the plurality of user-manipulable user interface elements;
based on an analysis of the plurality of simultaneous user inputs, matching each of the plurality of simultaneous user inputs to a respective input category selected from a plurality of predefined input categories for the subset of the plurality of user- manipulable user interface elements; and
controlling a software application based on each of the plurality of simultaneous user inputs and the respective input category.
14. The system of claim 13, wherein the instructions, when executed by the processor, further cause the processor to perform steps comprising:
translating the plurality of simultaneous user inputs to standardized inputs based on the respective input category; and
updating at least one of the plurality of user-manipulable user interface elements based on the standardized inputs.
15. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform steps comprising:
displaying, on a touch- sensitive display, a user interface element capable of accepting any of a plurality of different categories of predefined user input;
receiving, from a user, input associated with the user interface element via the touch-sensitive display;
based on an analysis of the input, matching the input to an input category selected from the plurality of predefined input categories for the user interface element; and
updating the user interface element based on the input and the input category.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/038,276 US9547428B2 (en) | 2011-03-01 | 2011-03-01 | System and method for touchscreen knob control |
US13/038,276 | 2011-03-01 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012118626A1 true WO2012118626A1 (en) | 2012-09-07 |
Family
ID=45809644
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2012/025519 WO2012118626A1 (en) | 2011-03-01 | 2012-02-16 | System and method for touchscreen knob control |
Country Status (2)
Country | Link |
---|---|
US (1) | US9547428B2 (en) |
WO (1) | WO2012118626A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
USD776126S1 (en) | 2014-02-14 | 2017-01-10 | Samsung Electronics Co., Ltd. | Display screen or portion thereof with a transitional graphical user interface |
US9628543B2 (en) | 2013-09-27 | 2017-04-18 | Samsung Electronics Co., Ltd. | Initially establishing and periodically prefetching digital content |
IT201900019892A1 (en) | 2019-10-28 | 2021-04-28 | Fater Spa | CONTAINER FOR HOUSEHOLD WASTE, IN PARTICULAR FOR USED ABSORBENT SANITARY PRODUCTS |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2013035744A1 (en) * | 2011-09-07 | 2015-03-23 | Necカシオモバイルコミュニケーションズ株式会社 | Terminal device, information input method and program |
US9601113B2 (en) | 2012-05-16 | 2017-03-21 | Xtreme Interactions Inc. | System, device and method for processing interlaced multimodal user input |
EP2898600B1 (en) * | 2012-09-21 | 2020-06-10 | Diehl AKO Stiftung & Co. KG | Virtual touch knob assembly |
US10678352B2 (en) * | 2012-09-21 | 2020-06-09 | Diehl Ako Stiftung & Co. Kg | Virtual touch knob assembly |
US10139937B2 (en) * | 2012-10-12 | 2018-11-27 | Microsoft Technology Licensing, Llc | Multi-modal user expressions and user intensity as interactions with an application |
US9569107B2 (en) * | 2012-10-16 | 2017-02-14 | Google Inc. | Gesture keyboard with gesture cancellation |
US9632584B2 (en) | 2013-01-08 | 2017-04-25 | 2236008 Ontario Inc. | On-demand user control |
CN110141689A (en) * | 2013-03-14 | 2019-08-20 | 史密夫和内修有限公司 | System and method for application decompression treatment |
US8731824B1 (en) | 2013-03-15 | 2014-05-20 | Honda Motor Co., Ltd. | Navigation control for a touch screen user interface |
US9195391B2 (en) | 2013-04-19 | 2015-11-24 | International Business Machines Corporation | Touch sensitive data entry using a continuous gesture |
US9880727B2 (en) | 2013-05-29 | 2018-01-30 | Microsoft Technology Licensing, Llc | Gesture manipulations for configuring system settings |
US9109921B1 (en) * | 2013-06-19 | 2015-08-18 | Amazon Technologies, Inc. | Contextual based navigation element |
CN104281380B (en) * | 2013-07-08 | 2018-01-30 | 纬创资通(昆山)有限公司 | The operating method of virtual adjusting knob |
WO2015031500A1 (en) | 2013-08-27 | 2015-03-05 | Duke Manufacturing Co. | Food management system |
EP3605286B1 (en) | 2013-09-03 | 2021-02-17 | Apple Inc. | User interface for manipulating user interface objects |
US11068128B2 (en) | 2013-09-03 | 2021-07-20 | Apple Inc. | User interface object manipulations in a user interface |
US10545657B2 (en) | 2013-09-03 | 2020-01-28 | Apple Inc. | User interface for manipulating user interface objects |
US10503388B2 (en) | 2013-09-03 | 2019-12-10 | Apple Inc. | Crown input for a wearable electronic device |
JP2016530659A (en) * | 2013-09-17 | 2016-09-29 | コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. | Simultaneous selection of range and value gestures |
US9582184B2 (en) * | 2013-11-04 | 2017-02-28 | Keysight Technologies, Inc. | Touch screen control for adjusting a numerical value |
CN116301544A (en) | 2014-06-27 | 2023-06-23 | 苹果公司 | Reduced size user interface |
US10146409B2 (en) | 2014-08-29 | 2018-12-04 | Microsoft Technology Licensing, Llc | Computerized dynamic splitting of interaction across multiple content |
TWI676127B (en) | 2014-09-02 | 2019-11-01 | 美商蘋果公司 | Method, system, electronic device and computer-readable storage medium regarding electronic mail user interface |
US10082892B2 (en) | 2014-09-02 | 2018-09-25 | Apple Inc. | Button functionality |
CN110072131A (en) | 2014-09-02 | 2019-07-30 | 苹果公司 | Music user interface |
US10073590B2 (en) | 2014-09-02 | 2018-09-11 | Apple Inc. | Reduced size user interface |
FR3026865B1 (en) * | 2014-10-03 | 2016-12-09 | Thales Sa | METHOD FOR DISPLAYING AND MANAGING INTERACTION SYMBOLS AND ASSOCIATED TOUCH-SURFACE VISUALIZATION DEVICE |
US9195225B1 (en) * | 2014-10-16 | 2015-11-24 | Handi Quilter, Inc. | Graphical user interface virtual handwheel for a controller of a motor |
US8997669B1 (en) | 2014-10-16 | 2015-04-07 | Handi Quilter, Inc. | Thread tensioner for a sewing machine |
US9394639B2 (en) | 2014-10-16 | 2016-07-19 | Handi Quilter, Inc. | Motorized thread tensioner for a sewing machine |
ES2972416T3 (en) * | 2014-11-07 | 2024-06-12 | Esab Group Inc | Techniques for controlling a portable welding system |
US10365807B2 (en) | 2015-03-02 | 2019-07-30 | Apple Inc. | Control of system zoom magnification using a rotatable input mechanism |
US9916032B2 (en) | 2016-05-18 | 2018-03-13 | Honeywell International Inc. | System and method of knob operation for touchscreen devices |
DE202016105915U1 (en) * | 2016-10-21 | 2016-11-16 | Ma Lighting Technology Gmbh | Lighting console with rotary control |
USD835648S1 (en) * | 2016-10-27 | 2018-12-11 | Smith & Nephew, Inc. | Display screen or portion thereof with a graphical user interface for a therapy device |
US11435830B2 (en) | 2018-09-11 | 2022-09-06 | Apple Inc. | Content-based tactile outputs |
DK179888B1 (en) | 2018-09-11 | 2019-08-27 | Apple Inc. | CONTENT-BASED TACTICAL OUTPUTS |
WO2020079729A1 (en) * | 2018-10-15 | 2020-04-23 | 三菱電機株式会社 | Touch panel input device, touch panel input method, and program |
US10884614B1 (en) * | 2018-11-30 | 2021-01-05 | Zoox, Inc. | Actuation interface |
US11435868B2 (en) | 2018-12-04 | 2022-09-06 | Baker Hughes Holdings Llc | Precise value selection within large value ranges |
US11119602B2 (en) | 2019-08-09 | 2021-09-14 | Cypress Semiconductor Corporation | Detecting the angle of a touch screen mounted passive dial |
US11494286B2 (en) | 2020-01-27 | 2022-11-08 | Salesforce.Com, Inc. | Dynamic adjustment of web product-based performance factors in database systems |
US11281563B2 (en) * | 2020-01-27 | 2022-03-22 | Salesforce.Com, Inc. | Actionable insights for performance of web products in database systems |
US11573880B2 (en) | 2020-01-27 | 2023-02-07 | Salesforce.Com, Inc. | Performance simulation for selected platforms for web products in database systems |
US11429577B2 (en) | 2020-01-27 | 2022-08-30 | Salesforce.Com, Inc. | Performance simulation and cost-benefit analysis for performance factors for web products in database systems |
USD997107S1 (en) * | 2020-12-04 | 2023-08-29 | Zoox, Inc. | Door interface system having an animated visual indicator |
US11803260B1 (en) * | 2022-08-11 | 2023-10-31 | Cypress Semiconductor Corporation | Detecting the angle of passive rotary knob partially located on touch screen |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1942401A1 (en) * | 2007-01-05 | 2008-07-09 | Apple Inc. | Multimedia communication device with touch screen responsive to gestures for controlling, manipulating and editing of media files |
EP1956446A2 (en) * | 2007-02-07 | 2008-08-13 | LG Electronics Inc. | Electronic device with a touchscreen displaying an analog clock |
Family Cites Families (70)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH01177120A (en) | 1988-01-06 | 1989-07-13 | Matsushita Electric Ind Co Ltd | Display device for operation key |
US5448686A (en) | 1992-01-02 | 1995-09-05 | International Business Machines Corporation | Multi-resolution graphic representation employing at least one simplified model for interactive visualization applications |
US5483261A (en) | 1992-02-14 | 1996-01-09 | Itu Research, Inc. | Graphical input controller and method with rear screen image detection |
DE69430967T2 (en) | 1993-04-30 | 2002-11-07 | Xerox Corp | Interactive copying system |
JP3290600B2 (en) | 1996-10-29 | 2002-06-10 | 新日鉄ソリューションズ株式会社 | Network system |
US6396507B1 (en) | 1996-09-13 | 2002-05-28 | Nippon Steel Corporation | Data storage/access network system for zooming image and method of the storage/access |
US6954899B1 (en) | 1997-04-14 | 2005-10-11 | Novint Technologies, Inc. | Human-computer interface including haptically controlled interactions |
US6073036A (en) | 1997-04-28 | 2000-06-06 | Nokia Mobile Phones Limited | Mobile station with touch input having automatic symbol magnification function |
EP1717684A3 (en) | 1998-01-26 | 2008-01-23 | Fingerworks, Inc. | Method and apparatus for integrating manual input |
US7760187B2 (en) | 2004-07-30 | 2010-07-20 | Apple Inc. | Visual expander |
US8479122B2 (en) | 2004-07-30 | 2013-07-02 | Apple Inc. | Gestures for touch sensitive input devices |
JP2002503854A (en) | 1998-02-17 | 2002-02-05 | サン・マイクロシステムズ・インコーポレーテッド | Estimating graphics system performance for polygons |
US6211856B1 (en) | 1998-04-17 | 2001-04-03 | Sung M. Choi | Graphical user interface touch screen with an auto zoom feature |
US6278443B1 (en) | 1998-04-30 | 2001-08-21 | International Business Machines Corporation | Touch screen with random finger placement and rolling on screen to control the movement of information on-screen |
US6421051B1 (en) | 1998-06-18 | 2002-07-16 | Spatial Corporation | Multi-resolution geometry |
US7358956B2 (en) | 1998-09-14 | 2008-04-15 | Microsoft Corporation | Method for providing feedback responsive to sensing a physical presence proximate to a control of an electronic device |
US6950539B2 (en) | 1998-09-16 | 2005-09-27 | Digital Persona | Configurable multi-function touchpad device |
JP2000163193A (en) | 1998-11-25 | 2000-06-16 | Seiko Epson Corp | Portable information equipment and information storage medium |
US7469381B2 (en) | 2007-01-07 | 2008-12-23 | Apple Inc. | List scrolling and document translation, scaling, and rotation on a touch-screen display |
JP2001134382A (en) | 1999-11-04 | 2001-05-18 | Sony Corp | Graphic processor |
JP4803883B2 (en) | 2000-01-31 | 2011-10-26 | キヤノン株式会社 | Position information processing apparatus and method and program thereof. |
US6766456B1 (en) | 2000-02-23 | 2004-07-20 | Micron Technology, Inc. | Method and system for authenticating a user of a computer system |
WO2001071478A2 (en) * | 2000-03-22 | 2001-09-27 | Sony Electronics Inc | Data entry user interface |
US20020057385A1 (en) | 2000-07-14 | 2002-05-16 | Mingui Sun | Data processing for fast transmission of webpages |
US20020015064A1 (en) | 2000-08-07 | 2002-02-07 | Robotham John S. | Gesture-based user interface to multi-level and multi-modal sets of bit-maps |
AU2001286645A1 (en) | 2000-08-31 | 2002-03-13 | Zframe, Inc | Gesture-based user interface to multi-level and multi-modal sets of bit-maps |
US20050134578A1 (en) | 2001-07-13 | 2005-06-23 | Universal Electronics Inc. | System and methods for interacting with a control environment |
US7171625B1 (en) | 2002-06-18 | 2007-01-30 | Actify, Inc. | Double-clicking a point-and-click user interface apparatus to enable a new interaction with content represented by an active visual display element |
US7385615B2 (en) | 2002-10-21 | 2008-06-10 | Microsoft Corporation | System and method for scaling images to fit a screen on a mobile device according to a non-linear scale factor |
JP4518955B2 (en) | 2002-11-29 | 2010-08-04 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | User interface using moved representation of contact area |
US7762665B2 (en) | 2003-03-21 | 2010-07-27 | Queen's University At Kingston | Method and apparatus for communication between humans and devices |
US20040207653A1 (en) | 2003-04-17 | 2004-10-21 | Stavely Donald J. | Systems and methods for controlling a display |
JP2004363892A (en) | 2003-06-04 | 2004-12-24 | Canon Inc | Portable apparatus |
US20060242607A1 (en) | 2003-06-13 | 2006-10-26 | University Of Lancaster | User interface |
JP2005026836A (en) | 2003-06-30 | 2005-01-27 | Canon Inc | Image display device |
US9024884B2 (en) * | 2003-09-02 | 2015-05-05 | Apple Inc. | Touch-sensitive electronic apparatus for media applications, and methods therefor |
JP4360871B2 (en) | 2003-09-10 | 2009-11-11 | 富士通テン株式会社 | Input device in information terminal |
US7411575B2 (en) | 2003-09-16 | 2008-08-12 | Smart Technologies Ulc | Gesture recognition method and touch system incorporating the same |
US7471827B2 (en) | 2003-10-16 | 2008-12-30 | Microsoft Corporation | Automatic browsing path generation to present image areas with high attention value as a function of space and time |
US7746404B2 (en) | 2003-11-10 | 2010-06-29 | Hewlett-Packard Development Company, L.P. | Digital camera with panoramic image capture |
JP4250543B2 (en) | 2004-02-06 | 2009-04-08 | キヤノン株式会社 | Imaging apparatus, information processing apparatus, and control method thereof |
US7542050B2 (en) | 2004-03-03 | 2009-06-02 | Virtual Iris Studios, Inc. | System for delivering and enabling interactivity with images |
US7519223B2 (en) | 2004-06-28 | 2009-04-14 | Microsoft Corporation | Recognizing gestures and using gestures for interacting with software applications |
US7653883B2 (en) | 2004-07-30 | 2010-01-26 | Apple Inc. | Proximity detector in handheld device |
US8381135B2 (en) | 2004-07-30 | 2013-02-19 | Apple Inc. | Proximity detector in handheld device |
JP4763695B2 (en) | 2004-07-30 | 2011-08-31 | アップル インコーポレイテッド | Mode-based graphical user interface for touch-sensitive input devices |
EP1774427A2 (en) | 2004-07-30 | 2007-04-18 | Apple Computer, Inc. | Mode-based graphical user interfaces for touch sensitive input devices |
US7728821B2 (en) | 2004-08-06 | 2010-06-01 | Touchtable, Inc. | Touch detecting interactive display |
US7721197B2 (en) | 2004-08-12 | 2010-05-18 | Microsoft Corporation | System and method of displaying content on small screen computing devices |
US20070198942A1 (en) | 2004-09-29 | 2007-08-23 | Morris Robert P | Method and system for providing an adaptive magnifying cursor |
US7688364B2 (en) | 2004-12-10 | 2010-03-30 | Ambarella, Inc. | Decimating and cropping based zoom factor for a digital camera |
US7489306B2 (en) | 2004-12-22 | 2009-02-10 | Microsoft Corporation | Touch screen accuracy |
US7760189B2 (en) | 2005-01-21 | 2010-07-20 | Lenovo Singapore Pte. Ltd | Touchpad diagonal scrolling |
US20060181519A1 (en) | 2005-02-14 | 2006-08-17 | Vernier Frederic D | Method and system for manipulating graphical objects displayed on a touch-sensitive display surface using displaced pop-ups |
US7605804B2 (en) | 2005-04-29 | 2009-10-20 | Microsoft Corporation | System and method for fine cursor positioning using a low resolution imaging touch screen |
US7487118B2 (en) | 2005-05-06 | 2009-02-03 | Crutchfield Corporation | System and method of image display simulation |
US20070061126A1 (en) | 2005-09-01 | 2007-03-15 | Anthony Russo | System for and method of emulating electronic input devices |
US8024768B2 (en) | 2005-09-15 | 2011-09-20 | Penthera Partners, Inc. | Broadcasting video content to devices having different video presentation capabilities |
US7480870B2 (en) | 2005-12-23 | 2009-01-20 | Apple Inc. | Indication of progress towards satisfaction of a user input condition |
US7649526B2 (en) | 2005-12-23 | 2010-01-19 | Apple Inc. | Soft key interaction indicator |
CN101379461A (en) | 2005-12-30 | 2009-03-04 | 苹果公司 | Portable electronic device with multi-touch input |
US7720552B1 (en) * | 2006-06-05 | 2010-05-18 | Rockwell Automation Technologies, Inc. | Virtual knob lever arm as analog control element |
US7940250B2 (en) | 2006-09-06 | 2011-05-10 | Apple Inc. | Web-clip widgets on a portable multifunction device |
KR20080073549A (en) | 2007-02-06 | 2008-08-11 | 삼성전자주식회사 | Manufacturing method of photoresist patten and manufacturing method of dispaly panel |
US20090089448A1 (en) | 2007-09-28 | 2009-04-02 | David Sze | Mobile browser with zoom operations using progressive image download |
KR101629645B1 (en) * | 2009-09-18 | 2016-06-21 | 엘지전자 주식회사 | Mobile Terminal and Operation method thereof |
US9547840B2 (en) * | 2009-12-28 | 2017-01-17 | Google Technology Holdings LLC | Methods for associating objects on a touch screen using input gestures |
US10996774B2 (en) * | 2010-04-30 | 2021-05-04 | Nokia Technologies Oy | Method and apparatus for providing interoperability between devices |
US8762893B2 (en) * | 2010-05-14 | 2014-06-24 | Google Inc. | Automatic derivation of analogous touch gestures from a user-defined gesture |
US20120068925A1 (en) * | 2010-09-21 | 2012-03-22 | Sony Corporation | System and method for gesture based control |
-
2011
- 2011-03-01 US US13/038,276 patent/US9547428B2/en active Active
-
2012
- 2012-02-16 WO PCT/US2012/025519 patent/WO2012118626A1/en active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1942401A1 (en) * | 2007-01-05 | 2008-07-09 | Apple Inc. | Multimedia communication device with touch screen responsive to gestures for controlling, manipulating and editing of media files |
EP1956446A2 (en) * | 2007-02-07 | 2008-08-13 | LG Electronics Inc. | Electronic device with a touchscreen displaying an analog clock |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9628543B2 (en) | 2013-09-27 | 2017-04-18 | Samsung Electronics Co., Ltd. | Initially establishing and periodically prefetching digital content |
USD776126S1 (en) | 2014-02-14 | 2017-01-10 | Samsung Electronics Co., Ltd. | Display screen or portion thereof with a transitional graphical user interface |
IT201900019892A1 (en) | 2019-10-28 | 2021-04-28 | Fater Spa | CONTAINER FOR HOUSEHOLD WASTE, IN PARTICULAR FOR USED ABSORBENT SANITARY PRODUCTS |
EP3816068A1 (en) | 2019-10-28 | 2021-05-05 | Fater S.p.A. | A domestic waste bin, in particular, for used absorbent sanitary products |
Also Published As
Publication number | Publication date |
---|---|
US9547428B2 (en) | 2017-01-17 |
US20120226977A1 (en) | 2012-09-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9547428B2 (en) | System and method for touchscreen knob control | |
US9146672B2 (en) | Multidirectional swipe key for virtual keyboard | |
US9110584B2 (en) | Controlling a cursor on a touch screen | |
US8878787B2 (en) | Multi-touch user input based on multiple quick-point controllers | |
US9304656B2 (en) | Systems and method for object selection on presence sensitive devices | |
US20140098142A1 (en) | System and method for generation and manipulation of a curve in a dynamic graph based on user input | |
US20140344765A1 (en) | Touch Sensitive UI Pinch and Flick Techniques for Managing Active Applications | |
US20100251112A1 (en) | Bimodal touch sensitive digital notebook | |
US20120223959A1 (en) | System and method for a touchscreen slider with toggle control | |
US9652142B2 (en) | System and method of mode-switching for a computing device | |
CN104007894A (en) | Portable device and method for operating multiapplication thereof | |
US20120026077A1 (en) | Mapping trackpad operations to touchscreen events | |
US20140215393A1 (en) | Touch-based multiple selection | |
JP2021002381A (en) | Method for input by touch sensitive surface-display, electronic apparatus, and method and system for input control by tactile sense-visual sense technology | |
US8842088B2 (en) | Touch gesture with visible point of interaction on a touch screen | |
CN103809903B (en) | Method and apparatus for controlling virtual screen | |
TWI564780B (en) | Touchscreen gestures | |
JP7109448B2 (en) | dynamic spacebar | |
US9081547B2 (en) | System and method of automatic switching to a text-entry mode for a computing device | |
CN103809794A (en) | Information processing method and electronic device | |
TWI681307B (en) | Electronic device, storage medium and method for processing letter input in electronic device | |
KR101366170B1 (en) | User Interface for controlling state of menu | |
US20160117000A1 (en) | Touchscreen input method and apparatus | |
Albanese et al. | A technique to improve text editing on smartphones | |
KR101791132B1 (en) | Method and apparatus for multi-touch input |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12707434 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 12707434 Country of ref document: EP Kind code of ref document: A1 |