US20110128224A1 - User interface device, such as a mouse or trackball, with a high-dimension joystick providing at least three independently adjustable parameters - Google Patents

User interface device, such as a mouse or trackball, with a high-dimension joystick providing at least three independently adjustable parameters Download PDF

Info

Publication number
US20110128224A1
US20110128224A1 US13/025,129 US201113025129A US2011128224A1 US 20110128224 A1 US20110128224 A1 US 20110128224A1 US 201113025129 A US201113025129 A US 201113025129A US 2011128224 A1 US2011128224 A1 US 2011128224A1
Authority
US
United States
Prior art keywords
user interface
interface device
signals
trackball
joystick
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/025,129
Inventor
Lester F. Ludwig
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BIORAM TECH LLC
Original Assignee
Ludwig Lester F
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ludwig Lester F filed Critical Ludwig Lester F
Priority to US13/025,129 priority Critical patent/US20110128224A1/en
Publication of US20110128224A1 publication Critical patent/US20110128224A1/en
Assigned to BIORAM TECH L.L.C. reassignment BIORAM TECH L.L.C. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LUDWIG, LESTER F.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03543Mice or pucks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03547Touch pads, in which fingers can move on a surface
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G06F3/0383Signal control means within the pointing device
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0414Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using force sensing means to determine a position
    • G06F3/04144Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using force sensing means to determine a position using an array of force sensing means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26Power supply means, e.g. regulation thereof
    • G06F1/266Arrangements to supply power to external peripherals either directly from the computer or under computer control, e.g. supply of power through the communication port, computer controlled power-strips
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038Indexing scheme relating to G06F3/038
    • G06F2203/0384Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04104Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04106Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03549Trackballs

Definitions

  • This invention relates to user interface devices for use with a computer, and in particular to computer mice, trackballs, touchpads, multiple-parameter pointing and data entry devices, and user interface metaphors.
  • User interface devices for data entry and graphical user interface pointing have been known for many years.
  • the most common devices include the computer mouse (usually attributed to English, Engelbart, and Bennan “Display-Selection Techniques for Text Manipulation, IEEE Transactions on Human Factors in Electronics, pp. 5-15, vol. HFE-8, No. 1, March 1967), the trackball, the touchpad in both finger-operated (for example, the various finger-operated devices produced by Symantec Corp., of Springfield, Oreg.) and stylus-operated (for example, products used with desktop workstation computers by Wacom Technology Corp., of Vancouver, Wash.) versions, and display-overlay touchscreens.
  • Other historical and exotic devices include various types of light pens and the Data GloveTM (produced by VPL Research, Inc., of Redwood City, Calif.).
  • Most user interface devices for data entry and graphical user interface pointing commonly used with computers or with equipment providing computer-like user interfaces have two wide-range parameter adjustment capabilities that are usually assigned to the task of positioning a screen cursor within a two-dimensional display.
  • one, two, or three binary-valued “discrete-event” controls are provided, typically in the form of spring-loaded push-buttons.
  • two independently adjustable and positionable cursors are employed in a visual interface for editing an electronic document.
  • the document may comprise text symbols, text objects, and graphics objects, among others. If both cursors are located in close proximity within the document, then these cursors may be simultaneously displayed in a single window. Otherwise, two areas of the document, each comprising one of the cursors, may be simultaneously displayed using a separate window for each cursor, or alternately selected and displayed in a single window. Copy or cut operations may be made with one cursor, while paste operations may be repeated using the second cursor. Cursor locations may be left unchanged, or they may be moved between or within editing operations.
  • the two cursors may be freely controlled by an enhanced pointing device, such as a mouse with an added touchpad or trackball, or sequentially selected and controlled by a conventional mouse.
  • a traditional hand-movable computer mouse is configured with an additional user interface sensor.
  • user interface sensor will be used herein to collectively refer to devices such as trackballs, touchpads, mouse devices, scroll-wheels, joysticks, and other such devices.
  • the addition of a user interface sensor provides alternative physical modalities for the same pair of adjustable parameters so that a user may switch between using the user interface device as a traditional hand-movable computer mouse and using the user interface device as a trackball or touchpad.
  • the addition of a user interface sensor provides alternative resolution modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one level of parameter adjustment resolution, and using the invention as a trackball or touchpad, for example, to obtain a different level of parameter adjustment resolution.
  • the addition of a user interface sensor provides alternative types of warping modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one type of parameter adjustment (for example, linear) and using the invention as a trackball or touchpad, for example, to obtain a different type of parameter adjustment (for example, logarithmic, gamma-corrected, arccosine, exponential, etc.).
  • the addition of a user interface sensor provides alternative offset modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one type of centering of parameter adjustment and using the invention as a trackball or touchpad, for example, to obtain a different centering of parameter adjustment.
  • the addition of a user interface sensor may be used to provide additional parameters that may be simultaneously controlled.
  • the addition of a user interface sensor may be used to provide additional parameters that are of a different isolated context from those assigned to a traditional hand-movable computer mouse.
  • the addition of a touchpad may be used to provide many additional parameters that are of a different context than those of a traditional hand-movable computer mouse.
  • the touchpad may be a null-contact touchpad adapted to measure at least one maximum spatial span of contact in a given direction.
  • the null-contact touchpad is adapted to measure at least one maximum spatial span of contact in a given direction at a specifiable angle.
  • the null-contact touchpad is adapted to measure pressure applied to the null-contact touchpad.
  • the touchpad may comprise a pressure sensor array touchpad adapted to measure, among other things, one or more of the following: the rocking position of a contacting finger in a given direction; the rotational position of a contacting finger; the pressure of a contacting finger; and parameters relating to a plurality of contacting fingers.
  • the addition of a user interface sensor may be realized via a replaceable module accepted by an adaptation of a traditional hand-movable computer mouse.
  • a user may initially obtain the invention in one configuration and field-modify it to another configuration.
  • a traditional hand-movable computer mouse may be implemented as a removable module in a laptop computer or other affiliated equipment, and may include a wireless link with the laptop computer or other affiliated equipment.
  • a traditional hand-movable computer mouse is implemented as a removable module in a laptop computer or other affiliated equipment, and the mouse further comprises a user interface sensor.
  • a traditional hand-movable computer mouse additionally comprises a trackball or touchpad, for example.
  • the mouse comprises a wireless link to an associated computer or other affiliated equipment.
  • a visual display is provided.
  • an auditory output is provided.
  • two or more individual user interface sensors may be combined without incorporation of such sensors with a traditional hand-movable computer mouse.
  • FIGS. 1 a - 1 i illustrate various exemplary implementations involving merging, selecting, multiplexing, and preprocessing distributed in various ways between the body of the user interface device and an associated piece of equipment;
  • FIGS. 2 a - 2 c depict an embodiment of the invention comprising a traditional mouse fitted with a trackball, illustrating three exemplary button configurations
  • FIGS. 3 a - 3 c depict an embodiment of the invention comprising a traditional mouse fitted with a touchpad, illustrating three exemplary button configurations
  • FIGS. 4 a - 4 d depict various exemplary degrees of freedom that may be measurably assigned to a trackball for interactively controlling parameters in a user interface
  • FIGS. 5 a - 5 d depict various exemplary degrees of freedom that may be measurably assigned to a touchpad for interactively controlling parameters in a user interface
  • FIG. 6 depicts an exemplary implementation of the invention directed towards the control of both a traditional text cursor and a dual-scrollbar in a typesetting application
  • FIG. 7 depicts an exemplary implementation of the invention directed towards the active selection from a clip-art or symbol library and adjustment of positioning or other attributes of the active selection in a drawing or layout application;
  • FIG. 8 is a flowchart showing exemplary operations and overhead involved in selecting and adjusting a specific pair of parameters from among a larger group of adjustable parameters
  • FIGS. 9 a - 9 b illustrate how the exemplary operations and overhead depicted in FIG. 8 introduce excessive overhead in situations where many parameters with a larger group of adjustable parameters must be adjusted in pairs;
  • FIGS. 10 a - 10 b illustrate one technique for adding an additional scroll-wheel to a conventional scroll-wheel mouse
  • FIGS. 11 a - 11 b illustrate a simple example of open adjustments being made within various levels of hierarchy of graphical object groupings
  • FIG. 12 illustrates aspects of the 3D orientation of an object in 3-dimensional space, and in particular the three coordinates of position and the three angles of rotation;
  • FIGS. 13 a - 13 b illustrate one technique for using two cursors in a text cut-and-paste operation
  • FIGS. 14 a - 14 d illustrate exemplary embodiments of a mouse where the traditional mouse buttons have been replaced by trackballs or touchpads.
  • an additional finger-wheel adjustment device providing a third, wide-range parameter adjustment capability is typically assigned to vertical scroll bar positioning.
  • such a design may be supplemented with a fourth, wide-range parameter adjustment capability so that a horizontal scroll bar position control may be achieved.
  • Some opportunities have wide-range applicability, such as in providing interactive separate adjustment of the selections for “cut” or “copy” operations from the interactive adjustment of insertion location or selection for a “paste” operation.
  • Other opportunities are more specialized but still widely applicable, such as making an active selection fiom a clip-art or symbol library and adjusting the position or other attributes of said active selection in a drawing or layout application.
  • Yet other opportunities may be very specialized, such as in 3D modeling, data visualization, advanced color adjustment, lighting control, machine control, or audio and image signal processing.
  • one user interface modality (such as normal mouse operation) may be used for normal parameter adjustment, while a second user interface modality may be used for adjustments involving a different resolution, warping (i.e., logarithmic, gamma-corrected, arccosine, exponential, etc.), centering offset, etc.
  • warping i.e., logarithmic, gamma-corrected, arccosine, exponential, etc.
  • centering offset etc.
  • Another important case is where the same two widely-varying parameters are controlled with the same resolution, warping, offset, etc., but in a different user interface modality (e.g., a trackball or touchpad may have some advantages in certain situations over use of a traditional mouse).
  • a more widely applicable example is that of responding to and preventing handlwrist/arm fatigue and injury.
  • a traditional mouse fitted with an additional user interface sensor allows a user to interchangeably enter information with either the mouse body or another user interface sensor, changing which user interface modality is used (obtaining
  • a user interface sensor provides many opportunities for alternative means of adjustment of a common pair of adjustable parameters.
  • the user may benefit from having both adjustment modalities available, changing modalities as needed or desired. For example:
  • the addition of another user interface sensor provides many opportunities for the simultaneous adjustment of additional parameters that may or may not require simultaneous interactive control.
  • the traditional computer mouse may be used to simultaneously adjust two parameters while the additional user interface sensor may be configured to allow the fingers to simultaneously adjust at least two additional parameters.
  • these additional parameters may be closely related to those assigned to the traditional computer mouse.
  • the traditional computer mouse may be used to simultaneously adjust the location within a window of a text, graphic, or other object, while the additional user interface sensor allows the fingers to be used to adjust the type or attributes of the text, graphic, or other object.
  • these additional parameters may be of a different isolated context from those assigned to the traditional computer mouse.
  • the traditional computer mouse may be used to simultaneously adjust two parameters dealing with affairs within an active application window, while the addition of another user interface sensor allows the fingers to be used to adjust at least two additional parameters dealing with broader window system affairs such as vertical and horizontal scrollbars, window selection, window resizing, etc., or intermediate-level affairs such as zoom control, help-window navigation, clip-art selection, etc.
  • Another application would be to provide separate adjustment of selections for “cut” or “copy” operations from the adjustment of insertion location or selection for a “paste” operation.
  • the touchpad may be configured and/or enhanced to allow the fingers to adjust three or more additional interactive measured parameters.
  • additional interactive measured parameters may be assigned to control more sophisticated interactive affairs such as 3-dimensional space position, 3-dimensional space orientation, color model navigation, image or audio processing parameter settings, etc.
  • the additional interactive measured parameters may be provided in a number of ways.
  • the touchpad may be a relatively low-cost null-contact touchpad that has been adapted to measure at least one maximum spatial span of contact in a given direction.
  • the user may also control an additional parameter by varying the width between the spatial extremes of a single point of contact (i.e., how much finger flesh makes contact with the pad) or multiple points of contact (i.e., the spread between two contacting fingers).
  • this provides the user with a method for controlling four total parameters from a touchpad.
  • rotational transformations or other methodologies may be used to measure the angle of rotation of an oblong contact profile.
  • the measured angle may be used as a fifth interactive parameter, and/or used to adapt the measurement of maximum spatial span of contact in an arbitrary angle.
  • the null-contact touchpad may be further adapted to measure pressure applied to the null-contact touchpad via, for example, use of an attached pressure sensor.
  • the pressure may be used as a sixth interactive parameter, and may or may not have rapid pressure changes recognized as ‘tap’ or ‘click’ events.
  • a pressure sensor array touchpad can be adapted to measure the rocking position of a contacting finger in two orthogonal directions, as well as the rotational position and average pressure of a contacting finger.
  • a pressure sensor array touchpad can be adapted to provide up to six widely variable interactive adjustable parameters from the contact of a single finger.
  • a pressure sensor array touchpad can be further adapted to measure parameters relating to a plurality of contacting fingers.
  • FIGS. 1 a - 1 i illustrate various exemplary implementations involving merging, selecting, multiplexing, and preprocessing distributed in various ways between the body of the user interface device and an associated piece of equipment.
  • FIGS. 1 a - 1 d concern the aggregated pair of user interface sensors in isolation, while FIGS. 1 e - 1 i address arrangements where some functions of the invention are performed in the associated external equipment. It is noted that the invention further provides for any of these exemplary functionalities, as well as other functionalities, to be combined or made selectable.
  • power may be supplied to these implementations by the associated external equipment or by other devices such as batteries, storage capacitors, photoelectric devices, and the like.
  • FIG. 1 a shows an implementation 100 a featuring two user interface sensors 101 , 102 , each of which may be a particular type of user interface sensor (which again may be a trackball, touchpad, mouse, or other user interface device) that can be collocated within the common physical enclosure 100 (demarcated by the dotted-line boundary).
  • First user interface sensor 101 produces signal 108 and second user interface sensor 102 produces another signal 109 , which are directed to a merge or select function 103 .
  • the merge or select function produces outgoing signal 110 which is provided to associated external equipment.
  • signals 108 , 109 (from first and second user interface sensors 101 , 102 ) would typically lose their individual identities within the outgoing signal 110 and as such may be used or processed interchangeably (without individual attribution to either the first or second user interface sensor) by the associated external equipment.
  • Merge or select function 103 may take several forms in various implementations. For example, in one embodiment it may simply be fixed to only perform a merge operation. In another embodiment it may only provide a selection function; here the selection function may be controlled by the user using a switch or some sort of action, or the selection may be remotely controlled by external equipment. As an alternative, merge or select function 103 may instead provide a user adjustable “merge” or “select” function.
  • FIG. 1 b shows an implementation that is similar to that of FIG. 1 a .
  • the primary difference is that the FIG. 1 b implementation 100 b replaces merge or select function 103 of FIG. 1 a with multiplex function 104 to produce outgoing signal 110 .
  • signals 108 , 109 (from first and second user interface sensors 101 , 102 ) retain their individual identities within outgoing signal 110 and as such may be used or processed separately by the associated external equipment.
  • FIG. 1 c shows implementation 100 c which is similar in many respects to that of FIG. 1 a .
  • the FIG. 1 c embodiment utilizes preprocessor 105 applied to signal 109 to produce processed signal 109 a.
  • the processed signal 109 a, along with signal 108 , is directed to merge or select function 103 , resulting in outgoing signal 110 .
  • Preprocessor 105 may thus introduce a pre-processing step (such as resolution modification, warping modification, offset modification, etc.) on signal 109 to produce a signal of distinguished value from that of signal 108 .
  • a pre-processing step such as resolution modification, warping modification, offset modification, etc.
  • FIG. 1 d illustrates another exemplary implementation 100 d which is similar to that of FIG. 1 c but with an additional preprocessor 106 applied to signal 108 .
  • Preprocessor 106 produces processed signal 108 a, which along with signal 109 a, is directed to merge or select function 103 to produce outgoing signal 110 .
  • Preprocessor 106 may therefore introduce a pre-processing step (such as resolution modification, warping modification, offset modification, etc.) on signal 108 to produce a signal of either equivalent or distinguished value fiom that of signal 109 a.
  • a pre-processing step such as resolution modification, warping modification, offset modification, etc.
  • FIG. 1 e shows implementation 100 b of FIG. 1 b (which features two user interface sensors 101 , 102 and a multiplex function 104 producing an outgoing signal 110 ) used in conjunction with subsequent functions provided by associated external equipment 150 a. These subsequent functions are shown within the functional boundary 150 of the associated external equipment 150 a.
  • Outgoing signal 110 from the common physical enclosure 100 is presented to demultiplexer 117 within external equipment 150 a.
  • Demultiplexer 117 produces signal 118 corresponding to or associated with pre-multiplexed signal 108 , and an additional signal 119 corresponding to or associated with the pre-multiplexed signal 109 .
  • signals 118 , 119 are presented to merge or select function 113 producing merged or selected signal 120 .
  • This implementation is functionally similar or equivalent to that of FIG. 1 a except that the various types of merge or selection functions 103 are provided within the associated external equipment 150 a (for example in software, perhaps within an application where it is customized for the needs of that application) rather than being provided within physical unit 100 b.
  • FIG. 1 f shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 b, and similar to that of FIG. 1 e , except signal 119 produced by demultiplexer 117 is directed to preprocessor 115 to produce processed signal 119 a before being sent to merge or selection function 113 .
  • This implementation is thus functionally similar or equivalent to that of FIG. 1 c except that the various types of merge or selection 103 and preprocessor 105 functions in FIG. 1 c are provided within the associated external equipment 150 b (for example in software, perhaps within an application where it is customized for the needs of that application) rather than being provided within the physical unit 100 b.
  • FIG. 1 g shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 c.
  • This arrangement expands on that shown in FIG. 1 f in that signal 118 produced by demultiplexer 117 is directed to preprocessor 116 to produce processed signal 118 a. The processed signal is then sent to merge or selection function 113 .
  • This implementation is thus functionally similar or equivalent to that of FIG. 1 d except that the various types of merge or selection 103 and preprocessor 105 , 106 functions in FIG. 1 d are provided within the associated external equipment 150 c (for example in software, perhaps within an application where it is—customized for the needs of that application) rather than being provided within the physical unit 100 b.
  • FIG. 1 h shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 d.
  • signal 119 produced by demultiplexer 117 is directed to preprocessor 115 to produce processed signal 119 a.
  • processed signal 119 a is not directed to merge or selection 113 function and retains its identity for use by a different destination from that of signal 118 within associated external equipment 150 d.
  • FIG. 1 i shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 e.
  • signals 118 , 119 produced by demultiplexer 117 are directed to preprocessors 115 , 116 to produce processed signals 118 a, 119 a.
  • processed signals 118 a, 119 a are not directed to merge or selection 113 function and thus retain their identity for use by differing destinations within associated external equipment 150 e.
  • a number of exemplary implementations of the invention utilizing various types of additional user interface sensors added to an original user interface sensor or device.
  • the first three sections address cases where the original user interface sensor is a movable mouse and the additional user interface sensor is a trackball, touchpad, or other exemplary technology, including additional scroll-wheels.
  • exemplary adaptations of trackballs and touchpads are extended to provide simultaneous adjustment of as many as six interactive widely-varying parameters and other forms of control.
  • This section continues by presenting exemplary implementations where the original user interface sensor is not a mouse, where there is a plurality of additional user interface sensors, where there is a visual display or auditory output, where the additional user interface sensor is a removable module, and where the implementation itself is a removable module.
  • FIGS. 2 a - 2 c illustrate a number of implementations provided for by the invention where a trackball controller is used as an additional user interface sensor apparatus incorporated into a traditional hand-movable computer mouse.
  • the trackball may be freely operated without disturbing previous or currently-varying parameter adjustments made by the mouse.
  • a trackball controller is added to the top surface of a conventional computer mouse as depicted in FIG. 2 a .
  • the conventional mouse buttons may be located in various places in view of the presence of the trackball and in synergy with it.
  • buttons 201 and 202 are located on the surface of mouse 200 ; button 201 being on the left of the trackball and button 202 being on the right of the trackball.
  • buttons 231 and 232 are now separated and located on the sides of mouse 230 as is the case with many trackball interfaces; button 231 being on the left side of mouse 230 and button 232 being on the right side of the mouse.
  • buttons 261 and 262 are shown located on the surface of the mouse 260 ; button 261 wraps around the left of the trackball and button 262 wraps around the right of the trackball.
  • the elongated buttons 261 and 262 may be positioned so that a user can readily and rapidly move fingers from trackball 265 to buttons 261 and 262 , or even operate one of these buttons with one finger while another finger contacts trackball 265 .
  • the trackball has an effectively unconfined range of contiguous data entry.
  • FIGS. 3 a - 3 c illustrate a number of exemplary implementations provided for by the invention where a touchpad controller is-used as an additional user interface sensor incorporated into a traditional hand-movable computer mouse.
  • the touchpad may be freely operated without disturbing previous or currently-varying parameter adjustments made by the mouse.
  • trackball 205 in FIG. 2 a can be replaced with touchpad 305 as shown in FIG. 3 a .
  • this touchpad implementation can also support the alternative button configurations of FIGS. 2 b and 2 c .
  • FIG. 3 b shows buttons 331 , 332 positioned on either side of the mouse body 330
  • FIG. 3 c shows elongated buttons 361 , 362 on the surface of the mouse wrapping around either side of touchpad 365 .
  • the touchpad typically has a confined maximum range of data entry by contiguous operation of a finger, stylus, etc.
  • the invention provides for still other types of additional user interface sensors.
  • any of these user interface sensors may be freely operated without disturbing previous or currently varying parameter adjustments made by the mouse or other associated device.
  • an X-Y joystick may be used in place of the trackball or touchpad described above.
  • the joystick may have a spring-return or may retain the last position it was placed. Similar to the touchpad and unlike the trackball, the X-Y joystick typically has a confined maximum range of travel.
  • two or more scrolling finger wheels may be used in place of the trackball or touchpad described above.
  • the scrolling finger wheels may be implemented with an unconfined maximum range of travel similar to the trackball, or with a confined range of travel like the touchpad and X-Y joystick.
  • embodiments 1000 are depicted comprising the usual components of a scroll-wheel mouse (mouse body 1001 , buttons 1011 and 1012 , and the usual scroll wheel 1021 ) complemented with an additional scroll wheel 1022 with adjustment direction perpendicular to that of finger scroll-wheel 1021 .
  • two or more rotating control knobs may be used in place of the trackball or touchpad described above.
  • the rotating control knobs may be implemented with an unconfined maximum range of travel like the trackball or with a confined range of travel like the touchpad and X-Y joystick.
  • the invention also provides for more exotic types of user interface sensor technologies—for example, proximity detectors of various types (RF, infrared, etc.), video cameras (using any of the techniques provided in U.S. Pat. No. 6,570,078), and other emerging and unforeseen technologies—to be used as the additional user interface sensor aggregated in the same physical enclosure as the first user interface sensor.
  • proximity detectors of various types RF, infrared, etc.
  • video cameras using any of the techniques provided in U.S. Pat. No. 6,570,078
  • other emerging and unforeseen technologies to be used as the additional user interface sensor aggregated in the same physical enclosure as the first user interface sensor.
  • additional user interfaces may be used to capture larger numbers (i.e., more than two) of widely-varying adjustable parameters from the additional user interface sensor.
  • FIGS. 4 a - 4 d address the case of the trackball.
  • FIG. 4 a illustrates a freely-rotating trackball 205 and the two principle orthogonal adjustment directions 401 , 402 that are responsively resolved and measured in traditional trackball user interface devices. However, at least two other physical degrees of freedom may be readily exploited, and at least six total parameters can be interactively adjusted and measured.
  • FIG. 4 b shows the application of downward pressure 403 on trackball 205 . Such pressure 403 may be applied without disturbing current values established in orthogonal adjustment directions 401 , 402 .
  • the trackball may be implemented so that downward pressure 403 may be applied while simultaneously adjusting trackball 205 in orthogonal adjustment directions 401 , 402 , particularly if the signal produced by the measurement of downward pressure 403 incorporates a modest “grace” zone of non-responsiveness for light pressure values.
  • Downward pressure impulses may alternatively be sensed and treated as discrete event “taps,” as commonly used in contemporary touchpad interfaces found in laptop computers, for example.
  • FIG. 4 c shows the application of “yaw” rotation 404 (i.e., rotation around the vertical axis) of trackball 205 .
  • This yaw rotation 404 may be applied without disturbing current values established in orthogonal adjustment directions 401 , 402 and can be readily measured and adjusted as a widely-varying parameter. Further, by grasping trackball 205 (or other operational methods), the yaw rotation 404 and traditional orthogonal adjustment directions 401 , 402 may be independently and simultaneously adjusted. It is also noted that in principle up to six widely-varying physical degrees of freedom can be simultaneously measured from a properly configured trackball 205 by placing the trackball 205 in a cradle that senses not only downward pressure 403 or displacement but also lateral pressure or displacement. As shown in FIG.
  • both forward-backward, non-rotational force 405 and left-right, non-rotational force 406 may be applied to the trackball in a manner that the values of force or displacement in each of these directions 405 , 406 can be independently measured.
  • three rotational directions of orientation 401 , 402 , 404 and three non-rotational directions of force or displacement 403 , 405 , 406 may be independently and simultaneously adjusted by a user and measured as six independent interactively adjustable user interface parameters. These correspond, effectively, to measurable adaptations of the six degrees of freedom of an orientable object in 3-dimensional space as found in classical mechanics and aeronautics—that is:
  • trackball sensing technologies use optically based techniques for sensing the two traditional components of rotation (“roll” and “pitch”) of the trackball.
  • the trackball itself may be configured with an optical pattern on it with spatially varying reflectivity for a range of the light spectrum.
  • the pattern may be such that it can spatially vary light reflectively in these two traditional components of trackball rotation.
  • two spatially varying reflectivity patterns each active at different ranges of the light spectrum or light polarization, may be superimposed or integrated with the trackball.
  • a second or third spatially varying reflectivity pattern active at, respectively, a second or third portion of the light spectrum (or light polarization if available) may be superimposed or integrated with the patterns employed for traditional “roll” and “pitch” rotation sensing, and an additional optical source and sensor is used to obtain measurement of the added varying reflectivity pattern.
  • sensor signals may be directly usable or may require processing of the three primitive signals measured by the sensors to obtain a clean decomposition of the measurement signals into independent “roll,” “pitch,” and “yaw” signals independently responsive to the “roll,” “pitch,” and “yaw” components of trackball rotation.
  • the trackball may include internally, or on its surface, or both, materials with spatially varying patterns of magnetic properties, capacitive properties, electromagnetic properties, ultrasonic acoustic properties, resonance phenomena of any of these properties, polarization phenomena of any of these properties, etc., individually or in combination, each of which may be active at specific ranges of polarization, frequencies, etc. These may be used together with or in place of optical measurement approaches.
  • sensor signals may be directly usable or may require processing of the three primitive signals measured by the sensors to obtain a clean decomposition of the measurement signals into independent “roll,” “pitch,” and “yaw” signals independently responsive to the “roll,” “pitch,” and “yaw” components of trackball rotation as is clear to one skilled in the art. It is also noted that the third component of rotation of the freely-rotating trackball may be interpreted or even measured as a discrete “click” event.
  • the trackball may be secured in a saddle allowing free rotation of the trackball but causing any displacement actions on the trackball to invoke displacements of the saddle.
  • the saddle displacement may be measured with a displacement sensor which itself may comprise one or more pressure, optical, resistive, capacitive, magnetic, electromagnetic, continuous-range sensors, switches, etc. It is also noted that one or more components of displacement of the freely-rotating trackball may be interpreted or even measured as a discrete “click” event.
  • FIGS. 5 a - 5 d turn now to the case of the touchpad.
  • FIG. 5 a illustrates touchpad 305 and the two principle orthogonal data entry directions 501 , 502 that are responsively resolved and measured in traditional touchpad user interface devices.
  • the touchpad shown in FIGS. 5 a - 5 d which provides at least four other physical degrees of freedom, may be implemented using the techniques presented in U.S. Pat. No. 6,570,078, for example.
  • FIG. 5 b illustrates the use of downward pressure 503 in the context of a touchpad.
  • downward pressure 503 is sensed and utilized as discrete event “taps.”
  • downward pressure 503 may also be measured and adjusted as an independent and simultaneously interactive widely-varying parameter.
  • the rotational angle 504 of a finger contacting a touchpad with rough-elliptical contact boundary can also be measured as a widely-varying parameter.
  • both forward-backward 505 and left-right 506 components of the tilt of a contacting finger can additionally be measured as independent and simultaneously interactive widely-varying parameters.
  • the sensing of multiple fingers, the application of contact syntaxes and grammars, and other user interface control expansions of an adequately configured touchpad may also be achieved using, for example, the techniques presented in U.S. Pat. No. 6,570,078.
  • the invention also provides for larger numbers (i.e., more than two) of widely varying adjustable parameters from other types of user interface sensor technologies.
  • the joystick may be configured to rotate on its axis, pulled in and out, fitted with a knob or trackball, etc., in a measurable fashion to provide additional and simultaneous interactively adjustable parameters.
  • the joystick may be configured to rotate on its axis, pulled in and out, fitted with a knob or trackball, etc., in a measurable fashion to provide additional and simultaneous interactively adjustable parameters.
  • three or more of these devices may be provided.
  • known techniques for the extraction of additional parameters may be used. Examples of the various types of video extraction techniques that may be used are presented in U.S. Pat. No. 6,570,078.
  • the invention involves the incorporation of two conventional user interface sensors (such as a mouse, trackball, touchpad, joystick, etc.) into an embodiment where the user may freely use both of the user interface sensors individually or simultaneously with the same hand.
  • the invention provides for implementations that do not include a mouse as one of the user interface sensors.
  • two or more individual user interface sensors can be combined without need of a traditional hand-movable computer mouse.
  • Such an implementation may be accomplished by implementing one of the possible user interface sensors in place of the traditional hand-movable computer mouse where taught in other descriptions of the invention. This configuration may be useful when built into a laptop computer, control console, musical instrument, and test instrument, among others.
  • a trackball and touchpad may be arranged in the same physical enclosure so that the front, middle, or back of the palm may freely operate a conventional trackball while one or more selected extended or arching fingers may simultaneously or alternatively operate a touchpad.
  • the touchpad may be a conventional touchpad providing two widely-varying simultaneously interactive parameters from a single finger contact, or the touchpad may be a more enhanced version providing as many as six widely-varying and simultaneously interactive parameters from a single finger contact.
  • the touchpad may also be configured to accept multiple points of contact, recognize gestures, support syntax and grammatical constructions using, for example, the teachings provided by U.S. Pat. No. 6,570,078.
  • two trackballs may be arranged in the same physical enclosure.
  • the two trackballs may be positioned so that they lie parallel to the length of the hand, enabling the front, middle, or back of the palm to freely operate a first trackball while one or more extended or arching fingers may simultaneously or alternatively operate the second trackball.
  • the two trackballs may be positioned so that they lie parallel to the width of the hand, so that the fingers and/or thumb on the left side of the hand may operate a leftmost trackball while the remaining fingers and/or thumb on the right side of the hand may individually or simultaneously operate a rightmost trackball.
  • either or both of the trackballs may be a conventional trackball providing two widely-varying and simultaneously interactive parameters, or it may be a more enhanced trackball providing as many as six widely-varying and simultaneously interactive parameters as described earlier.
  • a mouse user interface sensor is used.
  • a mouse user interface sensor is supplemented with at least two additional user interface sensors ergonomically arranged so that the two additional user interface sensors may be simultaneously or alternatively operated by the same hand. If these embodiments are further configured so the mouse body is readily moved with adequate precision via the back of the operating hand, then all three user interface sensors may be simultaneously or alternatively operated by the same hand in an ergonomically advantageous manner.
  • FIGS. 14 a - 14 d illustrate some exemplary embodiments of the just-described features.
  • FIG. 14 a illustrates a mouse where traditional mouse buttons have been replaced by trackballs 1405 a, 1405 b. These trackballs 1405 a, 1405 b may accept a downward pressure impulse and as such act as the traditional mouse buttons.
  • trackballs 1405 a, 1405 b are also adjustable and each readily provides two or more additional widely-variable and simultaneously adjustable parameters in addition to the two parameters adjusted by moving the mouse body 1400 .
  • FIG. 14 b shows a similar arrangement where traditional mouse buttons have been replaced by touchpads 1435 a, 1435 b.
  • these touchpads may accept a downward pressure impulse and as such act as traditional mouse buttons, but are also adjustable as touchpads and as such each readily provides two or more additional widely-variable and simultaneously adjustable parameters.
  • a single hand may be positioned to comfortably operate simultaneously or alternatively both trackballs or both touchpads. If these embodiments are further configured so the mouse body is readily moved with adequate precision via the back of the operating hand, then either of these embodiments readily provides six to twelve widely-variable and simultaneously adjustable parameters.
  • FIGS. 14 a and 14 b may be blended as depicted in FIG. 14 c , or in its mirror image.
  • FIG. 14 d illustrates a more extreme realization comprising a left-fingers/thumb trackball 1465 a, a right-fingers/thumb trackball 1465 b, a palm trackball 1465 c, and a traditional clickable scroll-wheel 1468 .
  • Yet another alternative is to replace one or more of the trackballs of the FIG. 14 d embodiment with a touchpad user interface sensor.
  • any of the mouse and non-mouse embodiments may further include a visual display.
  • the visual display may provide details of adjustable parameter values, operation modalities, etc.
  • the visual display may be physically associated with a physical enclosure (such as that of a traditional computer mouse), or may be displayed on the computer screen or display of other associated equipment.
  • any of the mouse and non-mouse embodiments may further provide auditory output.
  • the auditory output may provide details of adjustable parameter values, operation modalities, error conditions in usage of the invention, a condition relating to elapsed time or other metric of consistent use of a single usage modality, etc.
  • the auditory output associated with the invention may be physically associated with a physical enclosure (such as that of a traditional computer mouse), or may be produced by speakers or headsets affiliated with the computer or other associated equipment.
  • the invention also provides for the user interface sensor to be implemented using a replaceable module accepted by an adaptation of a traditional computer mouse.
  • a user may initially obtain the invention in one configuration and field modify it to another configuration.
  • the invention also provides for a traditional computer mouse to be implemented as a removable module in a laptop computer or other affiliated equipment, and may include a wireless link with such devices.
  • this removable module may further include one or more user interface sensors, with these sensors operable as a traditional trackball or touchpad when the invention is stowed in the laptop computer or other affiliated equipment in such a way that the invention's traditional hand-movable computer mouse modality is unmovable and hence unusable.
  • user interface parameters can be interchangeably adjusted with either the movement of the mouse body or the use of an integrated trackball or touchpad with identical task results.
  • a user with a tiring hand can change at will the user interface sensor employed according to how the hand feels or the nature of a specific task.
  • the user can also switch back and forth between moving the mouse body and using the trackball/touchpad either by free choice or by following auditory or visual prompting from a time or usage monitor.
  • Contemporary mice often feature a small rotating wheel between the buttons for use in operating the vertical scroll bar of a window without changing the position of the mouse.
  • the left-right sensing capability of the trackball or touchpad may be used to add a similar capability for horizontal scroll bars of a window.
  • a user can move the vertical bar 611 of FIG. 6 up by rotating trackball 205 away from him/herself, or one can move the vertical bar 621 down by rotating trackball 205 towards him/herself.
  • By rotating trackball 205 to the left the user can move the horizontal bar 621 left.
  • the user can move the horizontal bar 621 right by rotating the trackball 205 to the right.
  • a user can move scroll bar 611 up by sliding the finger away from her/himself or move scroll bar 611 down by sliding the contacting finger towards her/himself; similarly, the user can move the scroll bar 621 left by sliding the finger to the left or move the scroll bar 621 right by sliding the contacting finger to the right.
  • the vertical and horizontal scroll bars may be adjusted with a conventional scroll-wheel mouse that has been fitted with an additional scroll-wheel.
  • FIG. 10 a illustrates an arrangement where the additional scroll-wheel is located closer to the user while FIG. 10 b illustrates an arrangement where the additional scroll-wheel is located farther away from the user.
  • the two scroll-wheels are shown co-centered with respect to the mouse body; for simultaneous adjustment it may be advantageous to locate the additional scroll-wheel 1022 to one side or the other of the centered positions shown in FIGS. 10 a and 10 b .
  • One approach useful for supporting both left-handed and right-handed users, which may provide additional utility, would be to provide two off-centered additional scroll-wheels, one on either side of the center line of the mouse body 1001 and conventional scroll-wheel 1021 .
  • mouse operations are necessary for each of the many types of object attribute modification, etc.
  • these mouse operations are required because the mouse only allows for the interactive adjustment of two widely-varying parameters at a time, and the user must change context several times as the parameters adjusted by the mouse are chosen, adjusted, and then replaced with another pair of parameters.
  • the present invention is useful in many of these circumstances because it allows for more than two parameters to be adjusted at the same time.
  • FIG. 7 shows an example of a session involving the authoring of a viewgraph.
  • the viewgraph authoring task showcased in this example includes the creation of a flowchart diagram (here depicting a business workflow process) and as such also illustrates related needs and attributes of a 2D CAD program involving layout of a diagram (such as a circuit, algorithm, etc.) or physical object (such as a PC board, control panel, semiconductor photolithography mask, etc.).
  • a 2D CAD program involving layout of a diagram (such as a circuit, algorithm, etc.) or physical object (such as a PC board, control panel, semiconductor photolithography mask, etc.).
  • this example further involves drawing, the example also illustrates the related needs and attributes of a paint-box or electronic drafting application.
  • application window 700 comprising menu area 700 a and drawing area 700 b.
  • viewgraph title 701 and portion 702 of the flowchart to be drawn comprising thus far a sequence of flowgraph objects connected by arrows, have already been entered and rendered.
  • New flowgraph objects may be introduced in standard fashion by selecting the type of new object desired from a palette, initially putting an instance of the selected object type in a convenient place in the drawing area, adjusting the size, orientation, color, and/or other attributes, and putting into final position. Often a number of the last few steps are interactively cycled through multiple times before the newly introduced object is adequately drawn and the user directs their attention to the next task.
  • the palette of available objects of a specific high-level task is shown as an overlapping stack of three sub-class palettes 703 a, 703 b, 703 c, each providing a selection of available objects within that sub-class.
  • sub-class palette 703 a has been selected (as indicated by the heavy line) from other available objects 705 , within the sub-class palette 703 a.
  • an initial highly adjustable rendering of a specific instance 714 of the selected object 704 appears in a convenient location, which may be selected by the user.
  • the specific instance 714 rendered in this highly adjustable initial state, is typically surrounded by graphical handles 716 which facilitate sizing, positioning within the drawing area, and often at least rotational orientation (for example, using the mouse with the ALT key simultaneously held down to interactively adjust the angle of rotation of the object should that be needed).
  • the cursor controlled by the mouse 717 can be moved within object 714 to relocate it within drawing area 700 b or can, as shown in FIG. 7 , be positioned atop one of the graphical handles 716 to permit the mouse to adjust the horizontal and vertical scale of object 714 , i.e., adjust its size and aspect-ratio.
  • the latter adjustment is permitted to collapse the object through to ‘zero’ thickness in one of the adjustment dimensions and continue through to re-render the object in mirror image, thus additionally providing a form of vertical and horizontal flip by using the size and aspect-ratio resizing.
  • FIG. 8 is a flowchart showing tasks involved in selecting and adjusting one of a plurality of available pairs of adjustable parameters by using a user interface device permitting the adjustment of only one pair of parameters at a time.
  • the task goal is simply to adjust a pair of selected parameters 801 from a larger group of adjustable parameters.
  • the specific pair of parameters must first be selected. In most known graphical user interface systems and methods, this typically involves first using the user interface device to control the movement of a cursor to a selection area of the graphical interface in a first overhead step 811 and then selecting the adjustment context (parameter pair) in a second overhead step 821 .
  • the selected pair of parameters may be immediately adjusted in goal operation 801 , but typically the cursor must then at least be moved, in a third overhead operation 812 , to a location outside of the selection area affiliated with operations 811 and 821 to another location (such as a drawing or typing area) affiliated with the context (parameter pair) that has just been selected for adjustment.
  • the selected pair of parameters may be immediately adjusted in goal operation 801 , but typically the context must be activated (for example, by clicking in an open portion of a drawing area or selecting an existing object) in a fourth overhead operation 822 .
  • the cycle may then immediately be repeated in some variant form for another pair of parameters, but typically the parameters must be deselected (for example, by another click to set the final value) in a fifth overhead operation 823 before the cursor may be moved to the selection area in another instance of operation 811 .
  • the parameters in order to adjust one pair of parameters from a larger group of parameters, as many as five overhead operations (as many as two cursor movements 811 , 812 and as many as three select/deselect clicks 821 , 822 , 823 ) are commonly required.
  • FIGS. 9 a - 9 b show broader implications of the overhead called out in FIG. 8 .
  • FIG. 9 a depicts the sequential adjustment of pairs of parameters chosen from a larger group of pairs of parameters in a scenario suggestive of no interactive iteration.
  • One pair of parameters is adjusted with up to five overhead operations in action 901
  • a second pair of parameters is adjusted with up to five overhead operations in action 902
  • a third pair of parameters is adjusted with up to five overhead operations in action 903 , and so on.
  • the overhead slows things down but may not be a significant encumbrance to the broader goal of actions 901 , 902 , 903 , etc.
  • FIG. 9 b depicts an interactive adjustment of pairs of parameters from a larger group of parameters in a scenario suggestive of one where interactive iteration is required, as the setting of one pair of parameters is difficult to complete without setting other parameters.
  • the overhead is likely a significant encumbrance to the higher goal involving the pair-wise adjustment actions 901 , 902 , 903 , etc.
  • pair-wise adjustment actions 901 , 902 , 903 For example, consider the interactive adjustment of six parameters, one pair at a time, through pair-wise adjustment actions 901 , 902 , 903 ; not only are up to five operations of overhead involved for each of the pair-wise adjustment actions 901 , 902 , 903 , but a considerable extra number of passes must be made through these pair-wise adjustment actions 901 , 902 , 903 due to the fact that the adjustment of some parameters depends on or interacts heavily with the values of other parameters. The situation gets even more cumbersome should additional pair-wise adjustment operations be required.
  • 9 b further shows the potential for one or more additional adjustment actions 950 which in principle may be iterated as well as and combined with pair-wise adjustment actions 901 , 902 , 903 (as suggested by fully-connected iteration paths 921 , 922 , 923 ).
  • the present invention readily offers, for example, four, six, eight or even higher numbers of simultaneously adjustable parameters controllable by the same hand, which, when selected in a context, eliminate the many overhead operations depicted in FIGS. 8 and 9 a - 9 b , and the many additional iteration steps depicted in FIG. 9 b.
  • operations B, D, and E are almost always utilized, operations A, G, and I are frequently utilized, and operations C, F, H, and J are rarely utilized.
  • the user may freely assign user interface sensor parameters to operations A through J (and others as may be useful) for each of a number of steps as may match the task or tasks at hand.
  • These assignments may be stored for later retrieval and use, and may be named by the user.
  • the stored assignments may be saved along with specific files, specific applications, or as a general template the user may apply to a number of applications. It is noted that such variable assignments may be particularly useful to users as their hands fatigue, to prevent fatigue or injury, or as an adjustment for a temporary or permanent disability.
  • one user interface sensor for example, the mouse body
  • the additional user interface sensor for example, a trackball or touchpad
  • the user interface may be directed towards non-positional adjustments, such as the adjustment of a rotation angle or of the color of a graphic object; here multiple resolutions may be valuable to make careful adjustments and coarse adjustments as needed.
  • scroll bars for long documents may also benefit from rapid access to multiple resolution scales, for example one user interface sensor may be used to navigate within a page (using a fine-grained navigation scale) while a second user interface sensor may be used to navigate across pages (using a coarser-grained navigation scale).
  • some types of user interface sensors such as the touchpad and X-Y joystick for example, naturally have a limited maximum range of operation while others such as a mouse, trackball, and scroll-wheel have an essentially unlimited maximum range of operation.
  • most user interface sensors are interpreted in relative terms (that is, the stimulus from the sensor is interpreted as a command to move a cursor, scroll bar, etc., incrementally in some direction relative to a current position)
  • stimulus signals from any of these types of user interface signals may be interpreted in either a relative or absolute manner with varying degrees of naturalness or problematic qualities.
  • the present invention provides for one user interface sensor to be used for absolute positioning of a cursor, scroll bar, etc., or other means of parameter adjustment while another user interface sensor is used for traditional relative adjustment of such parameters.
  • a scroll bar may be adjusted in the usual fashion by a mouse body or trackball and in an absolute manner by a touchpad wherein the extreme values of the adjusted parameter correspond to the extreme positions at the edges of the touchpad.
  • these two user interface sensors may control the same parameters—here it is often the result that the two sensors adjust the same parameters with different resolutions. Further, in this situation it is fairly likely that at least one of the resolution scale factors will be adjusted automatically.
  • the resolution of the absolute positioning sensor will correspondingly vary (so that the extremities in range of, for example, a touchpad correspond to the top of the first page and end of the last page) while the relative positioning sensor may retain the same incrementing/decrementing vertical scrolling resolution scale regardless of the number of pages.
  • the present invention provides at least four simultaneous interactively adjustable parameters overall, it is thus potentially useful for fully interactive color adjustment within a complete color space model.
  • the additional user interface sensor may be such that it alone provides three simultaneously interactively adjustable parameters
  • the first user interface sensor for example, the mouse body
  • the additional user interface sensor may be used as a pointer to select objects and the additional user interface sensor may be used to adjust attributes of the selected object such as its color, border color, etc.
  • lower-level graphical or other objects may be grouped to form an aggregated object.
  • This aggregated or “grouped” object (collectively referred to herein as an “aggregated object”) can then be moved, rotated, flipped, resized, etc. as if it were a lower-level graphic or other object.
  • Grouping can also typically be done hierarchically and in mixed hierarchies, i.e., a plurality of lower-level graphical or other objects may first be grouped, and the resulting aggregated object may then itself be grouped with other aggregated objects and/or lower-level graphical or other objects.
  • one or more of the lower-level graphical or other objects comprising the aggregated object may need modification.
  • most applications permit modifications to be made to individual text objects within an aggregated object.
  • the aggregated object must be first disaggregated or “ungrouped” to completely free the involved lower-level graphical or other object from any grouping it had been involved in. After the modification, the grouping must be reconstructed. Often this becomes a cumbersome situation, particularly where the adjustments within the group are themselves an interactive response to other adjustments made within a drawing.
  • one user interface sensor may be used to navigate the levels of grouping and another user interface sensor may be used to perform operations on objects (lower-level or “grouped”) within that level of grouping of the overall aggregated object.
  • FIG. 11 a shows a portion 1100 of a larger drawing, the portion 1100 featuring box 1101 , two arrowed lines 1111 , 1112 , and grouped object 1102 (here itself comprising two triangles connected by a line).
  • grouped object 1102 is itself grouped with box 1101 to form a second grouped object, and this second grouped object is itself grouped with the two arrowed lines 1111 , 1112 to form a third grouped object.
  • the user's task is to modify FIG. 11 a so that it becomes FIG. 11 b. To do this, effectively the user must, in some order of operation:
  • one user interface sensor is used to select the second group level, and the second user interface sensor is used to perform insert, copy, paste, and position operations within this level of grouping without any form or type of ungrouping operation. If the vertically distributed ellipsis 1103 itself is realized as a fourth grouped object created from three aligned text elements, when it is pasted into the drawing via this modality its 1103 grouping is subordinated appropriately (i.e., structured as a peer to grouped objects 1102 , 1102 a ) within the second grouping level.
  • the aspects of the invention illustrated here can also be implemented with a conventional user interface sensor such as a traditional mouse, trackball, or touchpad.
  • the conventional user interface sensor such as a traditional mouse, trackball, or touchpad must first be used to select the level of grouping and then be used to make the desired modifications within that level of grouping; to make modifications at a different level of grouping, the new level of grouping must be selected in a separate operation, thus adding overhead, as depicted in FIGS. 8 and 9 a - 9 b .
  • CAD and drawing packages that enable 3D object placement and orientation within a 3D space typically extend the capabilities of traditional 2D layout, CAD, and graphics applications as described in Section 3.3 to serve additional geometric needs.
  • the placement and orientation of 3D object 1200 within a 3D space 1250 (oriented with respect to a reference point 1251 ) requires that one additional position dimension and two additional orientation angles be specified to complete the full collection of three position dimensions 1201 , 1202 , 1203 and three orientation angles 1211 , 1212 , 1213 .
  • the invention provides for a wide range of mappings between the six position and orientation parameters 1201 , 1202 , 1203 , 1211 , 1212 , 1213 involved in the placement and orientation of 3D object 1200 within a 3D space, and the large numbers of widely-adjustable and simultaneously interactive parameters facilitated by various realizations of the invention.
  • a mouse fitted with two trackballs as in FIG. 14 a may be used to specify these six parameters in various ways.
  • One technique is to use the position of mouse body 1400 to control two of the position coordinates (for example 1202 , 1203 ), one of the trackballs (for example 1405 a ) to control the orientation angles ( 1212 , 1213 ) corresponding to these two axes, and the remaining trackball ( 1405 b ) to control the remaining axis ( 1201 ) and its corresponding orientation angle ( 1211 ).
  • trackballs 1405 a, 1405 b are configured or used in 2-parameter modalities.
  • the mouse of FIG. 14 a is fitted with two trackballs
  • the trackballs may be configured in 3-parameter modalities with one of the trackballs used for controlling three position dimensions 1201 , 1202 , 1203 and the second trackball configured to correspondingly control the three orientation angles 1211 , 1212 , 1213 .
  • the position of mouse body 1400 may be used to control other aspects of drawing operations.
  • a touchpad configured for 4-parameter operation involving two parameters of finger position and two parameters of finger tilt may be combined with a trackball configured for 2-parameter operation.
  • finger position may be used to control two of the position coordinates (for example 1202 , 1203 )
  • finger tilt may be used to control the orientation angles ( 1212 , 1213 ) corresponding to these two axes
  • the trackball to control the remaining axis ( 1201 ) and its corresponding orientation angle ( 1211 ). If the configuration includes a mouse body, its position may be used to control other aspects of drawing operations.
  • a configuration like that of FIG. 14 d may use left-fingers/thumb trackball 1465 a to control a first position coordinate 1201 and its corresponding orientation angle 1211 , the right-fingers/thumb trackball 1465 b to control a second position coordinate 1202 and its corresponding orientation angle 1212 , and palm trackball 1465 c to control third position coordinate 1203 and its corresponding orientation angle 1213 .
  • the invention further provides for a wide range of additional mappings and geometric metaphors between the user interface sensor geometry and the three position dimensions 1201 , 1202 , 1203 and three orientation angles 1211 , 1212 , 1213 of a 3D object.
  • the invention additionally provides for a plurality of pairs of user interface sensor parameters to be used to control the respective positions of a corresponding plurality of individual cursors, selections, and/or insertion points.
  • Multiple cursors and associated operations of multiple selection and insertion points can have many applications. Below, a few of these possibilities that would be apparent to one skilled in the art are showcased in a cut-and-paste editing example.
  • FIG. 13 a illustrates a text editing example with text display window 1300 involving the selection of a clause 1320 (highlighted in this example) with the intention of relocating it to a new position 1351 .
  • the cursor is first used to select clause 1320 and then used to select the insertion position 1351 .
  • the cursor is initially in the spot where the material to be sequentially divided and relocated is originally aggregated; the user must repeatedly select the portion of the remaining aggregated material to be relocated and then lose that cursor position to go searching for the new destination insertion spot, perform the insertion, and then search again, perhaps taking considerable time, for the initial spot and re-establish the cursor location there.
  • the invention may be used in a minimal configuration capable of interactively specifying at least two pairs of widely adjustable interactive parameters.
  • one pair of parameters is used to set the location of first cursor 1301 which is used in a selection operation, while the second pair of parameters is used to set the location of second cursor 1351 which is to be used to independently set an insertion point.
  • the user may then perform the cut and paste operation with a single mouse click, resulting in the outcome depicted in FIG. 13 b .
  • the relocated text clause 1320 has been transferred to a position determined by the insertion cursor 1351 (here shown to the left of the cursor 1351 ; it could just as easily be to the right of it), and cursor 1301 used to make the selection remains in position. Either cursor 1301 or 1351 may now be moved and/or used for other cut, copy, paste, or (via the keyboard) new text insertion operations.
  • the two cursor locations were close enough to be displayed in the same window 1300 , the value of this application of the invention is significantly increased should the two positions be separated by many pages, many tens of pages, or even many hundreds of pages of text.
  • Such situations may be handled by any number of approaches as is clear to one skilled in the art.
  • the area comprising the cursor whose corresponding user interface sensor was last manipulated is displayed in the single display window.
  • a click event or other user interface stimulus may be used to toggle among the areas comprising the various cursor locations.
  • at least two windows may be rendered, with one of the cursors displayed and operable within one window and a second cursor displayed and operable in a second window.
  • the invention also provides for these general principles to be applied to other types of objects and applications, such as spreadsheet cells (involving data, formula objects, and cell formats), graphical objects, electronic CAD diagrams (where objects may be connected with formulas, dynamic models, etc.), and others as will be apparent to one skilled in the art.
  • Simulation, processing, and analysis applications typically involve a large number of parameters which are adjusted to model, affect or investigate the resulting behaviors, end results, and/or implications.
  • Conventional 2-parameter user interface devices such as a mouse/trackball/touchpad require the user to adjust these parameters pairwise (or individually with a knob box as has been done historically in some systems) involving complex repetitive passes among high-overhead operations as depicted in FIGS. 9 a - 9 b , for example among steps 901 , 902 , 903 .
  • the division among high-overhead operations for example moving among steps 901 , 902 , 903 , can be functionally disruptive as well as slow and inefficient.
  • the ability to interactively and freely adjust larger collections of parameters simultaneously is thus also of extremely high value.
  • the division among high-overhead operations for example moving among steps 901 , 902 , 903 , can be functionally disruptive as well as slow and inefficient.
  • the ability to interactively freely adjust larger collections of parameters simultaneously is thus also of extremely high value.
  • real-time machine control and plant operations also traditionally involve controlling a significant number of parameters requiring simultaneous interactive adjustment.
  • Conventional real-time machine control and plant operation consoles typically have large numbers of controls with carefully designed spatial layouts to facilitate the rapid and precise adjustment of multiple parameters via knobs, sliders, pushbuttons, toggle switches, etc.
  • the introduction of computer GUIs can add considerable value and new capabilities, including “soft” reconfigurable consoles and functional assignments, but in the bargain typically significantly encumber users—accustomed to rapid and precise operation of multiple parameters—with a 2-parameter mouse/trackball/touchpad having the overhead of iterative context-switching operations depicted in FIGS. 8 and 9 a - 9 b .
  • the division among high-overhead operations for example moving among steps 901 , 902 , 903 , can be functionally disruptive as well as slow and inefficient.
  • the ability to interactively freely adjust larger collections of parameters simultaneously is thus also of extremely high value.
  • a very few examples of this category of application where the invention may be useful include many forms of robotics control, computer-control manufacturing tools, industrial optical and electron microscopy, camera control (pan, tilt, zoom, focus, and/or iris), plant process elements (heaters, pumps, values, stirrers, aerators, actuators, activators, etc.), and a wide range of other related and divergent possibilities apparent to those skilled in the art.
  • the present invention at its core provides for a wide range of systems and methods for realizing and applying user interfaces providing, in many cases, at least four widely variable simultaneously interactively adjustable parameters.
  • the invention more broadly encompasses novel user interface structures, metaphors, and applications readily suggested and enabled by the core of the invention but which may be indeed realized in ways not involving the core of the invention.

Abstract

A user interface device having a housing and a joystick with at least three independent mechanical adjustments, each generating an associated signal responsive to the mechanical adjustment, and at least one sensor arrangement generating signals to control at least a cursor on an external device is described. Signals from the sensor arrangements and the joystick can be multiplexed together for transmission to the external device. In an implementation, the sensor arrangements generate signals responsive to changes in the position of the housing with respect to a surface on which the user interface device is placed. In another implementation, the user interface device includes a trackball, and at least one additional sensor arrangement generating signals responsive to the operation of the trackball.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • This application is a continuation of U.S. application Ser. No 12/618698 filed on Nov. 14, 2009, which is a continuation of U.S. application Ser. No. 10/779,368 filed on Feb. 13, 2004, now U.S. Pat. No. 7,620,915 issued on Nov. 17, 2009.
  • BACKGROUND OF THE INVENTION
  • 1. Field of the Invention
  • This invention relates to user interface devices for use with a computer, and in particular to computer mice, trackballs, touchpads, multiple-parameter pointing and data entry devices, and user interface metaphors.
  • 2. Description of the Related Art
  • User interface devices for data entry and graphical user interface pointing have been known for many years. The most common devices include the computer mouse (usually attributed to English, Engelbart, and Bennan “Display-Selection Techniques for Text Manipulation, IEEE Transactions on Human Factors in Electronics, pp. 5-15, vol. HFE-8, No. 1, March 1967), the trackball, the touchpad in both finger-operated (for example, the various finger-operated devices produced by Symantec Corp., of Springfield, Oreg.) and stylus-operated (for example, products used with desktop workstation computers by Wacom Technology Corp., of Vancouver, Wash.) versions, and display-overlay touchscreens. Other historical and exotic devices include various types of light pens and the Data Glove™ (produced by VPL Research, Inc., of Redwood City, Calif.).
  • Most user interface devices for data entry and graphical user interface pointing commonly used with computers or with equipment providing computer-like user interfaces have two wide-range parameter adjustment capabilities that are usually assigned to the task of positioning a screen cursor within a two-dimensional display. In many cases, one, two, or three binary-valued “discrete-event” controls are provided, typically in the form of spring-loaded push-buttons.
  • More recently, computer mice have emerged that provide an additional “scroll” finger-wheel adjustment (for example, between two control buttons) to provide a third wide-range parameter adjustment capability (for example, various products developed by Logitech Inc., of Fremont, Calif.). A mouse of this configuration is often referred to as a “Scroll Mouse” since this third wide-range parameter is typically assigned the task of positioning a vertical scroll bar in an actively selected window. This additional finger-wheel adjustment may also operate as a spring-loaded push-button, thus providing an additional binary-valued “discrete-event” control. Typically this additional binary-valued “discrete-event” control is used to turn on and off an automatic scrolling feature which controls the rate and direction of automatic scrolling according to vertical displacement of the displayed cursor.
  • SUMMARY OF THE INVENTION
  • In accordance with embodiments of the invention, two independently adjustable and positionable cursors are employed in a visual interface for editing an electronic document. The document may comprise text symbols, text objects, and graphics objects, among others. If both cursors are located in close proximity within the document, then these cursors may be simultaneously displayed in a single window. Otherwise, two areas of the document, each comprising one of the cursors, may be simultaneously displayed using a separate window for each cursor, or alternately selected and displayed in a single window. Copy or cut operations may be made with one cursor, while paste operations may be repeated using the second cursor. Cursor locations may be left unchanged, or they may be moved between or within editing operations. The two cursors may be freely controlled by an enhanced pointing device, such as a mouse with an added touchpad or trackball, or sequentially selected and controlled by a conventional mouse.
  • In accordance with embodiments of the invention, a traditional hand-movable computer mouse is configured with an additional user interface sensor. For convenience, the term “user interface sensor” will be used herein to collectively refer to devices such as trackballs, touchpads, mouse devices, scroll-wheels, joysticks, and other such devices.
  • In one aspect of the invention, the addition of a user interface sensor provides alternative physical modalities for the same pair of adjustable parameters so that a user may switch between using the user interface device as a traditional hand-movable computer mouse and using the user interface device as a trackball or touchpad.
  • In another aspect of the invention, the addition of a user interface sensor provides alternative resolution modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one level of parameter adjustment resolution, and using the invention as a trackball or touchpad, for example, to obtain a different level of parameter adjustment resolution.
  • In another aspect of the invention, the addition of a user interface sensor provides alternative types of warping modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one type of parameter adjustment (for example, linear) and using the invention as a trackball or touchpad, for example, to obtain a different type of parameter adjustment (for example, logarithmic, gamma-corrected, arccosine, exponential, etc.).
  • In another aspect of the invention, the addition of a user interface sensor provides alternative offset modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one type of centering of parameter adjustment and using the invention as a trackball or touchpad, for example, to obtain a different centering of parameter adjustment.
  • In another aspect of the invention, the addition of a user interface sensor may be used to provide additional parameters that may be simultaneously controlled.
  • In another aspect of the invention, the addition of a user interface sensor may be used to provide additional parameters that are of a different isolated context from those assigned to a traditional hand-movable computer mouse.
  • In a further more detailed aspect of the invention, the addition of a touchpad may be used to provide many additional parameters that are of a different context than those of a traditional hand-movable computer mouse.
  • In a further more detailed aspect of the invention, the touchpad may be a null-contact touchpad adapted to measure at least one maximum spatial span of contact in a given direction.
  • In a yet further detailed aspect of the invention, the null-contact touchpad is adapted to measure at least one maximum spatial span of contact in a given direction at a specifiable angle.
  • In an additional further detailed aspect of the invention, the null-contact touchpad is adapted to measure pressure applied to the null-contact touchpad.
  • In a further more detailed aspect of the invention, the touchpad may comprise a pressure sensor array touchpad adapted to measure, among other things, one or more of the following: the rocking position of a contacting finger in a given direction; the rotational position of a contacting finger; the pressure of a contacting finger; and parameters relating to a plurality of contacting fingers.
  • In another aspect of the invention the addition of a user interface sensor may be realized via a replaceable module accepted by an adaptation of a traditional hand-movable computer mouse. In this implementation, a user may initially obtain the invention in one configuration and field-modify it to another configuration.
  • In another aspect of the invention, a traditional hand-movable computer mouse may be implemented as a removable module in a laptop computer or other affiliated equipment, and may include a wireless link with the laptop computer or other affiliated equipment.
  • In yet a further aspect of the invention, a traditional hand-movable computer mouse is implemented as a removable module in a laptop computer or other affiliated equipment, and the mouse further comprises a user interface sensor.
  • In another aspect of the invention, a traditional hand-movable computer mouse additionally comprises a trackball or touchpad, for example. In this aspect, the mouse comprises a wireless link to an associated computer or other affiliated equipment.
  • In another aspect of the invention, a visual display is provided.
  • In another aspect of the invention, an auditory output is provided.
  • In another aspect of the invention, two or more individual user interface sensors may be combined without incorporation of such sensors with a traditional hand-movable computer mouse.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The above and other aspects, features and advantages of the present invention will become more apparent upon consideration of the following description of preferred embodiments taken in conjunction with the accompanying drawing figures, wherein:
  • FIGS. 1 a-1 i illustrate various exemplary implementations involving merging, selecting, multiplexing, and preprocessing distributed in various ways between the body of the user interface device and an associated piece of equipment;
  • FIGS. 2 a-2 c depict an embodiment of the invention comprising a traditional mouse fitted with a trackball, illustrating three exemplary button configurations;
  • FIGS. 3 a-3 c depict an embodiment of the invention comprising a traditional mouse fitted with a touchpad, illustrating three exemplary button configurations;
  • FIGS. 4 a-4 d depict various exemplary degrees of freedom that may be measurably assigned to a trackball for interactively controlling parameters in a user interface;
  • FIGS. 5 a-5 d depict various exemplary degrees of freedom that may be measurably assigned to a touchpad for interactively controlling parameters in a user interface;
  • FIG. 6 depicts an exemplary implementation of the invention directed towards the control of both a traditional text cursor and a dual-scrollbar in a typesetting application;
  • FIG. 7 depicts an exemplary implementation of the invention directed towards the active selection from a clip-art or symbol library and adjustment of positioning or other attributes of the active selection in a drawing or layout application;
  • FIG. 8 is a flowchart showing exemplary operations and overhead involved in selecting and adjusting a specific pair of parameters from among a larger group of adjustable parameters;
  • FIGS. 9 a-9 b illustrate how the exemplary operations and overhead depicted in FIG. 8 introduce excessive overhead in situations where many parameters with a larger group of adjustable parameters must be adjusted in pairs;
  • FIGS. 10 a-10 b illustrate one technique for adding an additional scroll-wheel to a conventional scroll-wheel mouse;
  • FIGS. 11 a-11 b illustrate a simple example of open adjustments being made within various levels of hierarchy of graphical object groupings;
  • FIG. 12 illustrates aspects of the 3D orientation of an object in 3-dimensional space, and in particular the three coordinates of position and the three angles of rotation;
  • FIGS. 13 a-13 b illustrate one technique for using two cursors in a text cut-and-paste operation; and
  • FIGS. 14 a-14 d illustrate exemplary embodiments of a mouse where the traditional mouse buttons have been replaced by trackballs or touchpads.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • In the following description, reference is made to the accompanying drawing figures which form a part hereof, and which show by way of illustration specific embodiments of the invention. It is to be understood by those of ordinary skill in this technological field that other embodiments may be utilized, and structural, electrical, as well as procedural changes may be made without departing from the scope of the present invention.
  • By way of overview, a number of different applications that take advantage of the functionality of additional, wide-range adjustment parameters will now be discussed. In one example, an additional finger-wheel adjustment device providing a third, wide-range parameter adjustment capability is typically assigned to vertical scroll bar positioning. In accordance with the invention, such a design may be supplemented with a fourth, wide-range parameter adjustment capability so that a horizontal scroll bar position control may be achieved. With the increasing popularity of the web (with many web pages wide enough to require horizontal scrolling) and publisher layout tools (typically involving pages wide enough to require horizontal scrolling), as well as the need for simultaneous interactive horizontal and vertical scrolling actions that do not disturb a screen cursor location when using “zoom” controls, a fourth wide-range parameter adjustment capability in traditional user interface devices for data entry and graphical user interface pointing is quite valuable.
  • There are many other potential uses for additional wide-range adjustment parameters in traditional user interface devices for data entry and graphical user interface pointing. Some opportunities have wide-range applicability, such as in providing interactive separate adjustment of the selections for “cut” or “copy” operations from the interactive adjustment of insertion location or selection for a “paste” operation. Other opportunities are more specialized but still widely applicable, such as making an active selection fiom a clip-art or symbol library and adjusting the position or other attributes of said active selection in a drawing or layout application. Yet other opportunities may be very specialized, such as in 3D modeling, data visualization, advanced color adjustment, lighting control, machine control, or audio and image signal processing.
  • There are many opportunities for adjusting the same two widely-varying parameters in more than one way. For example, one user interface modality (such as normal mouse operation) may be used for normal parameter adjustment, while a second user interface modality may be used for adjustments involving a different resolution, warping (i.e., logarithmic, gamma-corrected, arccosine, exponential, etc.), centering offset, etc. Another important case is where the same two widely-varying parameters are controlled with the same resolution, warping, offset, etc., but in a different user interface modality (e.g., a trackball or touchpad may have some advantages in certain situations over use of a traditional mouse). A more widely applicable example is that of responding to and preventing handlwrist/arm fatigue and injury. A traditional mouse fitted with an additional user interface sensor allows a user to interchangeably enter information with either the mouse body or another user interface sensor, changing which user interface modality is used (obtaining the same results with either) to relieve fatigue or pain, or prevent injury.
  • More specifically, the addition of a user interface sensor provides many opportunities for alternative means of adjustment of a common pair of adjustable parameters. The user may benefit from having both adjustment modalities available, changing modalities as needed or desired. For example:
      • A user may simply switch between using the invention as a traditional hand-movable computer mouse and using the invention as another kind of user interface sensor.
      • The user may benefit from having both modalities available to avoid or in response to hand fatigue.
      • The user may also benefit from having both modalities available due to the type of pointing or data entry interaction needed—depending on the case, one type of modality may perform better than another.
      • The trackball, touchpad, or other user interface sensor apparatus may be used to provide alternative resolution modalities so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one level of parameter adjustment resolution and using the invention as a user interface sensor to obtain a different level of parameter adjustment resolution.
      • The trackball or touchpad may be used to provide alternative warping modalities for the same pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one type of parameter adjustment (for example, linear) and using the invention as another kind of user interface sensor to obtain a different type of parameter adjustment resolution (for example, logarithmic, gamma-corrected, arccosine, exponential, etc.).
      • The user interface sensor may be used to provide alternative offset modalities for a common pair of adjustable parameters so that a user may switch between using the invention as a traditional hand-movable computer mouse to obtain one centering of parameter adjustment and using the invention as another kind of user interface sensor to obtain a different centering of parameter adjustment. These modalities can provide one or more “location bookmarks” for cursor location, each affiliated with a sub-context within an interactive application.
  • Further, the addition of another user interface sensor provides many opportunities for the simultaneous adjustment of additional parameters that may or may not require simultaneous interactive control. The traditional computer mouse may be used to simultaneously adjust two parameters while the additional user interface sensor may be configured to allow the fingers to simultaneously adjust at least two additional parameters. In some applications, these additional parameters may be closely related to those assigned to the traditional computer mouse. For example, the traditional computer mouse may be used to simultaneously adjust the location within a window of a text, graphic, or other object, while the additional user interface sensor allows the fingers to be used to adjust the type or attributes of the text, graphic, or other object. In other applications, these additional parameters may be of a different isolated context from those assigned to the traditional computer mouse. For example, the traditional computer mouse may be used to simultaneously adjust two parameters dealing with affairs within an active application window, while the addition of another user interface sensor allows the fingers to be used to adjust at least two additional parameters dealing with broader window system affairs such as vertical and horizontal scrollbars, window selection, window resizing, etc., or intermediate-level affairs such as zoom control, help-window navigation, clip-art selection, etc. Another application would be to provide separate adjustment of selections for “cut” or “copy” operations from the adjustment of insertion location or selection for a “paste” operation.
  • In instances of the invention involving the addition of a touchpad, the touchpad may be configured and/or enhanced to allow the fingers to adjust three or more additional interactive measured parameters. These additional interactive measured parameters may be assigned to control more sophisticated interactive affairs such as 3-dimensional space position, 3-dimensional space orientation, color model navigation, image or audio processing parameter settings, etc.
  • The additional interactive measured parameters (above the two typically associated with traditional touchpads) may be provided in a number of ways. For example, the touchpad may be a relatively low-cost null-contact touchpad that has been adapted to measure at least one maximum spatial span of contact in a given direction. The user may also control an additional parameter by varying the width between the spatial extremes of a single point of contact (i.e., how much finger flesh makes contact with the pad) or multiple points of contact (i.e., the spread between two contacting fingers). As there are two geometrically orthogonal sensing directions on a touchpad, this provides the user with a method for controlling four total parameters from a touchpad. Further, rotational transformations or other methodologies may be used to measure the angle of rotation of an oblong contact profile. The measured angle may be used as a fifth interactive parameter, and/or used to adapt the measurement of maximum spatial span of contact in an arbitrary angle. The null-contact touchpad may be further adapted to measure pressure applied to the null-contact touchpad via, for example, use of an attached pressure sensor. The pressure may be used as a sixth interactive parameter, and may or may not have rapid pressure changes recognized as ‘tap’ or ‘click’ events.
  • Another way to provide the additional interactive measured parameters (above the two typically associated with traditional touchpads) with a touchpad is to implement the touchpad with a pressure sensor array. Through use of operations effectively amounting to image processing, a pressure sensor array touchpad can be adapted to measure the rocking position of a contacting finger in two orthogonal directions, as well as the rotational position and average pressure of a contacting finger. Thus a pressure sensor array touchpad can be adapted to provide up to six widely variable interactive adjustable parameters from the contact of a single finger. A pressure sensor array touchpad can be further adapted to measure parameters relating to a plurality of contacting fingers.
  • All of these considerations and others demonstrate the potential value in providing the addition of another user interface sensor to a traditional hand-movable computer mouse. In the descriptions to follow, various implementations and exemplary applications of exemplary embodiments are considered and explained.
  • 1. Exemplary Signal Flow and Processing
  • The invention provides for a wide range of signal flow and processing configurations and implementations. FIGS. 1 a-1 i illustrate various exemplary implementations involving merging, selecting, multiplexing, and preprocessing distributed in various ways between the body of the user interface device and an associated piece of equipment. FIGS. 1 a-1 d concern the aggregated pair of user interface sensors in isolation, while FIGS. 1 e-1 i address arrangements where some functions of the invention are performed in the associated external equipment. It is noted that the invention further provides for any of these exemplary functionalities, as well as other functionalities, to be combined or made selectable. In any of the exemplary implementations disclosed herein, power may be supplied to these implementations by the associated external equipment or by other devices such as batteries, storage capacitors, photoelectric devices, and the like.
  • FIG. 1 a shows an implementation 100 a featuring two user interface sensors 101, 102, each of which may be a particular type of user interface sensor (which again may be a trackball, touchpad, mouse, or other user interface device) that can be collocated within the common physical enclosure 100 (demarcated by the dotted-line boundary). First user interface sensor 101 produces signal 108 and second user interface sensor 102 produces another signal 109, which are directed to a merge or select function 103. The merge or select function produces outgoing signal 110 which is provided to associated external equipment. Here signals 108,109 (from first and second user interface sensors 101,102) would typically lose their individual identities within the outgoing signal 110 and as such may be used or processed interchangeably (without individual attribution to either the first or second user interface sensor) by the associated external equipment.
  • Merge or select function 103 may take several forms in various implementations. For example, in one embodiment it may simply be fixed to only perform a merge operation. In another embodiment it may only provide a selection function; here the selection function may be controlled by the user using a switch or some sort of action, or the selection may be remotely controlled by external equipment. As an alternative, merge or select function 103 may instead provide a user adjustable “merge” or “select” function.
  • FIG. 1 b shows an implementation that is similar to that of FIG. 1 a. The primary difference is that the FIG. 1 b implementation 100 b replaces merge or select function 103 of FIG. 1 a with multiplex function 104 to produce outgoing signal 110. Here signals 108, 109 (from first and second user interface sensors 101, 102) retain their individual identities within outgoing signal 110 and as such may be used or processed separately by the associated external equipment.
  • FIG. 1 c shows implementation 100 c which is similar in many respects to that of FIG. 1 a. However, the FIG. 1 c embodiment utilizes preprocessor 105 applied to signal 109 to produce processed signal 109 a. The processed signal 109 a, along with signal 108, is directed to merge or select function 103, resulting in outgoing signal 110. Preprocessor 105 may thus introduce a pre-processing step (such as resolution modification, warping modification, offset modification, etc.) on signal 109 to produce a signal of distinguished value from that of signal 108.
  • FIG. 1 d illustrates another exemplary implementation 100 d which is similar to that of FIG. 1 c but with an additional preprocessor 106 applied to signal 108. Preprocessor 106 produces processed signal 108 a, which along with signal 109 a, is directed to merge or select function 103 to produce outgoing signal 110. Preprocessor 106 may therefore introduce a pre-processing step (such as resolution modification, warping modification, offset modification, etc.) on signal 108 to produce a signal of either equivalent or distinguished value fiom that of signal 109 a.
  • FIG. 1 e shows implementation 100 b of FIG. 1 b (which features two user interface sensors 101,102 and a multiplex function 104 producing an outgoing signal 110) used in conjunction with subsequent functions provided by associated external equipment 150 a. These subsequent functions are shown within the functional boundary 150 of the associated external equipment 150 a.
  • Outgoing signal 110 from the common physical enclosure 100, is presented to demultiplexer 117 within external equipment 150 a. Demultiplexer 117 produces signal 118 corresponding to or associated with pre-multiplexed signal 108, and an additional signal 119 corresponding to or associated with the pre-multiplexed signal 109. Here, signals 118, 119 are presented to merge or select function 113 producing merged or selected signal 120. This implementation is functionally similar or equivalent to that of FIG. 1 a except that the various types of merge or selection functions 103 are provided within the associated external equipment 150 a (for example in software, perhaps within an application where it is customized for the needs of that application) rather than being provided within physical unit 100 b.
  • FIG. 1 f shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 b, and similar to that of FIG. 1 e, except signal 119 produced by demultiplexer 117 is directed to preprocessor 115 to produce processed signal 119 a before being sent to merge or selection function 113. This implementation is thus functionally similar or equivalent to that of FIG. 1 c except that the various types of merge or selection 103 and preprocessor 105 functions in FIG. 1 c are provided within the associated external equipment 150 b (for example in software, perhaps within an application where it is customized for the needs of that application) rather than being provided within the physical unit 100 b.
  • FIG. 1 g shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 c. This arrangement expands on that shown in FIG. 1 f in that signal 118 produced by demultiplexer 117 is directed to preprocessor 116 to produce processed signal 118 a. The processed signal is then sent to merge or selection function 113. This implementation is thus functionally similar or equivalent to that of FIG. 1 d except that the various types of merge or selection 103 and preprocessor 105, 106 functions in FIG. 1 d are provided within the associated external equipment 150 c (for example in software, perhaps within an application where it is—customized for the needs of that application) rather than being provided within the physical unit 100 b.
  • FIG. 1 h shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 d. Here again, as in the arrangement of FIG. 1 f, signal 119 produced by demultiplexer 117 is directed to preprocessor 115 to produce processed signal 119 a. In contrast to other embodiments, processed signal 119 a is not directed to merge or selection 113 function and retains its identity for use by a different destination from that of signal 118 within associated external equipment 150 d.
  • As a final illustrative example in this series, FIG. 1 i shows implementation 100 b of FIG. 1 b used in conjunction with subsequent functions provided by associated external equipment 150 e. Here, as in the arrangement of FIG. 1 g, signals 118, 119 produced by demultiplexer 117 are directed to preprocessors 115, 116 to produce processed signals 118 a, 119 a. In contrast to other embodiments, processed signals 118 a, 119 a are not directed to merge or selection 113 function and thus retain their identity for use by differing destinations within associated external equipment 150 e.
  • Having presented various exemplary signal flows and processing realizations provided for by the invention, attention is now directed to exemplary implementations utilizing specific types of user interface sensors. It is to be understood that the various sensors, techniques and methods disclosed herein may be implemented using computer software, hardware, and firmware, and combinations thereof.
  • 2. Implementations Utilizing Specific Types of Additional User Interface Sensors
  • In this section, a number of exemplary implementations of the invention utilizing various types of additional user interface sensors added to an original user interface sensor or device. The first three sections address cases where the original user interface sensor is a movable mouse and the additional user interface sensor is a trackball, touchpad, or other exemplary technology, including additional scroll-wheels. Then exemplary adaptations of trackballs and touchpads, each traditionally used to provide simultaneous adjustment of two interactive widely-varying parameters, are extended to provide simultaneous adjustment of as many as six interactive widely-varying parameters and other forms of control. This section continues by presenting exemplary implementations where the original user interface sensor is not a mouse, where there is a plurality of additional user interface sensors, where there is a visual display or auditory output, where the additional user interface sensor is a removable module, and where the implementation itself is a removable module.
  •  2.1 Trackball Implementations of Additional User Interface Sensors
  • FIGS. 2 a-2 c illustrate a number of implementations provided for by the invention where a trackball controller is used as an additional user interface sensor apparatus incorporated into a traditional hand-movable computer mouse. In each of these implementations it is understood that the trackball may be freely operated without disturbing previous or currently-varying parameter adjustments made by the mouse.
  • In one implementation, a trackball controller is added to the top surface of a conventional computer mouse as depicted in FIG. 2 a. The conventional mouse buttons may be located in various places in view of the presence of the trackball and in synergy with it. In the configuration depicted in FIG. 2 a, buttons 201 and 202 are located on the surface of mouse 200; button 201 being on the left of the trackball and button 202 being on the right of the trackball.
  • In a second configuration depicted in FIG. 2 b, buttons 231 and 232 are now separated and located on the sides of mouse 230 as is the case with many trackball interfaces; button 231 being on the left side of mouse 230 and button 232 being on the right side of the mouse.
  • In a third possible configuration depicted in FIG. 2 c, elongated buttons 261 and 262 are shown located on the surface of the mouse 260; button 261 wraps around the left of the trackball and button 262 wraps around the right of the trackball. The elongated buttons 261 and 262 may be positioned so that a user can readily and rapidly move fingers from trackball 265 to buttons 261 and 262, or even operate one of these buttons with one finger while another finger contacts trackball 265.
  • It is noted that unlike the touchpad described below, the trackball has an effectively unconfined range of contiguous data entry.
  •  2.2 Touchpad Implementations of Additional User Interface Sensor
  • FIGS. 3 a-3 c illustrate a number of exemplary implementations provided for by the invention where a touchpad controller is-used as an additional user interface sensor incorporated into a traditional hand-movable computer mouse. In each of these implementations it is understood that the touchpad may be freely operated without disturbing previous or currently-varying parameter adjustments made by the mouse.
  • In one implementation, trackball 205 in FIG. 2 a can be replaced with touchpad 305 as shown in FIG. 3 a. Additionally, this touchpad implementation can also support the alternative button configurations of FIGS. 2 b and 2 c. By way of illustration, FIG. 3 b shows buttons 331, 332 positioned on either side of the mouse body 330, while FIG. 3 c shows elongated buttons 361, 362 on the surface of the mouse wrapping around either side of touchpad 365.
  • It is noted that unlike the trackball, the touchpad typically has a confined maximum range of data entry by contiguous operation of a finger, stylus, etc.
  •  2.3 Other Implementations of Additional User Interface Sensors.
  • In accordance with other embodiments, the invention provides for still other types of additional user interface sensors. In each of these embodiments it is understood that any of these user interface sensors may be freely operated without disturbing previous or currently varying parameter adjustments made by the mouse or other associated device.
  • As a first example, an X-Y joystick may be used in place of the trackball or touchpad described above. The joystick may have a spring-return or may retain the last position it was placed. Similar to the touchpad and unlike the trackball, the X-Y joystick typically has a confined maximum range of travel.
  • As another example, two or more scrolling finger wheels may be used in place of the trackball or touchpad described above. The scrolling finger wheels may be implemented with an unconfined maximum range of travel similar to the trackball, or with a confined range of travel like the touchpad and X-Y joystick. In this embodiment, it may be advantageous to have one or more finger scroll-wheels mounted with its adjustment direction perpendicular to that of another finger scroll-wheel so that each wheel may be appropriately associated with vertical and horizontal scroll bars of a window, or other useful orthogonally-based user interface metaphor. For example, looking ahead to FIGS. 10 a and 10 b, embodiments 1000 are depicted comprising the usual components of a scroll-wheel mouse (mouse body 1001, buttons 1011 and 1012, and the usual scroll wheel 1021) complemented with an additional scroll wheel 1022 with adjustment direction perpendicular to that of finger scroll-wheel 1021.
  • As another example, two or more rotating control knobs may be used in place of the trackball or touchpad described above. Like the scrolling finger wheels, the rotating control knobs may be implemented with an unconfined maximum range of travel like the trackball or with a confined range of travel like the touchpad and X-Y joystick.
  • The invention also provides for more exotic types of user interface sensor technologies—for example, proximity detectors of various types (RF, infrared, etc.), video cameras (using any of the techniques provided in U.S. Pat. No. 6,570,078), and other emerging and unforeseen technologies—to be used as the additional user interface sensor aggregated in the same physical enclosure as the first user interface sensor.
  •  2.4 Larger Numbers of Interactively Widely-Varying Adjustable Parameters from the Additional User Interface Sensor
  • In accordance with embodiments of the invention, additional user interfaces may be used to capture larger numbers (i.e., more than two) of widely-varying adjustable parameters from the additional user interface sensor. Some examples are provided here, but many others are possible as may be readily understood by one skilled in the art.
  • FIGS. 4 a-4 d address the case of the trackball. FIG. 4 a illustrates a freely-rotating trackball 205 and the two principle orthogonal adjustment directions 401,402 that are responsively resolved and measured in traditional trackball user interface devices. However, at least two other physical degrees of freedom may be readily exploited, and at least six total parameters can be interactively adjusted and measured. FIG. 4 b shows the application of downward pressure 403 on trackball 205. Such pressure 403 may be applied without disturbing current values established in orthogonal adjustment directions 401, 402. Further, the trackball may be implemented so that downward pressure 403 may be applied while simultaneously adjusting trackball 205 in orthogonal adjustment directions 401, 402, particularly if the signal produced by the measurement of downward pressure 403 incorporates a modest “grace” zone of non-responsiveness for light pressure values. Downward pressure impulses may alternatively be sensed and treated as discrete event “taps,” as commonly used in contemporary touchpad interfaces found in laptop computers, for example.
  • FIG. 4 c shows the application of “yaw” rotation 404 (i.e., rotation around the vertical axis) of trackball 205. This yaw rotation 404 may be applied without disturbing current values established in orthogonal adjustment directions 401,402 and can be readily measured and adjusted as a widely-varying parameter. Further, by grasping trackball 205 (or other operational methods), the yaw rotation 404 and traditional orthogonal adjustment directions 401,402 may be independently and simultaneously adjusted. It is also noted that in principle up to six widely-varying physical degrees of freedom can be simultaneously measured from a properly configured trackball 205 by placing the trackball 205 in a cradle that senses not only downward pressure 403 or displacement but also lateral pressure or displacement. As shown in FIG. 4 d, both forward-backward, non-rotational force 405 and left-right, non-rotational force 406 may be applied to the trackball in a manner that the values of force or displacement in each of these directions 405, 406 can be independently measured. Thus, by grasping trackball 205 (or other operational methods), three rotational directions of orientation 401, 402, 404 and three non-rotational directions of force or displacement 403, 405, 406 may be independently and simultaneously adjusted by a user and measured as six independent interactively adjustable user interface parameters. These correspond, effectively, to measurable adaptations of the six degrees of freedom of an orientable object in 3-dimensional space as found in classical mechanics and aeronautics—that is:
      • “roll” rotation (adapted to 401)
      • “pitch” rotation (adapted to 402)
      • “up-down: displacements (adapted to 403)
      • “yaw” rotation (adapted to 404)
      • “forward-backward” displacements (adapted to 405)
      • “left-right” displacements (adapted to 406).
  • Most trackball sensing technologies use optically based techniques for sensing the two traditional components of rotation (“roll” and “pitch”) of the trackball. The trackball itself may be configured with an optical pattern on it with spatially varying reflectivity for a range of the light spectrum. The pattern may be such that it can spatially vary light reflectively in these two traditional components of trackball rotation. Alternatively, two spatially varying reflectivity patterns, each active at different ranges of the light spectrum or light polarization, may be superimposed or integrated with the trackball.
  • A number of approaches may be used to obtain measurements for all three directions of rotation. In one completely optical approach, a second or third spatially varying reflectivity pattern active at, respectively, a second or third portion of the light spectrum (or light polarization if available) may be superimposed or integrated with the patterns employed for traditional “roll” and “pitch” rotation sensing, and an additional optical source and sensor is used to obtain measurement of the added varying reflectivity pattern. Depending on the pattern(s) used, sensor signals may be directly usable or may require processing of the three primitive signals measured by the sensors to obtain a clean decomposition of the measurement signals into independent “roll,” “pitch,” and “yaw” signals independently responsive to the “roll,” “pitch,” and “yaw” components of trackball rotation.
  • As another alternative, the trackball may include internally, or on its surface, or both, materials with spatially varying patterns of magnetic properties, capacitive properties, electromagnetic properties, ultrasonic acoustic properties, resonance phenomena of any of these properties, polarization phenomena of any of these properties, etc., individually or in combination, each of which may be active at specific ranges of polarization, frequencies, etc. These may be used together with or in place of optical measurement approaches. Again, depending on the pattern(s) used, sensor signals may be directly usable or may require processing of the three primitive signals measured by the sensors to obtain a clean decomposition of the measurement signals into independent “roll,” “pitch,” and “yaw” signals independently responsive to the “roll,” “pitch,” and “yaw” components of trackball rotation as is clear to one skilled in the art. It is also noted that the third component of rotation of the freely-rotating trackball may be interpreted or even measured as a discrete “click” event.
  • Similarly, a number of approaches may be used to obtain measurements for one, two, or three directions of non-rotational trackball displacement. For example, the trackball may be secured in a saddle allowing free rotation of the trackball but causing any displacement actions on the trackball to invoke displacements of the saddle. The saddle displacement may be measured with a displacement sensor which itself may comprise one or more pressure, optical, resistive, capacitive, magnetic, electromagnetic, continuous-range sensors, switches, etc. It is also noted that one or more components of displacement of the freely-rotating trackball may be interpreted or even measured as a discrete “click” event.
  • FIGS. 5 a-5 d turn now to the case of the touchpad. FIG. 5 a illustrates touchpad 305 and the two principle orthogonal data entry directions 501, 502 that are responsively resolved and measured in traditional touchpad user interface devices. The touchpad shown in FIGS. 5 a-5 d, which provides at least four other physical degrees of freedom, may be implemented using the techniques presented in U.S. Pat. No. 6,570,078, for example.
  • FIG. 5 b illustrates the use of downward pressure 503 in the context of a touchpad. In contemporary touchpad interfaces, such as those found in laptop computers for example, such downward pressure 503 is sensed and utilized as discrete event “taps.” However, downward pressure 503 may also be measured and adjusted as an independent and simultaneously interactive widely-varying parameter. Further, as illustrated in FIG. 5 c, the rotational angle 504 of a finger contacting a touchpad with rough-elliptical contact boundary can also be measured as a widely-varying parameter. In FIG. 5 d, both forward-backward 505 and left-right 506 components of the tilt of a contacting finger can additionally be measured as independent and simultaneously interactive widely-varying parameters.
  • The sensing of multiple fingers, the application of contact syntaxes and grammars, and other user interface control expansions of an adequately configured touchpad may also be achieved using, for example, the techniques presented in U.S. Pat. No. 6,570,078.
  • The invention also provides for larger numbers (i.e., more than two) of widely varying adjustable parameters from other types of user interface sensor technologies. In the case of an X-Y joystick, the joystick may be configured to rotate on its axis, pulled in and out, fitted with a knob or trackball, etc., in a measurable fashion to provide additional and simultaneous interactively adjustable parameters. In the cases of finger scroll-wheels and rotational knobs, three or more of these devices may be provided. When implementing video cameras, known techniques for the extraction of additional parameters may be used. Examples of the various types of video extraction techniques that may be used are presented in U.S. Pat. No. 6,570,078.
  •  2.5 Non-Mouse User Interface Sensors
  • In one of its most abstract forms, the invention involves the incorporation of two conventional user interface sensors (such as a mouse, trackball, touchpad, joystick, etc.) into an embodiment where the user may freely use both of the user interface sensors individually or simultaneously with the same hand. As such, the invention provides for implementations that do not include a mouse as one of the user interface sensors. For example, two or more individual user interface sensors can be combined without need of a traditional hand-movable computer mouse. Such an implementation may be accomplished by implementing one of the possible user interface sensors in place of the traditional hand-movable computer mouse where taught in other descriptions of the invention. This configuration may be useful when built into a laptop computer, control console, musical instrument, and test instrument, among others.
  • In one exemplary implementation of a non-mouse embodiment, a trackball and touchpad may be arranged in the same physical enclosure so that the front, middle, or back of the palm may freely operate a conventional trackball while one or more selected extended or arching fingers may simultaneously or alternatively operate a touchpad. In this example, the touchpad may be a conventional touchpad providing two widely-varying simultaneously interactive parameters from a single finger contact, or the touchpad may be a more enhanced version providing as many as six widely-varying and simultaneously interactive parameters from a single finger contact. The touchpad may also be configured to accept multiple points of contact, recognize gestures, support syntax and grammatical constructions using, for example, the teachings provided by U.S. Pat. No. 6,570,078.
  • In another non-mouse implementation, two trackballs may be arranged in the same physical enclosure. In one possible arrangement, the two trackballs may be positioned so that they lie parallel to the length of the hand, enabling the front, middle, or back of the palm to freely operate a first trackball while one or more extended or arching fingers may simultaneously or alternatively operate the second trackball.
  • In another arrangement, the two trackballs may be positioned so that they lie parallel to the width of the hand, so that the fingers and/or thumb on the left side of the hand may operate a leftmost trackball while the remaining fingers and/or thumb on the right side of the hand may individually or simultaneously operate a rightmost trackball. In each of these arrangements, either or both of the trackballs may be a conventional trackball providing two widely-varying and simultaneously interactive parameters, or it may be a more enhanced trackball providing as many as six widely-varying and simultaneously interactive parameters as described earlier.
  • In addition to the just-described embodiments, alternative arrangements, such as the combination of a palm-operated trackball and a recessed joystick, and others, are also provided for by the invention.
  •  2.6 Use of More than One Additional User Interface Sensor
  • Typically the arrangements of two non-mouse user interface sensors described above in Section 2.5 can also be applied to embodiments of the invention where a mouse user interface sensor is used. In such embodiments, a mouse user interface sensor is supplemented with at least two additional user interface sensors ergonomically arranged so that the two additional user interface sensors may be simultaneously or alternatively operated by the same hand. If these embodiments are further configured so the mouse body is readily moved with adequate precision via the back of the operating hand, then all three user interface sensors may be simultaneously or alternatively operated by the same hand in an ergonomically advantageous manner.
  • FIGS. 14 a-14 d illustrate some exemplary embodiments of the just-described features. FIG. 14 a illustrates a mouse where traditional mouse buttons have been replaced by trackballs 1405 a, 1405 b. These trackballs 1405 a, 1405 b may accept a downward pressure impulse and as such act as the traditional mouse buttons. However, trackballs 1405 a, 1405 b are also adjustable and each readily provides two or more additional widely-variable and simultaneously adjustable parameters in addition to the two parameters adjusted by moving the mouse body 1400.
  • FIG. 14 b shows a similar arrangement where traditional mouse buttons have been replaced by touchpads 1435 a, 1435 b. If desired, these touchpads may accept a downward pressure impulse and as such act as traditional mouse buttons, but are also adjustable as touchpads and as such each readily provides two or more additional widely-variable and simultaneously adjustable parameters. As described in Section 2.5, a single hand may be positioned to comfortably operate simultaneously or alternatively both trackballs or both touchpads. If these embodiments are further configured so the mouse body is readily moved with adequate precision via the back of the operating hand, then either of these embodiments readily provides six to twelve widely-variable and simultaneously adjustable parameters.
  • Other configurations are of course possible. For example, the configurations of FIGS. 14 a and 14 b may be blended as depicted in FIG. 14 c, or in its mirror image. As another example, FIG. 14 d illustrates a more extreme realization comprising a left-fingers/thumb trackball 1465 a, a right-fingers/thumb trackball 1465 b, a palm trackball 1465 c, and a traditional clickable scroll-wheel 1468. Yet another alternative is to replace one or more of the trackballs of the FIG. 14 d embodiment with a touchpad user interface sensor.
  •  2.7 Incorporation of Visual Display and Auditory Output
  • If desired, any of the mouse and non-mouse embodiments may further include a visual display. The visual display may provide details of adjustable parameter values, operation modalities, etc. The visual display may be physically associated with a physical enclosure (such as that of a traditional computer mouse), or may be displayed on the computer screen or display of other associated equipment.
  • Alternatively or additionally, any of the mouse and non-mouse embodiments may further provide auditory output. The auditory output may provide details of adjustable parameter values, operation modalities, error conditions in usage of the invention, a condition relating to elapsed time or other metric of consistent use of a single usage modality, etc. The auditory output associated with the invention may be physically associated with a physical enclosure (such as that of a traditional computer mouse), or may be produced by speakers or headsets affiliated with the computer or other associated equipment.
  •  2.8 Provisions for Field Installation or Replacement of Additional User Interface Sensor
  • The invention also provides for the user interface sensor to be implemented using a replaceable module accepted by an adaptation of a traditional computer mouse. In this implementation a user may initially obtain the invention in one configuration and field modify it to another configuration.
  •  2.9 Implementation as a Module Removable from Affiliated Equipment
  • The invention also provides for a traditional computer mouse to be implemented as a removable module in a laptop computer or other affiliated equipment, and may include a wireless link with such devices. In particular, this removable module may further include one or more user interface sensors, with these sensors operable as a traditional trackball or touchpad when the invention is stowed in the laptop computer or other affiliated equipment in such a way that the invention's traditional hand-movable computer mouse modality is unmovable and hence unusable.
  • 3. Exemplary Applications
  • Departing now from the range of extreme realization and embodiments of the invention, attention is directed towards particular applications of the invention. A number of examples of various embodiments of the invention in a wide range of applications will now be presented. Many of these applications are viable with only the simplest physical embodiments of the invention (for example, those suggested by FIGS. 2 a-2 c and 3 a-3 c). In the discussion that follows, particular note is directed towards the discussion in Section 3.3 involving FIGS. 8 and 9 a-9 b. Although the discussion is motivated by a graphical layout application, the principles of the discussion in Section 3.3 involving FIGS. 8 and 9 a-9 b are very general, and the discussion illustrates the power of the invention for various applications in almost directly quantifiable terms.
  •  3.1 Wrist/Hand/Arm-Fatigue Relief and Prevention Application
  • The danger and damage stemming from extensive continuous or mis-postured mouse usage to wrist, hand, and arms are sadly misfortunate and increasingly well recognized. As the present invention provides a plurality of different user interface sensors, it is well suited for use in responding to and preventing wrist/hand/arm fatigue due to overuse. In one exemplary implementation, user interface parameters can be interchangeably adjusted with either the movement of the mouse body or the use of an integrated trackball or touchpad with identical task results. Thus a user with a tiring hand can change at will the user interface sensor employed according to how the hand feels or the nature of a specific task. In addition, to prevent fatigue or injury, the user can also switch back and forth between moving the mouse body and using the trackball/touchpad either by free choice or by following auditory or visual prompting from a time or usage monitor.
  •  3.2 Double-Scrollbar Application
  • Contemporary mice often feature a small rotating wheel between the buttons for use in operating the vertical scroll bar of a window without changing the position of the mouse. In one particular application of the invention, the left-right sensing capability of the trackball or touchpad may be used to add a similar capability for horizontal scroll bars of a window.
  • In a trackball implementation, a user can move the vertical bar 611 of FIG. 6 up by rotating trackball 205 away from him/herself, or one can move the vertical bar 621 down by rotating trackball 205 towards him/herself. By rotating trackball 205 to the left, the user can move the horizontal bar 621 left. Similarly, the user can move the horizontal bar 621 right by rotating the trackball 205 to the right. In a touchpad implementation, a user can move scroll bar 611 up by sliding the finger away from her/himself or move scroll bar 611 down by sliding the contacting finger towards her/himself; similarly, the user can move the scroll bar 621 left by sliding the finger to the left or move the scroll bar 621 right by sliding the contacting finger to the right.
  • In another implementation, the vertical and horizontal scroll bars may be adjusted with a conventional scroll-wheel mouse that has been fitted with an additional scroll-wheel. FIGS. 10 a and 10 b depict exemplary embodiments 1000 of such an arrangement which comprise the usual components of a scroll-wheel mouse including mouse body 1001, buttons 1011 and 1012, and traditional scroll wheel 1021, with these components further complemented by an additional scroll wheel 1022 with adjustment direction perpendicular to that of finger scroll-wheel 1021. FIG. 10 a illustrates an arrangement where the additional scroll-wheel is located closer to the user while FIG. 10 b illustrates an arrangement where the additional scroll-wheel is located farther away from the user. In each of these arrangements, the two scroll-wheels are shown co-centered with respect to the mouse body; for simultaneous adjustment it may be advantageous to locate the additional scroll-wheel 1022 to one side or the other of the centered positions shown in FIGS. 10 a and 10 b. One approach useful for supporting both left-handed and right-handed users, which may provide additional utility, would be to provide two off-centered additional scroll-wheels, one on either side of the center line of the mouse body 1001 and conventional scroll-wheel 1021.
  •  3.3 Traditional 2D Layout, CAD, and Graphics Applications
  • In most contemporary 2-dimensional layout and graphics applications, such as those commonly used for viewgraphs, page layout, electronic CAD, etc., numerous mouse operations are necessary for each of the many types of object attribute modification, etc. Typically, these mouse operations are required because the mouse only allows for the interactive adjustment of two widely-varying parameters at a time, and the user must change context several times as the parameters adjusted by the mouse are chosen, adjusted, and then replaced with another pair of parameters. The present invention is useful in many of these circumstances because it allows for more than two parameters to be adjusted at the same time.
  • FIG. 7 shows an example of a session involving the authoring of a viewgraph. The viewgraph authoring task showcased in this example includes the creation of a flowchart diagram (here depicting a business workflow process) and as such also illustrates related needs and attributes of a 2D CAD program involving layout of a diagram (such as a circuit, algorithm, etc.) or physical object (such as a PC board, control panel, semiconductor photolithography mask, etc.). In that this example further involves drawing, the example also illustrates the related needs and attributes of a paint-box or electronic drafting application.
  • In this broadly representative application, application window 700 is shown comprising menu area 700 a and drawing area 700 b. Within the drawing area, viewgraph title 701 and portion 702 of the flowchart to be drawn, comprising thus far a sequence of flowgraph objects connected by arrows, have already been entered and rendered. New flowgraph objects may be introduced in standard fashion by selecting the type of new object desired from a palette, initially putting an instance of the selected object type in a convenient place in the drawing area, adjusting the size, orientation, color, and/or other attributes, and putting into final position. Often a number of the last few steps are interactively cycled through multiple times before the newly introduced object is adequately drawn and the user directs their attention to the next task. In this example, the palette of available objects of a specific high-level task is shown as an overlapping stack of three sub-class palettes 703 a, 703 b, 703 c, each providing a selection of available objects within that sub-class. Here, for example, sub-class palette 703 a has been selected (as indicated by the heavy line) from other available objects 705, within the sub-class palette 703 a. Upon selection, an initial highly adjustable rendering of a specific instance 714 of the selected object 704 appears in a convenient location, which may be selected by the user.
  • The specific instance 714, rendered in this highly adjustable initial state, is typically surrounded by graphical handles 716 which facilitate sizing, positioning within the drawing area, and often at least rotational orientation (for example, using the mouse with the ALT key simultaneously held down to interactively adjust the angle of rotation of the object should that be needed). Traditionally, the cursor controlled by the mouse 717 can be moved within object 714 to relocate it within drawing area 700 b or can, as shown in FIG. 7, be positioned atop one of the graphical handles 716 to permit the mouse to adjust the horizontal and vertical scale of object 714, i.e., adjust its size and aspect-ratio. In some application packages, the latter adjustment is permitted to collapse the object through to ‘zero’ thickness in one of the adjustment dimensions and continue through to re-render the object in mirror image, thus additionally providing a form of vertical and horizontal flip by using the size and aspect-ratio resizing.
  • As familiar and widely accepted as these sorts of operations are, there is considerable overhead involved in such sequences of repeated selecting and adjusting (and in some cases additional deselecting) pairs of parameters from a larger collection of parameters. To see several aspects of the power of the present invention, these operations are now examined in more detail in generalized form.
  • FIG. 8 is a flowchart showing tasks involved in selecting and adjusting one of a plurality of available pairs of adjustable parameters by using a user interface device permitting the adjustment of only one pair of parameters at a time. In FIG. 8, the task goal is simply to adjust a pair of selected parameters 801 from a larger group of adjustable parameters. However, since a larger number of parameters are available for adjustment than are available at one time with the user interface sensor, the specific pair of parameters must first be selected. In most known graphical user interface systems and methods, this typically involves first using the user interface device to control the movement of a cursor to a selection area of the graphical interface in a first overhead step 811 and then selecting the adjustment context (parameter pair) in a second overhead step 821.
  • In some situations the selected pair of parameters may be immediately adjusted in goal operation 801, but typically the cursor must then at least be moved, in a third overhead operation 812, to a location outside of the selection area affiliated with operations 811 and 821 to another location (such as a drawing or typing area) affiliated with the context (parameter pair) that has just been selected for adjustment. In some situations the selected pair of parameters may be immediately adjusted in goal operation 801, but typically the context must be activated (for example, by clicking in an open portion of a drawing area or selecting an existing object) in a fourth overhead operation 822.
  • After the selected pair of parameters are adjusted (for example, by sizing a rectangle, etc.) the cycle may then immediately be repeated in some variant form for another pair of parameters, but typically the parameters must be deselected (for example, by another click to set the final value) in a fifth overhead operation 823 before the cursor may be moved to the selection area in another instance of operation 811. In summary, in order to adjust one pair of parameters from a larger group of parameters, as many as five overhead operations (as many as two cursor movements 811, 812 and as many as three select/deselect clicks 821, 822, 823) are commonly required.
  • FIGS. 9 a-9 b show broader implications of the overhead called out in FIG. 8. FIG. 9 a depicts the sequential adjustment of pairs of parameters chosen from a larger group of pairs of parameters in a scenario suggestive of no interactive iteration. One pair of parameters is adjusted with up to five overhead operations in action 901, then a second pair of parameters is adjusted with up to five overhead operations in action 902, then a third pair of parameters is adjusted with up to five overhead operations in action 903, and so on. Here the overhead slows things down but may not be a significant encumbrance to the broader goal of actions 901, 902, 903, etc.
  • In contrast, FIG. 9 b depicts an interactive adjustment of pairs of parameters from a larger group of parameters in a scenario suggestive of one where interactive iteration is required, as the setting of one pair of parameters is difficult to complete without setting other parameters. Here the overhead is likely a significant encumbrance to the higher goal involving the pair-wise adjustment actions 901, 902, 903, etc. For example, consider the interactive adjustment of six parameters, one pair at a time, through pair-wise adjustment actions 901,902,903; not only are up to five operations of overhead involved for each of the pair-wise adjustment actions 901,902,903, but a considerable extra number of passes must be made through these pair-wise adjustment actions 901, 902,903 due to the fact that the adjustment of some parameters depends on or interacts heavily with the values of other parameters. The situation gets even more cumbersome should additional pair-wise adjustment operations be required. FIG. 9 b further shows the potential for one or more additional adjustment actions 950 which in principle may be iterated as well as and combined with pair-wise adjustment actions 901, 902, 903 (as suggested by fully-connected iteration paths 921, 922, 923). In contrast to such sequences or iterative graphs of pair-wise adjustment actions, the present invention readily offers, for example, four, six, eight or even higher numbers of simultaneously adjustable parameters controllable by the same hand, which, when selected in a context, eliminate the many overhead operations depicted in FIGS. 8 and 9 a-9 b, and the many additional iteration steps depicted in FIG. 9 b.
  • Returning now to the generalized graphical layout situation described earlier and depicted in FIG. 7, the following operations are routinely performed in 2D graphics, layout, and CAD applications:
      • A. Selection of palette containing object;
      • B. Selection of object from palette;
      • C. Selection of “layer” object is to be assigned to (common in CAD, but typically not used in standard draw and paint packages);
      • D. Adjustment of object placement in drawing;
      • E. Adjustment of object sizing;
      • F. Adjustment of object rotation;
      • G. Adjustment of object line thickness(es);
      • H. Adjustment of object line color(s);
      • I. Adjustment of object fill color(s); and
      • J. Adjustment of object fill pattern(s);
  • Of these, operations B, D, and E are almost always utilized, operations A, G, and I are frequently utilized, and operations C, F, H, and J are rarely utilized.
  • Thus, in one exemplary application of the invention, it may be advantageous to group specific collections of operations that are commonly used together (this may be application specific) so that the benefits of having four or more widely-adjustable interactive parameters simultaneously available can be applied to speed the execution of basic common operations. For example:
      • Employing a four-parameter version of the invention:
        • Operation 1: Mouse for operation B
        • Operation 2: Mouse for operation D, trackball or touchpad for operation E
      • Employing a six-parameter version of the invention comprising a 4-parameter touchpad:
        • Operation 1: Mouse for operation B, touchpad finger-location for operation D, touchpad finger-tilt for operation E.
      • Employing a six-parameter version of the invention comprising a mouse and two trackballs or touchpads:
        • Operation 1: Mouse for operation B, first trackball/touchpad for operation D, second trackball/touchpad for operation E.
  • Other operations can be later applied in groupings and operations appropriate for the application.
  • As a possible alternative to the preceding example, it may be advantageous to assign a principal one of the user interface sensors to the sequential adjustment of each of such universal (or otherwise principal) operations and reserve the additional user interface sensors for rapid “in-context” interactive access to less frequently used operations. For example:
      • Employing a four-parameter version of the invention:
        • Operation 1: Mouse for operation B, trackball or touchpad for operation A and/or operation C;
        • Operation 2: Mouse for operation D, trackball or touchpad for operation E and/or operation F;
        • Operation 3: Mouse for operation G, trackball or touchpad for operation H; and
        • Operation 4: Mouse for operation I, trackball or touchpad for operation J.
      • Employing a six-parameter version of the invention comprising a 4-parameter touchpad:
        • Operation 1: Mouse for operation B, touchpad finger-location for operation A and touchpad finger-tilt for operation C;
        • Operation 2: Mouse for operation D, touchpad finger-location for operation E and touchpad finger-tilt for operation F;
        • Operation 3: Mouse for operation G, touchpad finger-location (and touchpad finger-tilt as useful) for operation H; and
        • Operation 4: Mouse for operation I, touchpad finger-location (and touchpad finger-tilt as useful) for operation J.
      • Employing a six-parameter version of the invention comprising a mouse and two trackballs or touchpads:
        • Operation 1: Mouse for operation B, first trackball/touchpad for operation A and second trackball/touchpad for operation C;
        • Operation 2: Mouse for operation D, first trackball/touchpad for operation E and second trackball/touchpad for operation F;
        • Operation 3: Mouse for operation G, first trackball;/touchpad (and second trackball/touchpad as useful) for operation H; and
        • Operation 4: Mouse for operation I, first trackball/touchpad (and second trackball/touchpad as useful) for operation J.
  • As another alternative example, the user may freely assign user interface sensor parameters to operations A through J (and others as may be useful) for each of a number of steps as may match the task or tasks at hand. These assignments may be stored for later retrieval and use, and may be named by the user. The stored assignments may be saved along with specific files, specific applications, or as a general template the user may apply to a number of applications. It is noted that such variable assignments may be particularly useful to users as their hands fatigue, to prevent fatigue or injury, or as an adjustment for a temporary or permanent disability.
  •  3.4 Multi-Resolution Mouse Application
  • In another exemplary family of applications, one user interface sensor (for example, the mouse body) is used for course adjustment or fine adjustment of user interface parameters, while the additional user interface sensor (for example, a trackball or touchpad) is used for the remaining level of parameter adjustment resolution.
  • In most user-interface applications it is advantageous to have multiple scales of graphical user interface pointing and data entry. Many window systems provide an ‘acceleration’ setting which changes the pointing and data entry values on a more significant scale used for user interface changes made less frequently. Many applications further internally adjust the resolution as the corresponding visual display is “zoomed” in and out.
  • In many user interface applications, additional levels of resolution selection may be useful. For example, in pointing usage in text work, multiple resolutions would be advantageous in amending fine print or in making isolated changes in thumbnail overviews of 40% actual size or less. Similarly, in graphics work, fine resolution may be especially useful in making fine adjustments to figures. In the fine adjustment of figures, it may be further advantageous to employ each of the separate user interface sensors in conjunction with corresponding snap-grids of differing grid spacing, particularly if one of the grid spacings is a sub-multiple of the other. A potentially useful extension of this would be to impose locally-applicable grid spacing on individual graphic or other objects, particularly objects which have been resized and hence for which the standard snap-grid spacing is no longer useful.
  • In a further application, the user interface may be directed towards non-positional adjustments, such as the adjustment of a rotation angle or of the color of a graphic object; here multiple resolutions may be valuable to make careful adjustments and coarse adjustments as needed. Similarly, scroll bars for long documents may also benefit from rapid access to multiple resolution scales, for example one user interface sensor may be used to navigate within a page (using a fine-grained navigation scale) while a second user interface sensor may be used to navigate across pages (using a coarser-grained navigation scale).
  •  3.5 Provision of Both Absolute and Relative Positioning
  • As discussed earlier, some types of user interface sensors, such as the touchpad and X-Y joystick for example, naturally have a limited maximum range of operation while others such as a mouse, trackball, and scroll-wheel have an essentially unlimited maximum range of operation. Although most user interface sensors are interpreted in relative terms (that is, the stimulus from the sensor is interpreted as a command to move a cursor, scroll bar, etc., incrementally in some direction relative to a current position), stimulus signals from any of these types of user interface signals may be interpreted in either a relative or absolute manner with varying degrees of naturalness or problematic qualities.
  • The present invention provides for one user interface sensor to be used for absolute positioning of a cursor, scroll bar, etc., or other means of parameter adjustment while another user interface sensor is used for traditional relative adjustment of such parameters. For example, a scroll bar may be adjusted in the usual fashion by a mouse body or trackball and in an absolute manner by a touchpad wherein the extreme values of the adjusted parameter correspond to the extreme positions at the edges of the touchpad. In one embodiment or application setting these two user interface sensors may control the same parameters—here it is often the result that the two sensors adjust the same parameters with different resolutions. Further, in this situation it is fairly likely that at least one of the resolution scale factors will be adjusted automatically. For example, in a document editor, as the number of pages of the document varies, the resolution of the absolute positioning sensor will correspondingly vary (so that the extremities in range of, for example, a touchpad correspond to the top of the first page and end of the last page) while the relative positioning sensor may retain the same incrementing/decrementing vertical scrolling resolution scale regardless of the number of pages.
  •  3.6 Color-Selection Application
  • In color adjustment, three parameters are involved in the full interactive span of any complete color space (RGB, HSB, YUV, etc.). By adding additional parameters to the overall user interface, all three parameters can be adjusted simultaneously rather than simply two at a time. As the present invention provides at least four simultaneous interactively adjustable parameters overall, it is thus potentially useful for fully interactive color adjustment within a complete color space model. Further, should the additional user interface sensor be such that it alone provides three simultaneously interactively adjustable parameters, the first user interface sensor (for example, the mouse body) may be used as a pointer to select objects and the additional user interface sensor may be used to adjust attributes of the selected object such as its color, border color, etc.
  •  3.7 Multi-level Graphic Object Grouping and Editing Application
  • In many drawing applications, lower-level graphical or other objects (such as lines, basic shapes, and text areas) may be grouped to form an aggregated object. This aggregated or “grouped” object (collectively referred to herein as an “aggregated object”) can then be moved, rotated, flipped, resized, etc. as if it were a lower-level graphic or other object. Grouping can also typically be done hierarchically and in mixed hierarchies, i.e., a plurality of lower-level graphical or other objects may first be grouped, and the resulting aggregated object may then itself be grouped with other aggregated objects and/or lower-level graphical or other objects.
  • Often one or more of the lower-level graphical or other objects comprising the aggregated object may need modification. In the case of text, most applications permit modifications to be made to individual text objects within an aggregated object. However, for any isolated adjustment to any other lower-level graphical or other object the aggregated object must be first disaggregated or “ungrouped” to completely free the involved lower-level graphical or other object from any grouping it had been involved in. After the modification, the grouping must be reconstructed. Often this becomes a cumbersome situation, particularly where the adjustments within the group are themselves an interactive response to other adjustments made within a drawing.
  • The additional number of widely-adjustable simultaneously interactive parameters made possible by the invention may be advantageously applied to this problem. For example, one user interface sensor may be used to navigate the levels of grouping and another user interface sensor may be used to perform operations on objects (lower-level or “grouped”) within that level of grouping of the overall aggregated object.
  • As an illustrative example, FIG. 11 a shows a portion 1100 of a larger drawing, the portion 1100 featuring box 1101, two arrowed lines 1111, 1112, and grouped object 1102 (here itself comprising two triangles connected by a line). In this example it is given that grouped object 1102 is itself grouped with box 1101 to form a second grouped object, and this second grouped object is itself grouped with the two arrowed lines 1111, 1112 to form a third grouped object. The user's task is to modify FIG. 11 a so that it becomes FIG. 11 b. To do this, effectively the user must, in some order of operation:
      • Reposition grouped object 1102 from its original position in FIG. 11 a to a new position in FIG. 11 b;
      • Copy or otherwise reproduce grouped object 1102 to create an accompanying grouped object 1102 a, and position it within box 1101;
      • Introduce a vertically distributed ellipsis 1103 and position it within box 1101—typically, a vertically distributed ellipsis 1103 is either rotated text or itself a fourth grouped object created from three aligned text elements; and
      • Ensure elements 1102, 1102 a, and 1103 are in the end grouped with box 1101 to form the second grouped object, and this second grouped object is itself grouped with the two arrowed lines 1111, 1112 to form a third grouped object.
  • Utilizing the invention, one user interface sensor is used to select the second group level, and the second user interface sensor is used to perform insert, copy, paste, and position operations within this level of grouping without any form or type of ungrouping operation. If the vertically distributed ellipsis 1103 itself is realized as a fourth grouped object created from three aligned text elements, when it is pasted into the drawing via this modality its 1103 grouping is subordinated appropriately (i.e., structured as a peer to grouped objects 1102, 1102 a) within the second grouping level.
  • Although readily implemented using the novel user interface sensors described herein that make it particularly easy to simultaneously adjust a plurality of pairs of parameters, the aspects of the invention illustrated here can also be implemented with a conventional user interface sensor such as a traditional mouse, trackball, or touchpad. In this case, the conventional user interface sensor such as a traditional mouse, trackball, or touchpad must first be used to select the level of grouping and then be used to make the desired modifications within that level of grouping; to make modifications at a different level of grouping, the new level of grouping must be selected in a separate operation, thus adding overhead, as depicted in FIGS. 8 and 9 a-9 b. Although this novel ability to move and modify arbitrary graphic or other objects within groupings may be implemented in this way, having an additional number of widely-adjustable simultaneously interactive parameters—made possible by the main themes of the present invention—is clearly more efficient, as many or all of the overhead operations depicted in FIGS. 8 and 9 a-9 b can be eliminated via usage of the additional widely-adjustable simultaneously interactive parameters.
  •  3.8 3D Graphic Object Placement and Orientation Application
  • CAD and drawing packages that enable 3D object placement and orientation within a 3D space typically extend the capabilities of traditional 2D layout, CAD, and graphics applications as described in Section 3.3 to serve additional geometric needs. As illustrated in FIG. 12, the placement and orientation of 3D object 1200 within a 3D space 1250 (oriented with respect to a reference point 1251) requires that one additional position dimension and two additional orientation angles be specified to complete the full collection of three position dimensions 1201, 1202, 1203 and three orientation angles 1211, 1212, 1213.
  • To interactively adjust these parameters pairwise (or individually with a knob box as has been done historically in some systems) involves complex repetitive passes among high overhead operations as depicted in FIGS. 9 a-9 b, for example among steps 901, 902, 903. The necessity of making many high-overhead operations, for example moving among steps 901, 902, 903, can be functionally disruptive as well as slow and inefficient. The ability to interactively freely adjust the full collection of three position dimensions 1201, 1202, 1203 and three orientation angles 1211, 1212, 1213 is thus of extremely high value.
  • The invention provides for a wide range of mappings between the six position and orientation parameters 1201, 1202, 1203, 1211, 1212, 1213 involved in the placement and orientation of 3D object 1200 within a 3D space, and the large numbers of widely-adjustable and simultaneously interactive parameters facilitated by various realizations of the invention. As one example, a mouse fitted with two trackballs as in FIG. 14 a may be used to specify these six parameters in various ways. One technique is to use the position of mouse body 1400 to control two of the position coordinates (for example 1202, 1203), one of the trackballs (for example 1405 a) to control the orientation angles (1212, 1213) corresponding to these two axes, and the remaining trackball (1405 b) to control the remaining axis (1201) and its corresponding orientation angle (1211). In this example, trackballs 1405 a, 1405 b are configured or used in 2-parameter modalities.
  • In another example, the mouse of FIG. 14 a is fitted with two trackballs, the trackballs may be configured in 3-parameter modalities with one of the trackballs used for controlling three position dimensions 1201, 1202, 1203 and the second trackball configured to correspondingly control the three orientation angles 1211, 1212, 1213. Here the position of mouse body 1400 may be used to control other aspects of drawing operations.
  • In another implementation, a touchpad configured for 4-parameter operation involving two parameters of finger position and two parameters of finger tilt may be combined with a trackball configured for 2-parameter operation. In this example, finger position may be used to control two of the position coordinates (for example 1202, 1203), finger tilt may be used to control the orientation angles (1212, 1213) corresponding to these two axes, and the trackball to control the remaining axis (1201) and its corresponding orientation angle (1211). If the configuration includes a mouse body, its position may be used to control other aspects of drawing operations.
  • In another example, a configuration like that of FIG. 14 d may use left-fingers/thumb trackball 1465 a to control a first position coordinate 1201 and its corresponding orientation angle 1211, the right-fingers/thumb trackball 1465 b to control a second position coordinate 1202 and its corresponding orientation angle 1212, and palm trackball 1465 c to control third position coordinate 1203 and its corresponding orientation angle 1213.
  • The invention further provides for a wide range of additional mappings and geometric metaphors between the user interface sensor geometry and the three position dimensions 1201, 1202, 1203 and three orientation angles 1211, 1212, 1213 of a 3D object.
  •  3.9 Multiple Cursors and Cut and Paste Application
  • The invention additionally provides for a plurality of pairs of user interface sensor parameters to be used to control the respective positions of a corresponding plurality of individual cursors, selections, and/or insertion points. Multiple cursors and associated operations of multiple selection and insertion points can have many applications. Below, a few of these possibilities that would be apparent to one skilled in the art are showcased in a cut-and-paste editing example.
  • Cut, copy, and paste operations using traditional user interface devices usually involve multiple operations to switch between contexts introducing considerable overhead as depicted in FIGS. 9 a-9 b and 8. For instance, FIG. 13 a illustrates a text editing example with text display window 1300 involving the selection of a clause 1320 (highlighted in this example) with the intention of relocating it to a new position 1351. In such an operation with a traditional 2-parameter mouse/trackball/touchpad user interface, the cursor is first used to select clause 1320 and then used to select the insertion position 1351.
  • When writing or editing it is often the case that material needs to be fetched from elsewhere and put in the spot where one is currently writing. Here, the cursor is initially in the spot where the insertion is to occur and the user must then lose the cursor position currently set in this spot to go searching and then to select and cut or copy the material to be pasted; following this the user must then search again, perhaps taking considerable time, for said initial spot and re-establish the cursor location there. Equally often there are other situations where material must be split up and distributed in a number of far-flung places. Here, the cursor is initially in the spot where the material to be sequentially divided and relocated is originally aggregated; the user must repeatedly select the portion of the remaining aggregated material to be relocated and then lose that cursor position to go searching for the new destination insertion spot, perform the insertion, and then search again, perhaps taking considerable time, for the initial spot and re-establish the cursor location there. In both of these cases it would be advantageous if the user could “bookmark” an initial cursor location, search and perform the desired fetch or relocation operations, and readily return without search to the “bookmarked” location. Although this novel and advantageously valuable capability could be realized with a conventional mouse through context redirection operations involving the steps depicted in FIGS. 8 and 9 a-9 b, the present invention provides for a wide range of readily realized and easy-to-use implementations.
  • The invention may be used in a minimal configuration capable of interactively specifying at least two pairs of widely adjustable interactive parameters. Returning to the specific example associated with FIG. 13 a, one pair of parameters is used to set the location of first cursor 1301 which is used in a selection operation, while the second pair of parameters is used to set the location of second cursor 1351 which is to be used to independently set an insertion point. The user may then perform the cut and paste operation with a single mouse click, resulting in the outcome depicted in FIG. 13 b. The relocated text clause 1320 has been transferred to a position determined by the insertion cursor 1351 (here shown to the left of the cursor 1351; it could just as easily be to the right of it), and cursor 1301 used to make the selection remains in position. Either cursor 1301 or 1351 may now be moved and/or used for other cut, copy, paste, or (via the keyboard) new text insertion operations.
  • Although in this example the two cursor locations were close enough to be displayed in the same window 1300, the value of this application of the invention is significantly increased should the two positions be separated by many pages, many tens of pages, or even many hundreds of pages of text. Such situations may be handled by any number of approaches as is clear to one skilled in the art. In one approach involving a single display window, the area comprising the cursor whose corresponding user interface sensor was last manipulated is displayed in the single display window. In another approach involving a single display window, a click event or other user interface stimulus may be used to toggle among the areas comprising the various cursor locations. In yet another approach, at least two windows may be rendered, with one of the cursors displayed and operable within one window and a second cursor displayed and operable in a second window.
  • The invention also provides for these general principles to be applied to other types of objects and applications, such as spreadsheet cells (involving data, formula objects, and cell formats), graphical objects, electronic CAD diagrams (where objects may be connected with formulas, dynamic models, etc.), and others as will be apparent to one skilled in the art.
  •  3.10 Simulation, Processing, and Analysis Applications
  • Simulation, processing, and analysis applications typically involve a large number of parameters which are adjusted to model, affect or investigate the resulting behaviors, end results, and/or implications. Conventional 2-parameter user interface devices such as a mouse/trackball/touchpad require the user to adjust these parameters pairwise (or individually with a knob box as has been done historically in some systems) involving complex repetitive passes among high-overhead operations as depicted in FIGS. 9 a-9 b, for example among steps 901, 902, 903. As in the case of 3D object positioning and orientation, the division among high-overhead operations, for example moving among steps 901, 902, 903, can be functionally disruptive as well as slow and inefficient. The ability to interactively and freely adjust larger collections of parameters simultaneously is thus also of extremely high value.
  •  3.11 Live Signal Processing and Lighting Applications
  • In artistic performance, composition, and recording applications, control of large numbers of parameters requiring simultaneous interactive adjustment is common. Conventional recording, mixing, video, and light control consoles typically have large numbers of controls with carefully designed spatial layouts to facilitate the rapid and precise adjustment of multiple parameters via knobs, sliders, pushbuttons, toggle switches, etc. The introduction of computer GUIs has added considerable value and new capabilities, including “soft” reconfigurable consoles and functional assignments, but in the bargain, typically encumber users—accustomed to rapid and precise operation of multiple parameters—with a 2-parameter mouse/trackball/touchpad having the overhead of iterative context-switching operations depicted in FIGS. 8 and 9 a-9 b. As in the case of 3D object positioning and orientation, the division among high-overhead operations, for example moving among steps 901, 902, 903, can be functionally disruptive as well as slow and inefficient. The ability to interactively freely adjust larger collections of parameters simultaneously is thus also of extremely high value.
  •  3.12 Real-Time Machine Control and Plant Operations
  • Similarly, real-time machine control and plant (manufacturing, chemical, energy, etc.) operations also traditionally involve controlling a significant number of parameters requiring simultaneous interactive adjustment. Conventional real-time machine control and plant operation consoles typically have large numbers of controls with carefully designed spatial layouts to facilitate the rapid and precise adjustment of multiple parameters via knobs, sliders, pushbuttons, toggle switches, etc. The introduction of computer GUIs can add considerable value and new capabilities, including “soft” reconfigurable consoles and functional assignments, but in the bargain typically significantly encumber users—accustomed to rapid and precise operation of multiple parameters—with a 2-parameter mouse/trackball/touchpad having the overhead of iterative context-switching operations depicted in FIGS. 8 and 9 a-9 b. As in the case of 3D object positioning/orientation and artistic applications described earlier, the division among high-overhead operations, for example moving among steps 901, 902, 903, can be functionally disruptive as well as slow and inefficient. The ability to interactively freely adjust larger collections of parameters simultaneously is thus also of extremely high value.
  • A very few examples of this category of application where the invention may be useful include many forms of robotics control, computer-control manufacturing tools, industrial optical and electron microscopy, camera control (pan, tilt, zoom, focus, and/or iris), plant process elements (heaters, pumps, values, stirrers, aerators, actuators, activators, etc.), and a wide range of other related and divergent possibilities apparent to those skilled in the art.
  • 4. Concluding Remarks
  • The present invention at its core provides for a wide range of systems and methods for realizing and applying user interfaces providing, in many cases, at least four widely variable simultaneously interactively adjustable parameters. In so doing, the invention more broadly encompasses novel user interface structures, metaphors, and applications readily suggested and enabled by the core of the invention but which may be indeed realized in ways not involving the core of the invention.
  • While the invention has been described in detail with reference to disclosed embodiments, various modifications within the scope of the invention will be apparent to those of ordinary skill in this technological field. It is to be appreciated that features described with respect to one embodiment typically may be applied to other embodiments. Therefore, the invention properly is to be construed with reference to the claims.

Claims (20)

1. A user interface device for controlling an external device, comprising:
a housing;
a sensor arrangement for detecting a position change of the user interface device on a surface, wherein the sensor arrangement provides a first pair of signals to indicate changes in the position of the housing with respect to the surface on which the user interface device is placed;
a joystick comprising at least three independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least three associated signals forming a plurality of joystick signals;
a communications path to transmit the first pair of parameters and the plurality of joystick signals to an external device.
2. The user interface device of claim 1 wherein the first pair of signals is for at least controlling the location of a cursor displayed on a screen associated with the external device.
3. The user interface device of claim 1 wherein the plurality of joystick signals is for at least controlling an application executing on the external device.
4. The user interface device of claim 1 wherein the joystick comprises at least four independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least four associated signals forming a plurality of joystick signals
5. The user interface device of claim 1 wherein the joystick comprises at least five independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least five associated signals forming a plurality of joystick signals
6. The user interface device of claim 1 wherein the joystick comprises at least six independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least six associated signals forming a plurality of joystick signals
7. The user interface device of claim 1 further comprises a multiplexer configured to multiplex at least the first pair of signals and the plurality of joystick signals into a single multiplexed output signal.
8. The user interface device of claim 7 wherein the multiplexed output signal is transmitted electrically to the external device.
9. The user interface device of claim 7 wherein the multiplexed output signal is transmitted optically to the external device.
10. The user interface device of claim 7 wherein the multiplexed output signal is transmitted by a wireless link to the external device.
11. The user interface device of claim 1, wherein the user interface device further comprises a visual display attached to the housing.
12. The user interface device of claim 1, the user interface device further provides auditory output.
13. The user interface device of claim 1, wherein the user interface device further comprises a preprocessor.
14. A user interface device for controlling an external device, comprising:
a housing;
a trackball;
a sensor arrangement for detecting a position change of the trackball, wherein the sensor arrangement provides a first pair of signals to indicate changes in the position of the trackball;
a joystick comprising at least three independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least three associated signals forming a plurality of joystick signals;
a communications path to transmit the first pair of parameters and the plurality of joystick signals to an external device.
15. The user interface device of claim 1 wherein the first pair of signals is for at least controlling the location of a cursor displayed on a screen associated with the external device.
16. The user interface device of claim 1 wherein the plurality of joystick signals is for at least controlling an application executing on the external device.
17. The user interface device of claim 1 wherein the joystick comprises at least four independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least four associated signals forming a plurality of joystick signals
18. The user interface device of claim 1 wherein the joystick comprises at least five independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least five associated signals forming a plurality of joystick signals
19. The user interface device of claim 1 wherein the joystick comprises at least six independent mechanical adjustments, each independent mechanical adjustment generating an associated signal responsive to the mechanical adjustment, the resulting at least six associated signals forming a plurality of joystick signals
20. The user interface device of claim 1 further comprises a multiplexer configured to multiplex at least the first pair of signals and the plurality of joystick signals into a single multiplexed output signal.
US13/025,129 2004-02-13 2011-02-10 User interface device, such as a mouse or trackball, with a high-dimension joystick providing at least three independently adjustable parameters Abandoned US20110128224A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/025,129 US20110128224A1 (en) 2004-02-13 2011-02-10 User interface device, such as a mouse or trackball, with a high-dimension joystick providing at least three independently adjustable parameters

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US10/779,368 US7620915B2 (en) 2004-02-13 2004-02-13 Electronic document editing employing multiple cursors
US12/618,698 US20100064238A1 (en) 2004-02-13 2009-11-13 Electronic document editing employing multiple cursors
US13/025,129 US20110128224A1 (en) 2004-02-13 2011-02-10 User interface device, such as a mouse or trackball, with a high-dimension joystick providing at least three independently adjustable parameters

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US12/618,698 Continuation US20100064238A1 (en) 2004-02-13 2009-11-13 Electronic document editing employing multiple cursors

Publications (1)

Publication Number Publication Date
US20110128224A1 true US20110128224A1 (en) 2011-06-02

Family

ID=34838368

Family Applications (13)

Application Number Title Priority Date Filing Date
US10/779,368 Active 2025-04-21 US7620915B2 (en) 2004-02-13 2004-02-13 Electronic document editing employing multiple cursors
US10/806,694 Abandoned US20050179663A1 (en) 2004-02-13 2004-03-22 Freely rotating trackball providing additional control parameter modalities
US10/997,650 Active 2025-12-29 US7557797B2 (en) 2004-02-13 2004-11-24 Mouse-based user interface device providing multiple parameters and modalities
US10/997,097 Abandoned US20050179650A1 (en) 2004-02-13 2004-11-24 Extended parameter-set mouse-based user interface device offering offset, warping, and mixed-reference features
US11/008,892 Active 2029-07-27 US8816956B2 (en) 2004-02-13 2004-12-10 Mouse-based user interface device employing user-removable modules
US12/618,698 Abandoned US20100064238A1 (en) 2004-02-13 2009-11-13 Electronic document editing employing multiple cursors
US12/619,678 Abandoned US20100060607A1 (en) 2004-02-13 2009-11-16 User interface mouse with touchpad responsive to gestures and multi-touch
US13/025,129 Abandoned US20110128224A1 (en) 2004-02-13 2011-02-10 User interface device, such as a mouse or trackball, with a high-dimension joystick providing at least three independently adjustable parameters
US13/024,569 Abandoned US20110134039A1 (en) 2004-02-13 2011-02-10 User interface device, such as a mouse, with a plurality of scroll wheels
US14/469,453 Active 2024-07-25 US9417716B2 (en) 2004-02-13 2014-08-26 Mouse-based user interface device employing user-removable modules
US15/209,188 Active 2024-07-07 US11314340B2 (en) 2004-02-13 2016-07-13 User interface device with touch sensor
US17/823,475 Expired - Lifetime US11809643B2 (en) 2004-02-13 2022-08-30 Methods and systems for controlling applications using user interface device with touch sensor
US18/065,246 Expired - Lifetime US11797107B2 (en) 2004-02-13 2022-12-13 Method and user interface device with touch sensor for controlling applications

Family Applications Before (7)

Application Number Title Priority Date Filing Date
US10/779,368 Active 2025-04-21 US7620915B2 (en) 2004-02-13 2004-02-13 Electronic document editing employing multiple cursors
US10/806,694 Abandoned US20050179663A1 (en) 2004-02-13 2004-03-22 Freely rotating trackball providing additional control parameter modalities
US10/997,650 Active 2025-12-29 US7557797B2 (en) 2004-02-13 2004-11-24 Mouse-based user interface device providing multiple parameters and modalities
US10/997,097 Abandoned US20050179650A1 (en) 2004-02-13 2004-11-24 Extended parameter-set mouse-based user interface device offering offset, warping, and mixed-reference features
US11/008,892 Active 2029-07-27 US8816956B2 (en) 2004-02-13 2004-12-10 Mouse-based user interface device employing user-removable modules
US12/618,698 Abandoned US20100064238A1 (en) 2004-02-13 2009-11-13 Electronic document editing employing multiple cursors
US12/619,678 Abandoned US20100060607A1 (en) 2004-02-13 2009-11-16 User interface mouse with touchpad responsive to gestures and multi-touch

Family Applications After (5)

Application Number Title Priority Date Filing Date
US13/024,569 Abandoned US20110134039A1 (en) 2004-02-13 2011-02-10 User interface device, such as a mouse, with a plurality of scroll wheels
US14/469,453 Active 2024-07-25 US9417716B2 (en) 2004-02-13 2014-08-26 Mouse-based user interface device employing user-removable modules
US15/209,188 Active 2024-07-07 US11314340B2 (en) 2004-02-13 2016-07-13 User interface device with touch sensor
US17/823,475 Expired - Lifetime US11809643B2 (en) 2004-02-13 2022-08-30 Methods and systems for controlling applications using user interface device with touch sensor
US18/065,246 Expired - Lifetime US11797107B2 (en) 2004-02-13 2022-12-13 Method and user interface device with touch sensor for controlling applications

Country Status (1)

Country Link
US (13) US7620915B2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9007302B1 (en) 2011-11-11 2015-04-14 Benjamin D. Bandt-Horn Device and user interface for visualizing, navigating, and manipulating hierarchically structured information on host electronic devices
US10000268B1 (en) * 2015-08-20 2018-06-19 Brunswick Corporation Systems and methods for controlling a marine vessel having a joystick with adjustable display
US11010033B2 (en) * 2015-02-19 2021-05-18 Olympus Corporation Display control apparatus and methods for generating and displaying a related-item plate which includes setting items whose functions are related to a designated setting item

Families Citing this family (161)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6610917B2 (en) 1998-05-15 2003-08-26 Lester F. Ludwig Activity indication, external source, and processing loop provisions for driven vibrating-element environments
US9213365B2 (en) 2010-10-01 2015-12-15 Z124 Method and system for viewing stacked screen displays using gestures
US7620915B2 (en) * 2004-02-13 2009-11-17 Ludwig Lester F Electronic document editing employing multiple cursors
US20050225533A1 (en) * 2004-04-13 2005-10-13 Annie Cheng Mouse
JP4789277B2 (en) * 2004-04-22 2011-10-12 横河電機株式会社 Plant operation support device
TWI269206B (en) * 2004-05-05 2006-12-21 Sunplus Technology Co Ltd Setting device for changing the resolution of the track of a multi-step mouse
US20060274037A1 (en) * 2005-06-06 2006-12-07 Stewart Sheila K Onboard pointer system
US20060143571A1 (en) * 2004-12-29 2006-06-29 Wilson Chan Multiple mouse cursors for use within a viewable area for a computer
TW200627248A (en) * 2005-01-20 2006-08-01 Behavior Tech Computer Corp Pointer device with multi-directional window browsing function
TWI267772B (en) * 2005-06-24 2006-12-01 Transpacific Plasma Llc Pointing device
US7719519B2 (en) * 2005-06-24 2010-05-18 Hewlett-Packard Development Company, L.P. Input device which emits and/or reflects optical electromagnetic radiation for use on a display screen
US7433741B2 (en) * 2005-09-30 2008-10-07 Rockwell Automation Technologies, Inc. Hybrid user interface having base presentation information with variably prominent supplemental information
US8018440B2 (en) 2005-12-30 2011-09-13 Microsoft Corporation Unintentional touch rejection
KR100714707B1 (en) * 2006-01-06 2007-05-04 삼성전자주식회사 Apparatus and method for navigation in 3-dimensional graphic user interface
US20090128815A1 (en) * 2006-03-15 2009-05-21 Koninklijke Philips Electronics N.V. Remote control pointing technology with roll detection
US7459624B2 (en) 2006-03-29 2008-12-02 Harmonix Music Systems, Inc. Game controller simulating a musical instrument
TWI315840B (en) * 2006-04-07 2009-10-11 High Tech Comp Corp Intuitive image navigator and electrical equipment with the same
US8510454B2 (en) * 2006-05-04 2013-08-13 Digital River, Inc. Mapped parameter sets using bulk loading system and method
US7720552B1 (en) * 2006-06-05 2010-05-18 Rockwell Automation Technologies, Inc. Virtual knob lever arm as analog control element
EP1887452A1 (en) * 2006-08-09 2008-02-13 STMicroelectronics (Research & Development) Limited Human-Computer Interface
US8310446B1 (en) * 2006-08-25 2012-11-13 Rockwell Collins, Inc. System for integrated coarse and fine graphical object positioning
WO2008039497A2 (en) * 2006-09-25 2008-04-03 Nuvasive, Inc Embroidery using soluble thread
GB2445971A (en) * 2007-01-25 2008-07-30 Jonathan Mark Kelly A musical instrument teaching aid that captures and displays images of a playing area of the instrument
US20080270949A1 (en) * 2007-04-25 2008-10-30 Liang Younger L Methods and Systems for Navigation and Selection of Items within User Interfaces with a Segmented Cursor
US7904835B1 (en) 2007-05-05 2011-03-08 Adobe Systems Incorporated Synchronizing keyboard cursor and mouse cursor movements
US8555186B2 (en) 2007-05-08 2013-10-08 Adobe Systems Incorporated Interactive thumbnails for transferring content among electronic documents
US8063890B2 (en) * 2007-05-15 2011-11-22 Wacom Co., Ltd. Device, method, and computer readable medium for mapping a graphics tablet to an associated display
US20080284726A1 (en) * 2007-05-17 2008-11-20 Marc Boillot System and Method for Sensory Based Media Control
US8439733B2 (en) 2007-06-14 2013-05-14 Harmonix Music Systems, Inc. Systems and methods for reinstating a player within a rhythm-action game
US8678896B2 (en) 2007-06-14 2014-03-25 Harmonix Music Systems, Inc. Systems and methods for asynchronous band interaction in a rhythm action game
US20080309349A1 (en) * 2007-06-15 2008-12-18 Computer Access Technology Corporation Flexible interposer system
US8591584B2 (en) * 2007-11-19 2013-11-26 Nuvasive, Inc. Textile-based plate implant and related methods
KR20090065040A (en) * 2007-12-17 2009-06-22 삼성전자주식회사 Dual pointing device and method based on 3-d motion and touch sensors
US8797271B2 (en) * 2008-02-27 2014-08-05 Microsoft Corporation Input aggregation for a multi-touch device
US9019237B2 (en) 2008-04-06 2015-04-28 Lester F. Ludwig Multitouch parameter and gesture user interface employing an LED-array tactile sensor that can also operate as a display
US8418076B2 (en) * 2008-05-15 2013-04-09 Microsoft Corporation Managing inputs from a plurality of user input device actuators
US8232966B2 (en) * 2008-06-03 2012-07-31 Avago Technologies Ecbu Ip (Singapore) Pte. Ltd. Multiple input optical navigation system
US20090310027A1 (en) * 2008-06-16 2009-12-17 James Fleming Systems and methods for separate audio and video lag calibration in a video game
US8663013B2 (en) * 2008-07-08 2014-03-04 Harmonix Music Systems, Inc. Systems and methods for simulating a rock band experience
US8169414B2 (en) 2008-07-12 2012-05-01 Lim Seung E Control of electronic games via finger angle using a high dimensional touchpad (HDTP) touch user interface
US8345014B2 (en) 2008-07-12 2013-01-01 Lester F. Ludwig Control of the operating system on a computing device via finger angle using a high dimensional touchpad (HDTP) touch user interface
US8604364B2 (en) * 2008-08-15 2013-12-10 Lester F. Ludwig Sensors, algorithms and applications for a high dimensional touchpad
KR101481556B1 (en) * 2008-09-10 2015-01-13 엘지전자 주식회사 A mobile telecommunication terminal and a method of displying an object using the same
KR20100048090A (en) * 2008-10-30 2010-05-11 삼성전자주식회사 Interface apparatus for generating control command by touch and motion, interface system including the interface apparatus, and interface method using the same
US20100205529A1 (en) * 2009-02-09 2010-08-12 Emma Noya Butin Device, system, and method for creating interactive guidance with execution of operations
US9569231B2 (en) * 2009-02-09 2017-02-14 Kryon Systems Ltd. Device, system, and method for providing interactive guidance with execution of operations
US20100223571A1 (en) * 2009-02-27 2010-09-02 Morley Krete Apparatus and method for scrolling pages displayed on a handheld device
US8170346B2 (en) 2009-03-14 2012-05-01 Ludwig Lester F High-performance closed-form single-scan calculation of oblong-shape rotation angles from binary images of arbitrary size using running sums
US8355007B2 (en) * 2009-05-11 2013-01-15 Adobe Systems Incorporated Methods for use with multi-touch displays for determining when a touch is processed as a mouse event
US8465366B2 (en) 2009-05-29 2013-06-18 Harmonix Music Systems, Inc. Biasing a musical performance input to a part
US20100309129A1 (en) * 2009-06-09 2010-12-09 Research In Motion Limited Directional clickable trackball
US9703398B2 (en) * 2009-06-16 2017-07-11 Microsoft Technology Licensing, Llc Pointing device using proximity sensing
US8188969B2 (en) * 2009-06-26 2012-05-29 Panasonic Corporation Dual pointer management method using cooperating input sources and efficient dynamic coordinate remapping
US9098313B2 (en) * 2009-08-24 2015-08-04 Kryon Systems Ltd. Recording display-independent computerized guidance
US8918739B2 (en) * 2009-08-24 2014-12-23 Kryon Systems Ltd. Display-independent recognition of graphical user interface control
US9405558B2 (en) * 2009-08-24 2016-08-02 Kryon Systems Ltd. Display-independent computerized guidance
US20110055722A1 (en) * 2009-09-02 2011-03-03 Ludwig Lester F Data Visualization Environment with DataFlow Processing, Web, Collaboration, Advanced User Interfaces, and Spreadsheet Visualization
US20110066933A1 (en) 2009-09-02 2011-03-17 Ludwig Lester F Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization
CN102023678A (en) * 2009-09-21 2011-04-20 鸿富锦精密工业(深圳)有限公司 Electronic equipment with cursor control device
EP2480957B1 (en) 2009-09-22 2017-08-09 Apple Inc. Device, method, and graphical user interface for manipulating user interface objects
US8799826B2 (en) 2009-09-25 2014-08-05 Apple Inc. Device, method, and graphical user interface for moving a calendar entry in a calendar application
US8780069B2 (en) 2009-09-25 2014-07-15 Apple Inc. Device, method, and graphical user interface for manipulating user interface objects
US8832585B2 (en) 2009-09-25 2014-09-09 Apple Inc. Device, method, and graphical user interface for manipulating workspace views
US8766928B2 (en) * 2009-09-25 2014-07-01 Apple Inc. Device, method, and graphical user interface for manipulating user interface objects
US8816991B2 (en) * 2009-10-02 2014-08-26 Dedo Interactive, Inc. Touch input apparatus including image projection
US8702485B2 (en) 2010-06-11 2014-04-22 Harmonix Music Systems, Inc. Dance game and tutorial
EP2494432B1 (en) 2009-10-27 2019-05-29 Harmonix Music Systems, Inc. Gesture-based user interface
US9981193B2 (en) 2009-10-27 2018-05-29 Harmonix Music Systems, Inc. Movement based recognition and evaluation
US8614664B2 (en) * 2009-11-09 2013-12-24 Primax Electronics Ltd. Multi-touch multi-dimensional mouse
US8612884B2 (en) 2010-01-26 2013-12-17 Apple Inc. Device, method, and graphical user interface for resizing objects
US8539386B2 (en) 2010-01-26 2013-09-17 Apple Inc. Device, method, and graphical user interface for selecting and moving objects
US8539385B2 (en) 2010-01-26 2013-09-17 Apple Inc. Device, method, and graphical user interface for precise positioning of objects
US9411504B2 (en) 2010-01-28 2016-08-09 Microsoft Technology Licensing, Llc Copy and staple gestures
US8261213B2 (en) 2010-01-28 2012-09-04 Microsoft Corporation Brush, carbon-copy, and fill gestures
US9519356B2 (en) 2010-02-04 2016-12-13 Microsoft Technology Licensing, Llc Link gestures
US20110202934A1 (en) * 2010-02-12 2011-08-18 Ludwig Lester F Window manger input focus control for high dimensional touchpad (htpd), advanced mice, and other multidimensional user interfaces
US9965165B2 (en) 2010-02-19 2018-05-08 Microsoft Technology Licensing, Llc Multi-finger gestures
US9310994B2 (en) * 2010-02-19 2016-04-12 Microsoft Technology Licensing, Llc Use of bezel as an input mechanism
US9367205B2 (en) * 2010-02-19 2016-06-14 Microsoft Technolgoy Licensing, Llc Radial menus with bezel gestures
US9454304B2 (en) 2010-02-25 2016-09-27 Microsoft Technology Licensing, Llc Multi-screen dual tap gesture
US20110214088A1 (en) * 2010-02-26 2011-09-01 Research In Motion Limited Automatic scrolling of electronic messages
US10146427B2 (en) * 2010-03-01 2018-12-04 Nri R&D Patent Licensing, Llc Curve-fitting approach to high definition touch pad (HDTP) parameter extraction
US8874243B2 (en) 2010-03-16 2014-10-28 Harmonix Music Systems, Inc. Simulating musical instruments
US8562403B2 (en) 2010-06-11 2013-10-22 Harmonix Music Systems, Inc. Prompting a player of a dance game
US9358456B1 (en) 2010-06-11 2016-06-07 Harmonix Music Systems, Inc. Dance competition game
US9632344B2 (en) 2010-07-09 2017-04-25 Lester F. Ludwig Use of LED or OLED array to implement integrated combinations of touch screen tactile, touch gesture sensor, color image display, hand-image gesture sensor, document scanner, secure optical data exchange, and fingerprint processing capabilities
US9626023B2 (en) 2010-07-09 2017-04-18 Lester F. Ludwig LED/OLED array approach to integrated display, lensless-camera, and touch-screen user interface devices and associated processors
US8754862B2 (en) * 2010-07-11 2014-06-17 Lester F. Ludwig Sequential classification recognition of gesture primitives and window-based parameter smoothing for high dimensional touchpad (HDTP) user interfaces
US9098182B2 (en) 2010-07-30 2015-08-04 Apple Inc. Device, method, and graphical user interface for copying user interface objects between content regions
US8972879B2 (en) 2010-07-30 2015-03-03 Apple Inc. Device, method, and graphical user interface for reordering the front-to-back positions of objects
US9081494B2 (en) 2010-07-30 2015-07-14 Apple Inc. Device, method, and graphical user interface for copying formatting attributes
US9950256B2 (en) 2010-08-05 2018-04-24 Nri R&D Patent Licensing, Llc High-dimensional touchpad game controller with multiple usage and networking modalities
JP5053428B2 (en) * 2010-08-18 2012-10-17 ギガ−バイト テクノロジー カンパニー リミテッド Mouse and touch input coupling device
US20120054667A1 (en) * 2010-08-31 2012-03-01 Blackboard Inc. Separate and simultaneous control of windows in windowing systems
US20130091437A1 (en) * 2010-09-03 2013-04-11 Lester F. Ludwig Interactive data visulization utilizing hdtp touchpad hdtp touchscreens, advanced multitouch, or advanced mice
US9024166B2 (en) 2010-09-09 2015-05-05 Harmonix Music Systems, Inc. Preventing subtractive track separation
WO2012044717A1 (en) 2010-10-01 2012-04-05 Imerj LLC Drag move gesture in user interface
US20120146923A1 (en) * 2010-10-07 2012-06-14 Basir Mossab O Touch screen device
US20120092255A1 (en) * 2010-10-19 2012-04-19 Giga-Byte Technology Co., Ltd. Assembled type mouse and replaceable input component thereof
TWI452494B (en) * 2010-10-28 2014-09-11 Primax Electronics Ltd Method for combining at least two touch signals in a computer system
KR101784317B1 (en) * 2010-11-19 2017-10-11 삼성전자주식회사 Method and apparatus for operating function of e-book in a mobile device
KR101558200B1 (en) 2010-12-06 2015-10-08 한국전자통신연구원 Apparatus and method for controlling idle of vehicle
KR102113439B1 (en) 2010-12-31 2020-06-02 이베이 인크. Methods and systems for displaying content on multiple networked devices with a simple command
US20120204577A1 (en) 2011-02-16 2012-08-16 Ludwig Lester F Flexible modular hierarchical adaptively controlled electronic-system cooling and energy harvesting for IC chip packaging, printed circuit boards, subsystems, cages, racks, IT rooms, and data centers using quantum and classical thermoelectric materials
US9442652B2 (en) 2011-03-07 2016-09-13 Lester F. Ludwig General user interface gesture lexicon and grammar frameworks for multi-touch, high dimensional touch pad (HDTP), free-space camera, and other user interfaces
US9075497B1 (en) * 2011-03-15 2015-07-07 Symantec Corporation Enabling selective policy driven seamless user interface presentation between and among a host and a plurality of guests
US8863019B2 (en) * 2011-03-29 2014-10-14 International Business Machines Corporation Modifying numeric data presentation on a display
US9052772B2 (en) 2011-08-10 2015-06-09 Lester F. Ludwig Heuristics for 3D and 6D touch gesture touch parameter calculations for high-dimensional touch parameter (HDTP) user interfaces
US9720583B2 (en) * 2011-09-22 2017-08-01 Microsoft Technology Licensing, Llc User interface for editing a value in place
CN103019548A (en) * 2011-09-22 2013-04-03 致伸科技股份有限公司 Computer peripheral device and method for controlling different window scroll bars
US20130076654A1 (en) 2011-09-27 2013-03-28 Imerj LLC Handset states and state diagrams: open, closed transitional and easel
US9367230B2 (en) * 2011-11-08 2016-06-14 Microsoft Technology Licensing, Llc Interaction models for indirect interaction devices
TWI493387B (en) * 2011-11-18 2015-07-21 Primax Electronics Ltd Multi-touch mouse
US10430066B2 (en) 2011-12-06 2019-10-01 Nri R&D Patent Licensing, Llc Gesteme (gesture primitive) recognition for advanced touch user interfaces
US9823781B2 (en) 2011-12-06 2017-11-21 Nri R&D Patent Licensing, Llc Heterogeneous tactile sensing via multiple sensor types
US10423328B2 (en) * 2011-12-28 2019-09-24 Hiroyuki Ikeda Portable terminal for controlling two cursors within a virtual keyboard according to setting of movement by a single key at a time or a plurality of keys at a time
US9519419B2 (en) 2012-01-17 2016-12-13 Microsoft Technology Licensing, Llc Skinnable touch device grip patterns
GB2500566A (en) * 2012-01-31 2013-10-02 Avolites Ltd Automated lighting control system allowing three dimensional control and user interface gesture recognition
TW201337649A (en) * 2012-03-02 2013-09-16 Pixart Imaging Inc Optical input device and input detection method thereof
TWI450159B (en) * 2012-03-02 2014-08-21 Pixart Imaging Inc Optical touch device, passive touch system and its input detection method
CN103309353B (en) * 2012-03-12 2016-03-30 联想(北京)有限公司 A kind of robot long-distance control method and communication terminal
JP2013210906A (en) * 2012-03-30 2013-10-10 Sony Corp Control method, control device and program
JP6071107B2 (en) 2012-06-14 2017-02-01 裕行 池田 Mobile device
US9858244B1 (en) 2012-06-27 2018-01-02 Amazon Technologies, Inc. Sampling a part of a content item
US9582122B2 (en) 2012-11-12 2017-02-28 Microsoft Technology Licensing, Llc Touch-sensitive bezel techniques
US10220303B1 (en) 2013-03-15 2019-03-05 Harmonix Music Systems, Inc. Gesture-based music game
EP2830039B1 (en) * 2013-07-24 2018-10-03 Native Instruments GmbH Method, arrangement, computer program and computer-readable storage means for controlling at least one parameter or at least one object using capacity sensing input elements
US9977772B1 (en) * 2013-09-24 2018-05-22 Mehrdad Samadani Method and system for cellular text processor and display
US20150121314A1 (en) * 2013-10-24 2015-04-30 Jens Bombolowsky Two-finger gestures
US9589749B1 (en) * 2013-12-14 2017-03-07 Qiankun Wu Finger activated switching device
US9715286B2 (en) 2014-01-28 2017-07-25 Solid Art Labs, Inc. Hand-controllable signal-generating devices and systems
US9477337B2 (en) 2014-03-14 2016-10-25 Microsoft Technology Licensing, Llc Conductive trace routing for display and bezel sensors
US9542027B2 (en) * 2014-04-16 2017-01-10 At&T Intellectual Property I, L.P. Pressure-based input method for user devices
CN103970307A (en) * 2014-05-30 2014-08-06 陈亮 Mouse based on induction keys
CN108334268B (en) * 2014-07-30 2020-05-08 Oppo广东移动通信有限公司 Method and device for controlling rotation of camera and mobile terminal
USD788813S1 (en) * 2014-11-14 2017-06-06 Microsoft Corporation Display screen with icon
US9996232B2 (en) 2014-12-15 2018-06-12 International Business Machines Corporation Creating multiple cursors for duplicated entry
WO2017007422A1 (en) * 2015-07-09 2017-01-12 Razer (Asia-Pacific) Pte. Ltd. Input devices
US9778759B2 (en) 2015-08-21 2017-10-03 Kashif Huda User interface assembly
CN105808015B (en) * 2016-05-27 2018-10-12 京东方科技集团股份有限公司 Peep-proof user interaction means and method
US20180164996A1 (en) * 2016-12-12 2018-06-14 Logitech Europe S.A. Contextually-based functional assignment for a user-manipulable element on an input device
EP3340536B1 (en) * 2016-12-20 2019-05-22 Axis AB Controlling different states of operation of an electronic device over a communication network using a control device
WO2018136028A1 (en) * 2017-01-17 2018-07-26 Hewlett-Packard Development Company, L.P. Model controller
CN106874013B (en) * 2017-03-01 2020-10-23 网易(杭州)网络有限公司 Interface display method and device
CN108628469B (en) * 2017-03-24 2020-08-14 致伸科技股份有限公司 Mouse (Saggar)
KR102325329B1 (en) * 2017-04-20 2021-11-10 엘지디스플레이 주식회사 Touch Device And Method Of Driving The Same
US11099658B2 (en) 2017-04-21 2021-08-24 Hewlett-Packard Development Company, L.P. Determining orientation of a trackball
JP2018190268A (en) * 2017-05-10 2018-11-29 富士フイルム株式会社 Touch type operation device, operation method thereof, and operation program
CN109284015A (en) * 2017-07-21 2019-01-29 致伸科技股份有限公司 Mouse apparatus
BE1025596A9 (en) * 2017-09-29 2019-05-06 Inventrans Bvba METHOD AND DEVICE AND SYSTEM FOR PROVIDING DOUBLE MOUSE SUPPORT
WO2019074480A1 (en) 2017-10-09 2019-04-18 Hewlett-Packard Development Company, L.P. Removable button members for electronic pointing devices
WO2019083495A1 (en) 2017-10-23 2019-05-02 Hewlett-Packard Development Company, L.P. Input device with precision control
CN109901776B (en) * 2017-12-08 2022-05-06 珠海金山办公软件有限公司 Method and device for adjusting object attribute
USD859433S1 (en) * 2018-01-03 2019-09-10 Samsung Electronics Co., Ltd. Display screen or portion thereof with graphical user interface
USD923042S1 (en) * 2018-07-04 2021-06-22 The Fourth Paradigm (Beijing) Tech Co Ltd Display screen with transitional graphical user interface
CN108818552A (en) * 2018-07-05 2018-11-16 安徽横渡信息技术有限公司 A kind of toy for children robot
CN109491565B (en) * 2018-10-30 2020-10-02 武汉思普崚技术有限公司 Method and equipment for displaying component information of object in three-dimensional scene
US11656695B2 (en) * 2018-12-31 2023-05-23 Oahwip B.V. Computer mouse for different modes of use
US11086413B2 (en) * 2019-02-26 2021-08-10 Yong Wang Context-dependent wheels and buttons
US11003255B2 (en) * 2019-02-26 2021-05-11 Yong Wang Context-dependent touchbands
WO2021011571A1 (en) 2019-07-15 2021-01-21 Corindus, Inc. Systems and methods for a control station for robotic interventional procedures using a plurality of elongated medical devices

Citations (81)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3974493A (en) * 1974-04-29 1976-08-10 Vydec, Inc. Cursor find system for the display of a word processing system
US4988981A (en) * 1987-03-17 1991-01-29 Vpl Research, Inc. Computer data entry and manipulation apparatus and method
US5049863A (en) * 1989-02-10 1991-09-17 Kabushiki Kaisha Toshiba Cursor key unit for a computer having a mouse function removably mounted on a keyboard section of a base
US5095303A (en) * 1990-03-27 1992-03-10 Apple Computer, Inc. Six degree of freedom graphic object controller
US5327161A (en) * 1989-08-09 1994-07-05 Microtouch Systems, Inc. System and method for emulating a mouse input device with a touchpad input device
US5490039A (en) * 1993-11-24 1996-02-06 Dell Usa, L.P. Integrated mouse tray and mouse for portable computer having cavity therefore
US5565891A (en) * 1992-03-05 1996-10-15 Armstrong; Brad A. Six degrees of freedom graphics controller
US5586243A (en) * 1994-04-15 1996-12-17 International Business Machines Corporation Multiple display pointers for computer graphical user interfaces
US5615083A (en) * 1995-12-11 1997-03-25 Gateway 2000, Inc. Detachable joystick for a portable computer
US5666499A (en) * 1995-08-04 1997-09-09 Silicon Graphics, Inc. Clickaround tool-based graphical interface with two cursors
US5670990A (en) * 1991-03-19 1997-09-23 Logitech Incorporated Integral ball cage for pointing device
US5726684A (en) * 1996-07-26 1998-03-10 Ncr Corporation Detachable convertible mouse-trackball pointing device for use with a computer
US5805144A (en) * 1994-12-14 1998-09-08 Dell Usa, L.P. Mouse pointing device having integrated touchpad
US5852442A (en) * 1996-01-12 1998-12-22 Fujitsu Limited Method of drawing a three-dimensional object
US5881366A (en) * 1996-05-01 1999-03-09 Logitech, Inc. Wireless peripheral interface
US5917472A (en) * 1996-05-29 1999-06-29 International Computers Limited Cursor control system with multiple pointing devices
US5999169A (en) * 1996-08-30 1999-12-07 International Business Machines Corporation Computer graphical user interface method and system for supporting multiple two-dimensional movement inputs
US6057830A (en) * 1997-01-17 2000-05-02 Tritech Microelectronics International Ltd. Touchpad mouse controller
US6061051A (en) * 1997-01-17 2000-05-09 Tritech Microelectronics Command set for touchpad pen-input mouse
US6128006A (en) * 1998-03-26 2000-10-03 Immersion Corporation Force feedback mouse wheel and other control wheels
US6204837B1 (en) * 1998-07-13 2001-03-20 Hewlett-Packard Company Computing apparatus having multiple pointing devices
US6205021B1 (en) * 1998-02-26 2001-03-20 Micron Electronics, Inc. Method for operating an input device and a laptop computer
US6219037B1 (en) * 1997-10-02 2001-04-17 Samsung Electronics Co., Ltd. Pointing device provided with two types of input means for a computer system
US6232958B1 (en) * 1998-12-29 2001-05-15 Microsoft Corporation Input device with multiplexed switches
US6239803B1 (en) * 1999-04-14 2001-05-29 Stanley W. Driskell Method to achieve least effort selection from an item list of arbitrary length
US6239790B1 (en) * 1996-08-05 2001-05-29 Interlink Electronics Force sensing semiconductive touchpad
US6281881B1 (en) * 1996-01-02 2001-08-28 Microsoft Corporation System and method of adjusting display characteristics of a displayable data file using an ergonomic computer input device
US6295051B1 (en) * 1999-06-02 2001-09-25 International Business Machines Corporation Intelligent boundless computer mouse system
US20010033268A1 (en) * 2000-02-29 2001-10-25 Jiang Jiong John Handheld ergonomic mouse
US6323846B1 (en) * 1998-01-26 2001-11-27 University Of Delaware Method and apparatus for integrating manual input
US6329978B1 (en) * 1997-08-11 2001-12-11 Fu-Kuo Yeh Cursor control device
US20020005108A1 (en) * 1998-05-15 2002-01-17 Ludwig Lester Frank Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US6396477B1 (en) * 1998-09-14 2002-05-28 Microsoft Corp. Method of interacting with a computer using a proximity sensor in a computer input device
US20020080112A1 (en) * 2000-09-28 2002-06-27 Braun Adam C. Directional tactile feedback for haptic feedback interface devices
US20020084981A1 (en) * 2000-11-14 2002-07-04 Flack James F. Cursor navigation system and method for a display
US6424335B1 (en) * 1998-09-02 2002-07-23 Fujitsu Limited Notebook computer with detachable infrared multi-mode input device
US20020113776A1 (en) * 2001-02-21 2002-08-22 Ran Meriaz Trackball mouse
US6456275B1 (en) * 1998-09-14 2002-09-24 Microsoft Corporation Proximity sensor in a computer input device
US6489948B1 (en) * 2000-04-20 2002-12-03 Benny Chi Wah Lau Computer mouse having multiple cursor positioning inputs and method of operation
US20020190930A1 (en) * 2001-06-19 2002-12-19 Fujitsu Hitachi Plasma Display Limited Method of driving plasma display panel
US20030006961A1 (en) * 2001-07-09 2003-01-09 Yuly Shipilevsky Method and system for increasing computer operator's productivity
US20030030621A1 (en) * 1993-07-16 2003-02-13 Rosenberg Louis B. Force feeback device including flexure member between actuator and user object
US6525713B1 (en) * 1999-05-10 2003-02-25 Alps Electric Co., Ltd. Coordinate input device capable of inputting z-coordinate of image object
US20030063062A1 (en) * 2001-09-28 2003-04-03 Makoto Tsumura Image display device
US6556150B1 (en) * 2000-03-24 2003-04-29 Microsoft Corporation Ergonomic computer input device
US6570557B1 (en) * 2001-02-10 2003-05-27 Finger Works, Inc. Multi-touch system and method for emulating modifier keys via fingertip chords
US20030107552A1 (en) * 2001-12-11 2003-06-12 Koninklijke Philips Electronics N.V. Computer mouse with dual functionality
US20030107603A1 (en) * 2001-12-12 2003-06-12 Intel Corporation Scroll notification system and method
US6580420B1 (en) * 2000-03-15 2003-06-17 Yanqing Wang Convertible computer input device
US6587091B2 (en) * 2001-04-23 2003-07-01 Michael Lawrence Serpa Stabilized tactile output mechanism for computer interface devices
US6590564B1 (en) * 1999-12-17 2003-07-08 Microsoft Corporation Ergonomic computer input device
US20030169216A1 (en) * 2002-03-06 2003-09-11 Lg Electronics Inc. Method and apparatus for driving plasma display panel
US6623194B1 (en) * 1999-10-27 2003-09-23 Chung Ching Lip Position encoder system
US6646632B2 (en) * 2000-12-01 2003-11-11 Logitech Europe S.A. Tactile force feedback device
US6677932B1 (en) * 2001-01-28 2004-01-13 Finger Works, Inc. System and method for recognizing touch typing under limited tactile feedback conditions
US20040017358A1 (en) * 2002-07-19 2004-01-29 Kuo Hsiao Ming Housing assembly for computer mouse
US6704003B2 (en) * 2000-12-15 2004-03-09 Logitech Europe S.A. Adaptable input device support
US6714221B1 (en) * 2000-08-03 2004-03-30 Apple Computer, Inc. Depicting and setting scroll amount
US20040126171A1 (en) * 2001-04-30 2004-07-01 Microsoft Corporation Keyboard with improved lateral region
US20040147318A1 (en) * 1999-09-30 2004-07-29 Shahoian Erik J. Increasing force transmissibility for tactile feedback interface devices
US20040155865A1 (en) * 2002-12-16 2004-08-12 Swiader Michael C Ergonomic data input and cursor control device
US20040189605A1 (en) * 2003-03-25 2004-09-30 Ken Shih Mouse with multi-axis inputting device
US6909422B2 (en) * 2000-02-07 2005-06-21 Matsushita Electric Industrial Co., Ltd. Track ball device and electronic apparatus using the same
US20050179652A1 (en) * 2004-02-13 2005-08-18 Ludwig Lester F. Mouse-based user interface device employing user-removable modules
US20050193321A1 (en) * 2000-11-10 2005-09-01 Microsoft Corporation Insertion point bungee space tool
US20050275637A1 (en) * 1998-09-14 2005-12-15 Microsoft Corporation Method for displaying information responsive to sensing a physical presence proximate to a computer input device
US20060125803A1 (en) * 2001-02-10 2006-06-15 Wayne Westerman System and method for packing multitouch gestures onto a hand
US7168047B1 (en) * 2002-05-28 2007-01-23 Apple Computer, Inc. Mouse having a button-less panning and scrolling switch
US7358956B2 (en) * 1998-09-14 2008-04-15 Microsoft Corporation Method for providing feedback responsive to sensing a physical presence proximate to a control of an electronic device
US20080088602A1 (en) * 2005-03-04 2008-04-17 Apple Inc. Multi-functional hand-held device
US20080163130A1 (en) * 2007-01-03 2008-07-03 Apple Inc Gesture learning
US20080158169A1 (en) * 2007-01-03 2008-07-03 Apple Computer, Inc. Noise detection in multi-touch sensors
US20080168364A1 (en) * 2007-01-05 2008-07-10 Apple Computer, Inc. Adaptive acceleration of mouse cursor
US20080165140A1 (en) * 2007-01-05 2008-07-10 Apple Inc. Detecting gestures on multi-event sensitive devices
US20080165141A1 (en) * 2007-01-05 2008-07-10 Apple Inc. Gestures for controlling, manipulating, and editing of media files using touch sensitive devices
US20080204426A1 (en) * 2004-07-30 2008-08-28 Apple Inc. Gestures for touch sensitive input devices
US7463239B2 (en) * 2001-04-30 2008-12-09 Microsoft Corporation Input device including a wheel assembly for scrolling an image in multiple directions
US20080309632A1 (en) * 2007-06-13 2008-12-18 Apple Inc. Pinch-throw and translation gestures
US20100156818A1 (en) * 2008-12-23 2010-06-24 Apple Inc. Multi touch with multi haptics
US20100214250A1 (en) * 2001-05-16 2010-08-26 Synaptics Incorporated Touch screen with user interface enhancement
US7808479B1 (en) * 2003-09-02 2010-10-05 Apple Inc. Ambidextrous mouse

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5880411A (en) 1992-06-08 1999-03-09 Synaptics, Incorporated Object position detector with edge motion feature and gesture recognition
US6392634B1 (en) * 1993-07-08 2002-05-21 Dell Usa, L.P. Portable computer having reversible trackball/mouse device
US6756965B2 (en) * 1994-03-18 2004-06-29 International Business Machines Corporation Input device having two joysticks and touchpad with default template
US5708460A (en) * 1995-06-02 1998-01-13 Avi Systems, Inc. Touch screen
US6803905B1 (en) * 1997-05-30 2004-10-12 International Business Machines Corporation Touch sensitive apparatus and method for improved visual feedback
US6243080B1 (en) * 1998-07-14 2001-06-05 Ericsson Inc. Touch-sensitive panel with selector
US6078100A (en) * 1999-01-13 2000-06-20 Micron Technology, Inc. Utilization of die repattern layers for die internal connections
US6587093B1 (en) * 1999-11-04 2003-07-01 Synaptics Incorporated Capacitive mouse
US7006077B1 (en) * 1999-11-30 2006-02-28 Nokia Mobile Phones, Ltd. Electronic device having touch sensitive slide
EP1184414A3 (en) * 2000-08-30 2003-08-06 JSR Corporation Conjugated diene-based rubber and method of producing the same, oil extended rubber and rubber composition containing the same
JP4689812B2 (en) * 2000-11-17 2011-05-25 富士通コンポーネント株式会社 Wireless mouse
TW521205B (en) * 2001-06-05 2003-02-21 Compal Electronics Inc Touch screen capable of controlling amplification with pressure
US6999066B2 (en) * 2002-06-24 2006-02-14 Xerox Corporation System for audible feedback for touch screen displays
US20040041787A1 (en) * 2002-08-28 2004-03-04 Graves Robert A. Method and apparatus for a hybrid pointing device used with a data processing system
DE10262063A1 (en) * 2002-12-20 2004-09-16 Hewlett-Packard Co.(A Delaware Corporation), Palo Alto Interface device with a generator of electrical energy
US8832585B2 (en) * 2009-09-25 2014-09-09 Apple Inc. Device, method, and graphical user interface for manipulating workspace views
US9268457B2 (en) * 2012-07-13 2016-02-23 Google Inc. Touch-based fluid window management
US9851842B1 (en) * 2016-08-10 2017-12-26 Rovi Guides, Inc. Systems and methods for adjusting display characteristics

Patent Citations (97)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3974493A (en) * 1974-04-29 1976-08-10 Vydec, Inc. Cursor find system for the display of a word processing system
US4988981B1 (en) * 1987-03-17 1999-05-18 Vpl Newco Inc Computer data entry and manipulation apparatus and method
US4988981A (en) * 1987-03-17 1991-01-29 Vpl Research, Inc. Computer data entry and manipulation apparatus and method
US5049863A (en) * 1989-02-10 1991-09-17 Kabushiki Kaisha Toshiba Cursor key unit for a computer having a mouse function removably mounted on a keyboard section of a base
US5327161A (en) * 1989-08-09 1994-07-05 Microtouch Systems, Inc. System and method for emulating a mouse input device with a touchpad input device
US5095303A (en) * 1990-03-27 1992-03-10 Apple Computer, Inc. Six degree of freedom graphic object controller
US5670990A (en) * 1991-03-19 1997-09-23 Logitech Incorporated Integral ball cage for pointing device
US5565891A (en) * 1992-03-05 1996-10-15 Armstrong; Brad A. Six degrees of freedom graphics controller
US20030030621A1 (en) * 1993-07-16 2003-02-13 Rosenberg Louis B. Force feeback device including flexure member between actuator and user object
US5490039A (en) * 1993-11-24 1996-02-06 Dell Usa, L.P. Integrated mouse tray and mouse for portable computer having cavity therefore
US5586243A (en) * 1994-04-15 1996-12-17 International Business Machines Corporation Multiple display pointers for computer graphical user interfaces
US5805144A (en) * 1994-12-14 1998-09-08 Dell Usa, L.P. Mouse pointing device having integrated touchpad
US5666499A (en) * 1995-08-04 1997-09-09 Silicon Graphics, Inc. Clickaround tool-based graphical interface with two cursors
US5615083A (en) * 1995-12-11 1997-03-25 Gateway 2000, Inc. Detachable joystick for a portable computer
US6281881B1 (en) * 1996-01-02 2001-08-28 Microsoft Corporation System and method of adjusting display characteristics of a displayable data file using an ergonomic computer input device
US5852442A (en) * 1996-01-12 1998-12-22 Fujitsu Limited Method of drawing a three-dimensional object
US5881366A (en) * 1996-05-01 1999-03-09 Logitech, Inc. Wireless peripheral interface
US5917472A (en) * 1996-05-29 1999-06-29 International Computers Limited Cursor control system with multiple pointing devices
US5726684A (en) * 1996-07-26 1998-03-10 Ncr Corporation Detachable convertible mouse-trackball pointing device for use with a computer
US6239790B1 (en) * 1996-08-05 2001-05-29 Interlink Electronics Force sensing semiconductive touchpad
US5999169A (en) * 1996-08-30 1999-12-07 International Business Machines Corporation Computer graphical user interface method and system for supporting multiple two-dimensional movement inputs
US6057830A (en) * 1997-01-17 2000-05-02 Tritech Microelectronics International Ltd. Touchpad mouse controller
US6061051A (en) * 1997-01-17 2000-05-09 Tritech Microelectronics Command set for touchpad pen-input mouse
US6329978B1 (en) * 1997-08-11 2001-12-11 Fu-Kuo Yeh Cursor control device
US6219037B1 (en) * 1997-10-02 2001-04-17 Samsung Electronics Co., Ltd. Pointing device provided with two types of input means for a computer system
US6323846B1 (en) * 1998-01-26 2001-11-27 University Of Delaware Method and apparatus for integrating manual input
US20080128182A1 (en) * 1998-01-26 2008-06-05 Apple Inc. Sensor arrangement for use with a touch sensor
US20020015024A1 (en) * 1998-01-26 2002-02-07 University Of Delaware Method and apparatus for integrating manual input
US6888536B2 (en) * 1998-01-26 2005-05-03 The University Of Delaware Method and apparatus for integrating manual input
US6205021B1 (en) * 1998-02-26 2001-03-20 Micron Electronics, Inc. Method for operating an input device and a laptop computer
US6128006A (en) * 1998-03-26 2000-10-03 Immersion Corporation Force feedback mouse wheel and other control wheels
US6570078B2 (en) * 1998-05-15 2003-05-27 Lester Frank Ludwig Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US20020005108A1 (en) * 1998-05-15 2002-01-17 Ludwig Lester Frank Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US6204837B1 (en) * 1998-07-13 2001-03-20 Hewlett-Packard Company Computing apparatus having multiple pointing devices
US6424335B1 (en) * 1998-09-02 2002-07-23 Fujitsu Limited Notebook computer with detachable infrared multi-mode input device
US6396477B1 (en) * 1998-09-14 2002-05-28 Microsoft Corp. Method of interacting with a computer using a proximity sensor in a computer input device
US20050275637A1 (en) * 1998-09-14 2005-12-15 Microsoft Corporation Method for displaying information responsive to sensing a physical presence proximate to a computer input device
US7358956B2 (en) * 1998-09-14 2008-04-15 Microsoft Corporation Method for providing feedback responsive to sensing a physical presence proximate to a control of an electronic device
US6456275B1 (en) * 1998-09-14 2002-09-24 Microsoft Corporation Proximity sensor in a computer input device
US7256770B2 (en) * 1998-09-14 2007-08-14 Microsoft Corporation Method for displaying information responsive to sensing a physical presence proximate to a computer input device
US6232958B1 (en) * 1998-12-29 2001-05-15 Microsoft Corporation Input device with multiplexed switches
US6239803B1 (en) * 1999-04-14 2001-05-29 Stanley W. Driskell Method to achieve least effort selection from an item list of arbitrary length
US6525713B1 (en) * 1999-05-10 2003-02-25 Alps Electric Co., Ltd. Coordinate input device capable of inputting z-coordinate of image object
US6295051B1 (en) * 1999-06-02 2001-09-25 International Business Machines Corporation Intelligent boundless computer mouse system
US20040147318A1 (en) * 1999-09-30 2004-07-29 Shahoian Erik J. Increasing force transmissibility for tactile feedback interface devices
US6623194B1 (en) * 1999-10-27 2003-09-23 Chung Ching Lip Position encoder system
US6590564B1 (en) * 1999-12-17 2003-07-08 Microsoft Corporation Ergonomic computer input device
US6909422B2 (en) * 2000-02-07 2005-06-21 Matsushita Electric Industrial Co., Ltd. Track ball device and electronic apparatus using the same
US20010033268A1 (en) * 2000-02-29 2001-10-25 Jiang Jiong John Handheld ergonomic mouse
US6580420B1 (en) * 2000-03-15 2003-06-17 Yanqing Wang Convertible computer input device
US6556150B1 (en) * 2000-03-24 2003-04-29 Microsoft Corporation Ergonomic computer input device
US6489948B1 (en) * 2000-04-20 2002-12-03 Benny Chi Wah Lau Computer mouse having multiple cursor positioning inputs and method of operation
US6714221B1 (en) * 2000-08-03 2004-03-30 Apple Computer, Inc. Depicting and setting scroll amount
US20020080112A1 (en) * 2000-09-28 2002-06-27 Braun Adam C. Directional tactile feedback for haptic feedback interface devices
US20050193321A1 (en) * 2000-11-10 2005-09-01 Microsoft Corporation Insertion point bungee space tool
US20020084981A1 (en) * 2000-11-14 2002-07-04 Flack James F. Cursor navigation system and method for a display
US6646632B2 (en) * 2000-12-01 2003-11-11 Logitech Europe S.A. Tactile force feedback device
US6704003B2 (en) * 2000-12-15 2004-03-09 Logitech Europe S.A. Adaptable input device support
US6677932B1 (en) * 2001-01-28 2004-01-13 Finger Works, Inc. System and method for recognizing touch typing under limited tactile feedback conditions
US20060125803A1 (en) * 2001-02-10 2006-06-15 Wayne Westerman System and method for packing multitouch gestures onto a hand
US6570557B1 (en) * 2001-02-10 2003-05-27 Finger Works, Inc. Multi-touch system and method for emulating modifier keys via fingertip chords
US20020113776A1 (en) * 2001-02-21 2002-08-22 Ran Meriaz Trackball mouse
US6587091B2 (en) * 2001-04-23 2003-07-01 Michael Lawrence Serpa Stabilized tactile output mechanism for computer interface devices
US20040126171A1 (en) * 2001-04-30 2004-07-01 Microsoft Corporation Keyboard with improved lateral region
US7463239B2 (en) * 2001-04-30 2008-12-09 Microsoft Corporation Input device including a wheel assembly for scrolling an image in multiple directions
US20100214250A1 (en) * 2001-05-16 2010-08-26 Synaptics Incorporated Touch screen with user interface enhancement
US20020190930A1 (en) * 2001-06-19 2002-12-19 Fujitsu Hitachi Plasma Display Limited Method of driving plasma display panel
US20030006961A1 (en) * 2001-07-09 2003-01-09 Yuly Shipilevsky Method and system for increasing computer operator's productivity
US20030063062A1 (en) * 2001-09-28 2003-04-03 Makoto Tsumura Image display device
US20030107552A1 (en) * 2001-12-11 2003-06-12 Koninklijke Philips Electronics N.V. Computer mouse with dual functionality
US20030107603A1 (en) * 2001-12-12 2003-06-12 Intel Corporation Scroll notification system and method
US20030169216A1 (en) * 2002-03-06 2003-09-11 Lg Electronics Inc. Method and apparatus for driving plasma display panel
US7168047B1 (en) * 2002-05-28 2007-01-23 Apple Computer, Inc. Mouse having a button-less panning and scrolling switch
US20040017358A1 (en) * 2002-07-19 2004-01-29 Kuo Hsiao Ming Housing assembly for computer mouse
US20040155865A1 (en) * 2002-12-16 2004-08-12 Swiader Michael C Ergonomic data input and cursor control device
US20040189605A1 (en) * 2003-03-25 2004-09-30 Ken Shih Mouse with multi-axis inputting device
US7808479B1 (en) * 2003-09-02 2010-10-05 Apple Inc. Ambidextrous mouse
US20050179655A1 (en) * 2004-02-13 2005-08-18 Ludwig Lester F. Electronic document editing employing multiple cursors
US20050179652A1 (en) * 2004-02-13 2005-08-18 Ludwig Lester F. Mouse-based user interface device employing user-removable modules
US20050179650A1 (en) * 2004-02-13 2005-08-18 Ludwig Lester F. Extended parameter-set mouse-based user interface device offering offset, warping, and mixed-reference features
US7620915B2 (en) * 2004-02-13 2009-11-17 Ludwig Lester F Electronic document editing employing multiple cursors
US7557797B2 (en) * 2004-02-13 2009-07-07 Ludwig Lester F Mouse-based user interface device providing multiple parameters and modalities
US20050179663A1 (en) * 2004-02-13 2005-08-18 Ludwig Lester F. Freely rotating trackball providing additional control parameter modalities
US20080211775A1 (en) * 2004-07-30 2008-09-04 Apple Inc. Gestures for touch sensitive input devices
US20080204426A1 (en) * 2004-07-30 2008-08-28 Apple Inc. Gestures for touch sensitive input devices
US20080211783A1 (en) * 2004-07-30 2008-09-04 Apple Inc. Gestures for touch sensitive input devices
US20080211785A1 (en) * 2004-07-30 2008-09-04 Apple Inc. Gestures for touch sensitive input devices
US20080211784A1 (en) * 2004-07-30 2008-09-04 Apple Inc. Gestures for touch sensitive input devices
US20080231610A1 (en) * 2004-07-30 2008-09-25 Apple Inc. Gestures for touch sensitive input devices
US20080088602A1 (en) * 2005-03-04 2008-04-17 Apple Inc. Multi-functional hand-held device
US20080158169A1 (en) * 2007-01-03 2008-07-03 Apple Computer, Inc. Noise detection in multi-touch sensors
US20080163130A1 (en) * 2007-01-03 2008-07-03 Apple Inc Gesture learning
US20080168364A1 (en) * 2007-01-05 2008-07-10 Apple Computer, Inc. Adaptive acceleration of mouse cursor
US20080165140A1 (en) * 2007-01-05 2008-07-10 Apple Inc. Detecting gestures on multi-event sensitive devices
US20080165141A1 (en) * 2007-01-05 2008-07-10 Apple Inc. Gestures for controlling, manipulating, and editing of media files using touch sensitive devices
US20080309632A1 (en) * 2007-06-13 2008-12-18 Apple Inc. Pinch-throw and translation gestures
US20100156818A1 (en) * 2008-12-23 2010-06-24 Apple Inc. Multi touch with multi haptics

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9007302B1 (en) 2011-11-11 2015-04-14 Benjamin D. Bandt-Horn Device and user interface for visualizing, navigating, and manipulating hierarchically structured information on host electronic devices
US9829987B2 (en) 2011-11-11 2017-11-28 Benjamin D. Bandt-Horn Device and user interface for visualizing, navigating, and manipulating hierarchically structured information on host electronic devices
US11010033B2 (en) * 2015-02-19 2021-05-18 Olympus Corporation Display control apparatus and methods for generating and displaying a related-item plate which includes setting items whose functions are related to a designated setting item
US10000268B1 (en) * 2015-08-20 2018-06-19 Brunswick Corporation Systems and methods for controlling a marine vessel having a joystick with adjustable display

Also Published As

Publication number Publication date
US20050179655A1 (en) 2005-08-18
US11314340B2 (en) 2022-04-26
US11809643B2 (en) 2023-11-07
US7620915B2 (en) 2009-11-17
US8816956B2 (en) 2014-08-26
US20050179652A1 (en) 2005-08-18
US20150062014A1 (en) 2015-03-05
US20100060607A1 (en) 2010-03-11
US20230116511A1 (en) 2023-04-13
US20050179651A1 (en) 2005-08-18
US11797107B2 (en) 2023-10-24
US20050179650A1 (en) 2005-08-18
US20220404920A1 (en) 2022-12-22
US7557797B2 (en) 2009-07-07
US9417716B2 (en) 2016-08-16
US20100064238A1 (en) 2010-03-11
US20110134039A1 (en) 2011-06-09
US20170139491A1 (en) 2017-05-18
US20050179663A1 (en) 2005-08-18

Similar Documents

Publication Publication Date Title
US11797107B2 (en) Method and user interface device with touch sensor for controlling applications
US5936612A (en) Computer input device and method for 3-D direct manipulation of graphic objects
JP5379777B2 (en) Object position detector
US8587517B2 (en) Input device, input method, corresponding computer program, and corresponding computer-readable storage medium
Weiss et al. SLAP widgets: bridging the gap between virtual and physical controls on tabletops
Kurtenbach et al. Issues in combining marking and direct manipulation techniques
KR101541928B1 (en) visual feedback display
US8471822B2 (en) Dual-sided track pad
Moscovich et al. Navigating documents with the virtual scroll ring
WO1995031791A1 (en) Method and apparatus for noise filtering for an input device
WO1998000775A9 (en) Touchpad with scroll and pan regions
JP2007535726A (en) Method and apparatus for generating a one-dimensional signal with a two-dimensional pointing device
Tsandilas et al. Sketchsliders: Sketching widgets for visual exploration on wall displays
US20030222925A1 (en) Field control method and system
Dachselt et al. A Survey and Taxonomy of 3D Menu Techniques.
Bae et al. Tangible NURBS-curve manipulation techniques using graspable handles on a large display
Rossmy et al. TouchGrid–Combining Touch Interaction with Musical Grid Interfaces
Michelitsch et al. Multi-finger haptic interaction within the MIAMM project
Couturier et al. Pointing fingers: using multiple direct interactions with visual objects to perform music
Rooney et al. A new method for interacting with multi-window applications on large, high resolution displays
JP2548131Y2 (en) Pen-type input device
Wu Study and design of interaction techniques to facilitate object selection and manipulation in virtual environments on mobile devices
Avizzano et al. Haptic desktop for office automation and assisted design
JPH04367925A (en) Mouse device

Legal Events

Date Code Title Description
AS Assignment

Owner name: BIORAM TECH L.L.C., DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LUDWIG, LESTER F.;REEL/FRAME:027345/0099

Effective date: 20111114

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION