Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
  • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05G—CONTROL DEVICES OR SYSTEMS INSOFAR AS CHARACTERISED BY MECHANICAL FEATURES ONLY
  • G05G9/00—Manually-actuated control mechanisms provided with one single controlling member co-operating with two or more controlled members, e.g. selectively, simultaneously
  • G05G9/02—Manually-actuated control mechanisms provided with one single controlling member co-operating with two or more controlled members, e.g. selectively, simultaneously the controlling member being movable in different independent ways, movement in each individual way actuating one controlled member only
  • G05G9/04—Manually-actuated control mechanisms provided with one single controlling member co-operating with two or more controlled members, e.g. selectively, simultaneously the controlling member being movable in different independent ways, movement in each individual way actuating one controlled member only in which movement in two or more ways can occur simultaneously
  • G05G9/047—Manually-actuated control mechanisms provided with one single controlling member co-operating with two or more controlled members, e.g. selectively, simultaneously the controlling member being movable in different independent ways, movement in each individual way actuating one controlled member only in which movement in two or more ways can occur simultaneously the controlling member being movable by hand about orthogonal axes, e.g. joysticks
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
  • G06F3/0338—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
  • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
  • G06F3/0383—Signal control means within the pointing device

Definitions

• This invention relates in general to computer user input devices and more specifically to a method and apparatus for achieving a more accurate relative pointing device in a computer user interface.
• RPD relative pointing device
• Today's graphical computer user interfaces typically use some form of "relative pointing device” (RPD) for moving a pointer, cursor, or the like around on a display screen.
• RPDs include a mouse, joystick, touchpad, trackball, etc.
• One approach at implementing these devices uses a material that exhibits varying electrical properties as a function of applied pressure, or position, of a control object, or surface, manipulated by a user.
• the use of such materials introduces problems in that the measurement of the resistance, conductance, capacitance or other property of the material is not always constant, or consistent, enough to use the pointing device for precise, stable, and consistently accurate positioning in a computer user interface.
• Drift results from miscahbration of a "zero" position, or other reference position, for the pointing device.
• "Drift” is perceived by a user of the device as spurious or inaccurate movement of a pointer, or other image or object, that is controlled by the device.
• Differences in material, composition, components, temperature, geometry of finger placement, etc. all contribute to whether the device is properly centered, or whether there is drift.
• a properly calibrated device allows the user to easily make fine position motions. An improperly calibrated device will exhibit drift and be difficult to control.
• a common user manipulation is to position a pointer by using the RPD and then depress a button, or other control, to indicate selection of an item selected by the pointer.
• a button or other control
• the pointer does not change position.
• miscahbration There are other consequences of "drift”. If the user is trying to make a fine cursor movement while drift is present then the user is forced to compensate by applying pressure against the drift. This feels rather like trying to walk on very slippery ice in a strong windstorm. Another consequence of miscahbration is reduced dynamic range in the direction "against the drift". Some or all of the movement range of the device can be absorbed in an attempt to counter the drift, leaving less range available for intended manipulation.
• the present invention automatically compensates for variations in the components that make up a pointing device used as an input device in a computer system.
• a preferred embodiment of the invention is adapted to a finger-operated pointing device.
• Automatic calibration ensures that calibration is performed, as opposed to manual calibration or other conditionally-performed calibration. Automatic calibration also frees the user from having to understand and perform the calibration procedure and eliminates the cost and size requirements of devices (e.g., "trim pots" in typical joysticks) necessary to perform the calibration.
• Another aspect of the invention is to correct for differing geometry of users' hands and fingers as the user depresses the pointing device.
• the "natural" "home” position may be different for left-handers or right-handers, or if the device is operated with a thumb versus a forefinger.
• One aspect of the invention includes a system that monitors user activity and, from that activity, automatically selects X-Y values for auto-calibration.
• Another aspect of the invention makes specific use of "first touch” and “click” activities to select proper X-Y values for auto-calibration.
• Yet another aspect of the invention improves the accuracy in a "click from out of presence" manipulation.
• Yet another aspect of the invention uses hardware for detecting and calibrating a first touch.
• Fig. 1 shows a pointing device in an "out of presence" condition
• Fig. 2 shows a pointing device in an "in presence" condition
• Fig. 3 shows a "click,” or selection
• Fig. 4 shows a "click, or selection, on the fly
• Fig. 5 shows a contact point on a contact area
• Fig. 6 shows a contact point on an off-center portion of the contact area as a result of a user clicking while positioning at the same time
• Fig. 7 shows a center contact point surrounded by an insulated area
• Fig. 8 illustrates the basic file organization of the firmware
• Fig. 9 shows "main loop” processing of the firmware
• Fig. 10A illustrates first table values used to affect tracking
• Fig. 10B illustrates second table values used to affect tracking
• Fig. 10C illustrates third table values used to affect tracking
• Fig. 11 is a flowchart to illustrate the basic steps of the tracking procedures.
• In presence or "out of presence.” This refers to the ability of the device to allow movement of a control, such as a small "stick” control operated by a finger, in two different modes. In a first, “in presence” mode, a pointer is moved in accordance with the movement of the stick. In the second “out of presence” mode the pointer on the display screen does not move even when the stick is being moved.
• a user can decide whether the stick is moved in or out of presence by applying slight downward pressure while the stick is being manipulated. If downward pressure is applied as the stick moves, this is an "in presence” movement and the pointer is moved accordingly. When the downward pressure is released then the stick can be moved without moving the pointer.
• first touch the X-Y position of the device at the instant it first enters "presence" is detected. This location is used as a bias pair coordinate value for subsequent pointing activity.
• Another approach is available on a pointing device that has an integrated switch for clicking. It is possible to take the X-Y position of the device at the instant it "clicks”. Further refinements assess the users' intention to determine which "first touch” and "click” activities yield X-Y positions suitable for auto-calibration.
• Another embodiment detects when a user is clicking from out of presence. That is, when a user is indicating a selection by making a hard downward depressing movement of the control stick.
• This system works on a finger-operated pointing system with integrated "click" switch. The switch is built into the control so that a user can trigger the click by increasing the Z component of pointing force without having to remove the finger from the device to find and use external switches.
• One type of pointing device for which this invention is applicable has distinct “presence” and "not in presence” modes.
• Fig. 1 shows a specific device referred to as "VaraPoint,” manufactured by Varatouch Technology, Inc., In Fig. 1, the device is “not in presence.”
• Fig. 2 shows the VaraPoint device “in presence.”
• One embodiment of the present invention is a method of creating and using a "usage profile" that gives a reliable indication of which position of the device the user expects to be associated with zero motion. Earlier attempts at auto-calibration used the X-Y coordinates of the device at the instant in time when the user performed a click, or kept a running average of those X-Y pairs associated with clicks.
• Figs. 3 and 4 show two possible "click” events. This algorithm is different, in that it distinguishes between clicks for which the associated X-Y position is likely to be a good calibration point, and those for which it is not.
• the essence of the invention is the use of the X-Y location associated with the "out of presence” clicking for auto-calibration, and, especially, not using the X-Y data of a click that occurred "on the fly".
• the initial auto-calibration is performed on the first "out of presence” click, and subsequent re-calibration occurs either as a running average of subsequent "out of presence” clicks (the preferred embodiment), or by simply replacing centering bias X-Y values with those of subsequent "out of presence” clicks.
• a preferred embodiment of the invention uses source code, or firmware, included in the appendix.
• This firmware produces a system compatible with a Microsoft Mouse (either serial or PS/2).
• the VaraPoint sensor used in the invention provides an analog output signal.
• the firmware in the preferred embodiment digitizes the analog signal to 5 bit (32 step) resolution.
• Other interfaces e.g., a game controller interface, can be used in place of the mouse interface.
• the firmware produces code to be run on an Atmel 89C2051 CPU.
• the basic file organization of the firmware is illustrated in Fig. 8.
• the "main loop" processing is illustrated by the flowchart of Fig. 9.
• the firmware of the preferred embodiment can be adapted for use with any suitable microprocessor that meets the system requirements (see below).
• the code to support the VaraPoint device may coexist with other code in the microprocessor, for example, to scan buttons on a remote control device or keyboard.
• the software has the following options, which can be used to generate different versions via conditional assembly: a) MSMOUSE or PS2MOUSE
• This switch determines whether code will be generated to support a switch under the VaraPoint device to generate "mouse clicks".
• a click is commonly a part of most pointing activity, in which the user indicates an intent to take some action with the object to which the user is currently pointing.
• a separate switch is provided, which may be scanned in other code, or by some other processor.
• SNAPTOGRID This switch determines whether code will be generated to cause motion near a multiple of 90 degrees to "snap" to those directions. This switch is normally turned on when the application involves Graphical User Interface (GUI) menu navigation, etc. It would be less desirable if the primary application involved sketching and drawing.
• GUI Graphical User Interface
• NAVIGATEMENU This switch determines whether code will be generated to cause motion near a multiple of 45 to "snap" to those directions. It is much like the SNAPTOGRID switch, but allows “snapping" to the true axes and the 45-degree diagonals. As with SNAPTOGRID, the intended application would dictate whether this switch should be set.
• AUTOCENTER This switch determines whether code will be generated to cause the system to recalibrate itself for "centering", the position at which no motion is generated. In any joystick, or joystick-like pointing device, any of a number of situations can cause the "null" position to be other than where the user expects it.
• the Presence value helps the system to distinguish between two different types of clicks.
• the "in-presence” click is also known as the "click-on-the-fly” or “bombing-run” click - in which the user lets the cursor drift over the target, and clicks during movement. This type of click is unsuitable for auto-centering and must be avoided.
• the "out-of-presence” click occurs when the user has stopped pointing, and makes the click with a frozen cursor. This type of click is generally very suitable for auto-centering.
• the current algorithm is, on a click, to determine whether "presence" scans have taken place since initiating pointing.
• switches in the form "PD ⁇ " which cause code to be assembled to output diagnostic information on the serial port. These switches and diagnostic information are used for debugging purposes.
• the routine ScanSensor essentially implements an Analog to Digital (AT)) conversion to produce 5 bit data values representing the X and Y components of the deflection of the VaraPoint sensor.
• AT Analog to Digital
• This circuit also includes a wakeup circuit which can be used in battery operated applications where the microcontroller is to be put into a low power "sleep mode" when there is no VaraPoint activity. While the VaraPoint deflection is being measured (i.e. while the microcontroller is active) the signal "WAKE ENABLE" is at a high-impedance state, effectively taking the wakeup circuit out of consideration.
• the resistance between the direction outputs of the VaraPoint module and Vcc is measured by: 1) Fully discharging the .047uF timing capacitors through pins 0..3 of PORTB of the microcontroller (by configuring those pins as outputs and setting their level LOW), 2) Switching pins 0..3 of PORTB of the microcontroller to high-impedance input mode, and 3) Measuring the time it takes for the timing capacitors to charge to the point that the microcontroller reads an input HIGH at each of pins 0..3 of PORTB.
• the chain of conversions described above is summarized as 1) convert from joystick deflection to resistance, and 2) convert from resistance to capacitor charging time (which is easily measured by the microcontroller).
• the resistance of a VaraPoint direction is nominally 50K Ohms at minimum deflection (and >1M Ohm at no deflection).
• the time to charge a capacitor to the microcontroller's input HIGH level is 0.6*R*C, so at minimum deflection the charge time is 1.410 msec.
• the window of time that the microcontroller samples the direction inputs is 1.397 msec (127 samples of each direction input * 1 lusec per sampling loop — see assembly listing). So the microcontroller reads a minimum deflection at a slightly larger than minimum physical deflection on a nominal VaraPoint (this must be the case for a nominal setup so that we can always attain a minimum deflection taking variation of component values into account). As deflection is increased, the charge time is decreased, since the direction's resistance to Vcc is decreased. If the charge time is longer than the 1.397 msec sampling window, this is read as no deflection at all.
• Figs. 10A-C illustrate table values used to affect tracking.
• Fig. 11 is a flowchart to illustrate the basic steps of the tracking procedures. The source code appendix should be consulted for details.
• TblSpeedVect and TblDelayVect tables work together to implement the main part of VaraPoint tracking. These tables are normally adjusted together to account for differences in microprocessor speed and AID circuits (such as different reference voltages, resistor and capacitor values, etc.). Together, these tables implement a "Three Plateaus" approach to tracking: Fine Control, Navigation, and Blitz. A general strategy for adjusting these tables is to compare a trial implementation with a reference design VaraPoint, and to try to achieve a similar level of control. Fine-tuning is best accomplished with user-level tests. TblSpeedVect is the primary tracking table, working in conjunction with
• TblDelayVect contains 32 entries, for each of 32 possible input counts (coming from the A/D circuit). For each input count, TblSpeedVect gives the number of output counts ("Mickeys") that the firmware should report. These Mickeys correspond to the counts normally made by optical encoders in a typical 300-dpi mouse. Thus, when the A/D measures a force corresponding to NNN counts from the A/D, the system will report TblSpeedVect(NNN) Mickeys on each report out, which will occur every TblDelayVect(NNN) milliseconds.
• the TblDelayVect table works in conjunction with TblSpeedVect to condition the apparent rate of motion reported by the firmware.
• TblSpeedVect/TblDelayVect system is an alternative to "Fractional Mickey" tracking, which would have to be maintained if the motion would correspond to less than one Mickey (usually 1/3 00 of an inch) per packet of data sent out. By delaying the time between packets, the effective rate of motion is kept appropriately low when the intended motion is slow.
• TblSlowVect manages an important aspect of VaraPoint tracking — an alternative tracking during deceleration to manage the "overshoot" which so often otherwise characterizes joystick pointing.
• the firmware always remembers the last force vector magnitude, and continually compares it to the current force vector magnitude. When the magnitude is decreasing, then the user is attempting to slow down. The difference between last and current magnitude will be positive during deceleration, and is used as an index into the TblSlowVect to calculate an adjustment to the speed to help slow down motion faster and minimize overshoot. The value in the table is multiplied by previous (larger) magnitude, and that number is subtracted from the current magnitude.
• the system requires a processor capable of performing A D conversion on N-S and E-W signals of the R VaraPoint sensor, and processing the resulting digital information through tracking algorithms, detecting and reporting click information, and communicating the information at rates consistent with human factors requirements.
• the reference implementation uses a typical 8-bit microprocessor running at 12 MHz, but considerable design flexibility is available. The reference design occupies 1.5K bytes of program space.
• the system requires 4 pins for connection to X and Y sensors, but these pins can be shared with other functions.
• pins can be time-multiplexed for both functions.
• the lines should be true tn-state, but if the processor can set the ports to input mode without internal or external pull-ups, they can be used.
• Some microprocessors have integrated A D support. These circuits can be used to implement VaraPoint, provided that they can be adjusted to provide adequate resolution for the resistance ranges (typically 5 bits or better), and this can result in significant savings in program memory requirements, that occupied by the ScanSensor routine.
• the reference design firmware is written using the industry-standard 8051 instruction set.
• An embodiment of the invention provides "hardware help” for detecting and calibrating a "first touch.”
• Some pointing devices have no dome switches or other methods for detecting a "click” via increased pressure on the fingertip actuator. These systems typically implement one or more "click” switches via separate, external switches. With devices constructed in this manner, the device may not be in “presence” at all at the time of the click.
• Initial experimentation had involved the use of "first touch", which was to use the X-Y values detected at the transition from out-of-presence to presence operation. However, this system proved to be vulnerable to false centering when the user attempted to initiate pointing with immediate, fast movement.
• auto-calibration aspects of the present invention shield the user from having to know about, or to take active concern, for auto-calibration.
• the system requires an extra contact point and surrounding open area on the contact area as shown in Fig. 7.
• a microprocessor port (perhaps shared) is needed to "watch" for contact on the centering contact point.
• PD_SUP1 EQU PD_SENSE+PD_SCANSENSOR+PD_GETSYSDATA+PD_ ⁇ PUTSYSDATA
• PD_SUP2 EQU PD_CMD_FF+PD_SS_ABORT+PD_ABORT+PD_TRYERROR
• PushCntLatch EQU 37h Number of passes to latch
• PresenceCount EQU 40h Presence count (note address skip).
• Timer 1 Mode 2; Timer 0: Mode 1 setb trl ; Timer 1 on mov scon, #050h Mode 1, Enable Reception mov scon, #040h Mode 1, Disable Reception mov pl,#0ffh mov p3,#07fh ; Allow Txd and Rxd functions of P3 clr itO ,- Use level for into mov ipc,#000h ; No distinction in priority
• buttons are pushed
• buttons There are two buttons : Left and right .
• Sense () For each corner, enable its driver and sense the value by a call to Sense () . Clear the X and Y values.
• the Sense () routine will indicate a value which corresponds to the position of the joystick relative to that corner (N, S, W or E) .
• the joystick driver is on. This signal goes to each of the corners and is sensed. When the value exceeds the TTL threshold, the signal changes state from low to high. The timing of this change indicates to actual resistance/deflection. All four corners are sensed at the same time .
• the diagonals are sensed to add more directional capabilities. Currently, the only directions are N, S, E, W and the two diagonals. By adding diagonal sensors, eight additional directions are added. For instance, sensing the north sensor means that we have north deflection. If east is also sensed, that means we are on the diagonal. However, if north is sensed, but east is not and the diagonal sensor is activated, this means that we are 22.5 degrees off of north.
• ScanSensor mov 01dXCoord,XCoord ; Save old values mov OldYCoord, YCoord
• Not_in_Presence debug ' N ' clr InPresence IF PUSHDOWN mov PushReset,#5 ; Indicate: reset output coordinates ENDIF mov OutXCoord, #0 mov OutYCoord,#0 jmp gohome
• debug ' 1 ' jb HaveAutoCtrd, HaveACdB4 if we got here, we've never click-centered before, let's take this value as golden clr c mov a, RXCoord mov XOffsetH,a mov a, RYCoord mov Y ⁇ ffsetH,a mov a,#0 mov X ⁇ ffsetL,a mov Y ⁇ ffsetL,a setb HaveAutoCtrd ,-now that we've done it, remember jmp Have_Centered HaveACdB4 : ; if we got here, we have click-centered before, take rotating average debug ' 2 ' clr c mov a, RXCoord Get current coordinate subb a,X
• No_Ctr_Tch debug 'N' debug 'C debug • t'
• SnapBigY lcall GetSnapToGridPercentage mov b,rl ; Get Y mul ab mov a,r0 ; Get X clr c subb a, b Is X bigger than percentage of Y? jnc Snap99 ; NC - X is big enough mov XCoord, #0 ; Snap to axis sjmp Snap99
• SnapBigX lcall GetSnapToGridPercentage mov b,r0 ; Get X mul ab mov a,rl ; Get Y clr c subb a,b ; Is Y bigger than percentage of X? jnc Snap99 ; NC - Y is big enough mov YCoord, #0 ; Snap to axis
• Menu navigation limits variability of coordinates to one of eight directions based upon the X/Y coordinate. This is to assist the operator in moving around through a menu. The assumption is that if the button is not pushed or latched, then we are navigating (versus drawing) and some assistance can be offered.
• NavYAxis mov XCoord, #0 ; Lock to Y axis sjmp Nav99
• NavYDiag mov a, Coord ; What is current sign of X? jb acc7,NavYDXM ; Negative - jump mov XCoord, rl ; Get positive Y sjmp Nav99 NavYDX : clr a clr c subb a,rl mov XCoord, a Get negative Y sjmp Nav99
• NavBigX mov a,r0 clr c subb a,rl ; compute X-Y clr c subb a,rl jc NavXDiag ,- Force X-dominate diagonal
• NavXA is : mov YCoord, #0 ; Lock to X axis sjmp Nav99
• NavXDiag mov a , YCoord ; What is current sign of Y? jb acc7,NavXDYM ; Negative - jump mov YCoord, rO Get positive X sjmp Nav99
• PushNoReset IF PUSHDOWN mov a,PushReset ; Should I reset output coordinates? jz PushNoReset add a,#-l ,- Decrement reset count mov PushReset,a ; Save for next pass mov OutXCoord, #0 ; Do not let the switch move while pushing mov OutYCoord, #0 PushNoReset :

Landscapes

• Engineering & Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Automation & Control Theory (AREA)
• Position Input By Displaying (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=22412999

Family Applications (1)

Country Status (5)

Families Citing this family (4)

Family Cites Families (2)

• 2000
  • 2000-03-10 EP EP00916274A patent/EP1200954A1/en not_active Withdrawn
  • 2000-03-10 WO PCT/US2000/006477 patent/WO2000054247A1/en active Application Filing
  • 2000-03-10 JP JP2000604395A patent/JP2002539541A/ja active Pending
  • 2000-03-10 AU AU37403/00A patent/AU3740300A/en not_active Abandoned
• 2002
  • 2002-11-01 HK HK02107958.3A patent/HK1047338A1/zh unknown

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events