EP1200954A1 - Auto-calibration of pointing devices used in a computer user interface - Google Patents

Abstract

A pointing device in a computer system is automatically calibra ted by distinguishing between 'in presence' and 'out of presence' conditions. Calibration correction accomodates differing geometry of user's hands and fingers. Thus, the 'natural' 'home' position may be different for left-handers or right-handers, or if the device is operated with a thumb versus of forefinger. A system monitors user activity and from that activity automatically selects X-Y values for aut o-calibration. 'First touch' and 'click' activities are used to select proper X-Y values for auto-calibration. 'Click from out of presence' is used to determine user selection events for proper calibration. The invention provides 'hardware help' for detecting and calibrating a 'first touch'.

Description

AUTO-CALIBRATION OF POINTING DEVICES USED IN A

COMPUTER USER INTERFACE

CLAIM OF PRIORITY This patent application claims priority from co-pending provisional patent application Serial No. 60/124,137 filed March 12, 1999 entitled METHOD FOR AUTO- CALIBRATING OF POINTING DEVICES BASED ON RESISTIVE RUBBER which is hereby incorporated by reference as if set forth in full in this document.

COPYRIGHT NOTICE

A portion of the disclosure recited in this application contains material which is subject to copyright protection. Specifically, source code is provided for a computer program implementing portions of the invention as described herein. The copyright owner has no objection to the facsimile reproduction of the specification as filed in the Patent and Trademark Office. Otherwise all copyright rights are reserved.

BACKGROUND OF THE INVENTION

This invention relates in general to computer user input devices and more specifically to a method and apparatus for achieving a more accurate relative pointing device in a computer user interface.

Today's graphical computer user interfaces typically use some form of "relative pointing device" (RPD) for moving a pointer, cursor, or the like around on a display screen. Examples of RPDs include a mouse, joystick, touchpad, trackball, etc. One approach at implementing these devices uses a material that exhibits varying electrical properties as a function of applied pressure, or position, of a control object, or surface, manipulated by a user. The use of such materials introduces problems in that the measurement of the resistance, conductance, capacitance or other property of the material is not always constant, or consistent, enough to use the pointing device for precise, stable, and consistently accurate positioning in a computer user interface.

For example, where resistive rubber is used as the sensing material the undesirable property of "drift" presents itself. Drift results from miscahbration of a "zero" position, or other reference position, for the pointing device. "Drift" is perceived by a user of the device as spurious or inaccurate movement of a pointer, or other image or object, that is controlled by the device. Differences in material, composition, components, temperature, geometry of finger placement, etc., all contribute to whether the device is properly centered, or whether there is drift. A properly calibrated device allows the user to easily make fine position motions. An improperly calibrated device will exhibit drift and be difficult to control.

A common user manipulation is to position a pointer by using the RPD and then depress a button, or other control, to indicate selection of an item selected by the pointer. Before, during or after the user's act of depressing the control it is usually important that the pointer does not change position. When such an unwanted change in position takes place as a result of miscahbration it is referred to as "drift". There are other consequences of "drift". If the user is trying to make a fine cursor movement while drift is present then the user is forced to compensate by applying pressure against the drift. This feels rather like trying to walk on very slippery ice in a strong windstorm. Another consequence of miscahbration is reduced dynamic range in the direction "against the drift". Some or all of the movement range of the device can be absorbed in an attempt to counter the drift, leaving less range available for intended manipulation.

Thus, it is desirable to reduce or eliminate the problem of miscahbration and drift in a pointing device.

SUMMARY OF THE INVENTION The present invention automatically compensates for variations in the components that make up a pointing device used as an input device in a computer system. A preferred embodiment of the invention is adapted to a finger-operated pointing device. Automatic calibration ensures that calibration is performed, as opposed to manual calibration or other conditionally-performed calibration. Automatic calibration also frees the user from having to understand and perform the calibration procedure and eliminates the cost and size requirements of devices (e.g., "trim pots" in typical joysticks) necessary to perform the calibration. Another aspect of the invention is to correct for differing geometry of users' hands and fingers as the user depresses the pointing device. The "natural" "home" position may be different for left-handers or right-handers, or if the device is operated with a thumb versus a forefinger. One aspect of the invention includes a system that monitors user activity and, from that activity, automatically selects X-Y values for auto-calibration.

Another aspect of the invention makes specific use of "first touch" and "click" activities to select proper X-Y values for auto-calibration.

Yet another aspect of the invention improves the accuracy in a "click from out of presence" manipulation.

Yet another aspect of the invention uses hardware for detecting and calibrating a first touch.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a pointing device in an "out of presence" condition;

Fig. 2 shows a pointing device in an "in presence" condition;

Fig. 3 shows a "click," or selection;

Fig. 4 shows a "click, or selection, on the fly;

Fig. 5 shows a contact point on a contact area; Fig. 6 shows a contact point on an off-center portion of the contact area as a result of a user clicking while positioning at the same time;

Fig. 7 shows a center contact point surrounded by an insulated area;

Fig. 8 illustrates the basic file organization of the firmware;

Fig. 9 shows "main loop" processing of the firmware; Fig. 10A illustrates first table values used to affect tracking;

Fig. 10B illustrates second table values used to affect tracking;

Fig. 10C illustrates third table values used to affect tracking; and

Fig. 11 is a flowchart to illustrate the basic steps of the tracking procedures.

DESCRIPTION OF THE SPECIFIC EMBODIMENTS

System Description One type of pointing device has two states referred to as "in presence" or "out of presence." This refers to the ability of the device to allow movement of a control, such as a small "stick" control operated by a finger, in two different modes. In a first, "in presence" mode, a pointer is moved in accordance with the movement of the stick. In the second "out of presence" mode the pointer on the display screen does not move even when the stick is being moved. Typically, a user can decide whether the stick is moved in or out of presence by applying slight downward pressure while the stick is being manipulated. If downward pressure is applied as the stick moves, this is an "in presence" movement and the pointer is moved accordingly. When the downward pressure is released then the stick can be moved without moving the pointer. This is useful when the stick needs to be returned to a "home" position for further movement of the pointer. This is analogous to the mouse-and-pointer scenario where a user must lift up the mouse and then put the mouse back down in order to keep the mouse within an area (such as a mouse "pad" area) and to continue moving the pointer in the desired direction. In one embodiment, referred to as "first touch," the X-Y position of the device at the instant it first enters "presence" is detected. This location is used as a bias pair coordinate value for subsequent pointing activity. Another approach is available on a pointing device that has an integrated switch for clicking. It is possible to take the X-Y position of the device at the instant it "clicks". Further refinements assess the users' intention to determine which "first touch" and "click" activities yield X-Y positions suitable for auto-calibration.

Another embodiment detects when a user is clicking from out of presence. That is, when a user is indicating a selection by making a hard downward depressing movement of the control stick. This system works on a finger-operated pointing system with integrated "click" switch. The switch is built into the control so that a user can trigger the click by increasing the Z component of pointing force without having to remove the finger from the device to find and use external switches. One type of pointing device for which this invention is applicable has distinct "presence" and "not in presence" modes.

In a preferred embodiment, "presence" is denoted by contact between the resistive rubber and the conductive surface beneath it. Fig. 1 shows a specific device referred to as "VaraPoint," manufactured by Varatouch Technology, Inc., In Fig. 1, the device is "not in presence." Fig. 2 shows the VaraPoint device "in presence." One embodiment of the present invention is a method of creating and using a "usage profile" that gives a reliable indication of which position of the device the user expects to be associated with zero motion. Earlier attempts at auto-calibration used the X-Y coordinates of the device at the instant in time when the user performed a click, or kept a running average of those X-Y pairs associated with clicks. Figs. 3 and 4 show two possible "click" events. This algorithm is different, in that it distinguishes between clicks for which the associated X-Y position is likely to be a good calibration point, and those for which it is not.

In particular, there are two distinguishable patterns of clicking "out of presence" clicking, and "clicking on the fly". "Out of presence" clicking occurs when the user has positioned the tracking cursor carefully above a target, temporarily released finger pressure (to take the device out of "presence" mode), and then made a distinct click without intended motion. Fig. 3 shows a typical "out of presence" click. Clicking on the fly occurs when the user drifts the tracking cursor over a target, and clicks while it is moving. Fig. 4 shows a typical "on the fly" click. The key distinction between these behaviors is whether and for how long the device was "out of presence" before the click. The essence of the invention is the use of the X-Y location associated with the "out of presence" clicking for auto-calibration, and, especially, not using the X-Y data of a click that occurred "on the fly". The initial auto-calibration is performed on the first "out of presence" click, and subsequent re-calibration occurs either as a running average of subsequent "out of presence" clicks (the preferred embodiment), or by simply replacing centering bias X-Y values with those of subsequent "out of presence" clicks.

Software for performing the "out of presence" sensing and calibrating is included with this application in several files under the subdirectory label "pr25b." Such software should be consulted for more details on the preferred embodiment of this aspect. The directory paths for each file are listed at the bottom of each page of the source code listing associated with the file. Specifics of the software are next discussed.

Source Code (firmware)

A preferred embodiment of the invention uses source code, or firmware, included in the appendix. This firmware produces a system compatible with a Microsoft Mouse (either serial or PS/2). The VaraPoint sensor used in the invention provides an analog output signal. The firmware in the preferred embodiment digitizes the analog signal to 5 bit (32 step) resolution. Other interfaces, e.g., a game controller interface, can be used in place of the mouse interface. The firmware produces code to be run on an Atmel 89C2051 CPU. The basic file organization of the firmware is illustrated in Fig. 8. The "main loop" processing is illustrated by the flowchart of Fig. 9. The firmware of the preferred embodiment can be adapted for use with any suitable microprocessor that meets the system requirements (see below). In some applications, the code to support the VaraPoint device may coexist with other code in the microprocessor, for example, to scan buttons on a remote control device or keyboard. The

The software has the following options, which can be used to generate different versions via conditional assembly: a) MSMOUSE or PS2MOUSE

These two mutually exclusive switches determine whether the assembler will generate code to emulate a Microsoft Serial Mouse or a PS/2 Mouse. Naturally, it is possible to implement different interfaces for a pointing device in applications other than Microsoft Windows support. b) PUSHDOWN

This switch determines whether code will be generated to support a switch under the VaraPoint device to generate "mouse clicks". A click is commonly a part of most pointing activity, in which the user indicates an intent to take some action with the object to which the user is currently pointing. In some applications a separate switch is provided, which may be scanned in other code, or by some other processor. c) SNAPTOGRID This switch determines whether code will be generated to cause motion near a multiple of 90 degrees to "snap" to those directions. This switch is normally turned on when the application involves Graphical User Interface (GUI) menu navigation, etc. It would be less desirable if the primary application involved sketching and drawing. d) NAVIGATEMENU This switch determines whether code will be generated to cause motion near a multiple of 45 to "snap" to those directions. It is much like the SNAPTOGRID switch, but allows "snapping" to the true axes and the 45-degree diagonals. As with SNAPTOGRID, the intended application would dictate whether this switch should be set. e) AUTOCENTER This switch determines whether code will be generated to cause the system to recalibrate itself for "centering", the position at which no motion is generated. In any joystick, or joystick-like pointing device, any of a number of situations can cause the "null" position to be other than where the user expects it. This can cause a perception of "skating in the wind", or "drift". This switch would normally be turned on, except perhaps for some unusual applications in which the autocentering behavior conflicts with other design goals. With this switch turned on, the system automatically recalibrates itself based on a rolling average of contact points, with automatic compensation for outlying values. f) Presence

The Presence value helps the system to distinguish between two different types of clicks. The "in-presence" click is also known as the "click-on-the-fly" or "bombing-run" click - in which the user lets the cursor drift over the target, and clicks during movement. This type of click is unsuitable for auto-centering and must be avoided. On the other hand, the "out-of-presence" click occurs when the user has stopped pointing, and makes the click with a frozen cursor. This type of click is generally very suitable for auto-centering. The current algorithm is, on a click, to determine whether "presence" scans have taken place since initiating pointing. If so, we assume that there was so much time in the pointing immediately prior to the click that it should be deemed an "in-presence" click and made ineligible for auto-centering. If not, we then check to see if auto-centering has been performed yet. If we have never auto-centered yet, the current coordinates are taken as the autocentering values. If we have previously autocentered, we now initiate a rolling-average of the last four values, to guard against false auto-centering.

There are a number of switches in the form "PD ~" in the code which cause code to be assembled to output diagnostic information on the serial port. These switches and diagnostic information are used for debugging purposes.

The routine ScanSensor essentially implements an Analog to Digital (AT)) conversion to produce 5 bit data values representing the X and Y components of the deflection of the VaraPoint sensor.

The implementation of this task is illustrated using a Microchip PIC 1 6C54 microcontroller. This circuit also includes a wakeup circuit which can be used in battery operated applications where the microcontroller is to be put into a low power "sleep mode" when there is no VaraPoint activity. While the VaraPoint deflection is being measured (i.e. while the microcontroller is active) the signal "WAKE ENABLE" is at a high-impedance state, effectively taking the wakeup circuit out of consideration.

The resistance between the direction outputs of the VaraPoint module and Vcc is measured by: 1) Fully discharging the .047uF timing capacitors through pins 0..3 of PORTB of the microcontroller (by configuring those pins as outputs and setting their level LOW), 2) Switching pins 0..3 of PORTB of the microcontroller to high-impedance input mode, and 3) Measuring the time it takes for the timing capacitors to charge to the point that the microcontroller reads an input HIGH at each of pins 0..3 of PORTB. The chain of conversions described above is summarized as 1) convert from joystick deflection to resistance, and 2) convert from resistance to capacitor charging time (which is easily measured by the microcontroller).

The resistance of a VaraPoint direction is nominally 50K Ohms at minimum deflection (and >1M Ohm at no deflection). The time to charge a capacitor to the microcontroller's input HIGH level is 0.6*R*C, so at minimum deflection the charge time is 1.410 msec. The window of time that the microcontroller samples the direction inputs is 1.397 msec (127 samples of each direction input * 1 lusec per sampling loop — see assembly listing). So the microcontroller reads a minimum deflection at a slightly larger than minimum physical deflection on a nominal VaraPoint (this must be the case for a nominal setup so that we can always attain a minimum deflection taking variation of component values into account). As deflection is increased, the charge time is decreased, since the direction's resistance to Vcc is decreased. If the charge time is longer than the 1.397 msec sampling window, this is read as no deflection at all.

Figs. 10A-C illustrate table values used to affect tracking. Fig. 11 is a flowchart to illustrate the basic steps of the tracking procedures. The source code appendix should be consulted for details.

The TblSpeedVect and TblDelayVect tables work together to implement the main part of VaraPoint tracking. These tables are normally adjusted together to account for differences in microprocessor speed and AID circuits (such as different reference voltages, resistor and capacitor values, etc.). Together, these tables implement a "Three Plateaus" approach to tracking: Fine Control, Navigation, and Blitz. A general strategy for adjusting these tables is to compare a trial implementation with a reference design VaraPoint, and to try to achieve a similar level of control. Fine-tuning is best accomplished with user-level tests. TblSpeedVect is the primary tracking table, working in conjunction with

TblDelayVect. TblSpeedVect contains 32 entries, for each of 32 possible input counts (coming from the A/D circuit). For each input count, TblSpeedVect gives the number of output counts ("Mickeys") that the firmware should report. These Mickeys correspond to the counts normally made by optical encoders in a typical 300-dpi mouse. Thus, when the A/D measures a force corresponding to NNN counts from the A/D, the system will report TblSpeedVect(NNN) Mickeys on each report out, which will occur every TblDelayVect(NNN) milliseconds.

The TblDelayVect table works in conjunction with TblSpeedVect to condition the apparent rate of motion reported by the firmware. The

TblSpeedVect/TblDelayVect system is an alternative to "Fractional Mickey" tracking, which would have to be maintained if the motion would correspond to less than one Mickey (usually 1/3 00 of an inch) per packet of data sent out. By delaying the time between packets, the effective rate of motion is kept appropriately low when the intended motion is slow.

TblSlowVect manages an important aspect of VaraPoint tracking — an alternative tracking during deceleration to manage the "overshoot" which so often otherwise characterizes joystick pointing. During tracking, the firmware always remembers the last force vector magnitude, and continually compares it to the current force vector magnitude. When the magnitude is decreasing, then the user is attempting to slow down. The difference between last and current magnitude will be positive during deceleration, and is used as an index into the TblSlowVect to calculate an adjustment to the speed to help slow down motion faster and minimize overshoot. The value in the table is multiplied by previous (larger) magnitude, and that number is subtracted from the current magnitude.

In general, the system requires a processor capable of performing A D conversion on N-S and E-W signals of the R VaraPoint sensor, and processing the resulting digital information through tracking algorithms, detecting and reporting click information, and communicating the information at rates consistent with human factors requirements. The reference implementation uses a typical 8-bit microprocessor running at 12 MHz, but considerable design flexibility is available. The reference design occupies 1.5K bytes of program space.

Typically, the system requires 4 pins for connection to X and Y sensors, but these pins can be shared with other functions. For example, in an application with a keypad and a VaraPoint in which keypad keys and the VaraPoint are not used simultaneously, pins can be time-multiplexed for both functions. The lines should be true tn-state, but if the processor can set the ports to input mode without internal or external pull-ups, they can be used. Some microprocessors have integrated A D support. These circuits can be used to implement VaraPoint, provided that they can be adjusted to provide adequate resolution for the resistance ranges (typically 5 bits or better), and this can result in significant savings in program memory requirements, that occupied by the ScanSensor routine.

The reference design firmware is written using the industry-standard 8051 instruction set.

Conversion to other microprocessors using this basic instruction set should be straightforward. Conversion to processors with substantially different instruction sets should be a straightforward re-coding project, given the excellent internal documentation in the reference design source code.

Hardware Help

An embodiment of the invention provides "hardware help" for detecting and calibrating a "first touch." Some pointing devices have no dome switches or other methods for detecting a "click" via increased pressure on the fingertip actuator. These systems typically implement one or more "click" switches via separate, external switches. With devices constructed in this manner, the device may not be in "presence" at all at the time of the click. Initial experimentation had involved the use of "first touch", which was to use the X-Y values detected at the transition from out-of-presence to presence operation. However, this system proved to be vulnerable to false centering when the user attempted to initiate pointing with immediate, fast movement. In such activity, the first point of contact typically represented a location significantly far away from the preferred "null point" Close observation revealed that this type of activity (called "hitting the decks running") could be not be easily distinguished from initiating the pointing with an intent to have no motion. We thus observed two distinctly different types of first-touches: "hitting the deck running", and (for lack of a better name) "Ninja landing". What we did observe was that all "Ninja landings" occurred at the user's preferred null location, and this point was always with the R² contact point very near the geometrical center of the contact area (Fig. 5). A "hit the decks running" first-touch looked more like Fig. 6. Note that no part of the contact point in Fig. 5 is particularly near the geometrical center of the contact area. This lead us to the "hardware help" improvement on "first touch", which is shown in Fig. 7. Note that there is a special contact point at the center of the contact area. When a "first touch" also involves contact with this contact point, we are assured that this "first touch" X-Y is a suitable location for auto-calibration. The size of the contact point covers a range of possible first touches adequate to correct for variations in hand placement and finger placement, but is not large enough to confuse "hit the decks running" with "Ninja" landings. Reference should be made to the source code and other documentation provided as reference materials with this application. Specifically, the source code in several files under the subdirectory "fth-2btn" shows in detail those algorithms used to implement first touch with hardware help a preferred embodiment of the present invention.

Note that the auto-calibration aspects of the present invention shield the user from having to know about, or to take active concern, for auto-calibration.

For switched versions of the device, we note the following. If the user has just turned on a system that is somewhat out of calibration, the natural behavior will be to position the cursor above a target with some difficulty (because of the "drift"), release the device "out of presence", and then make a decisive click. This will automatically calibrate the device. The system requires only a few extra tens of bytes of code to implement this approach. The system must maintain state variables to track the state as "in presence" and "out of presence". The system must measure the time between entering "presence" and the time of the click, and compare it to a minimum amount of time.

For unswitched versions, the system requires an extra contact point and surrounding open area on the contact area as shown in Fig. 7. A microprocessor port (perhaps shared) is needed to "watch" for contact on the centering contact point.

Thus, the invention has been described with reference to particular embodiments thereof. However, variations from the disclosed specific embodiments are possible. The embodiments are but illustrative, and not restrictive, of the invention. The scope of the invention is to be determined solely by the appended claims.

$title (FTH-2btn: first-touch with hardware assist for non-switch centering+ 2 button support) $include (macros. inc) $include (cpyrite. inc) ******************************************************

Application constants:

; Select mouse emulation mdde

PS2MOUSE equ 0 PS/2 Mouse

MSMOUSE equ 1-PS2MOUSE Microsoft Serial Mouse

; Initial PS/2 mouse status PS2INITSTAT equ OOOh Disabled

; Select push-down switch option (set to number of scans)

; NSC NSC *************turn off pushdown for the duration of this test***********NSC NSC

PUSHDOWN equ 0 ; Have push-down switch option selected

; Auto center bad rubber

AUTOCENTER EQU 1 ; Auto center bad rubber

; NSC NSC *************turn off presence for the duration of this test***********NSC NSC

PRESENCE EQU 0 ; Max number of scans to be in presence

^'; Select menu navigation option.

NAVIGATEMENU EQU 0 ; Assist with menu navigation

; Select axis snap-to option

SNAPTOGRID EQU 1 ; Enable snap-to grid option

Enable debug output for various subroutines

PD_SENSE EQU 0 ; Print debug messages for Sense ()

PD_SCANSENSOR EQU 0 ; Print debug messages for ScanSensorO

PD_GETSYSDATA EQU 0 ; Print debug messages for GetSysData ()

PD_PUTSYSDATA EQU 0 ; Print debug messages for PutSysDataO

PD_CMD_FF EQU 0 Print debug messages for CMD_FF code

PD_SS_ABORT EQU 0 Print debug messages for SS_ABORT code

PD_ABORT EQU 0 Print debug messages for ABORT code

PD TRYERROR EQU 0 Print debug messages for TryError code

; Various support routines necessary for debugging

PD_SUP1 EQU PD_SENSE+PD_SCANSENSOR+PD_GETSYSDATA+PD_^PUTSYSDATA

PD_SUP2 EQU PD_CMD_FF+PD_SS_ABORT+PD_ABORT+PD_TRYERROR

PD SUPPORT EQU PD SUP1+PD SUP2

Sanity checks of configuration options

IF MSMOUSE AND PS2MOUSE ;You cannot have both mouse emulation modes selected. Choose one or the other. ENDIF

IF NOT MSMOUSE AND NOT PS2MOUSE ;You must select at least one of MSMOUSE or PS2MOUSE emulation modes. ENDIF

$include (history. inc)

; ??-Nov-1998 dhs Prototyped first-touch hardware assist

; Ol-Dec-1998 dhs Added/Fixed support for mouse buttons

$include (connect . inc) ******************************************************

External Connections

PI 0 12 RV Reference Voltage

PI 1 13 uv - Unknown Voltage

PI 2 14 We - West Enable

PI 3 15 Se - South Enable

PI 4 16 Ee - West Enable

PI 5 17 Ne - East Enable

P3 2 06 B2 Left Joystick Button

P3 4 08 Bl Right Joystick Button pi 7 19 Hardware help contact for autocentering

P3.0 02 MC - PS/2 Mouse Clock P3.1 03 TxD - RS-232 Transmit Data P3.3 07 MD - PS/2 Mouse Data P3.5 09 RTS - RS-232 Request To Send

P3.7 11 RC - Reference Control

****************************************************** $EJECT

$INCLUDE (8051. def)

$include (pins. inc)

$include (asciis.inc) IF PS2MOUSE StatBl EQU 20h Mouse Status

StatRM EQU OOh Right Mouse Button

StatLM EQU 02h Left Mouse Button

StatScale EQU 04h Mouse Scaling

StatEnabled EQU 05h Mouse Enabled/Disabled

StatRemote EQU 06h Mouse Remote Mode

StatB2 EQU 21h ; Mouse Resolution StatB3 EQU 22h ; Mouse Sampling Rate ENDIF

MiscFlags EQU 23h

InPresence EQU 18h ; Are we in Presence? (rubber incontact) HaveAutoCtrd EQU 19h Have we auto centered yet?

XCoord EQU 24h ,- X-Coordinate

YCoord EQU 25h ; Y-Coordinate

OldButtons EQU 26h ,- Old button status

IF PD SUPPORT+MSMOUSE

To add a character, put into ©XmtlnPtr and then increment ptr.

To remove a character, get from ©XmtOutPtr and then increment ptr.

If (XmtInPtr++ == XmtOutPtr) then buffer is full-

XmtBusy EQU 27h Is the transmitter busy?

XmtOutPtr EQU 28h Pointer to data to transmit

XmtlnPtr EQU 29h Pointer, to data to insert

ENDIF

OnLine EQU 2ah Current state of joystick-computer link

DelayTime EQU bh Amount of delay between scans

Distance EQU 2ch Distance formed by X & Y

OldDistance EQU 2dh Distance formed by X & Y

OldXCoord EQU 2eh Old X-Coordinate

OldYCoord EQU 2fh Old Y-Coordinate

OutXCoord EQU 30h Output X-Coordinate

OutYCoord EQU 31h Output Y-Coordinate

IF PUSHDOWN

PushNum EQU 32h Number of pushes at current level

PushVal EQU 33h Output value

PushReset EQU 34h Reset output values?

PushWait EQU 35h Wait until next pass to do something

PushLatch EQU 36h Are we latched?

PushCntLatch EQU 37h ; Number of passes to latch

ENDIF

IF AUTOCENTER

XOffsetH EQU 38h Offset from center

XOffsetL EQU 39h

YOffsetH EQU 3ah

YOffsetL EQU 3bh

IF PRESENCE

PresenceCount EQU 40h Presence count (note address skip!)

ENDIF

ENDIF

RXCoord EQU 3ch

RYCoord EQU 3dh

ORXCoord EQU 3eh

ORYCoord EQU 3fh

IF PD_SUPPORT+MSMOUSE XmtAddr EQU 48h ; Start of transmitted characters buffer XmtBufMaxLen EQU 3 Oh ; Size of transmit buffer

ENDIF

$EJECT $include (intrpts . inc) ****************************************************** main - main loop

mam:

Perform basic system initialization

mov iec,#080h ; allow all interrupts, ; however individually, they're off

Setup RS-232 link and Timer 0 wakeup timer.

Setup port for 1200 baud, mode 1 (10-bit characters) .

mov sp,#7 ; Reset stack mov psw,#0 ; Clear PSW - choose register bank #0

IF PD_SUPPORT mov pcon,#80h ; Double baud rate (SM0D=l) . Rest 0. mov pcon,#00h ; 9600 buad

ELSE mov peon, #0 ; SMOD, GF1, GF0 , PD, IDL all are 0

ENDIF

IF PD_SUPPORT mov tll,#0 mov thl,#253 9600 baud (11.0592 MHz)

ELSE

IF MSMOUSE mov tll,#0 mov thl,#230 1200 baud (12.0 Mhz) mov thl,#232 1200 baud (11.0592 Mhz)

ENDIF

ENDIF mov tmod,#21h ; Timer 1: Mode 2; Timer 0: Mode 1 setb trl ; Timer 1 on mov scon, #050h Mode 1, Enable Reception mov scon, #040h Mode 1, Disable Reception mov pl,#0ffh mov p3,#07fh ; Allow Txd and Rxd functions of P3 clr itO ,- Use level for into mov ipc,#000h ; No distinction in priority

Buffer management

Clear all memory from 20h to 80h mov rl,#20h ^• start mov r0,#80h-20h count

II: mov @rl,#0 Clear memory inc rl dj nz rO , Il ; and loop

IF PD_SUPPORT+MSMOUSE mov XmtOutPtr, #XmtAddr mov XmtlnPtr, #XmtAddr

; 0 mov XmtBusy,#0

ENDIF

; 0 mov OnLine,#0 ; Clear OnLine state not online

;0 mov XCoord,#0 Something safe

;0 mov YCoord,#0

;0 mov OldXCoord,#0 ; Something safe

;0 mov θldYCoord,#0

;0 mov OutXCoord,#0 ; Something safe

;0 mov OutYCoord,#0

;0 mov OldDistance,#0

;0 mov θldButtons,#0 ; Assume no buttons are pushed

IF AUTOCENTER

;0 mov XθffsetH,#0 mov XθffsetL,#080h

;0 mov YθffsetH,#0 mov YθffsetL,#080h

IF PRESENCE mov PresenceCount , #PRESENCE

ENDIF

ENDIF

IF PS2MOUSE mov StatB3,#100 ; Sampling Rate mov StatB2,#4 ; Resolution mov StatBl, #PS2INITSTAT ; Basic Status

ENDIF

IF PUSHDOWN mov PushNum, #PushDownCode

0 mov PushVal,#0 ; Initial output value

0 mov PushReset,#0 ; Do not reset output coordinates

0 mov PushWait,#0 ; Wait until next output to change

0 mov PushLatch,#0 ; Are we latched?

0 mov PushCntLatch,#0 ; Number of passes so far

ENDIF

IF MSMOUSE+PD_SUPPORT ; case when we need serial interrupts setb es ; turn on serial interrupts

ENDIF

$EJECT ******************************************************

MainLoop - this is where the primary work is done

IF PS2MOUSE ljmp PS2Init ENDIF IF MSMOUSE MainLoopTst : mov OnLine,#0 ; If here, then we cannot be online

MainLoop :

Check for Ident request - RTS low

jb RTS, MainLoopTst ; (2) RTS asserted? No - Jump

; RTS asserted... mov a,θnLine ; (1) Have I been here before? cjne a,#0,HaveRTS ; (2) Yes - jump: no need to send ident lcall Delay50ms ; Wait 50 msec mov OnLine,#l ; Flag that I am a MS Serial Mouse mov a,#CharM ; Ident response lcall Xmtlnsert lcall OutputBuffer ; Go send it sjmp MainLoop

HaveRTS : lcall ScanSensor

$EJECT ******************************************************

Output Coordinates based upon coordinates and buttons

If the coordinates are not zero, or if the buttons are not the same as before, then output the values. If the buttons are the same and the coordinates are zero, then output one value of zero to stop the host. If this goes on long enough, then go into IDLE mode.

Emulate a Microsoft mouse

; Check current button state - see if it has changed

IF PUSHDOWN mov PushWait,#0 ; Indicate we have output status change

ELSE ; NSC NSC ************NSC NSC

; use p3.2 temporarily mov a,#0 mov c, p3.2 ; (1) Get outside button status mov ace .7 , c mov c, p3.4 ;get the other button is this the right place to get it? mov acc.6,c ;is this the right place to put it? ; mabe complement c if we need to mov b , a ; ( 1) Save for a moment mov a, 01dButtons ; ( 1 ) anl a, #0f0h ; (1) Mask to Keep only old buttons swap a ; ( 1) orl a,b ; (1) Get new value mov 01dButtons,a ; (1) Save for later processing

ENDIF

IF PD_SCANSENSOR+PD_SENSE mov OutXCoord,#0 ; Clear output that has been sent mov OutYCoord,#0 sjmp DoNotSend

ENDIF mov a,01dButtons Get old and new button status

IF PUSHDOWN orl a,PushLatch ; Output left button if we are latched

ENDIF anl a,#0c0h ; Get new status mov b,a ; Hold for a moment mov a.OldButtons ; Get old and new button status

IF PUSHDOWN orl a,PushLatch ; Output left button if we are latched

ENDIF swap a ; Position old status correctly anl a,#0c0h ; Get old status cjne a,b,θk2Send ; (2) Change - go send the new coord. mov a, OutXCoord orl a,θutYCoord ; (1) jnz Ok2Send ; (2) Something there - go send it

; IDLE and ZERO check here sjmp DoNotSend ,- (2) Nothing to send now

****************************************************** If we are here, then a button event or joystick movement has occurred.

Ok2Send:

Process the buttons

There are two buttons : Left and right .

mov a,01dButtons ; (1) Get current buttons

IF PUSHDOWN orl a,PushLatch ; Output left button if we are latched

ENDIF rr a ; (1) Shift buttons to correct positions rr a ; (1) anl a,#30h ; (1) Get Left and Right button value mov b,a ; (1) Save for later insertion

; Process coordinates mov , OutXCoord rl a (1) Get upper 2 bits rl a (1) anl a,#03h ; (1) Keep only useful stuff orl a,#40h ; (1) Part of protocol orl a,b (1) Fold in buttons mov b, a ; (1) save for a moment mov a,θutYCoord ; ⁽1⁾ swap a ; (1) anl a,#0ch ,^• (1) Keep only useful stuff orl a,b ; (1) Create first character orl a,#80h ; (1) Simulate 7-bit data lcall Xmtlnsert ; (35+2) lcall StartOutputBuffer ; (32+2) mov a, OutXCoord (1) get rest of data anl a,#3fh (1) orl a,#80h (1) Simulate 7-bit data lcall Xmtlnsert (35+2) mov a,θutYCoord (1) get rest of data anl a,#3fh (1) orl a,#80h (1) Simulate 7-bit data lcall Xmtlnsert (35+2) lcall StartOutputBuffer ; (32+2) mov OutXCoord, #0 ; Clear output that has been sent mov OutYCoord,#0

DoNotSend:

; At this time xxxx cycles have passed since MainLoop

; At 11.0592 Mhz, this equates to 5.600 msec. mov rl,DelayTime ; Get delay time MGOS: mov a,rl ; Save current delay mov r2 , a mov rl,#l ; Delay 1 msec lcall Delaylmsec ; do the delay

IF PUSHDOWN lcall CheckPushDown ; Check button status ENDIF mov a, r2 mov rl , a djnz rl,MGOS ljmp MainLoop

ENDIF ;END OF MSMOUSE IF BLOCK

$EJECT

$include (outputbuffer .inc) $include (delays. inc) $include (ps2stuff . inc)

G:\HOME\CJK\varatouch\fth-2btn\fth-2bta.BAK ******** *** ******** * *********** ****** * *** ********** ***

General operation:

For '125 buffer version:

For each corner, enable its driver and sense the value by a call to Sense () . Clear the X and Y values. The Sense () routine will indicate a value which corresponds to the position of the joystick relative to that corner (N, S, W or E) .

For non-'125 buffer version:

The joystick driver is on. This signal goes to each of the corners and is sensed. When the value exceeds the TTL threshold, the signal changes state from low to high. The timing of this change indicates to actual resistance/deflection. All four corners are sensed at the same time .

For all versions :

After the 4 values are found, the location is calculated. This is done by simply adding the opposite directions to determine a value for that axis. For instance, a North value of 5 and a South value of 2 create a Y value of (-3) (Y = S - N, -3 == 2 - 5) . The X axis is determined by subtracting West from East (X = E - W) . After the X and Y values are determined, they are then translated into scaled valued based upon a LUT. These values are output to the host computer.

The diagonals are sensed to add more directional capabilities. Currently, the only directions are N, S, E, W and the two diagonals. By adding diagonal sensors, eight additional directions are added. For instance, sensing the north sensor means that we have north deflection. If east is also sensed, that means we are on the diagonal. However, if north is sensed, but east is not and the diagonal sensor is activated, this means that we are 22.5 degrees off of north.

ScanSensor : mov 01dXCoord,XCoord ; Save old values mov OldYCoord, YCoord

mov a,#ScanNorth ; (1) lcall Sense ; (1192+2) Go do the work jc Not_in_Presence ; Skip scan if not active mov a,#ScanNorth ; (1) lcall Sense ; Do it again because sensor must settle mov RYCoord,a ; (1) Save current Y value mov a,#ScanSouth ; (1) lcall Sense ; (1192+2) Go do the work jc Not_in_Presence ; Skip scan if not active clr c ; (1) subb a,RYCoord ; (1) Determine actual Y Coordinate mov RYCoord,a ; (1) Save coordinate mov a,#ScanWest ; (1) lcall Sense ; (1192+2) Go do the work jc Not_in_Presence ; Skip scan if not active mov RXCoord,a ; (1) Save current X value mov a,#ScanEast ; (1) lcall Sense ; (1192+2) Go do the work jc Not_in_Presence ; Skip scan if not active clr c ; (1) subb a,RXCoord ; (1) Determine actual X Coordinate mov RXCoord,a ; (1) Save coordinate mov XCoord, RXCoord mov YCoord, RYCoord

IF PD_SENSE lcall PrtCRLF lcall OutputBuffer ENDIF

IF PD_SCANSENSOR ; RX & RY mov a, # ' R ' lcall DumpXY lcall OutputBuffer ENDIF jmp Are_In_Presence

Not_in_Presence : debug ' N ' clr InPresence IF PUSHDOWN mov PushReset,#5 ; Indicate: reset output coordinates ENDIF mov OutXCoord, #0 mov OutYCoord,#0 jmp gohome

Are_In_Presence : debug ' P ' jnb InPresence, FirstTouch jmp NotFirstTouch

FirstTouch: setb InPresence debug ' F ' debug ' t ' IF AUTOCENTER ; Check button status - if there, then leave the transducer charged mov a, pi ; Get button status jnb acc7,Center_Tchd ; Not pushed - turn off transducer jmp No_Ctr_Tch Center_Tchd: debug ' C ' debug ' t ' ; calculate new offset

; if we have never click-autocentered yet - take this as absolute, otherwise rotating average

,- One might ask, why use a whole byte for a bit flag? It was easy. If I can figure out how to ; do it with a single bit, I'll change it. debug ' 1 ' jb HaveAutoCtrd, HaveACdB4 ,- if we got here, we've never click-centered before, let's take this value as golden clr c mov a, RXCoord mov XOffsetH,a mov a, RYCoord mov YθffsetH,a mov a,#0 mov XθffsetL,a mov YθffsetL,a setb HaveAutoCtrd ,-now that we've done it, remember jmp Have_Centered HaveACdB4 : ; if we got here, we have click-centered before, take rotating average debug ' 2 ' clr c mov a, RXCoord Get current coordinate subb a,XOffsetH mov r0,#6 lcall LShiftAB add a,XOffsetL calculate new offset mov XθffsetL,a mov a,XOffsetH addc a,b mov XθffsetH,a clr c mov a, RYCoord Get current coordinate subb a,YOffsetH mov r0,#6 lcall LShiftAB add a,YOffsetL calculate new offset mov YθffsetL,a mov a,YOffsetH addc a,b mov YθffsetH,a

Have_Centered :

IF PD_SCANSENSOR ; OX & OY mov a, # ' O ' lcall Xmtlnsert mov a,#' ' lcall Xmtlnsert mov a,XOffsetH lcall PrtHex2 mov a,XOffsetL lcall PrtHex2 mov a,#CharSlash lcall Xmtlnsert mov a,YOffsetH lcall PrtHex2 mov a,YOffsetL lcall PrtHex2 mov a, #CharSpace lcall Xmtlnsert lcall OutputBuffer

ENDIF jmp AsNotFirstTouch all done with first touch processing

No_Ctr_Tch: debug 'N' debug 'C debug ^•t'

ENDIF

NotFirstTouch:

; offset current coordinates debug 'N' debug ^•f ' debug ^•t'

AsNotFirstTouch:

IF AUTOCENTER mov a, XCoord ; G Geet current coordinate value clr C subb a,XOffsetH mov XCoord, a mov a, YCoord clr c subb a,YOffsetH mov YCoord, a

ENDIF mov pl,#Scanθff ; (1) Turn off all sensors

,- KILROY - rescale coordinates

mov a, XCoord lcall MkPos clr c subb a,#XYScale jc CheckY

,- We have exceeded valid values for X - limit them to XYScale mov a, XCoord jb acc7,NegX mov XCoord, #XYScale-l sjmp CheckY NegX: mov XCoord, #-XYScale+l

CheckY : mov a, YCoord lcall MkPos clr c subb a,#XYScale jc CheckDone

; We have exceeded valid values for Y - limit them to XYScale mov a, YCoord jb acc7,NegY mov YCoord, #XYScale-l sjmp CheckDone NegY: mov YCoord, #-XYScale+l

CheckDone :

Print out debug lines of raw distance for each coordinate

IF PD_SCANSENSOR ; QX & QY mov a,#CharQ lcall DumpXY lcall OutputBuffer ENDIF

DiagSkip:

; At this point, more than 4788 cycles have passed since MainLoop

Compute distance for use with delay time

mov OldDistance, Distance ; Save old distance mov a, XCoord ; (1) Get X Coordinate lcall MkPos (6+2) Make positive mov b, a (1) Compute X*X mul ab (4) mov rO , a (1) Save for next step mov rl,b (1) mov a, YCoord (1) Get Y Coordinate lcall MkPos (6+ 2) Make positive mov b,a (1) Compute Y*Y mul ab (4) ace has lsb. add a,r0 (1) Sum of X*X+Y*Y mov rO, a (1) mov a,b (1) addc a, rl (1) mov rl,a (1) lcall Sqrt ; (94+2) Compute square root of rO/rl into ace mov Distance, a ; (1) Save for later

Print out distance value

IF PD SCANSENSOR

Dxx mov a,#CharD lcall Xmtlnsert mov a, Distance lcall PrtHex2 mov a,#CharSpace lcall Xmtlnsert lcall OutputBuffer mov a, Distance ; Restore value for DelayVect ENDIF lcall DelayVect ; (5+2) Get Delay vector mov DelayTime,a ; (1) Store delay time

Print out delay time

IF PD SCANSENSOR

; Mxx mov a , #CharM lcall Xmtlnsert mov a,DelayTime lcall PrtHex2 mov a,#CharSpace lcall Xmtlnsert lcall OutputBuffer ENDIF IF SNAPTOGRID mov a, XCoord lcall MkPos mov rO , a mov a, YCoord lcall MkPos mov rl,a clr c subb a,r0 ; calculate Y-X jc SnapBigX

SnapBigY: lcall GetSnapToGridPercentage mov b,rl ; Get Y mul ab mov a,r0 ; Get X clr c subb a, b Is X bigger than percentage of Y? jnc Snap99 ; NC - X is big enough mov XCoord, #0 ; Snap to axis sjmp Snap99

SnapBigX : lcall GetSnapToGridPercentage mov b,r0 ; Get X mul ab mov a,rl ; Get Y clr c subb a,b ; Is Y bigger than percentage of X? jnc Snap99 ; NC - Y is big enough mov YCoord, #0 ; Snap to axis

Snap99:

ENDIF

IF NAVIGATEMENU

Handle menu navigation option

Menu navigation limits variability of coordinates to one of eight directions based upon the X/Y coordinate. This is to assist the operator in moving around through a menu. The assumption is that if the button is not pushed or latched, then we are navigating (versus drawing) and some assistance can be offered.

IF PUSHDOWN mov a,PushLatch ; Get button status

ELSE mov a, pi ; (1) Get outside button status cpl a ; (1) Invert status

ENDIF anl a,#0f0h ; Get button status jnz NoNavigation ; Button pushed - no navigation lcall NavigateVect ; get menu navigation control jz NoNavigation

Get absolute values of X and Y. Cases are:

IF X > Y THEN

IF X-Y > Y THEN

SNAP TO X ELSE

SNAP TO X/Y DIAGONAL (X == Y) ENDIF ELSE

IF Y-X > X THEN

SNAP TO Y ELSE

SNAP TO X/Y DIAGONAL (X == Y) ENDIF ENDIF

mov a, XCoord lcall MkPos mov rO , a mov a, YCoord lcall MkPos mov rl,a clr c subb a,rO ; Is X > Y ? jc NavBigX

NavBigY :

; ace has Y-X. Compare to X clr c subb a,rO jc NavYDiag ,^• Force Y-dominate diagonal

NavYAxis : mov XCoord, #0 ; Lock to Y axis sjmp Nav99

NavYDiag: mov a, Coord ; What is current sign of X? jb acc7,NavYDXM ; Negative - jump mov XCoord, rl ; Get positive Y sjmp Nav99 NavYDX : clr a clr c subb a,rl mov XCoord, a Get negative Y sjmp Nav99

NavBigX : mov a,r0 clr c subb a,rl ; compute X-Y clr c subb a,rl jc NavXDiag ,- Force X-dominate diagonal

NavXA is : mov YCoord, #0 ; Lock to X axis sjmp Nav99

NavXDiag: mov a , YCoord ; What is current sign of Y? jb acc7,NavXDYM ; Negative - jump mov YCoord, rO Get positive X sjmp Nav99

NavXDYM : clr a clr c subb a,r0 mov YCoord, a Get negative X sjmp Nav99

Nav99 :

NoNavigation: ENDIF

Calculate deceleration offset

mov a,θldDistance ; Get old distance clr c subb a, Distance ; Subtract current distance jc NoDecel ; no carry - positive acceleration jz NoDecel ; No acceleration - skip deceleration lcall SlowVect ; Get scalar push ace ; Save for Y Coordinate mov b,a ; Save scalar mov a,θldXCoord ; Get Coordinate lcall MkPos ; Make positive mul ab ; Compute offset: B has value mov a,θldXCoord ; Get old value jnb ACC7,θkX ; Was old coordinate negative? clr a Get a zero subb a,b Invert B mov b, a Put back

OkX: mov a, XCoord ; Get new XCoordinate clr c ; Prepare for subtraction subb a,b ; Adjust XCoord... mov XCoord, a ; ...and save. pop Get Scalar again for Y Coord mov a,θldYCoord ; Get Coordinate lcall MkPos ; Make positive mul ab ; Compute offset: B has value mov a,θldYCoord ,- Get old value jnb ACC7,θkY ; Was old coordinate negative? clr a Get a zero subb a,b Invert B mov b, a Put back

OkY: mov a, YCoord ; Get new YCoordinate clr c ; Prepare for subtraction subb a,b ; Adjust YCoord... mov YCoord, a ; ...and save.

Print out each decelerated coordinate

IF PD SCANSENSOR BX & BY mov a,#CharB lcall DumpXY lcall OutputBuffer ENDIF

NoDecel :

Translate each coordinate based upon rate table

mov a, XCoord (1) Remap X coordinate lcall Translate (15+2) Translate into rate scale mov XCoord, a (1) Save coordinate mov a, YCoord (1) Remap Y coordinate lcall Translate (15+2) Translate into rate scale mov YCoord, a (1) Save coordinate mov a, OutXCoord add a, XCoord mov OutXCoord, a mov a,θutYCoord add a, YCoord mov OutYCoord,a

Print out eac .translated coordinate

IF PD_SCANSENSOR ; TX & TY push XCoord push YCoord mov XCoord, OutXCoord mov YCoord, OutYCoord mov a,#CharT lcall DumpXY pop YCoord pop XCoord lcall OutputBuffer ENDIF

IF PUSHDOWN mov a,PushReset ; Should I reset output coordinates? jz PushNoReset add a,#-l ,- Decrement reset count mov PushReset,a ; Save for next pass mov OutXCoord, #0 ; Do not let the switch move while pushing mov OutYCoord, #0 PushNoReset :

IF PD_SCANSENSOR ; ZX & ZY push XCoord push YCoord mov XCoord, OutXCoord mov YCoord, OutYCoord mov a,#CharZ lcall DumpXY pop YCoord pop XCoord lcall OutputBuffer ENDIF

ENDIF

IF PD_SCANSENSOR lcall PrtCRLF lcall OutputBuffer

ENDIF gohome : ; At this point, more than 4974 cycles have passed since MainLoop ret

$EJECT

$ nclude sense . inc) $include curvemap . inc) $include translate. inc) $include lshiftab. inc) $include ps2scale. inc) $include sqrt.inc) $include mkpos . inc) $include checkpushdown. inc) $include dumpxy. inc) $include xmitonechar. inc) $include xmtinsert . inc) $include prthex2. inc) $include prtcrlf . inc) $include watermark. inc) $include delaytables . inc) END

Claims

WHAT IS CLAIMED IS: 32

1. A method for performing auto-calibration in a relative pointing device for a computer user interface, the system comprising the steps of detecting whether or not the user is indicating a selection with the device by sensing a change in state from an out of presence state to an in presence state; and using the detected indication to calibrate the device.

2. The method of claim 1, further comprising the steps of creating a profile of readings of the device for a plurality of detections; and using the profile to automatically calibrate a zero-position for the device.

3. The method of claim 2, further comprising the step of augmenting profiles of user use with special hardware features.

4. The method of claim 1, further comprising the step of automatically calibrating for finger geometry.