EP0861485A1 - Multi-touch input device, method and system that minimize the need for memorization - Google Patents

Multi-touch input device, method and system that minimize the need for memorization

Info

Publication number
EP0861485A1
EP0861485A1 EP19960941390 EP96941390A EP0861485A1 EP 0861485 A1 EP0861485 A1 EP 0861485A1 EP 19960941390 EP19960941390 EP 19960941390 EP 96941390 A EP96941390 A EP 96941390A EP 0861485 A1 EP0861485 A1 EP 0861485A1
Authority
EP
Grant status
Application
Patent type
Prior art keywords
touch
method
texmal
apparams
digit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP19960941390
Other languages
German (de)
French (fr)
Inventor
Michael J. Ure
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ure Michael J
Original Assignee
Michael J. Ure
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • G06F3/0235Character input methods using chord techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for entering handwritten data, e.g. gestures, text
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041Indexing scheme relating to G06F3/041 -G06F3/045
    • G06F2203/04104Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04808Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen

Abstract

A multi-touch input device, method and system are provided that minimize the need for memorization on the part of the user. Information is input to an electronic device (520, 620) using a pad (510, 610) having a pad surface (300), which pad distinguishes multiple simultaneous touches. Input steps includes: forming marking in relation to the pad suface, the markings including textual elements; a user, with the user's hand, touching one digit of the hand to a first area of the pad surface bearing a marking corresponding to desired textual element that the user desires to input and, at substantially the same time, touching another digit of the hand to a second area of the pad surface identified by the user by its occupying a predetermined position relative to the first area; detecting the first and second areas touched by the user; and inputting the desired textual element.

Description

MULTI-TOUCH INPUT DEVICE, METHOD AND SYSTEM THAT MINIMIZE THE NEED FOR MEMORIZATION

The present invention relates to input devices for electronic devices such as computers, and more especially to multi-touch input devices. With the rapid advance of computer and electronic technologies in recent years, the limitations of the computer keyboard as the principal input device for computers and electromc devices have become increasingly apparent. As a result, increasing attention has and will be turned to alternative input devices.

A promising technology for realizing multi-touch input devices is that of capacitive touchpads. Touchpads have been popularized by such companies as Cirque Inc. , Apple Computer Inc. , Alps Electronics USA Inc., Synaptics Inc. , and, most recently, Logitech Inc. Approximately 80% of newly manufactured laptop computers now include a touchpad.

Within the touchpad industry, there exists a trend toward having touchpads do more than just point. Concurrently, others in the computer industry have recognized the need for a new input device, and have further recognized the potential of gesture (hand motions richer in information than the simple finger strokes used in keying) to fill that need. An example of the need for a new input device is provided by the recent emergence of text entry systems using the telephone keypad in which a key bearing multiple letters is pressed, after which 1 , 2 or 3 is pressed to specify the first second or third letter on the key. A variant is to press the same key once, twice or three times to designate the first, second or third letter on that key.

Of particular interest in relation to possible new input devices is the second-generation TP2™ touchpad sold by Logitech. With relatively minor firmware modifications, the touchpad may be made to provide raw capacitance measurement data to the outside world. This raw data may be manipulated in various ways to realize enhanced functionality. Also of interest are various display-based touch sensor devices such as that described in U.S. Patent 5,194,862, incorporated herein by reference. One existing alternative is a prior-art multi-touch input system known as the DatO™ input system. The prior-art input system has the objective of realizing a very compact, touch-sensitive input device for inputting to digital electronic devices command/control information and/or text. Compactness is a key consideration for small portable electronic devices. A DatOPad™ input device (described more fully at www.dato.com) is a touchpad or other device that uses the prior-art DatO input system. The prior-art input system is described in U.S. Patent 5,203,704, incorporated herein by reference. One embodiment of a prior-art input device has a layout 100 as shown in Figure 1. The index finger ("primary indicator") touches within a region 101, pressing one of areas 1-9. The thumb ("first ancillary indicator") touches within a region 103 if applicable. The middle finger ("second ancillary indicator") touches within a region 105 if applicable. For each position 1-9 of the index finger, there are four possible combinations of the "ancillary indicators "-thumb only, middle finger only, both, or neither- for a total of 36 distinct gestures. Further in accordance with the prior-art input system, a DatOSet™ gesture set is a set of eight groups of 36 gestures, different groups being selected by tapping once or twice with different combinations of the ancillary indicators only, for a total of 288 mode-distinguishable gestures. A gesture set may be thought of as a set of "fonts," for example. Each font may in turn be thought of as four "tic-tac-toe" grids overlaid on top of one another. For purposes of illustration, the tic-tac-toe grids may be drawn side-by-side, whereby an "alpha" font, for example, might be represented as shown in Figure 2. Within each tic-tac-toe grid, each box corresponds to one ofthe primary indicator positions numbered 1-9.

During entry of a word, the index finger remains in contact with the pad, sliding from numbered position to numbered position. A space is indicated by lifting. No details concerning concerning whether or how case or punctuation might be provided for are currently available. In accordance with the prior-art input system, neglecting case and punctuation, the sentence "Gee, this is a great new input device! " would therefore be input as follows, where Xs are used to indicate the required action for each character and MF indicates the middle finger: TABLE 1.

Slide Press Land Land Lift Lift Lift to Area thumb MF thumb MF all g 7 e X 5 e 5

X t 2 X h X 8 X i X 9 s X 1 X

X i 9 s X 1 X

X a 1

X g 7 r X 9 X e X 5 X a X 1 t X 2 X

X n 5 X e 5 X w 5 X

X i 9 n X 5 X

P X 7 u X 3 X X t X 2

X d 4 e X 5

V X 4 X i X 9 X c X 3 e X 5

In the prior-art input system, as has been explained, the position of the index finger is important, but only the presence or absence (not position) of the thumb and index finger is important. In one respect, this allows the prior-art input system to be used with a very small device. This assumes, however, that the user has memorized the mapping between gestures and characters/commands, or if not, that the user refers to some external reference such as a display or reference card. To display the mapping would require additional display area. Referencing a reference card is an inconvenience to the user.

Hence, despite the potential of the prior-art input system, there remains a need for an improved input system, particularly an input system that minimizes the need for memorization on the part of the user.

The present invention, generally speaking, provides such a multi-touch input device that minimizes the need for memorization on the part of the user. In accordance with one aspect of the invention, a series of steps are followed to input information to an electronic device using a pad having a pad surface, which pad distinguishes multiple simultaneous touches. The steps include: forming markings in relation to the pad surface, the markings including textual elements; a user, with the user's hand, touching one digit of the hand to a first area of the pad surface bearing a marking corresponding to desired textual element that the user desires to input and, at substantially the same time, touching another digit of the hand to a second area of the pad surface identified by the user by its occupying a predetermined position relative to the first area; detecting the first and second areas touched by the user; and inputting the desired textual element. In accordance with another aspect of the invention, an input device for inputting information to an electronic device includes the following combination of inter-related elements: a touchpad surface formed in relation to a touch sensor array; markings formed in relation to the pad surface, the markings including textual elements; an integrated circuit controller coupled to the touch sensor array for receiving capacitance measurement data from the touch sensor array; and processing means for detecting a particular touch pattern in which a plurality of the areas on the touchpad surface are touched simultaneously and for, in response to the particular touch pattern, signalling for input to the electronic device a textual element corresponding to the particular touch pattern. The markings used in the present input system, preferably and advantageously, are a superset of those of the common telephone keypad, enhancing familiarity and lean-ability. Other aspects of the invention are set forth in the appended claims.

Figure 1 is a diagram of an exemplary prior art input device; Figure 2 is a diagram of a gesture group within a potentially larger gesture set of the input device of Figure 1 ;

Figure 3 is a diagram of the layout of an exemplary input device in accordance with the present invention;

Figure 4 is a simplified schematic representation of a second-generation touchpad that may be used in the present invention; Figure 5 is a block diagram of a first embodiment of an input device in accordance with the present invention;

Figure 6 is a block diagram of a second embodiment of an input device in accordance with the present invention; Figure 7 is a more detailed block diagram of the processing means of

Figure 5 and Figure 6;

Figure 8 is flow diagram of one-touch processing performed by the processing means of Figure 7;

Figure 9 is flow diagram of two-touch processing performed by the processing means of Figure 7;

Figure 10 is a flow diagram of one-touch touch-inflection processing that follows the processing of Figure 8; and

Figure 11 is a flow diagram of two-touch touch- inflection processing that follows the processing of Figure 9.

The present input device, method, and system is marketed under the tradename TracKey™, referring to the dual capabilities of tracking a pointer finger as in conventional touchpads, and also of keying in information. As distinguished from the prior-art input device, in the present input device, two "indicators, " thumb and middle finger (or index finger), are used to form two-touch patterns, or "chords. " The position of both indicators is important. Furthermore, characters/commands are screened or displayed on the touch surface, eliminating the need for memorization or an extemal reference. Preferably, all of the characters/commands are displayed at one time such that there are no separate modes or character groupings. An exemplary layout of the present input device is shown in Figure 3.

As compared to the minimum layout area (primary indicator region only) of about 2.6cm by 2.6cm for the prior art input device, the layout of the present input device occupies about 5.2cm by 3.9cm. However, most or all of the indicia found on a typical computer keyboard are clearly visible. In the present input system, two-touch chords are formed according to simple conventions. For letters, the middle finger touches the key on which the letter appears. The thumb touches the first, second or third key on the row below, depending on whether the letter is the first, second or third on its key. The bottom row of keys has no letters.

For punctuation and commands, the thumb, instead of touching a key in the row immediately below, touches the first, second or third key in a row once (or twice) removed. The first row is paired with the third row, the second row is paired with the fourth row, and the fourth row is paired with the first row. Hence, in the example of Figure 3, to enter "!, " the finger presses 1 and the thumb presses 7; to enter " % ," the finger presses 5 and the thumb presses 0; in the case of the bottom row, to enter "), " the thumb presses # and the finger presses 1.

Numbers are entered using a single touch in the same manner as a touchtone keypad.

Much of the effectiveness of the present input system derives from its use of "touch inflections"— slight touch variations made with minimal additional effort so as to convey additional information. Among the myriad possible touch inflections, the following touch inflections have been found to be particularly useful: "press, pick, roll" and "waggle. " These touch inflections are used as follows:

• "Press a Cap: " Designate an upper case letter by applying greater-than-normal pressure. Current touchpads are able to sense degrees of pressure. In a similar manner, a punctuation symbol in the second tier on a key is designated by applying an increment of pressure.

• "Pick a Space: " When releasing a chord, indicate a subsequent space by causing the finger to stroke the pad slightly in a downward direction. Other similar inflections may be used to indicate common punctuation. These inflection may include, upon release of a chord, stroking the finger slightly upward, stroking the thumb slightly rightward, or stroking the thumb slightly leftward. • "Roll a Digraph " (or a Function Key): Before fully releasing a chord, indicate one of two digraphs beginning with the chorded letter by rolling the finger slightly rightward or leftward. A digraph is a common two-letter combination such as "th. " Alternatively, while applying a single touch to a number 0-9, indicate a corresponding function key by rolling the finger slightly rightward. Indicate a function key corresponding to the number plus ten by rolling the finger slightly leftward.

• "Waggle a Trigraph: " Before fully releasing a chord, indicate one of two trigraphs beginning with the chorded letter by rolling the finger rightward and back or leftward and back.

Other touch pattems are used to provide for returns and tabs. In one embodiment, a return is entered by landing the thumb only. Because of the different size and shape of a "thumbprint" as compared to a "fingerprint", this gesture may be distinguished from a single finger touch used to enter a number. In a further exemplary embodiment, a tab is indicated similarly, for example by landing the thumb but in addition stroking the thumb slightly to the right.

Where a space commonly follows a particular punctuation mark, the space is preferably input automatically. For convenience, two distinct ". " characters are provided for, the traditional period and the now-popular "dot" found in e-mail and Web-page addresses. Preferably, the two are distinguished visually by color, e.g. , black for period and red for dot. In the embodiment of Figure 3, the red dot is located central to the pad on the 5 key. The red dot may be slightly raised, if desired, to provide tactile locality information to the user.

Cursor keys may also be provided for. Referring still to Figure 3, note that small arrows appear along each edge of the device, two arrows along each edge. The arrows coincide with the boundary between adjacent keys. When a single finger is touched on one of the arrows, the device recognizes the touch as a cursor key input. Depending on the direction of the arrow, the cursor is spaced upward, downward, rightward or leftward. Alternatively, depending on the current context of a program being run on an electronic device (for example if no text cursor is currently displayed), the cursor key inputs may be interpreted instead as scroll button inputs, causing the display view to scroll upward, downward, rightward or leftward.

The cursor key input are distinguished within the device from two similar types of inputs. The cursor key inputs are like number inputs in that a single touch is used. In the case of number inputs, however, the centra id of the single touch is required to fall squarely within a key corresponding to a number 0-9. Provision is also made for entering "border" textual elements— elements that will often be input by touch inflection— by independent touch gestures. In Figure 3, these border textual elements include space, period, comma and semicolon. Input of border textual elements directly rather than by touch inflection is accomplished by applying a single touch to the border region of the desired textual element— but not to an arrow. In the example of Figure 3 therefore, a space is entered by touching in the bottom-center border region of one of the star key, the 0 key or the pound key. A period is entered by touching in the top-center border region of one of the keys 1, 2 and 3. A comma is entered by touching in the right-center border region; and a semicolon is entered by touching in the left-center border region.

In a preferred embodiment, an input device in accordance with the present invention is realized through a combination of a standard second-generation (G2) touchpad or touchscreen device and custom driver software. An example of a suitable G2 touchpad is the TP2 touchpad of Logitech.

The basic principles of operation of the TP2 touchpad remain the same as its first-generation predecessors and are described in detail in the patent literature, for example U.S. Patent 5,543,588, incorporated herein by reference. However, whereas first-generation touchpads used extensive analog preprocessing, the TP2 touchpad takes a digital approach. Capacitive measurement data from conductors in an orthogonal array are therefore digitized and input directly to a microprocessor or the equivalent. In the case of a particular touchpad having dimensions of approximately 2 in. by 2 in. , a total of 42 separate capacitance measurements are taken, digitized and input to a microprocessor. The microprocessor uses firmware to perform various smoothing algorithms, tracking algorithms, etc.

Unlike other G2 touch devices such as the touchscreen described in U.S. Patent 5,194,862, for example, the TP2 touchpad does not form a bit-mapped touch image. Although both X and Y touch coordinate data are provided, these data are not correlated in X and Y. Some touch patterns may therefore potentially be confused. Referring to Figure 4, there is shown a simplified schematic representation of a touch sensor array having a multiplicity of conductors extending in each of the X and Y directions. Superimposed on the touch sensor array are two different two-touch touch pattems, one indicated in solid lines and one indicated in dashed lines. At the edges of the touchpad are shown two waveforms, one representative of capacitance variation across the conductors in the X direction and another representative of capacitance variation across the conductors in the Y direction. Note that the two different touch patterns potentially give identical waveforms.

In the case of the present input system, two-touch chords are formed using a finger and the thumb. The amount and spatial distribution of capacitance presented to the touch sensor array by the thumb is appreciably different than the amount and spatial distribution of capacitance presented to the touch sensor array by the finger. A touch by the thumb is characterized by a pulse waveform having an appreciably larger peak and an appreciably wider base. Hence the two different touch pattems may be distinguished in software/firmware, or in hardware if desired.

Referring now to Figure 5, a block diagram of an input device 500 in accordance with a first embodiment of the invention is shown. A touch sensor array 501 is coupled to a microprocessor 503 or equivalent. Together, the touch sensor array and the microprocessor represent a G2 touchpad or touchscreen 510, i.e. , a touchpad or touchscreen in which raw sensor data is digitized and input to a microprocessor where it may be processed or communicated to an external device. In accordance with the present disclosure, the microprocessor 503 is provided with processing means 505, e.g. , firmware, for recognizing various touch gestures in accordance with the present input system. The microprocessor 503 is coupled in m to an electronic device 520 to input information to the electronic device, including positional information, commands and textual information. In the embodiment of Figure 5, the processing means 505 is located accessible to and runs on the microprocessor 503 of the touchpad or touchscreen 510. Results of the processing are then sent to the electronic device 520.

Referring to Figure 6, processing means 625 may instead be accessible to and run on a microprocessor of the electronic device 620. The processing means 625 may, for example, take the form of a custom software driver for the touchpad or touchscreen 610. In this embodiment, the microprocessor 603 sends raw measurement data to the electronic device 620 for processing by the processing means 625. The processing means 625 then signals the results of the processing to other elements within the electronic device 620. The logical strucmre of the processing means 505 is shown in greater detail in Figure 7. Touch sensor data is input to various program segments. One segment 710 performs one-touch processing and another segment 720 performs two-touch processing. A third segment 730 may be provided to perform three-touch processing if desired. Results of one-touch and two-touch (and, if desired, three-touch) processing are provided to a touch-inflections segment 740 for further processing. If the outcome of touch- inflection processing is that a valid touch gesture has been recognized, then the touch gesture is communicated to a protocol segment 750 for commumcation to the electronic device. Processed track data, or pointing data, is also input to the protocol segment 750. The processed track data may be produced by conventional means 760 in accordance with current practice.

In the case of the processing means 625, a protocol layer between the touchpad or touchscreen 610 and the electronic device 620 provides for raw capacitance data and processed tracking data to be input tothe electronic device 630. The raw capacitance data and the tracking data are processed within the electromc device 520, and results of such processing are communicated to further elements within the electronic device 620, e.g., the device operating system.

Referring still to Figure 7, within the one-touch processing and touch- inflection processing segments, various subprocesses have been called out. More particularly, within the one-touch processing segment 710, there is shown a subprocess 713 that detects a thumb touch and a further subprocess

715 that detects a touch on an arrow. Within the touch- inflections processing segment 740, there is shown a subprocess 741 that detects added pressure, subprocesses that detect a roll gesture (743) and a waggle gesture (745), respectively, a subprocess 747 that detects a thumb stroke in particular, and a subprocess 749 that detects other stroke inflections.

Referring to Figure 8, a flow diagram is shown of one-touch touch processing. As described previously, single-touch processing may involve the thumb only or the finger only. If the touch is a thumb touch (801), then the routine checks to see whether the thumb is moved right some mimmum amount

(803). This checking continues for so long as the thumb touch continues (805).

If the thumb was moved right, then a tab is input (807) and the routine returns.

If not, then a carriage return is input (809) and the routine returns.

If instead the touch is a finger touch, then the routine checks to see if the centroid of the touch is within a central area of a single key (811). If so, then one of a number 0-9, an asterisk or a pound sign is input per the touched key, except as modified subsequently by the touch inflection routine (813).

Processing of the touch inflection routine then follows, beginning at point A.

If the touch is over one of the arrows (815), then a cursor key command is input per the touched arrow (817). The routine then returns.

As seen in block 819, if the touch is in the border region of the touchpad (but not over an arrow), then a border textual element (e.g. , space, period, comma, semicolon) is input depending on which border region is touched (821). The routine then returns. Two- touch touch processing is simple and predictable. As shown in

Figure 9, a textual element/command is input per the touched key combination except as modified subsequently by the touch inflection routine. Processing of the touch inflection routine then follows, beginning at point B.

Referring now to Figure 10, the processing of single-touch touch inflections begins at point A. First, the routine checks to see if there is a continuing touch of one of keys 0-9 (1001). If not, the routine returns. If so, the routine checks for a roll. If a roll is detected (1003), then a function key command is input per the touched key and the direction (right or left) of the roll (1005). The routine then returns. Finally, if no roll is detected, then the routine checks for a stroke inflection in which the travel is faster and/or farther than in the case of a roll. If a stroke inflection is detected (1007), then a border textual element (e.g. , space, period, comma, semicolon) is input depending on which border region is touched (1009). The routine then returns. If no stroke inflection is detected, then the routine checks again to see if there is a continuing touch, and the routine is repeated as described previously until such time as there is no continuing touch.

Referring now to Figure 11 , the processing of double-touch touch inflections begins at point B. First, the routine checks to see if there is a continuing touch (1101). If not, the routine returns. If so, the routine checks for a press, i.e., a touch in which an added increment of pressure is applied. If a press is detected (1103), then an upper case letter or upper tier puncmation mark/command is input per the touched combination (1105), and the routine returns. If no press is detected, next, the routine checks for a roll. If a roll is detected (1107), then the routine checks further whether there is a continuing touch (1109). If not, then a digraph is input per the touched key and the direction (right or left) of the roll (1111). The routine then returns. If there is a continuing touch, then the routine checks for a waggle. If a waggle is detected (1113), then a trigraph is input per the touched key and the initial direction (right or left) of the waggle (1115) and the routine returns. Otherwise, the routine loops back until the touch has been discontinued (1109) or a waggle (1113) has been detected. Finally, if no roll was detected (1107), then the routine checks for a stroke inflection in which the travel is faster and/or farther than in the case of a roll. If a stroke inflection is detected (1117), then a border texmal element (e.g., space, period, comma, semicolon) is input depending on which border region is touched (1119). The routine then returns. If no stroke inflection is detected, then the routine checks again to see if there is a continuing touch, and the routine is repeated as described previously until such time as there is no continuing touch.

Insofar as the input of textual information is concerned, the invention has been described principally in terms of inputting texmal information based on the English alphabet. The invention is equally applicable to inputting texmal information based on "strokes" used in ideographic languages such as Chinese, Japanese, Korean, etc. A second finger (instead of pressure) may be used to advantage to distinguish between different tiers of radicals on a single key. A character would be input by inputting the individual strokes of the character in their usual stroke order. It will be appreciated by those of ordinary skill in the art that the present invention may be embodiment in other specific forms without departing from the spirit or essential character thereof. The present description is therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims, and all changes which come within the meaning and range of equivalents therefore are intended to be embraced therein.

Claims

What is Claimed is:
1. A method of inputting information to an electronic device using a pad having a pad surface, which pad distinguishes multiple simultaneous touches, the method comprising the steps of: forming markings in relation to the pad surface, the markings including texmal elements; a user, with the user's hand, touching one digit of the hand to a first area of the pad surface bearing a marking corresponding to desired texmal element that the user desires to input and, at substantially the same time, touching another digit of the hand to a second area of the pad surface identified by the user by its occupying a predetermined position relative to the first area; detecting the first and second areas touched by the user; and inputting the desired texmal element.
2. The method of Claim 1 , wherein the one digit is one of the index finger and the middle finger, and the other digit is the thumb.
3. The method of Claim 2, wherein the markings delineate a matrix of keys.
4. The method of Claim 3, wherein the pad is a touchpad, the pad surface is a touchpad surface, and the keys are virtual keys.
5. The method of Claim 4, comprising the further steps of: displaying a cursor on a display of the electronic device; the user moving one digit of the hand across the touchpad surface; tracking motion of the one digit across the touchpad surface; and moving the cursor across the display in accordance with motion of the one digit across the touchpad surface.
6. The method of Claim 3, wherein the matrix comprises a twelve key matrix array in four rows and three columns.
7. The method of Claim 6, wherein respective ones of ten of the keys each bear a marking corresponding to a respective one of the numbers 0 through 9.
8. The method of Claim 7, comprising the further steps of: the user touching a single one of said ten keys, producing a single touch; detecting the single touch; and in response to the single touch, inputting a number.
9. The method of Claim 8, comprising the further steps of: the user touching with a single digit multiple ones of the virtual keys near a periphery of the touchpad surface, producing a single-digit multiple touch; detecting the single-digit multiple touch; and in response to the single-digit multiple touch, inputting one of a cursor-key command and a scroll-bar command.
10. The method of Claim 9, wherein the markings include an up arrow, a down arrow, a right arrow and a left arrow.
11. The method of Claim 7, wherein a plurality of keys each bear markings corresponding to a plural number of distinct texmal elements.
12. The method of Claim 11 , wherein the plural number of distinct texmal elements are letters of an alphabet.
13. The method of Claim 11 , wherein the plural number of distinct texmal elements are puncmation marks.
14. The method of Claim 11 , wherein the plural number of distinct texmal elements is greater than two.
15. The method of Claim 1, wherein the texmal elements include a period and, distinct from the period, a dot for use in computer addresses.
16. The method of Claim 15, wherein a marking corresponding to the period and a marking corresponding to the dot are distinguished by color.
17. The method of Claim 1 , comprising the further steps of: the user touching only a thumb to the pad such that a thumb touch is applied to the pad surface; detecting the thumb touch; and in response to the thumb touch, inputting a desired texmal element.
18. The method of Claim 17, wherein the desired texmal element is a return.
19. The method of Claim 1, comprising the further steps of: the user touching only a thumb to the pad; the user moving the thumb across the pad surface in a direction toward the hand, producing a thumb-stroke gesture; detecting the thumb-stroke gesture; and in response to the thumb-stroke gesture, inputting a desired texmal element.
20. The method of Claim 19, wherein the desire texmal element is a tab.
21. The method of Claim 1 , wherein the desired texmal element is a letter of an alphabet, comprising the further step of applying an added increment of pressure to the pad in order to designate an upper-case letter.
22. The method of Claim 1, comprising the further steps of: the user, while withdrawing one digit of the hand from the pad surface, stroking that digit across the pad surface, producing a stroked touch inflection; detecting the stroked touch inflection; and in response to the stroked touch inflection, inputting a further desired texmal element.
23. The method of Claim 1 , comprising the further steps of: the user touch at least one digit to the pad surface and, prior to removing the digit from the pad surface, rolling the digit in a first direction.
24. The method of Claim 23, wherein rolling the digit in a first direction produces a roll gesmre, the method comprising the further steps of: detecting the roll gesmre; and in response to the roll gesmre, inputting a plurality of texmal elements.
25. The method of Claim 23, wherein rolling the digit in a first direction produces a roll gesmre, the method comprising the further steps of: detecting the roll gesmre; and in response to the roll gesmre, inputting a fiinction key command.
26. The method of Claim 25, wherein respective ones of ten of the keys each bear a marking corresponding to a respective one of the numbers 0 through 9.
27. The method of Claim 26, wherein the user touches a single one of said tens keys, and wherein the function key command is identified with a number to which a marking on the single one of the ten keys corresponds.
28. The method of Claim 23, comprising the further steps of: prior to removing the digit from the pad surface, rolling the digit again in a second direction to produce a waggle gesmre.
29. The method of Claim 28, comprising the further steps of: detecting the waggle gesmre; and in response to the waggle gesmre, inputting at least three texmal elements.
30. An input device for inputting information to an electronic device, the input device comprising: a touchpad surface formed in relation to a touch sensor array; markings formed in relation to the pad surface, the markings including texmal elements; an integrated circuit controller coupled to the touch sensor array for receiving capacitance measurement data from the touch sensor array; and processing means for detecting a particular touch pattem in which a plurality of the areas on the touchpad surface are touched simultaneously and for, in response to the particular touch pattem, signalling for input to the electronic device a texmal element corresponding to the particular touch pattem.
31. The apparams of Claim 30, wherein said means for processing further comprises means for tracking motion of a touch across the touchpad surface and for signalling for input to the electronic device positional information.
32. The apparams of Claim 30, wherein the markings delineate a matrix of virtual keys.
33. The apparams of Claim 32, wherein the matrix comprises a twelve key matrix array in four rows and three columns.
34. The apparams of Claim 33, wherein respective ones of ten of the keys each bear a marking corresponding to a respective one of the numbers 0 through 9.
35. The apparams of Claim 34, wherein said means for processing further comprises means for detecting a single touch of one of said ten keys and, in response to the single touch, signalling a number for input to the electronic device.
36. The apparams of Claim 35, wherein said means for processing further comprises means for detecting a single-digit multiple-key touch and for signalling for input to the electronic device one of a cursor-key command and a scroll-bar command.
37. The apparams of Claim 36, wherein the markings include an up arrow, a down arrow, a right arrow and a left arrow.
38. The apparams of Claim 33, wherein a plurality of virtual keys each bear markings corresponding to a plural number of distinct texmal elements.
39. The apparams of Claim 38, wherein the plural number of distinct textual elements are letters of an alphabet.
40. The apparams of Claim 38, wherein the plural number of distinct texmal elements are puncmation marks.
41. The apparams of Claim 38, wherein the plural number of distinct texmal elements is greater than two.
42. The apparams of Claim 30, wherein the texmal elements include a period and, distinct from the period, a dot for use in computer addresses.
43. The apparams of Claim 42, wherein a marking corresponding to the period and a marking corresponding to the dot are distinguished by color.
44. The apparams of Claim 30, wherein said means for processing further comprises means for detecting a thumb touch and for, in response to the thumb touch, signalling a first texmal element for input to the electronic device.
45. The apparams of Claim 44, wherein the first texmal element is a return.
46. The apparams of Claim 30, wherein said means for processing further comprises means for detecting a thumb-stroke gesmre and for, in response to the thumb-stroke gesmre, signalling a second texmal element.
47. The apparams of Claim 46, wherein the second texmal element is a tab.
48. The apparams of Claim 30, wherein said processing means further comprises means for detecting an added increment of pressure applied to the pad and for signalling an upper-case letter for input to the electronic device.
49. The apparams of Claim 30, wherein said processing means further comprises means for detecting a stroked touch inflection and for, in response to the stroked touch inflection, signalling a texmal element for input to the electronic device.
50. The apparams of Claim 30, wherein said processing means further comprises means for detecting a roll gesmre and for, in response to the roll gesmre, signalling a plurality of texmal elements for input to the electronic device.
51. The apparams of Claim 30, wherein said processing means further comprises means for detecting a roll gesmre and for, in response to the roll gesmre, signalling a function key command for input to the electronic device.
52. The apparams of Claim 51 , wherein respective ones of ten of the keys each bear a marking corresponding to a respective one of the numbers 0 through 9.
53. The method of Claim 52, wherein the roll gesmre is characterized by a single touch of one of said tens keys, and wherein the function key command is identified with a number to which a marking on the single one of the ten keys corresponds.
54. The apparams of Claim 30, wherein said processing means further comprises means for detecting a waggle gesmre and for, in response to the waggle gesmre, signalling a plurality of at least three texmal elements for input to the electronic device.
EP19960941390 1995-11-16 1996-11-18 Multi-touch input device, method and system that minimize the need for memorization Withdrawn EP0861485A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
US672093 1991-03-19
US55902395 true 1995-11-16 1995-11-16
US559023 1995-11-16
US08672093 US6107997A (en) 1996-06-27 1996-06-27 Touch-sensitive keyboard/mouse and computing device using the same
US2598296 true 1996-09-09 1996-09-09
US25982P 1996-09-09
PCT/US1996/018517 WO1997018547A1 (en) 1995-11-16 1996-11-18 Multi-touch input device, method and system that minimize the need for memorization

Publications (1)

Publication Number Publication Date
EP0861485A1 true true EP0861485A1 (en) 1998-09-02

Family

ID=27362662

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19960941390 Withdrawn EP0861485A1 (en) 1995-11-16 1996-11-18 Multi-touch input device, method and system that minimize the need for memorization

Country Status (3)

Country Link
EP (1) EP0861485A1 (en)
JP (1) JP2000501526A (en)
WO (1) WO1997018547A1 (en)

Families Citing this family (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9292111B2 (en) 1998-01-26 2016-03-22 Apple Inc. Gesturing with a multipoint sensing device
US9239673B2 (en) 1998-01-26 2016-01-19 Apple Inc. Gesturing with a multipoint sensing device
EP1717681B1 (en) * 1998-01-26 2015-04-29 Apple Inc. Method for integrating manual input
CA2390439A1 (en) * 1999-10-27 2001-05-03 Firooz Ghassabian Integrated keypad system
US7254775B2 (en) * 2001-10-03 2007-08-07 3M Innovative Properties Company Touch panel system and method for distinguishing multiple touch inputs
US6995752B2 (en) 2001-11-08 2006-02-07 Koninklijke Philips Electronics N.V. Multi-point touch pad
US7808479B1 (en) 2003-09-02 2010-10-05 Apple Inc. Ambidextrous mouse
US7663607B2 (en) 2004-05-06 2010-02-16 Apple Inc. Multipoint touchscreen
JP4405335B2 (en) 2004-07-27 2010-01-27 株式会社ワコム Position detecting device, and an input system
US7614008B2 (en) 2004-07-30 2009-11-03 Apple Inc. Operation of a computer with touch screen interface
US8479122B2 (en) 2004-07-30 2013-07-02 Apple Inc. Gestures for touch sensitive input devices
US8381135B2 (en) 2004-07-30 2013-02-19 Apple Inc. Proximity detector in handheld device
CN102841713A (en) * 2005-09-15 2012-12-26 苹果公司 System and method for processing raw data of track pad device
US7719522B2 (en) 2004-09-24 2010-05-18 Apple Inc. Raw data track pad device and system
US7728823B2 (en) 2004-09-24 2010-06-01 Apple Inc. System and method for processing raw data of track pad device
KR100984630B1 (en) * 2004-09-24 2010-09-30 애플 인크. System and method for processing raw data of track pad device
JP2008515052A (en) * 2004-09-24 2008-05-08 アップル インコーポレイテッド Raw data track pad devices and systems
US7656393B2 (en) 2005-03-04 2010-02-02 Apple Inc. Electronic device having display and surrounding touch sensitive bezel for user interface and control
GB0515175D0 (en) 2005-07-25 2005-08-31 Plastic Logic Ltd Flexible resistive touch screen
US7538760B2 (en) 2006-03-30 2009-05-26 Apple Inc. Force imaging input device and system
US8279180B2 (en) 2006-05-02 2012-10-02 Apple Inc. Multipoint touch surface controller
US9063647B2 (en) 2006-05-12 2015-06-23 Microsoft Technology Licensing, Llc Multi-touch uses, gestures, and implementation
GB0611032D0 (en) 2006-06-05 2006-07-12 Plastic Logic Ltd Multi-touch active display keyboard
EP2027524B1 (en) 2006-06-09 2011-11-23 Apple Inc. Touch screen liquid crystal display
CN104965621B (en) 2006-06-09 2018-06-12 苹果公司 A touch screen liquid crystal display and an operation method for
GB2445372B (en) * 2007-01-03 2009-06-03 Motorola Inc Electronic device and method of touch screen input detection
US7855718B2 (en) * 2007-01-03 2010-12-21 Apple Inc. Multi-touch input discrimination
US8130203B2 (en) 2007-01-03 2012-03-06 Apple Inc. Multi-touch input discrimination
US8970503B2 (en) 2007-01-05 2015-03-03 Apple Inc. Gestures for devices having one or more touch sensitive surfaces
US9710095B2 (en) 2007-01-05 2017-07-18 Apple Inc. Touch screen stack-ups
US8144129B2 (en) 2007-01-05 2012-03-27 Apple Inc. Flexible touch sensing circuits
US7844915B2 (en) 2007-01-07 2010-11-30 Apple Inc. Application programming interfaces for scrolling operations
WO2008138046A1 (en) * 2007-05-11 2008-11-20 Rpo Pty Limited Double touch inputs
US9448721B2 (en) 2008-03-19 2016-09-20 Blackberry Limited Electronic device including touch-sensitive input device and method of determining selection
EP2104027B1 (en) * 2008-03-19 2013-10-23 BlackBerry Limited Electronic device including touch sensitive input surface and method of determining user-selected input
WO2009118221A1 (en) * 2008-03-28 2009-10-01 Oticon A/S Hearing aid with a manual input terminal comprising a touch sensitive sensor
CN102084326A (en) 2008-05-14 2011-06-01 3M创新有限公司 Systems and methods for assessing locations of multiple touch inputs
US9513705B2 (en) 2008-06-19 2016-12-06 Tactile Displays, Llc Interactive display with tactile feedback
US8115745B2 (en) 2008-06-19 2012-02-14 Tactile Displays, Llc Apparatus and method for interactive display with tactile feedback
DE102009008041A1 (en) 2009-02-09 2010-08-12 Volkswagen Ag A method of operating a motor vehicle with a touchscreen
JP5451181B2 (en) * 2009-05-25 2014-03-26 株式会社ジャパンディスプレイ Sensor apparatus for detecting contact or proximity of an object
US8654524B2 (en) 2009-08-17 2014-02-18 Apple Inc. Housing as an I/O device
US8804056B2 (en) 2010-12-22 2014-08-12 Apple Inc. Integrated touch screens
FR2970576B1 (en) 2011-01-19 2013-02-08 Matchic Labs A method for determining the gaze direction and device for its implementation
US9064436B1 (en) 2012-01-06 2015-06-23 Google Inc. Text input on touch sensitive interface
CN104331173A (en) * 2012-04-16 2015-02-04 宗刚 Computer processing method and system for character information

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2585487B1 (en) * 1985-07-29 1990-09-07 Guyot Sionnest Laurent Computer keyboards, <70 cm2 and less than 13 contacts operated combinedly by the fingers of one hand
US4914624A (en) * 1988-05-06 1990-04-03 Dunthorn David I Virtual button for touch screen
US5128672A (en) * 1990-10-30 1992-07-07 Apple Computer, Inc. Dynamic predictive keyboard

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO9718547A1 *

Also Published As

Publication number Publication date Type
WO1997018547A1 (en) 1997-05-22 application
JP2000501526A (en) 2000-02-08 application

Similar Documents

Publication Publication Date Title
Westerman Hand tracking, finger identification, and chordic manipulation on a multi-touch surface
US7705830B2 (en) System and method for packing multitouch gestures onto a hand
US7170496B2 (en) Zero-front-footprint compact input system
US5933134A (en) Touch screen virtual pointing device which goes into a translucent hibernation state when not in use
US5856824A (en) Reshapable pointing device for touchscreens
US20110154268A1 (en) Method and apparatus for operating in pointing and enhanced gesturing modes
US5812118A (en) Method, apparatus, and memory for creating at least two virtual pointing devices
US20090273571A1 (en) Gesture Recognition
US20060267951A1 (en) Control of an electronic device using a gesture as an input
US6104317A (en) Data entry device and method
US5796406A (en) Gesture-based input information processing apparatus
US20100100854A1 (en) Gesture operation input system
US20060197752A1 (en) Multiple-touch sensor
US5790104A (en) Multiple, moveable, customizable virtual pointing devices
US5521986A (en) Compact data input device
US6424338B1 (en) Speed zone touchpad
US20080036743A1 (en) Gesturing with a multipoint sensing device
US5250929A (en) Interactive overlay-driven computer display system
US20040140956A1 (en) System and method for continuous stroke word-based text input
US6029214A (en) Input tablet system with user programmable absolute coordinate mode and relative coordinate mode segments
US20100231522A1 (en) Method and apparatus for data entry input
US5832113A (en) Data input device with a display keyboard
US5757368A (en) System and method for extending the drag function of a computer pointing device
US20020109677A1 (en) Touchpad code entry system
US20050052427A1 (en) Hand gesture interaction with touch surface

Legal Events

Date Code Title Description
AK Designated contracting states:

Kind code of ref document: A1

Designated state(s): DE DK FR GB IT NL

17P Request for examination filed

Effective date: 19980616

18D Deemed to be withdrawn

Effective date: 20020601