WO2007017660A2

WO2007017660A2 - A data entry device and method

Info

Publication number: WO2007017660A2
Application number: PCT/GB2006/002943
Authority: WO
Inventors: Timothy Scanlan
Original assignee: Timothy Scanlan
Priority date: 2005-08-08
Filing date: 2006-08-07
Publication date: 2007-02-15
Also published as: WO2007017660A3; GB0516246D0

Abstract

A character entry device on an input means for use with a stylus, the device comprising: a plurality of areas each associated with one or more characters, the areas being arranged in a predetermined orientation; a detector which identifies the contact or sequence of contacts made in the or each area in the process of inputting a word or command with a stroke of the stylus; a predictor which determines the characters associated with each of the contact or sequence of contacts to determine the most likely string of characters forming the word or command to thereby identify the word or command; and a selector for selecting the correct word or command if there is more than one predicted string of characters.

Description

A data entry device and method

This invention relates to the field of a data entry device and method, particularly but not exclusively to such a device and method for use in a computing device, pda, mobile phone, etc¹.

The problem of entering data into a computer or equivalent device such as a pda or mobile telephone is well known. Data has been entered into these- types of devices with a keyboard, a mouse, a stylus and screen and many other type of data entry means . As the devices get smaller and portable the data entry means must get smaller be always part of the device. This means a data entry means in the form of for example a separate keyboard or mouse, is less attractive.

Accordingly more and more data entry means are taking the form of a stylus which enters data using a screen or other contact surface by making gestures or strokes. This in turn has lead to a plurality of different methods of entering letters words and other information. Many of these include the need to learn a specific key stroke sequence for each letter and adhere to that for every data entry. These types of method are hard to learn and often mean the user has to make many strokes to enter simple words and still more if they are to enter capitals, punctuation marks etc. Thus a need exists for a pen based data entry device and method which overcomes at least some of the problems associated with the prior art. An object of the invention is to provide a true pen-based user interface which is both simple and intuitive to use.

According to one aspect of the present invention there is provided a character entry device on an input means for use with a stylus, the device comprising: -a plurality of areas each associated with one or more characters, the areas being arranged in a predetermined orientation;

-a detector which identifies the contact or sequence of contacts made in the or each area in the process of inputting a word or command with a stroke of the stylus;

-a predictor which determines the characters associated with each of the contact or sequence of contacts to determine the most likely string of characters forming the word or command to thereby identify the word or command; and

-a selector for selecting the correct word or command if there is more than one predicted string of characters .

According to a second aspect of the present invention there is provided a method of entering a character for recognition in a computer related device, comprising: -detecting the contact or sequence of contacts of a stroke made by a stylus on one or more areas on an input means, the areas each associated with one or more characters and the areas being arranged in a predetermined orientation;

-predicting the possible character or characters for the contact or sequence of contacts to determine the most likely string of characters forming a word or command; and

-selecting the correct word or command if there is more than one possible word or command.

According to a third aspect of the present invention there is provided a method of recognising gestures entered into an input means of a computer related device by means of a stylus, the method comprising:

-detecting the contact or sequence of contacts of the gesture on one or more areas on the input means, the areas being arranged in a predetermined orientation;

-associating the contact or contacts with one or more characters associated the or each area;

Reference will now be made, by way of example, to the accompanying drawings, in which:

Figures 1 and 2 are screen views of the data entry screen showing an example of how to enter a word in accordance with the present invention; - Figure 3 shows the graphic interface for the input of several words in accordance with the present invention; figure 4 shows the graphic interface for the input of several words in accordance with the present invention; figure 5 is a screen view of entering an abbreviate phrase in accordance with the present invention; figure 6 shows examples of punctuation gestures in accordance with the present invention; figure 7 shows examples of command gestures in accordance with the present invention; figure 8 shows the screen shot for a new language after a control command gesture in accordance with the present invention; figure 9 shows the screen shot for changing the nature of the text in accordance with the present invention; figure 10 shows the loop gesture in accordance with the present invention; figure 11 shows how to enter letters in accordance with the present invention; figure 12 shows the edit gestures in accordance with the present invention; - figure 13 shows the main edit gesture in accordance with the present invention; figure 14 shows spatial awareness reasoning in accordance with the present invention; The "LX" input system

The "LX" system is described for pen-based machines . The "LX" system not only provides a convenient and rapid writing method for text input, including punctuation, but also handles program control via gestures and commands . An intuitive set of editing gestures completes the interface, making "LX" the first true pen-based input system.

Figure 1 shows an adaptation of the system for a mobile phone without a keypad, but equipped with a pen. The example includes the various user interface elements needed for a simple email application. At top left (Al) is a single line text field used to enter the recipient's email address. Area A2 is the text area for the mail message. The central area (A3) is used to show the word proposed by the system, and various alternates associated with this word. At the right is the "LX" input area (A4) .

Text entry

The text input area (A4) consists of a ring of eight coloured sectors corresponding to the letter groups

[abc] to [wxyz] . The user writes a word by tapping on the sectors, or by drawing pen strokes corresponding to the combinations of letters forming the word, thus forming what we shall call "word shapes". For example, the example above shows a single pen stroke used to obtain the word "word" . The stroke starts in sector [wxyz] , and traverses [mno] to terminate in [pqrs] . The system generates a list of words corresponding to (or starting with) one of the possible sequences of letters defined by these letter groups. As it happens, there are no three-letter words for the given combination, so the list starts with the most common four-letter words . "Word" is most likely, and the alternates are work, wore, worn, worm, and York (area A3 in the illustration) . At this stage, a tap of the pen inside the sector ring will send "word" to the message area.

Note that in this particular case, it is not necessary to specify all the letters in the word: in fact, just the first three letters, ^λw' , ^vo' and 'r' are enough to obtain "word" as the proposal. This situation occurs quite often, and it is rarely necessary to go beyond five or six letters in order to be sure of finding the required word amongst the alternates. Moreover, since the words are stored in order of frequency of use, the required word will often be the proposal .

To send the proposed word to the text area, the user can either tap inside the sector ring, or on the proposal itself. To send one of the alternates to the text area, the user taps on the required word.

For the sake of completeness, look at what happens if we draw the entire word-shape for "word" see figure 2. "Word" is still the first choice, but now we have only one other four-letter alternate, and all the other suggestions start with the four letters "word" . This illustrates the "look ahead" feature of "LX", which is particularly useful in languages where verbs are conjugated or nouns are declined. Here, tapping on "Wordsworth" would send that word to the text area, a ten-letter word for a very simple shape.

The next example illustrates some writing techniques are shown in figure 3.

The word "hello" can be easily drawn with a single stroke. The double letter '1' is achieved by exiting and re-entering the sector [jkl] . This technique of going outside of the input circle is very convenient, not only for double letters, but for other letter combinations using the same sector, such as "ed", ^«pr" or "sp" (twice in sector [pqrs] ) , and even "mon" (three times in [mno] ) . Note the simplicity and elegance of the word shape for "hello" .

The word "world" has been written with two strokes, corresponding to "wor", and "Id". The first stroke is exactly that shown for the first example, except that the stroke starts outside the sector circle, which works equally well. The system would at this stage show the same alternates as above (work, wore etc) . The second stroke, from sector [jkl] to [def] completes the word shape for "world". (Note: the alternates are worlds, worked, worker, yorker and workers) . The word shape for "world" could, of course, also be drawn as a single stroke. However, it is often easier to break words into syllables, for convenience and comfort, but also because one tends to automatically learn the shapes of various frequent letter combinations .

The examples shown in figure 4 show the word shapes corresponding to the phrase "the quick brown fox jumped over the lazy dog" (leaving out the second "the") :

A few notes relating to these word shapes :

1. The word "the" can be obtained with a single stroke into the [tuv] sector, or simply by tapping inside the sector. The dictionary is organized so that words are proposed first according to length, then according to frequency of utilisation. "The" is special, since it is the most common word in the English language: the letter "f is thus defined as an abbreviation (see next section) .

2. "Quick" and "brown" have been written with two strokes, whereas "fox" is nicely drawn with a single cusped shape.

3. "Jumped" is also broken into two: first the base word "jump", then the ending "ed". The latter is obviously very common, and it will be frequently convenient to break it off in this way. 4. While "over" could also be broken into two simple strokes "ov" and "er", the shape shown in the example is relatively easy to execute, and since "over" is a fairly common word, would probably be committed to memory by many users .

5. "Lazy" contains another double-sector combination "zy" .

6. The word "dog", even though it is short and simple, can be awkward to execute, because of the return stroke from [mno] to [ghi] . One solution is to draw the stroke for "do", then to tap in sector [ghi] to complete the word.

Abbreviations

The system provides a method for defining and using abbreviations, and word contractions (it's, don't etc) . Dictionary entries for abbreviations are marked such that when an abbreviation shape is drawn in the writing area, it is automatically replaced by its associated full string.

The system will provide some standard abbreviations, but the facility is mostly intended for the user's own definitions. As an example of a standard abbreviation, we might have ^λys' for "Yours Sincerely,". The stroke shown in the following example would then result in this useful string as is seen in figure 5. The system uses internal abbreviations in a few special cases :

(1) Word contractions, like "he's" or "don't". In these cases, the user would just enter the word without worrying about the apostrophe: the system will know that it is required. (Note that a few contractions, like "it's" and "we'll" will appear as alternates to an ordinary word) .

(2) Hyphenated words, or words containing periods : ("word-shape", "G.I", "U. S.A"). Note that upper-case letters appearing in the dictionary are always retained in the proposed text.

(3) Some words resulting from a tap in a sector (or a stroke entering the sector) are in fact abbreviations . This means that some very common words will appear ahead of shorter, but less common, words. Thus, [tuv] is defined as a system abbreviation for "the", and [mno] is "of".

Punctuation Gestures

The "LX" gestures are executed in the space inside the ring of letter sectors . This allows a rapid and seamless transition from pure text input, which is particularly important in the case of punctuation. Figure 6 shows the punctuation gestures defined for the "LX" system. They have been designed to be both intuitive for the user, and easy to recognize. Most of the shapes correspond directly with their written equivalent: period is drawn as a circle in the anticlockwise direction, and colon is two anti-clockwise circles. The comma gesture should be drawn with a definite curve to distinguish it from apostrophe (but sufficiently shallow to distinguish it from close parenthesis, which is best drawn as a full semicircle) .

Action Gestures and Commands

As shown in figure 7, a tap inside the sector ring introduces a Command (the Tap gesture) . The sectors change colour (to red) to show this fact, and the system will then subsequently propose command names in response to word shapes. This is how "LX" provides functions that are usually menu items in keyboard and mouse systems. As an example, let's see how to change the input language to German as shown in figure 17.

Figure 8 shows the result of writing "de" in Command mode. The demo system includes a command "Edit Contact" which would be passed to the phone application, if selected. This is proposed as an alternate command. A tap in the centre of the ring, or on the word "Deutsch" will change the current input language to German.

The language change is handled directly by the "LX" system. The first time any given language command is used, the dictionary will be loaded (the input area will stay red during the loading) . Later, the user can switch between languages at will ("en" for English, "fr" for Franςais, and so on) .

Note that all the action gestures have equivalent commands, in case the user forgets the gesture, or has problems with it. A Tap gesture in command mode will cause the system to revert back to normal text input.

Four gestures provide shortcuts for defining character Style: UP (Shift), UP/DOWN (Capslock) , RIGHT/LEFT (Bold) , and LEFT/RIGHT (Italic) . Executing one of these gestures in the centre of the sectors ring changes the current text input state. The letter labels around the input area change to show the new state. The illustration above shows the result of a Capslock gesture. Note that Bold, Italic, and Underline are mutually exclusive in the current system see figure 9.

If the user comes across a word that is not in the dictionary, he/she must enter it letter by letter. The system will allow the user to call up a virtual keyboard for this task, but the sectors ring can also be used to enter single letters. The "Loop" gesture turns letter mode on or off as shown in figure 10.

The sector ring changes size to indicate letter input. Letters are executed by entering a given letter sector the appropriate number of times (for instance, a line into or through [abc] , or a tap in the sector, gives a' . For ^vb' the sector must be entered twice, and for ^λc' three times, as shown in figure 11) .

Note that all the letters are drawn with a single stroke, entering the given sector the appropriate number of times . This means that each letter is naturally separated by pen-up, and there is no reason to introduce an artificial method for separating the letters, as is common on multi-tap keypad systems (usually by time-out or by defining a specific letter separator key) .

When the new word is complete, it can be added to the dictionary by selecting it (see Edit Gestures), and then executing the "Add Word" gesture, or by executing the Add Word gesture directly over the word. The Loop gesture can be used to resume normal text entry.

The downward loop (an inverted Letters gesture) will be used to call up a numeric keypad for number input.

When writing quickly, the user might sometimes accept the proposed word when intending to enter one of the alternates. In this case, rather than deleting the word and starting again, the user can use the Alternates gesture (a "Greater Than" sign) to redisplay the alternates. The gesture can either be drawn directly over the word, or the word can first be selected, and the gesture drawn inside the sector ring. (This second method can be easier if the word is short, or if the font is small) . The Reset (or Scribble) gesture, is used to cancel current input, usually following an error in an intermediate stroke. Note that if there is no word corresponding to a stroke, there will be no proposal.

Edit Gestures

All of the edit gestures shown in figure 12 operate directly on the text. The Insert gesture (a tap at the chosen place in the text) sets the position for text input (the cursor position in mouse systems) .

The Select gesture consists of a line drawn around a word, or a group of words. (The word group is restricted currently in the demo system to words on the same line) . You can also select a line of text, or several lines, by drawing a left or right bracket to encompass the required text.

The Crossout gesture consists of a roughly diagonal line drawn through a word, or a selected phrase. Text removed with Crossout is cut, and can be pasted back into the text area by means of the Paste gesture (an inverted "v" drawn at the new position) .

The Underline gesture is perfectly intuitive, and consists of a straight line drawn from left to right, or from right to left, at the base of the word to underline. (The "Underline" command exists for setting the current input state) . Figure 13 shows the main edit gestures:

Taking the gestures in order, we have: (1) Crossout "quick".

(2) Underline "jumped".

(3) Paste "quick" back in front of "lazy".

(4) Select "dog". The result is shown in the right-hand panel .

You can change the style of a word, or group of words, by selecting, then executing the appropriate gesture inside the sectors ring. Thus, the Shift gesture will change the case of the first letter of a word, Capslock will change a lower-case word to upper-case, and vice versa, while the Bold and Italic gestures will set or unset their respective styles. It is also possible to execute the gesture directly over a word, as long as the gesture stays roughly within the limits of the word boundary. You can also do the "Add Word" and "Alternates" gestures directly on the word.

It is possible to select several lines of text at a time by drawing a Brackets gesture to encompass the chosen lines, to the left or to the right of the text.

Input Fields

The "LX" system will provide an API (Application Programming Interface) to allow application programmers to define specialized fields that take advantage of intelligent word input. As a simple example, consider the address field ("To: ") in the mobile phone adaptation:

The application would provide a method for the user to associate contact names with their email addresses . The application would declare the names list as a dictionary to be associated with the "To: " field. Then, the user can enter a name by means of "LX" word- shapes, and the application can automatically translate the name into the corresponding email address, to be displayed in the address field. A similar procedure would associate contact names directly with phone numbers .

In the example in figure 14, a stroke from sector [tuv] to [ghi] is sufficient to define the dictionary entry "Tim Scanlan" (indeed, this is the only entry corresponding to these two sectors in the list used for the demo) . A tap on the proposal, or in the centre of the sectors ring will send the corresponding email address to the "To:" field, as shown.

It will be appreciated that the ring or annulus is one form the area of the entry device may take. However other orientations are also possible, for example a rectangle, square, grid or other shape. In addition, the numbers of letters or other symbols in each area could by different for different variations of the invention. The system and method described herein are by way of example and other variations forms and examples are possible.

Claims

1. A character entry device on an input means for use with a stylus, the device comprising: -a plurality of areas each associated with one or more characters, the areas being arranged in a predetermined orientation;

-a predictor which determines the characters associated with each of the contact or sequence of contacts to determine the most likely string of characters forming the word or command to thereby identify the word or command; and -a selector for selecting the correct word or command if there is more than one predicted string of characters.

2. The device of claim 1 , wherein the word or command includes more than two characters .

3. The device of claim 1 or claim 2, wherein the predetermined orientation of the areas is in the form of an annulus .

4. The device of claim 3, wherein the centre of the annulus forms at least one of the areas.

5. The device of any preceding claim, wherein some of the areas are associated with a number of letters of the alphabet .

6. The device of any preceding claim, wherein the stroke is a single pen-stroke passing through the areas which represent the characters forming the required word or command.

7. The device of any preceding claim, wherein the stroke for a particular word or character can take a plurality of different forms.

8. The device of claim 7, wherein the optimal form for a particular word may be determined.

9. The device of any preceding claim, wherein a shortcut stroke may be programmed to be recognised as a particular commonly entered word.

10. The device of any preceding claim, the predictor uses a look-up dictionary to predict the word or command entered.

11. A computer related device including a character entry device according to any of the preceding claims .

12. A method of entering a character for recognition in a computer related device, comprising: -detecting the contact or sequence of contacts of a stroke made by a stylus on one or more areas on an input means, the areas each associated with one or more characters and the areas being arranged in a predetermined orientation;

-predicting the possible character or characters for the contact or sequence of contacts to determine the most likely string of characters forming a word or command; and -selecting the correct word or command if there is more than one possible word or command.

13. A method of recognising gestures entered into an input means of a computer related device by means of a stylus, the method comprising:

-detecting the contact or sequence of contacts of the gesture on one or more areas on the input means , the areas being arranged in a predetermined orientation; -associating the contact or contacts with one or more characters associated the or each area;

14. An interface for interfacing the character entry device of any of claims 1 to 11 with a computer related device.