WO2004107146A2

WO2004107146A2 - A data input method for a computing device

Info

Publication number: WO2004107146A2
Application number: PCT/GB2004/002302
Authority: WO
Inventors: Martin Riddiford
Original assignee: Therefore Limited
Priority date: 2003-05-30
Filing date: 2004-06-01
Publication date: 2004-12-09
Also published as: WO2004107146A3; GB0412212D0; GB2402105A; GB0312465D0

Abstract

A data input method for a computing device using a touch screen; selection of a virtual key occurs when a user presses down on the screen overlying the virtual key with a force exceeding a first pre-defined force and confirmation of selection of a virtual key occurs when the user presses down on the screen overlying the virtual key with a force exceeding a second pre-defined force and actuates a mechanical switch underlying the touch screen. Confirmation of selection in this manner provides tactile feedback missing from conventional touch screen systems. Because it more closely mimics a real world action of hitting a real world key, it is both more satisfying and more easily understood by a user. The invention stands in contrast to the established view that confirmation of the selection of a virtual key should be achieved when a user lifts off from the screen and without any form of tactile feedback.

Description

A DATA INPUT METHOD FOR A COMPUTING DEVICE

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a data input method for a computing device using a touch screen. It enables 'virtual keys' to be selected.

2. Description of the Prior Art

The term 'computing device' covers any kind of computing device that a person interacts with by selecting virtual keys that are shown on a display. As such, the term covers PDAs, mobile telephones, dedicated devices such digital audio players, remote control units, desktop PCs etc. The term 'key' should be expansively construed to cover any object which a user would 'select' in some way (e.g. choose in order to: (a) input data, such as text/numeric data; (b) initiate an action; (c) control or manipulate a function; or (d) to identify data which is to be manipulated). It therefore covers keys or tiles from a keyboard (such as letter or number keys), a control button or slider, a jog dial, a joystick, or a roller button etc. These 'keys' may be generated by the computing device and shown on the display (e.g. a LCD display). Alternatively, the keys may be permanently formed on a layer which can be illuminated by an electroluminescent screen. A 'virtual key' is therefore an image of a key which only appears when caused to do so by a computing device. The term 'virtual key' also covers items that do not correspond with objects as such; in particular, it covers any icon or image (a) which a user might select and (b) which only appears when caused to do so by a computing device.

Operating a 'virtual key' can be done in several ways and is a two stage process. First, a user has to select a key. Subsequently, the user has to confirm selection. In a PC environment, the user could point at a virtual key to select that virtual key and then click with a mouse to confirm selection. Where the computing device has a touch screen, then the user would simply touch the area of the display overlying the virtual key with a finger, pen or stylus to select that virtual key; when he lifts his finger/pen off the screen, the selection of the virtual key is confirmed. A synthetic audio 'click' is sometimes generated to provide feedback of confirmation. It is a common mis-conception that touch screens confirm selection of a virtual key when pressure is applied; they in fact confirm only when pressure is released.

Touch screens typically comprise an ITO layer overlying a display (such as a LCD) and are pressure sensitive. When pressure is applied to the ITO overlay, the top flexible surface is locally deformed so that it makes contact with a stiff /solid base layer. An analogue voltage level across the ITO coating on the inside surface layer is interpreted by the computer to give x/y coordinates for the pressure point. Feedback indicating that a key has been selected is often given by highlighting the zone which has been touched on the underlying LCD, or animating a button to look like it has been pressed. Movement of the point of contact can then change the highlighted zone, highlight a word, or be used for pull down menus, or for digital ink etc. Confirmation of selection of a particular virtual key occurs only when the pressure is released.

Because conventional touch screens enable the contact point to be moved so that different screen areas can be selected whilst the pen/stylus is 'down', 'drag and drop' is generally avoided since it would require that a selected item is dragged when the contact point is moved.

SUMMARY OF THE INVENTION

The invention is a data input method for a computing device using a touch screen; selection of a virtual key occurs when a user presses down on the screen overlying the virtual key with a force exceeding a first pre-defined force and confirmation of selection of the virtual key occurs when the user presses down on the screen overlying the virtual key with a force exceeding a second pre-defined force and actuates a mechanical switch underlying the touch screen.

Confirmation of selection in this manner provides tactile feedback missing from conventional touch screen systems. Because it more closely mimics a real world action of hitting a real world key, it is both more satisfying and more easily understood by a user. The invention stands in contrast to the established view that confirmation of the selection of a virtual key should be achieved when a user lifts off from the screen and without any form of tactile feedback. It should be noted that the conventional approach of confirming selection only on release is simpler, but less effective. It is simpler because it requires no secondary mechanical switch; this inherent simplicity has led to the overwhelming technical bias in the field of touch screens being towards preserving pressure release as the indication of confirmation of key selection.

The step of selection of a key (which clearly has to precede confirmation of that selection) may also be associated with various forms of feedback to clearly indicate to the user that a particular key has been selected. For example, visual feedback is possible: when the user selects a virtual key (but without pressing on it so hard that the switch is actuated and selection thereby confirmed), the virtual key could change colour, or a preview window could open, showing the identity of the particular key that has been selected. Hence, as the user moves over different keys with this light pressure, different keys are shown as being selected.

Voice feedback is also possible: the name of a key that has just been selected could be spoken by a synthetic voice. This approach would be very useful when selecting keys on a device whilst driving - e.g. dialling a telephone number on a mobile telephone. It would also be very useful to blind/visually impaired users. As polyphonic sound quality improves, it will also be possible to use voices belonging to different pop stars, actors etc, with users downloading the appropriate sound files needed in the same way that ringtones can be bought and downloaded today. Another feedback approach is the use of low profile guide lines or tactile features on the touch screen itself, so that the user's sense of touch can also be used to aid the process of selection.

Actuation of the switch that occurs when the user presses down on a virtual key beyond the first pre-defined force may cause a control signal to be sent from the switch to result in that virtual key being selected. The switch then acts as a gate, so that the X-Y coordinates continuously sensed by the touch screen as the user brushes against the screen above the first pre-defined force but below the second pre-defined force threshold (i.e. without actuating the switch) does not result in erroneous key selection confirmation; instead, the user must positively press down in excess of the second threshold to actually confirm selection of the key. Actuation of the switch then triggers the immediate capture and use of X-Y co-ordinates of the area that is being pressed down upon, as measured by the touch screen system.

If the virtual key is an item that can be dragged, then a light touch on it, sufficient to identify the touch position to the touch screen but not enough to activate the switch, will select that item. The item can then be dragged and dropped in the normal manner.

The present invention can be implemented using a conventional ITO layer overlying a LCD display. The switch underlying the touch screen can be a single (or multiple) switch which is actuated when the touch screen is pressed with a force above a pre-defined threshold. In one implementation, the whole touchscreen and LCD assembly are mounted in a combined unit hinged at one edge with the back of the unit resting on the switch, with the switch mounted in a suitable position so that the force to actuate the switch in normal use does not vary beyond a pre-defined and limited extent across the entire touch screen - i.e. the hinge axis is far away from finger press area. Two switches may also be practical for larger areas. Other switch variants, such as that disclosed in EP0419145, are also possible. The switch is preferably an electro-mechanical contact switch.

The switch provides one or more of the following advantages: A. It gives tactile feedback, with the user sensing the slight movement of the hinged touch screen pivoting; it feels just like pressing mechanical keys, unlike a conventional touch screen. No such tactile feedback arises with conventional touch screens.

B. It positively confirms the selection of a virtual key, whilst still allowing hovering before confirmation — i.e. the user is made aware that he has actually confirmed selection of a virtual key through the movement associated with the screen pivoting. But the user can still draw his finger over an area without confirming selection of the underlying keys if the pressure is light enough, but once he presses with a force exceeding a threshold, the switch positively closes, confirming selection of the underlying key. Hence, the mechanical switch acts as a gate so that the X-Y co-ordinates continuously sensed as the user brushes against the screen at below a pre-defined force level (i.e. without actuating the switch) does not result in erroneous confirmation of key selection; instead, the user must positively press down in excess of a threshold to confirm selection of the key.

C. It switches at the same time as selection is confirmed, i.e. actuation of the switch triggers the immediate capture and use of the X-Y co-ords of the area that is being pressed down upon, as measured by the ITO system, unlike conventional touch screen systems in which a user can press down on a virtual key but the actual switching occurs only later on release.

D. It gives mechanical audio feedback when the switch is fully depressed, unlike a conventional touch screen which requires a synthetic audio feedback.

E. Drag and drop is possible, with an item to be dragged being selected by a light press down action that does not actuate the switch but can nevertheless be sensed by the touch screen; with pressure maintained, the user can then drag the selected item and drop it at the required destination by lifting up. Another approach would be for the item only to be dragged after confirmation of selection had been made — i.e. a press down hard enough to activate the mechanical switch. The item can then be dragged whilst pressure is maintained and dropped by ^• releasing. This mirrors how drag and drop is achieved using a normal mouse. This latter approach can also be used to highlight text and multiple icons in the same way that a mouse can highlight, i.e. the user presses down at the start of the region to be highlighted with a force sufficient to activate the switch and then, with pressure maintained, drags the contact point to the end of the region being highlighted.

F. Various forms of feedback are possible on selection (but prior to confirmation), such as audio, tactile or visual feedback.

The mechanical switch does not have to gate the touch screen output in the manner described above: it may simply provide tactile feedback. Then, a touch screen could employ a controller modified so that confirmation of key selection occurs on a 'pen down' rather than a 'pen up' action. Positive confirmation of key selection can then be achieved by having a touch screen that can position sense once the touch force exceeds one level, and trigger confirmation of key selection only if a force exceeding a higher level is applied. This approach does however require a more costly type of touch screen.

Several virtual keys may be displayed in proximity to one another and any ambiguity in the identify of the key which the user intended to select is resolved by disambiguation software running on the device. Hence, virtual keys for all letters of the alphabet could be displayed (e.g. a QWERTY or ABCD ordered keypad) and/or numerics 0 to 9. An input to the disambiguation software could be the proximity of a touch to a given area or point uniquely associated with a given key. Hence, the software relates the likelihood of a given key being the correct key to the proximity of the actual strike area on the touch screen to that of the key.

There may be no delineation between one or more of the virtual keys — e.g. the display simply shows an array of letters, with no apparent key tiles. This leads to a cleaner, less cluttered display. Another feature is that the device could display different virtual keys depending on the application being used by the user or the appropriate inputs needed for a given application. Hence, if the device was a smartphone, it could display a virtual QWERTY keyboard if the user activates a messaging application, or controls for a digital audio player if a digital audio application is opened.

The device may also comprise a touch screen ITO layer on the left side of the device to receive input from the left hand and a second touch screen ITO layer on the right side to receive input from the right side. This is useful for portable devices that are meant to allow two handed data input (e.g. to allow games to be played). Having two separate ITO layers, each with its own controller, overlying the virtual keys for each different hand allows simultaneous input from each hand: with a single ITO layer overlying the entire screen, a simultaneous input to the far left of the screen and to the far right would be averaged out to represent a touch to the center of the screen — clearly inappropriate for a device that should allow two handed input.

The device might use a backlit electroluminescent display which comprises several different layers, each layer comprising a different set of virtual keys. Hence, one layer could be a numeric keypad, another layer could be a QWERTY keypad, another control buttons for a TV remote control etc.

DETAILED DESCRIPTION

The present invention is implemented as the Cliqa™ virtual keyboard from Therefore Limited product design consultants of London, UK. The Cliqa virtual keyboard addresses the problem of how the smartphone keyboard should evolve, given that smartphone size will stay the same, yet screens will get bigger, applications more complicated and text input become more important.

The conventional solution is arguably a compromise in usability. For example, with a typical flip phone, the keyboard already requires at least 20 multi- function keys to manage the proliferation of applications, as shown in Figure 1.

The Cliqa virtual keyboard addresses this problem through a single intelligent key, as shown in the lower leaf of the flip phone in Figure 2. The Cliqa virtual keyboard is an assembly of a mono touch-screen LCD 1 with an electro-mechanical 'click' or dome switch 2 underneath, as shown in Figure 3. The screen 1 is hinged at pivot 3 running along one edge. The 'click' switch 2 is a conventional metal dome switch that deforms gradually under increasing force applied by a user's finger tip/stylus, and at a certain position, deforms rapidly and easily, giving the use a clear impression that the switch has been activated at that position. The Cliqa virtual keyboard displays virtual keys, always appropriate to the task at hand, but with a real mechanical key click action. An initial light press on the Cliqa virtual keyboard can select an item displayed on the touch-screen: the touch screen ITO has sensed the contact position over the item and can hence highlight that item. Only with a more positive press, sufficient to actuate the switch, is the item confirmed for selection, in the sense of being committed for data entry.

A firm press on the touch screen, as shown in Figure 4, causes the switch to actuate: the touch screen and display are a combined unit, hinged at one end and resting on the switch. As the combined unit is depressed beyond a certain distance, the switch is actuated. The switch acts as a gate, with actuation of the switch triggering the immediate capture and use of the X-Y co-ordinates of the area that is being pressed down upon, as measured by a conventional ITO touch screen system. This confirms selection of the virtual key being pressed. On lifting up from the screen, the combined touch screen and display pivot back upwards, as shown in Figure 5 under the action of a spring or deformable member The tactile experience is hence very close to a user pressing down on a real key.

Single-handed QWERTY input is achieved using enhanced predictive text (disambiguation) software. Normally, predictive text software cannot make any assumptions about which key the user intended to select prior to running its dictionary look up process. The Cliqa virtual keyboard can since it can measure the distance of the contact point or area to different keys and weight more heavily the key which is closest to the strike area.

The Cliqa virtual keyboard can be virtually split for two-handed operation (gaming), as suggested in Figure 6: two ITO layers are used; this will typically be a single physical layer divided into two logical sides, e.g. inputs to one side are independent of inputs to the other side and do not interfere with one another.

Figure 7 is an illustration of various possible Cliqa virtual keyboard phone interfaces, as amplified in Figure 8 — 13. Figure 8 is an illustration of how numeric virtual keys can be used for a smartphone interface. A clear preview window onto the LCD provides Caller ID when the phone is closed, without the need for a third LCD.

Figure 9 shows how a QWERTY letter virtual keyboard can be deployed. Because the keyboard is virtual, it can readily be made to show the actual font that will be deployed — e.g. font type, upper case/lower case, italics, bold etc. Figure 10 shows how a MP3 virtual keyboard could appear. Figure 11 shows how an in-car navigation interface might appear. Figure 12 shows how a photo album interface might appear. Figure 13 shows how a mouse pad interface might appear. All of these interfaces could be available on the same device at different times, depending on the application that was being run.

Figure 14 shows several devices; moving clockwise from the top left, there is a universal remote control, a bar phone and a Bluetooth watch/controller, all using the Cliqa virtual keyboard concept to provide tactile confirmation of virtual key selection. Figures 15 — 18 show another variant in which there is a hinged panel 4 that in one position overlies at least part of a display 5, the hinged panel 4 is the screen that the user presses down on to select and confirm selection of virtual keys. It itself comprises a translucent panel 6 through which virtual keys shown on the display 5 are visible. The hinged panel 4 may include the device microphone. This variant is easier to integrate into current device designs since it requires minimal changes: the display 5 is a touch screen display and can be activated by a user pressing on the translucent panel 6. The hinged panel 4 is mounted against a simple mechanical switch to detect pressure sufficient to confirm selection of a virtual key as described above. Figure 15 shows the device displaying a numeric keypad for dialing etc. Figure 16 shows the device with panel 4 open — this shows how the screen 5 displays the numeric keypad in an entirely conventional manner; the numeric keypad can still be used in the conventional way using touch screen 5. Other virtual keys can be displayed: Figure 17 shows a set of calculator virtual keys that the closed panel 4 overlies. The calculator virtual keyboard can be simply generated using conventional programming techniques and can float, as shown in Figures 18 and 19. Figures 15 - 18 depict a variant in which the display is sub-divided into different regions, with different touch related rules applying to the different regions. In the implementation described in Figures 15 - 18, the boundary between the region that operates the Cliqa system is clear — it is only the area underlying the translucent panel 6. For the rest of the display, conventional touch input behaviour still applies. The boundary need not however be so explicit; it is possible for a device to have a single display, fully overlaid with a hinged Cliqa system (or integrated with a Cliqa system) and for only certain regions of the display to activate Cliqa functionality. For example, the lower third could be a Cliqa touch pad zone, with the upper two thirds behaving conventionally. This may have some UI benefits — for example, all virtual keys could appear solely in the Cliqa touch pad zone, leaving the rest of the display able to accept conventional pen input.

Claims

1. A data input method for a computing device using a touch screen, in which (a) selection of a virtual key occurs when a user presses down on the screen overlying the virtual key with a force exceeding a first pre-defined force and (b) confirmation that a virtual key is selected occurs when the user presses down on the screen overlying the virtual key with a force exceeding a second pre-defined force, and actuates a switch underlying the touch screen.

2. The method of Claim 1 in which actuation of the switch that occurs when the user presses down on a virtual key causes a control signal to be sent from the switch that results in that virtual key being confirmed as selected.

3. The method of Claim 2 in which the step of actuation of the switch triggers the immediate capture and use of X-Y co-ordinates of the area that is being pressed down upon, as measured by an ITO touch screen system.

4. The method of Claim 1 in which the switch provides mechanical audio feedback of its actuation.

5. The method of Claim 1 in which 'drag and drop' is possible, in which the step of selecting a virtual key to be dragged is achieved when the user presses down on the virtual key with a force above the first pre-defined force but not exceeding the second pre-defined force, the step of dragging is achieved when the user alters the contact point at which he presses down and the step of dropping is achieved when the user lifts off from the screen.

6. The method of Claim 1 in which 'drag and drop' is possible, in which the step of selecting a virtual key to be dragged is achieved when the user presses down on the virtual key with a force above the second pre-defined force, the step of dragging is achieved when the user alters the contact point at which he presses down and the step of dropping is achieved when the user lifts off from the screen.

7. The method of Claim 1 in which highlighting is possible, in which the step of selecting a virtual key to be highlighted is achieved when the user presses down on the virtual key with a force above the second pre-defined force, the step of marking text to be highlighted is achieved when the user moves the contact point at which he presses down over all of the text to be highlighted and the step of confirming selection is achieved when the user lifts off from the screen.

8. The method of Claim 1 in which the mechanical switch is arranged so that the second pre-defined force does not vary beyond a pre-defined and limited extent across the entire touch screen.

9. The method of Claim 1 in which several virtual keys are displayed in proximity to one another and any ambiguity in the identify of the key which the user intended to select is resolved- by disambiguation software running on the device which relates the likelihood of a given key being the correct key to the proximity of the actual strike area on the touch screen to that of the key.

10. The method of Claim 9 in which virtual keys for all letters of the alphabet are displayed.

11. The method of Claim 10 in which there is no delineation between one or more of the virtual keys.

12. The method of Claim 7 in which a numeric keypad is displayed.

13. The method of Claim 1 in which the device can display different virtual keys depending on the application being used by the user or the appropriate inputs needed for a given application.

14. The method of Claim 1 in which the step of selection of a key that occurs prior to confirmation of that selection is accompanied by audio, visual and/or tactile feedback.

15. The method of Claim 14 in which the audio feedback is the name of the key generated by a synthetic voice.

16. The method of Claim 14 in which the visual feedback is the identity of the key displayed in a preview window.

17. The method of Claim 14 in which the tactile feedback arises from low profile guide lines or features on the touch screen.

18. A computing device comprising a touch screen displaying a virtual key, in which (a) selection of a virtual key occurs when a user presses down on the screen overlying the virtual key with a force exceeding a first pre-defined force and (b) confirmation that a virtual key is selected occurs when the user presses down on the screen overlying the virtual key with a force exceeding a second pre-defined force, and actuates a switch underlying the touch screen.

19. The device of Claim 18 operable so that actuation of the switch that occurs when the user presses down on a virtual key causes a control signal to be sent from the switch that results in that virtual key being confirmed as selected.

20. The device of Claim 18 operable so that actuation of the switch triggers the immediate capture and use of X-Y co-ordinates of the area that is being pressed down upon, as measured by an ITO touch screen system.

21. The device of Claim 18 in which the switch provides mechanical audio feedback of its actuation.

22. The device of Claim 18 operable to allow 'drag and drop', in which the step of selecting a virtual key to be dragged is achieved when the user presses down on the virtual key with a force exceeding the first pre-defined force but not exceeding the second pre-defined force, the step of dragging is achieved when the user alters the contact point at which he presses down and the step of dropping is achieved when the user lifts off from the screen.

23. The device of Claim 18 in which 'drag and drop' is possible, in which the step of selecting a virtual key to be dragged is achieved when the user presses down on the virtual key with a force above the second pre-defined force, the step of dragging is achieved when the user alters the contact point at which he presses down and the step of dropping is achieved when the user lifts off from the screen.

24. The device of Claim 18 in which highlighting is possible, in which the step of selecting a virtual key to be highlighted is achieved when the user presses down on the virtual key with a force above the second pre-defined force, the step of marking text to be highlighted is achieved when the user moves the contact point at which he presses down over all of the text to be highlighted and the step of confirming selection is achieved when the user lifts off from the screen.

25. The device of Claim 18 in which the switch is arranged so that the pre-defined force does not vary beyond a pre-defined and limited extent across the entire touch screen.

26. The device of Claim 18 in which several virtual keys are displayed in proximity to one another and any ambiguity in the identify of the key which the user intended to select is resolved by disambiguation software running on the device which relates the likelihood of a given key being the correct key to the proximity of the actual strike area on the touch screen to that of the key.

27. The device of Claim 26 in which virtual keys for all letters of the alphabet are displayed.

28. The device of Claim 27 in which there is no delineation between one or more of the virtual keys.

29. The device of Claim 26 in which a numeric keypad is displayed.

30. The device of Claim 18 in which the device can display different virtual keys depending on the application being used by the user or the appropriate inputs needed for a given application.

31. The device of Claim 18 which comprises a touch screen layer on the left side of the device to receive input from the left hand and a second touch screen layer on the right side to receive input from the right side.

32. The device of Claim 18 in which the display is a LCD display.

33. The device of Claim 18 in which the display is a backlit electroluminescent display.

34. The device of Claim 33 in which the electroluminescent display comprises several different layers, each layer comprising a different set of virtual keys.

35. The device of Claim 18 which operates such that the step of selection of a key that occurs prior to confirmation of that selection is accompanied by audio, visual and/or tactile feedback.

36. The device of Claim 35 in which the audio feedback is the name of the key generated by a synthetic voice.

37. The device of Claim 35 in which the visual feedback is the identity of the key displayed in a preview window.

38. The device of Claim 35 in which the tactile feedback arises from low profile guide lines or tactile features on the touch screen.

39. The device of Claim 18 comprising a hinged panel that in one position overlies at least part of a display, the hinged panel itself being the screen that the user presses down on and itself comprises a translucent panel through which virtual keys shown on the display are visible.

40. The device of Claim 39 in which the hinged panel is a flip panel that comprises a microphone.