WO2003065192A1

WO2003065192A1 - Method, system and device for distinguishing pointing means

Info

Publication number: WO2003065192A1
Application number: PCT/FI2002/000072
Authority: WO
Inventors: Pekka Pihlaja
Original assignee: Nokia Corporation
Priority date: 2002-01-31
Filing date: 2002-01-31
Publication date: 2003-08-07
Also published as: EP1483656A1

Abstract

The present invention relates to a method for distinguishing pointing means (10, 11). A touch of a pointing means (10, 11) on a touch-operable device (2) is recognized. In the method at least one physical characteristic, which is generated by the touch of the pointing means (10, 11) on the touch-operable device (2), is examined and compared with at least one pre-determined reference characteristic. A decision on which one of different pointing means (10, 11) has touched the touch-operable device (2) is performed on the basis of said comparison. The invention also relates to a system and a device, in which the method is applied.

Description

Method, system and device for distinguishing pointing means

The present invention relates to a method for distinguishing pointing means, in which a touch of a pointing means on a touch-operable device is recognized. The invention also relates to a system comprising a touch-operable device, and means for recognizing a touch of a pointing means on said touch-operable device. The invention further relates to a device comprising a touch-operable device, and means for recognizing a touch of a pointing means on said touch-operable device.

BACKGROUND

Devices are known in which touch panels are used to perform opera- tions on the devices. For example, many personal digital assistants (PDA), information kiosks and palmtop computers can be operated and controlled by touching a touch panel by a pointing device. The touch panel recognizes the touch and typically also the co-ordinates of the touching point on the touch panel. The user of the device, which com- prises a touch panel, can perform e.g. selection operations, input data by handwriting on the touch panel with a pointing means, etc. Some devices accept a finger of the user to be used as the pointing means. The user can then use any finger to perform operations on the device, such as select a control operation from multiple control operations.

The touch panel is one example of such touch-operable (touch-sensitive) devices. Other touch-operable devices that could be mentioned are graphic tablets and touch pads. It is also possible to implement a switch based touch pad comprising a set of switches, wherein the switches operate as the means for recognizing a touch.

The devices provided with a touch panel typically comprise a display means located under the touch panel, wherein the touch panel is at least partly transparent. This kind of combination of a touch panel and a display means is also called a touch screen. The display means can be used to display different kind of information to the user of the device. For example, different selection areas with appropriate infor- mation can be presented on the display. An operation or set of operations is preferably defined for the selection areas. The corresponding selection areas are arranged to be recognized by the touch panel. When the user touches one of the selection areas on the touch panel, the co-ordinates of the contact point (touching point) are defined and compared with the selection area information. If the contact point is inside a selection area, the operation or set of operations defined for that particular selection area are performed. The touch panel can be used e.g. as a keyboard of a device. This is especially advantageous in palmtop computers to eliminate the need for a separate keyboard and thus reducing the size of the palmtop computer.

One disadvantage of prior art devices comprising a touch panel is that the touch panel does not necessarily recognize other pointing means than the one designed to be used with that particular touch panel. Therefore, if the user looses the pointing means, the touch panel cannot be used until the pointing means is found or a similar one is bought. The patent US-5,977,959 discloses a position-pointing device, which is arranged to be used with a position detecting apparatus. The device comprises communication means to communicate with the position detecting apparatus, and pressure detectors. Each detector can have a different function, for example, color. The user can select the function by pressing one of these pressure detectors. Therefore it is necessary to use the certain pointing device to operate the position detecting apparatus. It is also required that the user presses the correct pressure detector to perform the corresponding function. This kind of arrangement requires quite a complicated pointing device.

Another disadvantage of prior art systems and devices is that they cannot make a distinction between different pointing means. For example, the device with a touch panel cannot distinguish if the user has touched the touch panel with a pointing device or with a finger if the touch panel is sensitive to them both. Therefore, the operations that can be performed by touching the touch panel are mainly limited to certain range of selection and/or handwriting operations. SUMMARY OF THE INVENTION

It is an aim of the present invention to provide a new method, system and device for distinguishing pointing means touching a touch-operable device. The invention is based on the idea that different pointing means are distinguished by examining at least one physical effect that is generated by the touch of the pointing means. The effect or effects are then compared with certain preset parameters and a decision on which of the different pointing means has touched the touch-operable device is performed on the basis of this comparison. More specifically, the method according to the invention is primarily characterized in that at least one physical characteristic, which is generated by the touch of the pointing means on the touch-operable device, is examined and compared with at least one pre-determined reference characteristic, and a decision on which of the different pointing means has touched the touch-operable device is performed on the basis of said comparison. The system according to the invention is primarily characterized in that the system comprises means for examining at least one physical characteristic generated by the touch of the pointing means on the touch-operable device, means for comparing the examined at least one physical characteristic with at least one pre-determined reference characteristic, and means for deciding on the basis of said comparison, which of the different pointing means has touched the touch-operable device. The device according to the invention is primarily characterized in that the device comprises means for examining at least one physical characteristic, which is generated by the touch of the pointing means on the touch-operable device, means for comparing the examined at least one physical characteristic with at least one pre-determined reference characteristic, and means for deciding on the basis of said comparison, which of the different pointing means has touched the touch-operable device.

Considerable advantages are achieved by the present invention compared to the methods and systems of prior art. The touch or selec- tion of an area, can perform different tasks; tasks that are associated with different pointing means. Thus, new operations based on touch- operable devices can be introduced and devices comprising a touch- operable device can be made usable and user-friendly compared to devices of prior art. More operations can be performed with pointing means, wherein the device according to the invention is more user- friendly than prior art devices.

These and other features, aspects, and advantages of embodiments of the present invention will become apparent with reference to the following description in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for the purposes of illustration and not to define the limits of the invention, for which reference should be made to the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following the invention will be described in more detail with reference to the appended drawings, in which

Fig. 1 is a general block diagram of a device according to an advantageous embodiment of the present invention,

Fig. 2 illustrates the arrangement of sensing means in a device according to an advantageous embodiment of the present invention,

Fig. 3 illustrates the arrangement of sensing means in a device according to another advantageous embodiment of the present invention,

Fig. 4 illustrates an example of a stylus comprising two pointing means, and

Figs. 5a and 5b illustrate examples of using finger as a pointing means. DETAILED DESCRIPTION OF THE INVENTION

In the following the invention will be described in connection with a device 1 with reference to Fig. 1. The device 1 comprises a touch- operable device 2, e.g. a touch panel and/or a touch pad, which is placed on top of a display means 3. The operations of the device 1 are mainly controlled by a controller 4, such as a micro-processing unit (MPU). The device 1 further comprises memory means 5 for storing program codes for the controller 4, data, etc. The device 1 advanta- geously comprises audio means 6 and means 7 for mobile communication, such as a transmitter and a receiver for communicating e.g. with a GSM-cellular network or a UMTS-cellular network (not shown). The device can also have a keyboard or a keypad 8, but it is not necessary if input operations are only performed by the touch-operable device 2.

Sensing means 9 are arranged in connection with the touch-operable device 2. These sensing means 9 are arranged to perform operations to examine, which kind of effects the touching of a pointing means 10, 11 (Fig. 4) on the touch-operable device 2 affects. This will be explained later in this description.

To be able to use the touch-operable device 2 with different pointing means it may be necessary to set up the touch-operable device 2, i.e. to train the touch-operable device 2 to recognize different pointing means and to distinguish them from each other. However, the present invention is not limited solely to such embodiments in which such training or set-up procedure is needed. In the following, one advantageous example of the set-up procedure is described. The device 1 is set to a set-up mode by e.g. selecting from a menu. Now, the device 1 presents, for example on the display means 3, some guidance to the user about the set-up procedure. The user is asked to touch the touch panel 2 with one of the pointing means 10, 11. The touch-operable device recognises the touch of the pointing means 10, 11. Similarly, the sensing means 9 measure at least one physical characteristic related to the touch and store some information about the measured physical characteristic into the memory means 5. Then, the user may be asked to touch the touch panel 2 with another pointing means 10, 11. The touch-operable device again recognises the touch and the sensing means 9 measure and store at least one physical characteristic related to the touch. If necessary, the set-up procedure can be performed for more than two different pointing means. It may be necessary to recali- brate the touch-operable device 2 during the operation lifetime of the touch-operable device 2, because the pointing means and/or the touch- operable device 2 may wear due to the touching operations and/or the physical characteristics may change.

In some embodiments of the set-up procedure it is sufficient to perform the set-up procedure to all but one of those pointing means 10, 11 that are intended to be used with the touch-operable device 2. This means that, for example, if two pointing means 10, 11 will be used, the set-up procedure is performed for only one of them. Then, during the normal operation, the sensing means compare the stored characteristics to the measured characteristics and if the comparison result indicates sufficient similarity between them, it is decided that the pointing means corresponding to the stored characteristics touched the touch-operable device 2. Otherwise, it is decided that the other pointing means touched the touch-operable device. It is obvious that the above described set-up procedure is just an example, and different set-up procedures, if any, can be applied with different kinds of pointing means and with different embodiments of the present invention.

In the first embodiment of the method of the present invention it is assumed that two pointing means 10, 11 are used and that they are arranged as a stylus 12. This is illustrated in Fig. 4. The pointing means 10, 11 are advantageously located at opposite ends of the stylus 12. The first pointing means 10 is preferably made of at least partly differ- ent material and/or shape having a different hardness than the second pointing means 11. The usage of different materials and/or shapes having a different hardness affects that the touching of the touch panel generates a different effect depending on the pointing means 10, 11 used. For example, the first pointing means 10 can be made of a harder material than the second pointing means 11 or the shape of the first pointing means 10 can be sharper than the shape of the second pointing means 11. An advantageous embodiment of the sensing means 9 is shown in Fig. 2. The sensing means 9 are arranged to examine a physical characteristic, e.g. impact, which is generated by the pointing means 10, 11 which touches the touch-operable device 2 , the physical characteristic being different in connection with different pointing means 10, 11. The sensing means 9 comprise advantageously pressure-sensing means, such as an accelerometer or a set of force sensors. In the embodiment of the sensing means according to Fig. 2 pressure sensing means 9a, 9b, 9c, 9d are arranged in proximity of each corner of the touch-operable device 2.

Fig. 2 also illustrates a situation in which the first pointing means 10 of the stylus touches the surface of the touch-operable device 2.

In a situation in which the user touches the touch-operable device 2 with any of the pointing means 10, 11 , the following operations are performed in the device 1 according to an advantageous embodiment of the present invention. The touch-operable device 2 generates an indication of the touch and this indication is preferably location dependent. This indication is transmitted to an analogue-to-digital converter 13 (Fig. 1). The analogue-to-digital converter 13 converts the indication to digital form. This digital indication is then sent to the controller 4. The controller 4 detects that a new indication has arrived and begins to perform necessary operations. The detection can be e.g. interrupt-based or polling-based. This is known technique by an expert in the field wherein it is not necessary to explain it in more detail in this context. The controller 4 uses the digital indication to calculate the coordinates of the contact point. The calculated contact point is advanta- geously stored in memory means 5 for later use.

The sensing means 9 are used to measure the acceleration/time relationship and/or the force/time relationship caused by the impact of the pointing means 10, 11 touching the touch-operable device 2. The sensing means preferably generate a set of measurement results wherein the impact can be examined over a sufficient time period. The measurement results obtained are advantageously converted to digital form by the analogue-to-digital converter 13 and transmitted to the controller 4. The controller 4 receives the measurement results and stores them into memory means 5 for analysis. Some normalisation of the measurement results may be necessary, if significantly different signal strength values are obtained with different pointing means. After the measurement period is lapsed, the controller 4 starts to examine the measurement results. The controller 4 can calculate some deterministic feature from the measurement results, e.g. the change rate of the acceleration/time ratio. The controller 4 then compares the results to pre-determined reference characteristics in order to find the pre-determined characteristic, that most resembles the calculated results. If such pre-determined reference characteristic is found, the controller 4 can then determine that the properties of the pointing means 10, 11 , which touched the touch-operable device 2, are likely to be substan- tially similar with the properties corresponding to the pre-determined reference characteristic. The pre-determined reference characteristics can be determined, for example, by using the set-up procedure described earlier in this description.

The pre-determined reference characteristics are advantageously defined in a manner that the impact of, for example, two or more known pointing means are measured for a certain time (one pointing means at a time) and the measurement results are stored into the memory means 5. Each pre-determined reference characteristic can therefore contain more than one value to describe the impact in more detail.

It is also possible that only one pre-determined reference characteristic is stored in the memory means 5. In this kind of situation the above described measurement, calculation and comparison are carried out and, if the calculation result is similar or substantially similar than the pre-determined reference characteristic, it is determined that the properties of the pointing means is substantially similar to the properties corresponding to the pre-determined reference characteristic. Otherwise the other alternative is selected. For example, if the pre-deter- mined reference characteristic corresponds to a pointing means having with a relatively hard surface, the controller 4 preferably determines that a hard pointing means was used if the calculation results corre- spond to the pre-determined reference characteristic. Touching with a softer pointing device would cause different measurement and calculation results, wherein the controller 4 determines that the pointing means was not a pointing means having a hard surface.

Normally, the harder the pointing means, the higher the change rate of the acceleration/time ratio and the force/time ratio.

The present invention can also be applied in a manner that the user can select two or more pointing means for use from a larger number of pointing means. The user then informs the touch-operable device on the type of the pointing means the user intends to use. For example, the pointing means consists of the two different ends 10, 11 of the stylus 12, fingertips 14 and fingernails 15. The user informs e.g. by using a set-up menu which pointing means (s)he intends to use as the different pointing means, e.g. the first pointing means and the second pointing means. This kind of arrangement may be necessary if the physical characteristics of some of the pointing means are too close to each other for the touch-operable device to distinguish between all of them.

There are also other alternatives for implementing the pointing means than the stylus 12 presented above. In an advantageous embodiment of the invention the user can use her/his fingers to perform the selec- tion. The device 1 can differentiate between fingertips 14 and fingernails 15 by using the method according to the invention. Fig. 5a illustrates a situation in which the fingertip 14 is used as one pointing means and Fig. 5b illustrates a situation in which the fingernail 15 is used as another pointing means. Hence, the user can perform certain operations by touching the touch-operable device 2 with a fingertip 14 and other operations by touching the touch-operable device 2 with a fingernail 15.

Yet another alternative embodiment of the pointing means 10, 11 is a pen or pencil comprising preferably a pointed end and a blunt end. The sensing means 9 can then differentiate between the two ends by analysing some physical characteristic(s) of the touch. It is obvious to a person skilled in the art that the above-mentioned embodiments of different pointing means are only non-restrictive examples and other kind of pointing means can be applied with the present invention.

It is also possible that the device 1 can differentiate between more than two different pointing means, for example, between the pointing means 10, 11 of the stylus 12 and fingertips 14 and fingernails 15.

Some prior art touch-operable devices can distinguish between the following kinds of input operations performed by the user: tapping, double-tapping, sweeping and more complicated touching patterns as in character input. By having two or more distinguishable pointing means the number of possible input operations can be increased, if necessary, and/or those operations can be simplified by using the method according to the invention.

Many different operations can be defined for different kinds of pointing means. The user can, for example, perform a selection operation by using the first pointing means 10 (harder pointing means), or a fingernail, and a scrolling operation by using the second pointing means 11 (softer pointing means), or a fingertip. The scrolling can preferably be performed two-dimensionally, i.e. both vertical and horizontal directions of the touch-operable device 2. Another example is that keys of a keyboard are shown on the display means 3 of the device 1. The user can then type the base characters by fingertips 14 and the capitalized/shifted characters with fingernails 15, or vice versa. The coordinate information is used to determine which character was touched and the information of the sensing means 9 is used to determine whether the user touched the character with the first pointing means 10 (or fingernail) or with the second pointing means 11 (or fingertip). Yet another example is that the fingernail and fingertips correspond to different buttons of a pointing device, such as a computer mouse. Therefore, the device 1 interprets a touching of the touch-operable device with a fingertip 14 as e.g. a left mouse button click and a touching of the touch-operable device with a fingernail 15 as a right mouse button click, or vice versa. It is also possible to define a positive action (e.g. a selection of an application or a function, or step forward function, etc.) to one of the different pointing means and a negative action (e.g. quit an application or a function, an undo function, step- back function, escape function, etc.) to another one of the different pointing means.

A computer mouse comprising a wheel is known. The wheel can be used e.g. to scroll information on the display means 3. Similar opera- tion can be performed by the device of the present invention. For example, if a user trails a fingertip 14 on the touch-operable device 2, the device can interpret this as a scrolling of a mouse wheel.

The method according to the invention can also be applied in hand writing and erasing similarly as with a pencil and an eraser. The user can use the first pointing means 10 for handwriting on the touch- operable device 2 and the second pointing means 11 for erasing information by tapping on the touch-operable device 2 with the second pointing means 11 and trailing it on the surface of the touch-operable device 2.

In the embodiment of the sensing means 9 shown in Fig. 2, four pressure-sensing means 9a, 9b, 9c, 9d, one in each corner, are used. The distance between the touching point and the pressure sensing means 9a, 9b, 9c, 9d can affect the signal level of the pressure sensing means. Therefore, some kind of normalisation of the signal may be performed, if the signal level is too low or too high. However, the distance does not significantly affect the acceleration/time ratio and/or the force/time ratio caused by the impact of the pointing means 10, 11 touching the touch-operable device 2.

In a second advantageous embodiment of the present invention the sensing means 9 comprise e.g. acoustic means such as piezo-electric transformer, which transforms an acoustic signal to an electric signal. One example of this advantageous embodiment is illustrated in Fig. 3. In this embodiment the acoustical properties of the touch are examined and compared to the pre-determined reference characteris- tic/characteristics. The sound generated by the touch is different in connection with different pointing means 10, 11 , wherein the acoustic information can be used to determine, which kind of pointing means (hard/soft) touched the touch panel. There are many possibilities to perform the examination. For example, one or more low-pass filters, high-pass filters and/or band-pass filters can be used. The electric signal representing the acoustic signal generated by the sensing means is input to e.g. a low-pass filter and to a high-pass filter. If the signal is less attenuated when passing through the low-pass filter than when passing through the high-pass filter, the acoustic signal consists of lower frequencies and vice versa. Normally, softer material produces lower frequencies than harder material.

By using the combination of the low-pass filter and the high-pass filter and selecting the cut-off frequencies of the filters appropriately it is also possible to distinguish a third pointing means, which generates frequencies which are in between the frequencies generated by the first pointing means and the second pointing means.

Yet another alternative is that the electric signal generated by the sensing means and representing the acoustic signal is first transformed to a frequency domain to form the frequency spectrum of the signal. Then, the frequency components of the frequency spectrum are examined to determine, which frequencies (high/low) are present and how strong the different frequency components are. The result of the determination can then be used to determine the type of the pointing means 10, 11 in question. It should be noted here that it is not necessary to examine all the frequency components of the frequency spectrum but only some of them. For example, a frequency limit can be defined, wherein it is determined whether the strongest frequency components of the frequency spectrum are located below or above that limit. This determination can then be used to distinguish the pointing means. It is also possible that the set-up procedure is performed to train the sensing means 9 to distinguish between different pointing means. The principles of the set-up procedure have been illustrated hereinabove. Fig. 3 also illustrates a situation in which the second pointing means 11 of the stylus touches the surface of the touch-operable device 2.

Preferably, the sensing means 9 are not used alone but as a combina- tion with the touch-operable device, wherein the sensing means 9 are only activated for the distinction between different pointing means when the touch-operable device 2 has detected a touch.

The distinction between the pointing means may also be based on other signals than those mentioned above. If the touch-operable device 2 produces a signal which depends on the area where the pointing means touches the touchable surface of the touch-operable device, this signal can be used to distinguish the pointing means. Moreover, especially when the pointing means is made of a soft and/or elastic material, the area varies with relation to the force the user taps on the touch-operable device 2, wherein the force (and the force/time ratio) can be determined from said signal and no external force sensor nor accelerometer is needed.

The present invention can be applied with many devices which comprise some kind of human input functions. For example, the invention can be applied with wireless communication devices, information kiosks, PDAs, laptop computers, palmtop computers, portable media terminals, graphic tablets, control terminals, etc.

It is obvious that the present invention is not limited to the above- described embodiments, but it can be modified within the scope of the appended claims.

Claims

Claims:

1. A method for distinguishing pointing means (10, 11), in which a touch of a pointing means (10, 11) on a touch-operable device (2) is recognized, characterized in that at least one physical characteristic generated by the touch of the pointing means (10, 11) on the touch- operable device (2), is examined and compared with at least one predetermined reference characteristic, and a decision on which of the different pointing means (10, 11) has touched the touch-operable device (2) is performed on the basis of said comparison.

2. A method according to Claim 1 , characterized in that said at least one physical characteristic is acceleration/time relationship.

3. A method according to Claim 1 or 2, characterized in that said at least one physical characteristic is force/time relationship.

4. A method according to Claim 1 , 2 or 3, characterized in that said at least one physical characteristic is the acoustical properties of the touch.

5. A method according to any of the Claims 1 to 4, characterized in that said at least one physical characteristic is the surface area of the touch.

6. A method according to any of the Claims 1 to 5, characterized in that the hardness of the pointing means is evaluated by said at least one physical characteristic.

7. A method according to any of the Claims 1 to 6, characterized in that a set-up procedure is performed to determine said at least one pre-determined reference characteristic.

8. A system comprising a touch-operable device (2), and means for recognizing a touch of a pointing means (10, 11) on said touch- operable device (2), characterized in that the system comprises means (9) for examining at least one physical characteristic generated by the touch of the pointing means (10, 11) on the touch-operable device (2), means for comparing the examined at least one physical characteristic with at least one pre-determined reference characteristic, and means (4) for deciding on the basis of said comparison, which of the different pointing means (10, 11) has touched the touch-operable device (2).

9. A system according to Claim 8, characterized in that said means (9) for examining at least one physical characteristic comprise an accelerometer.

10. A system according to Claim 8 or 9, characterized in that means (9) for examining at least one physical characteristic comprise a force sensor.

11. A system according to Claim 8, 9 or 10, characterized in that means (9) for examining at least one physical characteristic comprise a piezo-electric transformer.

12. A device comprising a touch-operable device (2), and means for recognizing a touch of a pointing means (10, 11) on said touch-operable device (2), characterized in that the device comprises means (9) for examining at least one physical characteristic generated by the touch of the pointing means (10, 11) on the touch-operable device (2), means for comparing the examined at least one physical characteristic with at least one pre-determined reference characteristic, and means (4) for deciding on the basis of said comparison, which of the different pointing means (10, 11) has touched the touch-operable device (2).

13. A device according to Claim 12, characterized in that it comprises display means (3), and that said touch-operable device (2) is located on said display means (3).

14. A device according to Claim 12 or 13, characterized in that a setup procedure is performed to determine said at least one pre-determined reference characteristic, wherein the device (1) comprises means (5) to store said at least one pre-determined reference characteristic.

15. A device according to Claim 12, 13 or 14, characterized in that it comprises means (9) for mobile communication.