AT519427B1 - Method for controlling a data processing system - Google Patents

Abstract

The invention relates to a method for controlling a data processing system (5) with a position-resolving touch-sensitive input surface (1) and any number and type of input objects, wherein at least one input object is an input device (2) comprising a contact sensor (3) and an inertial sensor (3). 4).

Description

The invention relates to a method for controlling a data processing system with a position-resolving touch-sensitive input surface and any number and type of input objects, at least one input object being an input device which has a contact sensor and an inertial sensor.

A data processing system in the sense of this document is a device that processes input signals from input devices and generates data or output signals for output devices therefrom.

An input surface in the sense of this document is a device which is touch-sensitive, that is, senses touches or approaches of objects, and is position-resolving, i.e. detects the positions at which the objects are approached, or the touches take place and this information in the form sends a signal to a data processing system.

An input object in the sense of this document is any object whose contact or approach is perceived by the input surface.

An input device in the sense of this document is any object, the touch or approach of which is perceived by the input surface and which additionally has a contact sensor and an inertial sensor.

A contact sensor in the sense of this document is a sensor that already detects the approach of the input device to a surface or only detects the touch of the surface by the input device.

An inertial sensor in the sense of this document is a sensor that detects the relative movement of the input device in space in or around at least one spatial direction, for example in the form of acceleration.

[0008] US2016139690A1 shows an input device in the form of an input pen, which can have, among other things, a contact sensor and an inertial sensor, wherein the input device can be used on a position-resolving touch-sensitive input surface for data input. Paragraph [0069] states that several styluses can be used to control devices or applications with the sensor data of the inertial sensors. However, US2016139690A1 does not describe a solution as to how a distinction can be made in the case of input with several input objects on a position-resolving touch-sensitive input surface, which input object is responsible for which touch point on the input surface.

The US2014146021A1 also shows an input device, in the form of an input pen, which may include a contact sensor and an inertial sensor. Paragraph [0031] states that multiple styluses can be used to transmit data to a display device. However, US2014146021A1 also does not describe a solution as to how a distinction can be made when entering with multiple input objects on a position-resolving touch-sensitive input surface, which input object is responsible for which touch point on the input surface.

The US2015049022A1 describes a method for entering data on a position-resolving touch-sensitive input surface with an input device in the form of a computer mouse, that is, an input device that detects the two-dimensional movement of itself along a surface. It is proposed to compare the absolute two-dimensional movement signal of a point of contact on the input surface and the relative two-dimensional movement signal of the computer mouse to determine whether these movements are identical. A disadvantage of using an input device in the form of a computer mouse, both in the case of an optical mouse or a ball mouse, is that its sensor for detecting the relative movement is susceptible to damage or ver

1.17

AT519 427 B1 2019-02-15 Austrian

Patent office dirt is and / or some surfaces do not allow exact detection of the relative movement.

The object underlying the invention is based on US2015049022A1 to create a method to enable the simultaneous use and differentiation of several identical input devices in the control of a data processing system with any position-resolving touch-sensitive input surface, being identical in this context means that their identity cannot be determined solely from the signals of the position-resolving touch-sensitive input surface, without having to equip the input devices with the functionality of a computer mouse as suggested in US2015049022A1.

To solve the problem, it is proposed to equip each input device with a contact sensor, for example mechanical, capacitive or optical, and inertial sensor, preferably in or near a tip of the input device, and to continuously send the data of these sensors to the data processing system, wherein In addition, the positions of all touch points that the position-resolving touch-sensitive input surface perceives, are continuously sent by it to the data processing system and for each perceived touch point and for the data sent by an input device, the probability with which this data is correlated is calculated, first it is determined whether there is a temporal correlation of the occurrence of a signal from the contact sensor of one or more input devices with the occurrence of one or more contact points and, if so, the probability with which the course occurs it touch point on the position-resolving touch-sensitive input surface and the course of the spatial movement of an input device detected by the inertial sensor describe the same or a correlating movement.

The assignment of input objects and input devices can be done according to the following scheme:

As soon as the data processing system perceives the appearance of a touch point on an input surface, it determines whether the contact sensors of one or more input devices have become active within a short time before or after the appearance of the touch point. If so, the possible combinations of touch points on the input surface and touch signals from contact sensors are stored. The following options are available:

- There is no temporal correlation of a touch signal from a contact sensor with the appearance of a touch point, then the touch point is treated as "nopen touch", that is, as not caused by an input device.

- The appearance of exactly one point of contact on the input surfaces correlates in time with exactly one touch signal from a contact sensor, then this point of contact can be treated as a "pen touch", that is, as caused by an input device.

If there are several temporal correlations between the appearance of touch points on the input surface and the appearance of touch signals from contact sensors, each possible correlation combination is assigned a probability value which indicates how likely it is that an input device is associated with that of the correlation combination Contact sensor has touched an input surface at the location where the contact point associated with the correlation combination has appeared. In this case, steps a-c are carried out.

[0018] a. The probability value of each correlation combination is subsequently updated at very short, for example regular, time intervals. For this purpose, on the one hand the data of the contact sensor belonging to the correlation combination are compared with the activity of the contact point belonging to the correlation combination, on the other hand the movement data of the correlation

2.17

AT519 427 B1 2019-02-15 Austrian

Patent office combination associated inertial sensor with the movement trajectory of the point of contact associated with the correlation combination. If there is a high match of the matched data, the probability value of the correlation combination increases, otherwise it decreases.

B. After each update of the probability values for all correlation combinations occurring at a point in time, any possible correlation combination whose value falls below a certain threshold is deleted and / or the point of contact of that correlation combination, the probability value of which increases by a certain factor over the probability value of all competing correlation combinations, as “ pen touch ”and the competing correlation combinations were deleted.

[0020] c. It is then checked for each contact point whether there are still possible correlation combinations. If there is none, the contact point is treated as a "no-pen touch". If there is exactly one, the point of contact is treated as a "pen touch".

Correlation combination is to be understood as a possible combination of input device and contact point. The number of correlation combinations results from the number of touch points multiplied by the number of touch signals whose occurrence correlates in time.

If the occurrence of three contact points on the input surface and two touch signals from input devices is correlated in time, the number of possible correlation combinations is six.

Competing correlation combinations are correlation combinations that have an input device or a touch point in common, so that these competing correlation combinations are mutually exclusive. If, for example, one of two possible input devices is assigned to one of three possible touch points as "pen-touch", then only two possible touch points are available for the second input device. A correlation combination was therefore confirmed and three competing correlation combinations were deleted, so that only two possible correlation combinations remain.

An advantage of the present invention is that the input devices, even if they do not have the functionality of a computer mouse, can be distinguished from one another and from other input objects. The input devices can therefore have a simpler structure than a computer mouse, in particular a closed and thus dirt-resistant housing.

The subject invention thus advantageously allows the differentiation of input devices, such as known from US2016139690A1 and US2014146021A1, among themselves and from other input objects when inputting to one or more input surfaces and therefore the different handling of the inputs of different input devices and input objects.

[0026] The invention is illustrated with reference to drawings of exemplary embodiment variants according to the invention:

Fig. 1: shows schematically the components that are required to implement the inventive method.

2: schematically illustrates the method according to the invention, in which the signals are shown, on the basis of which the data processing system carries out the assignment of input devices to points of contact.

3: shows schematically the assignment of input devices to points of contact made by the data processing system, based on the signals obtained from FIG. 2.

3.17

AT519 427 B1 2019-02-15 Austrian

Patent Office [0030] FIG. 4: shows schematically three points of contact that occur simultaneously on the input surface and the signals of the contact sensors from two input devices at the time of the occurrence.

Fig. 5: shows schematically the times of the disappearance of the three simultaneous contact points on the input surface and the signals of the contact sensors of two input devices after the disappearance of all touch points.

6: shows schematically the assignment of input devices to points of contact made by the data processing system, on the basis of the signals received from FIGS. 3 and 4.

Fig. 7: shows schematically a variant using an optical sensor in the input device.

In Fig. 1, the system which is required to implement the method according to the invention is shown schematically. The system comprises a touch-sensitive, position-resolving input surface 1, which can be in the form of a touchpad or a touchscreen. The input surface 1 is capable of recognizing several touch events at the same time. Such an input area is referred to as a multi-touchable input area. The touch event can be triggered when an object touches the input surface 1, or when the object approaches, if the object falls below a certain distance from the input surface 1.

The system further comprises an input device 2, which has at least one contact sensor 3 and at least one inertial sensor 4. The contact sensor 3 supplies a signal which indicates that the input device 2 is resting on a surface at the end at which the contact sensor 3 is arranged. As an alternative or in addition, the contact sensor 3 can already provide a signal before touching it, which indicates that the input device 2 is in the immediate vicinity of a surface, that is to say it is less than a certain distance from the surface. The contact sensor 3 can work, for example, according to the mechanical, capacitive or optical principle.

The inertial sensor 4 detects accelerations or rotation rates in or about at least one spatial axis. The inertial sensor 4 detects accelerations in at least one preferably two or particularly preferably all three spatial directions. The inertial sensor 4 preferably also detects rotation rates by at least one, preferably at least two or particularly preferably all three spatial directions. The inertial sensor 4 can particularly preferably be present as an inertial measuring unit for detecting the six possible kinematic degrees of freedom, comprising three acceleration sensors (translation sensors), each orthogonal to one another, for detecting the translatory movement in the x-, y- or z-axis and three orthogonal to one another attached rotation rate sensors (gyroscopic sensors) for the detection of rotating (circular) movements in the x-, y- or z-axis. The inertial sensor 4 is preferably located as close as possible to the end or tip with which the input device 2 contacts the input surface 1, so that the movement perceived by the inertial sensor 4 corresponds as exactly as possible to the movement of the tip of the input device 2 and thus that of the input surface 1 corresponds to the detected movement of the point of contact.

The system further comprises a data processing system 5, which receives and processes the signals from the input surface 1 and the input devices 2.

The system can preferably also comprise at least one additional input device 6 which, like the input device 2, has at least one contact sensor 7 and at least one inertial sensor 8.

In addition, the system can comprise any number of further input objects, which can also be equipped with sensors or can be present without sensors and which are of the touch-sensitive, position-resolving input surface 1 when touched

4.17

AT519 427 B1 2019-02-15 Austrian

Patent office or approximation are perceived. These further input objects include in particular the fingers of users of the system.

A trajectory 9 of a point of contact is shown on the input surface 1. The problem on which the present invention is based is that the data processing system 5 cannot determine which input object caused the movement path 9 without further information. This problem is irrelevant for the known control of applications, since this does not assign different meanings to the input with different input objects. For example, it makes no difference whether a file is opened on a touch screen with a stylus or with a finger.

The inventive solution to the problem provides that an input device 2 is used for input, which has a contact sensor 3 as a proximity or touch sensor and a sensor for the detection of relative movements in space, for example at least one inertial sensor 4, the data processing system 5 for assignment first checks whether the movement path 9 has appeared on the input surface 1 at the same time as the signal from the contact sensor 3. Since even in this case an incorrect assignment is possible, namely if the input device 2 is placed on any surface at the same time as another input object on the input surface 1, the second step is to check whether the movement path of the input device 2 is identical to the movement path 9 can be what the signal of the inertial sensor 4 and the signal of the input surface 1 are compared. If only one input device 2 is used, the second step could be omitted, since it seems unlikely that the input device 2 will be placed on another surface at the same time as any input object on the input surface 1. The second step is in any case for a clear assignment of one Input device 2 at a point of contact is particularly valuable if a large number of input objects are used simultaneously for data input on input surface 1, since in this case it can often happen that input objects are placed on input surface 1 at the same time or in a timely manner.

2 illustrates a situation in which two input devices 2, 6 are used, which send the data of their contact sensors 3, 7 and the data of their inertial sensors 4, 8 to the data processing system 5. In addition, an input surface 1 is used, which sends the position data of contact points to the data processing system 5.

In the example in FIG. 2, the input surface 1 detects three points of contact that occur simultaneously at the time t ₀ . Due to the temporal correlation of the occurrence of three touch points on the input surface 1 and two touch signals from input devices 2, 6, the number of possible correlation combinations is six. The objects are shifted differently (in terms of acceleration, speed and / or direction) up to the point in time of the snapshot in FIG. 2 along the surface of the input surface 1. The displacement of the touching objects on the input surface 1 results in a displacement of the touch points, which results in a movement path 9, 10, 11 in the plane of the input surface 1 for each touch point. Since the input surface 1 only transmits the trajectories 9, 10, 11 of the points of contact, the data processing system 5 cannot determine which object has caused which point of contact.

By using the input devices 2, 6, the data processing system 5 also receives the movement data of the inertial sensors 4, 8 of the respective input device 2, 6. The input device 2, for example, travels a movement path 12 in space, the inertial sensor 4 at least along the acceleration a direction of the input device 2 detected. The second input device 6 covers, for example, a path of motion 13 in space, the inertial sensor 8 detecting at least the acceleration along a direction of the second input device 6. The inertial sensors 4, 8 detect, for example, at least accelerations in a spatial direction, which is shown by the arrow, starting from the respective inertial sensor 4, 8. If the inertial sensor 4, 8 measures accelerations in only one or two spatial directions, these spatial directions preferably lie in one plane

5.17

AT519 427 B1 2019-02-15 Austrian

Patent Office

Angle 90 ° to the longitudinal direction of the input device 2, 6, which extends from the tip to the end thereof. Since the longitudinal direction of the input device 2, 6 is at least not aligned at an angle when touching a surface, there is a reference of the sensor signal of the inertial sensor 4, 8 along the spatial direction mentioned and the movement path of contact points detected by the input surface 1.

In addition, the data processing system 5 receives the signals from the contact sensors 3, 7 of the two input devices 2, 6. These, for example, also deliver a signal at time t ₀ , after which they touch a surface, that is to say at the same time as the contact points on the input surface 1.

The data processing system 5 is not able to clearly assign an input device 2, 6 to a contact point at time t ₀ , since each input device 2, 6, whose contact sensor 3, 7 delivers a touch signal at time t ₀ , at each time t ₀ occurring touch point could be placed.

In the method according to the invention, when at least one touch point occurs on the position-resolving touch-sensitive input surface 1 and / or each time a touch signal from a contact sensor 3, 7 of one or more input devices 2, 6 occurs, it is determined in the first step whether a temporal correlation of the occurrence of a Touch signal of a contact sensor 3, 7 of one or more input devices 2, 6 with the occurrence of one or more touch points on the position-resolving touch-sensitive input surface 1 is present.

Since this requirement is met, it is determined in a second step for all signals which have occurred at the same time in very short time intervals, with what probability the temporal course of a contact point, i.e. the respective movement path 9, 10, 11, on the position-resolving touch-sensitive input area 1 and the course of a movement detected by an inertial sensor 4, 8, that is to say a spatial movement path 12, 13 of an input device 2, 6 or at least one component thereof, describes the same or a correlating movement, or in other words, for every possible correlation combination in very short time intervals a probability value is determined and compared with the probability values of all other correlation combinations.

In the third step, an input device 2, 6 is assigned to a point of contact if the probability that the time course of a point of contact, ie the respective movement path 9, 10, 11, and the course of the movement detected by the inertial sensor 4, 8, ie a movement path 12, 13 of an input device 2, 6 describe the same or a correlating movement, exceeds a threshold value, or the probability value of a correlation combination is a certain threshold value greater than that of all competing correlation combinations.

In the example of FIG. 2, the movement path 9 on the input surface 1 correlates with the spatial movement path 12 of the input device 2 from the start time t ₀ , so that the data processing system 5 can clearly assign the input device 2 to the point of contact with the movement path 9. In the example in FIG. 2, the movement path 10 on the input surface 1 correlates with the spatial movement path 13 of the input device 6 in space, so that the data processing system 5 can clearly assign the input device 6 to the point of contact with the movement path 10. The point in time at which this unambiguous assignment can take place is variable and depends on the course of the signals and the probability values of all possible correlation combinations calculated therefrom.

Since none of the spatial trajectories 12, 13 correlate with the trajectory 11 on the input surface 1, this trajectory 11 is assigned by the data processing system 5 the meaning of any input object 14 which can be clearly distinguished from the touch events by input devices 2, 6.

If there are several points of contact by any further input objects 14 which themselves have no sensors, the data processing system can be used

6.17

AT519 427 B1 2019-02-15 Austrian

Patent office 5 would not distinguish which of these input objects 14 is located at which position. However, the input devices 2, 6 and each additional input device 2, 6 can still be uniquely assigned to a touch event.

The clear assignment of input devices 2, 6, ie a meaning “pen touch” to contact paths 9, 10 and the assignment of the meaning “not caused by input devices” or “no-pen touch” to all further contact paths 11, can 3 determine the actual arrangement of objects on the input surface 1 and thus assign a different meaning to the inputs of each input device 2, 6.

For example, the first input device 2 can be assigned to a first user by a control specification stored in the data processing system 5, to which user other functions of the data processing system 5 are available than to the user to whom the data processing system 5 assigns the second input device 6. In addition, different control rules can of course be stored in the data processing system 5 for input devices 2, 6 and other input objects 14, so that, for example, inputs can only be made with input devices 2, 6 and inputs with other input objects 14 are ignored by the data processing systems 5. In this way, incorrect entries can advantageously be recognized and avoided by unintentionally touching the input surface 1. An example of this is the unintentional touch by the ball of the hand (palm touch), which can be clearly distinguished from inputs with input devices 2, 6 by the present invention.

4 to 6 show an additional identification option, which is advantageously carried out in parallel or in addition to the identification described above. For each touch event, in addition to the start time t _{0 of} the touch with the end time of the touch, there is another characteristic time which can be used to identify the input devices 2, 6. This is particularly helpful when short-term entries are made without moving the input objects, as is the case, for example, when clicking on icons or buttons. In this case, the input surface 1 registers no movement of the touch point. 4, if several contact points 15, 16, 17 occur on the input surface 1 at the same time at the time t ₀ and one or more contact sensors 3, 7 give a signal at the same time at the time t ₀ , after which they are on a surface, then At time t ₀ the data processing system 5 cannot see whether one and which input device 2, 6 has caused a contact point 15, 16, 17. If neither the input surface 1 nor the inertial sensor 4, 8 of one or both input devices 2, 6 subsequently detect a movement, then no further assignment can take place. In this case, the assignment of input devices 2, 6 to touch points 15, 16, 17 can be determined via the respective point in time at which the touch points 15, 16, 17 and the touch signals of the contact sensors 3, 7 disappear again. 5 shows the signal curve of the contact sensors 3, 7 of two input devices 2, 6 below the input surface 1. 5, the disappearance of the touch signal of the contact sensor 3 of the first input device 2 and the disappearance of the touch point 17 take place at the same time L. The disappearance of the touch signal of the contact sensor 7 of the second input device 6 and the disappearance of the touch point 16 take place at the same time t ₃ . The disappearance of the touch point 15 does not coincide with the disappearance of a touch signal from a contact sensor 3, 7. As shown in FIG. 6, the data processing system 5 can assign the input device 2 to the contact point 17 from the time L and an input object 14, which is not defined in more detail, to the touch point 15 from the time t _2. The input device 6 can with high probability the contact point from the time t2 16 are assigned, wherein the simultaneous disappearance of the touch signal of the contact sensor 7 of the second input device 6 and the disappearance of the touch point 16 can be used as confirmation of the assignment that may have already taken place.

It should be noted that also in the example of FIGS. 4-6 the movement path of the contact point 17 detected by the input surface 1 with the movement path of the input gene

7.17

AT519 427 B1 2019-02-15 Austrian

Patent Office 2 correlates and correlates the movement path of the contact point 16 detected by the input surface 1 with the movement path of the second input device 6. However, the spatial trajectories of the input devices 2, 6 correlate with each of the trajectories of the contact points 15, 16, 17 up to a certain time t ^ The correlation of the trajectory of the input device 2 with the trajectories of the contact points 15 and 16 ends in any case Time G, the correlation of the movement path of the second input device 6 with the movement path of the contact point 17 also ending at the time. Specifically, the inertial sensor 4 of the input device 2 registers a movement at the time, while the touch points 15 and 16 do not move any further, so that the input device 2 can also be clearly assigned to the touch point 17 on the basis of the movement registered by the inertial sensor 4.

In general, it can therefore be stated that the trajectories of all contact points 15, 16, 17 occurring at the same time on the input surface 1 are compared with the trajectories of all input devices 2, 6 applied to a surface at the same time until, for each input device 2, 6 either a touch point 15, 16, 17 can be assigned with sufficient probability, or the correlation with each touch point 15, 16, 17 can be excluded with sufficient probability. If no assignment has taken place until a contact point 15, 16, 17 disappears, the further course of the signal of the inertial sensor 4, 8 and / or the elimination of the contact signal of the contact sensor 3, 7 can be used to subsequently determine the correlation or that Confirm that there is no correlation. Which touch events are rated as simultaneous depends largely on the measurement frequency and transmission rate of the input surface 1 and the input devices 2, 6, so that all touch points 15, 16, 17 of the input surface 1 and touch signals of the contact sensors 3, 7 occurring within a certain time span as simultaneous be treated. All touch points 15, 16, 17 and touch signals that potentially represent the same touch event that occur within a time period are referred to as temporally correlating.

A preferred embodiment variant provides that a plurality of input surfaces 1 are present, wherein a plurality of input surfaces 1 can be connected to a data processing system 5, or each input surface 1 has its own data processing system 5, these data processing systems 5 advantageously being connected to one another via intranet or the Internet, Radio, LAN or WLAN are connected.

The inventive method can be determined by one or more data processing systems 5, which input surface 1 is touched by which input device 2, 6. If there are several data processing systems 5, it is advantageous if the input device 2, 6 transmits the signals of its inertial sensor 4, 8 and its contact sensor 3, 7 wirelessly to the data processing systems 5, for example by radio or WLAN. For example, in addition to the signals from their inertial sensor 4, 8 and their contact sensor 3, 7, input devices 2, 6 can transmit a unique identifier such as a serial number or a digital signature to the data processing systems 5, so that the data processing systems 5 according to a stored control rule each input device 2, 6 can assign an individual meaning. For example, the meaning of an administrator input device can be assigned to an input device 2, with which all existing data processing systems 5 can be registered and simply accessed by simply touching or moving the input device 2 on the input surface 1. On the other hand, a conventional user can be assigned to an input device 2, so that this user can, for example, provide full access to his data processing system 5 and only has the functions and / or drives released for him available in other data processing systems 5.

It can advantageously be recorded and stored with which input device 2, 6 which inputs were made to which data processing system 5.

Applications or programs can be run on each data processing system 5

8.17

AT519 427 B1 2019-02-15 Austrian

Patent office leads that process both inputs with an input device 2, 6 and inputs by any non-identifiable input objects and can treat them differently. For example, input devices 2, 6 can be used to draw or write freely, while other input objects can select icons or objects and / or can rotate or zoom views or objects.

Each input device 2, 6 can have further sensors, buttons or switches, so that the data of the sensors and states of the buttons or switches can be assigned to a contact point 15, 16, 17 on the input surface 1. For example, by pressing a button on the input device 2, 6, a virtual color can be selected with which the input device 2, 6 is used to draw on the input surface 1.

For example, an input device 2, 6 can have a biometric sensor such as a fingerprint scanner, so that it can identify its user itself, or the data processing system 5 can determine the user from the data of the biometric sensor.

When using a touchscreen as input surface 1, after the input devices 2, 6 have been identified or assigned to touch points 15, 16, 17, an individual input marking (cursor or mouse pointer) for each input device 2, 6 at the respective touch point 15, 16 is preferred , 17 appear in order to enable a visual control of the assignment by the user. With short entries, the individual entry marking can be displayed beyond the duration of the entry. When controlling any screen with a touchpad as input surface 1, the input markings shown on the screen, which represent touch positions of input devices 2, 6 on the touchpad, can be displayed individually for the input devices 2, 6 or for the users assigned to the input devices 2, 6.

In the figures, the input device 2, 6 is illustrated purely by way of example, whereby this can have any shape, it being advantageous to design that end or ends with which the input surface 1 is to be touched with contact sensors 3, 7 as a tip , In the event that capacitive input surfaces 1 are used, it is advantageous to make ends of the input device 2, 6, on which a contact sensor 3, 7 is located, somewhat larger, since very small peaks with a capacitive input surface 1 are difficult to detect. In the event that an inductive input surface 1 should also be usable, a coil or conductor loop can be integrated in the tip of the input device 2.

If the input device 2, 6 has a wide, as shown, for example, round head with an inserted coil, this can be used both for input on capacitive input surfaces 1 and for input on inductive input surfaces 1. Any input objects are suitable for input on optical input surfaces 1 (for example, using total internal reflection), pressure-sensitive input surfaces or resistive input surfaces 1, so that the method according to the invention can be used with an input device 2, 6 with a larger head and coil on all common input surfaces 1.

Both the inertial sensor 4 and the contact sensor 3 are preferably mounted in a closed housing of the input device 2. The contact sensor 3 can be an inductive or capacitive distance sensor. As an alternative to the detection of touches, the contact sensor 3 can also perceive forces, for example by including strain gauges or piezo elements. The contact sensor 3 can be a button. The contact sensor 3 can be or comprise an optical sensor, for example an optical distance sensor or an intensity sensor, which is preferably mounted behind a transparent or at least partially transparent section of the housing of the input device 2.

If the input device 2 has at least one optical sensor, this can perceive, for example, the light intensity or the color of the surface to which the input device 2 is directed or against which it is applied, these data being used

9.17

AT519 427 B1 2019-02-15 Austrian

Patent office on a touch screen with the light intensity or the color of the pixels at the detected touch points 15, 16, 17 on the touch screen can be compared. For example, a distinction can be made as to whether the input device 2 is used on a touchscreen or another surface. In addition, the assignment of an input device 2 to a point of contact 15, 16, 17 can take place via the correlation of the course of the light intensity and / or color, by determining in repeated short time intervals whether the course of the intensity or color perceived by the optical sensor with the The course of the pixels shown on the touchscreen along the path of movement of a point of contact 15, 16, 17 on the touchscreen corresponds. Analogously, a probability value for each possible correlation combination can be set up for all signals of optical sensors from input devices 2 occurring simultaneously and for all contact points 15, 16, 17 occurring simultaneously, which indicates with what probability the course of the light intensity or the course of the color value of an optical Sensor with the light intensity or the course of the color value at the point of contact 15, 16, 17 or along its path of movement on the touch screen. The assignment of the meaning “pen touch” and “no-pen touch” is carried out according to the criteria already described, depending on the increase and decrease in the probability values of the correlation combinations or depending on the number of possible correlation combinations remaining after recalculation of the probability values.

This variant with an optical sensor can advantageously be implemented in addition to the method protected in the claims. For example, however, the contact sensor 3 can also be designed as such an optical sensor, so that in addition to approximation it can also acquire further information about the detected surface or can even receive signals from the detected surface.

Since, as illustrated in FIG. 7, the embodiment variant with an optical sensor 20 on the input device 21, 22 and at least one touch screen 18 as input surface 1 is also suitable for solving the task according to the invention on its own, the applicant reserves the right intend to submit this as a divisional application at a later date. This embodiment variant is advantageous since each input device 21, 22 only has to have an optical sensor 20, that is to say does not need to have an inertial sensor 4 or a contact sensor 3. Instead of perceiving intensities or colors selectively with the optical sensor 20, the latter can also record a small image section, that is to say several pixels in the form of an image, the course of this image with the corresponding image sections at the positions of the contact points 15, 16, 17 on the touchscreen 18 is compared. From the scaling of the image, the distance of the optical sensor 20 from the touch screen 18 can also be determined, if necessary. As shown in FIG. 7, an individual light signal 19 (for example intensity, color, two-dimensional pattern or the like) can advantageously be detected by the data processing system 5 at each touch point 15, 16, 17 when one or more touch points 15, 16, 17 are detected on the touch screen 18 temporal course) are shown. The light signals 19 are detected by the optical sensors 20 of the input devices 21, 22 and sent back to the data processing system 5. Since the data processing system 5 knows at which point of contact 15, 16, 17 from the touch screen 18, which light signal 19 was displayed, it can quickly and easily assign the input devices 21, 22 to the points of contact 15, 16, 17 on the touch screen 18. For example, the data processing system 5 can uniquely assign the input device 21 to the contact point 15, since the time profile of the signal sent from the input device 21 to the data processing system 5 only matches the time profile of the light signal 19 at the contact point 15. Individual light-dark or white-black changes of the pixels at the point of contact 15, 16, 17 or within a certain radius around the point of contact 15 are particularly suitable as light signals 19 at the points of contact 15, 16, 17 on the touchscreen 18 , 16, 17 or approach point, since this can be detected particularly reliably as a binary signal by a simple photodiode or another intensity sensor. The individual light signals 19 can be transmitted so subtly or briefly that they cannot be perceived by the human eye. Preferably, each touch point 15, 16, 17 can be assigned an input mark (= cursor) with an individual color, so that the input markings

10/17

AT519 427 B1 2019-02-15 Austrian

Patent office can be distinguished both from suitable optical sensors 20 of the input devices 21, 22 and from the users.

A disadvantage of a variant with only an optical sensor 20 in the input device 21, 22 is that it can only be identified on touch screens 18 and not on touch pads. The detection of the movement or a movement component of the input device 2 in space by at least one inertial sensor 4 is therefore also useful in this variant.

In particular, the signals of the inertial sensor 4 can be used to track the movement of the input device 2, 6 away from the input surface 1, so that after the initial identification of the input device 2, 6 at a point of contact 15, 16, 17 of an input surface 1, the absolute position of the input device 1 2 can be determined or estimated in space, so that it can be determined or estimated from the extent and or the direction of the movement of the input device 2 in the space whether the input device 2 for a later contact point 15, 16, 17 on the same or one other input surface 1 may be responsible. For example, if the input device 2, 6 has been assigned to a first point of contact on the input surface 1 and subsequently only experiences acceleration in a direction of approximately 90 ° to the surface of the input surface 1, then it is likely that a new point of contact of this input device 2, 6 only can occur near the first point of contact. In order to detect the movement of the input device 2, 6 in the room as precisely as possible, it is advantageous to design the inertial sensor 4, 8 as an inertial measuring unit and, if necessary, also to equip it with additional sensors in order to also, in addition to relative movements, the orientation of the input device 2, 6 in the room to capture. With the contact points 15, 16, 17, the input surface 1 advantageously provides the absolute position of the input device 2, 6 at certain times, as a result of which the relative position data of the inertial measuring unit can be continuously converted into absolute positions in space in relation to the input surface 1.

Claims (13)

claims 1. Method for controlling at least one data processing system (5) with at least one input surface (1), which is position-resolving and touch-sensitive and any number and type of input objects, at least one input object being an input device (2) which is a contact sensor (3) and has an inertial sensor (4) and continuously sends the data of these sensors to at least one data processing system (5), the positions of all points of contact, which an input surface (1) perceives, are also continuously sent by it to at least one data processing system (5), characterized in that each time at least one touch point occurs on an input surface (1) and / or each time a touch signal from a contact sensor (3) of one or more input devices (2) occurs, it is determined in the first step whether a temporal correlation of the occurrence of a Touch signal of the contact sensor (3) one or me her input devices (2) with the occurrence of one or more touch points on one or more input surfaces (1) and in the second step with several temporal correlations between the appearance of touch points on an input surface (1) and the appearance of touch signals from contact sensors (3) of input devices (2), a possible value is assigned to each possible correlation combination, which indicates how likely it is that an input device (2) with the contact sensor (3) belonging to the correlation combination has touched the input surface (1) at the location at which the point of contact associated with the correlation combination has appeared, this probability value being subsequently updated in very short time intervals for each correlation combination, for this purpose, on the one hand, the data of the contact sensor (3) belonging to the correlation combination with the activity of the correlation combination associated contact point and, on the other hand, the movement data of the inertial sensor (4) belonging to the correlation combination are compared with the movement trajectory of the contact point belonging to the correlation combination, the probability value of a correlation combination increasing if there is a high match of the compared data and otherwise decreasing, with the third step involving Input device (2) is assigned to a point of contact if the probability value of a correlation combination exceeds a threshold value and / or only one possible correlation combination remains through the exclusion of correlation combination and / or no input device (2) is assigned to a touch point if no possible correlation combination remains. 2. The method according to claim 1, characterized in that it is determined whether there is a temporal correlation of the disappearance of a touch signal of the contact sensor (3) of one or more input devices (2) with the disappearance of one or more touch points on an input surface (1), wherein an input device (2) is assigned to a point of contact if exactly one touch signal and exactly one point of contact occur and disappear at the same time. 3. The method according to claim 1, characterized in that a point of contact on the input surface (1) is treated as a "no-pen touch", that is to say as not caused by an input device (2), if there is no temporal correlation of a contact signal from a contact sensor ( 3) with the appearance of a point of contact on the input surface (1). 4. The method according to claim 1, characterized in that the appearance of exactly one point of contact on an input surface (1) is treated as a "pen touch", that is, as caused by an input device (2), if this occurs only with the appearance of exactly one touch signal of a contact sensor (3) of an input device (2) correlated in time. 12/17 AT519 427 B1 2019-02-15 Austrian Patent Office 5. The method according to claim 1, characterized in that after each update of the probability values for all correlation combinations occurring at a point in time, every possible correlation combination whose value drops below a certain threshold is deleted. 6. The method according to claim 1, characterized in that after each update of the probability values for all correlation combinations occurring at a point in time, that correlation combination whose probability value increases by a certain factor over the probability values of all competing correlation combinations is treated as a "pen touch" and the competing ones Correlation combinations are deleted. 7. The method according to any one of claims 5 or 6, characterized in that it is then checked for each touch point whether there are still possible correlation combinations, a touch point being treated as “no-pen touch” if none exist and as “pen touch”. is treated if there is still exactly one possible correlation combination overall. 8. The method according to any one of claims 1 to 7, characterized in that at least two input devices (2, 6) with an identical structure for the simultaneous input of data on at least one input surface (1) are used. 9. The method according to any one of claims 1 to 8, characterized in that a plurality of input surfaces (1) are present, the method according to the invention determining which input surface (1) is touched at which position by which input device (2). 10. The method according to any one of claims 1 to 9, characterized in that the contact sensor (3) of the input device (2) is an optical sensor (20) which detects the light intensity or color of the surface on which the input device (2) with this is present or the input device (2) has such an optical sensor (20) in addition to the contact sensor (3). 11. The method according to claim 10, characterized in that the input surface (1) is a touchscreen (18) and the data processing system (5) at each contact point detected by the touchscreen (18) represents an individual light signal (19) which is emitted by the optical sensor ( 20) can be detected and the input device (2) sends the signal detected by the optical sensor (20) to the data processing system (5). 12. The method according to claim 11, characterized in that the data processing system (5) assigns an input device (2) to a point of contact on the touch screen (18) when that detected by the optical sensor (20) and by the input device (2) to the data processing system (5 ) transmitted signal coincides with the light signal (19) exactly one point of contact. 13. The method according to any one of claims 1 to 12, characterized in that after the initial assignment of an input device (2) to a point of contact on the input surface (1), the location of the next point of contact occurring in time by the same input device (2) on the input surface ( 1) whose inertial sensor (4) is at least approximately determined on the basis of the sensor signals, for which purpose the inertial sensor (4) is preferably an inertial measuring unit.