RU2674879C1 - Method for managing of graphic indicator movements on touch screen devices using the properties of motion vector-functions (time-coordinate dependent) of motion of a finger point with a screen for switching between control modes - Google Patents

Abstract

FIELD: data input devices.SUBSTANCE: invention relates to information input devices. Such result is achieved by the fact that the method of controlling the movement of a graphic pointer includes two modes: control mode 1 and motion mode 2. In the first mode, moving the touch point of a finger with a screen is performed without changing the position of the pointer. This mode is used to set the desired pointer offset relatively to the touch point. In the second mode, moving the touch point of the finger with the screen causes an equal movement of the pointer. Command for switching between modes is transmitted through the properties of the motion vector function (time-coordinate dependence) of the point of contact of the finger with the screen.EFFECT: technical result is to provide an opportunity to hold the pointer along a straight path with the ends at any pre-set points, even if these points are at the edges of the screen, using only one finger and not lifting it from the screen.1 cl, 1 dwg

Description

FIELD OF TECHNOLOGY

The invention relates to computing, in particular to methods for inputting information by a person into computing devices. It allows you to control a moving pointer (cursor) on devices into which information is entered by touching the fingers of the touch screen, and can be used to create programs for such devices.

BACKGROUND OF THE INVENTION, BACKGROUND

One of the main elements of the interface used to control computing devices is a movable graphic pointer, often called a mouse cursor. The most common means for controlling this pointer when using desktop computers are physical input devices: a computer mouse and trackpad. These devices translate the trajectory of the hand on the plane into the trajectory of the pointer. The need to use rolling pointers remains on devices with a touch screen, but with a mouse and trackpad. The use of a mouse and trackpad can be modeled using various methods of influencing the screen of a touch device, while hand actions on the screen, similar to actions with a physical input device, lead to the required movement of the pointer.

Modeling the use of physical input devices occurs through the creation of virtual devices on the touch screen. At the same time, an area is created on the screen, all the touches in which relate only to the created virtual device and are intended only for controlling the pointer. We will call this region the control region. We can say that the control area covers part of the screen for performing normal actions: for example, simply moving your finger in this area will no longer cause the contents of the window to scroll, and touching it will not trigger the activation of the program control. Thus, the virtual device on the screen consists of two parts: the control area and pointers. The present invention proposes one of the possible ways to control pointers through exposure to the control area.

It should immediately indicate which categories of known solutions will be excluded from comparison with the proposed invention. To do this, we first clarify some concepts. Firstly, no matter what name its authors give their invention, we will agree to call any method of controlling a moving pointer on a device with a touch screen a virtual mouse, in which the control area moves around the screen, accompanying the hand touching this area. Secondly, we call the conversion of the trajectory of the finger to the trajectory of the pointer non-scalable if moving the finger 1 cm leads to the movement of the pointer 1 cm. The following categories of solutions are excluded from the comparison (with reasons).

1) Virtual trackpads (that is, pointer control methods in which the control area does not accompany the moving arm). Causes:

A) The control area covers a larger part of the screen than in the case of a virtual mouse.

B) To simulate the movement with the pressed button, you need to use more than one finger.

C) In the case of an unscaled transformation of the trajectory of the finger into the trajectory of the pointer, significant movement of the pointer should be performed in several steps with the finger detached.

2) Virtual mice with scaled path conversion.

Cause:

As practice shows, and as indicated in the descriptions of many inventions, the greatest control over the movement of the pointer is achieved when the pointer accompanies a moving hand.

3) Virtual mice used on devices where exposure to the screen is performed using a special pointed instrument (pen, stylus).

Cause:

On such devices, the problem solved by the present invention is absent.

So, the subject of the invention is a method for moving a pointer that implements a virtual mouse with unscaled conversion of the path of the finger to the path of the pointer on devices where exposure to the screen is performed by fingers.

A constant difficulty in implementing this category of virtual devices is the following drawback. Since a finger moving across the screen closes a spot with a diameter of about 1.5 cm from the user's eyes, the pointer should be away from the finger. During the movement, the relative position of the finger (or, equivalently, the control area) and the pointer remains unchanged. If at the beginning of the movement the pointer was, for example, on the left side of the finger, then at the moment the finger reaches the right edge of the screen, the pointer will be at a significant distance from this edge and will not be able to continue moving in its direction. So in the vicinity of the edges of the screen, dead zones are formed, inaccessible to the pointer. To correct this drawback, you must either sacrifice zones along the edges of the screen, excluding them from the workspace, or propose a suitable method for changing the relative position of the pointer and the control area, which would allow the pointer to fall into these zones.

Below are links to known solutions to this problem with a brief indication of the solution. All references are references to US Patent Documents.

US20150212698 A1 A virtual mouse where the relative position of the cursor and control area remains unchanged. (Paragraph 21 of the patent claims, paragraph 31 of the description). The problem is solved, judging by the images, by adding fields to the workspace. US20100214218 A1 A virtual mouse, where the relative position of the cursor and the control area remains unchanged (Paragraph 12 of the patent formula, paragraph 37 of the description). US5835079 A A virtual mouse, where the relative position of the cursor and the control area is changed by moving the mouse. The mouse is considered raised if the hand is completely torn off the screen, while using the imprint of the whole hand. (Alternatively, the user could merely lift his hand and place it at a desired location, whereby the OS would re-create the virtual pointing device under the user's fingers at the new location.) US20110018806 A1 A virtual mouse, where the relative position of the cursor and the control area is carried out by turning the control area, which is performed with two fingers. (Paragraph 6 of the patent claims, paragraph 58 of the description) US20060132460 A1 A virtual mouse, where the relative position of the cursor and the control area is set at the beginning of the movement, and after that remains unchanged (Paragraph 33 of the description) US20140313130 A1 A virtual mouse, where the cursor is at a predetermined distance from the control area in one of the predefined directions (left-up, right-up, and so on). Relative positioning occurs automatically and stepwise based on the direction of movement of the control region at the initial moment (Clause 17 of the patent formula, paragraphs 126-149 of the description) US6411283 B1 A virtual mouse where the cursor is at a predetermined distance from the control area in one of the predefined directions (left, up, right, and so on). Changing the relative position occurs automatically and spasmodically at the moment the control region enters the left or right border strip of the screen (Claim 3 of the patent formula) US20160378251 A1 A virtual mouse, where the relative position of the cursor and the control area is set at the beginning of the movement, and the gesture serves as a means to adjust the relative position (Claim 1 of the patent claims, paragraph 46 of the description) US9582107 B2 A virtual mouse, which uses two different modes: regulation mode 1 (disposition mode) and motion mode 2 (movement mode). In the control mode, the control region can move with a fixed pointer, which allows you to set their arbitrary relative position. In motion mode, the pointer accompanies the movement of the control region with an offset set during the control mode. It is indicated that the method of switching from driving mode to control mode is lowering the second finger onto the screen. (Claim 6 of the patent claims, fifth embodiment from the description).

Consider the following task that must be performed using a virtual mouse.

Task A:

"Using one finger and not lifting it from the screen, draw a pointer along a path that is a line segment with ends at arbitrary points on the screen."

Such a problem arises, for example, in text editors, when the user tries to select a line in the text with the mouse. The line segment is usually (although not essential) horizontal, its beginning is in front of the first character of the selected text, and the end is behind the last character. Detaching a finger from the screen means releasing the mouse button and ending the selection. This problem cannot be solved by known methods.

Indeed, since the ends are arbitrary, we should have access to points in the border areas of the screen, possibly from different sides of the screen. Therefore, the relative position of the pointer and the control area must change during the movement of the finger. Therefore, the methods described in documents US20150212698, US20100214218, US20060132460, US20160378251 do not allow to solve the problem, since the relative position of the pointer and the control region during finger movement does not change.

The methods described in documents US5835079, US20110018806, US9582107, do not allow to solve the problem, because in them to change the relative position of the pointer and the control area requires more than one finger. The need to use several fingers in these methods is irremovable and stems from the features of patented inventions.

The listed automatic methods (described in documents US20140313130, US6411283) are not suitable, since they cause jump-like changes in the position of the pointer, which do not allow the pointer to move along a straight path with arbitrary ends.

Closest to solving this problem is the method specified in the last of these patents (US9582107). This solution can be selected as a prototype of the invention. The only condition of task A, which remains unfulfilled when using the method of US9582107, is to use one finger. The present invention allows to be limited to one finger.

Using two fingers is undesirable for the following reasons:

1) The movement to move the pointer is less convenient when you need to use two fingers, and performing different actions - one finger slides all the time on the screen, the other at this time falls, slides and rises.

2) From the point of view of the interface implementation, the participation of the second finger for operating the mouse causes additional difficulties. We have an alternative - either to expand the control area so that the second finger fits easily on it, moreover, at a distance from the first finger (because the sensor devices do not recognize simultaneous close touches as separate events), or take into account the second finger touching the screen outside the control area. In the latter case, independent processing of touches outside the control area during mouse operation is excluded. For example, we will no longer be able to move the pointer with the mouse and simultaneously scroll the contents of the screen with the other finger.

SUMMARY OF THE INVENTION

The technical problem solved by the present invention is the control of a movable pointer on devices into which information is entered by touching the fingers of the touch screen. It is required to ensure that the pointer moves along arbitrary paths (in particular, straight lines), including such paths whose ends are at the edges of the screen. In this case, the movement of the touch point should cause equal movement of the pointer. The technical result is the ability to perform this task with one finger without tearing off the screen.

In the proposed method, in the same way as in the prototype, in the process of controlling the pointer, control modes 1 and motion 2 are distinguished. To switch between modes, you need to send the appropriate command to the computing device. Commands can be issued in various ways. The prototype proposes to issue a command to touch the screen with another finger. In this invention, the command to switch between modes is proposed to be transmitted through the properties of the motion vector function (coordinate versus time) of the touch point of the finger with the screen - this is the essence of the invention.

 The difference between the FREQUENCY of the finger on the screen and the VECTOR FUNCTION of the finger on the screen is that the trajectory connects two quantities (X, Y), and the vector function - three values (X, Y, t). Typically, a program running on a computing device consumes just the trajectory as input, that is, the relationship of the two values. There is a third quantity in a vector function, which means that the vector function contains redundant information. Therefore, the use of the information contained in the vector function theoretically allows SIMULTANEOUSLY with the introduction of the trajectory in the computing device to submit to this device any other information. For example, suppose we agreed to use Morse code. They advanced a finger along the trajectory, stopped, and advanced again - the time between stops sets points and dashes. At the same time, a trajectory and any text are entered. This is one of an infinite number of possible ways to transmit information using a vector function in addition to entering a path. One application of such information may be to instruct the device. The input of commands using the vector function can be done not only simultaneously with the input of the trajectory, but also alternately with it. In the latter case, in one solid line along which the contact point of the finger moves with the screen, the pieces used to enter the path can alternate with the pieces used to encode commands. Pieces of the second kind are usually called gestures. In any case, to supply the commands encoded using the vector function to the device, ONE finger is enough, and the finger may not tear off the surface of the screen.

Since in this method the relative position of the pointer and the control area is changed, dead zones do not appear on the screen, and the pointer can be moved to any place on the screen - thus achieving the technical task. And since the command to switch between modes is transmitted through the properties of the vector function of the touch point of the finger with the screen, it is possible to perform this task with one finger without interruption from the screen.

Thus, the implementation of this invention is sufficient to obtain a technical result.

BRIEF DESCRIPTION OF THE DRAWINGS

Attached to FIG. 1 drawing is used as an illustration to the description of the existing implementation of the invention, links to it are contained in the section entitled IMPLEMENTATION OF THE INVENTION.

DETAILED DESCRIPTION OF THE INVENTION

In a specific implementation, a special gesture is used to switch from the motion mode to the control mode, which is performed while the pointer is moving with the same finger, which moves the control area and does not affect the pointer trajectory. This method is implemented in the Tabula program, which works on devices with a touch screen running Android OS. Let us describe in detail how this method solves problem A. The description is illustrated in FIG. 1, where the thin solid line shows the trajectory of the point of touch of the finger with the screen (in short - the trajectory of the finger).

For definiteness, suppose that you need to move the pointer along a horizontal line segment with a beginning in the vicinity of the left edge of the screen and with an end in the vicinity of the right edge of the screen (dotted line AC in Fig. 1). We also assume that before the start of the movement, the pointer is located to the left of the control zone - this can be done by known methods.

Step 1. The finger drops to the control zone (point 1 in Fig. 1), which corresponds to the click of a mouse button, and starts moving around the screen from left to right. Initially, the mouse is in motion mode, so at the same time as the finger moves across the screen, the pointer starts moving in the same direction (dotted line section AB in Fig. 1).

Step 2. Since the required end point of the index path is in the vicinity of the right edge of the screen, we cannot achieve it if the location of the index to the left of the finger (control zone) is preserved. Therefore, we must switch to the regulation mode of the relative position of the pointer and the control zone. To do this, in an arbitrary place before the end of the trajectory of the finger, you must start a gesture, which, after its recognition, will serve as a signal to the program about the beginning of the regulation mode. The gesture is recognized in this particular implementation (in the Tabula program) when the following conditions are met:

a) there was an acceleration of movement in the original direction (over a fraction of a centimeter) - the section between points 2 and 3 on the path of the finger

b) immediately after acceleration, the finger trajectory had a kink approximately at a right angle - point 3 on the finger trajectory

c) after a kink in the movement of the finger, the direction opposite to the original direction was found - return (that is, in this case, the finger moved from right to left at some point) - point 4 on the path of the finger.

If the program after acceleration detects a kink (conditions a) and b)) are satisfied, then it stops the pointer movement until it is clear whether condition c) is fulfilled. If condition c) is fulfilled, then the transition to mode 1 is recognized, and the pointer is in the place that it reached during the passage of the finger through the break point (point B in Fig. 1) until it switches back to mode 2 If condition c) is not fulfilled, then after some time the pointer resumes movement from the point at which it paused and catches up with the position that it would have if it had just followed the finger all the time.

The specified combination of conditions, on the one hand, allows you to not distort the desired cursor path in cases where switching to mode 1 is necessary, on the other hand, allows you to move the cursor along complex paths, including kinks. When moving even along such complex trajectories, at least one of the conditions a) - c) is usually not satisfied, and switching does not occur.

After the program has switched to mode 1, we can move the finger (control zone) as necessary in order to establish the relative position of the pointer and the control zone, which will be convenient for further movement of the pointer. In this particular case, the control zone should move to the right side of the pointer to the left side.

Step 3. When the new location of the pointer is set, you must again switch to mode 2 to resume the movement of the pointer. To signal to the program about switching to this mode, exclusively the properties of the vector function of the movement of the finger’s touch point with the screen can also be used. In this particular implementation (in the Tabula program), switching to mode 2 occurs under the following conditions:

a) If you want to resume movement in the original direction, the switch occurs immediately after the fracture of the trajectory of the finger in this direction (at point 5 in Fig. 1).

b) Another way to switch is to suspend the movement of a finger for a split second.

Step 4. Finish moving the pointer along the remaining portion of the segment specified in the condition of problem A (the dotted line portion BC in Fig. 1). Raising a finger from the surface of the screen (at point 6 in Fig. 1), which means releasing the mouse button.

Externally, the implementation of these steps looks like just a rectilinear movement of the finger across the screen with a quick loop or curl in the middle of the movement. The pointer in this case moves strictly rectilinearly with a stop at the time of execution of the curl.

The described implementation of the invention is one of an infinite number of possible implementations. For example, to send a command to switch to mode 1, you can use the message "SOS" encoded in Morse code and transmitted by alternating intervals of movement and stopping the finger along the entered path. In the latter case, the command will also be transmitted through the properties of the vector function of the movement of the touch point of the finger with the screen.

Claims (1)

A method for controlling the movement of a graphic pointer on devices with a touch screen, in which two modes are distinguished: mode 1, when the movement of the touch point of the finger with the screen occurs when the pointer is stationary, and which serves to establish the offset of the pointer relative to the point of touch of the finger with the screen, mode 2, when the movement of the touch point of the finger with the screen is accompanied by the movement of the pointer with an offset set relative to the touch point, characterized in that the switching command between the modes is transmitted via ARISING motion vector function (coordinates versus time) touch point the finger to the screen.

Images