DE112007000278T5 - Gesturing with a multipoint scanning device - Google Patents

Abstract

Electronic system comprising:
a multi-point sampling device that provides a multipoint sampling area for receiving inputs from one or more objects;
a gesture module configured to determine a gesture set for a particular input device received by the multipoint scan region of the multipoint sensing device, to monitor the particular input device for one or more gesture events contained in the gesture set and to initiate input actions associated with a gesture input Gesture event are associated when the gesture event is performed with the input device.

Description

CROSS REFERENCE TO RELATED REGISTRATIONS

These Registration claims the priority of the provisional U.S. Patent Application No. 60 / 763,605, filed January 30 2006, entitled "GESTURING WITH A MULTI-POINT SENSING DEVICE", which is incorporated herein by reference.

• U. S. Patenanmeldung Nr. 10/903,964, eingereicht am 330. Juli 2004 mit dem Titel "GESTEN FÜR BERÜHRUNGSEMPFINDLICHE EINGABEVORRICHTUNGEN";
• U. S. Patentanmeldung Nr. 11/015,434, eingereicht am 17. Dezember 2004 mit dem Titel "VERFAHREN UND VORRICHTUNG ZUM INTEGRIEREN MANUELLER EINGABE", die die Priorität des U. S. Patents: 6,323,846 beansprucht, erteilt am 27. November 2001, welches die Priorität der vorläufigen U. S. Patentanmeldung Nr. 60/072,509 beansprucht, eingereicht am 26. Januar 1998;
• U. S. Patentanmeldung Nr. 10/654,108, eingereicht am 2. September 2003 mit dem Titel 'BEIDHÄNDIGE MAUS";
• U. S. Patentanmeldung Nr. 10/789,676, eingereicht am 27. Februar 2004 mit dem Titel "FORM DETEKTIERENDE EINGABEVORRICHTUNG";
• U. S. Patentanmeldung Nr. 10/840,862, eingereicht am 6. Mai 2004 mit dem Titel "MEHRPUNKT-BERÜHRUNGSBILDSCHIRM";
• U. S. Patentanmeldung Nr. 11/115,539, eingereicht am 26. April 2005 mit dem Titel "HANDHALTBARE ELEKTRONISCHE VORRICHTUNG MIT MEHRFACHBERÜHRUNGS-ABTASTVORRICHTUNGEN";
• U. S. Patentanmeldung Nr. 11/241,839, eingereicht am 30. Juli 2004 mit dem Titel "NÄHEDETEKTOR IN HANDHALTBARER VORRICHTUNG";
• Vorläufige U. S. Patentanmeldung Nr. 60/658,777, eingereicht am 4. März 2005 mit dem Titel "MULTIFUNKTIONALE HANDHALTBARE VORRICHTUNG";
• U. S. Patentanmeldung Nr. 10/903,964, eingereicht am 30. Juli 2004 mit dem Titel "GESTEN FÜR BERÜHRUNGSEMPFINDLICHE EINGABE-VORRICHTUNGEN";
• U. S. Patentanmeldung Nr. 11/038,590, eingereicht am 18. Januar 2005 mit dem Titel "MODUSBASIERTE GRAPHISCHE BENUTZERSCHNITTSTELLEN FÜR BERÜHRUNGSEMPFINDLICHE EINGABEVORRICHTUNGEN";
• U. S. Patentanmeldung Nr. 117048,264, eingereicht am 31. Januar 2005 mit dem Titel "GESTEN FÜR BERÜHRUNGSEMPFINDLICHE EINGABE-VORRICHTUNGEN";
• U. S. Patentanmeldung Nr. 11/228,737, eingereicht am 16. September 2005 mit dem Titel "AKTIVIEREN VIRTUELLER TASTEN EINER VIRTUELLEN BERÜHRUNGSBILDSCHIRM-TASTATUR";
• U. S. Patentanmeldung Nr. 11/228,758, eingereicht am 16. September 2005 mit dem Titel "VIRTUELLE EINGABEVORRICHTUNGSANORDNUNG AUF EINER BERÜHRUNGSBILDSCHIRM-BENUTZERSCHNITTSTELLE";
• U. S. Patentanmeldung Nr. 11/228,700, eingereicht am 16. September 2005 mit dem Titel "OPERATION EINES COMPUTERS MIT BERÜHRUNGS-BILDSCHIRM-SCHNITTSTELLE";
• U. S. Patentanmeldung Nr. 10/927,925, eingereicht am 26. August 2004 mit dem Titel "VIRTUELLER ERWEITERER";
• U. S. Patentanmeldung Nr. 10/927,575, eingereicht am 25. August 2004 mit dem Titel "BREITES BERÜHRUNGSFELD AUF EINEM TRAGBAREN COMPUTER";
• U. S. Patentanmeldung Nr. 11/619,553, eingereicht am 3. Januar 2007 mit dem Titel "WÖRTERBUCH FÜR MEHRFACHBERÜHRUNGSGESTEN" und
• U. S. Patentanmeldung Nr. 11/619,571, eingereicht am 3. Januar 2007 mit dem Titel "WÖRTERBUCH FÜR MEHRFACHBERÜHRUNGSGESTEN".

This application also relates to the following patent applications, each incorporated herein by reference:

• U.S. Patent Application No. 10 / 903,964, filed July 330, 2004, entitled "GESTURES FOR TOUCH-SENSITIVE INPUT DEVICES";
• US Patent Application No. 11 / 015,434 filed on Dec. 17, 2004, entitled "METHOD AND DEVICE FOR INTEGRATING MANUAL INPUT", which is the priority of US Patents: 6,323,846 granted November 27, 2001, which claims priority to US Provisional Patent Application No. 60 / 072,509, filed January 26, 1998;
• US Patent Application No. 10 / 654,108, filed on September 2, 2003, entitled "BRAID MOUSE";
• U.S. Patent Application No. 10 / 789,676, filed February 27, 2004, entitled "FORM DETECTING INPUT DEVICE";
• U.S. Patent Application No. 10 / 840,862 filed May 6, 2004, entitled "MULTIPURPOSE TOUCH SCREEN";
• U.S. Patent Application No. 11 / 115,539, filed April 26, 2005, entitled "HANDMARKABLE ELECTRONIC DEVICE WITH MULTIPLE TOUCH SCREENS";
• U.S. Patent Application No. 11 / 241,839, filed July 30, 2004, entitled "Sewing Detector in Hand-held Device";
• Preliminary US Patent Application No. 60 / 658,777, filed March 4, 2005, entitled "MULTIFUNCTIONAL HAND-HELD DEVICE";
• U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, entitled "GESTURES FOR TOUCH SENSITIVE INPUT DEVICES";
• US Patent Application No. 11 / 038,590, filed January 18, 2005, entitled "MODE-BASED GRAPHIC USER INTERFACES FOR TOUCH-SENSITIVE INPUT DEVICES";
• U.S. Patent Application No. 117048,264, filed January 31, 2005, entitled "GESTURES FOR TOUCH SENSITIVE INPUT DEVICES";
• US Patent Application No. 11 / 228,737, filed September 16, 2005, entitled "ACTIVATING VIRTUAL TOPS OF A VIRTUAL TOUCH SCREEN KEYBOARD";
• US Patent Application No. 11 / 228,758, filed September 16, 2005, entitled "VIRTUAL INPUT DEVICE ARRANGEMENT ON A TOUCH SCREEN USER INTERFACE";
• US Patent Application No. 11 / 228,700, filed September 16, 2005, entitled "OPERATING A COMPUTER WITH A TOUCH SCREEN INTERFACE";
• U.S. Patent Application No. 10 / 927,925, filed August 26, 2004, entitled "VIRTUAL EXTENDER";
• US Patent Application No. 10 / 927,575, filed on August 25, 2004, entitled "WIDE TOUCH FIELD ON A PORTABLE COMPUTER";
• US Patent Application No. 11 / 619,553, filed January 3, 2007, entitled "DICTIONARY FOR MULTIPLE TOUGHEST GESTURES" and US Ser
• US Patent Application No. 11 / 619,571, filed January 3, 2007, entitled "DICTIONARY FOR MULTILAYER GESTURES".

BACKGROUND OF THE INVENTION

1. FIELD OF THE INVENTION

The The present invention relates generally to gesturing, which refers to multipoint scanning devices.

2. DESCRIPTION OF THE RELATED TECHNIQUE

It Today there are several types of input devices to perform of operations in a computer system. The operations correspond to Generally moving a cursor and dialing on a display screen. The operations can also Scroll, scroll, pan, zoom, and so on. To the For example, the input devices may be buttons, Switches, keyboards, mice, rolling balls, touch pads, joysticks, Touch screens and the like include. Each of these devices has advantages and disadvantages that are considered need to be when designing the computer system.

With touch pads, the movement of the input pointer corresponds to the relative movements of the user's finger (or stylus) as the finger is moved along a surface of the touchpad. On the other hand, touch screens are a type of display screen that has a touch-sensitive translucent panel that covers the screen. If he has a touch A user makes a selection on the display screen by directly pointing to GUI objects on the screen (usually with a pen or finger). In general, the touch device recognizes the touch and position of the touch, and the computer system interprets the touch and then performs an action based on the touch event.

Around additional functionality available too gestures have been implemented with some of these input devices. For example, in touchpad selections be made when one or more taps on the surface of the Touchpad is detected. In some cases can be typed on any section of the touchpad, and in other cases may be on a particular section be touched the touchpad. About selections In addition, scrolling can be accomplished by using a finger gesture on the edge of the finger Touch field be prompted.

Unfortunately gesturing is severely limited by the fact that most touch technologies only able are to report a single point, even if multiple objects placed on the scanning surface. This means they lack the ability to make multiple contact points simultaneously to track. In resistance and capacitance technologies becomes an average of all concurrent points of contact determined and a single point somewhere between the points of contact falls, is reported. In surface wave and Infrared technology is impossible because of masking to distinguish the exact position from multiple points of contact, which fall on the same horizontal or vertical line. In both cases erroneous results are generated.

On Based on the above, there is a need for Multi-point scanning devices and methods for implementing Gestures with multipoint scanners.

SUMMARY OF THE INVENTION

The Invention relates in one embodiment to a electronic system. The electronic system comprises a multipoint scanning device, which is a multipoint scan area for receiving inputs from one or multiple objects. The electronic System also includes a gesture module that is configured to be one Set of gestures to determine for a given input device which are received by the multipoint scanning area of the multi-point scanning apparatus becomes the given input arrangement for one or more Monitor gesture events included in the gesture set are and trigger input actions with a gesture event are associated when the gesture event with the input device is performed. The input device can, for. B. an arrangement from fingers and / or other parts of the hand.

The Invention relates in another embodiment on a gesture control process. The method includes detecting multiple points within a scan area at the same time. The method also includes determining a chord if one or several points within the scanning area are detected. Of the Chord is a special arrangement of points within the scanning area. The method further comprises determining a gesture set that Combine commands with one or more gesture events. The procedure further includes monitoring points for gesture events. Furthermore, the method comprises executing a command, which communicates with the gesture event when a gesture event is recognized.

The Invention relates in another embodiment to a control operation. The control operation includes detecting a touch or near-touch. The operation also includes determining a gesture set for the touch. The gesture set includes one or more gesture events for triggering or initiating a command. The operation also includes monitoring the Touch for a gesture event. The operation includes additionally initiating a command when a gesture event, that is associated with the gesture set is executed.

The Invention relates in another embodiment on a gesture operation. The operation includes monitoring a touch movement. The operation also includes distinguishing the Contact movement between a first and second state. The operation further includes performing a first Action when the touch movement with the first state is associated. The operation also includes performing a second action when the movement associated with the second state becomes.

The invention relates in a further embodiment to a control operation. The control operation includes providing a first input device and a second input device different from the first input device. The first input device comprises an object-scanning device, such as a touch-scanning device, for providing input events. The operation also includes monitoring the first input device for input events. The operation further includes simultaneous monitoring of the second input device for input events. The operation further includes performing input operations in accordance with input events associated with the first input device. Furthermore, the method comprises concurrently performing input operations in accordance with input events associated with the second input device.

The Invention relates in a further embodiment to a control operation. The control operation provides a list of input functions. The input functions have commands and gesture events, which are connected to the commands. The commands are available with the Input function in connection. The operation also includes mapping from input functions to chords. The operation includes about it adding an input function to a chord when the chord is recognized.

The Invention relates in a further embodiment on a control panel. The control panel includes displaying a menu of input functions. The control panel also includes displays of a gesture set, shows the commands with a selected input function and gesture events associated with the commands are. The control operation further includes displaying a list of chords. To set up a gesture image, select a user a desired input function from a List of input functions and selects a desired one Chord from the list of chords.

The Invention relates in a further embodiment on a gesture design process. The design process includes the Providing a list of chords and arranging the chords from the simplest to the most difficult. The design process also includes arranging the frequency of operations of most frequently least often. The design process continues to include Mapping the simplest chords with the most common operations and assigning the most difficult chords with the rarest operations. The design process may additionally fine tune Chord / operation grouping include.

The Invention relates in a further embodiment on a gesture operation. The gesture operation involves detecting a first finger. The gesture operation also includes determining the condition of the finger. The state of the finger can z. B. moving or be stuck. The gesture operation further includes detecting a or several additional fingers. For example, a second finger are detected. The gesture operation includes about it In addition, determining the state of the additional fingers. The state of the additional finger can z. B. be that they are present or not. Furthermore, the method comprises implementing different input modes based on the temporal assignment of States of the first and the additional fingers based on each other. The different modes can z. As pointing modes, drawing modes and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The The invention will become readily apparent from the following detailed description in conjunction with the appended drawings where the same relational signs are the same structural elements denote and in which:

1 a gesture control operation in accordance with an embodiment of the present invention;

2 a control operation in accordance with an embodiment of the present invention;

3 a control operation in accordance with an embodiment of the present invention;

4 a control operation in accordance with an embodiment of the present invention;

5 a control operation in accordance with an embodiment of the present invention;

6 a control operation in accordance with an embodiment of the present invention;

7 a gesture operation in accordance with an embodiment of the present invention;

8th a gesture operation in accordance with an embodiment of the present invention;

9 a gesture operation in accordance with an embodiment of the present invention;

10 a gesture operation in accordance with an embodiment of the present invention;

11 a gesture operation in accordance with an embodiment of the present invention;

12 a gesture operation in agreement mood with an embodiment of the present invention;

13 a gesture operation in accordance with an embodiment of the present invention;

14 a gesture operation in accordance with an embodiment of the present invention;

15 a gesture operation in accordance with an embodiment of the present invention;

16 a gesture operation in accordance with an embodiment of the present invention;

17 a gesture operation in accordance with an embodiment of the present invention;

18 a gesture operation in accordance with an embodiment of the present invention;

19 a gesture operation in accordance with an embodiment of the present invention;

20 a control operation in accordance with an embodiment of the present invention;

21 a control operation in accordance with an embodiment of the present invention;

22 a control operation in accordance with an embodiment of the present invention;

23 a control operation in accordance with an embodiment of the present invention;

24 a control operation in accordance with an embodiment of the present invention;

25 a control operation in accordance with an embodiment of the present invention;

26 a control operation in accordance with an embodiment of the present invention;

27 a control operation in accordance with an embodiment of the present invention;

28 a control operation in accordance with an embodiment of the present invention;

29A to G are diagrams of a control panel in accordance with an embodiment of the present invention;

30 a control operation in accordance with an embodiment of the present invention;

31 a gesture assigning method in accordance with an embodiment of the present invention;

32 a gesture assignment method in accordance with an embodiment of the present invention;

33 is a method of gesture feedback in accordance with an embodiment of the present invention;

34 Fig. 10 is a diagram of a gesture viewing window in accordance with an embodiment of the present invention;

35 a keyboard method implemented via a touch screen in accordance with an embodiment of the present invention;

36 a gesture design method in accordance with an embodiment of the present invention;

37 is a matrix for evaluating chords in accordance with an embodiment of the present invention;

38 a matrix for ranking the frequency of operations for a few operations in accordance with an embodiment of the present invention;

39 a control operation in accordance with an embodiment of the present invention;

40 a control operation in accordance with an embodiment of the present invention;

41 a control operation in accordance with an embodiment of the present invention;

42 Fig. 12 is a diagram of a gesture board in accordance with an embodiment of the present invention;

43 Fig. 12 is a diagram of a gesture board in accordance with an embodiment of the present invention;

44 Fig. 12 is a diagram of a gesture board in accordance with an embodiment of the present invention;

45 Figure 12 is a diagram of a gesture set in accordance with an embodiment of the present invention;

46 Figure 12 is a diagram of a gesture set in accordance with an embodiment of the present invention;

47 Figure 12 is a diagram of a gesture set in accordance with an embodiment of the present invention;

48 Figure 12 is a diagram of a gesture set in accordance with an embodiment of the present invention;

49 a gesture operation in accordance with an embodiment of the present invention;

50 a gesture operation in accordance with an embodiment of the present invention;

51 a gesture operation in accordance with an embodiment of the present invention;

52 a gesture operation in accordance with an embodiment of the present invention;

53 a gesture operation in accordance with an embodiment of the present invention;

54 Fig. 10 is a block diagram of an electronic system in accordance with an embodiment of the present invention;

DETAILED DESCRIPTION THE INVENTION

gestures and method of implementing gestures with scanners are revealed. Specifically, gestures and methods for implementing gestures with multipoint scanners. Have multi-point scanners a number of advantages over conventional ones Single-point devices, in that they are more than one object (Finger) at the same time or almost simultaneously. In most cases, multi-point scanners monitor and systems using such devices have a surface with respect to a touch or near-touch event. When such an event occurs, it may have the different Contact area (s) determine and the type of events over identify their geometric features and their geometric arrangement. As soon as you are identified, touch or near-touch events are monitored, to determine if they correspond to multiple gesture events.

One Gesture event may be considered a stylized interaction with the sensing surface be defined, the one or more special calculation operations is assigned. Gesture events can be different Hand, especially finger movements, typing, pressures, dwell times and / or the like. Because the surface based on multipoint technology, can be complex gesturing with any number of fingers or other contact sections to be done by hand. In fact, therefore, one can great gesture language to be developed, which is analogous to sign language is. Gestural language (or figure) can z. B. a command set comprising a contact arrangement (e.g., chords) that detects Recognition of gesture events (eg movements) and recognizes one or multiple software agents via the gesture events informed and / or about which action (s) in response to perform on the gesture events is / are. Examples for multipoint gesturing, for example, In U.S. Patent Application Nos. 10 / 903,964, 11 / 038,590 and 11 / 015,434 are found, all be incorporated herein by reference.

A wide array of different gestures can be used with multipoint scanners. For example, a gesture may be a single-point or a multi-point gesture; a static or dynamic gesture, a continuous or segmented gesture; and / or the like. Single-point gestures are those gestures that are performed with a single point of contact, e.g. For example, the gesture is performed with a single touch such as from a single finger, palm or stylus. Multipoint gestures are those gestures that can be performed with multiple points, e.g. For example, the gesture will be performed with multiple touches, such as multiple fingers, fingers and palms, a finger and a stylus, multiple styluses, and / or any combination thereof. Static gestures can be those gestures that occur in the Essentially, there are no Geste events (eg, chords), and dynamic gestures can be those gestures that involve significant gesture events (eg, movements, typing, etc.).

continuous Gestures can be those gestures that are in a single gesture Train can be performed, and segmented gestures can those gestures that are in a sequence of different Steps or trains are performed. Even though Here are several examples to be given, it should be recognized that this is exemplary and not restrictive.

Multipoint sensing can be implemented in several ways, including - but not limited to - standard size touch fields, large expanded palm panels, touch screens, touch-sensitive housing, etc. Continue can multi-point scanners on many form factors be attached, including - but not limited to - tablet computers, laptop computers, Desktop computers and handheld computer devices such as Media players, PDAs, mobile phones and the like. The Multipoint scanning devices can also be dedicated Input devices such as touchscreen monitors, Keyboards, navigation panels, tablets, mice and the like found become. Examples of multipoint scanning devices may be in U.S. Patent Application Nos. 10 / 840,862, 10 / 927,575 will be included here by reference.

Several embodiments will now be described below with reference to FIGS 1 to 54 to be discussed. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes, as the invention extends beyond these limited embodiments.

1 shows the illustrative gesture control operation 10 in accordance with an embodiment of the present invention. The operation 10 can at block 12 begin where a multipoint scanner is provided. The multipoint scanner is capable of detecting multiple contact or near contact points simultaneously. The multi-point scanning device may, for. B. comprise a multipoint sensing surface capable of simultaneously scanning multiple objects on the touch surface. Alternatively or additionally, the multi-point scanning device may include capabilities of scanning objects that are in close proximity to a surface but that do not touch the surface (eg, proximity scan). The multi-point scanning device may be implemented as a touch screen, touch pad, touch-sensitive palm rest, touch-sensitive housing, and the like.

The block 12 Following the operation after block 14 where a determination is made as to whether a touch or near-touch is detected by the multi-point sensing device. If no touch is detected, the operation may wait. If a touch is detected, the operation may block 16 continue where a chord associated with the touch is determined. A chord can be z. A particular arrangement of contacts or near-contacts associated with any input functionality. In general, when objects are placed near or on a touch-sensitive surface, pads are created, and these pads form a pattern that can be identified. The pattern can be made with any selection of objects, including e.g. Mechanical features such as pins or punches and / or portions of one or more hands such as fingers, thumb, palm, ankles, etc.

Of the Chord can vary widely and may depend on many factors including the size of the touch surface, whether the touch surface is a touch screen or a touchpad, etc. Furthermore the chords on a number of unknown contacts or one based special arrangement of known contacts. The chords can continue to be based on whether the contacts are close together, in a neutral position or widely scattered. The chords can still be based on whether the contacts adjacent or offset from each other. The contacts can continue to be based on whether they are from a left and / or right Hand are.

Determining the chord may include analyzing the touch (the image created by objects that touch or nearly touch the touch surface) and detecting a particular arrangement of contacts. Specifically, the determination may include classifying or identifying the contacts about geometric features of each contact, as well as the geometric arrangement of contacts, and then referencing a database of expected chords (eg, gesture mapping). If the contact arrangement matches an expected chord, then it can be assumed that the contact arrangement is the expected chord. These steps may also include filtering. For example, some contacts may not actually be part of a chord. For example, in the case of a palm touch panel, contacts formed by the palm can be ignored as they serve as a bearing surface for the fingers of the hand (e.g. convenient to hold palms over the touch surface for long periods of time).

in the Case of a tablet PC with a touch screen can Contacts formed by the thumb of at least one hand, be ignored because they are used primarily to to hold the tablet PC.

Examples One-handed chords are shown in Table 1 below. It should be recognized that Table 1 is not an exhaustive list is and that they are as an example and not as a limitation will be shown. For example, the palm can act as a contact and any of those shown in Table 1 Combinations are added to create more chords. It should also be noted that many chord combinations are more can be formed by the list of Table 1 with the same list from the opposite hand combined becomes. It should also be noted that although there are many combinations There are some chords that can not be made from different ones Reasons, including ease of use, Ergonomics, intuitiveness, etc.

Once the chord has been determined, the operation may block 18 go on, where a gesture set that connects actions to one or more gesture events is determined. A gesture set can be an image that associates actions with gesture events. The gesture set may depend on the particular chord, but it may also depend on other factors including location of the chord, open application, state or mode of the application, other touch characteristics, etc. Essentially, the chord selects an input channel, each channel having a different gesture set associated with it. For organization and ease of use, in some cases the channels are associated with a particular input functionality, such as navigation operations, file operations, editing operations, view operations, formatting operations, tool operations, web browsing operations, and so on. Each of these input functionalities may have an associated instruction set associated with the gesture events.

The Actions can be state commands or manipulative commands be. A state command is a single implemented command. Examples of state commands include New, Open, Close, Delete, Rename, Select All, Cut, Copy, Paste, Undo, Repeat, Group / Group Dissolve, Italic, Fat, Underline, Previous / Next, Play / Pause, Mute etc. Other examples of state commands include starting of a particular program, starting a particular mode of a Program, starting a webpage, opening a Toolbar, opening a menu, arrow keys, Delete key, space, enter, tab, caps lock, Functions, Zoom, Zoom In / Zoom Out, Scroll, Volume, etc. A manipulative command is a command, which continuously manipulates the selected object. Examples manipulative commands include pointing, tracking, dragging, Scrolling, panning, zooming, resizing, Stretching, scrolling, volume, etc. It should be recognized will be that these examples are not limiting, and that other commands can be used.

As As noted above, a gesture event can be any stylized physical Be action on or above the touch surface is carried out. Examples of gesture events may e.g. As movements, typing, pressure changes, lingering, etc. include. In some cases, the gesture events be performed by the chord. In other cases The gesture events can be represented by a subset of the chord be performed. In other cases you can the gesture events through new contacts in addition to or be performed separately from the initial chord. Gesture events will be described in more detail below.

In block 20 the touch can be monitored for gesture events and in block 22 a determination can be made as to whether gesture events are recognized or not.

The Recognition of gesture events can. Analyze the touch properties of contacts (contacts associated with the chord are and / or new contacts), identifying a particular pattern and reference to the gesture set and its list of expected Gesture events. If the pattern matches an expected gesture event, then it can be assumed that the pattern is the expected gesture event is. The contact characteristic may, for. B. Considerations first order, such as movement, typing, pressure change, Lingering and second-order considerations, such as Speed (absolute or relative), direction (absolute or relative), orientation (absolute or relative), size (absolute or relative), duration (absolute or relative), form (absolute or relative), length (absolute or relative) and / or the like.

In In one embodiment, gesture events designed or chosen to be different from others Gesture events are distinguishable to overlays or to avoid errors in recognizing the gesture event (e.g. For example, you typically want gesture events from others Gesture events have distinguished).

Again, Gesture events can be as movement, typing, pressure, lingering etc. are classified. If they are exercise, you can the gesture events are exemplified as follows: Sliding in any direction, slow linear brushing to the right, left, up and down; a quick linear snap to the right, left, up and down; a slow clockwise rotation or counterclockwise; a fast clockwise rotation or counterclockwise, contracting or unfolding two or multiple contacts (for example, spreading and closing a Separation between contacts). If you are a typist, you can the gesture events will continue to be exemplified as follows: ½ typing, all typing, multiple typing, fast Typing, slow typing, and so on. If they're printing, you can the gesture events continue to be exemplified as light pressure or strong pressure. If you The gesture events can continue to be exemplary be presented as a long stationary arrest or short one stationary arrest. It should be recognized that this List is not a restriction and that gesture events be classified in other ways or with other orientations can. For example, linear motion gestures are not on limited right and left, up and down and can continue to include top right, top left, bottom right, bottom left.

The Gesture events can be basic or advanced be. Basic gestures can be those gestures with be an action, such as B. Swipe linear to the right. advanced Gestures can be those gestures with multiple actions such as B. Swipe linearly to the right followed by a tap or swipe linearly to the right; long stop, stroke linear to the left; or turning counterclockwise with raised or reduced pressure. Advanced gestures can be one include any number of actions. Basic gestures can for ease of use and processing complexity reasons be used.

If a gesture event is detected, the operation may block 24 proceed where the action (s) associated with the gesture events are performed. Block 24 may refer to the gesture set and find the action (s) associated with the detected gesture events. Once found, the action (s) can be initiated. If the action is a status command, the command may be initiated each time the gesture event is performed (eg, the gesture event works as a button). In some cases, the gesture event may be designed to generate repetitive state commands. If the user z. For example, as the gesture event expands or continues, multiple state commands may be initiated. This can be helpful in a fast fire play mode or in an edit mode when undoing work or in a scrolling mode when scrolling through pages. If z. For example, if a single undo command is initiated by a fast linear swipe to the right, a repeating undo command can be initiated by a fast linear swipe followed by a slow linear swipe in a continuous motion (during touch). In this example, the undo command may be repeatedly initiated in accordance with the slow linear swipe (eg, the undo command is fired every 100 milliseconds during slow linear swipe). If the action is a manipulative command, the command may be generated continuously during the gesture event (eg, one-to-one correspondence between command and gesture event). If the command z. Scrolling, scrolling may be performed as the gesture event is performed and in accordance with the gesture event properties (eg, speed, direction, etc.).

After block 24 can the operation after block 26 continue where a determination is made as to whether or not a switching event has been executed. A switching event may refer to an event that resets the operation or initiates a chord change. The switching event can be implemented in different ways. For example, it may be implemented by removing all contacts for a predetermined period of time (eg, raising the hand from the touch surface). It can also be implemented by changing the base chord during touch (for example, adding / removing contacts). It can also be implemented by adding / removing contacts of the opposite hand (eg, putting one or more fingers on the opposite hand while the other hand is still touching). It can also be implemented by pausing (for example, if the contacts remain stationary for a set period of time). It can also be implemented by keystroke or key click from a standard keyboard or mouse. It can also be implemented by a gesture event. If a switching event has occurred, the operation goes back to block 12 , If no switching event has occurred, the operation goes back to block 20 ,

Table 1 - Examples of one-handed chords

Unknown contacts - standard arrangement (no palm or palm ignored)

• Any finger
• Any two fingers
• Any three fingers
• Any four fingers
• Thumb + any finger
• Thumb + any two fingers
• Thumb + any three fingers
• Thumb + four fingers

Unknown contacts - varied Arrangement (no palm or palm ignored)

• Two fingers lying next to each other
• Two non-adjacent fingers
• Two adjacent fingers + one not adjacent finger
• Thumb + two fingers next to each other
• Thumb + two non-adjacent fingers
• Thumb + two adjacent fingers + one not adjacent finger
• Any two adjacent fingers closed
• Any two fingers spread apart
• Any three adjacent fingers closed
• Any three fingers spread next to each other
• Four fingers next to each other closed
• Four fingers spread next to each other
• Thumb + two fingers next to each other closed
• Thumb + two fingers spread apart
• Thumb + three fingers next to each other closed
• Thumb + three fingers spread next to each other
• Thumb + four fingers next to each other closed
• Thumb + four fingers spread apart

Known contacts (no palm or palm ignored)

• index finger
• middle finger
• ring finger
• Pinkie finger
• Index finger + middle finger
• Index finger + ring finger
• Index finger + little finger
• Middle finger + ring finger
• Middle finger + little finger
• Ring finger + little finger
• Thumb + index finger
• Thumb + middle finger
• Thumb + ring finger
• Thumb + little finger
• Thumb + index finger + middle finger
• Thumb + index finger + ring finger
• Thumb + index finger + little finger
• Thumb + middle finger + ring finger
• Thumb + middle finger + little finger
• Thumb + ring finger + little finger
• Index finger + middle finger + ring finger
• Index finger + middle finger + little finger
• Index finger + ring finger + little finger
• Middle finger + ring finger + little finger
• Thumb + index finger + middle finger + ring finger
• Thumb + index finger + middle finger + little finger
• Thumb + index finger + ring finger + little finger
• Thumb + middle finger + ring finger + little finger
• Index finger + middle finger + ring finger + little finger
• Thumb + index finger + middle finger + ring finger + little finger

Other (palm down)

• Finger closed to fist or palm wound
• Index finger + remaining fingers closed to fist or wrapped to palm
• Index finger + middle finger + remaining fingers closed to fist or wrapped to palm
• Index finger + middle finger + ring finger + little finger too Fist closed or wrapped to palm
• Thumb + remaining fingers closed to fist or closed Palm wrapped
• Thumb + index finger + remaining fingers closed to fist or wrapped to palm
• Thumb + index finger + middle finger + remaining fingers too Fist closed or wrapped to palm
• Thumb + little finger + middle finger + ring finger + smaller Finger closed to fist or wrapped to palm
• Thumb + index finger + remaining fingers closed to fist or wrapped to palm
• Thumb + index finger + middle finger + remaining fingers too Fist closed or wrapped to palm
• Thumb + index finger + middle finger + ring finger + little finger closed to fist or wrapped to palm

other

• Right hand side
• Left hand side
• Hand back
• Hand front (whole)

Other (ankle down - fist blow)

• Finger closed to fist or palm wound
• Index finger open + remaining fingers closed to fist or wrapped to palm
• Index finger open + middle finger open + remaining fingers too Fist closed or wrapped to palm
• Index finger open + middle finger open + ring finger open + little finger closed or wrapped in a fist
• Thumb + fingers closed to fist or palm wound
• Thumb + index finger open + remaining fingers closed to fist or wrapped to palm
• Thumb + index finger open + middle finger open + left Finger closed to fist or wrapped to palm
• Thumb + index finger open + middle finger open + ring finger open + little finger closed to fist or palm wound

2 shows an illustrative control operation 50 in accordance with an embodiment of the present invention. The surgery may be at block 52 begin where a touch or near-touch is detected.

After block 52 can block the operation 54 go on where a gesture set is determined for the touch. The gesture set may depend on many factors, including touch characteristics, touch location, application, application mode, or the like. In most cases, the gesture set will be based, at least in part, on the contact placement during weaning.

After block 54 can block the operation 56 continue where touch is monitored for gesture events associated with gestures. The gesture set may include one or more gesture events for initiating or initializing a command (eg, commands may be tied to or associated with certain gesture events).

After block 56 can block the operation 58 continue where one or more commands are initiated when gesture events are performed. For example, a user may slip a particular finger arrangement to invoke or initiate a scrolling event.

3 shows an illustrative control operation 60 in accordance with an embodiment of the present invention. The control operation 60 can z. B. the block 54 in 2 correspond. The control operation 60 can at block 62 begin where the initial contact arrangement is detected. After that, in block 64 comparing the initial contact arrangement with a stored set of contact arrangements. For example, the system may refer to a gesture map that includes a list of initial contact arrangements and gesture sets associated with them. If there is a match, the operation may block 66 continue where the gesture set associated with the detected initial contact arrangement is loaded.

4 shows an illustrative control operation 70 in accordance with an embodiment of the present invention. The control operation 70 can at block 72 begin where a touch is detected. After that, in block 74 the location of the touch will be determined. After that, in block 76 the contact arrangement associated with the touch can be determined (eg, touch pattern). After that, in block 78 the active application will be determined. After that, in block 80 the current state of the application can be determined (eg current mode). After that, in block 82 an appropriate gesture set may be set or selected based on one or more particular attributes mentioned above (blocks 74 to 80 ). Using the determined attributes, a system can e.g. For example, refer to a stored gesture map that associates each of the attributes mentioned above with a particular gesture set. The stored gesture image may be generated by default or it may be user-customizable. The user can z. For example, use a control panel to change settings associated with gesture mapping, or use a training sequence to program gesture mapping.

5 shows an illustrative control operation 100 in accordance with an embodiment of the present invention. The control operation can be at block 102 begin where a determination is made as to whether a touch has been detected or not. If a touch has been detected, the operation may go to block 104 go on where the contact arrangement is detected. block 104 can the subblocks 106 and 108 contain. In block 106 a determination is made as to whether the contact points can be accurately identified. For example, if a contact point can be an index finger or a thumb or a palm. If they can not be accurately identified, then the operation may be after block 108 continue where a number of contact points are determined. For example, if there are two contact points, three contact points, etc. After block 104 can the operation after block 110 continue where the recognized contact arrangement is compared with stored contact arrangement in a gesture map. If there is no match, then the operation can go to block 102 decline. If there is an agreement mood gives, then the operation after block 112 continue where, after reference to the gesture map, the gesture set associated with the initial contact layout is loaded. After that, in block 116 the touch is monitored for gesture events associated with the gesture set. When a gesture event is performed, the operation may proceed to block 118 proceed where the gesture associated with the gesture event is executed.

6 shows an illustrative control operation 120 in accordance with an embodiment of the present invention. The control operation 120 can at block 122 begin where a touch or near-touch is detected. After that, in block 124 a chord to be determined for the touch. After that, in block 126 determine the input functionality associated with the chord. The input functionality may describe an operation or group of operations. Examples of operations include navigation operations, file operations, editing operations, view operations, insert operations, formatting operations, tool operations, web operations, etc. The input functionality may continue to be based on environmental conditions, including the touchpad chord position, applications, modes, and so forth 128 the gesture set associated with the input functionality is activated or loaded. A gesture set may be a group of gesture events, each associated with a different action related to the input functionality. The gesture event may be performed by the initial chord alone, either as a group, a subset of the group, or relative to another. Alternatively, they may be performed by objects that are not associated with the chord (eg, pads added after the base chord is set). The gesture event may include first order parameters such as motion, typing, lingering, pressure and / or the like. The gesture event may also include second order parameters that define first order parameters such as speed, direction, shape, timing / duration, length, and / or the like. After that, in block 130 Actions associated with the gesture events are implemented when the gesture events are performed.

7 shows an illustrative gesture operation 140 in accordance with an embodiment of the present invention. The operation 140 can at block 142 begin where a touch is detected. After block 144 can the operation after block 144 continue where a contact arrangement is detected. After that, in block 146 the chord associated with the recognized contact arrangement is determined. After block 146 can the operation after block 148 continue where the gesture set associated with the chord is loaded. The gesture set contains gesture events that can be performed with any contact arrangement during the touch. After that, in block 150 monitoring the touch for a gesture event performed by any contact arrangement. When a gesture event is performed by any contact arrangement, the operation may proceed to block 152 continue where the control / command associated with the gesture event is initiated. If no gesture event has been performed, the operation may move to block 154 go on where a determination is made as to whether a touch is still detected or not. If a touch is still detected, the operation may block 150 decline. If a touch is not detected, the operation may go back to block 142 walk. That is, a lift followed by a touch resets the chord and thus the gesture set.

The above in 7 The initial chord selection model described advantageously allows a user to lift or drop additional fingers without touching the gesture set. This is more ergonomic in that a user who starts in a base chord of two fingers can drop the remaining three fingers so that all five fingers support the hand. As a result, gesturing can be performed during longer periods of time. Essentially, changes to the base chord are ignored. Furthermore, it allows users to expand / collapse scrolling into limited-surface touch surfaces by raising all but one finger and allowing other fingers to reach or slide to their destination.

8th shows an illustrative gesture operation 160 in accordance with an embodiment of the present invention. The operation 160 can at block 162 begin where a determination is made as to whether 2 adjacent fingers are detected or not. If so, the operation may block 164 continue where pointing operations are performed in accordance with motion. If not, the operation may block 166 go on, where a determination is made as to whether 3 adjacent fingers are detected or not. If so, the operation may go to block 168 proceed where pulling operations are performed in accordance with movement. If not, the operation may be after block 170 go on where a determination is made as to whether a thumb and two adjacent fingers are detected or not. If so, the operation may go to block 172 continue where secondary drag operations are performed. If not, the operation can go back to block 162 continue. This process can be reset every time all fingers are lifted off the touch surface (eg touch is no longer detected).

9 shows an illustrative gesture operation 180 in accordance with an embodiment of the present invention. The operation 180 can at block 182 begin where a base chord is determined. Thereafter, the operation may perform three different processes either separately or simultaneously (parallel blocks 184 to 188 ). In block 184 movement can be detected with the base chord. After that, in block 190 Pointing operations are performed in accordance with the movement. In block 186 a new first finger can be detected. That is, a new first finger that is not associated with the base chord can be detected (in addition to the base chord). After that, in block 192 a first command is initiated each time the new first finger is detected. In some cases, a user may perform repetitive commands by continuously tapping the new first finger. In block 188 a new second finger can be detected (in addition to the base chord). That is, a new second finger that is not associated with the base chord can be detected. After that, in block 194 a second command is initiated each time the new second finger is detected. In some cases, a user may perform repetitive commands by tapping the new second finger continuously, or simply by holding the finger down. In the 9 The described operation is sometimes called a model for selecting the current chord.

In an example of 9 can the base chord 3 Be fingers (either index finger middle finger ring finger or thumb middle finger ring finger). Thus, three fingers can be used for pointing operations. Furthermore, the new first finger may be the thumb or forefinger if the thumb is already part of the base. Thus, the thumb or forefinger can initiate the first loading. Furthermore, the new second finger can be the little finger. Thus, the little finger can initiate the second command.

The The above example may be useful in game modes. The basic chord can be used to pointing, the first new finger can to Triggering can be used and the second new finger can to be used for a weapon change. Alternatively, you can in standard modes the base chord can be used for pointing, the First new fingers can be used for primary clicking and dragging can be used and the second new finger can be for one Secondary click and drag are used. The three Finger support provides adequate support in both game and standard modes.

10 shows an illustrative gesture operation 200 in accordance with an embodiment of the present invention. The operation 200 can at block 202 begin where a touch is detected. After block 204 can the operation after block 204 continue where a contact arrangement is detected. After that, in block 206 , the chord associated with the recognized contact arrangement is determined. After block 206 can the operation after block 208 continue where the gesture set associated with the chord is loaded. The gesture set may include gesture events that may be performed with any contact arrangement during the first touch. After that, in block 210 the touch is conveyed with respect to a gesture event performed by any contact arrangement. If a gesture event has been performed by any contact arrangement, the operation may proceed to block 212 continue where the controller / command associated with the gesture event is initiated. If a gesture event has not been performed, the operation may move to block 214 go on, where a determination is made as to whether the base chord or the current contact arrangement has been paused during the touch. If so, it can be assumed that the pause is a chord change event and the operation can go back to block 204 continue. If not, the operation may be after block 216 go on where a determination is made as to whether a touch is still detected or not. If a touch is still detected, the operation may go back to block 210 walk. If a touch is not detected, the operation may go back to block 202 continue.

In the in 10 Changeover selection model described above can only take place when the original fingers are stopped or move rather slowly. Thus, users can relax from a finger to five-finger pointing as long as the finger-down is in the middle of a pointing move.

11 shows an illustrative gesture operation 220 in accordance with an embodiment of the present invention. The operation 220 can at block 222 begin where a determination is made whether 1 finger is detected or not. If so, the operation may be after block 224 continue where pointing operations are performed in accordance with motion. If not, the operation may be after block 226 go on where a determination is made, whether two next to each other lying fingers are detected or not. If so, the operation may go to block 228 proceed where pulling operations are performed in accordance with movement. In some cases, the pull lock / extension can be initiated by clicking and in other cases it is initiated by dropping the thumb (two adjacent fingers + thumbs). If not, the operation may be after block 230 go on where a determination is made as to whether or not two non-adjacent fingers are detected. If so, the operation may go to block 232 continue where secondary drag operations are performed. In some cases, the pull lock / extension can be initiated by clicking and in other cases it is initiated by dropping the thumb (two non-adjacent fingers + thumbs). If not, the operation may be after block 234 go on where a determination is made whether three or four fingers are detected or not. If so, scrolling can be initiated in accordance with motion (block 236 ). If not, the operation can go back to block 222 continue. This process can be reset every time all fingers are lifted off the touch surface (eg, touch is no longer detected), or when the chord is paused for a moment.

12 shows an illustrative gesture operation 240 in accordance with an embodiment of the present invention. The operation 240 can at block 242 begin where a determination is made as to whether 5 fingers are detected or not. If so, the operation can continue to block 244 go where pointing is initiated until all fingers are lifted off the touch surface. In block 246 a determination is made as to whether or not one of the 5 fingers makes a lift and a tap while pointing. If so, a click can be performed (block 248 ). In block 250 a determination is made as to whether or not two of the five fingers take off and tap during pointing. If so, pulling can be done (block 252 ). After that, in block 253 a determination is made as to whether one or two of the five fingers make a lift and tap while dragging or not. If so, dropping can be performed (block 254 ). If not, pulling can continue. After the blocks 250 and 254 can the operation back to block 244 continue. This process can be reset every time all fingers are lifted off the touch surface (ie, touch is no longer detected) or when the chord is paused for a moment.

In an alternative embodiment of 12 For example, lifting and tapping a left-oriented finger of the five-finger base chord may require clicking the left button, and raising and tapping a right-oriented finger of the base chord may initiate a click of the right button.

13 shows the illustrative gesture operation 260 in accordance with an embodiment of the present invention. The operation 260 can at block 262 begin where a touch is detected. After that, in block 264 the side of the little finger can be recognized. The side of the little finger creates a long (highly eccentric) horizontal pad which is different from all other contacts (except possibly a flattened thumb). Thus, the gesture system may be able to classify horizontally eccentric but thin contacts consistently as a small finger (or possibly a thumb). After that, in block 266 a strike with the side of the little finger are detected.

After that, in block 268 a control or command associated with your stroking little finger.

Wipe with the side of the little finger can be used for simple control gestures, like volume up / down, sleep, screensaver etc. be useful. Painting with the side of the small one Fingers has the simplicity of having a touch to the side of the little finger as she than from the normal 1-fingertip pointing / -clicking is recognizable differently. It is also very ergonomic because while touching with the side of the small one Fingers showing the thumb straight up in the air. This is the most neutral comfortable of all wrist positions.

14 shows an illustrative gesture operation 280 in accordance with an embodiment of the present invention. The operation 280 can at block 282 Start where a thumbnail picture is provided of high-slip after uncovering and sliding down to instrument panel.

dashboard is a control panel that provides customizable gadgets (gadgets) that provides the user with instant information - weather forecasts, Stock prices, yellow pages, airline flights, sports scores etc. In most cases, the dashboard is in View brought when activated, and when disabled the instrument panel is removed from view. The user can provide up-to-date and timely information from the internet with one Key click and then let them disappear immediately, if the button is released.

Expose is a window manager. Window management programs are konfigu to help navigate through or alleviate window clutter (the condition in which it is difficult to find documents or to see the desk because there are so many open windows and / or applications). Expose can have or be selected from three different modes of operation that can be controlled by gesturing. The first mode is All windows or Tile, Scale and Show all. When working in this mode, all open windows are tiled and scaled so that all the open windows can be seen simultaneously within the display screen. That is, performing the specified gesture immediately tilts all open windows - scales them down and arranges them nicely so that the user can view the content in each window. The scaling quantity or scaling rate may be tied to any property of the gesture, such as a mouse. Pressure or speed. The second mode is Application Window or Lift Current Application. This mode works similar to the first mode, except that it only works on a specific application. For example, performing a particular gesture may immediately tile the open windows of a particular application while causing all other open applications to dimming to a gray tone. The third mode is Desk or Hide All. In this mode, all open windows are moved to the edges of the screen, which opens the desktop. That is, passing the particular gesture may hide all open windows, giving the user instant access to his desk.

After that, in block 284 the operations include detecting a base chord (eg, two or more fingers). After that, in block 286 a determination is made as to whether high glide is detected with the base chord or not. If so, the operation may go to block 288 Continue where All Expose is opened. After that, in block 290 a determination is made as to whether or not a slide-down with the base chord is detected. If so, the operation may go to block 292 Continue where All Expose is closed. After that, the operation can go back to block 286 continue. With reference to block 286 For example, if high-slip is not detected by the base chord, the operation may move to block 294 go on, where a determination can be made as to whether or not a downshift with the base chord is detected. If so, the operation may go to block 296 go on, where instrument panel is opened. After that, in block 298 a determination is made as to whether high-slip is detected or not. If so, the operation may go to block 300 continue where the instrument panel is closed. After that, the operation can go back to block 286 continue. Referring back to block 294 if the operation is not detected, the operation may move to block 301 continue where a determination is made as to whether the chord has changed. For example, raising a finger. If the chord has changed, the operation may move to block 302 continue where pointing is performed with the changed chord. After that, in block 304 a determination is made as to whether the chord has been changed back to the base chord or not. If so, the operation can go back to block 286 continue. The gesture operation 280 sometimes called Cancel in opposite direction.

In additional embodiments, the in 14 The operation shown may be modified to include right and left swipe, each associated with a different functionality of exposition.

A static command gesture is a gesture made by simply placing a particular contact arrangement performed on a contact surface becomes. A command is initiated when the particular contact arrangement is identified at the impact. There are no gesture events that follow the initial placement. The command will simply generated when the touch is detected. The static Command gesture may analogously be a sign language where specific Hand arrangements mean different things. In connection with a static command gesture initiates each hand arrangement (or different contact arrangement) a different command.

15 shows the illustrative gesture operation 310 in accordance with an embodiment of the present invention. The gesture operation 310 can at block 312 begin where a touch or a near touch is detected on a touch surface. After block 312 can the operation after block 314 continue where a pad assembly associated with the touch is identified. The pad arrangement may be made by one or two hands, or alternatively or additionally by elements such as a pin or punch. When the pads are created by hand, they can be made through any portion of the hand, including fingers, palms, etc., and further by any orientation of the hand, any position of the fingers relative to each other, and any alignment of fingers. As should be appreciated, various hand orientations and different finger positions / orientations will produce a different pad arrangement. Examples of different orientations include open hand down, of fene hand up, open hand side down (thumb up), closed hand - palm down, closed hand - back of hand, closed hand - knuckle, closed hand - side down. Examples of different finger positions include spread, neutral, grouped, etc. Each contact arrangement can be considered (sign language).

After block 314 can the operation after block 316 continue where an instruction bound to a particular contact arrangement is initiated when the particular contact arrangement is identified. The commands can vary greatly.

16 shows an illustrative gesture operation 320 in accordance with an embodiment of the present invention. The gesture operation 320 can at block 322 begin where a touch is monitored for movement. The touch can z. B. be designed as a special contact arrangement. If there is movement, the gesture operation can continue to block 324 where the movement is distinguished between a click and a swipe. A click can be a quick short move. A swipe can be a slow, slow movement. If the movement is a click, then a first action may be taken when the click is detected (Block 326 ). If the movement is cancel, then a second action may be taken if the strike is detected (block 328 ).

17 shows an illustrative gesture operation 330 in accordance with an embodiment of the present invention. The gesture operation 330 can at block 332 begin where a touch is detected on a touch surface (e.g., touch pad, touch screen, etc.). After block 332 can the operation after block 334 continue where two contacts associated with the touch are identified. After that, the operation in block 336 Monitoring the touch for a gesture event. In block 338 For example, a spin command may be initiated when a first contact is stationary and the second contact is rotated about the first contact.

Many users keep multiple windows open at the same time. This allows them to go back and forth between windows as needed. This can be useful when copying from one window and pasting into another window. In a typical process, the user would select one of the windows, thereby enabling the window, and then performing actions with the window. The user would then select another window, which activates the new window and deactivates the old window and then performs actions with the new window. This is awkward and consists of many shadows. To remedy this, the present invention provides a method 18 which allows the user to toggle between windows without enabling and disabling them. The windows become active when a pointing mechanism is over the window. The pointing mechanism may be a pointer or cursor in the case of a touchpad or a finger in the case of a touchscreen.

18 shows an illustrative gesture operation 340 in accordance with an embodiment of the present invention. The gesture operation 340 includes making several windows available 342 , The windows can z. B. be open programs on a desk surface. After that, the operation in block 344 Detecting detecting a pointing mechanism over the first window. In the case of a touchpad, the pointing mechanism may e.g. B. be a cursor. In the case of a touch screen, the pointing mechanism may be a finger or a finger group. After that, in block 346 the touch is monitored to determine if a chord associated with the pointing mechanism has been presented. In the case of a touchpad, this can be done after the cursor is placed over the window. In the case of a touchscreen, this can be accomplished while the finger is positioned over the window. After that, in block 348 a gesture set associated with the chord and the first window are initiated (eg, the gesture set depends on both the chord and the window). After that, in block 350 Actions in the first window are performed when gesture events associated with the gesture set are performed on the touch surface. After that, in block 352 the operation comprises detecting a pointing mechanism over the second window. This can be one of block 344 be accomplished at different times or it may be simultaneously with block 344 happen. The pointing mechanism may be the same pointing mechanism used for the first window (eg, the pointing mechanism is moved from one window to the other), or alternatively, the pointing mechanism may be a second pointing mechanism (eg, second cursor or second finger). , After that, in block 354 the touch surface may be monitored to determine if a chord associated with the second pointing mechanism has been presented. After that, in block 356 a gesture set associated with the chord and the second window are initiated. After that, in block 358 Actions in the second window are performed when gesture events associated with the gesture set are performed on the touch surface.

In an example of 18 using a touch screen, when a user performs copying and pasting, the user can position his / her fingers from one hand over the first window and select an object to be copied. Thereafter, the user may make a copy gesture over the first window. The user may also position fingers from a second hand over the second window and select a location to insert. Thereafter, the user may perform an insertion gesture over the second window.

In another example of 18 in which a touch screen is used, when the user performs a copy and paste operation, a user can position his / her fingers from one hand over the first window and select an object to be copied. Thereafter, the user may make a copy gesture over the first window. The user can then move his hand to the second window. The user can place their fingers over the second window and select a place to insert. Thereafter, the user may perform an insertion gesture over the second window.

19 shows an illustrative gesture operation 360 in accordance with an embodiment of the present invention. The gesture operation may be at block 364 begin where the contact assembly associated with a first hand is detected. After block 364 can the operation after block 366 continue where a gesture set is loaded based on a contact arrangement associated with the first hand. After block 366 can the operation after block 368 continue where a second contact arrangement associated with a second hand is detected. After block 368 can the operation after block 370 continue where the gesture set of the first contact arrangement is changed or modified on the basis of the second contact arrangement. As should be appreciated, the second hand can be used to quickly modulate between different gesture sets as needed. This may also allow for a wider range of possible gesture sets. If z. For example, if the first hand can provide five gesture sets based on the number of fingers and the second hand can provide five modifiers based on the number of fingers, then 100 gesture sets can be generated.

To give an example of 19 To quote where a touchpad is used, the user can place two fingers on the touchpad and thereby initiate a gesture set associated with the two fingers. The user can perform gesture events using the two fingers. The user may decide that he must perform an action that is not offered by the two-finger gesture set. The user can therefore lower two fingers from the opposite hand. This can load a completely new gesture set to be done by the first hand.

20 shows an illustrative control operation 380 in accordance with an embodiment of the present invention. The control operation can be at block 382 begin where a mouse and a touch surface are provided. For example, a mouse may be connected to a laptop computer that includes a touchpad. After block 382 can the operation after block 384 continue where the mouse is monitored for mouse events (e.g., moving a cursor, key clicking, scrolling). At the same time, in block 386 the touch surface is monitored for gesture events. In block 388 Mouse operations may be performed in accordance with mouse events. At the same time, in block 390 Gesture operations are performed in accordance with gesture events.

This Method allows a user to use a mouse Hand to steer while he is facing the opposite Hand gestures. The user can therefore be more productive.

In an example of 20 For example, in a spreadsheet program, a user may select cells using the mouse, then make a copy gesture with the opposite hand on a touch surface. The user can then scroll or swipe through the table using a scroll or swipe gesture. Once a desired location is found, the user can select the desired cell or cell for insertion using the mouse and then perform an insertion gesture.

21 shows an illustrative control operation 400 in accordance with an embodiment of the present invention. The control operation can be at block 402 begin where a mouse and a touch surface are provided. After block 402 can the operation after block 404 continue where the mouse is monitored for mouse events (eg, moving a cursor, clicking, clicking, scrolling). At the same time, in block 406 the touch surface is monitored for gesture events. After that, in block 408 the mouse mode may be changed based on a touch or gesture events made on the touch surface. For example, the touching hand can be used to control the functionality of the mouse operated with the other hand, to modify. For example, the presence of a finger on the touch surface may cause the mouse to switch from a tracking mode to a scrolling mode.

22 shows an illustrative control operation 410 in accordance with an embodiment of the present invention. The surgery may be at block 412 begin where a first touch on a touch surface is detected. After block 412 can the operation after block 414 continue where the first contact arrangement is detected. This can be z. B. can be achieved with a first hand. At the same time in the blocks 416 and 418 a second touch on the touch surface are detected and a second contact arrangement can be detected. This can be z. B. can be achieved with a second hand. In block 420 For example, the gesture set for the first contact arrangement can be loaded. At the same time, in block 422 the gesture set for the second contact arrangement are loaded. In block 424 For example, the first touch may be monitored for first gesture events. At the same time, in block 426 the second touch is monitored for second gesture events. In block 428 For example, actions associated with the first gesture events may be implemented when the first gesture events are performed. At the same time, in block 430 Actions associated with the second gesture events are implemented when the second gesture events are performed.

23 shows an illustrative control operation 440 in accordance with an embodiment of the present invention. The operation 440 can at block 442 start where a GUI object is displayed. After that, in block 444 several pointers are detected at different points of the displayed GUI object. In the case of a touch screen, the first pointers may be a finger of a first hand and the second pointer may be a finger of a second hand. In the case of a touchpad, the first pointer may be a cursor controlled by a first hand and the second pointer may be a cursor controlled by a second hand. Once the hands are detected, the pointers to the displayed object in block 445 be arrested. This can be z. B. be achieved by stopping. For example, the user may position the pointers at points on the GUI object and then pause there until the hands are locked. After that, in block 446 the position of the pointers are monitored relative to their locked position. After that, in block 448 the displayed object may be modified when the position of one or more pointers is moved relative to its locked position. For example, the attributes of the GUI object, the z. As alignment, size and shape include modified.

To give an example of 23 For example, to rotate an object, the user can position their pointers on the GUI object and rotate one or two pointers when locked. The GUI object may follow the rotation of the one or two pointers, thereby changing its orientation. To resize, the user can position the pointers on the GUI object and push the two pointers together or away from each other when locked. When they are pushed together, the size of the object can be reduced. When they are pushed apart, the size of the object can be increased. To change its shape, the user can position the pointers on the GUI object and move a pointer up, down, right, left when locked. If it is pushed away from the object, the object can be made larger in the sliding direction. If it is pushed towards the object, the object can be made smaller in the sliding direction.

24 shows an illustrative control operation 450 in accordance with an embodiment of the present invention. The surgery may be at block 452 begin where a first touch on a touch surface is detected. This can be z. B. can be achieved by a first hand. After block 452 can the operation after block 454 continue where the first contact arrangement associated with the first touch is detected. At the same time in the blocks 456 and 458 a second touch on the touch surface are detected and a second contact arrangement can be detected. This can be z. B. can be achieved with a second hand. In block 460 For example, the first touch may be monitored for first gesture events. At the same time, in block 462 the second touch is monitored for second gesture events. In block 464 For example, pointing operations associated with the first gesture events may be implemented when the first gesture events are performed. At the same time, in block 466 Pointing operations associated with the second gesture events are implemented when the second gesture events are performed.

25 shows an illustrative control operation 470 in accordance with an embodiment of the present invention. The control operation can be at block 472 begin where a group of selectable items is displayed. The group of selectable items may, for. B. a color palette or a toolbar. After block 472 can the operation after block 474 continue where a first pointing mechanism detek over the group of selectable objects is being done. In the case of a touch screen, the pointing mechanism may be a finger or a finger group. In the case of a touchpad, the pointing mechanism may be a cursor or a tool controlled by a finger or a finger group. In block 476 The item can be activated (selected) under the first pointing mechanism. The activated item may be based on the position of the pointing mechanism. A visual feedback may be provided so that the user receives a confirmation of his selection. For example, the activated item may be highlighted when the first pointing mechanism is positioned over the item. In block 478 For example, a second pointing mechanism may be simultaneously detected over a window (eg, both pointing mechanisms operate simultaneously). The window can z. B. be a canvas in a drawing or painting program. In the case of a touch screen, the second pointing mechanism may be a finger or a finger group (from the opposite hand as the first pointing mechanism). In the case of a touchpad, the second pointing mechanism may be a cursor or tool controlled by a finger or a finger group. In block 480 the selected item can be applied to the second pointing mechanism. For example, in the case of a toolbar, the tool functionality of the second pointing mechanism may be based on the selected item. Furthermore, in the case of a touchpad, the tool UI element may change with functionality so that the user knows which tool is active (eg, pencil, brush, etc.). In the case of a palette, the output of the second pointing mechanism may be based on the selected item. If z. For example, if, in the case of painting, the user selects a color on the palette, the output of the second pointer changes to that color. The user can therefore adjust the color applied to the canvas immediately during a painting operation.

26 shows an illustrative control operation 500 in accordance with an embodiment of the present invention. The control operation can be at block 502 begin when a group of media items is displayed. The media items can z. As songs, video clips, movies, photos, games and the like. The media items can z. B. be controlled by a media management program. iPhoto and iTunes, manufactured by Apple Inc. of Cupertino, are examples of media management programs. In block 504 For example, a pointing mechanism may be detected over an object and thereby highlight the object. The pointing mechanism can, for. B. be controlled by a mouse or a hand chord (dominant hand). In block 506 For example, a touch on a touch surface may be detected. The touch may be performed by the hand opposite the hand associated with the pointing mechanism. After that, in block 508 the contact arrangement associated with the contact can be recognized. After that, in block 510 Metadata bound to the recognized contact arrangement are inserted into the highlighted object. Each of the contact arrangements may have different metadata. For example, a single finger may have first metadata, two fingers may have second metadata, three fingers may have third metadata, four fingers may have fourth metadata, five fingers may have fifth metadata, and so on. The metadata for each contact arrangement may have the same functionality or they can have completely different functionalities.

To give an example of 26 To cite, the above process can be used to rate or classify media items. For example, a single finger can bind a 1-star cast to a song, two fingers can bind a 2-star cast to a song, three fingers can bind a 3-star cast to a song, four fingers can bind a 4-star cast to a song and five fingers can tie a 5 star arrangement to a song.

For another example of 26 To cite, the above process can be used to identify media objects. For example, a single finger may bind a first person (eg, wife Darcy) to a photo, a second finger may bind a second person (eg, daughter Halley) to a photo, a third finger may be a third person ( eg cousin Camden) to a photo, a fourth finger can bind a fourth person (Uncle Chance) to a photo and a fifth finger can bind a fifth person (eg grandmother Shery) to a photo. Tap can be used to enlarge the list. For example, the number of fingers at the first settling of the number of fingers at the second settling may be added to produce a final contact arrangement. Three fingers followed by two fingers can still mean the fifth person. While five fingers and one finger (6 fingers in total) can bind a sixth person (eg grandmother Terry) to a photo. Furthermore, five fingers and two fingers can bind a seventh person (eg, dog Prima) to a photo, five fingers and three fingers can bind an eighth person (friend Lisa) to a photo, five fingers and four fingers can hold a ninth person bind to a photo and five fingers and five fingers can bind a tenth person to a photo. Alternatively, the metadata may not be associated with a person but with another property of photos (eg vacation, California, landscape, family, etc.).

in the Generally, it may require the media management program to work in to be brought into a metadata insertion mode. This can be done conventionally by controlling on the screen or alternatively be achieved by the chord associated with the pointing mechanism connected is.

27 shows an illustrative control operation 520 in accordance with an embodiment of the present invention. The control operation can be at block 522 begin where a control object is displayed. The control object can, for. As a button, dial, slide, switch, etc. on the screen. In block 524 a touch above the control object can be detected. After that, in block 526 the contact arrangement associated with the contact can be recognized. In the block 527 a determination may be made as to whether or not the contact arrangement agrees with the contact arrangement associated with control of the control object. If it does not match, the operation may block 528 go on where the touch is ignored. If it matches, the operation may block 530 continue where the touch is monitored for a gesture event associated with controlling the control object. When a gesture event is performed, the action associated therewith can be applied to the control object so that the control object can generate its commands (Block 432 ).

In one embodiment of 27 For example, a contact arrangement that initiates control of the control object may be based on multiple contacts, while the contact arrangement that is ignored may be based on a single contact.

28 shows an illustrative control operation 540 in accordance with an embodiment of the present invention. The control operation 540 can be done using a control field like the one in 29 be performed shown. The surgery may be at block 542 begin where a list of input functions is provided. The input functions may have commands and gesture events associated with the commands. The commands may refer to the input functions. If the input function z. For example, if you are editing, the commands can be Cancel, Undo, Paste, Backward Tab, Copy, Tab, Cut, Restore, and Select All. The gesture events may be actions performed by an accord. The gesture events may include rotations (clockwise and counterclockwise), shifts as a group (up, down, right, left), shifts relative to each other (spread, closed), jogging, etc. The gesture events may be linked to commands in an intuitive manner become. For example, the clip command may be initiated by closing the fingers to each other similar to a clip action. Furthermore, complementary instructions (cut, paste) are typically associated with complementary gesture events (spreading / closing). In block 544 The input function can be assigned to a specific chord. A chord can be a finger arrangement. The chords can be displayed in a list so that the user highlights a particular chord and then selects a particular input function. Chords can be assigned to multiple input functions.

In block 544 For example, the user may assign input functions to a particular chord by selecting an input function from a menu of input functions and then selecting a chord from a menu list of chords (eg, highlight input function and then select Chord). Once the chord is selected, the input function to that chord can be bound for future operations (Block 546 ). In some cases, the shortcut menu may include an indication of the input function associated with it. In this particular embodiment, commands may already be associated with gesture events.

The 29A to G are diagrams of a control panel 550 in accordance with an embodiment of the present invention. 29A shows a basic control panel while 29B to 29G show several examples of a control panel embodiment.

As in 29A The control panel can be shown 550 a gesture mapping control 552 in the form of a pull-down window. When a user selects the pull-down window, one or more gesture-mapping options may be presented to the user. The user may be presented with a default setting (normally set by the manufacturer) or, alternatively, the user may select a user setting so that he can make his own gesture mapping. There may be more than one user picture. For example, different users can create different pictures. When a user opens the control panel, he can select the desired user image by selecting gesture gesture control and then moving through the list of user images via a highlight bar. Once a desired image is found, the user can simply highlight it and then make a first click to set the desired gesture image. After that, the user can change their settings.

The control panel 550 can also have a menu of input functions 554 also in the form of a window for pulling down. When a user selects the pull down window, the user can be presented with a list of one or more input functions. The input functions can, for. No, show, show / click, scroll, scroll and secondary click, primary click and drag, secondary click and drag, move window, uncover and dashboard, web browser operations, edit operations, file operations, and the like. To highlight one of the input functions, the user can be provided with a highlight bar that runs through the list in accordance with the user controls.

control field 550 can also have a gesture set section 556 comprising commands associated with the input function that is highlighted and gesture events associated with the commands. The gesture set portion may only serve feedback, but may also include the functionality of associating commands with gesture events (or vice versa). Although the window may be greatly modified, in the illustrated embodiment, the window may contain separate fields, each containing a gesture event and a command associated therewith. In this embodiment, the fields may be laid out in a 3x3 matrix and therefore there may be 9 different gesture events and 9 different commands for each input function. In most cases, opposite gesture events may be placed opposite each other. For example, counterclockwise rotation may be counterclockwise rotation, spreading may be arranged against contraction, left-side brushing may be opposite to kick-offs, swipe-up may be facing down swipe. If possible, complementary commands can be linked to opposite gesture events, making them more intuitive.

control field 550 can also be a chord menu 558 include. The chords can be stacked and move through a window using a scroll bar. The chords may contain text identifiers as well as chord identifiers that show examples of how to make the chord. In the illustrated embodiment, the chord indicator may be in the shape of a hand, with darkened circles covering the fingers used to make the chord. The chords may also have text identifiers for the input function associated with the chord.

In In one embodiment, the user may view the gesture image to set up, highlight an input function and then move to the chord menu. When the chord menu arrived, the user can choose a desired Select chord for this input function by he goes through the chord list, the desired one Chord highlights and then a selection event such. B. over make a click. After that, the input function be assigned to the chord. In some cases, as soon as the user assigns the input function to the chord, the input function identifier in the chord menu together with the chord. Furthermore, the input function may contain an identifier, indicating that it has been assigned. For example, there may be a catch be placed by the assigned input function. While Operations can occur when a user places the finger assembly in the Chord menu is shown on the multipoint touch surface placed, the finger arrangement can be recognized as the chord, and after that, the gesture set associated with the input function is assigned to the chord as the primary or active gesture set.

29B FIG. 15 shows an example in which the input function menu can be opened and the user can allocate the input-function-instrument panel to associate with the 4-finger chord. Furthermore, the gesture set may show the commands associated with the exposure / instrument panel input function and the gesture events used to initiate the commands. For example, swiping up can initiate all uncovering, swiping right can initiate desktop uncovering, swiping left can initiate applications uncovering, and swipe down can initiate instrument panel.

29C shows an example in which the input function menu can be opened and the user can assign the input function primary click and drag to the 3-finger chord. Furthermore, the gesture set may show the commands associated with the click and drag input functions and the gesture events used to initiate the commands. For example, directional moves in each direction can be used to draw an object, and jogging can be used to initiate a click.

29D shows an example in which the input function menu can be opened and the user can assign the input function file operation to the chord thumb + 1 finger. Furthermore, the gesture set may show the commands associated with the file operation input function and the gesture events used to initiate the commands. For example, a counterclockwise rotation can be used can be used to initiate close, a clockwise spin can be used to initiate close, spreading can be used for new, and contracting can be used for save.

29E shows an example in which the input function menu can be opened and the user can assign the input function editing operations the chord thumb + 2 fingers. Furthermore, the gesture set may show the commands associated with the edit operation input function and the gesture events used to initiate the commands. For example, a swipe may initiate undo, a swipe to the right may initiate a tab, a swipe to the left may initiate backward tab applications, and a swipe down may initiate restoring. Furthermore, a counter-clockwise rotation swipe may initiate abort, a clockwise rotation swipe may initiate all-select, spreading may initiate insertion, and contraction may initiate cut-off. Furthermore, typing can initiate copying.

29F shows an example in which the input function menu can be opened and the user can assign the web browser operation input function to the finger + 3 finger accord. Furthermore, the gesture set may show the commands associated with the web browser operation input function and the gesture events used to initiate the commands. For example, a swipe to the top may initiate pause, a swipe to the right may precede, a swipe to the left may initiate back, a swipe down may initiate reload. Furthermore, swiping to the top left may initiate startup, swiping to the top right can initiate highlighting, swiping to the bottom left can initiate searches, and swiping to the bottom right can initiate bookmarks.

29G shows an example in which the input function menu can be opened and the user can assign the pointing input function to the chord thumb + 4 fingers. Furthermore, the gesture set may show the commands associated with the pointing input function and the gesture events used to initiate the commands. For example, directional movements in any direction may be used for pointing.

30 shows an illustrative control operation 580 in accordance with an embodiment of the present invention. The control operation 580 can be done using a control panel, like the one in 29 be performed shown. The surgery can be in block 582 begin where a list of input functions can be provided. In the block 584 a list of commands associated with the input functions may be provided. In block 586 a chord list can be made available. In block 588 may be provided a list of gesture events associated with the chords. In the block 590 the input function can be assigned to a chord (or vice versa). In the block 592 The commands can be assigned to the gesture events. In an alternative operation, the operation may be 580 include providing a list of all available commands instead of having a list associated with a particular input function. The user can assign any command to the input function. Also in an alternative operation, the operation may include providing a list of all available gesture events rather than having a list associated with a particular chord. The user can assign any gesture event to the chord.

With reference to 29 For example, the user may assign input functions to a particular chord by selecting an input function from an input function menu and then selecting a chord from a chord menu list. (for example, highlight Input and then select Chord). Once the chord is selected, the input function to that chord can be bound for future operations. In some cases, the chord menu may include an indication of the input function associated therewith. Furthermore, the user can assign commands to gesture events by dragging the commands over a gesture. For example, the user may select and drag a particular command and drop it over the desired gesture event. Thereafter, the gesture event may be bound for future operations.

31 shows an illustrative gesture mapping method 600 in accordance with an embodiment of the present invention. The picture 600 can at block 602 begin where the system is placed in a gesture mapping mode. Once it is in gesture imaging mode, a touch can be in block 604 be detected. After that, in block 606 the arrangement of the contacts associated with the touch are stored. For example, an image of the contact arrangement may be generated. After that, in block 608 the touch is monitored for a gesture event. After that, in block 610 the gesture event will be saved. After that, in block 612 be assigned a command of the contact arrangement / the gesture event. Once it is out of the gesture mapping mode, the command can be initiated by moving the contact assembly off is set that it can be recognized and by the gesture event is performed.

32 shows an illustrative gesture mapping method 620 in accordance with an embodiment of the present invention. The gesture imaging method 620 can at block 622 begin where a command menu can be opened. The command menu may be any of the menus found in the header of an application (eg, file, edit, view, insert, format, tools, etc.). When the command menu is opened, a list of commands can be displayed (Block 624 ). After block 624 For example, a highlight bar may be moved via a command in the command list via a user control. After that, in block 626 a touch on a touch surface are detected while a command is highlighted. After that, in block 628 the contact arrangement associated with the touch is mapped and stored. After that, in block 630 the touch is monitored for a gesture event. After that, in block 632 a gesture event will be mapped and saved when it is performed. After that, in block 634 the initial contact arrangement and the gesture events are associated with the highlighted command. Once out of the gesture mapping mode, the command may be initiated by placing the contact assembly so that it can be recognized and by performing the gesture event.

33 shows an illustrative gesture feedback method 640 in accordance with an embodiment of the present invention. The procedure 640 can at block 642 begin where a touch is detected. After that, in block 644 the contact arrangement can be detected. After that, in block 646 based on the gesture set based at least partially on the contact arrangement. After that, in block 648 a pause is detected during the touch (eg, no gesture event is performed during a period of time). After that, in block 650 the gesture set can be displayed so that the user can view commands and gesture events that are bound to him (see eg 34 ). After that, in block 652 the gesture set will be removed from the display when a gesture event is started or after a preset timeout.

34 shows an illustrative gesture viewing window 660 in accordance with an embodiment of the present invention. A gesture view window may be a gesture menu that opens and contains information about how to perform a gesture event and what command will be initiated when the gesture event is performed. The user can therefore quickly memorize the gesture set before performing a gesture event. The gesture window 660 may be a semitransparent or opaque window displayed near the first touch point (or pause) and over any other displayed information. Although the window may vary widely, in the illustrated embodiment, the window may 660 separate fields 662 each of which contains a gesture event 664 and a command 667 contains, which is associated with him. In this embodiment, the fields may be laid out in a 3x3 matrix and therefore there may be 9 different gesture events and 9 different commands. In most cases, opposite gesture events may be placed opposite each other. For example, counterclockwise rotations may be clockwise relative to rotations, struts may be opposite to contraction, left strokes may be opposite to strokes, swipe upward may be down against swipes. If possible, complementary commands can be linked to opposite gesture events, making them more intuitive. In the case of cut and paste, for. For example, clipping may be associated with contraction, while insertion may be associated with clipping. Furthermore, another can be associated with a tap (represented by a circle).

35 shows an illustrative keyboard method 680 implemented via a touch screen in accordance with an embodiment of the present invention. The procedure 680 can at block 682 begin where a touch screen keyboard is activated (eg, displayed and ready to receive input). After that, in block 684 the touch surface is monitored for key inputs (eg, a finger tapping a displayed key). At the same time, in block 686 the touch surface is monitored for stroking gesture events. If the stroking gesture event is a legal strike, a space command may be performed (blocks 688 and 690 ). If the stroking gesture event is a left stroke, a backspace / delete command may be performed (blocks 692 and 694 ). If the stroking gesture event is swipe down, an enter / rewind command may be performed (blocks 696 and 698 ). If the stroking gesture event is swipe-up, a page break command may be performed (blocks 700 and 702 ). It should be noted, however, that the method is not limited to these instructions and that other instructions may be performed.

With so many chord combinations, gesture events, commands for evaluation It may be helpful to gain strength and weakness too isolate and develop a guidance theory which chords and gesture events go well with which commands. It is in the Generally accepted that long-term user experience through assignment the best finger combinations with the most common commands can be maximized. Mediocre finger combinations should only be used for less common commands become. Because of subtle issues like overlaying However, it is not enough to follow this principle alone to create a great user experience.

36 shows an illustrative gesture design process 720 in accordance with an embodiment of the present invention. The design process 720 includes block 702 where the simplest chords are rated or listed. The chords can be z. Based on speed, reliability, comfort and ease of use, simplicity, etc. This may also be due to a size of the touch surface, as smaller surfaces may not accommodate all finger combinations. 37 shows an example of a matrix for evaluating chords. It should be appreciated that this matrix is exemplary and not limiting.

The design process also includes block 704 where the frequency of operations is assessed or arranged. For example, how many times a particular user or group of users performs a particular set of operations. Frequency orders may vary according to the proportions of operations relative to one another (eg, web surfing versus editing). 38 shows an example of a matrix for classifying operation frequency for a few operations. It should be known that this list is only an example and not a delineation since there are many more operations.

The design process can also block 706 where the simplest chords are essentially grouped or paired with the most common operations, and the most difficult chords are essentially grouped or mated with the rarest operations.

Of the Design process may also include fine tuning of groups. The Fine tuning can be based on many factors, including complementary ones Commands, Intuitivity, Overlay, Meaningful Arrangement, etc. As should be appreciated, the design process may not be as simple as merging the simplest Chords with the most common surgery. In some cases can be the simplest chord for the most common surgery Does not make any sense. As such, the chords and operations be moved while still trying, just with most common and difficult with least often maintain.

39 shows an illustrative control operation 740 in accordance with an embodiment of the present invention. The control operation can be at block 742 start where a basic gesture mapping is provided. The base gesture map may include gesture preference settings, such as Chords, commands, gesture events, etc. After block 742 can the operation after block 744 continue where a first program is activated. The program can z. A word processor, a media manager, a spreadsheet, a media editor, a drawing program, etc. In block 746 For example, the base gesture map may be adjusted in accordance with the first program while the first program is active. For example, the base gesture map program may order to change the meanings of chords and gesture sets for that particular program. In the block 748 a second program can be activated. In block 750 For example, the base gesture map may be adjusted in accordance with the second program while the second program is active.

In an example 39 For example, a media editing program can customize gesture mapping as follows: a finger for pointing, two fingers for moving, scaling, rotating objects; three fingers for primary pulling; four fingers for a scrolling pan; 5 fingers to point at, while a word processor can adjust gesture mapping as follows: 2 fingers to point with text cursor, 3 fingers to select text, 4 fingers to scroll, start of line and end of line.

40 shows an illustrative control operation 760 in accordance with an embodiment of the present invention. The control operation may be associated with a gesture set, such as that implemented with a three finger chord. The control operation can be at block 762 begin where a determination is made as to whether a swipe to the right is detected or not. If so, the operation may go to block 764 progress where a first exemption function is initiated. If not, the operation may be after block 766 go on, where a determination is made as to whether a swipe to the left is detected or not. If so, the operation may block 768 where a second exemption function is initiated. If not, the operation may be after block 770 go on where a determination is made as to whether or not a swipe is detected up. If so, the operation may block 772 go on where a third freistel is initiated. If not, the operation may be after block 774 go on where a determination is made as to whether or not a swipe down is detected. If so, the operation may block 776 continue where an instrument panel function is initiated. If not, the operation can go back to 762 continue.

41 shows an illustrative control operation 780 in accordance with an embodiment of the present invention. The control operation can be at block 782 start where a base chord is detected. The base chord can z. B. be a three finger chord. During the touch stop a pause is detected (block 784 ). After that, in block 786 a context menu will be opened because of the pause of the base chord. After that, in block 788 a determination is made as to whether or not slip is detected up or down. If so, the operation may block 790 continue where a highlight bar is moved through the context menu in accordance with the sliding movement. After that, in block 792 a determination is made as to whether a jog is detected or not. If so, the operation may block 794 continue where the functionality of the highlighted entry is implemented.

dashboard houses widgets: gadgets for performing common tasks can and which provides quick access to information put. With a swipe can dashboard appear, complete with widgets that give a user a world of Bring information - real-time weather, stock tickers, flight information and further - immediately. Instrument panel can be just as easy disappear so that a user can return to what he just did. Dashboard is manufactured by Apple Inc. of Cupertino, California, produced.

release Allows users to instantly access any open window with a single swipe - and amazing style access that can never be imitated. Show all open windows as thumbnails view windows of the current application or hide them all Window to quickly locate a file on your desktop. Crops is manufactured by Apple Inc. of Cupertino, California.

The 42 to 44 show various gesture illustrations in accordance with various embodiments of the present invention. Each figure comprises four headings: 1) hand, 2) # on fingers (chord), 3) gesture event, and 4) behavior or action associated with the gesture event.

The 45 to 48 show various gesture sets in accordance with several embodiments of the present invention. In all of these figures, the gesture sets include various movements shown by arrows (wiping, rotating, spreading / contracting) and one or more taps indicated by circles. They also include commands that are tied to the movements they are lying over. The gesture sets can be applied to different chords depending on user needs. In one example, the gesture set may be off 45 to be applied to a chord of a thumb + 1 finger, the gesture set out 46 can be applied to a chord of a thumb + 2 fingers, the gesture set out 47 can be applied to a chord of a thumb + 3 fingers and the gesture set 48 can be applied from a chord of splayed thumb and 3 fingers. It should be understood that these gesture sets as well as the chords are given as an example and not for demarcation. For example, in the case of gesture sets, different commands may be used, different gesture events may be used, different mappings may be made, and so on.

Additional embodiments, referring to the 8th to 11 will now be described.

49 shows an illustrative gesture operation 800 in accordance with an embodiment of the present invention. The operation 800 can at block 802 begin where only a first finger is detected. After that, in block 804 the condition of the first finger can be determined. The state can z. B. a specific orientation of the finger, the position of the finger relative to the contact surface, whether the finger is moving or stationary, the direction of the finger movements and the like. After that, in block 806 one or more additional fingers are detected. For example, a different second finger may be detected besides the first finger. In block 808 the state of the one or more additional fingers can be determined. After that, in block 810 various input modes are implemented based on a time history of the states of the first and the additional fingers relative to each other. The various input modes may be greatly varied and may be selected from any of those discussed herein or any suitable or known input mode. An example of an input mode may be a pointing mode or a drag mode. Other modes can z. As scrolling, panning or the like include.

In In one example, the state of the first finger may be stationary or moving, and the state of a second finger can include whether the second finger is present or not. In this Example, pointing operations can be implemented if only the first finger is detected. Furthermore you can Show operations are switched to drawing operations, if the second finger is detected when the first finger is stationary, and pointing operations are maintained when the second finger is detected when the first finger moves. This can a user can easily switch between pointing-drag operations. The user can z. B. a finger over the touch surface move to him or a cursor over a file folder to position (eg pointing), the finger or cursor over stop the file folder, a second finger on the touch surface settle so that it appears that you want the To drag file folder, and then his first finger to another position move to drag the file folder to a new location. Of the User can continue his fingers and thus the file folder to move around, or simply raise the first finger to the File folder finally within the screen to arrange.

50 shows an illustrative gesture operation 820 in accordance with an embodiment of the present invention. The operation 820 can at block 822 begin where only a first finger is detected. After that, in block 824 the condition of the first finger can be determined. The state can z. As a specific orientation of the finger, the position of the finger relative to the contact surface, whether the finger is moving or stationary, the direction of the finger movements or the like correspond. After that, in block 826 a first input mode may be implemented with the first finger when the first finger is in a first state. In block 828 can be looked for a modifier finger event. The modifier finger event may be provided by a finger other than the first finger. It can, for. B. be a second finger. The event may be the presence of a second finger or it may be some type of gesturing event or chord. After that, in block 830 a second may be implemented when the first finger is in a second state and the modifier finger event is detected. The modifying event may be e.g. B. be the presence of a second finger next to the first finger. In one example, the first mode is a pointing mode and the second mode is a drag mode. Of course, the embodiment may be applied to other modes as needed.

51 shows an illustrative gesture operation 840 in accordance with an embodiment of the present invention. The operation 840 can at block 842 begin where a first finger is initially detected. After that, in block 844 a first mode associated with the first finger may be implemented when the first finger is moved. After that, in block 826 a second finger can be detected after the first finger is detected. In block 828 For example, the first mode may be switched to a second mode when the first finger is stationary and the presence of a second finger is detected. The presence of the second finger may be position dependent, such. From a certain position on the touch surface or a certain position relative to the first finger. In one example, the second finger near the first finger should be detected. After that, in block 830 the second mode may be implemented with at least the first finger when the first finger is moved. In one example, the first mode may be a pointing mode and the second mode may be a drag mode. Of course, the embodiment may be applied to other modes as needed.

52 shows an illustrative gesture operation 860 in accordance with an embodiment of the present invention. The operation 860 can at block 862 begin where a determination is made as to whether a single finger is detected. The single finger can be identified or not, depending on the needs of the system. In some cases, the operations z. For example, only if a particular finger is identified, while in other cases the method can be implemented if any single finger is detected. If a single finger is detected, the operation may go to block 864 go on where a determination is made whether the finger is moved or not. When the finger is moved, in block 866 Pointing operations are performed with the first finger movement. If the first finger is no longer detected, the pointing operations may end. If the finger is not moved, in block 868 a determination is made as to whether a second finger is detected while the first finger is stationary. If not, the operation can go back to block 862 continue. If so, the operation may go to block 870 go on, where a determination is made as to whether the first finger is moved after the second finger is detected. The second finger may or may not move with the first finger. In most cases, the second finger can be lifted so that it is no longer detected. If the first finger is not moved, the operation may go back to block 868 continue. When the first finger is moved, the operation may go to block 872 proceed where pulling operations are performed in accordance with the first finger movement. If the first finger is no longer detected, the drag operations may end.

53 shows an illustrative gesture operation 880 in accordance with an embodiment of the present invention. The operation 880 can at block 882 begin where a determination is made whether a first finger is detected or not. The first finger can be identified or not, depending on the needs of the system. In some cases, the operation z. For example, only if a particular finger is identified, while in other cases the operation can be implemented if any single finger is detected. If a finger is detected, the operation may go to block 884 continue where a pointing mode is implemented. After that, in block 886 a determination is made as to whether the finger is moved or not. When the finger is moved, in block 888 subsequent modification events are ignored. For example, additional detected fingers may be ignored and only the movement of the first finger may be relevant. After that, in block 890 Showing operations are carried out in accordance with the moving finger. If the first finger is no longer detected, the pointing operations may end. If the first finger is stationary, the operation may go to block 892 go on where to look for a modify event. A modification event may e.g. For example, the addition of a finger, or a gesture made by a new finger, and so on 894 a determination is made as to whether or not a particular modification event is detected. If not, the operation can go back to block 886 continue. If so, the operation may go to block 896 continue where the system is switched from the pointing mode to a pulling mode. When the first finger is moved, the operation may go to block 898 proceed where pulling operations are performed in accordance with the first finger movement. If the first finger is no longer detected, the drag operations may end.

54 shows an illustrative electronic system 950 in accordance with an embodiment of the present invention. The electronic system may be configured with any of the foregoing embodiments. The electronic system 950 may correspond to a computer system, such as a personal computer system that includes desktops, laptops, tablets, or handheld computers. The electronic system may also correspond to other electronic user devices, such. Hand-held devices, including cell phones, PDAs, special media players, and the like. Of course, the electronic system can be applied to other electronic devices.

The exemplary system 950 , this in 54 shown can be a processor 956 that is configured to execute instructions and perform operations associated with the system 950 are associated. For example, the processor 956 using instructions that z. B. are received from the memory, the reception and manipulation of input and output data between components of the computer system 950 Taxes. The processor 956 can be implemented on a single chip, multiple chips or multiple electronic components. For example, different architectures can be used for the processor 956 including a dedicated or embedded processor, single-task processor, controller, ASIC, etc.

In most cases, the processor can 956 working together with an operating system to execute computer code and produce usage data. Operating systems are generally well known and will not be described in greater detail. The operating system can z. OS / 2, DOS, Unix, Linux, Palm OS, and the like. The operating system can also be an operating system with a special task, such as z. B. can be used for household computer devices with limited purpose. The operating system, other computer code and data may be within a memory block 958 which is operational with the processor 956 connected is. memory block 958 can provide a place to store computer code and data through the system 950 be used. For example, the memory block 958 Read-Only Memory (ROM), Random Access Memory (RAM), Hard Disk Drive, and / or the like. The information may be on a removable storage medium or on the system 950 loaded or installed if needed. Removable storage media include, for. CD-ROM, PC-CARD, memory card, floppy disk, magnetic tape and a network component.

The system 950 can also be a display device 968 include, which is operational with the processor 956 connected is. The display device 968 may be a liquid crystal display (LCD) (eg active matrix, passive matrix and the like). Alternatively, the display device 968 a monitor, such as a monochrome display, color graphics adapter (CGA) display, enhanced graphics adapter (EGA) display, variable-graphics-array (VGA), super VGA display, cathode ray tube (cathode ray tube, CRT) and the like. The display device may also correspond to a plasma display or display implemented with electronic inks.

The display device 968 can be configured to have a graphical user interface (GUI) 969 which provides an easy-to-use interface between the user of the system and the operating system or application running on it. In general, the GUI represents 969 Programs, files and operating options with graphic images. The graphical images may include windows, boxes, dialog boxes, menus, icons, switches, cursors, scroll bars, and so forth. Such images may be arranged in predefined layouts or they may be generated dynamically to serve particular actions taken by a user. During the operation, the user may select and activate various graphical images to initiate functions and tasks associated therewith. For example, a user may select a button that opens, closes, minimizes or maximizes a window, or an icon that launches a particular program. The GUI 969 In addition, or alternatively, information such as non-interactive text and graphics may be displayed to the user on the display device 968 Show.

The system 950 can also have one or more input devices 970 include, which is operational with the processor 956 are connected. The input devices 970 can be configured to transfer data from the outside world to the system 950 transferred to. The input devices 970 can in the system 950 be integrated or they may be separate components (eg peripheral devices). In some cases, the input devices 970 with the system 950 be connected by wire connections (cables / ports). In other cases, the input devices 970 with the system 950 be connected by wireless connections. The data connection can z. PS / 2, USB, IR, RF, Bluetooth or similar.

The input device (s) 970 can / can z. B. can be used to perform tracking and GUI choices 969 on the display 968 close. The input device 970 can also be used to place commands in the system 950 leave.

In one embodiment, the input devices 970 comprise a touch-sensitive device configured to receive input from a touch of the user and send that information to the processor 956 to send. The touch-sensitive device may, for. As a touch panel, a touch screen and / or a touch-sensitive housing.

In many cases, the touch-sensitive device may detect touches (or near-touches) as well as the position and magnitude of the touches on the touch-sensitive surface. The touch sensing means may contact the processor 956 report and the processor 956 can interpret the touches in accordance with its programming. For example, the processor 956 initiate a task in accordance with a specific touch. A dedicated processor can be used to process the touches locally and reduce the demand for the main processor of the computer system. The touch-sensitive device may be based on scanning technologies including, but not limited to, capacitive scanning, resistive scanning, acoustic wave scanning surfaces, pressure sensing, optical scanning, and / or the like.

Farther Can the touch-sensitive agents on Single point or multipoint sampling based. Single point sampling is only able to distinguish a single touch while multipoint scanning is capable of multiple touches to distinguish that occur at the same time. Alternatively or In addition, proximity sensing devices may be used be used. Sewing scanning devices are capable of Objects in the room instead of on a touch surface scan.

In one embodiment, the input devices 970 comprise a multi-point input device. Multipoint input devices have advantages over conventional single-point devices in that they can simultaneously distinguish more than one object (finger). As an example, a multipoint device used herein is described in more detail in US Patent Application Nos. 10 / 840,862 and 11 / 015,434, both of which are incorporated herein by reference.

In accordance with an embodiment of the present invention, the system 950 designed to be gestures 985 detects that on one or more input devices 970 (In particular, touch devices) are applied and aspects of the system 950 based on the gestures 985 controls. As previously noted, a gesture may be defined as a stylized interaction with an input device that maps to one or more particular computer operations. The gestures 985 can be made by various hand and especially finger movements. Alternatively or additionally, the gestures can be made with a pen. In all these cases, at least one of the input devices 970 such as a touch device, the gestures 985 received and the processor 956 can execute instructions and perform operations with the gestures 985 are associated. In addition, the memory block 958 a gesture operating program 988 which may be part of an operating system or a separate application. The gesture operating program 988 can contain a set of instructions that indicate the appearance of gestures 985 recognize and one or more software agents on the gestures 985 inform and / or what action (s) in response to the gestures 985 to do.

When a user performs one or more gestures, the input device may 970 Gesture information to the processor 956 pass on. Using instructions from the store 958 and in particular the gesture operating program 988 can the processor 956 the gestures 985 interpret and different components of the system 950 such as the memory 958 , an ad 968 based on the gestures 985 Taxes. The gestures 985 can be used as commands to perform actions in applications that are in memory 958 are stored, modify the GUI objects on the display 968 to modify data stored in memory 958 are stored and / or perform actions in I / O devices. For example, the commands may be associated with zooming, panning, scrolling, scrolling, rotating, resizing, and the like. As further examples, the commands may also allow for starting a particular program, opening a file or a particular document, viewing a menu, making a selection, executing instructions, logging into a computer system, authorized individual access to restricted areas of the computer system a user profile associated with and / or associated with a user-preferred arrangement on the computer desk.

In In one embodiment, the number of fingers may be in contact display an input mode with the input device. For example can a single touch, such. B. by a single finger indicate the desire to conduct follow-up, d. H. Pointer or cursor movements or selections, whereas several Touches such. By a group of fingers, the desire can indicate to perform other gestures. The number on fingers to implement gestures can be greatly modified become. For example, two fingers may have a first input mode three fingers can enter a third input mode Alternatively, any number of fingers, i. H. more are used as one for the same input mode, which may include one or more gesture controls. The alignment The finger can be used similarly to the one you want Mode. The profile of the fingers can be detected to enable various modal operations based on this whether the user z. B. has used his thumb or index finger.

In In a particular embodiment, a single finger may be used Initiate tracking (or selection) and two or more fingers In close proximity to each other can scroll or panning. Two fingers can be simple back and forth switching between one and two fingers deliver, d. H. the user can switch between modes very easily by He simply raises or places an additional finger. This has the advantage of being more intuitive than other forms of the mode switching back and forth. While tracking can be cursor movement be controlled by the user, a single finger on the touch-sensitive surface of a touch-sensitive device moves. The sensor arrangement The touch-sensitive device may be finger-moving interpret and signals to generate appropriate motion of the cursor on the display. While scrolling Screen motion can be controlled by the user who dual fingers on the touch-sensitive surface the touch-sensitive device moves. If the combined fingers can be moved in the vertical direction the motion is interpreted as a vertical scrolling event and when the combined fingers are moved in the horizontal direction, can interpret the movement as a horizontal scrolling event become. The same can be said for panning, though Pan in all directions instead of just horizontal and vertical Direction can occur.

The Embodiments described herein may be through hardware, software or a combination of hardware and Software to be implemented. The software can also be called computer readable Code be executed on a computer-readable medium. The computer-readable medium is any data storage device, can store the data subsequently read by a computer system can be. Examples of the computer readable Medium include read-only memory, read-write memory, CD-ROMs, DVDs, magnetic tape, optical data storage devices and carrier waves. The computer-readable medium can also be network-connected Computer systems are distributed so that the computer-readable code is up distributed manner is stored and executed.

While this invention with respect to several embodiments there are changes, permutations and equivalents, which fall within the scope of the invention. Although z. B. the invention is primarily directed to touch devices, should be noted that the invention also with Näheabtastungsvorrichtungen can be implemented. In addition, you can Procedures are implemented to automatically gesture preferences to synchronize between different devices. If z. B. a Palm device is connected to a host computer. It should also be noted that there are many alternative ways of Implementation of the methods and apparatus of the present Invention exists. Although z. For example, the invention is primarily directed to methods which are associated with multi-point scanners, It should be noted that some of the methods are based on single point scanners can be directed. It is therefore intended that the following appended claims are interpreted as such be that they have all such modifications, permutations and equivalents include, as in the true spirit and fall within the scope of the present invention.

SUMMARY

method and systems of implementing gestures with scanners are revealed. More specifically, methods and systems are disclosed referring to gesturing with multipoint scanners.

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited patent literature

• US 6323846 [0002]

Claims (73)

Electronic system comprising: a multipoint scanning device, which is a multipoint sampling area for receiving inputs from provides one or more objects; one Gesture module that is configured to gesture set for to determine a particular input device, which is passed through the multipoint sampling area of the Multipoint scanning device is received, the particular input device to monitor one or more gesture events, contained in the gesture set and to initiate input actions, that are associated with a gesture event when the gesture event is performed with the input device. The electronic system of claim 1, wherein the Multipoint scanning device is a touch-sensitive Device is. The electronic system of claim 1, wherein the Multipoint scanning device, a proximity scanning device is. The electronic system of claim 1, wherein the Gesture module by hardware, software or a combination of hardware and software is implemented. The electronic system of claim 1, wherein the electronic system is associated with a computer system, including Desktops, laptops, tablets or handheld computers. The electronic system of claim 1, wherein the electronic system with handheld electronic user devices including mobile phones, PDAs or dedicated media players is associated. The electronic system of claim 1, wherein the Gesture set is also determined by one or more factors including position of the input device within the scan area, an open application, an open application state, Input functionality or additional input characteristics. The electronic system of claim 1, wherein the Input actions are state commands. The electronic system of claim 1, wherein the Input actions are manipulative commands. The electronic system of claim 1, wherein the Gesture events performed by the input device become. The electronic system of claim 1, wherein the Gesture events performed by a subset of the input device become. The electronic system of claim 1, wherein the Determining the gesture set Detecting a particular input device and detecting wherein monitoring the particular input device of gesture events associated with the gesture set are. The electronic system of claim 12, wherein detecting gesture events Analyze characteristics of each Input of the input device comprises identifying a particular one Pattern that is associated with the input device and references on the gesture set to see if the particular pattern with a Gesture event associated with the gesture set is. The electronic system of claim 1, wherein the Gesture events associated with touch are out Movements, typing, printing and staying are selected can. The electronic system of claim 14, wherein movements selected from gliding, swiping, clicking and rotation become. The electronic system of claim 14, wherein movements Contraction or expansion between two or more inputs. The electronic system of claim 14, wherein jogging from a half tap, full tap, multiple tap, fast Tap or slow tap to be selected. The electronic system of claim 14, wherein pressure associated with hard and light presses. The electronic system of claim 14, wherein lingering associated with a long or short stationary break becomes. The electronic system of claim 14, wherein the Gesture set from a matrix of different gesture events and various commands. The electronic system of claim 20, wherein the Gesture set from a 3 × 3 matrix of 9 different gesture events and 9 different commands. The electronic system of claim 20, wherein the 9 different gesture events rotate counterclockwise, rotating clockwise meaning, spreading, contracting, swiping, swiping, swiping, swiping, and tapping. Gesture control method, comprising: detect multiple points within a scan area at the same time; Determine of a chord when one or more points within the scan area be detected, the chord a special arrangement of points within the scanning area; Determining a gesture set, associate commands with one or more gesture events; Monitor of points for gesture events; and To run a command associated with the gesture event when a gesture event is detected. The method of claim 23, wherein the points with connected to the touches on a touch-sensitive Surface occur. The method of claim 23, wherein the points with Near-touches are associated with the over a surface occur. The method of claim 23, wherein the chord with mechanical features is performed. The method of claim 23, wherein the chord with all or parts of one or more hands becomes. The method of claim 23, wherein the chord is on based on a number of unknown points. The method of claim 23, wherein the chord is on based on a special arrangement of known points. The method of claim 23, wherein the chord thereon based, whether the points close to each other, in a neutral position or are widely scattered. The method of claim 23, wherein the chords thereon based on whether the contacts are adjacent or offset from each other are. The method of claim 23, wherein the chords thereon based on whether they are from the left or right hand. The method of claim 23, wherein determining of the chord, analyzing an image generated by objects which touch or nearly touch the scanning area, and recognizing a particular arrangement of dots from the image, having. The method of claim 23, wherein the gesture set based on the chord. The method of claim 34, wherein the gesture set also based on additional factors, including Location of chord, open applications, state or mode of application and / or properties of the points. The method of claim 23, wherein the gesture set connected to an input functionality. The method of claim 36, wherein the input functionality is selected from navigation operations, file operations, Editing operations, view operations, formatting operations, tool operations or web browsing operations. The method of claim 37, wherein the instructions include Status commands and manipulative commands selected become. The method of claim 23, wherein detecting the Gesture events include analyzing the properties of the points Identify a specific pattern and reference it Has gesture set and its list of gesture events when a pattern matches a gesture event of the gesture set, then it is assumed that the pattern is the gesture event. The method of claim 39, wherein the properties of points including first order considerations Movement, typing, pressure or lingering, and considerations second order including speed, directions, orientation, Size, duration, shape or length. The method of claim 23, wherein the gesture events associated with touch, touch, pressure and lingering are. The method of claim 41, wherein movements from Sliding, swiping, clicking and rotation are selected. The method of claim 41, wherein movements contract or extending between two or more inputs. The method of claim 41, wherein jogging is from a half tap, full tap, multiple tap, fast tap or slow tapping. The method of claim 41, wherein pressure with hard and slight pressure. The method of claim 41, wherein Verwei is associated with a long or short stationary break. The method of claim 23, wherein the gesture set from a matrix of different gesture events and different ones Commands exists. The method of claim 47, wherein the gesture set from a 3 × 3 matrix of 9 different gesture events and 9 different commands. The method of claim 48, wherein the 9 different Gesture events rotate counterclockwise, rotate clockwise, Spreading, contraction, left-stroking, legal strike, swipe-up, Move down and tap. The method of claim 23, further comprising a determination of whether a switching event has been performed or not, where a switching event refers to an event can reset the operation or a chord change initiated. The method of claim 23, wherein the switching event implemented by one of the following operations: a) Remove all points from the scanning range for a predetermined Time span, b) changing the chord, c) adding / subtracting from Points from the scanning area, d) stationary stopping of the points for a predetermined period of time, e) detecting a special key input or key click, or f) Detect a gesture event. Control operation, comprising: Detect a Touch or near-touch; if one Touch is detected, determining a gesture set for the touch, with the gesture set one or several gesture events to trigger or initiate a Command includes; Monitoring the touch for a gesture event associated with the gesture set is; and Initiate a command when a gesture event, the is associated with the gesture set is executed. The operation of claim 52, wherein the gesture set of touch characteristics, touch location, open Application and / or application mode. The operation of claim 53, wherein the touch or near-touch by a touch device like a touchpad, a touch screen or a touch-sensitive housing detected becomes. Gesture operation, comprising: Monitor a touch movement; Distinguishing the contact movement between a first and second state; Carry out a first action, if the contact movement with the first state is associated; and Perform a second action, if the contact movement with the second State is associated. The gesture operation of claim 55, wherein the touch movement is a sliding movement and wherein the first state is a click is and the second state is a swipe. Control operation, comprising: to disposal provide a first input device and a second input device, wherein the first input device is a touch-sensitive Surface has; Monitor the first Input device after input events; simultaneous monitoring the second input device after input events; Carry out input operations in accordance with input events, associated with the first input device; simultaneous Perform input operations in accordance with input events associated with the second input device are. The control operation of claim 57, wherein the first Input device is a mouse. Control operation, comprising: to disposal make a list of input functions, using the input functions Have commands and gesture events associated with the commands are, with the commands related to the input function stand; Assigning input functions to chords; and Connect an input function with a chord when the chord is recognized becomes. The control operation of claim 59, wherein the input function from navigation operations, file operations, editing operations, View operations, formatting operations, tool operations or web browsing operations is selected. The control operation of claim 59, wherein the gesture events Are actions that are performed by the chord. The control operation of claim 59, wherein the gesture events from rotations, shifts as a group, shifts relative each other and / or typing be selected. The control operation of claim 59, wherein the gesture events are connected to commands in an intuitive way. A control operation according to claim 59, wherein complementary Commands associated with complementary gesture events become. The control operation of claim 59, wherein a chord is a finger arrangement. The control operation of claim 59, wherein associating from input functions to chords selecting an input function from a menu of input functions and then selecting of a chord from a menu list of chords. Control field comprising: Displaying a menu of input functions; View a gesture set containing commands shows that are associated with a selected input function, and gesture events associated with the instructions; Show a list of chords, whereby, to establish a gesture image, a user a desired input function from a List of input functions and selects a desired one Chord selected from the list of chords. Gesture design method, comprising: arrange from chords from the simplest to the most difficult to perform; arrange by frequency of operations of most frequent least often; essentially assign the simplest chords with the most common operations and assigning the most difficult chords with the rarest operations; and Fine tuning of chord / operation grouping. The method of claim 68, wherein the chords be arranged on the basis of one or more attributes, the speed, reliability, comfort, ease of use, simplicity or size of a scanning area selected become. The method of claim 68, wherein said fine tuning may be based on one or more factors, including complementary commands, intuitiveness, overlay or arrangement. Gesture operation, comprising: Detect a first finger; Determining the condition of the finger; detect one or more additional fingers; Determine the state of the additional fingers; and To implement different input modes based on the temporal assignment of States of the first and the additional fingers based on each other. A gesture operation according to claim 71, wherein, if the first finger in a first state of the two states is when the extra fingers are in a first State of the two states are, then becomes a first Implemented input mode, otherwise, a second input mode implemented. The gesture operation of claim 71, wherein the state the first finger moving or stationary and has the condition of the additional fingers being a presence or not, and if the first finger is moving, then an input pointing mode is implemented, and if the first Finger is stationary when the extra fingers present, then a pull mode is implemented.