RU2574830C2 - Gesture shortcuts - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: user movement or body position is captured by a capture device of a system and is used as input data to control the system. For a system-recognised gesture, there may be a full version of the gesture and a shortcut of the gesture. Where the system recognises that either the full version of the gesture or the shortcut of the gesture has been performed, it sends an indication that the system-recognised gesture was observed to a corresponding application. Where the shortcut comprises a subset of the full version of the gesture, and both the shortcut and the full version of the gesture are recognised as the user performs the full version of the gesture, the system recognises that only a single performance of the gesture has occurred.

EFFECT: faster information input.

20 cl, 23 dwg

Description

BACKGROUND

Many computer applications, such as computer games, multimedia applications, office applications or the like, use controls to allow users to manipulate gaming characteristics or other aspects of the application. Typically, such controls are input devices using, for example, controllers, remote controls, keyboards, mice, or the like. Unfortunately, such controls can be difficult to learn, thus creating a barrier between the user and similar games and applications. In addition, such controls may differ from actual game actions or other application actions for which the controls are used. For example, a game control that changes a game characteristic to swing a baseball bat may not correspond to the actual movement of the rising baseball bat.

SUMMARY OF THE INVENTION

Disclosed herein are systems and methods for gesture abbreviations.

In one embodiment of the invention, the computing device receives a series of image data from the camera. This camera may contain a color camera (in particular a red-green-blue color rendering system or RGB), a depth camera, and a three-dimensional (3D) camera. This data may include separate images of depth and color, a combined image that combines information about depth and color, or an analyzed image in which objects are identified, for example, people who are displayed as a skeleton. This data captures movements or poses made by at least one user. The indicated movements or poses can be recognized by the computing device as a type of input data - input data of the gesture. For a given gesture (for example, navigation up), there may be a full version of the gesture that the user can do, and reduction of the gesture that the user can do, reduction of the gesture generally requires less time, movement or difficulty of movement from the user.

If the computing device recognizes that the user has performed either a gesture reduction or a full version of the gesture, it sends a signal about this to the application, which uses gestures as input.

In an embodiment of the invention in which the abbreviation includes a subset of the full version of the gesture, and the abbreviated and full version of the gesture are recognized as the full version of the gesture executed by the user, the computing device recognizes that only a single execution of the gesture has occurred and accordingly indicates the application.

The foregoing is a brief description and thus includes, as appropriate, simplifications, generalizations, and omission of details. It should be understood that this brief description is illustrative only and is not intended to limit in any way.

BRIEF DESCRIPTION OF THE DRAWINGS

Systems, methods and computer-readable media for gestures in accordance with this specification are further described with reference to the accompanying drawings, in which:

1A and 1B illustrate an example implementation of a target recognition, analysis and tracking system by a user playing a game.

Figure 2 illustrates an example implementation of a capture device that can be used in a target recognition, analysis, and tracking system.

3A illustrates an example implementation of a computing environment that can be used to interpret one or more gestures in a target recognition, analysis, and tracking system.

FIG. 3B illustrates another embodiment of a computing environment that can be used to interpret one or more gestures in a target recognition, analysis, and tracking system.

FIG. 4A illustrates a skeletal display of a user that has been generated based on the target recognition, analysis, and tracking system of FIG. 2.

Fig. 4B illustrates additional details of the architecture gesture of the recognition device shown in Fig. 2.

5A and 5B illustrate how gesture filters can be stacked to create more sophisticated gesture filters.

6A, 6B, 6C, 6D, and 6E illustrate an example of a gesture that a user can perform to signal a “fair capture” in an American football video game.

FIGS. 7A, 7B, 7C, 7D, and 7E illustrate examples of the “fair capture” gesture of FIGS. 6A-E in a state after each frame of image data has been analyzed to create a skeletal display of a user.

Fig. 8A illustrates a user performing a running movement in full.

FIG. 8B illustrates a user performing a running movement in a reduced volume, the reduced gesture includes a subset of the full volume movement of the gesture of FIG. 8A.

FIG. 8C illustrates a user performing a second type of reduced gesture volume, a second type of reduced gesture volume includes movement independent of the total gesture volume of FIG. 8A.

Fig.9 depicts an example sequence of operations for gestures.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

As will be described herein, a user can control an application running in a computing environment, such as a game console, computer, or the like, by performing one or more gestures. According to one embodiment of the invention, gestures can be received, for example, using a capture device. For example, a capture device may capture the depth of an image in a scene. In one embodiment of the invention, the capture device may determine whether one or more targets or objects in the scene matches a human object, such as a user. To determine whether a target or object in a scene corresponds to a human object, each of the goals can be painted over and compared with a model of a model of the human body. Each target or object that matches the model of the human body can then be scanned to create the skeletal model associated with it. The skeleton model can then be provided to the computing environment, so that the computing environment can track the skeletal model, present an avatar associated with the skeletal model, and can determine which controls to execute in an application running on the basis of the computer environment, for example, user gestures that were recognized based on the skeletal model. A gesture recognition processor, the architecture of which is described more fully below, is used to determine when a particular gesture was executed by the user.

1A and 1B illustrate an embodiment of a configuration of a target recognition, analysis and tracking system 10 with a user 18 playing a boxing game. In an embodiment of the invention, target recognition, analysis, and tracking systems 10 can be used to recognize, analyze, and / or track a human object, such as user 18.

As shown in FIG. 1A, a target recognition, analysis, and tracking system 10 may include a computing environment 12. Computing environment 12 may be a computer, game system, or console, or similar device. According to an embodiment of the invention, computing environment 12 may include hardware components and / or software components, such that computing environment 12 can be used to execute applications such as gaming applications, non-gaming applications, or the like.

As shown in FIG. 1A, the target recognition, analysis and tracking system 10 may further include a capture device 20. The capture device 20 may be, for example, a camera that can be used to visually monitor one or more users, such as user 18 so that gestures made by one or more users can be captured, analyzed, and tracked to perform one or more controls or actions within the application, as will be described in more detail below.

According to one embodiment of the invention, the target recognition, analysis and tracking system 10 may be connected to an audiovisual device 16, such as a television, monitor, high definition television (HDTV) or the like, which can provide a game or visual and / or audio application to a user, such as user 18. For example, computing environment 12 may include a video adapter, such as a video card and / or an audio adapter, such as a sound card, that can provide audio-visual signals, ssotsiirovannye a game application is not a game application or the like. The audiovisual device 16 may receive audiovisual signals from the computing environment 12 and may then output the game, visual and / or audio application associated with the audiovisual signals of the game, or the application to the user 18. According to one embodiment of the invention, the audiovisual device 16 may be connected to the computing environment 12, for example, via an S-video cable, coaxial cable, HDMI cable, DVI cable, VGA cable, or the like.

As shown in FIGS. 1A and 1B, a target recognition, analysis and tracking system 10 can be used to recognize, analyze and / or track a human object, such as user 18. For example, user 18 can be tracked using capture device 20, so that the movements of the user 18 can be interpreted as controls that can be used to influence the application executed by the computer environment 12. Thus, according to one embodiment of the invention, the user 18 can t move his body to control the application.

As shown in FIGS. 1A and 1B, in an embodiment of the invention, an application executed in computing environment 12 may be a boxing game that user 18 can play. For example, computing environment 12 may use audio-visual device 16 to provide a visual representation of a boxing opponent 22 to user 18. Computing medium 12 can also use audiovisual device 16 to provide a visual representation of the avatar of player 24, which user 18 can control with his movements. For example, as shown in FIG. 1B, user 18 may strike in physical space, causing the avatar of player 24 to strike in game space. Thus, according to an embodiment of the invention, the computer environment 12 and the capture device 20 of the target recognition, analysis and tracking system 10 can be used to recognize and analyze the blow of the user 18 in the physical space so that the blow can be interpreted as a game avatar control of the player 24 in the playing space.

Other movements of the user 18 can also be interpreted as other controls or actions, such as bobsledding, skating, shuffling, blocking, punching, jabbing, or sweeping multiple punches of various strengths. In addition, some movements can be interpreted as controls, which may correspond to actions other than controlling the avatar of player 24. For example, a player can use movements to end a game, pause or save a game to select a level, see the best results, chat with friends and so extra.

In embodiments of the invention, a human object, such as user 18, may possess an object. In such embodiments of the invention, the user of the electronic game can hold the object so that the player’s movements and the given object can be used to adjust and / or control the game parameters. For example, the movement of a player holding a racket can be tracked and applied to control a virtual racket in an electronic sports game. In another embodiment, the movement of the player holding the object can be tracked and used to control virtual weapons in an electronic combat game.

According to other embodiments of the invention, the target recognition, analysis and tracking system 10 can also be used to interpret target movements as an operating system and / or application controls that are outside the scope of the games. For example, in essence, any controllable aspect of the operating system and / or application can be controlled by the movements of the target, such as user 18.

FIG. 2 illustrates an embodiment of a capture device 20 that can be used in a target recognition, analysis, and tracking system 10. According to an embodiment of the invention, the capture device 20 may be configured to capture a video image with depth information, including image depth, which may include depths using any suitable technique, including, for example, travel time, structured lighting, stereo imaging, or the like. According to one embodiment of the invention, the capture device 20 can group the calculated depth information into “Z-layers” or layers that can be perpendicular to the Z-axis, propagating from the depth camera along its observation line.

As shown in FIG. 2, the capture device 20 may include an image camera component 22. According to an embodiment of the invention, the image camera component 22 may be a depth camera that can capture the depth of the image of the scene. The depth of the image may include a two-dimensional (2-D) raster region of the captured scene, where each pixel in the 2-D raster region can represent a range, for example, in centimeters, millimeters, or the like. an object in a captured scene from the camera.

As shown in FIG. 2, according to an embodiment of the invention, the image camera component 22 may include an IR light component 24, a three-dimensional (3-D) camera 26, and an RGB camera 28 that can be used to capture the depth of the image of the scene. For example, in the analysis of the travel time, the IR light component 24 of the capture device 20 can emit infrared light in the scene and then can use sensors (not shown) to find reflected light from the surface of one or more targets or objects in the scene using, for example, a 3-D camera 26 and / or RGB camera 28. In some embodiments of the invention, pulsed infrared light can be used so that the time between the outgoing light pulsation and the matched incoming light signal can be measured and used for determining the physical distance from the pickup device 20 to a specified location on the purposes or objects in the scene. In addition, in other embodiments, the phase of the outgoing light wave can be compared with the phase of the incoming light wave to determine the phase shift. The phase shift can then be used to determine the physical distance from the capture device to a specific location on targets or objects.

According to another embodiment of the invention, the analysis by the transmission method can be used to indirectly determine the physical distance from the capture device 20 to a specific location on targets or objects by analyzing the intensity of the reflected light flux in the dynamics over time using various technologies, including, for example, shuttered light technology pulse imaging.

In another embodiment, the capture device 20 may use structured light to capture depth information. In such an analysis, patterned light (i.e., light incident in the form of a known pattern, such as a net pattern or a striped pattern) can be projected onto the scene, for example, using the IR light component 24. When one or more targets are reached, or objects in the scene, the figure may be deformed under the influence of the surface. Such deformation of the pattern can be captured, for example, by a 3-D camera 26 and / or RGB camera 28, and then can be analyzed to determine the physical distance from the capture device to a specific location on targets or objects.

According to another embodiment of the invention, the capture device 20 may include two or more physically separated cameras that can view the scene from different angles to receive visual stereo data that is resolved to generate depth information.

The capture device 20 may further include a microphone 30. The microphone 30 may include a sensor or sensor that can receive and convert sound into an electrical signal. According to one embodiment of the invention, the microphone 30 can be used to simplify the feedback between the capture device 20 and the computing environment 12 in the target recognition, analysis and tracking system 10. In addition, the microphone 30 can be used to receive an audio signal, which can also be submitted by the user to application controls, such as gaming applications, non-gaming applications, or the like, which may be executed by computing environment 12.

In an embodiment of the invention, the capture device 20 may further include a processor 32, which may be in operative communication with a component of the image camera 22. The processor 32 may include a standard processor, a special processor, a microprocessor, or the like, which may execute commands that may include commands for receiving the depth of the image, determining whether a suitable target can be included in the depth of the image, converting the suitable target into a skeleton representation, or del given goal, or any other suitable team.

The capture device 20 may further include a memory component 34 that may store instructions that may be executed by the processor 32, images or frames of images captured by a 3-D camera or RGB camera, or any other suitable information, image or similar to them. According to an embodiment of the invention, the memory component 34 may include random access memory (RAM), read-only memory (ROM), cache, flash memory, hard disk, or any other suitable storage components. As shown in FIG. 2, in one embodiment of the invention, the memory component 34 may be a separate component in connection with the image pickup component 22 and the processor 32. According to another embodiment of the invention, the memory component 34 may be integrated into the processor 32 and / or the capture component images 22.

As shown in FIG. 2, the capture device 20 may communicate with computing environment 12 via a communication channel 36. The communication channel 36 may be a physical connection including, for example, a USB connection, a Firewire connection, an Ethernet cable connection or the like, and / or may be a wireless connection, such as an 802.11b, g, a or n wireless connection. According to one embodiment of the invention, computing environment 12 may provide a clock to a capture device 20, which may be used to determine the capture time, for example, of a scene through communication channel 36.

In addition, the capture device 20 may provide depth information and images captured, for example, by a 3-D camera 26 and / or RGB camera 28, and a skeletal model that can be created by the capture device 20 for computing environment 12 via a communication channel 36 Computing environment 12 may then use a skeletal model, depth information, and captured images, for example, to recognize user gestures and in the response of an application control such as a game or word processor. For example, as shown in FIG. 2, computing environment 12 may include gesture recognition mechanism 190. Gesture recognition mechanism 190 may include a collection of gesture filters, each of which includes information regarding a gesture that can be performed by a skeletal model (how the user moves). Data captured by cameras 26, 28 and device 20 in the form of a skeletal model and the movements associated with them can be compared with gesture filters in gesture recognition mechanism 190 for identification when the user (as represented by the skeletal model) performs one or more gestures. These gestures can be associated with various application controls. Thus, the computing environment 12 can use the gesture recognition mechanism 190 to interpret the movements of the skeletal model and to control the application based on the movements.

FIG. 3A illustrates an embodiment of a computing environment that can be used to implement the computing environment 12 of FIGS. 1A-2. The computing environment may be a multimedia device 100, such as a game console. As shown in FIG. 3A, the multimedia device 100 has a central processing unit (CPU) 101 having a level 1 cache 102, a level 2 cache 104, and a flash ROM (Read Only Memory) 106. Level 1 cache 102 and a cache Level 2 104 temporarily stores data, and therefore reduces the number of memory access cycles, while improving processing speed and performance. CPU 101 can be provided with more than one core and, accordingly, with additional caches of the 1st and 2nd levels 102 and 104. Flash ROM 106 can store executable code that is loaded during the initial stage of the boot process when the multimedia device is turned on one hundred.

A graphics processor (GPU) 108 and a video encoder / video codec (encoder / decoder) 114 form a video processing channel for high-speed and high-resolution graphics processing. Data is received from the graphics processor 108 to the video encoder / video codec 114 via a bus. The video processing channel outputs data to the A / V (audio / video) port 140 for transmission to a television or other display. The memory controller 110 is connected to the GPU 108 to facilitate the task of the processor in accessing various types of memory 112, for example, but without limitation, RAM (Random Access Memory).

The multimedia device 100 includes an input / output controller 120, a system control controller 122, an audio signal processing unit 123, a network interface controller 124, a first USB root controller 126, a second USB controller 128, and a front panel input / output subunit 130, which is mainly implemented in module 118. USB controllers 126 and 128 act as the primary for peripheral controllers 142 (1) -142 (2), a wireless adapter 148 and an external memory device 146 (for example, flash memory, an external CD / DVD drive, removable media, etc.) .d.). Network interface 124 and / or wireless adapter 148 provides access to a network (e.g., the Internet, a home network, etc.) and can be any of a wide range of wired or wireless adapter components including an Ethernet card, modem, Bluetooth module, cable modem and the like.

System memory 143 is provided for storing application data that is downloaded during the boot process. A media drive 144 is provided and may include a DVD / CD drive, a hard disk, or another removable media drive, etc. The media drive 144 may be internal or external to the multimedia device 100. These applications can be accessed through the media drive 144 for execution, playback, etc. through the multimedia device 100. A media drive 144 is connected to the I / O controller 120 via a bus, such as a Serial ATA bus or other high-speed connection (eg, IEEE 1394).

The system control controller 122 provides various service functions related to making the multimedia device 100 accessible. The audio signal processing unit 123 and the audio codec 132 form a corresponding audio processing channel with high-quality and stereo processing. Audio data is transferred between the audio signal processing unit 123 and the audio codec 132 via a communication channel. The audio processing channel outputs data to the A / V port 140 for playback by an external audio player or device having sound reproduction.

The front panel input / output subunit 130 supports the functionality of the power button 150 and the eject button 152, as well as any LEDs or other indicators located on the outer surface of the multimedia device 100. The system power supply 136 provides power to the components of the multimedia device 100. The fan 138 cools circuitry inside multimedia device 100.

A CPU 101, a GPU 108, a memory controller 110, and various other components within the multimedia device 100 are connected through one or more buses, including serial and parallel buses, a memory bus, a peripheral bus, and a processor or local bus using any of various bus architectures. By way of example, such architectures may include a local peripheral device bus (PCI), a PCI-Express bus, etc.

When the multimedia device 100 is turned on, application data can be downloaded from system memory 143 to memory 112 and / or caches 102, 104 and executed by the CPU 101. The application can provide a graphical user interface that provides a consistent user interface for navigation in various types of environments, available on the multimedia device 100. When the application and / or other media located in the media drive 144 is operational, it can be started or played from the media drive 144 to provide additional new features of the multimedia device 100.

The multimedia device 100 can function as a standalone device by simply connecting the system to a television or other display. In this standalone mode, the multimedia device 100 allows one or more users to interact with the system, watch a movie, or listen to music. However, in the case of integrating the broadband connection provided through the direct network interface 124 or the wireless adapter 148, the multimedia device 100 can also function as a member in a large-scale network community.

When the multimedia device 100 is turned on, a set amount of hardware resources is reserved for the multimedia device system by the operating system. These resources may include memory redundancy (e.g., 16MB), CPU and GPU cycles (e.g., 5%), bandwidth (e.g., 8 kbps), etc. Due to the fact that these resources are reserved during the boot process, the reserved resources are not available from the application.

In particular, memory redundancy is predominantly large enough to contain a boot kernel, parallel system applications, and drivers. CPU redundancy is predominantly permanent, such that if the reserved CPU usage is not used by system applications, the wait thread will use any unused cycles.

Considering GPU redundancy, lightweight messages generated by system applications (such as pop-ups) are displayed by the GPU, interrupting the distribution of code to represent the pop-up window in the overlay. The amount of memory required for the overlay segment depends on the size of the overlay zone, and the overlay is preferably scaled in accordance with the screen resolution. In cases where the full user interface is used by the parallel system application, it is preferable to use a resolution independent of the resolution of the application. The scaling unit can be used to set this resolution so that the need to change the frequency and the reason for TV resynchronization disappears.

After the multimedia device 100 is loaded and system resources are reserved, parallel system applications are executed to provide system functionality. The functionality of the system consists in a set of system applications that are executed in the reserved system resources described above. The kernel of the operating system defines threads that are threads of system applications, in contrast to threads of game applications. System applications are prioritized for execution in the CPU 101 at a specified time and intervals in the order that the corresponding system resource is viewed for the application. Machine time allocation is required to minimize cache failures for a game application running on the console.

When a parallel system application requests sound, sound processing is allocated asynchronously to the game application due to the time factor. The application manager of the multimedia device (described below) controls the sound level of the game application (for example, turning it off, decreasing it) if system applications are active.

Input devices (e.g., controllers 142 (1) and 142 (2)) are shared by gaming applications and system applications. Input devices are not reserved resources, but must be switched between system applications and the game application, so that everyone will have the focus of the device. The application manager preferably controls the switching of the input stream without knowing the details of the gaming applications, and the driver stores the state switching focus information. The cameras 26, 28 and the capture device 20 may determine additional input devices for the console 100.

FIG. 3B illustrates another embodiment of a computing environment 220, which may be the computing environment 12 shown in FIGS. 1A-2 used to interpret one or more gestures in a target recognition, analysis, and tracking system. The environment of computing system 220 is only one embodiment of a suitable computing environment, and it is not intended to introduce any restrictions both in the field of use and functionality of the present disclosure of the subject invention. Also, computing environment 220 should not be interpreted as having any dependency or need regarding any component or combination of components illustrated in typical operating environment 220. In some embodiments of the invention, the various computing elements described may include a circuit configured to illustrate specific aspects of the current disclosure. SUMMARY OF THE INVENTION For example, the term “circuitry” used in the disclosure of the invention may include specialized hardware components configured to perform the function (s) by firmware or switches. In other embodiments of the invention, the term “circuit” may include a general-purpose processing unit, memory, etc., configured by program instructions that include working logic to perform functions (functions). In embodiments of the invention in which the circuit includes a combination of hardware and software, the designer can write source code including logic, and the source code can be compiled into machine-executable code that can be processed by a general-purpose processing unit. Since it is possible to understand that the level of technological development has reached a point where there is a slight difference between a hardware solution, a software solution or a combination of a hardware solution / software solution, the choice of a hardware solution compared to a software solution to perform specific functions is a design decision, a designer. In particular, one skilled in the art can appreciate that a software process can be transformed into an equivalent hardware structure, and a hardware structure can itself be transformed into an equivalent software process. Thus, the choice of hardware implementation in comparison with software implementation is one of the design solutions and remains with the designer.

3B, computing environment 220 includes a computer 241, which typically includes a number of computer-readable media. A computer-readable storage medium may be any available medium that can be accessed by computer 241 and includes a variable and non-changing medium, removable and non-removable medium. System memory 222 includes computer memory in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 223 and random access memory (RAM) 260. Basic input / output system 224 (BIOS), which contains basic programs that help transfer information between elements within the computer 241, for example, during startup, are usually stored in ROM 223. RAM 260 usually contains data and / or program modules that instantly become available and / or immediately b can be executed by a data processing unit 259. As an example and without limitation, FIG. 3B illustrates an operating system 225, application programs 226, other program modules 227, and program data 228.

Computer 241 may also include other removable / non-removable, volatile / non-volatile computer storage device. Only by way of example, FIG. 3B illustrates a hard disk 238 that reads from or writes to a non-removable, non-volatile magnetic medium, a magnetic disk drive 239 that reads from or writes to a removable, non-volatile magnetic disk 254, and an optical disk 240, which reads from or writes to a removable non-volatile optical disk 253, such as a CD-ROM, or other optical medium. Another removable / non-removable, volatile / non-volatile computer storage device that can be used in a typical operating environment, including but not limited to magnetic tape tapes, flash memory cards, universal digital disks, digital video tapes, solid state RAM, solid state ROM and the like. The hard disk 238 is usually connected to the system bus 221 via a non-removable memory interface such as interface 234, and the magnetic disk drive 239 and the optical disk drive 240 are usually connected to the system bus 221 by a removable memory interface such as interface 235.

The drives and their associated computer storage device, discussed above and illustrated in FIG. 3B, provide storage of computer-readable instructions, data structures, program modules and other data for computer 241. In FIG. 3B, for example, hard disk 238 is illustrated as an operating system storage 258, application program 257, other program modules 256, and program data 255. It should be noted that these components may either be the same as or may differ from operating system 225, application program programs 226, other program modules 227, and program data 228. Operating system 258, application program 257, other program modules 256, and program data 255 are given different numbers here to illustrate that they are at least different copies. The user can enter commands and information into the computer 241 through input devices, such as a keyboard 251 and a pointing device 252, usually called a mouse, trackball or touchpad. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the data processing unit 259 via a user input interface 236 that interfaces with the system bus, but can be connected using another interface and bus structures such as a parallel port, game port, or universal serial bus (USB) . Cameras 26, 28 and the capture device 20 may determine additional input devices for the console 100. A monitor 242 or other type of display device is also connected to the system bus 221 via an interface such as a video interface 232. In addition to the monitor, computers may also include other peripheral output devices, such as speakers 244 and a printer 243, which can be connected through an interface 233 of peripheral output devices.

Computer 241 may operate in a network environment using logical connections with one or more remote computers, such as remote computer 246. Remote computer 246 may be a personal computer, server, router, network PC, device of a peer-to-peer network, or other conventional network node, and typically includes many or all of the elements described above related to computer 241, although only storage device 247 is illustrated in FIG. 3B. The logical connections depicted in FIG. 3B include a local area network (LAN) 245 and a wide area network (WAN) 249, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet.

When used in a LAN network environment, computer 241 connects to LAN 245 through a network interface or adapter 237. When used in a WAN network environment, computer 241 typically includes a modem 250 or other designed to communicate over WAN 249, such as the Internet. The modem 250, which may be internal or external, may be connected to the system bus 221 via a user input interface 236 or other suitable mechanism. In a networked environment, program modules depicted associated with a computer 241 or parts thereof may be stored in a remote storage device. As a non-limiting example, FIG. 3B illustrates a remote application program 248 as being permanently stored in a memory device 247. It should be understood that the network connections shown are typical and other ways of achieving a communication connection between computers can be used.

Fig. 4A depicts an exemplary skeletal image of a user that can be created by the gripper 20. In this embodiment, various joints and bones are identified: each arm 302, each forearm 304, each elbow 306, each biceps 308, each shoulder 310, each hip joint 312, each hip 314, each knee 316, each lower leg 318, each foot 320, head 322, torso 324, upper 326 and lower part 328 of the spinal column and waist 330. In case most of the points are tracked, additional body parameters, such as bones and joints of the fingers or toes, or individual face parameters, such as nose and eyes.

By moving in the space of his body, the user can create gestures. The gesture includes the movement or pose of the user, which can be captured as image data and interpreted in terms of content. The gesture may be dynamic, containing movement, such as simulating a ball toss. The gesture may be a static pose, such as holding the crossed forearm 304 by the user in front of his torso 324. The gesture may also include the use of props, such as swinging with an imaginary sword. The gesture may include more than one part of the body, such as clapping hands 302, continuous or barely noticeable movement, such as pursing the lips.

Gestures can be used to enter a general computational context. For example, various movements of the hands 302 or other parts of the body may correspond to the general tasks of the entire system, such as navigating up or down in a hierarchically arranged list, opening a file, closing a file, and saving the file. Gestures can also be used in the specific context of a video game, depending on the game. For example, in the case of a driving game, various movements of the hands 302 and feet 320 may correspond to steering the car in direction, shifting gears, accelerating and braking.

The user can form a gesture that corresponds to walking or running, by walking or running in place. The user can alternately raise and lower each leg 312-320 to emulate walking without movement. The system can interpret this gesture by analyzing each hip joint 312 and each hip 314. A step can be recognized when one corner of the hip joint (which is measured relative to a vertical line where the standing leg has a hip angle of 0 ° and the leg stretched horizontally forward has a hip angle joint 90 °) exceeds a certain threshold value relative to the other hip. Walking or running can be recognized after a number of successive stages of leg movement. The time between the last two steps can be considered a period. After a certain number of periods where the threshold angle value has not been met, the system can determine that the walking or running gesture has stopped.

If there is a “walk or run” gesture, the application can set values for the parameters associated with the specified gesture. These parameters may include the threshold angle value, the number of steps required to initialize the walking or running gesture, the number of periods where there were no steps to the end of the gesture, and the threshold period that determines whether it was a walking or running gesture. A fast period may correspond to running at a time when the user will move his legs quickly, and a slow period may correspond to walking.

A gesture can be associated with a set of default parameters initially, which the application can reinstall along with its own parameters. In such a scenario, the application does not initiate the provision of parameters, but can instead use a set of default parameters that allow the gesture to be recognized if the application does not have the specified parameters.

There is a series of output that may be associated with a gesture. There may be a yes or no baseline regarding whether the gesture is being implemented. There may also be a level of confidence that corresponds to the likelihood that the user's tracking is consistent with the gesture. It should be a linear scale that runs through the values of numbers from 0 to 1 inclusive. Where an application receiving this gesture information cannot accept false positives as input, it can only use recognized gestures that have a high level of certainty, such as at least .95. If an application needs to recognize every instance of a gesture, even when counting false positives, it can use gestures that have at least a much lower level of confidence, such as gestures with a level of confidence just higher than .2. The gesture can have data output for the time between the last two steps, and in cases where only the first step is registered, this can be set to a reserved value, such as -1 (since the time between the two steps must be positive). The gesture may also have data output for the largest hip angle achieved during the very last step.

Another typical gesture is a “heel lift jump”. In this case, the user can make a gesture, lifting his heels off the ground, but leaving his toes in place. Alternatively, the user can jump up, where his feet 320 are completely off the ground. The system can interpret the skeleton for a given gesture by analyzing the ratio of the angle of the shoulders 310, the hip joints 312 and the knees 316 to see if they are in a position corresponding to a straightened position. Then these points and the upper 326 and lower 328 points of the spinal column can be monitored for any acceleration up. A reasonable combination of acceleration can activate a jump gesture.

If you have the specified “jump with heels up” gesture, the application can set values for the parameters associated with this gesture. The parameters may include the previously mentioned acceleration threshold value, which determines how quickly a combination of the shoulders of the user 310, the hip joints 312 and the knees 316 must move up to activate the gesture, as well as the maximum alignment angle between the shoulders 310, the hip joints 312 and the knees 316, in which the jump can still be activated.

The findings may include a level of confidence, as well as the angle of the user's body at the time of the jump.

Setting parameters for the gesture, based on the details of the application that will receive the gesture, is important for the accuracy of gesture identification. Properly identifiable gestures and intentions of the user are very helpful in creating a positive user experience. If the gesture recognition processor is too sensitive, and even a slight movement of the hand forward 302 is interpreted as a throw, the user may be disappointed because the gestures are recognized when he did not intend to make a gesture, and thus he does not have control over the system. If the gesture recognition processor is not sensitive enough, the system may not recognize the user's attempts to make a throw-up gesture, disappointing him in a similar way. At any end of the sensitivity spectrum, the user is disappointed because he cannot properly provide input to the system.

Another parameter for the gesture may be the distance of movement. In the case when user gestures control the actions of an avatar in a virtual environment, the avatar can be at arm's length from the ball. If the user wishes to interact with the ball and grab it, the user may be required to extend his arm 302-310 to the full length while performing the grab gesture. In such a situation, such an exciting gesture in cases where the user only partially extends his arm 302-310 may not achieve the result of interacting with the ball.

A gesture or part of it can have as a parameter the amount of space in which it should occur. This volume of space can usually be expressed in connection with the body, in cases where the gesture contains the movement of the body. For example, a football throw gesture for a right-handed user can only be recognized in a space no lower than the right shoulder 310a, and on the same side of head 322 as the throwing hand 302a-310a. It may not be necessary to define all boundaries of the volume, for example, in the case of the indicated throwing gesture, where the external boundary far from the body remains indefinite, and the volume indefinitely expands to the corner of the scene, which is controlled.

FIG. 4B provides additional details of one typical embodiment of the gesture recognition mechanism 190 of FIG. 2. As shown, the gesture recognition mechanism 190 may include at least one filter 418 for determining a gesture or gestures. Filter 418 includes information defining a gesture 426 (hereinafter referred to as “gesture”) in accordance with parameters 428 or metadata for the specified gesture. For example, a throw, which includes the movement of one of the hands from position behind the body to the position in front of the body, can be implemented as a gesture 426, including information representing the movement of one of the hands of the user from position behind the body to the position in front of the body, given that motion would be captured by a depth camera. Parameters 428 can then be set for the specified gesture 426. In case the gesture 426 is a throw, parameter 428 can be a threshold value of the speed that the hand can reach, the distance the hand must go (either absolute or relative to the size of the user as a whole) , and a confidence assessment determined by the recognition mechanism that the gesture has occurred. The indicated parameters 428 for gesture 426 may differ in different applications, in different contexts of an individual application, or within the same context of one application over time.

Filters can be modular or interleaved. In an embodiment of the invention, the filter has a number of inputs, each of these inputs has a type and a number of outputs, each of these outputs has a type. In this situation, the first filter can be replaced by a second filter, which has the same number and types of inputs and outputs as the first filter without changing any other aspect of the recognition engine architecture. For example, there may be a first filter for driving, which receives, as input, skeletal data and displays the certainty that the gesture associated with the filter takes place and displays the steering angle. If someone wants to replace the specified first filter for driving with a second filter for driving, possibly because the second filter is more efficient and requires less processing resources - someone can simply replace the first filter with a second filter for this purpose, provided that the second filter has the same inputs and outputs - one input of the skeletal data type and two outputs of the type of reliability and type of angle.

A filter does not need a parameter. For example, the user growth filter, which returns the user's growth, may not include any parameters that can be configured. An alternative filter “user height” can have customizable parameters - for example, whether to take into account user shoes, hair, headgear and position when determining the height of the user.

Inputs to the filter may include elements such as positional data of the user's joints, such as angles formed by bones that are connected to the joints, RGB color data from the scene, and the rate of change of the aspect of the user. The exits from the filter may include elements, such as the certainty that the specified gesture is being executed, the speed at which the gesture is moving, and the time it takes for the gesture to complete.

A context can be a cultural context, and it can be an environmental context. Cultural context refers to the culture of the user using the system. Different cultures can use similar gestures to set definitely different meanings. For example, an American user who wants to tell another user “look” or “use your eyes” can bring his index finger to his head, to the outer corner of the eye. However, for the Italian user, this gesture can be interpreted as a reference to the mafia.

Similarly, there may be different contexts among different environments of one application. Take a first-person shooter game that involves driving a car. When the user is standing on his feet, folding his fingers into a fist in the direction of the earth and holding his fist forward and at a certain distance from the body, he can represent a gesture of impact. When the user is in the context of driving, the same movement may represent a “gear shift” gesture. There can also be one or more menus of environments where the user can save his game, choose from the equipment of his character or perform similar actions that do not constitute a direct gameplay. In such an environment, the indicated one and the same gesture may have a third meaning, for example, select something or move to another screen.

Gesture recognition mechanism 190 may have a basic recognition mechanism 416 that provides the functionality of gesture filter 418. In an embodiment of the invention, the functionality that recognition mechanism 416 implements includes an input archive over time that tracks recognized gestures and other input, an implementation of hidden Markov models (where the simulated system is supposed to be a Markov process - a process where the current state encapsulates any information about the past state yanii, which is necessary to determine the future status, which is why other state information about the past should not be stored for this purpose - with unknown parameters, and hidden parameters are determined from observed data), as well as other functionality required to address specific implementations of gesture recognition.

Filters 418 are loaded and implemented on top of the base recognition engine 416 and can use the services provided by the mechanism 416 for all filters 418. In an embodiment of the invention, the base recognition engine 416 processes the received data to determine if it meets the requirements of any filter 418. From the moment providing these services, for example, analysis of input data, they are provided by the basic recognition engine 416 at a time, and are not processed by each filter 418, such a service needs to be processed atyvat only once per period, in contrast to one processing of each filter 418 over time, so the processing required to determine the sign decreases.

An application may use filters 418 provided by the recognition engine 190, or it may provide its own filter 418 that is integrated into the underlying recognition engine 416. In an embodiment of the invention, all filters 418 have a common interface to provide said plug-in characteristic. Additionally, all filters 418 can use parameters 428, which is why a single gesture tool, as described below, can be used to debug and configure the entire filter system 418.

The indicated parameters 428 can be configured for the application or for the application context by the gesture tool 420. In an embodiment of the invention, the gesture tool 420 includes a plurality of sliders 422, each slide 422 corresponds to a parameter 428, as well as a visual representation of the body 424. Since the parameter 428 is configured with of the corresponding slider 422, the body 424 can demonstrate actions that will be recognized as a gesture with these parameters 428, and actions that will be recognized as a gesture with indicated parameters 428 identified as such. Said visualization of gesture parameters 428 provides an effective way to debug and fine-tune the gesture.

5 depicts more complex gestures or filters 418 created from stacked gestures or filters 418. Gestures can be stacked one after another. That is, more than one gesture can be depicted by the user at a time. For example, instead of refusing to enter any information other than a throw when performing a throwing gesture or requiring the user to remain motionless, with the exception of the components of the gesture (for example, to stand motionless when performing a throwing gesture in which only one hand is involved). Where there is a stack of gestures, the user can make a jump gesture and a throw gesture at the same time, and both of these gestures will be recognized by the gesture mechanism.

5A depicts a simple gesture filter 418 according to the stack paradigm. The IFilter 502 filter is a basic filter 418 that can be used in every gesture filter. IFilter 502 receives the position data of user 504 and outputs a confidence level 506 that the gesture has occurred. It also downloads this position information 504 to the steering wheel filter 508, which takes it as input and displays the angle the user turned the steering wheel (for example, 40 degrees to the right of the user's current direction) 510.

FIG. 5B depicts a more complex gesture that stacks filters 418 on the gesture filter of FIG. 5A. In addition to the IFilter 502 and the steering wheel of the car 508, there is an ITracking 512 filter that receives position data 504 from the IFilter 502 and displays the degree of completion of a gesture made by the user 514. ITracking 512 also downloads position data 504 in the GreaseLightning 516 and EBrake 518, which are filters 418 that read other gestures that can be made while driving, such as using a hand brake.

6 depicts an example of a gesture by which user 602 may signal an “honest capture” in an American football video game. These figures depict the user at points in time, with FIG. 6A being the first time point and FIG. 6E being the last time point. Each of these figures may correspond to a picture or image frame captured by a camera of depth 402, even optionally consecutive frames of image data, since a camera of depth 402 may be able to capture frames at a greater frequency than the user can walk the distance. For example, the specified gesture can occur in a period of 3 seconds, and if the depth camera captures data at a speed of 40 frames per second, 60 frames of image data will be captured, while the user 602 made this gesture of “honest capture”.

In FIG. 6A, user 602 begins to lower his arms along body 604. He then raises his arms up and over his shoulders, as shown in FIG. 6B, and then further upward, approximately to his head level, as shown in FIG. 6C. From this position, he lowers his hands 604 to shoulder level, as shown in fig.6D, and then again raises them up, approximately to the level of his head, as shown in fig.6E. If the system captures the indicated positions of user 602 without any intermediate position that can signal that the gesture has been canceled or another gesture is being performed, the honest capture gesture filter can output data with a high level of certainty that user 602 made an honest capture gesture ".

FIG. 7 depicts an example of a “fair capture” gesture of FIG. 5 when each frame of image data has been interpreted to represent a skeletal display of a user. A system that presents a skeletal display based on the depth of the user's image can now determine how the user's body moves in time and, based on this, interpret the gesture.

In FIG. 7A, the shoulders of the user 310 are located above the elbows 306, which in turn are located above his hands 302. Then, in FIG. 7 B, the shoulders 310, the elbows 306 and the hands 302 are at the same level. The system in FIG. 7C then determines that the hands 302 are above the elbows that are above the shoulders 310. In FIG. 7D, the user has returned to the position of FIG. 7B, where the shoulders 310, elbows 306 and hands 302 are on the same level. In the final position of the gesture shown in FIG. 7E, the user returns to the position of FIG. 7C, where the hands 302 are above the elbows that are above the shoulders 310.

While the capture device 20 captures a series of still images, such that the user is still in any one image, the user moves throughout this gesture (unlike the static gesture, as discussed above). The system is able to receive data from a series of poses in each static image, and from which it determines the level of confidence of the moving gesture that the user makes.

When performing the gesture, it is unlikely that the user will be able to make an angle that is formed by his right shoulder 310a, right elbow 306a and right hand brush 302a and, for example, from 140 ° to 145 °. Thus, an application using the filter 418 for the gesture of fair capture 426 can configure the associated parameters 428 to better serve the specific features of the application. For example, the positions in FIGS. 7C and 7E can be recognized at any time when the user has his hands 302 above his shoulders 310, without considering the position of the elbow 306. A number of parameters that are more stringent may require the hands 302 to be above the head 310 and so that both elbows 306 are above the shoulders 310 and between the head 322 and the brushes 302. In addition, the parameters 428 for the gesture of fair capture 426 may require the user to move from the position of figa to the position of figa for a set period time, for example 1.5 seconds, and if the user It takes more than 1.5 seconds to move through the indicated positions, the movement will not be recognized as an “honest capture” 418, and a very low level of confidence can be inferred.

8A-C illustrate a user making a similar system-recognized running movement using various captured movements and poses.

8A illustrates a user performing a full running gesture. The user 18 is captured by the capture device 20. The user 18 creates a full running gesture by running in place - alternately raising each knee to approximately the waist, then lowering the leg down to the ground. The indicated version of the full running gesture is a cyclic gesture in which the user 18 repeats the movements that include the gesture for the time that the gesture lasts at will.

FIG. 8B illustrates a user making a shortened running gesture, the shortened gesture includes a subset of the full running gesture of FIG. 8A. To make the indicated version of the reduced running gesture, the user 18 lifts one leg so that his knee is approximately at hip level and remains in the specified position. Said gestural contraction includes a subset of the full gesture movement of FIG. 8A, where the user of FIG. 8A re-lifts and lowers his knees, here the user lifts his knee and remains in that position. While the full gesture of FIG. 8A contains a repeating movement, in this embodiment, the abbreviated gesture comprises a series of non-repeating movements or a series of movements where the series as a whole were not repeated. In an embodiment of the invention, the user 18 lowers his knee down to a standing position when he wishes to complete the gesture. In an embodiment of the invention, said knee lowering action may also include a subset of the full gesture. In an embodiment of the invention, computing environment 12 determines that a given movement is completing a gesture of contraction, rather than producing a full gesture when user 18 holds his knee at about hip level for more than a set period of time.

FIG. 8C illustrates a user doing a second type of abbreviated running gesture, the second type of abbreviated running gesture includes movement separate from the full running gesture of FIG. 8A. Here, user 18 takes one step forward and remains in the indicated position, with one foot in front of the other, both feet on the ground, for the time that he wants to make a running gesture. This position was not found in the full running gesture of FIG. 8A. User 18 can end the gesture by stepping back into a standing position. This gesture is similar to the gesture of FIG. 8B in that both gestures provide movement to start the gesture, then maintaining a pose to maintain the gesture and movement to end the gesture.

9 illustrates exemplary operating procedures for gestures. As discussed above, a single gesture input to a computing device can be recognized by the computing device as a result of a variety of methods performed by the user. In an embodiment of the invention, said plurality of ways in which a gesture can be performed by a user includes the full version of the gesture and the reduction of the gesture.

Gesture abbreviations can be used in a number of application contexts. For example, gesture running reductions can be used in applications that include running, such as a treadmill and playing on the field. The text input of abbreviations can be used in the text input of the application context. For example, a user may use sign language to enter text. The full version of the word gesture may include signing each letter of the word, since H-E-A-R-T. The abbreviation for the gesture of the word "heart" may include one heart gesture, similar to the image of a heart folded by the hands of the user. Such sign language may include American Sign Language (ASL).

Gesture contraction may involve other parts of the body than the corresponding full version of the gesture. For example, if the user cannot use the legs, and the full version of the running gesture includes running in place, reducing the gesture may include simulating the running movement of the user's hands.

Optional action 902 depicts the reception of user input corresponding to a gesture reduction definition. For example, a user, either by warning a computing device, or by instructing the computing device to do so, can make a movement or pose that is captured by the capture device and saved as a way to perform a gesture.

In an embodiment of the invention in which the user has determined a gesture reduction by means of his movement or posture, he can then specify the gesture reduction in the system. For example, if gestures are recognized using filters and corresponding parameters, he can adjust the parameters of his gesture reduction in the ways described above.

In an embodiment, the gesture reduction corresponds to the full version of the second gesture. Gesture abbreviation can correspond to many full gestures, and where the user defines the abbreviation, he can show that the abbreviation corresponds to many full gestures. For example, in the context of printing a text file, the User can define one gesture abbreviation that corresponds to the full gesture of selecting the vertical orientation of the paper, the full gesture of choosing to print four copies, and the full gesture of choosing a particular printer to print the form.

In an embodiment of the invention in which gestures are recognized through gesture filters and parameters, gesture reduction and the full version of the gesture can use the same gesture filter, but a different value for one or more parameters. For example, the full version of the “ball thrown” gesture may require the user to move his hand about the length of his arm in front of the torso due to the torso. Contraction can reduce the required distance that the brush must travel so that the brush does not extend either far back or far forward. This can be accomplished by changing the value or values of the parameter, such as the parameter "minimum brush distance".

Step 904 depicts the reception of data captured by the capture device, the data corresponding to a gesture made by the user. The capture device can capture a scene that contains the entire user, for example, from floor to ceiling and to the wall on each side of the room, at a distance at which the user is located. The capture device can also capture a scene that contains only part of the user, for example, the user from the waist up when he or she is sitting at the table. The capture device can also capture a user-controlled object, such as a props camera, which the user holds in his hand.

Optional action 906 depicts a definition from a data processing application with a gesture reduction. An application may limit abbreviations that are used as input in some way. For example, in a game on the field or on a treadmill, running may be considered an integral part of the process, and the application may not allow or block the abbreviation for the running gesture, requiring the user to make a full gesture when he wants to run. For comparison, in a first-person shooter game, running can be seen as an aid in using the game, therefore, the use of shortening for the running gesture can be allowed. In this first-person shooter game, a mechanic for discharging a firearm may be considered an integral part of the process, and the application may not allow or block the reduction for the aiming or shooting gesture.

In an embodiment of the invention, the user can perform both abbreviations for gestures and full versions of gestures that are recognized in the same way that the user can execute several gestures simultaneously, as described above. Using the example of a first-person shooter game, a user can simultaneously make a shortcut for a running gesture and a full version of the aiming gesture.

In an embodiment of the invention, said definition for processing data with a gesture reduction comes from a user. For example, which abbreviations for processing may correspond to the complexity level of the application that the user selects. In case the user selects the lower level of difficulty, all abbreviations can be processed. When the user increases the difficulty level, the number of allowed cuts for processing can be reduced, and ending with the highest level of difficulty, where cuts are not processed.

The specified definition may vary in time over time to adapt to the capabilities of the user. For example, the default settings for allowed abbreviations can be implemented at the start of an application session and allowed abbreviations can be increased or decreased during the session when the user demonstrates his ability to perform gestures well or does not have such an opportunity. In addition, since the user gets tired during the session or an ability improves, the allowed reductions can be increased or decreased to match his current state of ability.

Act 908 depicts data processing for determining output corresponding to whether the user has performed a gesture reduction, the gesture reduction corresponding to the full version of the gesture. In an embodiment of the invention, said output may include a level of certainty that the gesture has occurred. In an embodiment of the invention, the output may include an indication in the form of whether a full version of the gesture or gesture reduction has been observed.

Step 910 depicts sending output corresponding to a gesture reduction to an application. In cases where the current actions are performed by the application, the output can be sent to the application component, which receives the processed user input and converts them for actions in the application.

Optional action 912 depicts data processing to determine the output corresponding to whether the user has completed the full version of the gesture and sending the output corresponding to the full version of the gesture to the application.

In an embodiment of the invention, gesture reduction includes a user movement that contains a subset of the user movement that includes the full version of the gesture.

In an embodiment of the invention, the output corresponding to the gesture reduction corresponds to the high probability that the user began to perform the gesture, the output corresponding to the full version of the gesture corresponds to the high probability that the user began to perform the gesture, and the application recognizes only one user gesture. In cases where the gesture reduction includes a subset of the full version of the gesture, if the user performs the full version of the gesture, he will also perform the gesture reduction. Thus, for one intended input of a gesture, two gestures can be recognized. In an embodiment of the invention in which both the gesture reduction and the full version of the gesture are recognized within a set period of time (a period of time that may be specific to the gesture and / or user), only one is used as input and the other is not accepted into account.

In an embodiment of the invention, the output corresponding to the gesture reduction corresponds to the high probability that the user began to perform the gesture, the output corresponding to the full version of the gesture corresponds to the high probability that the user began to perform the gesture, and the application uses the output corresponding to the full version gesture to add detail to the gesture. In cases where the gesture reduction is recognized and processed (for example, the corresponding animation or result is shown on the display device), as input, by the application, and then the full version of the gesture is recognized, while the gesture reduction is processed, the output of the full version of the gesture can be used in processing.

For example, the full version of the “jump” gesture may include a jumping user. Reducing the indicated “jump” gesture may include the initial movements of the full version of the gesture — crouching and lifting — and the output of the level of confidence that the gesture was completed. As a result of observing a reduction in the jump gesture, the application can process it by displaying a jumping user avatar. While this happens, if the user completes the full version of the jump gesture by continuing to lift and lift both feet off the ground, the application can use the height that the user physically jumps to display the avatar jumping to the appropriate height to add detail to the current one being processed shorten the jump. If the user performs only a reduction in the gesture of the jump, the application can display the avatar jumping in height, which is set by default.

In an embodiment of the invention, in cases where the user performs a gesture reduction, it may correspond to less execution in the application than in cases where the user performs the full version of the gesture. For example, in a skateboard game that categorizes user performance using points, if a user performs a given trick using a reduction in the gesture of a skateboard trick, the user can take fewer points than he would have if he performed the full version of this trick using the full version of the skateboard gesture trick.

CONCLUSION

While the present disclosure is described in connection with preferred aspects, as illustrated in various figures, it should be understood that other similar aspects may be used, or modifications and additions may be made to the described aspects to perform the same function of the present disclosure without deviations from them. Thus, the present disclosure of the invention should not be limited to any particular aspect, but rather be interpreted in the scope and scope according to the attached claims. For example, the various procedures described herein may be implemented in hardware or software, or in a combination of both. Thus, the methods and equipment of the disclosed embodiments of the invention or some aspects or parts thereof can take the form of program code (namely, instructions) embodied on tangible media such as floppy disks, CD-ROMs, hard drives or any other computer-readable media . When the program code is loaded into a machine and executed by a machine, such as a computer, the machine becomes equipment configured to apply the disclosed embodiments of the invention. In addition to the specific implementation options that are provided here explicitly, other aspects and implementations will be apparent from consideration of the description disclosed herein. It should be understood that the description and illustrative options for implementation are considered only as examples.

Claims (20)

1. A method of using an abbreviated gesture in a system that takes user gestures as input to an application containing the steps of:
receive data captured by the capture device, and this data corresponds to the movement or position of the user;
determining, based on said data, the first output identifying that said movement or pose of the user corresponds to that the user is likely to perform a full gesture reduction, wherein performing a full gesture reduction identifies a command also identified by performing a full gesture, wherein the full gesture reduction contains a subset user movements or postures making up a complete gesture;
send the first output to the application;
determining, on the basis of said data, a second output identifying that said movement or posture of the user corresponds to the fact that the user is likely to perform a full gesture;
send second output to the application; and in response to recognizing a full gesture reduction and a full gesture within a predetermined amount of time, the application uses only either a full gesture reduction or a full gesture as input for said command. 2. The method according to claim 1, further comprising the step of: receiving user input corresponding to the definition of the movement or posture of the user used to complete the full gesture reduction. 3. The method of claim 1, wherein reducing the full gesture corresponds to a second gesture that identifies the second command, the first output identifying that the second command has been called. 4. The method according to claim 1, wherein the abbreviation of the full gesture corresponds to the gesture filter and the parameter, and the full gesture corresponds to the mentioned filter of the gesture and the parameter, the value of this parameter to reduce the full gesture differs from that for the full gesture, while determining based on said first output data identifying whether the user's movement or posture corresponds to the user performing a full gesture reduction, said data is processed using said gesture filter and param tra. 5. The method according to claim 1, wherein when determining, based on said data, the first output identifying whether the user's movement or posture corresponds to the user performing a full gesture reduction, it is further determined from the application to process the said data to determine whether full gesture reduction is performed prior to processing the data to determine the first output. 6. The method according to claim 1, wherein when determining, on the basis of said data, the first output data identifying whether the movement or pose of the user corresponds to the user performing a full gesture reduction, the first output data identifying is additionally determined based on said data whether the user's movement or posture that the user is performing a full gesture reduction is additionally in response to receiving from the user an indication that the abbreviations should be recognized e gestures. 7. The method according to p. 1, in which the first output shows the call of the command, the application associates the call of this command with the result or achievement, and when sending the first output to the application, the first output is sent to the application, while the application binds the first output with a smaller result or achievement than the result or achievement that the mentioned team had when it was called by performing a complete gesture. 8. The method of claim 1, wherein the first output contains a confidence level identifying the likelihood that said movement or posture of the user corresponds to the user performing a full gesture reduction, the method further comprising the step of responding to the reception the first output, the application recognizes that the command was called, and in response to receiving the second output, the application adds details to the called command based on the second output. 9. The method of claim 8, wherein the second output shows the distance the user has moved a part of the body, or the position of that part of the body. 10. The method of claim 8, further comprising the step of: in response to determining that the user has completed the full gesture reduction without subsequently performing the full gesture, the third output is sent to the application indicating that the user is performing the full gesture reduction, In this application recognizes that the command was called, and in response to the determination that the user did not complete the gesture, the default value is used to call the command. 11. A machine-readable storage device on which machine-readable instructions for using an abbreviated gesture are stored in a system that takes user gestures as input to an application, and machine-readable instructions, when executed by a processor, instruct the processor to perform operations comprising:
receiving data captured by the capture device, and this data corresponds to the movement or position of the user;
determining, based on said data, the first output identifying that said movement or posture of the user corresponds to the user probably performing a full gesture reduction, wherein performing a full gesture reduction identifies a command also identified by performing a full gesture, wherein the full gesture reduction contains a subset user movements or postures making up a complete gesture;
sending the first output to the application;
determining, based on said data, a second output identifying that said movement or posture of the user corresponds to the fact that the user is likely to perform a full gesture;
sending second output to the application; and
in response to recognizing a full gesture reduction and a full gesture within a predetermined amount of time, the application uses only either a full gesture reduction or a full gesture as input for the command. 12. The computer-readable storage device according to claim 11, further comprising computer-readable instructions that, when executed by the processor, instruct the processor to perform operations comprising receiving user input corresponding to the definition of a user's movement or posture used to perform a full gesture reduction. 13. The machine-readable storage device according to claim 11, wherein the abbreviation of the full gesture corresponds to a second gesture that identifies the second command, the first output identifying that the second command has been called. 14. The computer-readable storage device of claim 12, wherein the first output shows a call to said command, the application associating a call to this command with a result or achievement, while sending the first output to the application, the first output is sent to the application and the application associates the first output with a smaller result or achievement than the result or achievement that the mentioned team had when it was called by performing a complete gesture. 15. The computer-readable storage device according to claim 12, wherein the first output data contains a confidence level identifying the probability that said movement or posture of the user corresponds to the user performing a full gesture reduction, the computer-readable storage device further comprising computer-readable instructions that, when their execution by the processor instruct the processor to perform the steps in which:
In response to receiving the first output, the application recognizes that the command was called, and,
In response to receiving the second output, the application adds details to the called command based on the second output. 16. A computer system for using gesture reduction in a system that takes user gestures as input to an application, comprising: a processor; and
a memory connected to the processor with the ability to exchange data during the operation of the computer system, while the memory contains instructions executed by the processor, which, when executed by the processor, require the computer system to at least:
receive data captured by the capture device, and this data corresponds to the movement or position of the user;
determine, based on said data, the first output identifying that said movement or posture of the user corresponds to that the user is likely to perform a full gesture reduction, wherein performing a full gesture reduction identifies a command also identified by performing a full gesture, wherein the full gesture reduction contains a subset user movements or postures making up a complete gesture;
send the first output to the application;
determine, based on said data, a second output identifying that said movement or posture of the user corresponds to the fact that the user is likely to perform a full gesture;
send second output to the application; and
in response to recognizing a full gesture reduction and a full gesture within a predetermined amount of time, the application uses only either a full gesture reduction or a full gesture as input for the command. 17. The system of claim 16, wherein the memory further comprises instructions executed by the processor, which, when executed by the processor, instruct the system to at least receive user input corresponding to the definition of a user's movement or posture used to perform a full gesture reduction. 18. The system of claim 16, wherein the abbreviation of the full gesture corresponds to a second gesture that identifies the second command, the first output identifying that the second command has been called. 19. The system of claim 16, wherein the first output shows a call to the said command, the application associating the call to this command with a result or achievement, while the instructions executed by the processor, which, when executed by the processor, instruct the processor to at least send the first output the data to the application further instructs the system to at least send the first output to the application, while the application associates the first output with a smaller result or achievement than the result or and the achievement that the said team had when it was called through a complete gesture. 20. The system of claim 16, wherein the first output contains a confidence level that identifies the likelihood that said movement or posture of the user corresponds to the user performing a full gesture reduction, and in response to receiving the first output, the application recognizes that said the command was called, and in response to receiving the second output, the application adds details to the called command based on the second output.

Images