WO2010103482A2 - Interfaçage 3d amélioré pour dispositifs distants - Google Patents

Interfaçage 3d amélioré pour dispositifs distants Download PDF

Info

Publication number
WO2010103482A2
WO2010103482A2 PCT/IB2010/051055 IB2010051055W WO2010103482A2 WO 2010103482 A2 WO2010103482 A2 WO 2010103482A2 IB 2010051055 W IB2010051055 W IB 2010051055W WO 2010103482 A2 WO2010103482 A2 WO 2010103482A2
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
user
display screen
user interface
arc
Prior art date
Application number
PCT/IB2010/051055
Other languages
English (en)
Other versions
WO2010103482A3 (fr
Inventor
Aviad Maizels
Alexander Shpunt
Shai Litvak
Original Assignee
Primesense Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Primesense Ltd. filed Critical Primesense Ltd.
Publication of WO2010103482A2 publication Critical patent/WO2010103482A2/fr
Publication of WO2010103482A3 publication Critical patent/WO2010103482A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Definitions

  • This invention relates generally to user interfaces for computerized systems and specifically to user interfaces with three-dimensional characteristics.
  • tactile interface devices include the computer keyboard, mouse and joystick.
  • Touch screens detect the presence and location of a touch by a finger or other object within the display area.
  • Infrared remote controls are widely used, and "wearable" hardware devices have been developed, as well, for purposes of remote control.
  • U.S. Patent 7,348,963 whose disclosure is incorporated herein by reference, describes an interactive video display system, in which a display screen displays a visual image, and a camera captures three-dimensional information regarding an object in an interactive area located in front of the display screen. A computer system directs the display screen to change the visual image in response to the object.
  • An embodiment of the invention provides a method for operating a computerized system, which is carried out by presenting user interface elements on a display screen of the computerized system and detecting a first gesture made in a three-dimensional space by a distal portion of an upper extremity of a user while a segment of the distal portion thereof rests on a surface.
  • a first gesture made in a three-dimensional space by a distal portion of an upper extremity of a user while a segment of the distal portion thereof rests on a surface.
  • a first gesture made in a three-dimensional space by a distal portion of an upper extremity of a user while a segment of the distal portion thereof rests on a surface.
  • a first gesture made in a three-dimensional space by a distal portion of an upper extremity of a user while a segment of the distal portion thereof rests on a surface.
  • a second gesture made by the distal portion while the segment continues to rest on the surface is detected so as to select one of the user interface
  • the method further includes mapping an operation to the corresponding user interface element, wherein the second gesture causes the operation to be performed.
  • the method further includes mapping a three- dimensional location of the distal portion of the upper extremity to two parameters of a two- dimensional parametric surface, which is a section of a sphere that corresponds to possible locations of the distal portion of the upper extremity while the segment of the distal portion rests on a surface, and mapping the two parameters to corresponding parameters in a planar two-dimensional coordinate system of the display screen where the user interface elements are located
  • the first gesture describes a first arc-like movement forming at least a portion of a horizontal circular arc mapped to the section of the sphere as a pointing command
  • the second gesture includes a second arc-like movement mapped to the section of the sphere as a selection command.
  • the pointing command includes pointing to a letter on the display screen and the selection command includes inputting the letter to the system.
  • a threshold is defined, and includes displaying a subset of letters when a magnitude of the arc-like movement is less than the threshold and inputting the letter when the magnitude of the arc-like movement is greater than the threshold.
  • the subset of letters is shifted on the display screen using a language model to determine a probability of a preferred letter, and wherein inputting the letter is performed in a single continuous motion with high probability.
  • the segment of the distal portion may include an elbow, a wrist, or a forearm.
  • An embodiment of the invention provides a method for operating a computerized system, which is carried out by presenting user interface elements on a display screen of the computerized system and detecting a first gesture made in a three-dimensional space by a part of a body of a user. An area of the display screen selected by the user is identified responsively to the first gesture, and a magnification level of one or more of the user elements appearing in the selected area on the display screen is increased. After increasing the magnification level, a second gesture made by the part of the body of the user is detected so as to select one of the user elements that appear in the selected area.
  • a third gesture made by the part of the body is detected, and the magnification level is decreased in response to the third gesture.
  • the first and the third gesture include circular motions of a hand of the user in opposite, respective directions.
  • detecting the second gesture includes actuating a shortcut on the display in response to the second gesture.
  • selecting the magnified alphanumeric symbol includes adding the selected magnified alphanumeric symbol to a word spelled on the display screen, wherein the method includes detecting a third gesture made by the part of the body, opposite to the second gesture, and removing one or more symbols from the word in response to the third gesture.
  • the first gesture includes a three- dimensional movement by the part of the body of the user.
  • presenting user interface elements includes displaying a plurality of symbols arranged in at least one arc.
  • displaying the plurality of symbols includes presenting a set of symbols in a first arc, and increasing the magnification level includes presenting a magnified subset of the set of symbols in a second arc adjacent to the first arc.
  • detecting the first gesture includes detecting an arcuate movement of a hand of the user, and associating the arcuate movement with the plurality of symbols in the at least one arc.
  • presenting user interface elements includes presenting a sequence of textual characters, and increasing the magnification level includes displaying further characters for addition to the sequence using a language model to select the further characters.
  • detecting the first gesture includes scrolling forward or backward along the sequence responsively to first movements of a hand of the user in first and second directions along the sequence
  • detecting the second gesture includes selecting the further characters for addition to the sequence in response to second movements of the hand in at least a third direction perpendicular to the first and second directions.
  • An embodiment of the invention provides a computer software product for operating a computer system, including a sensing device, which is configured to detect at least a part of a body of a user, a display screen, which is configured to present user interface elements, and a processor, which is coupled to the sensing device so as to detect a first gesture made in a three-dimensional space by the part of the body.
  • the processor is additionally configured to identify an area of the display screen selected by the user in response to the first gesture, and to increase a magnification level of one or more of the user interface elements appearing in the selected area on the display screen, and after increasing the magnification level, to detect a second gesture made by the part of the body so as to select one of the user interface elements that appears in the selected area.
  • An embodiment of the invention provides a computer software product for operating a computerized system, including a computer storage medium in which computer program instructions are stored, which instructions, when executed by a computer, cause the computer to present user interface elements on a display screen of the computerized system, to detect a first gesture made in a three-dimensional space by a part of a body of a user, to identify an area of the display screen selected by the user in response to the first gesture, and to increase a magnification level of one or more of the user interface elements appearing in the selected area on the display screen. After increasing the magnification level, the instructions cause the computer to detect a second gesture made by the part of the body of the user so as to select one of the user interface elements that appears in the selected area.
  • a method for operating a computerized system including the steps of presenting user interface elements on a display screen of the computerized system and detecting a gesture made in a three-dimensional space by a part of a body of a user. While the user performs the gesture, one or more of the user interface elements on the display screen are continuously modified responsively to a direction of the gesture.
  • continuously modifying the one or more of the user interface elements includes increasing or decreasing a magnification level of at least one of the user interface elements, typically by zooming in on a user interface element toward which the gesture is directed.
  • presenting the user interface elements includes presenting a sequence of textual characters, which is continuously modifying by adding characters to the sequence while scrolling over the sequence responsively to the gesture.
  • Adding the characters typically includes presenting choices of further characters to add to the sequence, using a language model to determine the choices, and selecting at least one of the choices responsively to the gesture.
  • Presenting the choices may include determining, based on the language model, a respective likelihood of correctness of each of the choices, and displaying the choices so that an effort required by the user to select a given choice is a decreasing function of the likelihood.
  • continuously modifying the one or more of the user interface elements may include scrolling forward or backward along the sequence responsively to first movements of a hand of the user in first and second directions along the sequence, and selecting further characters for addition to the sequence responsively to second movements of the hand in at least a third direction perpendicular to the first and second directions.
  • FIG. 1 is a schematic, pictorial illustration of a system for remote gesture- mediated information input, in accordance with an embodiment of the present invention
  • Fig. 2 is a view of portions of a system operating under remote control of a user in accordance with an embodiment of the present invention
  • FIG. 3 is a view of portions of a system operating under remote control of a user in accordance with an embodiment of the present invention
  • FIG. 4 is a schematic, pictorial illustration of a system for remote information input, in accordance with an embodiment of the present invention.
  • FIG. 5 is a view of portions of a system operating under remote control of a user in accordance with an embodiment of the present invention
  • FIG. 6 is a view of portions of a system operating under remote control of a user in accordance with an embodiment of the present invention.
  • FIG. 7 is a view of portions of a system operating under remote control of a user in accordance with an embodiment of the present invention.
  • Fig. 8 is a flow chart of a method for remotely interfacing with a computer system, in accordance with an embodiment of the present invention.
  • Embodiments of the present invention that are described hereinbelow provide improved methods, products and systems for providing remote input to an electronic device. When a user interacts remotely with a device that requires accurate input and navigation decisions, these embodiments provide an intuitive and streamlined interface.
  • Remote input may be provided for interaction with a remote device such as a gaming console, an interactive television, a computerized cellular phone, or a computer.
  • a remote device such as a gaming console, an interactive television, a computerized cellular phone, or a computer.
  • the term "remote device” herein refers to any remotely governable device containing a processing unit.
  • a sensing device may be used to detect a virtual control, such as a virtual keyboard.
  • the sensing device is typically a three- dimensional camera that detects information that includes the position of a body (or at least parts of the body) of the user or other tangible entities wielded or operated by the user for interacting with a computer application running on the remote device, all of which are sometimes referred to herein for convenience as "control entities”.
  • the sensing device detects the presence and changes of position of a control entity, i.e. its speed and direction.
  • the remote device interprets movements detected by the sensing device as described hereinbelow.
  • the virtual control is positioned in front of the user, usually between the user and a display.
  • the remote device presents user interface elements on the display.
  • the sensing device detects the movements of the control entity in a three-dimensional space, such as a user's hand manipulating the virtual control, and translates them into commands for the remote device. For example, movement of the control entity using a circular gesture may be interpreted by the remote device as a command to adjust a magnification (or zoom) level of a remote information input interface comprising the user interface elements on the display.
  • magnification in this context is not limited to simple visual magnification: New details or relevant potentially selected options may be exposed.
  • a distance between the control entity and the sensing device varies as a gesture, e.g., the circular gesture. Movements in which the distance between the control entity and the sensing device remains substantially constant are classified as either “1- dimensional” or "2 -dimensional movements”.
  • Performing a clockwise circular gesture with the control entity may command the remote device to increase the magnification level of the display, whereas a counter-clockwise circular movement may cause minification.
  • clockwise and “counterclockwise” are used arbitrarily herein to distinguish two meaningful gestures. These terms have no necessary physical meanings with respect to the actual configuration of the embodiments.
  • the remote device causes the interface to zoom in and out, with a potentially infinite zoom range.
  • the actual distance between the user and the remote device remains largely constant.
  • the result of this arrangement is that the interface is essentially three-dimensional, and the elements of the interface may be continuously regrouped in three-dimensional space so as to move toward or away from the user as required, depending on the direction of the user's gesture.
  • the display may zoom in on a user interface element toward which a gesture is directed. This zoom may continue until the user's hand reaches the virtual location of the user element in the three-dimensional space, whereupon the element is selected (with or without an additional selection gesture).
  • the virtual control may be a virtual keyboard for remote information input, such as remote text input.
  • the display presents a remote information input interface representing the virtual keyboard.
  • the symbol positions e.g., the relative positions of the letters in the virtual keyboard, may be held constant.
  • the relative size of the symbols may be varied on the display in response to movements by the control entity.
  • the distance of the control entity from the sensing unit and the relative motion of the control entity with respect to the sensing unit may be interpreted as a manipulation of the virtual control, and hence as commands to the remote device as described hereinbelow.
  • the virtual control may be interpreted by the remote device to be constructed or oriented so as to accommodate right or left handed operation as the case may be.
  • the positions of the symbols in the virtual keyboard may be held constant, while the relative size of the symbols may be varied to reflect expected relevance or likelihood of each symbol being a suggested next symbol for remote input, e.g., based upon context awareness.
  • Movement of the control entity forward towards the sensing unit may cause a selected symbol, e.g., a letter to be input. Movement of the control entity away from the sensing unit may cause the previously input symbol to be erased. Selection is typically performed in two stages. A first selection stage may be used to determine a specific group of symbols. A second selection stage is used to input an individual symbol from the specific group of symbols. In some embodiments, the first selection stage is made by an arc-like movement of the control entity, e.g., clockwise or counter-clockwise, followed by a slight movement towards the sensing unit.
  • a threshold which may be fixed or adaptive, is defined to enable the remote device to detect completion of each selection stage.
  • the remote device interprets a movement with a magnitude less than the threshold as completion of the first selection stage.
  • the remote device interprets the movement as completion of the second stage.
  • the values of the threshold may depend upon the resolving capabilities of the sensing device, and are typically set so as to detect significant motion, while ignoring "jitter" by the control entity. In some cases, there is no pause between the stages, e.g., when a single continuous movement is performed.
  • the second selection stage may constitute an additional arc-like movement by the control entity towards the sensing unit and downwards, as described in more detail hereinbelow. [0046] Selection stages may be combinations of continuous and discrete, curved and linear movements in many directions with respect to the sensing unit.
  • the user can execute the combinations while resting his elbow (or other relatively proximal segment of his upper extremity, e.g., a wrist or forearm) on a surface such as an armrest of a chair or a table, while using a distal portion as the control entity.
  • the selection may be performed by the user's hand, fingers or forearm, or combinations thereof without tiring the arm.
  • shortcut refers to an option on a remote information input interface that appears upon user interaction, which offers a choice that is available in the current state, or context, of the computer application. That is to say, a user interface element, e.g. a shortcut, may be mapped to an operation to be performed upon selection of the shortcut.
  • the operation may cause the word shown in the shortcut to be input to the remote device.
  • shortcuts include a context menu that appears upon a mouse click operation such as a "right-click", and an iconic link whose activation triggers some function in a remote device.
  • the activation threshold may be customized to vary according to specific symbols and application scenarios.
  • the display may be altered to reflect potential likelihood or relevance of a next letter, group of letters, or symbol based upon an analysis of previously input symbols, e.g., by highlighting likely symbols on the display.
  • a subset of letters may be shown on the display to simplify navigation, and may themselves be linked to shortcuts.
  • the relative location of the subset of letters may be shifted on the display so as to enable the user to select a preferred letter in a single selection motion. Typically, the relative location of the subset of letters is altered without any change to the order of the letters.
  • the distance and motion relative to the display unit, rather than to the sensing unit are used for interpreting the commands.
  • Fig. 1 is a schematic, pictorial illustration of a system 10 for gesture-mediated remote information input, in accordance with an embodiment of the present invention.
  • System 10 incorporates a sensing device 12, typically a three-dimensional camera, which detects information that includes the body (or at least parts of the body) of a user 14 or other control entities for controlling a computer application as described hereinabove.
  • the control entity is described using an example of a hand 16 for providing remote input to system 10.
  • control entities could include portions of objects being manipulated by user 14, e.g., as hockey sticks, golf clubs, bows, and tennis rackets.
  • sensing device 12/352,622, filed January 13, 2009, which is hereby incorporated by reference, is suitable for use in system 10 as sensing device 12.
  • Other known three-dimensional cameras may also be employed as sensing device 12.
  • this embodiment relates to one particular system for providing remote information input, the principles of providing remote information input that are implemented in system 10 may similarly be applied, mutatis mutandis, in other types of remote information input or gesture control systems, using other techniques for providing remote information input or remote control via specialized gesture.
  • sensing device 12 Information detected by sensing device 12 is processed by a remote device 18, which drives a display screen 20 accordingly.
  • Sensing device 12 is connected to remote device 18 via a sensing interface 22, which may comprise a Bluetooth® adapter, an Infrared Data Association (IrDA) device, a cable connection, a universal serial bus (USB) interface, or any communication interface for outputting sensor data that allows remote device 18 to import remote sensing data.
  • Remote device 18 typically comprises a general-purpose computer processor, which is programmed in software to carry out the functions described hereinbelow.
  • the software may be downloaded to the processor in electronic form, over a network, for example, or it may alternatively be provided on tangible storage media, such as optical, magnetic, or electronic memory media.
  • remote device 18 is shown in Fig. 1, as a separate unit from sensing device 12, some or all of the processing functions of remote device 18 may be performed by suitable dedicated circuitry within the housing of sensing device 12 or otherwise associated with sensing device 12.
  • Display screen 20 presents user interface elements comprising a pointer 24 and a remote information input interface 26, which comprises symbols 28, 30, 32, 34, 36.
  • a display interface 38 connects display screen 20 to remote device 18, and may comprise a Bluetooth® adapter, an IrDA device, a cable connection, or any communication interface for outputting image data that allows remote device 18 to export visual display data, e.g., in the form of a compressed image.
  • the symbol selection layout provides a simplified example for the purposes of illustration. In the present example, symbols 28, 30, 32 represent numerals, whereas symbols 34 and 36 represent actions. Each symbol may be remotely selected or actuated to control the computer application.
  • Remote information input interface 26 may also comprise a zoom level indicator 40 to provide a visual indicator of the zoom level of remote information input interface 26.
  • Zoom level indicator 40 may be shown as a slider, similar to sliders utilized in web browsers and other applications.
  • the zoom level is typically allowed to range within certain limits, e.g., from 50% to 500%.
  • zoom level indicator 40 shows an initial value of 100%.
  • a scale indicator 42 shows symbols 28, 30, 32 having an initial height of one unit on remote information input interface 26.
  • FIG. 2 is a view of portions of system 10 (Fig. 1) operating under remote control of user 14 in accordance with an embodiment of the present invention.
  • Fig. 1 and Fig. 2 may be viewed as a sequence of actions.
  • user 14 is about to perform a specialized gesture, and in Fig. 2 completes the gesture.
  • User 14, who is typically viewing display screen 20, performs the specialized gesture, e.g., a clockwise circular gesture from the perspective of user 14, using hand 16 as described on a reference coordinate system 44 by directed broken lines 46.
  • the specialized gesture e.g., a clockwise circular gesture from the perspective of user 14, using hand 16 as described on a reference coordinate system 44 by directed broken lines 46.
  • other types of hand movements may be used to invoke image zoom and other functions, such as moving the hand toward and away from the display screen.
  • the x, y, and z axes are to be interpreted as horizontal, vertical, and depth coordinates, respectively, with respect to sensing device 12.
  • the distance between hand 16 and sensing device 12 vary as the specialized gesture is performed.
  • the inclination of the plane of the circle may be significant, according to specifications programmed in remote device 18.
  • a circle described vertically in the yz plane may be interpreted by remote device 18 differently from a circle in the xy plane or a horizontally executed circular gesture, e.g., as a pointing command.
  • Substantially circular gestures described in various oblique planes may be given even more specialized interpretations.
  • the clockwise circular gesture is recognized by sensing device 12, and remote device 18 interprets the gesture as a zoom command.
  • the clockwise circular gesture thus commands remote device 18 to smoothly increase the zoom (or magnification) level of remote information input interface 26 on display screen 20 using pointer 24 as a reference point for the zooming. That is to say, remote device 18 identifies an area of display screen 20 around pointer 24 as having been selected by user 14 for the zoom command.
  • hand 16 may move from an initial position 48 to a final position 50.
  • Scale indicator 42 shows symbols 28, 30, 32 having a final height of 1.5 units on remote information input interface 26.
  • zoom level indicator 40 shows that, in comparison with Fig. 1 , the clockwise circular gesture has increased the zoom level from the initial value of 100% immediately prior to the gesture to a final value of 150%.
  • a corresponding increase in the size of symbols 28, 30, 32 is shown on display screen 20 as a result of the zoom command. That is to say, one or more of the user interface elements appearing in the selected area on display screen 20 are magnified.
  • Fig. 3 is a view of portions of system 10 (Fig. 1) operating under control of user 14 in accordance with an embodiment of the present invention.
  • user 14 performs a leftward, substantially horizontal gesture using hand 16 as indicated by an arrow 52 on reference coordinate system 44.
  • the leftward gesture commands remote device 18 to move pointer 24 to the left on remote information input interface 26.
  • the leftward movement of pointer 24 (Fig. 2) executes from a first position (indicated by a cursor 54 (shown in broken outline) to a second position, indicated by a cursor 56 (shown in solid outline) as a result of the command.
  • hand 16 may move from an initial position 58 to a final position 60.
  • a rightward gesture may be interpreted as a command to move pointer 24 to the right from the perspective of user 14, while gestures performed upward and downward may similarly be interpreted by remote device 18 as commands to move pointer 24 upward and downward, respectively.
  • Suitable calibration of sensing device 12 and remote device 18 assures a desired sensitivity, i.e., a correspondence between a spatiotemporal displacement of the control entity and the effect on elements shown on remote information input interface 26. It is recommended to compensate for the viewing distance and viewing angle of user 14 using known methods. The compensation techniques described in U.S. Patent Application Publication No. 2009/0009593, entitled “Three-dimensional Projection Display” may be applied for this purpose.
  • FIG. 4 is a schematic, pictorial illustration of system 10 (Fig. 1) for remote information input, in accordance with an embodiment of the present invention.
  • a first symbol arc 62 is shown within remote information input interface 26 on display screen 20, in an embodiment that implements a T9® text input layout.
  • T9 text input represents "text on 9 keys," a method for streamlining input of text on numeric keypads, typically for mobile devices, available at the T9 web site (t9.com).
  • T9 text input represents "text on 9 keys," a method for streamlining input of text on numeric keypads, typically for mobile devices, available at the T9 web site (t9.com).
  • First symbol arc 62 comprises an arcuate, nearly semi-circular display of groups of alphanumeric symbol buttons 64 to simulate relaxed movement of hand 16 while user 14 sits comfortably on a chair 66.
  • First symbol arc 62 may comprise additional symbol buttons 68 to support input of special symbols, e.g., space, backspace, or carriage return.
  • Sensing device 12 detects the movement and remote device 18 interprets the movement by highlighting each of additional symbol buttons 68 and alphanumeric symbol buttons 64 sequentially as hand 16 moves through semi-circular arc 70 from a first position 72 to a second position 74. Provision of an arcuate display enables hand 16 to move while an elbow 118 of the same upper extremity as hand 16 rests on chair 66.
  • a portion of a sphere 122 is shaded within a spherical coordinate system 124 using an axis based upon elbow 118 to indicate an approximate range of motion of hand 16 when elbow 118 rests on a surface.
  • a three-dimensional space is mapped to spherical coordinate system 124, and is also mapped to a two-dimensional coordinate system. The latter can be conveniently appreciated as a plane in reference coordinate system 44.
  • the movement of semi-circular arc 70 be parallel to the xy plane in reference coordinate system 44. However, the movement may also be made so that the angle between a plane of the motion of semi-circular arc 70 and the xy plane is above 0 degrees, typically up to 45 degrees.
  • an emphasized symbol button 76 is shown on display screen 20 to indicate that performance of a second gesture, described hereinbelow as a selection gesture, will result in selection of the symbols displayed therein. That is to say, remote device 18 identifies emphasized symbol button 76 as the area of display screen 20 that is currently selected by user 14.
  • Corresponding selection gestures performed by hand 16 at other points along semi-circular arc 70 would select corresponding symbols of first symbol arc 62.
  • the first gesture and the second gesture may be recognized by remote device 18 according to time-varying coordinates on the two-dimensional coordinate system and the spherical coordinate system, respectively.
  • Fig. 5 is a view of portions of system 10 (Fig. 1) operating under remote control of user 14 in accordance with an embodiment of the present invention.
  • the selection gesture is typically performed in two stages, as described hereinabove.
  • User 14 may perform a first stage of the selection gesture by moving hand 16 downward in a vertical arc 78 generally directed toward display screen 20.
  • Remote device 18 uses the threshold, described hereinabove, to determine completion of each selection stage.
  • the selection gesture may pivot about the elbow or shoulder, whichever is applicable. Of course, when pivoting about the shoulder the advantages of resting a portion of the arm on a surface are lost.
  • hand 16 may move from an initial position 80 to an intermediate position 130 while performing the first stage, and then to a final position 82 while performing a second stage.
  • Sensing device 12 detects the movement, and remote device 18 interprets the selection gesture as a command to display a second symbol arc 84 directly below first symbol arc 62, comprising individually delineated symbol buttons 86, which are grouped together in highlighted symbol button 76.
  • user 14 next moves hand 16 in another arc-like movement, which is detected by sensing device 12.
  • Remote device 18 interprets the movement by highlighting each of individually delineated symbol buttons 86 as described hereinabove.
  • Fig. 4 and Fig. 5 may be viewed as a sequence of actions, whereby in Fig. 4 user 14 selects an area of display screen 20, e.g., one of additional symbol buttons 68, and in Fig. 5 selects one of the user interface elements, e.g., highlighted symbol button 76, in order to display second symbol arc 84 and to input one of individually delineated symbol buttons 86.
  • arcuate displays like first symbol arc 62 for remotely inputting information may provide particularly enhanced ergonomic value.
  • the motions involved in their use for remote information input are not fatiguing, e.g., in comparison with a standard "QWERTY" keyboard layout.
  • Virtual keyboard layouts such as the QWERTY keyboard layout may not as conveniently permit remote information input with a resting or fixed elbow position.
  • the selection gesture is made by moving hand 16 downwards. That is, it involves a forward displacement of the hand in the z-axis with respect to sensing device 12.
  • the motion involves both a component in the xy plane and a component in yz plane.
  • user 14 may move hand 16 directly downwards, such that only a component in the yz plane exists.
  • remote device 18 is able to distinguish the selection gesture from other linear or curved movements in the xy plane alone, by calculating the displacement of the hand along the z-axis.
  • remote device 18 interprets the movement as a selection gesture and ignores the motion component in the xy plane. In both cases, remote device 18 may provide enhanced ergonomic value when recognizing these selection gestures, as they allow user 14 to use a natural selection motion, as indicated by the location of hand 16 on the xy axis.
  • remote device 18 may cause a corresponding user interface element, e.g., shortcuts 96, 98 to be shown on remote information input interface 26, offering suggestions for completing a word.
  • a corresponding user interface element e.g., shortcuts 96, 98
  • letters "S" and "A” have been previously input, and the autocomplete feature of remote device 18 provides shortcuts 96 and 98 for selection.
  • Previously input information may be emphasized on shortcuts 96 and 98.
  • FIG. 6 is a view of portions of system 10 (Fig. 1) operating under remote control of user 14 in accordance with an embodiment of the present invention.
  • a domain-specific language model 132 may be used to determine the probability of a symbol or next letter being preferred by user 14.
  • language model 132 is shown as a computer program module operated by device 18.
  • user 14 has previously input the letters "INVENTIO”.
  • User 14 next moves hand 16 over emphasized symbol button 76.
  • Device 18 uses the domain-specific language model and determines that the probability of a letter "N" is significantly higher than another letter shown in emphasized symbol button 76, as shown in a shortcut 126.
  • device 18 uses the domain-specific language model to shift an adjusted second symbol arc 128 to place the letter with a highest probability beneath emphasized symbol button 76. Both stages of the selection gesture may be performed by user 14 in a continuous motion to select a preferred letter, thus minimizing required movement by hand 16.
  • language model herein refers to any suitable statistical model for assigning a probability to a sequence of letters or words by means of a probability distribution.
  • Fig. 7 is a view of portions of system 10 (Fig. 1) operating under remote control of user 14 in accordance with an embodiment of the present invention.
  • User 14 may choose to perform a deselection gesture after inputting information remotely, or to deselect second symbol arc 84.
  • the deselection gesture may comprise raising hand 16 as indicated by an upwardly directed vertical arc 100 which is a reversal of vertical arc 78 (Fig. 5).
  • Sensing device 12 detects the deselection gesture, and remote device 18 interprets the movement as a command to cancel the selection of second symbol arc 84 (Fig. 5) which has responsively been removed from remote information input interface 26 in Fig. 7.
  • any movement by hand 16 after the deselection gesture has been performed is interpreted by remote device 18 as a command to resume highlighting alphanumeric symbol buttons 64 on first symbol arc 62 for selection.
  • hand 16 moves from an initial position 102 to a final position 104.
  • remote device 18 automatically removes second symbol arc 84 without requiring the deselection gesture to be performed.
  • remote information input requires less movement by user 14 than in the previous embodiment.
  • Device 18 typically requires user 14 to return hand 16 to final position 104 before recognizing a new selection.
  • Embodiments of the present invention that utilize the T9 text input layout as symbol arcs on remote information input interface 26 may provide an advantage whereby input is provided remotely without the need to move the control entity in three dimensions.
  • moving hand 16 in an arcuate motion along semi-circular arc 70 (Fig. 4) is interpreted by remote device 18 as movement within two dimensions, e.g., leftward, rightward, upward and downward.
  • semi-circular arc 70 By limiting semi-circular arc 70 to motions substantially parallel to the xz plane, 3 -dimensional interpretation issues are avoided.
  • a complex movement in three dimensions, e.g., to perform the point-and-click gesture is not required, thus simplifying interpretation of the gesture and thereby facilitating remote information input.
  • Fig. 8 is a flow chart of a method for remotely interfacing with a computer system, in accordance with an embodiment of the present invention.
  • user 14 may need to search a large volume of media without using a physical keyboard or other interface connected to a remote device.
  • User 14 would thus need to perform efficient, streamlined search commands to interact remotely with a computer application running on the remote device.
  • the process steps are described below in a particular linear sequence for clarity of presentation. However, it will be evident that some of them can be performed in parallel, asynchronously, or in different orders. The process can be performed, for example, by system 10.
  • User interface elements comprising a remote information input interface to a computer application are presented to a user on a display screen in a display presentation step 106.
  • the computer application may be a media search and presentation system. It is assumed that the computer application has been loaded, and that a three-dimensional sensing device is in operation.
  • the sensing device can be any three-dimensional sensor or camera, provided that it generates data for interpretation by the remote device.
  • the user performs a first gesture in a three-dimensional space using a control entity, e.g., a part of the user's body.
  • a sensing device such as sensing device 12 (Fig. 1), detects the gesture made by the control entity, e.g., hand 16, in a gesture detecting step 108.
  • the computer iteratively analyzes three-dimensional data provided by the sensing device, for example by constructing a three-dimensional map as described in commonly assigned co-pending U.S. Application No. 12/683,452, which is herein incorporated by reference.
  • an area of the display screen is identified by the computer in a selected area identification step 110.
  • the first gesture is recognized by the computer as a command to increase the magnification level of user interface elements within the selected area on the display screen in a magnification level adjusting step 112.
  • Any gesture recognition algorithm may be employed to carry out magnification level adjusting step 112, so long as the system can relate the user gesture to a recognized command and a location of interest on the remote information input interface.
  • a second gesture is recognized by the computer as a command to select one of the user interface elements within the selected area in a selection gesture detecting step 114.
  • the second gesture can be for any purpose, for example to perform another zoom command, to input a symbol, or to alter the remote information input interface in accordance with the gesture identified.
  • the clockwise circular gesture command described with respect to Fig. 2 might correspond to an instruction to increase the zoom level of the remote information input interface on the display screen, while a counter-clockwise circular gesture, in which the direction of the motion is reversed, could result in an instruction to decrease the zoom level.
  • Many such combinations will occur to a developer of computer applications or other signal processing systems.
  • An updated display screen results, and is shown in subsequent iterations of the method. In practice the process iterates so long as the remote device is active or some error occurs.
  • the method then terminates at a final step 116.
  • the circular gestures comprise requiring at least one complete circle to be performed by the control entity before the zoom level is changed.
  • multiple control entities are used to perform the specialized gesture.
  • the zoom command may be input using a second hand (not shown) to complement hand 16 (Fig. 1).
  • remote device 18 recognizes the second hand by analyzing input from sensing device 12, movement of the second hand farther away from hand 16 may be interpreted as the zoom command to increase the zoom level, and vice versa.
  • Using multiple control entities may provide an advantage wherein pointer 24 is not moved prior to the change in zoom level. Thus, the changes in magnification may be performed around pointer 24.
  • language model 132 is used to assign a probability to each letter on remote information input interface 26.
  • Device 18 may order the letters accordingly, displaying the letters in a continuous ungrouped series of individual letters, rather than in groups, e.g., the group of three letters displayed in emphasized symbol button 76.
  • Device 18 may invite access to letters on the display having relatively high probabilities, e.g., by presenting them in closer proximity to the center of remote information input interface 26.
  • letters having relatively high and low probabilities may be grouped together and placed into secondary symbol arcs (not shown).
  • the spatial distribution of letters in a symbol arc may reflect their respective probabilities.
  • letters having relatively high and low probabilities of selection may be spaced apart and crowded together, respectively, in varying degrees.
  • text input created by the user is shown as a linear stream of characters running across the screen, from left to right, for example.
  • the user may perform a special gesture, recognized by the remote device, to mark a neutral reference position. Movement of the user's hand in a direction along the sequence, such as to the right of the reference position will then cause the display to advance to the right along the text stream whereas movement to the left will scroll backward through the text stream.
  • the scroll speed presented by the remote device on the display may initially be slow when the user gestures sideways to the right or left and may gradually accelerate the longer the user's hand is in the advance or reverse position.
  • the above-mentioned language model may be used to display alternative choices of additional characters and even words to append to the stream. These choices may be displayed above and/or below the existing line of characters, with the likeliest choices typically vertically closest to the line and possibly magnified.
  • the user selects the desired choice by upward and downward motions of the hand, perpendicular to the direction of the text sequence.
  • the user can add text quickly and efficiently using simple right/left and up/down motions.
  • the user's right/left and up/down hand motions may be made in a generally planar space or, if the elbow is resting on a surface as in some of the embodiments described above, may be over a generally spherical surface.
  • the right/left and up/down motions are not limited to a two-dimensional plane, but may be mapped to a two-dimensional coordinate system by the remote device. Within this latter coordinate system, one dimension of hand movement controls the speed of scrolling forward and back (wherein backward movement may delete characters previously appended to the stream), while the other dimension controls the selection of new characters.
  • This sort of embodiment may be used to present and add text input in a sort of continuous "flight mode": As the user scrolls to the right (forward) to add text to the stream being created, various potential continuations of the existing text are presented to the right of the existing text, above and/or below the text line.
  • the potential continuations may be ordered or otherwise presented in such a way that the effort necessary to select a given continuation is a decreasing function of the likelihood that the given continuation is the correct one, based on the language model (higher likelihood yields lower effort). For example, likelier continuations may be presented with larger size and/or in closer proximity to the current cursor position.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)

Abstract

Le fonctionnement d'un système informatisé (10) comprend la présentation d'éléments d'interface utilisateur (28, 30, 32, 34, 36) sur un écran d'affichage (20). Un premier geste fait dans un espace tridimensionnel par une partie (16) d'un corps d'un utilisateur est détecté. En réponse au premier geste, une zone de l'écran d'affichage sélectionnée par l'utilisateur est identifiée, et un niveau d'agrandissement d'un ou de plusieurs des éléments d'utilisateur apparaissant dans la zone sélectionnée sur l'écran d'affichage est augmenté. Après augmentation du niveau d'agrandissement, un second geste fait par la partie du corps de l'utilisateur est détecté de façon à sélectionner l'un des éléments d'interface utilisateur qui apparaissent dans la zone sélectionnée.
PCT/IB2010/051055 2009-03-13 2010-03-11 Interfaçage 3d amélioré pour dispositifs distants WO2010103482A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15980809P 2009-03-13 2009-03-13
US61/159,808 2009-03-13

Publications (2)

Publication Number Publication Date
WO2010103482A2 true WO2010103482A2 (fr) 2010-09-16
WO2010103482A3 WO2010103482A3 (fr) 2010-11-04

Family

ID=42728878

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2010/051055 WO2010103482A2 (fr) 2009-03-13 2010-03-11 Interfaçage 3d amélioré pour dispositifs distants

Country Status (2)

Country Link
US (3) US20100235786A1 (fr)
WO (1) WO2010103482A2 (fr)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2474950A1 (fr) 2011-01-05 2012-07-11 Softkinetic Software Procédés et systèmes d'interface d'utilisateur à base de geste naturel
WO2013038089A1 (fr) 2011-09-16 2013-03-21 Prynel Procédé et système d'acquisition et de traitement d'images pour la détection du mouvement
US8615108B1 (en) 2013-01-30 2013-12-24 Imimtek, Inc. Systems and methods for initializing motion tracking of human hands
US8655021B2 (en) 2012-06-25 2014-02-18 Imimtek, Inc. Systems and methods for tracking human hands by performing parts based template matching using images from multiple viewpoints
US8830312B2 (en) 2012-06-25 2014-09-09 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching within bounded regions
WO2014200874A1 (fr) * 2013-06-13 2014-12-18 Microsoft Corporation Génération de texte au moyen d'une interface sans toucher
US9092665B2 (en) 2013-01-30 2015-07-28 Aquifi, Inc Systems and methods for initializing motion tracking of human hands
US9298266B2 (en) 2013-04-02 2016-03-29 Aquifi, Inc. Systems and methods for implementing three-dimensional (3D) gesture based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US9310891B2 (en) 2012-09-04 2016-04-12 Aquifi, Inc. Method and system enabling natural user interface gestures with user wearable glasses
EP3009919A1 (fr) * 2011-12-30 2016-04-20 Samsung Electronics Co., Ltd. Appareil électronique et son procédé de commande
US9507417B2 (en) 2014-01-07 2016-11-29 Aquifi, Inc. Systems and methods for implementing head tracking based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US9504920B2 (en) 2011-04-25 2016-11-29 Aquifi, Inc. Method and system to create three-dimensional mapping in a two-dimensional game
US9600078B2 (en) 2012-02-03 2017-03-21 Aquifi, Inc. Method and system enabling natural user interface gestures with an electronic system
US9619105B1 (en) 2014-01-30 2017-04-11 Aquifi, Inc. Systems and methods for gesture based interaction with viewpoint dependent user interfaces
US9798388B1 (en) 2013-07-31 2017-10-24 Aquifi, Inc. Vibrotactile system to augment 3D input systems
US9857868B2 (en) 2011-03-19 2018-01-02 The Board Of Trustees Of The Leland Stanford Junior University Method and system for ergonomic touch-free interface

Families Citing this family (108)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103778635B (zh) * 2006-05-11 2016-09-28 苹果公司 用于处理数据的方法和装置
US8726194B2 (en) 2007-07-27 2014-05-13 Qualcomm Incorporated Item selection using enhanced control
US9035876B2 (en) 2008-01-14 2015-05-19 Apple Inc. Three-dimensional user interface session control
US8933876B2 (en) 2010-12-13 2015-01-13 Apple Inc. Three dimensional user interface session control
US8555207B2 (en) 2008-02-27 2013-10-08 Qualcomm Incorporated Enhanced input using recognized gestures
US9482755B2 (en) 2008-11-17 2016-11-01 Faro Technologies, Inc. Measurement system having air temperature compensation between a target and a laser tracker
JP5093080B2 (ja) * 2008-12-05 2012-12-05 富士通モバイルコミュニケーションズ株式会社 携帯端末
US9898675B2 (en) * 2009-05-01 2018-02-20 Microsoft Technology Licensing, Llc User movement tracking feedback to improve tracking
US9232167B2 (en) * 2009-08-04 2016-01-05 Echostar Technologies L.L.C. Video system and remote control with touch interface for supplemental content display
US8565479B2 (en) * 2009-08-13 2013-10-22 Primesense Ltd. Extraction of skeletons from 3D maps
US8963829B2 (en) 2009-10-07 2015-02-24 Microsoft Corporation Methods and systems for determining and tracking extremities of a target
US7961910B2 (en) 2009-10-07 2011-06-14 Microsoft Corporation Systems and methods for tracking a model
US8867820B2 (en) * 2009-10-07 2014-10-21 Microsoft Corporation Systems and methods for removing a background of an image
US8564534B2 (en) 2009-10-07 2013-10-22 Microsoft Corporation Human tracking system
US8787663B2 (en) * 2010-03-01 2014-07-22 Primesense Ltd. Tracking body parts by combined color image and depth processing
US9772394B2 (en) 2010-04-21 2017-09-26 Faro Technologies, Inc. Method and apparatus for following an operator and locking onto a retroreflector with a laser tracker
US8537371B2 (en) 2010-04-21 2013-09-17 Faro Technologies, Inc. Method and apparatus for using gestures to control a laser tracker
US9377885B2 (en) 2010-04-21 2016-06-28 Faro Technologies, Inc. Method and apparatus for locking onto a retroreflector with a laser tracker
US9400170B2 (en) 2010-04-21 2016-07-26 Faro Technologies, Inc. Automatic measurement of dimensional data within an acceptance region by a laser tracker
US8724119B2 (en) 2010-04-21 2014-05-13 Faro Technologies, Inc. Method for using a handheld appliance to select, lock onto, and track a retroreflector with a laser tracker
US8619265B2 (en) 2011-03-14 2013-12-31 Faro Technologies, Inc. Automatic measurement of dimensional data with a laser tracker
US8422034B2 (en) 2010-04-21 2013-04-16 Faro Technologies, Inc. Method and apparatus for using gestures to control a laser tracker
US8594425B2 (en) 2010-05-31 2013-11-26 Primesense Ltd. Analysis of three-dimensional scenes
US8966400B2 (en) * 2010-06-07 2015-02-24 Empire Technology Development Llc User movement interpretation in computer generated reality
US20110304649A1 (en) * 2010-06-10 2011-12-15 Microsoft Corporation Character selection
US9009594B2 (en) * 2010-06-10 2015-04-14 Microsoft Technology Licensing, Llc Content gestures
US9600153B2 (en) * 2010-07-05 2017-03-21 Lg Electronics Inc. Mobile terminal for displaying a webpage and method of controlling the same
US9201501B2 (en) 2010-07-20 2015-12-01 Apple Inc. Adaptive projector
JP2012027515A (ja) * 2010-07-20 2012-02-09 Hitachi Consumer Electronics Co Ltd 入力方法及び入力装置
JP5791131B2 (ja) 2010-07-20 2015-10-07 アップル インコーポレイテッド 自然な相互作用のための相互作用的現実拡張
JP5625599B2 (ja) * 2010-08-04 2014-11-19 ソニー株式会社 情報処理装置、情報処理方法、及びプログラム
US8582867B2 (en) 2010-09-16 2013-11-12 Primesense Ltd Learning-based pose estimation from depth maps
EP2620849B1 (fr) * 2010-09-22 2019-08-28 Shimane Prefectural Government Appareil d'entrée d'opération, procédé d'entrée d'opération, et programme
US8959013B2 (en) 2010-09-27 2015-02-17 Apple Inc. Virtual keyboard for a non-tactile three dimensional user interface
US8872762B2 (en) 2010-12-08 2014-10-28 Primesense Ltd. Three dimensional user interface cursor control
KR101858531B1 (ko) 2011-01-06 2018-05-17 삼성전자주식회사 모션에 의해 제어되는 디스플레이 장치 및 그 모션 제어 방법
KR101795574B1 (ko) 2011-01-06 2017-11-13 삼성전자주식회사 모션에 의해 제어되는 전자기기 및 그 제어 방법
US8825734B2 (en) * 2011-01-27 2014-09-02 Egain Corporation Personal web display and interaction experience system
US9285874B2 (en) 2011-02-09 2016-03-15 Apple Inc. Gaze detection in a 3D mapping environment
US10338672B2 (en) * 2011-02-18 2019-07-02 Business Objects Software Ltd. System and method for manipulating objects in a graphical user interface
GB2518769A (en) 2011-03-03 2015-04-01 Faro Tech Inc Target apparatus and method
US9164173B2 (en) 2011-04-15 2015-10-20 Faro Technologies, Inc. Laser tracker that uses a fiber-optic coupler and an achromatic launch to align and collimate two wavelengths of light
JP2014516409A (ja) 2011-04-15 2014-07-10 ファロ テクノロジーズ インコーポレーテッド レーザトラッカの改良位置検出器
US9686532B2 (en) 2011-04-15 2017-06-20 Faro Technologies, Inc. System and method of acquiring three-dimensional coordinates using multiple coordinate measurement devices
US9482529B2 (en) 2011-04-15 2016-11-01 Faro Technologies, Inc. Three-dimensional coordinate scanner and method of operation
US9417703B2 (en) 2011-04-20 2016-08-16 Koninklijke Philips N.V. Gesture based control of element or item
US8854321B2 (en) * 2011-05-02 2014-10-07 Verizon Patent And Licensing Inc. Methods and systems for facilitating data entry by way of a touch screen
US9507427B2 (en) 2011-06-29 2016-11-29 Intel Corporation Techniques for gesture recognition
US9459758B2 (en) 2011-07-05 2016-10-04 Apple Inc. Gesture-based interface with enhanced features
US8881051B2 (en) 2011-07-05 2014-11-04 Primesense Ltd Zoom-based gesture user interface
US9377865B2 (en) 2011-07-05 2016-06-28 Apple Inc. Zoom-based gesture user interface
US9030498B2 (en) 2011-08-15 2015-05-12 Apple Inc. Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface
US9218063B2 (en) 2011-08-24 2015-12-22 Apple Inc. Sessionless pointing user interface
US9122311B2 (en) 2011-08-24 2015-09-01 Apple Inc. Visual feedback for tactile and non-tactile user interfaces
US9002099B2 (en) 2011-09-11 2015-04-07 Apple Inc. Learning-based estimation of hand and finger pose
KR101457116B1 (ko) * 2011-11-07 2014-11-04 삼성전자주식회사 음성 인식 및 모션 인식을 이용한 전자 장치 및 그의 제어 방법
KR20130081580A (ko) * 2012-01-09 2013-07-17 삼성전자주식회사 표시 장치 및 그 제어 방법
JP6099675B2 (ja) 2012-01-27 2017-03-22 ファロ テクノロジーズ インコーポレーテッド バーコード識別による検査方法
US9423877B2 (en) * 2012-02-24 2016-08-23 Amazon Technologies, Inc. Navigation approaches for multi-dimensional input
US9229534B2 (en) 2012-02-28 2016-01-05 Apple Inc. Asymmetric mapping for tactile and non-tactile user interfaces
CN104246682B (zh) 2012-03-26 2017-08-25 苹果公司 增强的虚拟触摸板和触摸屏
US9047507B2 (en) 2012-05-02 2015-06-02 Apple Inc. Upper-body skeleton extraction from depth maps
JP5962249B2 (ja) * 2012-06-21 2016-08-03 富士通株式会社 文字入力プログラム、情報処理装置および文字入力方法
US9019267B2 (en) 2012-10-30 2015-04-28 Apple Inc. Depth mapping with enhanced resolution
DE102012110460A1 (de) * 2012-10-31 2014-04-30 Audi Ag Verfahren zum Eingeben eines Steuerbefehls für eine Komponente eines Kraftwagens
KR102059598B1 (ko) * 2013-01-29 2019-12-26 삼성전자주식회사 디지털 촬영 장치 및 그의 제어 방법
KR20140099111A (ko) 2013-02-01 2014-08-11 삼성전자주식회사 카메라 장치의 동작을 제어하는 방법 및 상기 카메라 장치
US9052746B2 (en) * 2013-02-15 2015-06-09 Microsoft Technology Licensing, Llc User center-of-mass and mass distribution extraction using depth images
US9342230B2 (en) * 2013-03-13 2016-05-17 Microsoft Technology Licensing, Llc Natural user interface scrolling and targeting
KR101872426B1 (ko) * 2013-03-14 2018-06-28 인텔 코포레이션 깊이 기반 사용자 인터페이스 제스처 제어
US10180029B2 (en) * 2013-03-15 2019-01-15 Springs Window Fashions, Llc Window covering motorized lift and control system motor and operation
US9041914B2 (en) 2013-03-15 2015-05-26 Faro Technologies, Inc. Three-dimensional coordinate scanner and method of operation
US10620775B2 (en) * 2013-05-17 2020-04-14 Ultrahaptics IP Two Limited Dynamic interactive objects
US9436288B2 (en) 2013-05-17 2016-09-06 Leap Motion, Inc. Cursor mode switching
US20140380223A1 (en) * 2013-06-20 2014-12-25 Lsi Corporation User interface comprising radial layout soft keypad
US10152136B2 (en) 2013-10-16 2018-12-11 Leap Motion, Inc. Velocity field interaction for free space gesture interface and control
US20150121314A1 (en) * 2013-10-24 2015-04-30 Jens Bombolowsky Two-finger gestures
US9740296B2 (en) 2013-12-16 2017-08-22 Leap Motion, Inc. User-defined virtual interaction space and manipulation of virtual cameras in the interaction space
US20150201124A1 (en) * 2014-01-15 2015-07-16 Samsung Electronics Co., Ltd. Camera system and method for remotely controlling compositions of self-portrait pictures using hand gestures
US9891794B2 (en) 2014-04-25 2018-02-13 Dropbox, Inc. Browsing and selecting content items based on user gestures
US10089346B2 (en) 2014-04-25 2018-10-02 Dropbox, Inc. Techniques for collapsing views of content items in a graphical user interface
KR102207856B1 (ko) * 2014-05-14 2021-01-26 한국전자통신연구원 사용자의 손 영역을 검출하기 위한 사용자 손 검출 장치 및 그 동작 방법
US20150370472A1 (en) * 2014-06-19 2015-12-24 Xerox Corporation 3-d motion control for document discovery and retrieval
US9395174B2 (en) 2014-06-27 2016-07-19 Faro Technologies, Inc. Determining retroreflector orientation by optimizing spatial fit
US10585485B1 (en) * 2014-11-10 2020-03-10 Amazon Technologies, Inc. Controlling content zoom level based on user head movement
JP6519075B2 (ja) * 2015-02-10 2019-05-29 任天堂株式会社 情報処理装置、情報処理プログラム、情報処理システム、および、情報処理方法
JP6603024B2 (ja) 2015-02-10 2019-11-06 任天堂株式会社 情報処理プログラム、情報処理装置、情報処理システム、および、情報処理方法
JP6534011B2 (ja) 2015-02-10 2019-06-26 任天堂株式会社 情報処理装置、情報処理プログラム、情報処理システム、および、情報処理方法
JP6561400B2 (ja) * 2015-02-10 2019-08-21 任天堂株式会社 情報処理装置、情報処理プログラム、情報処理システム、および、情報処理方法
US9696795B2 (en) 2015-02-13 2017-07-04 Leap Motion, Inc. Systems and methods of creating a realistic grab experience in virtual reality/augmented reality environments
US10429923B1 (en) 2015-02-13 2019-10-01 Ultrahaptics IP Two Limited Interaction engine for creating a realistic experience in virtual reality/augmented reality environments
TWI571768B (zh) * 2015-04-29 2017-02-21 由田新技股份有限公司 使用者介面同步系統、裝置、方法、電腦可讀取記錄媒體、及電腦程式產品
US10120454B2 (en) * 2015-09-04 2018-11-06 Eyesight Mobile Technologies Ltd. Gesture recognition control device
US10043279B1 (en) 2015-12-07 2018-08-07 Apple Inc. Robust detection and classification of body parts in a depth map
US11854308B1 (en) 2016-02-17 2023-12-26 Ultrahaptics IP Two Limited Hand initialization for machine learning based gesture recognition
US11841920B1 (en) 2016-02-17 2023-12-12 Ultrahaptics IP Two Limited Machine learning based gesture recognition
US11714880B1 (en) * 2016-02-17 2023-08-01 Ultrahaptics IP Two Limited Hand pose estimation for machine learning based gesture recognition
JP6728865B2 (ja) * 2016-03-25 2020-07-22 富士ゼロックス株式会社 手上げ検出装置及び手上げ検出プログラム
US10366278B2 (en) 2016-09-20 2019-07-30 Apple Inc. Curvature-based face detector
US11875012B2 (en) 2018-05-25 2024-01-16 Ultrahaptics IP Two Limited Throwable interface for augmented reality and virtual reality environments
CN110581987A (zh) * 2018-06-07 2019-12-17 宏碁股份有限公司 具备手势感测功能的三维显示器
KR102582863B1 (ko) 2018-09-07 2023-09-27 삼성전자주식회사 사용자 의도 기반 제스처 인식 방법 및 장치
US11301049B2 (en) * 2019-09-17 2022-04-12 Huawei Technologies Co., Ltd. User interface control based on elbow-anchored arm gestures
KR102145523B1 (ko) * 2019-12-12 2020-08-18 삼성전자주식회사 카메라 장치의 동작을 제어하는 방법 및 상기 카메라 장치
US11500454B2 (en) * 2020-12-29 2022-11-15 Snap Inc. Body UI for augmented reality components
EP4272406B1 (fr) 2020-12-29 2024-10-02 Snap Inc. Interface utilisateur corporelle pour composants de réalité augmentée
US20230116341A1 (en) * 2021-09-30 2023-04-13 Futian ZHANG Methods and apparatuses for hand gesture-based control of selection focus
US11914789B2 (en) * 2022-01-20 2024-02-27 Htc Corporation Method for inputting letters, host, and computer readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6977654B2 (en) * 2002-10-30 2005-12-20 Iviz, Inc. Data visualization with animated speedometer dial charts
US20060010400A1 (en) * 2004-06-28 2006-01-12 Microsoft Corporation Recognizing gestures and using gestures for interacting with software applications
US7042442B1 (en) * 2000-06-27 2006-05-09 International Business Machines Corporation Virtual invisible keyboard
US20080094371A1 (en) * 2006-09-06 2008-04-24 Scott Forstall Deletion Gestures on a Portable Multifunction Device

Family Cites Families (96)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2007402A (en) * 1931-01-02 1935-07-09 Ericsson Telephones Ltd Totalizator
US4550250A (en) * 1983-11-14 1985-10-29 Hei, Inc. Cordless digital graphics input device
US4789921A (en) * 1987-02-20 1988-12-06 Minnesota Mining And Manufacturing Company Cone shaped Fresnel reflector
US4988981B1 (en) * 1987-03-17 1999-05-18 Vpl Newco Inc Computer data entry and manipulation apparatus and method
US5588139A (en) * 1990-06-07 1996-12-24 Vpl Research, Inc. Method and system for generating objects for a multi-person virtual world using data flow networks
US5264836A (en) * 1991-01-15 1993-11-23 Apple Computer, Inc. Three dimensional cursor
US5973700A (en) * 1992-09-16 1999-10-26 Eastman Kodak Company Method and apparatus for optimizing the resolution of images which have an apparent depth
US5495576A (en) * 1993-01-11 1996-02-27 Ritchey; Kurtis J. Panoramic image based virtual reality/telepresence audio-visual system and method
US5594469A (en) * 1995-02-21 1997-01-14 Mitsubishi Electric Information Technology Center America Inc. Hand gesture machine control system
DE69634913T2 (de) * 1995-04-28 2006-01-05 Matsushita Electric Industrial Co., Ltd., Kadoma Schnittstellenvorrichtung
US6535210B1 (en) * 1995-06-07 2003-03-18 Geovector Corp. Vision system computer modeling apparatus including interaction with real scenes with respect to perspective and spatial relationship as measured in real-time
US5852672A (en) * 1995-07-10 1998-12-22 The Regents Of The University Of California Image system for three dimensional, 360 DEGREE, time sequence surface mapping of moving objects
RU2109336C1 (ru) * 1995-07-14 1998-04-20 Нурахмед Нурисламович Латыпов Способ погружения пользователя в виртуальную реальность и устройство для его реализации
JPH0981309A (ja) * 1995-09-13 1997-03-28 Toshiba Corp 入力装置
EP0768511A1 (fr) * 1995-10-16 1997-04-16 European Community Procédé optique de profilométrie tridimensionelle basé sur le traitement d'images-speckle en lumière partiellement cohérente, et interféromètre utilisant ce procédé
US5862256A (en) * 1996-06-14 1999-01-19 International Business Machines Corporation Distinguishing gestures from handwriting in a pen based computer by size discrimination
US5864635A (en) * 1996-06-14 1999-01-26 International Business Machines Corporation Distinguishing gestures from handwriting in a pen based computer by stroke analysis
US6084979A (en) * 1996-06-20 2000-07-04 Carnegie Mellon University Method for creating virtual reality
US6002808A (en) * 1996-07-26 1999-12-14 Mitsubishi Electric Information Technology Center America, Inc. Hand gesture control system
EP0959444A4 (fr) * 1996-08-14 2005-12-07 Nurakhmed Nurislamovic Latypov Procede de suivi et de representation de la position et de l'orientation d'un sujet dans l'espace, procede de presentation d'un espace virtuel a ce sujet, et systemes de mise en oeuvre de ces procedes
US6118888A (en) * 1997-02-28 2000-09-12 Kabushiki Kaisha Toshiba Multi-modal interface apparatus and method
US5917937A (en) * 1997-04-15 1999-06-29 Microsoft Corporation Method for performing stereo matching to recover depths, colors and opacities of surface elements
US6049327A (en) * 1997-04-23 2000-04-11 Modern Cartoons, Ltd System for data management based onhand gestures
US6008813A (en) * 1997-08-01 1999-12-28 Mitsubishi Electric Information Technology Center America, Inc. (Ita) Real-time PC based volume rendering system
US6720949B1 (en) * 1997-08-22 2004-04-13 Timothy R. Pryor Man machine interfaces and applications
EP0905644A3 (fr) * 1997-09-26 2004-02-25 Matsushita Electric Industrial Co., Ltd. Dispositif de reconnaissance de gestes de la main
US6072494A (en) * 1997-10-15 2000-06-06 Electric Planet, Inc. Method and apparatus for real-time gesture recognition
US6064387A (en) * 1998-01-23 2000-05-16 Dell, Usa, L.P. Animated cursor and icon for computers
US6076928A (en) * 1998-06-15 2000-06-20 Fateh; Sina Ideal visual ergonomic system for computer users
US6064354A (en) * 1998-07-01 2000-05-16 Deluca; Michael Joseph Stereoscopic user interface method and apparatus
US6252988B1 (en) * 1998-07-09 2001-06-26 Lucent Technologies Inc. Method and apparatus for character recognition using stop words
US6681031B2 (en) * 1998-08-10 2004-01-20 Cybernet Systems Corporation Gesture-controlled interfaces for self-service machines and other applications
US6501515B1 (en) * 1998-10-13 2002-12-31 Sony Corporation Remote control system
CN1145872C (zh) * 1999-01-13 2004-04-14 国际商业机器公司 手写汉字自动分割和识别方法以及使用该方法的系统
US7003134B1 (en) * 1999-03-08 2006-02-21 Vulcan Patents Llc Three dimensional object pose estimation which employs dense depth information
US6614422B1 (en) * 1999-11-04 2003-09-02 Canesta, Inc. Method and apparatus for entering data using a virtual input device
JP2000350865A (ja) * 1999-06-11 2000-12-19 Mr System Kenkyusho:Kk 複合現実空間のゲーム装置、その画像処理方法およびプログラム記憶媒体
GB9913687D0 (en) * 1999-06-11 1999-08-11 Canon Kk Image processing apparatus
US6512385B1 (en) * 1999-07-26 2003-01-28 Paul Pfaff Method for testing a device under test including the interference of two beams
US6512838B1 (en) * 1999-09-22 2003-01-28 Canesta, Inc. Methods for enhancing performance and data acquired from three-dimensional image systems
US7548874B2 (en) * 1999-10-21 2009-06-16 International Business Machines Corporation System and method for group advertisement optimization
WO2001095061A2 (fr) * 1999-12-07 2001-12-13 Frauenhofer Institut Fuer Graphische Datenverarbeitung Table virtuelle etendue: rallonge optique pour systemes de projection de type table
US6507353B1 (en) * 1999-12-10 2003-01-14 Godot Huard Influencing virtual actors in an interactive environment
JP2001307134A (ja) * 2000-04-19 2001-11-02 Sony Corp 三次元モデル処理装置および三次元モデル処理方法、並びにプログラム提供媒体
US6456262B1 (en) * 2000-05-09 2002-09-24 Intel Corporation Microdisplay with eye gaze detection
US7227526B2 (en) * 2000-07-24 2007-06-05 Gesturetek, Inc. Video-based image control system
US6686921B1 (en) * 2000-08-01 2004-02-03 International Business Machines Corporation Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object
JP3974359B2 (ja) * 2000-10-31 2007-09-12 株式会社東芝 オンライン文字認識装置及び方法並びにコンピュータ読み取り可能な記憶媒体及びオンライン文字認識プログラム
US6816615B2 (en) * 2000-11-10 2004-11-09 Microsoft Corporation Implicit page breaks for digitally represented handwriting
JP3631151B2 (ja) * 2000-11-30 2005-03-23 キヤノン株式会社 情報処理装置、複合現実感提示装置及びその方法並びに記憶媒体
EP1371019A2 (fr) * 2001-01-26 2003-12-17 Zaxel Systems, Inc. Point de vue virtuel temps r el en environnement de r alit simul e
US20040104935A1 (en) * 2001-01-26 2004-06-03 Todd Williamson Virtual reality immersion system
US8035612B2 (en) * 2002-05-28 2011-10-11 Intellectual Ventures Holding 67 Llc Self-contained interactive video display system
US20040135744A1 (en) * 2001-08-10 2004-07-15 Oliver Bimber Virtual showcases
US6741251B2 (en) * 2001-08-16 2004-05-25 Hewlett-Packard Development Company, L.P. Method and apparatus for varying focus in a scene
JP4050055B2 (ja) * 2002-01-10 2008-02-20 株式会社リコー 手書き文字一括変換装置、手書き文字一括変換方法およびプログラム
WO2003071410A2 (fr) * 2002-02-15 2003-08-28 Canesta, Inc. Systeme de reconnaissance de geste utilisant des capteurs de perception de profondeur
US20030227453A1 (en) * 2002-04-09 2003-12-11 Klaus-Peter Beier Method, system and computer program product for automatically creating an animated 3-D scenario from human position and path data
US7203356B2 (en) * 2002-04-11 2007-04-10 Canesta, Inc. Subject segmentation and tracking using 3D sensing technology for video compression in multimedia applications
US7348963B2 (en) * 2002-05-28 2008-03-25 Reactrix Systems, Inc. Interactive video display system
US20050122308A1 (en) * 2002-05-28 2005-06-09 Matthew Bell Self-contained interactive video display system
US7710391B2 (en) * 2002-05-28 2010-05-04 Matthew Bell Processing an image utilizing a spatially varying pattern
US6857746B2 (en) * 2002-07-01 2005-02-22 Io2 Technology, Llc Method and system for free-space imaging display and interface
US8905834B2 (en) * 2007-11-09 2014-12-09 Igt Transparent card display
US20040174770A1 (en) * 2002-11-27 2004-09-09 Rees Frank L. Gauss-Rees parametric ultrawideband system
US7576727B2 (en) * 2002-12-13 2009-08-18 Matthew Bell Interactive directed light/sound system
US9177387B2 (en) * 2003-02-11 2015-11-03 Sony Computer Entertainment Inc. Method and apparatus for real time motion capture
KR100465241B1 (ko) * 2003-03-17 2005-01-13 삼성전자주식회사 가상 필기 평면을 이용한 모션 인식 시스템 및 그 인식방법
KR100518824B1 (ko) * 2003-03-17 2005-10-05 삼성전자주식회사 필기 모션 획 구분 인식 시스템 및 그 인식방법
US7762665B2 (en) * 2003-03-21 2010-07-27 Queen's University At Kingston Method and apparatus for communication between humans and devices
US8745541B2 (en) * 2003-03-25 2014-06-03 Microsoft Corporation Architecture for controlling a computer using hand gestures
US7372977B2 (en) * 2003-05-29 2008-05-13 Honda Motor Co., Ltd. Visual tracking using depth data
US7590941B2 (en) * 2003-10-09 2009-09-15 Hewlett-Packard Development Company, L.P. Communication and collaboration system using rich media environments
US7536032B2 (en) * 2003-10-24 2009-05-19 Reactrix Systems, Inc. Method and system for processing captured image information in an interactive video display system
EP1676442A2 (fr) * 2003-10-24 2006-07-05 Reactrix Systems, Inc. Procede et systeme pour gerer un systeme d'affichage video interactif
US7963652B2 (en) * 2003-11-14 2011-06-21 Queen's University At Kingston Method and apparatus for calibration-free eye tracking
GB2411532B (en) * 2004-02-11 2010-04-28 British Broadcasting Corp Position determination
US7182465B2 (en) * 2004-02-25 2007-02-27 The University Of North Carolina Methods, systems, and computer program products for imperceptibly embedding structured light patterns in projected color images for display on planar and non-planar surfaces
US8487879B2 (en) * 2004-10-29 2013-07-16 Microsoft Corporation Systems and methods for interacting with a computer through handwriting to a screen
US20070130547A1 (en) * 2005-12-01 2007-06-07 Navisense, Llc Method and system for touchless user interface control
US7971156B2 (en) * 2007-01-12 2011-06-28 International Business Machines Corporation Controlling resource access based on user gesturing in a 3D captured image stream of the user
US8726194B2 (en) * 2007-07-27 2014-05-13 Qualcomm Incorporated Item selection using enhanced control
US7949157B2 (en) * 2007-08-10 2011-05-24 Nitin Afzulpurkar Interpreting sign language gestures
JP5559691B2 (ja) * 2007-09-24 2014-07-23 クアルコム，インコーポレイテッド 音声及びビデオ通信のための機能向上したインタフェース
JP5317169B2 (ja) * 2008-06-13 2013-10-16 洋 川崎 画像処理装置、画像処理方法およびプログラム
US9445193B2 (en) * 2008-07-31 2016-09-13 Nokia Technologies Oy Electronic device directional audio capture
KR20100101389A (ko) 2009-03-09 2010-09-17 삼성전자주식회사 사용자 메뉴를 제공하는 디스플레이 장치 및 이에 적용되는ｕｉ제공 방법
US8672854B2 (en) 2009-05-20 2014-03-18 Sotera Wireless, Inc. System for calibrating a PTT-based blood pressure measurement using arm height
KR20110010906A (ko) * 2009-07-27 2011-02-08 삼성전자주식회사 사용자 인터랙션을 이용한 전자기기 제어 방법 및 장치
US8659658B2 (en) * 2010-02-09 2014-02-25 Microsoft Corporation Physical interaction zone for gesture-based user interfaces
US8787663B2 (en) * 2010-03-01 2014-07-22 Primesense Ltd. Tracking body parts by combined color image and depth processing
US20110254765A1 (en) * 2010-04-18 2011-10-20 Primesense Ltd. Remote text input using handwriting
US20110289455A1 (en) 2010-05-18 2011-11-24 Microsoft Corporation Gestures And Gesture Recognition For Manipulating A User-Interface
US20110292036A1 (en) * 2010-05-31 2011-12-01 Primesense Ltd. Depth sensor with application interface
US20110310010A1 (en) * 2010-06-17 2011-12-22 Primesense Ltd. Gesture based user interface
US8959013B2 (en) * 2010-09-27 2015-02-17 Apple Inc. Virtual keyboard for a non-tactile three dimensional user interface

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7042442B1 (en) * 2000-06-27 2006-05-09 International Business Machines Corporation Virtual invisible keyboard
US6977654B2 (en) * 2002-10-30 2005-12-20 Iviz, Inc. Data visualization with animated speedometer dial charts
US20060010400A1 (en) * 2004-06-28 2006-01-12 Microsoft Corporation Recognizing gestures and using gestures for interacting with software applications
US20080094371A1 (en) * 2006-09-06 2008-04-24 Scott Forstall Deletion Gestures on a Portable Multifunction Device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LA VIOLA JR. ET AL.: 'Whole-Hand and Speech Input in Virtual Environments' 1996, pages 1 - 137 *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2474950A1 (fr) 2011-01-05 2012-07-11 Softkinetic Software Procédés et systèmes d'interface d'utilisateur à base de geste naturel
WO2012093147A1 (fr) 2011-01-05 2012-07-12 Softkinetic Software Procédés et systèmes d'interface utilisateur basés sur des gestes naturels
US9857868B2 (en) 2011-03-19 2018-01-02 The Board Of Trustees Of The Leland Stanford Junior University Method and system for ergonomic touch-free interface
US9504920B2 (en) 2011-04-25 2016-11-29 Aquifi, Inc. Method and system to create three-dimensional mapping in a two-dimensional game
WO2013038089A1 (fr) 2011-09-16 2013-03-21 Prynel Procédé et système d'acquisition et de traitement d'images pour la détection du mouvement
EP3009919A1 (fr) * 2011-12-30 2016-04-20 Samsung Electronics Co., Ltd. Appareil électronique et son procédé de commande
US9600078B2 (en) 2012-02-03 2017-03-21 Aquifi, Inc. Method and system enabling natural user interface gestures with an electronic system
US9111135B2 (en) 2012-06-25 2015-08-18 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching using corresponding pixels in bounded regions of a sequence of frames that are a specified distance interval from a reference camera
US9098739B2 (en) 2012-06-25 2015-08-04 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching
US8934675B2 (en) 2012-06-25 2015-01-13 Aquifi, Inc. Systems and methods for tracking human hands by performing parts based template matching using images from multiple viewpoints
US8655021B2 (en) 2012-06-25 2014-02-18 Imimtek, Inc. Systems and methods for tracking human hands by performing parts based template matching using images from multiple viewpoints
US8830312B2 (en) 2012-06-25 2014-09-09 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching within bounded regions
US9310891B2 (en) 2012-09-04 2016-04-12 Aquifi, Inc. Method and system enabling natural user interface gestures with user wearable glasses
US9129155B2 (en) 2013-01-30 2015-09-08 Aquifi, Inc. Systems and methods for initializing motion tracking of human hands using template matching within bounded regions determined using a depth map
US9092665B2 (en) 2013-01-30 2015-07-28 Aquifi, Inc Systems and methods for initializing motion tracking of human hands
US8615108B1 (en) 2013-01-30 2013-12-24 Imimtek, Inc. Systems and methods for initializing motion tracking of human hands
US9298266B2 (en) 2013-04-02 2016-03-29 Aquifi, Inc. Systems and methods for implementing three-dimensional (3D) gesture based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
WO2014200874A1 (fr) * 2013-06-13 2014-12-18 Microsoft Corporation Génération de texte au moyen d'une interface sans toucher
US9798388B1 (en) 2013-07-31 2017-10-24 Aquifi, Inc. Vibrotactile system to augment 3D input systems
US9507417B2 (en) 2014-01-07 2016-11-29 Aquifi, Inc. Systems and methods for implementing head tracking based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US9619105B1 (en) 2014-01-30 2017-04-11 Aquifi, Inc. Systems and methods for gesture based interaction with viewpoint dependent user interfaces

Also Published As

Publication number Publication date
US10719214B2 (en) 2020-07-21
WO2010103482A3 (fr) 2010-11-04
US20100235786A1 (en) 2010-09-16
US20140304647A1 (en) 2014-10-09
US20180059925A1 (en) 2018-03-01

Similar Documents

Publication Publication Date Title
US10719214B2 (en) Enhanced 3D interfacing for remote devices
US20240272723A1 (en) Hand gesture input for wearable system
Brandl et al. Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces
US10268339B2 (en) Enhanced camera-based input
US9857868B2 (en) Method and system for ergonomic touch-free interface
US20170228138A1 (en) System and method for spatial interaction for viewing and manipulating off-screen content
CN108052202A (zh) 一种3d交互方法、装置、计算机设备及存储介质
US20150261310A1 (en) One-dimensional input system and method
US20130120282A1 (en) System and Method for Evaluating Gesture Usability
CN105980965A (zh) 用于非接触式打字的系统、设备和方法
US10180714B1 (en) Two-handed multi-stroke marking menus for multi-touch devices
WO2010008835A1 (fr) Entrée de caractère améliorée à l'aide de gestes reconnus
Vogel et al. Hand occlusion with tablet-sized direct pen input
Au et al. Multitouch finger registration and its applications
CN108700957A (zh) 用于虚拟环境中文本输入的电子系统和方法
CN112527112A (zh) 一种多通道沉浸式流场可视化人机交互方法
Raees et al. VEN-3DVE: vision based egocentric navigation for 3D virtual environments
Xiao et al. A hand gesture-based interface for design review using leap motion controller
Chun et al. A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment
Bai et al. Asymmetric Bimanual Interaction for Mobile Virtual Reality.
CN111007977A (zh) 智能虚拟交互方法及装置
Uddin Improving Multi-Touch Interactions Using Hands as Landmarks
CN105242795A (zh) 一种利用方位手势输入英文字母的方法
Mahdikhanlou et al. Object manipulation and deformation using hand gestures
Ullerich et al. ThumbPitch: Enriching thumb interaction on mobile touchscreens using deep learning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10750444

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10750444

Country of ref document: EP

Kind code of ref document: A2