Connect public, paid and private patent data with Google Patents Public Datasets

Multiple input modes in overlapping physical space

Download PDF

Info

Publication number
US20030174125A1
US20030174125A1 US10367609 US36760903A US2003174125A1 US 20030174125 A1 US20030174125 A1 US 20030174125A1 US 10367609 US10367609 US 10367609 US 36760903 A US36760903 A US 36760903A US 2003174125 A1 US2003174125 A1 US 2003174125A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
input
mode
device
user
keyboard
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10367609
Inventor
Ilhami Torunoglu
Apurva Desai
Cheng-Feng Sze
Gagan Prakash
Abbas Rafii
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canesta Inc
Original Assignee
Canesta Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/18Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/18Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
    • G01S5/22Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F1/00Details of data-processing equipment not covered by groups G06F3/00 - G06F13/00, e.g. cooling, packaging or power supply specially adapted for computer application
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1626Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F1/00Details of data-processing equipment not covered by groups G06F3/00 - G06F13/00, e.g. cooling, packaging or power supply specially adapted for computer application
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1632External expansion units, e.g. docking stations
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F1/00Details of data-processing equipment not covered by groups G06F3/00 - G06F13/00, e.g. cooling, packaging or power supply specially adapted for computer application
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1662Details related to the integrated keyboard
    • G06F1/1673Arrangements for projecting a virtual keyboard
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F1/00Details of data-processing equipment not covered by groups G06F3/00 - G06F13/00, e.g. cooling, packaging or power supply specially adapted for computer application
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/169Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/0202Constructional details or processes of manufacture of the input device
    • G06F3/0221Arrangements for reducing keyboard size for transport or storage, e.g. foldable keyboards, keyboards with collapsible keys
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0421Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen
    • G06F3/0423Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen using sweeping light beams, e.g. using rotating or vibrating mirror
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • G06F3/0426Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking fingers with respect to a virtual keyboard projected or printed on the surface
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/043Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/043Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
    • G06F3/0433Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/043Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
    • G06F3/0436Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the screen or tablet into independently controllable areas, e.g. virtual keyboards, menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/22Image acquisition using hand-held instruments
    • G06K9/222Image acquisition using hand-held instruments the instrument generating sequences of position coordinates corresponding to handwriting; preprocessing or recognising digital ink
    • G06K9/224Image acquisition using hand-held instruments the instrument generating sequences of position coordinates corresponding to handwriting; preprocessing or recognising digital ink in three dimensions
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06Systems determining position data of a target
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/80Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using ultrasonic, sonic or infrasonic waves
    • G01S3/802Systems for determining direction or deviation from predetermined direction
    • G01S3/808Systems for determining direction or deviation from predetermined direction using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
    • G01S3/8083Systems for determining direction or deviation from predetermined direction using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems determining direction of source

Abstract

In a sensory input system that detects movement of a user's fingers on an inert work surface, two or more input modes (for instance, keyboard and mouse) are provided within an overlapping or coextensive physical space. Depending on the currently active mode, the invention interprets the finger motions as input according to one of the input modes. Automated and/or manual mode-switching are provided.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • [0001]
    This application claims priority from U.S. provisional patent application Serial No. 60/357,735 for “Method and Apparatus for Accomplishing Two or More Input Methods in the Same Physical Space Using a Sensory Input System,” filed Feb. 15, 2002, the disclosure of which is incorporated herein by reference.
  • [0002]
    The present application is a continuation-in-part of U.S. patent application Ser. No. 10/313,939 for “Portable Sensory Input Device,” filed Dec. 5, 2002, the disclosure of which is incorporated herein by reference, and which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/339,234 for “Method and Apparatus for Stability and Alignment of a Portable Sensory Input Device,” filed Dec. 7, 2001, and which in turn is a continuation-in-part of the following U.S. patent applications, the disclosures of which are incorporated herein by reference:
  • [0003]
    U.S. patent application Ser. No. 09/502,499 for “Method and Apparatus for Entering Data Using a Virtual Input Device,” filed Feb. 11, 2000, which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/163,445 for “Method and Device for 3D Sensing of Input Commands to Electronic Devices,” filed Nov. 4, 1999.
  • [0004]
    U.S. patent application Ser. No. 09/948,508 for “Quasi-Three-Dimensional Method and Apparatus To Detect and Localize Interaction of User-Object and Virtual Transfer Device,” filed Sep. 7, 2001, which in turn claims priority from U.S. Provisional Patent Application Ser. No. 60/231,184 for “Application of Image Processing Techniques for A Virtual Keyboard System,” filed Sep. 7, 2000.
  • [0005]
    U.S. patent application Ser. No. 10/245,925 for “Measurement of Depth from Thickness or Separation of Structured Light with Application to Virtual Interface Devices,” filed Sep. 17, 2002, which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/382,899 for “Measurement of Distance in a Plane from the thickness of a Light Beam from the Separation of Several Light Beams,” filed May 22, 2002.
  • [0006]
    U.S. patent application Ser. No. 10/246,123 for “Method and Apparatus for Approximating Depth of an Object's Placement into a Monitored Region with Applications to Virtual Interface Devices,” filed Sep. 17, 2002, which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/382,899 for “Measurement of Distance in a Plane from the thickness of a Light Beam from the Separation of Several Light Beams,” filed May 22, 2002.
  • [0007]
    U.S. patent application Ser. No. 10/115,357 for “Method and Apparatus for Approximating a Source Position of a Sound-Causing Event for Determining an Input Used in Operating an Electronic Device,” filed Apr. 2, 2002, which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/281,314 for “A Localization System Based on Sound Delays,” filed Apr. 3, 2001.
  • [0008]
    U.S. patent application Ser. No. 10/187,032 for “Detecting, Classifying, and Interpreting Input Events Based on Stimuli in Multiple Sensory Domains,” filed Jun. 28, 2002, which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/337,086 for “Sound-Based Method and Apparatus for Detecting the Occurrence and Force of Keystrokes in Virtual Keyboard Applications,” filed Nov. 27, 2001.
  • [0009]
    U.S. patent application Ser. No. 10/179,452 for “Method and Apparatus to Display a Virtual Input Device,” filed Jun. 24, 2002, which in turn claims priority from U.S. Provisional Patent Application Serial No. 60/300,542 for “User Interface Projection System,” filed Jun. 22, 2001.
  • BACKGROUND OF THE INVENTION
  • [0010]
    1. Field of the Invention
  • [0011]
    The present invention relates to input devices for portable electronic devices, and more particularly to an input device that accommodates multiple input modes in the same physical space.
  • [0012]
    2. Description of the Background Art
  • [0013]
    In a virtual keyboard system, a user taps on regions of a surface with his or her fingers or with another object such as a stylus, in order to interact with an electronic device into which data is to be entered. The system determines when a user's fingers or stylus contact a surface having images of keys (“virtual keys”), and further determines which fingers contact which virtual keys thereon, so as to provide input to a PDA (or other device) as though it were conventional keyboard input. The keyboard is virtual, in the sense that no physical device need be present on the part of surface that the user contacts, henceforth called the work surface.
  • [0014]
    A virtual keyboard can be implemented using, for example, a keyboard guide: a piece of paper or other material that unfolds to the size of a typical keyboard, with keys printed thereon to guide the user's hands. The physical medium on which the keyboard guide is printed is simply an inert surface and has no sensors or mechanical or electronic component. The input to the PDA (or other device) does not come from the keyboard guide itself, but rather is based on detecting contact of the user's fingers with areas on the keyboard guide. Alternatively, a virtual keyboard can be implemented without a keyboard guide, so that the movements of a user's fingers on any surface, even a plain desktop, are detected and interpreted as keyboard input. Alternatively, an image of a keyboard may be projected or otherwise drawn on any surface (such as a desktop) that is defined as the work surface or active area, so as to provide finger placement guidance to the user. Alternatively, a computer screen or other display may show a keyboard layout with icons that represent the user's fingers superimposed on it. In some applications, nothing is projected or drawn on the surface.
  • [0015]
    U.S. Pat. No. 6,323,942, for “CMOS Compatible 3-D Image Sensor,” the disclosure of which is incorporated herein by reference, discloses a three-dimensional imaging system including a two-dimensional array of pixel light sensing detectors and dedicated electronics and associated processing circuitry to measure distance and velocity data in real time using time-of-flight (TOF) data.
  • [0016]
    The related patent applications referenced above disclose additional data input methods, modes, and apparatuses including direct three-dimensional sensing, planar range sensors, and vertical triangulation. Such techniques detect user input by sensing three-dimensional data to localize the user's fingers as they come in contact with a surface.
  • [0017]
    The applications further describe several data input methods, modes, and apparatuses for sensing object movements with a sensing device (either 3D, planar, vertical triangulation, or otherwise) and interpreting such movements into digital data (such as keystrokes). In some of the above applications, techniques are described for combining stimuli detected in two or more sensory domains in order to improve performance and reliability in classifying and interpreting user gestures. These data input methods are used for entering data into any kind of electronic equipment such as mobile devices (e.g. PDA, cell-phone, pen-tablet, computer, etc.) and provide significant benefits over existing methods due to their ease of use, portability, speed of data entry, power consumption, weight, and novelty. Many of the described techniques are implemented in a virtual keyboard input system in which a user may strike an inert surface, such as a desktop, on which a keyboard pattern is being projected.
  • [0018]
    The Senseboard product, offered by Senseboard Technologies AB of Stockholm, Sweden, captures and interprets the motion of a user's fingers in order to allow keyboard-like input without a physical keyboard.
  • [0019]
    Conventional sensing devices are typically adapted to detect one particular type of input in a particular defined area, such as for example keyboard input. However, in many situations if it is desirable to provide two or more input modes. For example, most personal computers now provide both mouse and keyboard input devices, both of which are often used in quick succession to provide input and to specify command and control functions. Conventional sensing devices that operate by detecting finger motion are unable to perform both input functions in a given detection area.
  • [0020]
    MultiTouch products offered by FingerWorks Inc. of Townsend, Del. provide limited capability for receiving typing, mouse, and gesture input in the same overlapping area of an input pad. These products use an input detection pad and are not able to function on an inert surface such as an ordinary desktop. The overall input area is limited to that covered by the active surface, thus reducing the flexibility and portability of the device, particularly if it is to be used with personal digital assistants (PDAs) or other devices that are usually carried around by users.
  • [0021]
    What is needed, then, is a system and method for facilitating two or more input modes in a sensory input device. What is further needed is a system and method for facilitating two or more input modes without requiring separate physical space to be designated for each. What is further needed is a system and method for facilitating multiple input modes in a small space and on an inert surface such as a desktop. What is further needed is a system and method for facilitating multiple input modes in a sensory input device without requiring a user to reposition his or her fingers when switching from one mode to another. What is further needed is a system and method for facilitating two or more input modes while preserving the flexibility, portability, and other advantages of a sensory input device.
  • SUMMARY OF THE INVENTION
  • [0022]
    This invention enables two or more input modes (for instance, keyboard and mouse) in an overlapping or coextensive physical space using a sensory input system. The invention is operable on an inert surface such as a desktop. The user moves his or her fingers as though interacting with an ordinary input device; the system of the invention detects the finger motions and interprets them accordingly. Depending on the currently active mode, the invention interprets the finger motions as input according to one of the input modes, and changes its sensory input interpretation techniques so as to be better adapted to receive and interpret input in the current input mode.
  • [0023]
    In one embodiment, the user can switch from one mode to another by specifying a mode switch command. In another embodiment, the system of the invention automatically detects, from the nature of the user's input, that the input mode should be switched, and performs the mode switch accordingly. For example, in an embodiment that provides a keyboard mode and a mouse mode, the sensing device of the invention detects whether a user appears to be tapping (as one would interact with a keyboard) or gliding across the work surface (as one would interact with a mouse). Depending on the detected input type, the system of the invention automatically switches to the corresponding input mode and interprets the user's finger motions accordingly.
  • [0024]
    In one embodiment, the system of the invention projects an input guide onto the work surface, so as to help the user in positioning his or her fingers properly. In one embodiment, the invention changes the input guide when the input mode changes, so as to provide a guide that is appropriate to the current input mode. In another embodiment, the projected input guide does not change when the mode changes. In another embodiment, the system of the invention projects input guides for two or more modes simultaneously. In yet another embodiment, the user is able to configure the system regarding whether or not to change, the projected input guide when the mode changes.
  • [0025]
    The present invention is able to operate in conjunction with any of the various implementations and designs described in the above-referenced related applications. For example, the present invention may be implemented in a device that uses techniques for combining stimuli in multiple sensory domains as described in U.S. patent application Ser. No. 10/187,032 for “Detecting, Classifying, and Interpreting Input Events Based on Stimuli in Multiple Sensory Domains.”
  • [0026]
    The present invention thus provides many of the advantages of sensory input systems that can operate on an inert surface, and provides the further advantage of being able to accept input in multiple modes within the same physical space. In addition, the present invention is able to change its sensory input interpretation techniques depending on the current mode, so as to more accurately capture and interpret input in different modes. Furthermore, although the description herein is focused primarily on keyboard and mouse input modes, one skilled in the art will recognize that the techniques of the present invention can be applied to any sensory input system offering multiple input modes, and that the input modes can correspond to any type of physical or virtual input mechanism, including for example: musical instruments, joysticks, trackballs, jog/dial controllers, pen-based tablets, and the like.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0027]
    [0027]FIG. 1A is a diagram depicting an integrated multiple-mode input device displaying a keyboard guide according to one embodiment of the present invention.
  • [0028]
    [0028]FIG. 1B is a diagram depicting an integrated multiple-mode input device displaying a mouse guide according to one embodiment of the present invention.
  • [0029]
    [0029]FIG. 1C is a diagram depicting an integrated multiple-mode input device displaying a combination keyboard/mouse guide according to one embodiment of the present invention.
  • [0030]
    [0030]FIG. 2 is block diagram of an embodiment of the present invention.
  • [0031]
    [0031]FIG. 3 is an example of a keyboard guide for one embodiment of the present invention.
  • [0032]
    [0032]FIG. 4 is a flowchart depicting a method for providing multiple input modes in an overlapping physical space, according to one embodiment of the present invention.
  • [0033]
    [0033]FIG. 5 is block diagram depicting dispatching events to appropriate event queues, according to one embodiment of the present invention.
  • [0034]
    [0034]FIG. 6 is a diagram depicting a stand-alone multiple-mode input device displaying a keyboard guide according to one embodiment of the present invention.
  • [0035]
    [0035]FIG. 7 is a diagram depicting a stand-alone multiple-mode input device displaying a mouse guide according to one embodiment of the present invention.
  • [0036]
    [0036]FIG. 8 is an example of a use case illustrating key occlusion.
  • [0037]
    [0037]FIG. 9 is another example of a use case illustrating key occlusion.
  • [0038]
    [0038]FIG. 10 is another example of a use case illustrating key occlusion.
  • [0039]
    The Figures depict preferred embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
  • DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT
  • [0040]
    The following description of system components and operation is merely exemplary of embodiments of the present invention. One skilled in the art will recognize that the various designs, implementations, and techniques described herein may be used alone or in any combination, and that many modifications and equivalent arrangements can be used. Accordingly, the following description is presented for purposes of illustration, and is not intended to limit the invention to the precise forms disclosed.
  • [0041]
    Architecture
  • [0042]
    Referring now to FIGS. 1A through 1C, there is shown a diagram of an integrated device 101 that includes apparatus for providing input functionality according to one embodiment of the present invention. Referring also to FIGS. 6 and 7, there is shown a diagram of a stand-alone device housing 600 that includes apparatus for providing input functionality according to one embodiment of the present invention. In general, the present invention operates to provide input for any type of device 101, which may be a personal digital assistant (PDA), cell phone, or the like. The invention may be implemented in an apparatus enclosed within device 101 (as shown in FIGS. 1A through 1C) or in a separate housing 600 (as shown in FIGS. 6 and 7) that includes apparatus for sending input signals to a host device. In one embodiment, the present invention provides mechanisms for implementing data input methods, modes, and apparatuses including direct three-dimensional sensing, planar range sensors, and vertical triangulation. Such techniques detect user input by sensing three-dimensional data to localize the user's fingers as they come in contact with surface 204. In one embodiment, surface 204 is an inert work surface, such as an ordinary desktop.
  • [0043]
    Referring also to FIG. 2, there is shown a block diagram depicting an input device according to an embodiment of the present invention. In one embodiment, one or two (or more) sensor circuits 106, 108 are provided, each including a sensor 107, 109. Sensors 107, 109 may be implemented, for example, using charge-coupled device (CCD) and/or complementary metal-oxide semiconductor (CMOS) digital cameras as described in U.S. Pat. No. 6,323,942, to obtain three-dimensional image information. While many of the embodiments shown herein include one sensor 107, one skilled in the art will recognize that any number of sensors can be used, and thus references to “a sensor” are understood to include multiple sensor embodiments. It is beneficial, in some embodiments using three-dimensional sensing technology, to position sensors 107, 109 at the bottom of device 101, so as to more accurately detect finger motions and contact with the work surface in the proximity of the bottom of such device. Alternatively, it may be preferable in some embodiments to position sensors 107, 109 at the side and towards the center or above device 101. Such a location may be advantageous to provide an improved vantage point relative to the location of the user's fingers on the work surface when using two-dimensional sensors such as CCD or CMOS cameras.
  • [0044]
    Central processing unit (CPU) 104 runs software stored in memory 105 to detect input events, and to communicate such events to an application running on host device 101. In implementations where the input device of the present invention is provided in a separate housing 600 from host device 101 (as shown in FIGS. 6 and 7), CPU 104 communicates with device 101 via any known port 102 or communication interface, such as for example serial cable, Universal Serial Bus (USB) cable, Infrared Data Association (irDA) port, Bluetooth port, or the like. Light source 111 illuminates the area of interest on the work surface so that sensors 107, 109 can detect activity.
  • [0045]
    In one embodiment, sensor circuit 106, sensor 107, memory 105, and CPU 104, as well as circuitries for controlling optional projector 110 and light source 111, are integrated into a single CMOS chip or multi-chip module 103, also referred to as a sensor subsystem 103. One skilled in the art will recognize that in alternative embodiments the various components of module 103 may be implemented separately from one another.
  • [0046]
    In one embodiment, projector 110 projects an input guide (shown variously as 203A, 203B, and 203C in the drawings) onto work surface 204. Guide 203A, 203B, 203C has a virtual layout that mimics the layout of a physical input device appropriate to the type of input being detected. For example, in FIG. 1A and FIG. 6, guide 203A has a layout resembling a standard QWERTY keyboard for entering text. In the examples of FIG. 1B and FIG. 7, mouse input guide 203B is projected, to show the user the active area for virtual mouse movement. In the example of FIG. 1C, a combination keyboard/mouse input guide 203C is projected, drawn as a mouse guide overlaying a keyboard guide. In one embodiment, whenever a combination keyboard/mouse input guide 203C is projected, the mouse guide is projected in a different color than the keyboard guide, to further clarify the distinction between the two. Guide 203C indicates that the user can either type or perform mouse movements, in the same area of work surface 204. In one embodiment, as will be described in more detail below, device 110 is able to receive mouse input even when keyboard input guide 203A is projected, and even when no mouse input guide is projected. In general, one skilled in the art will recognize that input guide 203 can take any form appropriate to the currently active input mode.
  • [0047]
    Multiple Modes
  • [0048]
    The present invention accepts user input in two or more modes. Two or more input modes can be implemented in a sensing device by providing separate detection areas for each input mode. Thus, a mouse area and a keyboard area might be defined, possibly having separating sensing apparatus for each. A user wishing to provide mouse input moves his or her fingers within the defined mouse area. When the user wishes to provide keyboard input, he or she moves his or her fingers within the defined keyboard area. In such an implementation, the input mode areas are non-overlapping.
  • [0049]
    In a preferred embodiment, the detection areas for the input modes overlap one another, at least in part. Such an approach allows each detection area to be made larger, and therefore facilitates input within a relatively small desktop area, without compromising input detection area size. In addition, such an approach reduces or eliminates the requirement that the user move his or her fingers from one physical area to another when switching between input modes. In one embodiment, one detection area wholly includes or is coextensive with another detection area, so that the user can keep his or her fingers in the same physical area even when the device switches from one input mode to another.
  • [0050]
    For illustrative purposes, in the following description the invention will be described in terms of keyboard and mouse input modes. However, one skilled in the art will recognize that the techniques of the present invention can be used to implement other input modes in any combination. Thus, the invention is not intended to be limited to the particular example of a keyboard mode and a mouse mode.
  • [0051]
    When in a keyboard mode, device 101 interprets the user's finger motions as keyboard input. Based on sensor 107 detection of the user's finger positions at the time the user taps on work surface 204, device 101 determines which keystroke was intended.
  • [0052]
    When in a mouse mode, device 101 interprets the user's finger motions as though it were input from a pointing device such as a mouse, trackball, trackpad, or the like. Based on sensor 107 detection of the user's finger positions and movements on work surface 204, device 101 moves an onscreen cursor, activates onscreen objects, highlights onscreen text and objects, and performs other activities commonly associated with and controlled by pointing devices such as mice.
  • [0053]
    When in other input modes, device 101 interprets the user's finger motions in a manner appropriate to the currently active mode.
  • [0054]
    In one embodiment, device 101 switches from one mode to another in response to a command from the user. The user may request a mode switch by pressing a designated button on device 101, or by performing a predefined gesture or finger movement detected by sensor 107 and interpreted by device 101, or by speaking, tapping, or issuing some other auditory command that is detected and interpreted by device 101 according to conventional voice recognition or auditory recognition techniques. In one embodiment, a number of different mechanisms for commanding a mode switch may be provided, allowing the user to select the mechanism that is most convenient at any given time. Recognizing that users often switch rapidly and repeatedly from one mode to another, the present invention makes it very easy and convenient to perform such switches.
  • [0055]
    Additional examples of mode change mechanisms and commands include, without limitation:
  • [0056]
    Pressing a designated virtual key or keys changes into a new mode until the same key is pressed again.
  • [0057]
    Pressing a designated virtual key or keys changes into a new mode only while the key or keys are depressed.
  • [0058]
    Pressing a specific sequence of virtual keys (e.g. Ctrl-Shift-1) changes into a new mode.
  • [0059]
    Specific finger movements change mode. For example, a double tap on work surface 204 enters a mode, and triple tap exits a mode. Since a sensing system is being used, the finger movements are not limited to traditional computing finger movements. New operations such as a “pinch,” “flick,” “wiggle,” “scrub,” or other type of defined finger movement could also change modes.
  • [0060]
    One skilled in the art will recognize that the above list is merely exemplary, and that many other techniques for providing and interpreting mode change commands can be used without departing from the essential characteristics of the invention. In addition, node change commands (and other commands) need not be limited to movement along work surface 204. Gestures or other body movements could be used to change modes in a 3-dimensional environment. For instance, a thumbs-up or thumbs-down gesture could enter and/or exit a mode. Making a fist could change mode, grasping hands together could change mode, and so on. Kicking a leg or shaking hips could also change mode.
  • [0061]
    In another embodiment, device 101 automatically switches from one mode to another depending on the current context of the user interaction, or under control of the host device. For example, a numeric virtual keyboard mode can be activated when the context of the user interaction dictates that numeric input is expected.
  • [0062]
    In another embodiment, device 101 automatically switches from one mode to another, based on the nature of the detected finger positions and motions of the user. For example, if sensor 107 detects that the user has his or her fingers in a typing position or is moving his or her fingers in a manner consistent with typing, device 101 automatically switches to keyboard mode, and interprets finger movements as keystrokes. If sensor 107 detects that the user is gliding his fingers along surface 204 in a manner consistent with moving a mouse or interacting with a trackpad, device 101 automatically switches to mouse mode, and interprets finger movements as mouse movements.
  • [0063]
    In one embodiment, keyboard and mouse input are distinguished from one another by analysis of finger image blob motion. Blob motion representing keyboard input tends to be essentially vertical, corresponding to the tapping of keys, so that when the device detects a quick descent followed by an abrupt stop, it can assume keyboard input. By contrast, blob motion representing mouse input tends to have small amounts of vertical motion; thus, when the device detects movement parallel to the plane of the work surface with minimal vertical movement, it can assume mouse input.
  • [0064]
    In one embodiment, even if automatic mode switches are provided, device 101 allows the user to temporarily disable and/or override automatic mode switches. Thus, in the event the user's finger movements cause device 101 to make incorrect assumptions regarding the input mode, or if the user's current activity is specialized or limited to one mode, the user is able to control the manner in which his or her actions are interpreted.
  • [0065]
    In one embodiment, the invention provides seamless integration of the multiple mode sensory input system with an existing host system such as a personal computer or standalone PDA. Referring again to FIG. 2 and also to FIG. 5, CPU 104 communicates, via port 102, with a device driver 501 on device 101 that interprets the incoming events (such as keystrokes, joystick action, or mouse movements) and dispatches those events to an appropriate standard event queue 502-504 for those “virtual” devices. For instance, the keystrokes are dispatched to key event queue 502, the joystick actions to joystick event queue 503, and the mouse events to mouse event queue 504. Once the events are in the appropriate queue, device 101 processes the events as if they were coming from an actual physical device (such as a physical keyboard, joystick, or mouse). The invention thereby facilitates “plug-and-play” operation in connection with applications already written for the supported device types (such as keyboard, joystick, or mouse). In some embodiments, event queue 502-504 is implemented as another device driver or is embedded inside another device driver. In this case, the invention manipulates the other device drivers to insert the events in the driver directly.
  • [0066]
    This device driver system does not limit the functionality to compatibility with old applications, however. New applications that can support a more rich or enhanced set of event information are also support by dispatching this more rich set of event information directly to them. The invention thereby works with existing legacy applications, but also supports new applications with additional functionality.
  • [0067]
    Input Guide
  • [0068]
    As described above, and as depicted in FIGS. 1A through 1C, 6, and 7, in one embodiment device 101 includes projector 110 for projecting input guide 203 onto work surface 204. Input guide 203 is not an essential element of the invention, and in some embodiments the user provides input by moving his or her fingers on work surface 204 without the need for any input guide 203. For example, if the user is controlling the movement of an onscreen cursor in a mouse mode, the user is generally able to provide accurate input without any input guide 203. Accordingly, in some embodiments, input guide 203 may be switched on or off by the user, by activating a command, or input guide 203 may switch on or off automatically depending on which input mode is active. In other embodiments, projector 110 may be omitted without departing from the essential characteristics of the invention.
  • [0069]
    In embodiments that do include one or more input guides 203, projector 110 may project a different input guide 203 for each mode. Thus, the particular input guide 203 being projected depends on and is appropriate to the current input mode. If the currently active mode is a keyboard mode, projector 110 projects a keyboard guide 203A, as depicted in FIGS. 1A and 6. If the currently active mode is a mouse mode, projector 110 projects a mouse guide 203B, as depicted in FIGS. 1B and 7. Projector 110 switches from one guide to another in response to input mode changes.
  • [0070]
    Alternatively, in another embodiment projector 110 does not switch guides 203 automatically. Users may find repeated guide-switching distracting. Accordingly, in one embodiment, input guide 203 for a first input mode (e.g. keyboard mode) continues to be projected even when device 101 switches to a second input mode (e.g. mouse mode). In another embodiment, input guide 203 is projected as the superposition of input guides 203 for two or more input modes. For example, in FIG. 1C, input guide 203C is the superposition of a keyboard input guide and a mouse input guide. For clarity, in one embodiment, the two input guides being superimposed are projected in different colors, or are otherwise rendered visually distinct from one another.
  • [0071]
    In some embodiments, any or all of the above-described input guide 203 projection schemes are user-configurable. Thus, for example, device 101 may provide configuration options allowing a user to specify whether, and under which conditions, a particular type of input guide 203 is projected at any given time.
  • [0072]
    In yet other embodiments, some or all of the guides described above are printed on a flat surface (such as a piece of paper), rather than or in addition to being projected by projector 110.
  • [0073]
    In yet other embodiments, one or more three-dimensional guides may be used. A three-dimensional guide could be implemented as a two-dimensional drawing of a three-dimensional action that accomplishes a mode-change (or performs some other action) or it could, in fact, be a three-dimensional image projected, for example, as a hologram.
  • [0074]
    Method of Operation
  • [0075]
    Referring now to FIG. 4, there is shown a flowchart depicting a method of providing multiple input modes in the same physical space according to one embodiment of the invention.
  • [0076]
    Device 101 starts 400 in one of the input modes (for example, it may start in the keyboard mode). An appropriate input guide 203 is projected 401. The user provides input via finger movements on work surface 204 (for example, by typing on virtual keys), and device 101 detects and interprets 402 the finger movements using techniques described in the above-referenced related patent applications.
  • [0077]
    Device 101 detects 403 a mode-change command, which instructs device 101 to change to another input mode. As described above, in some embodiments, a mode-change command is not needed, and device 101 can change modes automatically depending on the nature of the detected input.
  • [0078]
    Device 101 then changes input mode 404 so that it now detects movements corresponding to the new mode. For example, if the user indicated that he or she is about to start performing mouse input, device 101 would change to a mouse input mode. In one embodiment, the input modes are implemented using lookup tables defining each layout and multiple-state machines.
  • [0079]
    If additional mode changes are specified, steps 402 through 404 are repeated. Otherwise, if the user is done with input 405, the method ends 406.
  • EXAMPLE
  • [0080]
    Referring now to FIG. 3, there is shown an example of a keyboard guide 203AA, that projector 110 projects onto an inert work surface 204 according to one embodiment, and that facilitates both a keyboard mode and a mouse mode.
  • [0081]
    Sensors 107, 109 detect the user's finger movements with respect to the virtual keys shown in the keyboard guide 203AA. As described in related applications cross-referenced above, sensors 107, 109 detect user contact with the virtual keys, and device 101 interprets the contact as a keystroke.
  • [0082]
    The user touches cross-hair 301 to switch to a mouse input mode. In one embodiment, some indication of mouse input mode is presented, for example by altering the color, brightness, or other characteristic of keyboard guide 203AA. The user places a finger on cross-hair 302 and moves his or her finger around to control an on-screen cursor. Sensors 107, 109 detect the x-y coordinates of the touch point as the user moves his or her finger around. Device 101 interprets these coordinates as mouse movement commands, and can further detect and interpret common mouse behaviors such as acceleration, clicking, double-clicking, and the like.
  • [0083]
    Changing Sensory Input Interpretation Techniques According to Mode
  • [0084]
    In one embodiment, the present invention changes its sensory input interpretation techniques depending on the current mode, so as to more accurately capture and interpret input in different modes. For example, the device may use different sensory input techniques for detecting mouse input, as opposed to detecting keyboard input. Mouse input movement differs from keyboard input movement. For example, keyboard input tends to include tapping movements that are perpendicular to the work surface; mouse input tends to include movement in the plane of the mouse pad.
  • [0085]
    Events associated with a mouse are different from keyboard events as well. While the mouse buttons are processed similarly to keyboard keys, the mouse pointer (or other pointing device) has no up or down event; it either rolls on the surface, or it does not. Lifting a mouse or leaving it in place has approximately the same effect. In addition, the main output from a mouse device driver is a sequence of coordinate pairs (plus button events), while keyboard output generally includes key identifiers corresponding to the struck keys. Finally, users often wish to shift the position of the mouse without moving the on-screen cursor, an operation that is typically done with a physical mouse by lifting the mouse off of the surface and replacing it in a different position; this is referred to as “planing.”
  • [0086]
    When interpreting keyboard input, the system of the invention determines whether there is contact between a finger and surface 204 either by comparing the height of the finger's image blob against a calibration table of expected blob heights, or by analyzing the blob's motion. As described above, since keyboard motion is essentially vertical, contact can be deemed to have occurred when a blob descends quickly and then stops abruptly.
  • [0087]
    When interpreting mouse input, as mentioned above, vertical motion tends to be small and unpredictable. Thus, rather than detect abrupt changes in blob descent, the invention uses blob height to distinguish sliding (moving the virtual mouse with the intention of providing input) from planing (adjusting the position of the virtual mouse without intending to provide input). Furthermore, as a finger slides on the pad, the height of its image blob can change as a result of rather unpredictable factors, such as variations in the tilt and orientation of the finger, and in the pressure it exerts on the pad, which in turn causes the fingertip to deform.
  • [0088]
    In one embodiment, the system of the invention establishes thresholds that are used to determine whether the user intends to glide or plane. If the user's fingers are above a certain threshold height, the motion is considered to be a plane, and the onscreen cursor is not moved. If the user's fingers are below the threshold height, the motion is considered to be a glide, and the onscreen cursor is moved accordingly.
  • [0089]
    In one embodiment, two thresholds are defined, one for contact and one for release. The release threshold is smaller than the contact threshold. When a finger first appears, the height of its image blob must exceed the contact threshold before contact is declared. Contact is terminated when the blob height goes below the release threshold. This hysteresis confers some stability to the discrimination between sliding and planing.
  • [0090]
    Key Occlusions
  • [0091]
    As described above, in one embodiment the device of the present invention changes modes in response to a user's finger movement specifying a mode change command. In some situations, the finger movement specifying a mode change command may be obscured from the view of sensor 107. For example, it is often the case that one of the user's fingers obscures another finger. In one embodiment, upon detection of an up-event (keystroke release) the present invention delays the up-event for a key for a certain number of frames. If after the predetermined number of frames have passed, sensor 107 still detects the finger in the down position, the up-event is discarded; the assumption is made that the up-event was merely the result of an occlusion. If the finger is in the up position after the predetermined number of frames has passed, the up-event is passed to the application.
  • [0092]
    One skilled in the art will recognize that finger occlusion may take place in connection with any finger movements, and is not limited to mode change commands. Thus, the following discussion is applicable to any user input, and is not restricted in applicability to mode change commands.
  • [0093]
    Referring now to FIGS. 8 through 10, there are shown additional examples of occlusion. The following description sets forth a technique for handling these occlusion situations according to one embodiment of the invention.
  • [0094]
    In FIG. 8, Finger A descends 800, and then Finger B descends behind finger A 801. Finger B becomes visible when finger A ascends 802. Finger B then ascends 803. Since Finger B is occluded by Finger A, sensor 107 does not detect the keypress represented by Finger B until Finger A has ascended in 802. The system therefore recognizes a down event in 802 rather than in 801. In one embodiment, the system transmits the down event to host device 101 two frames after Finger A has ascended in 802.
  • [0095]
    In FIG. 9, Finger B descends 900, and then Finger A moves in front of finger B 901. Finger B ascends while Finger A stays down 902, and then Finger A ascends 903. Since sensor 107 cannot detect Finger B's ascent in 902, an up event for Finger B is recognized in 903 rather than in 902.
  • [0096]
    In FIG. 10, Finger B descends 1000, and then Finger A moves in front of finger B 1001. Finger A ascends while Finger B stays down 1002, and then Finger B ascends 1003. This case should behave exactly as a mechanical keyboard.
  • [0097]
    Other Applications
  • [0098]
    The above descriptions sets for the invention as applied to keyboard and mouse input modes. One skilled in the art will recognize that other virtual input device combinations can be implemented using the present invention. Examples of such virtual input device combinations include, without limitation:
  • [0099]
    Keyboard/gesture-editing facilities. A virtual keyboard is used to type characters and then a secondary function is implemented to allow editing using finger or hand gestures.
  • [0100]
    Musical instruments. A virtual electronic piano or other instrument could be created that allows users to play musical notes as well as different percussion instruments (such as percussion instruments) within the same area of work surface 204.
  • [0101]
    Video Games. Enabling different functions within the same physical area can optimize Videogames. For instance, one set of functions causes a missile to be fired, while a second set of functions causes a bomb to be dropped.
  • [0102]
    Point-of-sale terminals. A keyboard is used to enter data specific to a product (for instance, push a button to purchase a product) while a secondary function could be used to enter a name to track the order. Depending on context, the input mode would change from being a general keyboard to a product-specific one.
  • [0103]
    Context-sensitive input device. Input mode can be changed depending on the context of the interaction, or under host system control or instruction. For example, a numeric virtual keyboard mode can be activated when the context of the user interaction dictates that numeric input is expected.
  • [0104]
    Automotive applications. A set of gestures (for example, thumbs-up and thumbs-down) may be used to turn the radio up and down in the car. Another set of gestures (for example, point upwards and point downwards) may be used to turn the air-conditioning up and down.
  • [0105]
    In the above description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these specific details. In other instances, structures and devices are shown in block diagram form in order to avoid obscuring the invention.
  • [0106]
    Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.
  • [0107]
    As will be understood by those familiar with the art, the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. For example, the particular architectures depicted above are merely exemplary of one implementation of the present invention. The functional elements and method steps described above are provided as illustrative examples of one technique for implementing the invention; one skilled in the art will recognize that many other implementations are possible without departing from the present invention as recited in the claims. Likewise, the particular capitalization or naming of the modules, protocols, features, attributes, or any other aspect is not mandatory or significant, and the mechanisms that implement the invention or its features may have different names or formats. In addition, the present invention may be implemented as a method, process, user interface, computer program product, system, apparatus, or any combination thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

Claims (23)

What is claimed is:
1. An input device having for detecting user input in at least two input modes, comprising:
a sensor, for:
responsive to the input device being in a first input mode, detecting user movement on or proximate to an inert surface within a first physical space, and generating a signal responsive to the detected movement; and
responsive to the input device being in a second input mode, detecting user movement on or proximate to an inert surface within a second physical space, and generating a signal responsive to the detected movement; and
a processor, coupled to the sensor, for:
responsive to the input device being in the first input mode, receiving and processing the detected signal according to the first input mode; and
responsive to the input device being in the second input mode, receiving and processing the detected signal according to the second input mode;
wherein at least a portion of the second physical space overlaps at least a portion of the first physical space.
2. The input device of claim 1, wherein the first input mode is a keyboard mode and the second input mode is a mouse input mode.
3. The input device of claim 1, wherein the second physical space is coextensive with the first physical space.
4. The input device of claim 1, further comprising:
a mode controller, coupled to the processor, for switching from one of the input modes to another of the input modes.
5. The input device of claim 1, wherein:
the processor switches from one of the input modes to another of the input modes.
6. The input device of claim 1, further comprising:
a mode controller, coupled to the processor, for, responsive to a user command, switching from one of the input modes to another of the input modes.
7. The input device of claim 1, wherein:
responsive to a user command, the processor switches from one of the input modes to another of the input modes.
8. The input device of claim 1, further comprising:
a mode controller, coupled to the sensor, for, responsive to at least one characteristic of the detected finger movement, automatically switching from one of the input modes to another of the input modes.
9. The input device of claim 1, wherein:
responsive to at least one characteristic of the detected finger movement, the processor automatically switches from one of the input modes to another of the input modes.
10. The input device of claim 1, further comprising:
a projector, for projecting an input guide adapted to assist the user in providing input according to at least one of the input modes.
11. The input device of claim 1, further comprising:
a projector, for:
responsive to the input device being in the first input mode, projecting an input guide adapted to assist the user in providing input according to the first input mode; and
responsive to the input device being in the second input mode, projecting an input guide adapted to assist the user in providing input according to the second input mode.
12. The input device of claim 1, further comprising:
a projector, for simultaneously projecting at least two input guides adapted to assist the user in providing input according to at least two of the input modes.
13. The input device of claim 12, wherein the projector projects each input guide in a different color.
14. A method for detecting user input in at least two input modes, comprising:
responsive to the input device being in a first input mode:
detecting user movement on or proximate to an inert surface within a first physical space;
generating a signal responsive to the detected movement; and
processing the detected signal according to a first input mode; and
responsive to the input device being in a second input mode:
detecting user movement on or proximate to an inert surface within a second physical space;
generating a signal responsive to the detected movement; and
processing the detected signal according to a first input mode;
wherein at least a portion of the second physical space overlaps at least a portion of the first physical space.
15. The method of claim 14, wherein the first input mode is a keyboard mode and the second input mode is a mouse input mode.
16. The method of claim 14, wherein the second physical space is coextensive with the first physical space.
17. The method of claim 14, further comprising:
switching from one of the input modes to another of the input modes; and
repeating the detecting, generating, and processing steps.
18. The method of claim 14, further comprising:
receiving a user command indicating a mode switch; responsive to the user command, switching from one of the input modes to another of the input modes; and
repeating the detecting, generating, and processing steps.
19. The method of claim 14, further comprising:
responsive to at least one characteristic of the detected user movement, automatically switching from one of the input modes to another of the input modes; and
repeating the detecting, generating, and processing steps.
20. The method of claim 14, further comprising:
projecting an input guide adapted to assist the user in providing input according to at least one of the input modes.
21. The method of claim 14, further comprising:
responsive to the input device being in the first input mode, projecting an input guide adapted to assist the user in providing input according to the first input mode; and
responsive to the input device being in the second input mode, projecting an input guide adapted to assist the user in providing input according to the second input mode.
22. The method of claim 14, further comprising:
simultaneously projecting at least two input guides adapted to assist the user in providing input according to at least two of the input modes.
23. The method of claim 22, wherein simultaneously projecting at least two input guides comprises projecting each input guide in a different color.
US10367609 1999-11-04 2003-02-13 Multiple input modes in overlapping physical space Abandoned US20030174125A1 (en)

Priority Applications (11)

Application Number Priority Date Filing Date Title
US16344599 true 1999-11-04 1999-11-04
US09502499 US6614422B1 (en) 1999-11-04 2000-02-11 Method and apparatus for entering data using a virtual input device
US23118400 true 2000-09-07 2000-09-07
US28131401 true 2001-04-03 2001-04-03
US30054201 true 2001-06-22 2001-06-22
US33708601 true 2001-11-27 2001-11-27
US33923401 true 2001-12-07 2001-12-07
US35773502 true 2002-02-15 2002-02-15
US38289902 true 2002-05-22 2002-05-22
US10313939 US20030132921A1 (en) 1999-11-04 2002-12-05 Portable sensory input device
US10367609 US20030174125A1 (en) 1999-11-04 2003-02-13 Multiple input modes in overlapping physical space

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10367609 US20030174125A1 (en) 1999-11-04 2003-02-13 Multiple input modes in overlapping physical space
PCT/US2003/004530 WO2003071411A1 (en) 2002-02-15 2003-02-14 Multiple input modes in overlapping physical space

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
US09502499 Continuation-In-Part US6614422B1 (en) 1999-11-04 2000-02-11 Method and apparatus for entering data using a virtual input device
US10313939 Continuation-In-Part US20030132921A1 (en) 1999-11-04 2002-12-05 Portable sensory input device

Publications (1)

Publication Number Publication Date
US20030174125A1 true true US20030174125A1 (en) 2003-09-18

Family

ID=28047088

Family Applications (1)

Application Number Title Priority Date Filing Date
US10367609 Abandoned US20030174125A1 (en) 1999-11-04 2003-02-13 Multiple input modes in overlapping physical space

Country Status (1)

Country Link
US (1) US20030174125A1 (en)

Cited By (77)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020075239A1 (en) * 2000-12-15 2002-06-20 Ari Potkonen Method and arrangement for accomplishing a function in an electronic apparatus and an electronic apparatus
US20030132950A1 (en) * 2001-11-27 2003-07-17 Fahri Surucu Detecting, classifying, and interpreting input events based on stimuli in multiple sensory domains
US20040246338A1 (en) * 2002-06-26 2004-12-09 Klony Lieberman Multifunctional integrated image sensor and application to virtual interface technology
US20050111700A1 (en) * 2003-10-03 2005-05-26 O'boyle Michael E. Occupant detection system
US20050141752A1 (en) * 2003-12-31 2005-06-30 France Telecom, S.A. Dynamically modifiable keyboard-style interface
WO2005064439A2 (en) * 2003-12-31 2005-07-14 France Telecom Dynamically modifiable virtual keyboard or virtual mouse interface
US6968073B1 (en) 2001-04-24 2005-11-22 Automotive Systems Laboratory, Inc. Occupant detection system
WO2006013345A2 (en) * 2004-08-03 2006-02-09 Anthony Allison A touchpad device
US20060241371A1 (en) * 2005-02-08 2006-10-26 Canesta, Inc. Method and system to correct motion blur in time-of-flight sensor systems
US20070029373A1 (en) * 2005-08-03 2007-02-08 Bumiller George B Handheld electronic device providing assisted entry of contact information, and associated method
US20070109278A1 (en) * 2005-11-11 2007-05-17 Samsung Electronics Co., Ltd. Input apparatus and method using optical sensing, and portable terminal using the same
WO2007144014A1 (en) * 2006-06-15 2007-12-21 Nokia Corporation Mobile device with virtual keypad
US20080030380A1 (en) * 2007-01-25 2008-02-07 Microsoft Corporation Input device
US20080030470A1 (en) * 2007-01-25 2008-02-07 Microsoft Corporation Automatic mode determination for an input device
GB2444852A (en) * 2006-12-13 2008-06-18 Compurants Ltd Interactive Food And/Or Drink Ordering System
WO2008131544A1 (en) * 2007-04-26 2008-11-06 University Of Manitoba Pressure augmented mouse
US20080316183A1 (en) * 2007-06-22 2008-12-25 Apple Inc. Swipe gestures for touch screen keyboards
US20090147272A1 (en) * 2007-12-05 2009-06-11 Microvision, Inc. Proximity detection for control of an imaging device
US20090231281A1 (en) * 2008-03-11 2009-09-17 Microsoft Corporation Multi-touch virtual keyboard
US20090244019A1 (en) * 2008-03-26 2009-10-01 Lg Electronics Inc. Terminal and method of controlling the same
DE102008036762A1 (en) * 2008-08-07 2010-02-11 Airbus Deutschland Gmbh Control and display system for use in e.g. aircraft, has processor analyzing camera signals such that position within reflection surface is assigned to manual control process, and control signal is displayed for assigned position
US20100053591A1 (en) * 2007-12-05 2010-03-04 Microvision, Inc. Scanned Proximity Detection Method and Apparatus for a Scanned Image Projection System
US20100077857A1 (en) * 2008-09-30 2010-04-01 Zhou Ye Inertia sensing module
US20100134421A1 (en) * 2008-11-30 2010-06-03 Lenovo (Singapore) Pte. Ltd., Singapore Combined tap sequence and camera based user interface
US20100194870A1 (en) * 2007-08-01 2010-08-05 Ovidiu Ghita Ultra-compact aperture controlled depth from defocus range sensor
GB2470653A (en) * 2009-05-26 2010-12-01 Zienon L L C Configuring the settings of a virtual input device.
US20110099299A1 (en) * 2009-10-28 2011-04-28 Microsoft Corporation Mode Switching
US8018579B1 (en) * 2005-10-21 2011-09-13 Apple Inc. Three-dimensional imaging and display system
WO2011147561A3 (en) * 2010-05-28 2012-04-12 Chao Zhang Mobile unit, method for operating the same and network comprising the mobile unit
US20120162077A1 (en) * 2010-01-06 2012-06-28 Celluon, Inc. System and method for a virtual multi-touch mouse and stylus apparatus
WO2012094489A1 (en) * 2011-01-05 2012-07-12 Autodesk, Inc. Multi-touch integrated desktop environment
US20120218188A1 (en) * 2011-02-24 2012-08-30 Tatsuki Kashitani Information processing apparatus, information processing method, and terminal apparatus
US20120242659A1 (en) * 2011-03-25 2012-09-27 Hon Hai Precision Industry Co., Ltd. Method of controlling electronic device via a virtual keyboard
US20120249409A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing user interfaces
US20130076697A1 (en) * 2004-04-29 2013-03-28 Neonode Inc. Light-based touch screen
US20130082935A1 (en) * 2011-09-30 2013-04-04 Microsoft Corporation Dynamic command presentation and key configuration for keyboards
US20130120319A1 (en) * 2005-10-31 2013-05-16 Extreme Reality Ltd. Methods, Systems, Apparatuses, Circuits and Associated Computer Executable Code for Detecting Motion, Position and/or Orientation of Objects Within a Defined Spatial Region
US20130127718A1 (en) * 2011-11-23 2013-05-23 Phihong Technology Co.,Ltd. Method for Operating Computer Objects and Computer Program Product Thereof
WO2013009482A3 (en) * 2011-07-12 2013-05-30 Google Inc. Methods and systems for a virtual input device
US20130246565A1 (en) * 2011-09-19 2013-09-19 Qualcomn Incorporated Sending human input device commands over internet protocol
US20130271369A1 (en) * 2012-04-17 2013-10-17 Pixart Imaging Inc. Electronic system
US20140055364A1 (en) * 2012-08-23 2014-02-27 Celluon, Inc. System and method for a virtual keyboard
US20140145958A1 (en) * 2012-11-27 2014-05-29 Inventec Corporation Tablet computer assembly, accessory thereof, and tablet computer input method
US20150054730A1 (en) * 2013-08-23 2015-02-26 Sony Corporation Wristband type information processing apparatus and storage medium
US8971572B1 (en) 2011-08-12 2015-03-03 The Research Foundation For The State University Of New York Hand pointing estimation for human computer interaction
US8988366B2 (en) 2011-01-05 2015-03-24 Autodesk, Inc Multi-touch integrated desktop environment
WO2013126905A3 (en) * 2012-02-24 2015-04-02 Moscarillo Thomas J Gesture recognition devices and methods
WO2015077486A1 (en) * 2013-11-22 2015-05-28 Pekall, LLC Wearable projection device
US9052583B2 (en) 2012-07-13 2015-06-09 Lite-On Technology Corporation Portable electronic device with multiple projecting functions
US20150160738A1 (en) * 2012-08-20 2015-06-11 David S. LITHWICK Keyboard projection system with image subtraction
WO2015095218A1 (en) * 2013-12-16 2015-06-25 Cirque Corporation Configuring touchpad behavior through gestures
US20150205374A1 (en) * 2014-01-20 2015-07-23 Beijing Lenovo Software Ltd. Information processing method and electronic device
US9092665B2 (en) 2013-01-30 2015-07-28 Aquifi, Inc Systems and methods for initializing motion tracking of human hands
US9092136B1 (en) * 2011-06-08 2015-07-28 Rockwell Collins, Inc. Projected button display system
US9098739B2 (en) 2012-06-25 2015-08-04 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching
US9111135B2 (en) 2012-06-25 2015-08-18 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching using corresponding pixels in bounded regions of a sequence of frames that are a specified distance interval from a reference camera
US9129155B2 (en) 2013-01-30 2015-09-08 Aquifi, Inc. Systems and methods for initializing motion tracking of human hands using template matching within bounded regions determined using a depth map
US9164625B2 (en) 2012-10-14 2015-10-20 Neonode Inc. Proximity sensor for determining two-dimensional coordinates of a proximal object
US9262005B2 (en) 2011-01-05 2016-02-16 Autodesk, Inc. Multi-touch integrated desktop environment
US9298266B2 (en) 2013-04-02 2016-03-29 Aquifi, Inc. Systems and methods for implementing three-dimensional (3D) gesture based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US20160092031A1 (en) * 2014-09-25 2016-03-31 Serafim Technologies Inc. Virtual two-dimensional positioning module of input device and virtual device with the same
US9310891B2 (en) 2012-09-04 2016-04-12 Aquifi, Inc. Method and system enabling natural user interface gestures with user wearable glasses
US20160139676A1 (en) * 2008-03-03 2016-05-19 Disney Enterprises, Inc. System and/or method for processing three dimensional images
US9430035B2 (en) 2011-12-30 2016-08-30 Intel Corporation Interactive drawing recognition
US9477310B2 (en) * 2006-07-16 2016-10-25 Ibrahim Farid Cherradi El Fadili Free fingers typing technology
US9504920B2 (en) 2011-04-25 2016-11-29 Aquifi, Inc. Method and system to create three-dimensional mapping in a two-dimensional game
US9507417B2 (en) 2014-01-07 2016-11-29 Aquifi, Inc. Systems and methods for implementing head tracking based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US9600078B2 (en) 2012-02-03 2017-03-21 Aquifi, Inc. Method and system enabling natural user interface gestures with an electronic system
US9600090B2 (en) 2011-01-05 2017-03-21 Autodesk, Inc. Multi-touch integrated desktop environment
US9619105B1 (en) 2014-01-30 2017-04-11 Aquifi, Inc. Systems and methods for gesture based interaction with viewpoint dependent user interfaces
US20170161903A1 (en) * 2015-12-03 2017-06-08 Calay Venture S.á r.l. Method and apparatus for gesture recognition
US9703389B2 (en) * 2012-12-24 2017-07-11 Peigen Jiang Computer input device
US9741184B2 (en) 2012-10-14 2017-08-22 Neonode Inc. Door handle with optical proximity sensors
US9740338B2 (en) 2014-05-22 2017-08-22 Ubi interactive inc. System and methods for providing a three-dimensional touch screen
US9798388B1 (en) 2013-07-31 2017-10-24 Aquifi, Inc. Vibrotactile system to augment 3D input systems
US9857868B2 (en) 2011-03-19 2018-01-02 The Board Of Trustees Of The Leland Stanford Junior University Method and system for ergonomic touch-free interface
US9880629B2 (en) 2013-02-25 2018-01-30 Thomas J. Moscarillo Gesture recognition devices and methods with user authentication

Citations (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4131760A (en) * 1977-12-07 1978-12-26 Bell Telephone Laboratories, Incorporated Multiple microphone dereverberation system
US4295706A (en) * 1979-07-30 1981-10-20 Frost George H Combined lens cap and sunshade for a camera
US4311874A (en) * 1979-12-17 1982-01-19 Bell Telephone Laboratories, Incorporated Teleconference microphone arrays
US4485484A (en) * 1982-10-28 1984-11-27 At&T Bell Laboratories Directable microphone system
US4914624A (en) * 1988-05-06 1990-04-03 Dunthorn David I Virtual button for touch screen
US5404458A (en) * 1991-10-10 1995-04-04 International Business Machines Corporation Recognizing the cessation of motion of a pointing device on a display by comparing a group of signals to an anchor point
US5461441A (en) * 1993-06-25 1995-10-24 Nikon Corporation Camera with switching mechanism for selective operation of a retractable lens barrel and closeable lens barrier and method of operation
US5477323A (en) * 1992-11-06 1995-12-19 Martin Marietta Corporation Fiber optic strain sensor and read-out system
US5528263A (en) * 1994-06-15 1996-06-18 Daniel M. Platzker Interactive projected video image display system
US5691748A (en) * 1994-04-02 1997-11-25 Wacom Co., Ltd Computer system having multi-device input system
US5767842A (en) * 1992-02-07 1998-06-16 International Business Machines Corporation Method and device for optical input of commands or data
USD395640S (en) * 1996-01-02 1998-06-30 International Business Machines Corporation Holder for portable computing device
US5784504A (en) * 1992-04-15 1998-07-21 International Business Machines Corporation Disambiguating input strokes of a stylus-based input devices for gesture or character recognition
US5838495A (en) * 1996-03-25 1998-11-17 Welch Allyn, Inc. Image sensor containment system
US5864334A (en) * 1997-06-27 1999-01-26 Compaq Computer Corporation Computer keyboard with switchable typing/cursor control modes
US5917476A (en) * 1996-09-24 1999-06-29 Czerniecki; George V. Cursor feedback text input method
US5959612A (en) * 1994-02-15 1999-09-28 Breyer; Branko Computer pointing device
US5995026A (en) * 1997-10-21 1999-11-30 Compaq Computer Corporation Programmable multiple output force-sensing keyboard
US6002808A (en) * 1996-07-26 1999-12-14 Mitsubishi Electric Information Technology Center America, Inc. Hand gesture control system
US6037882A (en) * 1997-09-30 2000-03-14 Levy; David H. Method and apparatus for inputting data to an electronic system
US6043805A (en) * 1998-03-24 2000-03-28 Hsieh; Kuan-Hong Controlling method for inputting messages to a computer
US6097374A (en) * 1997-03-06 2000-08-01 Howard; Robert Bruce Wrist-pendent wireless optical keyboard
US6115482A (en) * 1996-02-13 2000-09-05 Ascent Technology, Inc. Voice-output reading system with gesture-based navigation
US6128007A (en) * 1996-07-29 2000-10-03 Motorola, Inc. Method and apparatus for multi-mode handwritten input and hand directed control of a computing device
US6191773B1 (en) * 1995-04-28 2001-02-20 Matsushita Electric Industrial Co., Ltd. Interface apparatus
US6195589B1 (en) * 1998-03-09 2001-02-27 3Com Corporation Personal data assistant with remote control capabilities
US6204852B1 (en) * 1998-12-09 2001-03-20 Lucent Technologies Inc. Video hand image three-dimensional computer interface
US6211863B1 (en) * 1998-05-14 2001-04-03 Virtual Ink. Corp. Method and software for enabling use of transcription system as a mouse
USD440542S1 (en) * 1996-11-04 2001-04-17 Palm Computing, Inc. Pocket-size organizer with stand
US6232960B1 (en) * 1995-12-21 2001-05-15 Alfred Goldman Data input device
US6252598B1 (en) * 1997-07-03 2001-06-26 Lucent Technologies Inc. Video hand image computer interface
US6266048B1 (en) * 1998-08-27 2001-07-24 Hewlett-Packard Company Method and apparatus for a virtual display/keyboard for a PDA
US6275214B1 (en) * 1999-07-06 2001-08-14 Karl C. Hansen Computer presentation system and method with optical tracking of wireless pointer
US6281878B1 (en) * 1994-11-01 2001-08-28 Stephen V. R. Montellese Apparatus and method for inputing data
US6283860B1 (en) * 1995-11-07 2001-09-04 Philips Electronics North America Corp. Method, system, and program for gesture based option selection
US6323942B1 (en) * 1999-04-30 2001-11-27 Canesta, Inc. CMOS-compatible three-dimensional image sensor IC
US6356442B1 (en) * 1999-02-04 2002-03-12 Palm, Inc Electronically-enabled encasement for a handheld computer
US6535199B1 (en) * 1999-02-04 2003-03-18 Palm, Inc. Smart cover for a handheld computer
US6611253B1 (en) * 2000-09-19 2003-08-26 Harel Cohen Virtual input environment
US6611252B1 (en) * 2000-05-17 2003-08-26 Dufaux Douglas P. Virtual data input device
US6650318B1 (en) * 2000-10-13 2003-11-18 Vkb Inc. Data input device
US6657654B2 (en) * 1998-04-29 2003-12-02 International Business Machines Corporation Camera for use with personal digital assistants with high speed communication link
US6690357B1 (en) * 1998-10-07 2004-02-10 Intel Corporation Input device using scanning sensors
US6750849B2 (en) * 2000-12-15 2004-06-15 Nokia Mobile Phones, Ltd. Method and arrangement for accomplishing a function in an electronic apparatus and an electronic apparatus
US6904535B2 (en) * 2000-08-18 2005-06-07 Fujitsu Limited Information processing device selecting normal and exclusive operational modes according to wake up instructions from a communication interface section or an input/output device
US6952198B2 (en) * 1999-07-06 2005-10-04 Hansen Karl C System and method for communication with enhanced optical pointer
US6977643B2 (en) * 2002-01-10 2005-12-20 International Business Machines Corporation System and method implementing non-physical pointers for computer devices

Patent Citations (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4131760A (en) * 1977-12-07 1978-12-26 Bell Telephone Laboratories, Incorporated Multiple microphone dereverberation system
US4295706A (en) * 1979-07-30 1981-10-20 Frost George H Combined lens cap and sunshade for a camera
US4311874A (en) * 1979-12-17 1982-01-19 Bell Telephone Laboratories, Incorporated Teleconference microphone arrays
US4485484A (en) * 1982-10-28 1984-11-27 At&T Bell Laboratories Directable microphone system
US4914624A (en) * 1988-05-06 1990-04-03 Dunthorn David I Virtual button for touch screen
US5404458A (en) * 1991-10-10 1995-04-04 International Business Machines Corporation Recognizing the cessation of motion of a pointing device on a display by comparing a group of signals to an anchor point
US5767842A (en) * 1992-02-07 1998-06-16 International Business Machines Corporation Method and device for optical input of commands or data
US5784504A (en) * 1992-04-15 1998-07-21 International Business Machines Corporation Disambiguating input strokes of a stylus-based input devices for gesture or character recognition
US5477323A (en) * 1992-11-06 1995-12-19 Martin Marietta Corporation Fiber optic strain sensor and read-out system
US5461441A (en) * 1993-06-25 1995-10-24 Nikon Corporation Camera with switching mechanism for selective operation of a retractable lens barrel and closeable lens barrier and method of operation
US5959612A (en) * 1994-02-15 1999-09-28 Breyer; Branko Computer pointing device
US5691748A (en) * 1994-04-02 1997-11-25 Wacom Co., Ltd Computer system having multi-device input system
US5528263A (en) * 1994-06-15 1996-06-18 Daniel M. Platzker Interactive projected video image display system
US6281878B1 (en) * 1994-11-01 2001-08-28 Stephen V. R. Montellese Apparatus and method for inputing data
US6191773B1 (en) * 1995-04-28 2001-02-20 Matsushita Electric Industrial Co., Ltd. Interface apparatus
US6283860B1 (en) * 1995-11-07 2001-09-04 Philips Electronics North America Corp. Method, system, and program for gesture based option selection
US6232960B1 (en) * 1995-12-21 2001-05-15 Alfred Goldman Data input device
USD395640S (en) * 1996-01-02 1998-06-30 International Business Machines Corporation Holder for portable computing device
US6115482A (en) * 1996-02-13 2000-09-05 Ascent Technology, Inc. Voice-output reading system with gesture-based navigation
US5838495A (en) * 1996-03-25 1998-11-17 Welch Allyn, Inc. Image sensor containment system
US6002808A (en) * 1996-07-26 1999-12-14 Mitsubishi Electric Information Technology Center America, Inc. Hand gesture control system
US6128007A (en) * 1996-07-29 2000-10-03 Motorola, Inc. Method and apparatus for multi-mode handwritten input and hand directed control of a computing device
US5917476A (en) * 1996-09-24 1999-06-29 Czerniecki; George V. Cursor feedback text input method
USD440542S1 (en) * 1996-11-04 2001-04-17 Palm Computing, Inc. Pocket-size organizer with stand
US6097374A (en) * 1997-03-06 2000-08-01 Howard; Robert Bruce Wrist-pendent wireless optical keyboard
US5864334A (en) * 1997-06-27 1999-01-26 Compaq Computer Corporation Computer keyboard with switchable typing/cursor control modes
US6252598B1 (en) * 1997-07-03 2001-06-26 Lucent Technologies Inc. Video hand image computer interface
US6037882A (en) * 1997-09-30 2000-03-14 Levy; David H. Method and apparatus for inputting data to an electronic system
US5995026A (en) * 1997-10-21 1999-11-30 Compaq Computer Corporation Programmable multiple output force-sensing keyboard
US6195589B1 (en) * 1998-03-09 2001-02-27 3Com Corporation Personal data assistant with remote control capabilities
US6043805A (en) * 1998-03-24 2000-03-28 Hsieh; Kuan-Hong Controlling method for inputting messages to a computer
US6657654B2 (en) * 1998-04-29 2003-12-02 International Business Machines Corporation Camera for use with personal digital assistants with high speed communication link
US6211863B1 (en) * 1998-05-14 2001-04-03 Virtual Ink. Corp. Method and software for enabling use of transcription system as a mouse
US6266048B1 (en) * 1998-08-27 2001-07-24 Hewlett-Packard Company Method and apparatus for a virtual display/keyboard for a PDA
US6690357B1 (en) * 1998-10-07 2004-02-10 Intel Corporation Input device using scanning sensors
US6204852B1 (en) * 1998-12-09 2001-03-20 Lucent Technologies Inc. Video hand image three-dimensional computer interface
US6535199B1 (en) * 1999-02-04 2003-03-18 Palm, Inc. Smart cover for a handheld computer
US6356442B1 (en) * 1999-02-04 2002-03-12 Palm, Inc Electronically-enabled encasement for a handheld computer
US6323942B1 (en) * 1999-04-30 2001-11-27 Canesta, Inc. CMOS-compatible three-dimensional image sensor IC
US6952198B2 (en) * 1999-07-06 2005-10-04 Hansen Karl C System and method for communication with enhanced optical pointer
US6275214B1 (en) * 1999-07-06 2001-08-14 Karl C. Hansen Computer presentation system and method with optical tracking of wireless pointer
US6611252B1 (en) * 2000-05-17 2003-08-26 Dufaux Douglas P. Virtual data input device
US6904535B2 (en) * 2000-08-18 2005-06-07 Fujitsu Limited Information processing device selecting normal and exclusive operational modes according to wake up instructions from a communication interface section or an input/output device
US6611253B1 (en) * 2000-09-19 2003-08-26 Harel Cohen Virtual input environment
US6650318B1 (en) * 2000-10-13 2003-11-18 Vkb Inc. Data input device
US6750849B2 (en) * 2000-12-15 2004-06-15 Nokia Mobile Phones, Ltd. Method and arrangement for accomplishing a function in an electronic apparatus and an electronic apparatus
US6977643B2 (en) * 2002-01-10 2005-12-20 International Business Machines Corporation System and method implementing non-physical pointers for computer devices

Cited By (114)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020075239A1 (en) * 2000-12-15 2002-06-20 Ari Potkonen Method and arrangement for accomplishing a function in an electronic apparatus and an electronic apparatus
US6750849B2 (en) * 2000-12-15 2004-06-15 Nokia Mobile Phones, Ltd. Method and arrangement for accomplishing a function in an electronic apparatus and an electronic apparatus
US6968073B1 (en) 2001-04-24 2005-11-22 Automotive Systems Laboratory, Inc. Occupant detection system
US20030132950A1 (en) * 2001-11-27 2003-07-17 Fahri Surucu Detecting, classifying, and interpreting input events based on stimuli in multiple sensory domains
US20040246338A1 (en) * 2002-06-26 2004-12-09 Klony Lieberman Multifunctional integrated image sensor and application to virtual interface technology
US7307661B2 (en) * 2002-06-26 2007-12-11 Vbk Inc. Multifunctional integrated image sensor and application to virtual interface technology
US20050111700A1 (en) * 2003-10-03 2005-05-26 O'boyle Michael E. Occupant detection system
US7406181B2 (en) 2003-10-03 2008-07-29 Automotive Systems Laboratory, Inc. Occupant detection system
US20050141752A1 (en) * 2003-12-31 2005-06-30 France Telecom, S.A. Dynamically modifiable keyboard-style interface
WO2005064439A2 (en) * 2003-12-31 2005-07-14 France Telecom Dynamically modifiable virtual keyboard or virtual mouse interface
WO2005064439A3 (en) * 2003-12-31 2006-04-20 France Telecom Dynamically modifiable virtual keyboard or virtual mouse interface
US20130076697A1 (en) * 2004-04-29 2013-03-28 Neonode Inc. Light-based touch screen
WO2006013345A2 (en) * 2004-08-03 2006-02-09 Anthony Allison A touchpad device
WO2006013345A3 (en) * 2004-08-03 2006-09-28 Anthony Allison A touchpad device
US20060241371A1 (en) * 2005-02-08 2006-10-26 Canesta, Inc. Method and system to correct motion blur in time-of-flight sensor systems
US20070029373A1 (en) * 2005-08-03 2007-02-08 Bumiller George B Handheld electronic device providing assisted entry of contact information, and associated method
US8010610B2 (en) * 2005-08-03 2011-08-30 Research In Motion Limited Handheld electronic device providing assisted entry of contact information, and associated method
US20110298798A1 (en) * 2005-10-21 2011-12-08 Apple Inc. Three-dimensional imaging and display system
US8743345B2 (en) 2005-10-21 2014-06-03 Apple Inc. Three-dimensional imaging and display system
US9766716B2 (en) 2005-10-21 2017-09-19 Apple Inc. Three-dimensional imaging and display system
US8780332B2 (en) * 2005-10-21 2014-07-15 Apple Inc. Three-dimensional imaging and display system
US8018579B1 (en) * 2005-10-21 2011-09-13 Apple Inc. Three-dimensional imaging and display system
US20130120319A1 (en) * 2005-10-31 2013-05-16 Extreme Reality Ltd. Methods, Systems, Apparatuses, Circuits and Associated Computer Executable Code for Detecting Motion, Position and/or Orientation of Objects Within a Defined Spatial Region
US9046962B2 (en) * 2005-10-31 2015-06-02 Extreme Reality Ltd. Methods, systems, apparatuses, circuits and associated computer executable code for detecting motion, position and/or orientation of objects within a defined spatial region
US20070109278A1 (en) * 2005-11-11 2007-05-17 Samsung Electronics Co., Ltd. Input apparatus and method using optical sensing, and portable terminal using the same
US7961176B2 (en) * 2005-11-11 2011-06-14 Samsung Electronics Co., Ltd Input apparatus and method using optical sensing, and portable terminal using the same
US20100214267A1 (en) * 2006-06-15 2010-08-26 Nokia Corporation Mobile device with virtual keypad
WO2007144014A1 (en) * 2006-06-15 2007-12-21 Nokia Corporation Mobile device with virtual keypad
US9477310B2 (en) * 2006-07-16 2016-10-25 Ibrahim Farid Cherradi El Fadili Free fingers typing technology
GB2444852A (en) * 2006-12-13 2008-06-18 Compurants Ltd Interactive Food And/Or Drink Ordering System
GB2444852B (en) * 2006-12-13 2010-01-27 Compurants Ltd Interactive food and drink ordering system
US8614675B2 (en) 2007-01-25 2013-12-24 Microsoft Corporation Automatic mode determination for an input device
US20080030470A1 (en) * 2007-01-25 2008-02-07 Microsoft Corporation Automatic mode determination for an input device
US8928499B2 (en) 2007-01-25 2015-01-06 Microsoft Corporation Input device with multiple sets of input keys
US20080030380A1 (en) * 2007-01-25 2008-02-07 Microsoft Corporation Input device
US20100127983A1 (en) * 2007-04-26 2010-05-27 Pourang Irani Pressure Augmented Mouse
WO2008131544A1 (en) * 2007-04-26 2008-11-06 University Of Manitoba Pressure augmented mouse
US20080316183A1 (en) * 2007-06-22 2008-12-25 Apple Inc. Swipe gestures for touch screen keyboards
US8542206B2 (en) 2007-06-22 2013-09-24 Apple Inc. Swipe gestures for touch screen keyboards
WO2009002787A2 (en) * 2007-06-22 2008-12-31 Apple Inc. Swipe gestures for touch screen keyboards
WO2009002787A3 (en) * 2007-06-22 2009-05-22 Apple Inc Swipe gestures for touch screen keyboards
US8059101B2 (en) 2007-06-22 2011-11-15 Apple Inc. Swipe gestures for touch screen keyboards
US20100194870A1 (en) * 2007-08-01 2010-08-05 Ovidiu Ghita Ultra-compact aperture controlled depth from defocus range sensor
US8251517B2 (en) 2007-12-05 2012-08-28 Microvision, Inc. Scanned proximity detection method and apparatus for a scanned image projection system
US20090147272A1 (en) * 2007-12-05 2009-06-11 Microvision, Inc. Proximity detection for control of an imaging device
US20100053591A1 (en) * 2007-12-05 2010-03-04 Microvision, Inc. Scanned Proximity Detection Method and Apparatus for a Scanned Image Projection System
US20160139676A1 (en) * 2008-03-03 2016-05-19 Disney Enterprises, Inc. System and/or method for processing three dimensional images
US20090231281A1 (en) * 2008-03-11 2009-09-17 Microsoft Corporation Multi-touch virtual keyboard
US9274681B2 (en) * 2008-03-26 2016-03-01 Lg Electronics Inc. Terminal and method of controlling the same
US20090244019A1 (en) * 2008-03-26 2009-10-01 Lg Electronics Inc. Terminal and method of controlling the same
DE102008036762A1 (en) * 2008-08-07 2010-02-11 Airbus Deutschland Gmbh Control and display system for use in e.g. aircraft, has processor analyzing camera signals such that position within reflection surface is assigned to manual control process, and control signal is displayed for assigned position
US20100077857A1 (en) * 2008-09-30 2010-04-01 Zhou Ye Inertia sensing module
US8042391B2 (en) * 2008-09-30 2011-10-25 Cywee Group Limited Inertia sensing module
US20100134421A1 (en) * 2008-11-30 2010-06-03 Lenovo (Singapore) Pte. Ltd., Singapore Combined tap sequence and camera based user interface
US8797274B2 (en) * 2008-11-30 2014-08-05 Lenovo (Singapore) Pte. Ltd. Combined tap sequence and camera based user interface
GB2470653A (en) * 2009-05-26 2010-12-01 Zienon L L C Configuring the settings of a virtual input device.
GB2470653B (en) * 2009-05-26 2015-04-29 Zienon L L C Enabling data entry based on differentiated input objects
US8214546B2 (en) 2009-10-28 2012-07-03 Microsoft Corporation Mode switching
US20110099299A1 (en) * 2009-10-28 2011-04-28 Microsoft Corporation Mode Switching
US8941620B2 (en) * 2010-01-06 2015-01-27 Celluon, Inc. System and method for a virtual multi-touch mouse and stylus apparatus
US20120162077A1 (en) * 2010-01-06 2012-06-28 Celluon, Inc. System and method for a virtual multi-touch mouse and stylus apparatus
WO2011147561A3 (en) * 2010-05-28 2012-04-12 Chao Zhang Mobile unit, method for operating the same and network comprising the mobile unit
US9600090B2 (en) 2011-01-05 2017-03-21 Autodesk, Inc. Multi-touch integrated desktop environment
US8988366B2 (en) 2011-01-05 2015-03-24 Autodesk, Inc Multi-touch integrated desktop environment
US9262005B2 (en) 2011-01-05 2016-02-16 Autodesk, Inc. Multi-touch integrated desktop environment
WO2012094489A1 (en) * 2011-01-05 2012-07-12 Autodesk, Inc. Multi-touch integrated desktop environment
US9612743B2 (en) 2011-01-05 2017-04-04 Autodesk, Inc. Multi-touch integrated desktop environment
US20120218188A1 (en) * 2011-02-24 2012-08-30 Tatsuki Kashitani Information processing apparatus, information processing method, and terminal apparatus
US8957861B2 (en) * 2011-02-24 2015-02-17 Sony Corporation Information processing apparatus, information processing method, and terminal apparatus for superimposing a display of virtual keys upon an input unit
US9857868B2 (en) 2011-03-19 2018-01-02 The Board Of Trustees Of The Leland Stanford Junior University Method and system for ergonomic touch-free interface
US20120242659A1 (en) * 2011-03-25 2012-09-27 Hon Hai Precision Industry Co., Ltd. Method of controlling electronic device via a virtual keyboard
US20120249409A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing user interfaces
WO2012131584A2 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing user interfaces
WO2012131584A3 (en) * 2011-03-31 2012-11-22 Nokia Corporation Method and apparatus for providing projected user interfaces on various surfaces
US9504920B2 (en) 2011-04-25 2016-11-29 Aquifi, Inc. Method and system to create three-dimensional mapping in a two-dimensional game
US9092136B1 (en) * 2011-06-08 2015-07-28 Rockwell Collins, Inc. Projected button display system
US9069164B2 (en) 2011-07-12 2015-06-30 Google Inc. Methods and systems for a virtual input device
WO2013009482A3 (en) * 2011-07-12 2013-05-30 Google Inc. Methods and systems for a virtual input device
US8971572B1 (en) 2011-08-12 2015-03-03 The Research Foundation For The State University Of New York Hand pointing estimation for human computer interaction
US9372546B2 (en) 2011-08-12 2016-06-21 The Research Foundation For The State University Of New York Hand pointing estimation for human computer interaction
US9106651B2 (en) * 2011-09-19 2015-08-11 Qualcomm Incorporated Sending human input device commands over internet protocol
US20130246565A1 (en) * 2011-09-19 2013-09-19 Qualcomn Incorporated Sending human input device commands over internet protocol
US20130082935A1 (en) * 2011-09-30 2013-04-04 Microsoft Corporation Dynamic command presentation and key configuration for keyboards
US20130127718A1 (en) * 2011-11-23 2013-05-23 Phihong Technology Co.,Ltd. Method for Operating Computer Objects and Computer Program Product Thereof
US9430035B2 (en) 2011-12-30 2016-08-30 Intel Corporation Interactive drawing recognition
US9600078B2 (en) 2012-02-03 2017-03-21 Aquifi, Inc. Method and system enabling natural user interface gestures with an electronic system
WO2013126905A3 (en) * 2012-02-24 2015-04-02 Moscarillo Thomas J Gesture recognition devices and methods
US9454257B2 (en) * 2012-04-17 2016-09-27 Pixart Imaging Inc. Electronic system
US20130271369A1 (en) * 2012-04-17 2013-10-17 Pixart Imaging Inc. Electronic system
US9111135B2 (en) 2012-06-25 2015-08-18 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching using corresponding pixels in bounded regions of a sequence of frames that are a specified distance interval from a reference camera
US9098739B2 (en) 2012-06-25 2015-08-04 Aquifi, Inc. Systems and methods for tracking human hands using parts based template matching
US9052583B2 (en) 2012-07-13 2015-06-09 Lite-On Technology Corporation Portable electronic device with multiple projecting functions
US20150160738A1 (en) * 2012-08-20 2015-06-11 David S. LITHWICK Keyboard projection system with image subtraction
US20140055364A1 (en) * 2012-08-23 2014-02-27 Celluon, Inc. System and method for a virtual keyboard
US8937596B2 (en) * 2012-08-23 2015-01-20 Celluon, Inc. System and method for a virtual keyboard
US9310891B2 (en) 2012-09-04 2016-04-12 Aquifi, Inc. Method and system enabling natural user interface gestures with user wearable glasses
US9741184B2 (en) 2012-10-14 2017-08-22 Neonode Inc. Door handle with optical proximity sensors
US9164625B2 (en) 2012-10-14 2015-10-20 Neonode Inc. Proximity sensor for determining two-dimensional coordinates of a proximal object
US20140145958A1 (en) * 2012-11-27 2014-05-29 Inventec Corporation Tablet computer assembly, accessory thereof, and tablet computer input method
US9703389B2 (en) * 2012-12-24 2017-07-11 Peigen Jiang Computer input device
US9129155B2 (en) 2013-01-30 2015-09-08 Aquifi, Inc. Systems and methods for initializing motion tracking of human hands using template matching within bounded regions determined using a depth map
US9092665B2 (en) 2013-01-30 2015-07-28 Aquifi, Inc Systems and methods for initializing motion tracking of human hands
US9880629B2 (en) 2013-02-25 2018-01-30 Thomas J. Moscarillo Gesture recognition devices and methods with user authentication
US9298266B2 (en) 2013-04-02 2016-03-29 Aquifi, Inc. Systems and methods for implementing three-dimensional (3D) gesture based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US9798388B1 (en) 2013-07-31 2017-10-24 Aquifi, Inc. Vibrotactile system to augment 3D input systems
US20150054730A1 (en) * 2013-08-23 2015-02-26 Sony Corporation Wristband type information processing apparatus and storage medium
WO2015077486A1 (en) * 2013-11-22 2015-05-28 Pekall, LLC Wearable projection device
WO2015095218A1 (en) * 2013-12-16 2015-06-25 Cirque Corporation Configuring touchpad behavior through gestures
US9507417B2 (en) 2014-01-07 2016-11-29 Aquifi, Inc. Systems and methods for implementing head tracking based graphical user interfaces (GUI) that incorporate gesture reactive interface objects
US20150205374A1 (en) * 2014-01-20 2015-07-23 Beijing Lenovo Software Ltd. Information processing method and electronic device
US9619105B1 (en) 2014-01-30 2017-04-11 Aquifi, Inc. Systems and methods for gesture based interaction with viewpoint dependent user interfaces
US9740338B2 (en) 2014-05-22 2017-08-22 Ubi interactive inc. System and methods for providing a three-dimensional touch screen
US20160092031A1 (en) * 2014-09-25 2016-03-31 Serafim Technologies Inc. Virtual two-dimensional positioning module of input device and virtual device with the same
US20170161903A1 (en) * 2015-12-03 2017-06-08 Calay Venture S.á r.l. Method and apparatus for gesture recognition

Similar Documents

Publication Publication Date Title
US7168047B1 (en) Mouse having a button-less panning and scrolling switch
US8059101B2 (en) Swipe gestures for touch screen keyboards
US5252951A (en) Graphical user interface with gesture recognition in a multiapplication environment
US6654001B1 (en) Hand-movement-sensing input device
US20110296333A1 (en) User interaction gestures with virtual keyboard
US20060012580A1 (en) Automatic switching for a dual mode digitizer
US20070247431A1 (en) Keypad and sensor combination to provide detection region that overlays keys
US20110006991A1 (en) Image Processing for Camera Based Motion Tracking
US5157384A (en) Advanced user interface
US20090146968A1 (en) Input device, display device, input method, display method, and program
US7576732B2 (en) Scroll control method using a touchpad
US20030048280A1 (en) Interactive environment using computer vision and touchscreens
US20030201982A1 (en) Computer keyboard and cursor control system and method with keyboard map switching
US20100164897A1 (en) Virtual keypad systems and methods
US20100328351A1 (en) User interface
US20030132950A1 (en) Detecting, classifying, and interpreting input events based on stimuli in multiple sensory domains
US20110279397A1 (en) Device and method for monitoring the object's behavior
US20080005703A1 (en) Apparatus, Methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20120019488A1 (en) Stylus for a touchscreen display
US6958749B1 (en) Apparatus and method for manipulating a touch-sensitive display panel
US20030048260A1 (en) System and method for selecting actions based on the identification of user's fingers
US20120235949A1 (en) Dual- sided track pad
CN201156246Y (en) Multiple affair input system
US20050046621A1 (en) Method and device for recognizing a dual point user input on a touch based user input device
US20030043113A1 (en) Coordinates input apparatus having divided coordinates input surface

Legal Events

Date Code Title Description
AS Assignment

Owner name: CANESTA, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TORUNOGLU, ILHAMI;DESAI, APURVA;SZE, CHENG-FENG;AND OTHERS;REEL/FRAME:014071/0859

Effective date: 20030502