EP2941678B1 - Computerschnittstellensystem - Google Patents

Computerschnittstellensystem Download PDF

Info

Publication number
EP2941678B1
EP2941678B1 EP13857223.5A EP13857223A EP2941678B1 EP 2941678 B1 EP2941678 B1 EP 2941678B1 EP 13857223 A EP13857223 A EP 13857223A EP 2941678 B1 EP2941678 B1 EP 2941678B1
Authority
EP
European Patent Office
Prior art keywords
measurements
user
tap
accelerometer
acceleration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP13857223.5A
Other languages
English (en)
French (fr)
Other versions
EP2941678A4 (de
EP2941678A1 (de
Inventor
Eric Jeffrey KELLER
Vinh Vi LAM
Frank Peter LAMBRECHT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Opdig Inc
Original Assignee
Opdig Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Opdig Inc filed Critical Opdig Inc
Publication of EP2941678A1 publication Critical patent/EP2941678A1/de
Publication of EP2941678A4 publication Critical patent/EP2941678A4/de
Application granted granted Critical
Publication of EP2941678B1 publication Critical patent/EP2941678B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233—Character input methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/033—Indexing scheme relating to G06F3/033
    • G06F2203/0331—Finger worn pointing device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0383—Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN

Definitions

  • This disclosure relates to systems and methods for human-computer interaction through hand motions.
  • Computing devices are often equipped with one or more interfaces for interpreting hand motions of a user.
  • the Qwerty keyboard and mouse were the dominant interface devices in the computing field. More recently, the touch screen interface has gained popularity.
  • Another relatively recent development has been the use of hand-held gaming controllers that use accelerometers to detect hand motions of a user.
  • US 2012/139708 A1 describes tracking finger and hand gestures using passive RFID tags.
  • WO 2010/034795 A1 describes a device and method for commanding appliances using gestures.
  • US 6 515 669 B1 describes an operation input device for performing an input operation on the basis of the shape, spatial position, posture and motion of the hand of an operator.
  • the subject matter described in this specification can be embodied in systems that include a first accelerometer and a first fastening article attached to the first accelerometer, the first fastening article capable of holding the first accelerometer in place on a portion of a thumb of a user.
  • the systems may include a second accelerometer and a second fastening article attached to the second accelerometer, the second fastening article capable of holding the second accelerometer in place on a portion of a wrist of the user, where the wrist and the thumb are of a same hand of the user.
  • the first fastening article may be a ring band.
  • the second fastening article may be a wrist band.
  • a system may include a processing device configured to perform operations comprising receiving acceleration measurements from the first accelerometer and detecting, based at least in part on the first set of acceleration measurements, when the thumb of the user wearing the first fastening article and the second fastening article has been tapped against a surface.
  • the processing device may be further configured to perform operations comprising receiving a second set of acceleration measurements from the second accelerometer, when a tap of the thumb is detected, determining, based at least in part on the first set of acceleration measurements and the second set of acceleration measurements, which tap target from among a set of tap targets located on fingers of the hand of the user have been tapped by the thumb.
  • the processing device may be further configured to perform operations comprising mapping the tap of the tap target to a symbol associated with the tap target and transmitting, storing or displaying the symbol.
  • Determining, based at least in part on the first set of acceleration measurements and the second set of acceleration measurements, which tap target from among a set of tap targets located on fingers of the hand of the user have been tapped by the thumb may comprise determining, based at least in part on the first set of acceleration measurements and the second set of acceleration measurements, an orientation of the first accelerometer relative to the second accelerometer and identifying, based on the orientation, a tap target, from among the set of tap targets that was touched during the tap.
  • the operations may include assigning a symbol associated with the tap target to the tap.
  • the operations may include transmitting, storing, or displaying the symbol.
  • a first set of magnetic flux measurements may be received from a first magnetometer that is attached to the thumb of the user.
  • a second set of magnetic flux measurements may be received from a second magnetometer that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on the first set of magnetic flux measurements and the second set of magnetic flux measurements.
  • Determining the orientation may include filtering the first set of acceleration measurements to determine a first estimate of acceleration experienced by the first accelerometer while the thumb was at rest with respect to a hand of the user on the tap target; filtering the second set of acceleration measurements to determine a second estimate of acceleration experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the first set of magnetic flux measurements to determine a first estimate of magnetic flux experienced by the first magnetometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the second set of magnetic flux measurements to determine a second estimate of magnetic flux experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; and determining a rotation that will approximately align the first estimate of acceleration with the second estimate of acceleration and that also will approximately align the first estimate of magnetic flux with the second estimate of magnetic flux.
  • a first set of angular rate measurements may be received from a first gyroscope that is attached to the thumb of the user.
  • a second set of angular rate measurements may be received from a second gyroscope that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on first set of angular rate measurements and the second set of angular rate measurements.
  • Determining the orientation may include integrating the first set of acceleration measurements and the first set of angular rate measurements to determine a first estimate of an orientation, with respect to a reference position, of the first accelerometer at a time during the tap; integrating the second set of acceleration measurements and the second set of angular rate measurements to determine a second estimate of an orientation, with respect to a reference position, of the second accelerometer at the time during the tap; and comparing the first estimate of the orientation of the first accelerometer to the second estimate of the orientation of the second accelerometer.
  • An estimate may be determined, based on the second set of acceleration measurements, of an orientation of an acceleration experienced by the second accelerometer with respect to the axes of the second accelerometer during the tap.
  • a system may include a first gyroscope attached to the first fastening article and a second gyroscope attached to the second fastening article.
  • a system may include a processing device configured to receive acceleration measurements from the first accelerometer and determine, based at least in part the received acceleration measurements, when a thumb of a user wearing the first fastening article and the second fastening article has been tapped against a surface.
  • the processing device may be further configured to receive acceleration measurements from the second accelerometer, receive angular rate measurements from the first gyroscope and from the second gyroscope, and when a tap of the thumb is detected, determine, based at least in part on acceleration measurements from the first accelerometer and the second accelerometer and on angular rate measurements from the first gyroscope and the second gyroscope, which tap target from among a set of tap targets located on fingers of a hand of the user have been tapped by the thumb.
  • the processing device may be further configured to map the tap of the tap target to a symbol associated with the tap target.
  • a system may include two or more wires that connect the first fastening device to the second fastening device.
  • a system may include a wireless transmitter attached to the second fastening article, and the wireless transmitter may be configured to transmit data based on acceleration measurements from the second accelerometer to a computing device.
  • a system may include a wireless transmitter attached to the second fastening article, and the wireless transmitter may be configured to transmit data based on acceleration measurements from the first accelerometer to a computing device.
  • a system may include a processing device configured to receive acceleration measurements from the first accelerometer and determine, based at least in part the received acceleration measurements, when a thumb of a user wearing the first fastening article and the second fastening article has been tapped against a surface.
  • the processing device may be further configured to receive acceleration measurements from the second accelerometer and when a tap of the thumb is detected, determine, based at least in part on acceleration measurements from the first accelerometer and the second accelerometer, which tap target from among a set of tap targets located on fingers of a hand of the user have been tapped by the thumb.
  • the processing device may be further configured to map the tap of the tap target to a symbol associated with the tap target.
  • a system may include a display device configured to present images produced by the processing device and the images may include graphic representations of symbols selected based the data.
  • the display device may be a liquid crystal display.
  • the display device may be a projector.
  • the processing device may be attached to the second fastening article.
  • the processing device may be part of a remote computing device.
  • the subject matter described in this specification can be embodied in methods that include receiving a first set of acceleration measurements from a first accelerometer that is attached to a thumb of a user.
  • the methods may include receiving a second set of acceleration measurements from a second accelerometer that is attached to a wrist of the user, where the wrist and the thumb are of a same hand of the user.
  • the methods may include detecting, based at least in part on the first set of acceleration measurements, when the thumb is tapped on a surface.
  • the methods may include when a tap of the thumb is detected, determining, based at least in part on the first set of acceleration measurements and the second set of acceleration measurements, an orientation of the first accelerometer relative to the second accelerometer.
  • the methods may include identifying, based on the orientation, a tap target, from among a plurality of tap targets located on fingers of the hand of the user, that was touched during the tap.
  • the methods may include assigning a symbol associated with the tap target to the tap.
  • the methods may include transmitting, storing, or displaying the symbol.
  • a first set of magnetic flux measurements may be received from a first magnetometer that is attached to the thumb of the user.
  • a second set of magnetic flux measurements may be received from a second magnetometer that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on the first set of magnetic flux measurements and the second set of magnetic flux measurements.
  • Determining the orientation may include filtering the first set of acceleration measurements to determine a first estimate of acceleration experienced by the first accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the second set of acceleration measurements to determine a second estimate of acceleration experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the first set of magnetic flux measurements to determine a first estimate of magnetic flux experienced by the first magnetometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the second set of magnetic flux measurements to determine a second estimate of magnetic flux experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; and determining a rotation that will approximately align the first estimate of acceleration with the second estimate of acceleration and that also will approximately align the first estimate of magnetic flux with the second estimate of magnetic flux.
  • a first set of angular rate measurements may be received from a first gyroscope that is attached to the thumb of the user.
  • a second set of angular rate measurements may be received from a second gyroscope that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on first set of angular rate measurements and the second set of angular rate measurements.
  • Determining the orientation may include integrating the first set of acceleration measurements and the first set of angular rate measurements to determine a first estimate of an orientation, with respect to a reference position, of the first accelerometer at a time during the tap; integrating the second set of acceleration measurements and the second set of angular rate measurements to determine a second estimate of an orientation, with respect to a reference position, of the second accelerometer at the time during the tap; and comparing the first estimate of the orientation of the first accelerometer to the second estimate of the orientation of the second accelerometer.
  • An estimate may be determined, based on the second set of acceleration measurements, of an orientation of an acceleration experienced by the second accelerometer with respect to the axes of the second accelerometer during the tap.
  • Assigning the symbol to the tap may include selecting the symbol from among a plurality of symbols associated with the tap target based on the orientation of the acceleration experienced by the second accelerometer.
  • the acceleration experienced by the second accelerometer may be dominated by acceleration caused by the gravitational force of the Earth and the selection may be made based at least in part on an estimate of an angle between the gravitational acceleration and an axis running approximately parallel to a forearm of the user.
  • the user may be enabled to indicate a choice from among the plurality of the symbols associated with the tap target, by adjusting an angle of the wrist or an angle of a forearm of the user with respect to the gravitational force of the Earth during the tap.
  • the user may be enabled to turn a palm side of the wrist down to face towards the Earth to enter a cursor manipulation mode, in which acceleration measurements from the second accelerometer are used to move a cursor in a virtual space.
  • Determining the orientation may include filtering the first set of acceleration measurements to determine an estimate of a deceleration vector caused by the impact of the thumb with the tap target.
  • Assigning the symbol to the tap may include selecting the symbol from among a plurality of symbols associated with the tap target based on the orientation of the acceleration experienced by the second accelerometer.
  • the acceleration experienced by the second accelerometer may be dominated by acceleration caused by the gravitational force of the Earth and the selection may be made based at least in part on an estimate of an angle between the gravitational acceleration and an axis running approximately parallel to a forearm of the user.
  • the user may be enabled to indicate a choice from among the plurality of the symbols associated with the tap target, by adjusting the angle of wrist or the angle of a forearm of the user with respect to the gravitational force of the Earth during the tap.
  • the subject matter described in this specification can be embodied in systems that include a wireless network interface configured to receive a first set of acceleration measurements from a first accelerometer that is attached to a thumb of a user.
  • the systems may include a wireless network interface configured to receive a second set of acceleration measurements from a second accelerometer that is attached to a wrist of the user.
  • the systems may include a module configured to detect, based at least in part on the first set of acceleration measurements, when the thumb is tapped on a surface.
  • the systems may include a module configured to, when a tap of the thumb is detected, determine, based at least in part on the first set of acceleration measurements and the second set of acceleration measurements, an orientation of the first accelerometer relative to the second accelerometer.
  • the systems may include a module configured to identify, based on the orientation, a tap target, from among a plurality of tap targets located on fingers of the user, that was touched during the tap.
  • the systems may include a module configured to assign a symbol associated with the tap target to the tap.
  • the systems may include a module configured to transmit, store, or display the symbol.
  • a first set of magnetic flux measurements may be received from a first magnetometer that is attached to the thumb of the user.
  • a second set of magnetic flux measurements may be received from a second magnetometer that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on the first set of magnetic flux measurements and the second set of magnetic flux measurements.
  • Determining the orientation may include filtering the first set acceleration measurements to determine a first estimate of acceleration experienced by the first accelerometer while the thumb was at rest with respect to a hand of the user on the tap target; filtering the second set of acceleration measurements to determine a second estimate of acceleration experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the first set magnetic flux measurements to determine a first estimate of magnetic flux experienced by the first magnetometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the second set of magnetic flux measurements to determine a second estimate of magnetic flux experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; and determining a rotation that will approximately align the first estimate of acceleration with the second estimate of acceleration and that also will approximately align the first estimate of magnetic flux with the second estimate of magnetic flux.
  • a first set of angular rate measurements may be received from a first gyroscope that is attached to the thumb of the user.
  • a second set of angular rate measurements may be received from a second gyroscope that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on first set of angular rate measurements and the second set of angular rate measurements.
  • Determining the orientation may include integrating the first set of acceleration measurements and the first set of angular rate measurements to determine a first estimate of an orientation, with respect to a reference position, of the first accelerometer at a time during the tap; integrating the second set of acceleration measurements and the second set of angular rate measurements to determine a second estimate of an orientation, with respect to a reference position, of the second accelerometer at the time during the tap; and comparing the first estimate of the orientation of the first accelerometer to the second estimate of the orientation of the second accelerometer.
  • An estimate may be determined, based on the second set of acceleration measurements, of an orientation of an acceleration experienced by the second accelerometer with respect to the axes of the second accelerometer during the tap.
  • Assigning the symbol to the tap may include selecting the symbol from among a plurality of symbols associated with the tap target based on the orientation of the acceleration experienced by the second accelerometer.
  • the acceleration experienced by the second accelerometer may be dominated by acceleration caused by the gravitational force of the Earth and the selection may be made based at least in part on an estimate of an angle between the gravitational acceleration and an axis running approximately parallel to a forearm of the user.
  • the user may be enabled to indicate a choice from among the plurality of the symbols associated with the tap target, by adjusting the angle of wrist or the angle of a forearm of the user with respect to the gravitational force of the Earth during the tap.
  • the subject matter described in this specification can be embodied in one or more computer-readable storage media encoded with a computer program.
  • the computer program may include instructions that when executed by data a processing apparatus cause the data processing apparatus to perform operations including receiving a first set of acceleration measurements from a first accelerometer that is attached to a thumb of a user.
  • the operations may include receiving a second set of acceleration measurements from a second accelerometer that is attached to a wrist of the user.
  • the operations may include detecting, based at least in part on the first set of acceleration measurements, when the thumb is tapped on a surface.
  • the operations may include when a tap of the thumb is detected, determining, based at least in part on the first set of acceleration measurements and the second set of acceleration measurements, an orientation of the first accelerometer relative to the second accelerometer.
  • the operations may include identifying, based on the orientation, a tap target, from among a plurality of tap targets located on fingers of the user, that was touched during the tap.
  • the operations may include assigning a symbol associated with the tap target to the tap.
  • the operations may include transmitting, storing, or displaying the symbol.
  • a first set of magnetic flux measurements may be received from a first magnetometer that is attached to the thumb of the user.
  • a second set of magnetic flux measurements may be received from a second magnetometer that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on the first set of magnetic flux measurements and the second set of magnetic flux measurements.
  • Determining the orientation may include filtering the first set acceleration measurements to determine a first estimate of acceleration experienced by the first accelerometer while the thumb was at rest with respect to a hand of the user on the tap target; filtering the second set of acceleration measurements to determine a second estimate of acceleration experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the first set magnetic flux measurements to determine a first estimate of magnetic flux experienced by the first magnetometer while the thumb was at rest with respect to the hand of the user on the tap target; filtering the second set of magnetic flux measurements to determine a second estimate of magnetic flux experienced by the second accelerometer while the thumb was at rest with respect to the hand of the user on the tap target; and determining a rotation that will approximately align the first estimate of acceleration with the second estimate of acceleration and that also will approximately align the first estimate of magnetic flux with the second estimate of magnetic flux.
  • a first set of angular rate measurements may be received from a first gyroscope that is attached to the thumb of the user.
  • a second set of angular rate measurements may be received from a second gyroscope that is attached to the wrist of the user.
  • the orientation may be determined based at least in part on first set of angular rate measurements and the second set of angular rate measurements.
  • Determining the orientation may include integrating the first set of acceleration measurements and the first set of angular rate measurements to determine a first estimate of an orientation, with respect to a reference position, of the first accelerometer at a time during the tap; integrating the second set of acceleration measurements and the second set of angular rate measurements to determine a second estimate of an orientation, with respect to a reference position, of the second accelerometer at the time during the tap; and comparing the first estimate of the orientation of the first accelerometer to the second estimate of the orientation of the second accelerometer.
  • An estimate may be determined, based on the second set of acceleration measurements, of an orientation of an acceleration experienced by the second accelerometer with respect to the axes of the second accelerometer during the tap.
  • Assigning the symbol to the tap may include selecting the symbol from among a plurality of symbols associated with the tap target based on the orientation of the acceleration experienced by the second accelerometer.
  • the acceleration experienced by the second accelerometer may be dominated by acceleration caused by the gravitational force of the Earth and the selection may be made based at least in part on an estimate of an angle between the gravitational acceleration and an axis running approximately parallel to a forearm of the user.
  • the user may be enabled to indicate a choice from among the plurality of the symbols associated with the tap target, by adjusting the angle of wrist or the angle of a forearm of the user with respect to the gravitational force of the Earth during the tap.
  • the subject matter described in this specification can be embodied in systems that include a ring band, a first accelerometer attached to the ring band, a first magnetometer attached to the ring band, a wrist band, a second accelerometer attached to the wrist band, and a second magnetometer attached to the wrist band.
  • the systems may include a gyroscope attached to the wrist band.
  • the systems may include a gyroscope attached to the ring band.
  • the systems may include a wireless transmitter attached to the wrist band.
  • the systems may include a wireless transmitter attached to the ring band.
  • the systems may include two or more wire connecting the ring band to the wrist band.
  • the systems may include a processing device attached to the wrist band.
  • the processing device may be a micro-controller.
  • the processing device may be a microprocessor.
  • the systems may include a processing device attached to the ring band.
  • the subject matter described in this specification can be embodied in systems that include a ring band, a first accelerometer attached to the ring band, a first magnetometer attached to the ring band, a wrist band, a second accelerometer attached to the wrist band, a second magnetometer attached to the wrist band, and a processor configured to receive sensor readings from the first accelerometer, the first magnetometer, the second accelerometer, and the second magnetometer and to detect hand gestures based on these received sensor readings.
  • Detecting hand gestures may include calculating the orientation of the first accelerometer relative to the second accelerometer. Detecting hand gestures may include calculating the orientation of the first accelerometer relative to the second accelerometer. Detecting hand gestures may include calculating the orientation of the first magnetometer relative to the second magnetometer.
  • the subject matter described in this specification can be embodied in methods that include attaching a first sensor component to a first body part so that the sensor component is held in a fixed orientation with respect to the first body part and attaching a second sensor component to a second body part so that the sensor component is held in a fixed orientation with respect to the second body part.
  • the methods may include receiving signal from the first sensor component reflecting readings from a sensor in the first sensor component and receiving signal from the second sensor component reflecting readings from a sensor in the second sensor component.
  • the methods may include estimating the relative orientation of the first sensor component and the second sensor component based in part on the readings from the sensors in the first and second sensor components.
  • the first sensor component may include an accelerometer and a magnetometer
  • the second sensor component may include an accelerometer and a magnetometer
  • the sensor readings may include acceleration measurements and magnetic flux measurements from the first sensor component and the second sensor component.
  • the first sensor component may include an accelerometer and a gyroscope
  • the second sensor component may include an accelerometer and a gyroscope
  • the sensor readings may include acceleration measurements and angular rate measurements from the first sensor component and the second sensor component.
  • the subject matter described in this specification can be embodied in systems that include a fastening article for attaching a first sensor component to a first body part so that the sensor component is held in a fixed orientation with respect to the first body part and a fastening article for attaching a second sensor component to a second body part so that the sensor component is held in a fixed orientation with respect to the second body part.
  • the systems may include a wireless networking interface for receiving signal from the first sensor component reflecting readings from a sensor in the first sensor component and receiving signal from the second sensor component reflecting readings from a sensor in the second sensor component.
  • the systems may include a module configured to estimate the relative orientation of the first sensor component and the second sensor component based in part on the readings from the sensors in the first and second sensor components.
  • the first sensor component may include an accelerometer and a magnetometer
  • the second sensor component may include an accelerometer and a magnetometer
  • the sensor readings may include acceleration measurements and magnetic flux measurements from the first sensor component and the second sensor component.
  • the first sensor component may include an accelerometer and a gyroscope
  • the second sensor component may include an accelerometer and a gyroscope
  • the sensor readings may include acceleration measurements and angular rate measurements from the first sensor component and the second sensor component.
  • the subject matter described in this specification can be embodied in systems that include a ring band, a first accelerometer attached to the ring band, a fastening article configured to hold an object in a fixed orientation with respect to the hand or forearm of a user, and a second accelerometer attached to a fastening article.
  • the systems may include a gyroscope attached to the first fastening article.
  • the systems may include a gyroscope attached to the second fastening article.
  • the systems may include a wireless transmitter attached to the first fastening article.
  • the systems may include a wireless transmitter attached to the second fastening article.
  • the systems may include two or more wire connecting the first fastening article to the second fastening article.
  • the systems may include a processing device attached to the second fastening article
  • the processing device may be a micro-controller.
  • the processing device may be a microprocessor.
  • the systems may include a processing device attached to the first fastening article.
  • the subject matter described in this specification can be embodied in systems that include an accelerometer, a transmitter connected to the accelerometer and configured to transmit information derived from the accelerometer to an external processing device, and a ring band connected to the accelerometer that is capable of holding the accelerometer in place on a thumb of a user.
  • the systems may include a gyroscope connected to the ring band.
  • the systems may include a processing device attached to the ring band.
  • the processing device may be a micro-controller.
  • the processing device may be a microprocessor.
  • the systems may include a receiver configured to receive configuration commands from the external processing device.
  • the subject matter described in this specification can be embodied in systems that include a first accelerometer, a first transmitter connected to the first accelerometer and configured to transmit information derived from the first accelerometer to an external processing device, a ring band connected to the first accelerometer that is capable of holding the first accelerometer in place on a finger of a user, a second accelerometer; a second transmitter connected to the second accelerometer and configured to transmit information derived from the second accelerometer to the external processing device, and a wrist band connected to the second accelerometer that is capable of holding the second accelerometer in place on the wrist of a user.
  • the systems may include a magnetometer attached to the wrist band.
  • the systems may include a magnetometer attached to the ring band.
  • the systems may include a gyroscope attached to the wrist band.
  • the systems may include a gyroscope attached to the ring band.
  • the systems may include a wireless transmitter attached to the wrist band.
  • the systems may include a wireless transmitter attached to the ring band.
  • the systems may include two or more wire connecting the ring band to the wrist band.
  • the systems may include a processing device attached to the wrist band.
  • the processing device may be a micro-controller.
  • the processing device may be a microprocessor.
  • the systems may include a processing device attached to the ring band.
  • the subject matter described in this specification can be embodied in methods that include affixing two or more sensors to fingers and wrists, wherein signals from the sensors are processed to determine locations of movement in two-dimensional or three-dimension space and to determine locations of taps in two-dimensional or three-dimensional space.
  • the subject matter described in this specification can be embodied in methods that include affixing a first accelerometer to the thumb of a user and affixing a second accelerometer to a reference location on the same hand or wrist of the user.
  • the methods may include processing signals from the first accelerometer to detect taps of the user thumb and processing signals from the first and second accelerometers to determine which of a plurality of target locations on the hand was tapped by the thumb upon detection of a tap.
  • Signals from the second accelerometer may be processed to determine the orientation of the user hand in relation to the radius of the earth.
  • a value may be selecting among a plurality of values assigned to tap targets based in part on the orientation of the hand to the radius of the earth at the time of the taps.
  • Processing of signals from the first and second accelerometers may include tracking the change in position of the first accelerometer relative to the second accelerometer.
  • Processing of signals from the first and second accelerometers may include determining an angle between vectors emanating from the first accelerometer and the second accelerometer.
  • Processing of signals from the first and second accelerometers may include determining a linear displacement of either the first or second accelerometers between two time points.
  • the subject matter described in this specification can be embodied in systems that include a first accelerometer and a first fastening article attached to the first accelerometer, the first fastening article capable of holding the first accelerometer in place on a portion of a thumb of a user.
  • the systems that include a second accelerometer and a second fastening article attached to the second accelerometer, the second fastening article capable of holding the second accelerometer in place on a portion of a wrist of a user.
  • the systems that include a processing device receiving data based on acceleration measurements from the first accelerometer and the second accelerometer via one or more wireless communication links.
  • the subject matter described in this specification can be embodied in systems that include a first accelerometer and a first fastening article attached to the first accelerometer, the first fastening article capable of holding the first accelerometer in place on a portion of a thumb of a user.
  • the systems may include a second accelerometer and a second fastening article attached to the second accelerometer, the second fastening article capable of holding the second accelerometer in place on a portion of a wrist of a user.
  • the systems may include a processing device receiving data based on acceleration measurements from the first accelerometer and the second accelerometer via one or more wireless communication links.
  • the systems may include a projector connected to the processing device, the projector configured to project images produced by the processing device and wherein the images including graphic representations of symbols selected based the data.
  • the subject matter described in this specification can be embodied in methods that include detecting an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the methods may include estimating a first angle between a first axis that is approximately parallel to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the methods may include assigning one of the plurality of symbols to the instance of the gesture based in part on the estimate of the first angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between a second axis and the acceleration experienced by the accelerometer attached to the forearm may be estimated, where the second axis is approximately perpendicular to the first axis.
  • the instance of the gesture may be associated with one of the plurality of symbols based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in systems that include a data processing apparatus and a data storage device storing instructions executable by the data processing apparatus that upon execution by the data processing apparatus cause the data processing apparatus to perform operations including detecting an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the operations may include estimating a first angle between a first axis that is approximately parallel to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the operations may include assigning one of the plurality of symbols to the instance of the gesture based in part on the estimate of the first angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between a second axis and the acceleration experienced by the accelerometer attached to the forearm may be estimated, where the second axis is approximately perpendicular to the first axis.
  • the instance of the gesture may be associated with one of the plurality of symbols based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in systems that include a module configured to detect an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the systems may include a module configured to estimate a first angle between a first axis that is approximately parallel to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the systems may include a module configured to assign one of the plurality of symbols to the instance of the gesture based in part on the estimate of the first angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between a second axis and the acceleration experienced by the accelerometer attached to the forearm may be estimated, where the second axis is approximately perpendicular to the first axis.
  • the instance of the gesture may be associated with one of the plurality of symbols based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in one or more computer-readable storage media encoded with a computer program.
  • the computer program may include instructions that when executed by data a processing apparatus cause the data processing apparatus to perform operations including detecting an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the operations may include estimating a first angle between a first axis that is approximately parallel to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the operations may include assigning one of the plurality of symbols to the instance of the gesture based in part on the estimate of the first angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between a second axis and the acceleration experienced by the accelerometer attached to the forearm may be estimated, where the second axis is approximately perpendicular to the first axis.
  • the instance of the gesture may be associated with one of the plurality of symbols based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in methods that include detecting an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the methods may include estimating an angle between an axis that is approximately perpendicular to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the methods may include assigning one of the plurality of symbols to the instance of the gesture based in part on the estimate of the angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between an acceleration experienced by an accelerometer attached to the forearm and a second axis that is also approximately perpendicular to a forearm of the hand may be estimated.
  • the one of the plurality of symbols may be assigned to the instance of the gesture based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in systems that include a data processing apparatus and a data storage device storing instructions executable by the data processing apparatus that upon execution by the data processing apparatus cause the data processing apparatus to perform operations including detecting an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the operations may include estimating an angle between an axis that is approximately perpendicular to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the operations may include assigning one of the plurality of symbols to the instance of the gesture based in part on the estimate of the angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between an acceleration experienced by an accelerometer attached to the forearm and a second axis that is also approximately perpendicular to a forearm of the hand may be estimated.
  • the one of the plurality of symbols may be assigned to the instance of the gesture based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in systems that include a module configured to detect an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the systems may include a module configured to estimate an angle between an axis that is approximately perpendicular to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the systems may include a module configured to assign one of the plurality of symbols to the instance of the gesture based in part on the estimate of the angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between an acceleration experienced by an accelerometer attached to the forearm and a second axis that is also approximately perpendicular to a forearm of the hand may be estimated.
  • the one of the plurality of symbols may be assigned to the instance of the gesture based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in one or more computer-readable storage media encoded with a computer program.
  • the computer program may include instructions that when executed by data a processing apparatus cause the data processing apparatus to perform operations including detecting an instance of a gesture made with a hand, wherein the gesture is associated with a plurality of symbols.
  • the operations may include estimating an angle between an axis that is approximately perpendicular to a forearm of the hand and an acceleration experienced by an accelerometer attached to the forearm.
  • the operations may include assigning one of the plurality of symbols to the instance of the gesture based in part on the estimate of the angle.
  • the acceleration may be caused by gravitational force of the Earth.
  • a second angle between an acceleration experienced by an accelerometer attached to the forearm and a second axis that is also approximately perpendicular to a forearm of the hand may be estimated.
  • the one of the plurality of symbols may be assigned to the instance of the gesture based in part on the estimate of the second angle.
  • the subject matter described in this specification can be embodied in methods that include receiving a first set of sensor measurements from a first sensor module that is attached to a thumb of a user, wherein the first sensor module comprises an accelerometer and the first set of sensor measurements includes acceleration measurements.
  • the methods may include receiving a second set of sensor measurements from a second sensor module that is attached to a wrist of the user, wherein the second sensor module includes an accelerometer and the second set of sensor measurements includes acceleration measurements.
  • the methods may include determining, based at least in part on the first set of measurements and the second set of measurements, an orientation of the thumb relative to the wrist.
  • the methods may include detecting, based at least in part on acceleration measurements in the second set of sensor measurements, when a hand of the user is moved.
  • the methods may include, when the orientation satisfies a condition and movement of the hand is detected, moving a cursor in a virtual space in a direction and by an amount determined based at least in part on acceleration measurements in the second set of sensor measurements.
  • the methods may include causing the motion of the cursor in the virtual space to be reflected in a visual representation of the virtual space that is presented on a display.
  • the condition may require the angle between an axis of a sensor in the first sensor module that is approximately parallel to a portion of the thumb and an axis of a sensor in the second sensor module that is approximately parallel a forearm of the user to be below a threshold.
  • Tap events may be detected, based on the first set of sensor measurements and the second set of sensor measurements, when the thumb taps a tap target located on another finger of the hand. Detection of a thumb tap on the tap target may cause an interaction with an object in the virtual space that the cursor is in contact with.
  • the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the selected object in virtual space may be rotated, based at least in part on the angular rate measurements in the second set of sensor measurements.
  • the second sensor module may include a magnetometer and the second set of sensor measurements may include magnetic flux measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the object in virtual space may be rotated, based at least in part on acceleration measurements and magnetic flux measurements in the second set of sensor measurements.
  • the first sensor module may include a gyroscope and the first set of sensor measurements may include angular rate measurements and the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • Determining the orientation may include determining, based on the first set of sensor measurements, a first estimate of orientation of the first sensor module relative to a reference orientation; determining, based on the second set of sensor measurements, a second estimate of orientation of the second sensor module relative to a reference orientation; and comparing the first estimate of orientation to the second estimate if orientation.
  • the subject matter described in this specification can be embodied in systems that include a data processing apparatus and a data storage device storing instructions executable by the data processing apparatus that upon execution by the data processing apparatus cause the data processing apparatus to perform operations including receiving a first set of sensor measurements from a first sensor module that is attached to a thumb of a user, wherein the first sensor module comprises an accelerometer and the first set of sensor measurements includes acceleration measurements.
  • the operations may include receiving a second set of sensor measurements from a second sensor module that is attached to a wrist of the user, wherein the second sensor module includes an accelerometer and the second set of sensor measurements includes acceleration measurements.
  • the operations may include determining, based at least in part on the first set of measurements and the second set of measurements, an orientation of the thumb relative to the wrist.
  • the operations may include detecting, based at least in part on acceleration measurements in the second set of sensor measurements, when a hand of the user is moved.
  • the operations may include, when the orientation satisfies a condition and movement of the hand is detected, moving a cursor in a virtual space in a direction and by an amount determined based at least in part on acceleration measurements in the second set of sensor measurements.
  • the operations may include causing the motion of the cursor in the virtual space to be reflected in a visual representation of the virtual space that is presented on a display.
  • the condition may require the angle between an axis of a sensor in the first sensor module that is approximately parallel to a portion of the thumb and an axis of a sensor in the second sensor module that is approximately parallel a forearm of the user to be below a threshold.
  • Tap events may be detected, based on the first set of sensor measurements and the second set of sensor measurements, when the thumb taps a tap target located on another finger of the hand. Detection of a thumb tap on the tap target may cause an interaction with an object in the virtual space that the cursor is in contact with.
  • the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the selected object in virtual space may be rotated, based at least in part on the angular rate measurements in the second set of sensor measurements.
  • the second sensor module may include a magnetometer and the second set of sensor measurements may include magnetic flux measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the object in virtual space may be rotated, based at least in part on acceleration measurements and magnetic flux measurements in the second set of sensor measurements.
  • the first sensor module may include a gyroscope and the first set of sensor measurements may include angular rate measurements and the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • Determining the orientation may include determining, based on the first set of sensor measurements, a first estimate of orientation of the first sensor module relative to a reference orientation; determining, based on the second set of sensor measurements, a second estimate of orientation of the second sensor module relative to a reference orientation; and comparing the first estimate of orientation to the second estimate if orientation.
  • the subject matter described in this specification can be embodied in systems that include a wireless networking interface configured to receive a first set of sensor measurements from a first sensor module that is attached to a thumb of a user, wherein the first sensor module includes an accelerometer and the first set of sensor measurements includes acceleration measurements.
  • the systems may include a wireless networking interface configured to receive a second set of sensor measurements from a second sensor module that is attached to a wrist of the user, wherein the second sensor module includes an accelerometer and the second set of sensor measurements includes acceleration measurements.
  • the systems may include a module configured to determine, based at least in part on the first set of measurements and the second set of measurements, an orientation of the thumb relative to the wrist.
  • the systems may include a module configured to detect, based at least in part on acceleration measurements in the second set of sensor measurements, when a hand of the user is moved.
  • the systems may include, a module configured to, when the orientation satisfies a condition and movement of the hand is detected, move a cursor in a virtual space in a direction and by an amount determined based at least in part on acceleration measurements in the second set of sensor measurements.
  • the systems may include a module configured to cause the motion of the cursor in the virtual space to be reflected in a visual representation of the virtual space that is presented on a display.
  • the condition may require the angle between an axis of a sensor in the first sensor module that is approximately parallel to a portion of the thumb and an axis of a sensor in the second sensor module that is approximately parallel a forearm of the user to be below a threshold.
  • Tap events may be detected, based on the first set of sensor measurements and the second set of sensor measurements, when the thumb taps a tap target located on another finger of the hand. Detection of a thumb tap on the tap target may cause an interaction with an object in the virtual space that the cursor is in contact with.
  • the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the selected object in virtual space may be rotated, based at least in part on the angular rate measurements in the second set of sensor measurements.
  • the second sensor module may include a magnetometer and the second set of sensor measurements may include magnetic flux measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the object in virtual space may be rotated, based at least in part on acceleration measurements and magnetic flux measurements in the second set of sensor measurements.
  • the first sensor module may include a gyroscope and the first set of sensor measurements may include angular rate measurements and the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • Determining the orientation may include determining, based on the first set of sensor measurements, a first estimate of orientation of the first sensor module relative to a reference orientation; determining, based on the second set of sensor measurements, a second estimate of orientation of the second sensor module relative to a reference orientation; and comparing the first estimate of orientation to the second estimate if orientation.
  • the subject matter described in this specification can be embodied in one or more computer-readable storage media encoded with a computer program.
  • the computer program may include instructions that when executed by data a processing apparatus cause the data processing apparatus to perform operations including receiving a first set of sensor measurements from a first sensor module that is attached to a thumb of a user, wherein the first sensor module comprises an accelerometer and the first set of sensor measurements includes acceleration measurements.
  • the operations may include receiving a second set of sensor measurements from a second sensor module that is attached to a wrist of the user, wherein the second sensor module includes an accelerometer and the second set of sensor measurements includes acceleration measurements.
  • the operations may include determining, based at least in part on the first set of measurements and the second set of measurements, an orientation of the thumb relative to the wrist.
  • the operations may include detecting, based at least in part on acceleration measurements in the second set of sensor measurements, when a hand of the user is moved.
  • the operations may include, when the orientation satisfies a condition and movement of the hand is detected, moving a cursor in a virtual space in a direction and by an amount determined based at least in part on acceleration measurements in the second set of sensor measurements.
  • the operations may include causing the motion of the cursor in the virtual space to be reflected in a visual representation of the virtual space that is presented on a display.
  • the condition may require the angle between an axis of a sensor in the first sensor module that is approximately parallel to a portion of the thumb and an axis of a sensor in the second sensor module that is approximately parallel a forearm of the user to be below a threshold.
  • Tap events may be detected, based on the first set of sensor measurements and the second set of sensor measurements, when the thumb taps a tap target located on another finger of the hand. Detection of a thumb tap on the tap target may cause an interaction with an object in the virtual space that the cursor is in contact with.
  • the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the selected object in virtual space may be rotated, based at least in part on the angular rate measurements in the second set of sensor measurements.
  • the second sensor module may include a magnetometer and the second set of sensor measurements may include magnetic flux measurements.
  • a gesture may be detected based on the first set of sensor measurements and the second set of sensor measurements, the gesture indicating a selection of an object in the virtual space that the cursor is in contact with.
  • the object in virtual space may be rotated, based at least in part on acceleration measurements and magnetic flux measurements in the second set of sensor measurements.
  • the first sensor module may include a gyroscope and the first set of sensor measurements may include angular rate measurements and the second sensor module may include a gyroscope and the second set of sensor measurements may include angular rate measurements.
  • Determining the orientation may include determining, based on the first set of sensor measurements, a first estimate of orientation of the first sensor module relative to a reference orientation; determining, based on the second set of sensor measurements, a second estimate of orientation of the second sensor module relative to a reference orientation; and comparing the first estimate of orientation to the second estimate if orientation.
  • the subject matter described in this specification can be embodied in systems that include a first gyroscope; a first fastening article attached to the first gyroscope, the first fastening article capable of holding the first gyroscope in place on a portion of a thumb of a user; a second gyroscope; and a second fastening article attached to the second gyroscope, the second fastening article capable of holding the second gyroscope in place on a portion of a wrist of a user.
  • the first fastening article may be a ring band.
  • the second fastening article may be a wrist band.
  • a system may include a processing device configured to receive angular rate measurements from the first gyroscope and determine, based at least in part on the received angular rate measurements, when a thumb of a user wearing the first fastening article and the second fastening article has been tapped against a surface.
  • the processing device may be further configured to receive angular rate measurements from the second gyroscope and, when a tap of the thumb is detected, determine, based at least in part on angular rate measurements from the first gyroscope and the second gyroscope, which tap target from among a set of tap targets located on fingers of a hand of the user have been tapped by the thumb.
  • the processing device may be further configured to map the tap of the tap target to a symbol associated with the tap target.
  • a system may include two or more wires that connect the first fastening device to the second fastening device.
  • a system may include a wireless transmitter attached to the second fastening article, and the wireless transmitter may be configured to transmit data based on angular rate measurements from the second gyroscope to a computing device.
  • a system may include a wireless transmitter attached to the second fastening article, and the wireless transmitter may be configured to transmit data based on angular rate measurements from the first gyroscope to a computing device.
  • a system may include a display device configured to present images produced by the processing device and the images may include graphic representations of symbols selected based the data.
  • the display device may be a liquid crystal display.
  • the display device may be a projector.
  • the processing device may be attached to the second fastening article.
  • the processing device may be part of a remote computing device.
  • Implementations may include zero or more of the following advantages. Some implementations may reliably detect and classify hand gestures to allow a user to control a computing device. Some implementations may include sensor components that are comfortably wearable on a thumb and/or wrist of a user. Some implementations may enable a user to input alpha-numeric text or other symbols to a computing device. Some implementations may enable a user to manipulate a cursor in a two dimensional or a three dimensional virtual workspace. Some implementations may be robust to environmental noise such as vibrations or accelerations experienced in a moving vehicle. Some implementations may enable a user to enter text on mobile device without using limited display space to present keys. Some implementations may enable a user to enter symbols or commands to a computing device by tapping tap targets without looking at those targets.
  • a computer interface includes a sensor module that is attached to fastening article (e.g., a ring band, an adhesive substrate, or a glove with a thumb sleeve) that is capable of holding the sensor module in place on a portion of a thumb of a user.
  • the sensor module may include an accelerometer, a magnetometer, and/or a gyroscope.
  • a computing interface also includes a reference sensor module that may be attached to fastening article (e.g., a wrist band or sleeve) that is capable of holding the reference sensor module in place on a portion of the wrist of a user (or some other reference location on the hand or forearm of the user).
  • a reference sensor module may include an accelerometer, a magnetometer, and/or a gyroscope.
  • a sensor module of a computing interface may also include a micro-controller or microprocessor, a wireless transmitter, and/or a battery.
  • two sensor modules of an interface may be connected by two or more wires (e.g., a serial port cable), and only one of the sensor modules includes a battery that supplies power to both sensor modules.
  • each sensor module has its own battery and is configured to transmit measurements (e.g., acceleration measurements, magnetic flux measurements, and/or angular rate measurements) from one or more sensors in the sensor module to a remote computing device via a wireless communications link (e.g., a Bluetooth link).
  • a wireless communications link e.g., a Bluetooth link
  • a sensor module transmits (e.g., via wireless communications link) its sensor measurements to another sensor module in the interface rather than directly to a remote computing device.
  • Data based on sensor measurements from multiple sensor modules may be transmitted from one of the sensor modules (e.g., a reference sensor module attached to wrist) to a computing device that the user seeks to control or provide input to.
  • measurements from all the sensors in an interface may be forwarded to the target computing device via a transmitter (e.g., a Bluetooth transmitter) included in a reference sensor module.
  • one of the sensor modules includes a processing device (e.g., a micro-controller or a microprocessor) that analyzes sensor measurements from sensors of the interface and transmits other data based on those measurements to the target computing device. For example, symbols assigned to thumb taps detected by the interface may be transmitted to a target computing device.
  • a processing device e.g., a micro-controller or a microprocessor
  • processing to interpret the measurements from one or more sensors of an interface is performed by an application or device driver that runs on the target computing device.
  • Example processes are described for interpreting measurements from sensors in various interface configurations.
  • the example interfaces with corresponding processes may enable a computing device to determine when a thumb of a user wearing an interface is tapped against a surface. For example, a user's thumb may be tapped against one of a set of configured tap targets on the other fingers of the user. These tap events may be detected and classified to identify which tap target was tapped and to map that tap gesture to a corresponding symbol that the user intends to input to the target computing device.
  • the orientation of a user's wrist may be determined and used to select among multiple symbols assigned to an individual tap target.
  • an interface may support a cursor manipulation mode that enables a user to interact with objects in a virtual space (e.g., a two dimensional or three dimensional virtual space). For example, when in cursor manipulation mode, acceleration measurements from an accelerometer in the interface analyzed to control the movement of a cursor in the virtual space. In some implementations, angular rate measurements from a gyroscope in an interface may be interpreted to enable a user to rotate objects in virtual space that have been selected with a cursor while in a cursor manipulation mode.
  • a computing interface may include a ring 100 that may be worn on a user's thumb.
  • the ring includes a band 110 and one or more accelerometers, magnetometers, or gyroscopes (collectively, a set of sensors) that are located in an electronic component housing 120.
  • the ring may include a single tri-axial or multiple dual-axis and /or single axis sensors to span the three dimensional space.
  • the axes of different types of sensors in the component housing 120 may be aligned.
  • the axes of different types of sensors in the component housing 120 may be aligned electronically via a calibration process.
  • the ring may also include a radio frequency transmitting device, such as a Bluetooth transmitter, and a battery.
  • the electronic component housing may include a switch 130 for powering the electronics components up and down.
  • the accelerometer may measure the spatial orientation of the thumb and its motion. For example, when the thumb is tapped against a target, such as phalanges on the other fingers of the hand, an abrupt deceleration results that is detected by the accelerometer.
  • a transmitter may be used to send sensor measurements from the ring to an external processing device.
  • a transmitter may also be used to send information about events derived from sensor measurements to an external processing device.
  • the ring band 110 serves to hold the interface ring 100 in place on a user's thumb.
  • the ring band may be made of plastic, or another flexible material.
  • the ring band may be of approximately circular shape with a single gap 140 that allows the ring band to flex to surround most of the circumference of a user's thumb.
  • the ring band may alternatively be formed into a complete loop that completely encircles the user's thumb when worn.
  • the ring band may be made of a material, such as nylon, that is capable of stretching in a longitudinal direction.
  • the ring band may be rigid and fitted to a particular thumb size.
  • a ring 100 may also include a wireless receiver (e.g., a Bluetooth receiver) for receiving information from a target computing device or an intermediary device.
  • a wireless receiver e.g., a Bluetooth receiver
  • the ring may receive configuration commands from target computing device that set operating parameters of the ring, such as a usage mode (e.g., to enter a cursor control mode), a power-saving mode, a sampling rate for one or more sensors, etc.
  • a portion of the ring that includes a Bluetooth transmitter may be detachable from the band. This portion may also include a speaker and microphone that allow the detachable component to be used as a Bluetooth enabled earbud for a cellphone.
  • an example computing interface may include a bracelet 200 that may be worn on a user's wrist.
  • the bracelet includes a wristband 210 and one or more accelerometers, magnetometers, or gyroscopes (collectively, a set of sensors) that are located in an electronic component housing 220.
  • the bracelet may include a single tri-axial or multiple dual-axis and /or single axis sensors to span the three dimensional space.
  • the axes of different types of sensors in the component housing 220 may be aligned.
  • the axes of different types of sensors in the component housing 220 may be aligned electronically via a calibration process.
  • the bracelet may also include a radio frequency transmitting device, such as a Bluetooth transmitter, and a battery.
  • the accelerometer may measure spatial orientation of the wrist and its motion.
  • a transmitter may be used to send sensor measurements from the bracelet to an external processing device.
  • a transmitter may also be used to send information about events derived from the sensor measurements to an external processing device.
  • the electronic component housing may include a switch 230 for powering the electronics components up and down.
  • the wristband 210 serves to hold component(s) of the interface in place on a user's wrist.
  • the wristband may be made a flexible material, such as rubber, nylon, or plastic.
  • the wristband may include an adjustable fastening device, such a Velcro strip, snaps, cable tie, or a buckle.
  • the wristband may be formed into a complete loop that completely encircles the user's wrist when worn.
  • the wristband may be a continuous loop made of a material, such as rubber or nylon that is capable of stretching in a longitudinal direction to allow the band to slide over the hand of the user and still fit the wrist tight enough to hold an accelerometer in place on the wrist.
  • a bracelet 200 may also include a wireless receiver (e.g., a Bluetooth receiver) for receiving information from a target computing device or an intermediary device.
  • a wireless receiver e.g., a Bluetooth receiver
  • the ring may receive configuration commands from target computing device that set operating parameters of the ring, such as a usage mode (e.g., to enter a cursor control mode), a power-saving mode, a sampling rate for one or more sensors, etc.
  • an interface 300 may include multiple components worn on different parts of a user's hands or arms.
  • the interface includes a ring 100 and a bracelet 200 worn on the same hand 310.
  • a position tracker module may be initialized.
  • the position of the sensors in both the ring and the bracelet are tracked by integrating the dynamic motion detected by both components.
  • the change in position experienced by the bracelet serves as reference for determining how the position of the thumb has changed in relationship to the rest of the hand. In this manner the effects of unrelated user movement, such as turning, sitting, standing, walking or riding in a vehicle, on the position of the thumb may be controlled for to isolate changes in the position of the thumb relative to the rest of the hand.
  • accelerometer readings are sampled at a frequency of about 1 kHz and the resulting digital signals are processed to detect when thumb taps occur and classify taps according to the tap targets that are hit. All or part of the processing may be performed by a microprocessor located on the ring or the bracelet. All or part of the processing of accelerometer readings may be performed on a data processing device that receives readings via a radio frequency transmission from the transmitters on the ring and/or on the bracelet.
  • the data processing device may be an internal or external processing device, such as a cellphone, that runs software configured to receive sensor readings or information (e.g., filtered signals and/or symbols) based on those readings from the interface.
  • the processing device may be a stand-alone device configured to receive information based on sensor readings from the interface via the radio frequency transmission.
  • the stand-alone processing device may in turn pass information such as detected target tap events to an external processing device, such as a computer, via another interface, such as a USB (Universal Serial Bus) port.
  • an external processing device such as a computer
  • another interface such as a USB (Universal Serial Bus) port.
  • USB Universal Serial Bus
  • the devices in the interface system may communicate with each other by means of radio frequency.
  • low-power wireless transmission from a ring with a short range e.g., 1 foot range
  • a processing device attached to a bracelet may in turn interpret those measurements and/or forward they to a target computing device via a higher power wireless communications link (e.g., a Bluetooth link).
  • a higher power wireless communications link e.g., a Bluetooth link
  • they may communicate with each other through wired connections.
  • the ring 100 and bracelet 200 may communicate sensor readings through wired connections to determine their individual spatial orientations.
  • the bracelet may hold an energy storage that supplies power to the ring through the wire connections.
  • Accelerometers in the wristband may also be used to detect the spatial orientation of the hand by measuring the static acceleration caused by the gravitational pull of the Earth, which is a vector along the Earth radius, extending from the Earth center through the wearer of the interface.
  • the orientation of the accelerometer to the user's wrist may be fixed by the wristband.
  • the axes of the three dimensions sensed by the accelerometers may be fixed with respect to the orientation of the user's wrist.
  • the angle of the Earth-radius vector with respect to the reference frame extended by the axes is calculated to determine the orientation of the wrist with respect to the Earth radius.
  • the angle of the Earth-radius vector with respect to the reference frame extended by the axes of the ring accelerometers is calculated to determine the angle of a phalanx of the thumb with respect to the Earth radius.
  • the Earth-radius angles of the thumb and wrist may be compared to estimate a component of the angle between the thumb and the wrist outside of the plane orthogonal to the Earth radius.
  • the angle between the thumb and the wrist at the time that a tap is detected may be used to distinguish tap targets on the hand.
  • Information about the current angle between the thumb and the wrist may be used in conjunction with information from a position tracking module to classify tap events by assigning them to a tap target.
  • magnetometers may be used in conjunction with accelerometers to to determine the relative orientations of the thumb ring and a reference device, such as a reference device located on the wrist.
  • the thumb ring may include a tri-axial accelerometer and a tri-axial magnetometer. The axes of these magnetometer and the accelerometer may be aligned.
  • the reference device may also include an accelerometer and a magnetometer whose axes are aligned.
  • a rotation that represents the relative orientation of the thumb device and the reference device during the tap event may be determined from those four vectors. That rotation may be determined in stages by first determining two component rotations, R m and R a , and then combining them. First a rotation that aligns the two magnetic field vectors is calculated by taking a cross product of r m and t m to determine the axis of a minimum-angle rotation that aligns the two vectors as well as the magnitude of the angle of rotation. A dot product is also calculated to disambiguate the quadrant of the angle. These calculations yield an axis/angle representation of the first component rotation.
  • the first rotation matrix is then applied to the thumb acceleration vector, t a , to determined the rotated thumb acceleration vector, t ⁇ a ′ .
  • t ⁇ a ′ R m t ⁇ a
  • a second component rotation that aligns t ⁇ a ′ with the reference acceleration, r a may be determined next.
  • the second component rotation may be constrained to use an axis of rotation aligned with the reference magnetic field, r m , so that alignment of the two magnetic field vectors is preserved by the second component rotation. That can be done, for example, using the projections of r a and t ⁇ a ′ onto the plane perpendicular to r m .
  • This second component rotation may also be computed from an axis/angle representation and may be represented as a matrix, R a .
  • the two component rotations may then be combined by multiplying the two matrices in the proper order to produce a matrix representation of the relative orientation of the two devices, R .
  • R R a R m
  • the relative orientation of the thumb and reference devices may be converted from the matrix representation to a lower dimensional representation to enable more efficient slicing to quantize the orientation estimate into a symbol estimate.
  • the matrix representation, R may be converted to an axis /angle representation using an eigenvalue decomposition. Since the axis of the rotation is a unit vector, the axis /angle may be expressed as a three-tuple by multiplying the axis by the angle of rotation. These three-tuples may then be assigned to symbol estimates by slicing in the three dimensional space.
  • a slicer for the orientation estimates may be generated using standard techniques applied to corpus of tap orientation measurements taken during a known tap sequence. For example the centroids of clusters of orientation estimates corresponding to a particular tap may be used. Slicing may be accomplished by determining the nearest tap centroid to a new orientation estimate. Slicer regions may be determined based on aggregated data for many users or for a particular user by using training sequences to collect data from that user. In some cases an abbreviated training sequence may be used to customize generic slicer regions to a particular user.
  • the order of the decomposition of the orientation rotation into components may be reversed.
  • the component required to align the acceleration vectors may be determined first and then a constrained component rotation to approximately align the magnetic field vectors may be subsequently determined and then combined.
  • the selection of the order in which the decomposition is performed may be informed by the signal-to-noise ratios (SNRs) experienced by the accelerometers and the magnetometers.
  • SNRs signal-to-noise ratios
  • a calibration sequence may be performed by a user before the first use of the interface.
  • the user may be prompted to execute each step of the calibration process using a display connected to a processing device that the interface is inputting data to.
  • the prompt instructs the user to touch one or more of the targets on the hand and data is recorded as taps are detected.
  • the data may be used only for the current session, or stored in memory as a user profile. In this manner, the interface may be trained to respond to the geometry of the hand and tendencies of a particular user.
  • ring 100 may include a thermometer that is used to dynamically adjust an output amplifier gain for one or more of the sensors (e.g., an accelerometer) that have a response characteristics that vary with temperature.
  • sensors e.g., an accelerometer
  • one or more accelerometers located in a second ring worn on one of the proximal phalanges of the other fingers on the hand may be used as a reference for determining position and angles of the thumb in relation to the rest of the hand.
  • tap targets are located on phalanges of fingers of a user's hand that the user may ergonomically tap with the thumb while wearing an example interface including ring 100 and bracelet 200.
  • An example layout 400 of tap targets on the hand is shown in Fig. 4 .
  • the tap targets are centered on the inside surface of the distal 410, middle 420, and proximal 430 phalanges.
  • a mapping is established that assigns different symbols to taps of each of the targets.
  • each target is labeled by an associated symbol.
  • the tap targets on an index finger may be centered on the side of the index finger closest to the thumb. Locating tap targets on the fingers of the user may allow a user to conduct taps without looking at the tap targets.
  • An interface may include a matching set of components for the other hand so that both hands may be used for data entry.
  • different symbols may be assigned to the corresponding tap targets on each hand to double the size of the symbol set.
  • taps on both hands may be combined to expand the symbol set even more. For example, tapping and holding the distal phalanx of the left index finger while tapping the phalanges on the other hand may be mapped to one set of symbols; tapping and holding the medial phalanx of the left index finger while tapping the phalanges on the other hand may be mapped to another set of symbols. In this way, at least 144 (12 phalanges on the left hand times 12 phalanges on the right hand) symbols may be produced from combining the taps on the two hands.
  • the angle of the wrist to the Earth radius may be used to distinguish multiple symbols assigned to a single tap target.
  • one of the three axes, the z-axis 560, of the accelerometers in the bracelet 200 is approximately parallel to the forearm of the user 510 and the other two axes are labeled x and y.
  • the angle 565 of the z-axis to the Earth radius 550 may be determined and compared to thresholds to distinguish multiple sets of symbols assigned to the targets on a hand.
  • Fig. 5 shows a user 510 wearing an interface including a ring 100 and a bracelet 200 with the forearm oriented at approximately ninety degrees to the Earth radius 550.
  • the user 510 is able to input one set of symbols by tapping the targets on the hand with the thumb.
  • the user 510 may access other symbols in scenario 600 by bending the elbow to, for example, raise the forearm to an angle 665 of approximately fifty degrees to the Earth Radius 550, as depicted in Fig. 6 .
  • the user 510 may input a second set of symbols by tapping the same targets on the hand. In this manner the multiple symbol sets may be assigned to different ranges of the angle between the user's forearm and the Earth radius.
  • more sets of symbols may be distinguished by detecting rotations of the wrist.
  • one position for data entry may be with the user's wrist turned so the palm of the hand faces up, as depicted by the direction of the dashed arrow 760 in Fig. 7 .
  • An axis (e.g., an x-axis) of an accelerometer in bracelet 200 may be approximately parallel to line 760 which is perpendicular to the surface of the palm of user 710.
  • the angle of this axis and/or the angle of a second axis that is also approximately orthogonal to a line parallel to the length of the user's forearm with an acceleration experienced by the accelerometer during a tap may be determined and used estimate the orientation of the user's wrist with respect to the Earth Radius 750. In this manner it can be determined that the palm of user 710 is facing up and a certain set of symbols may be assigned to tap targets on the hand.
  • Usage scenario 800 shows another data entry position, in which the wrist is rotated so that the palm faces to the side, as depicted in Fig. 8 .
  • the dark circles 860 illustrate an axis (e.g., an x-axis) of an accelerometer in the bracelet 200 that points out of the page.
  • the axis 860 that is approximately perpendicular to the surface of the palm is also perpendicular to the Earth Radius 750, while another axis that is also approximately perpendicular to the length of the user's forearm is approximately parallel with the Earth Radius 750.
  • these wrist rotation positions may be distinguished by comparing the angle between the x or y axes of the accelerometers in the bracelet and the Earth radius to thresholds. In this manner the number of wrist twist positions distinguished can further multiply the number of symbols that may be signaled with the interface.
  • wrist twists may be detected by tracking fast changes in position of accelerometers of an interface.
  • the thumb-up and thumb-down gestures may be detected for signaling approval (OK) and disapproval (CANCEL), respectively, selections in computer user interface and interaction (Ul and UX).
  • Wrist orientations may also be used to enter different input modes for an interface. For example, turning the wrist so that the palm faces down could be used to enter a cursor control mode. In cursor control mode, the hand may be moved in the three dimensional space in front of the user to control a cursor in one or more dimensions. Thumb orientation in relation to the reference frame on the wrist may be used to determine whether the cursor is engaged or not, so that the cursor can continue to be moved in a direction beyond the reach of the user, much like a user may pick up a mouse or disengage the finger from a trackball.
  • the cursor may be disengaged when the thumb is oriented approximately perpendicular to the length of the forearm of the user (e.g., held in a thumb-up gesture) and the cursor may be engaged when the thumb is closer to parallel with the length of the forearm.
  • an angle between an axis of a sensor attached to the thumb that is approximately parallel to a portion of the thumb and an axis of a senor attached to the wrist that is approximately parallel to the forearm may be estimated to determine whether the cursor is engaged.
  • Tap targets may be assigned different symbols in such a mode. For example, a tap target may be tapped to select or deselect item highlighted by a cursor.
  • certain tap targets may be assigned meta symbols (e.g., 'shift' or 'ctrl') that change the interpretation of target taps by one thumb while the other thumb is held in place on the meta symbol target.
  • meta symbols e.g., 'shift' or 'ctrl'
  • user input received through an interface may be processed and/or used by a computing device in a variety of ways.
  • One way is to present graphical representations of symbols indicated by the user gestures (e.g., thumb taps) made while wearing an interface.
  • FIG. 9 illustrates an example interface system in which a user wearing an interface including a ring 100 and a bracelet 200 performs a thumb tap to cause a combined processing and display device 910 (e.g., a Bluetooth enabled internet television) to display an alpha-numeric character associated with the thumb tap gesture.
  • a combined processing and display device 910 e.g., a Bluetooth enabled internet television
  • an interface e.g., including ring 100 and bracelet 200
  • a smartphone with an touchscreen display e.g., a tablet device with a touchscreen display
  • a computing device controlling a projector e.g., a computing device controlling other actuators (e.g., an environmental control system in an automobile), among many other computing devices.
  • an interface may be used to distinguish a large number of symbols.
  • An example table that maps tap targets and hand orientations to symbols is depicted in FIGS. 10A-10C .
  • the mapping of tap targets to symbols may be memorized by the user. As needed, the mapping of symbols to targets may be depicted on the hand by wearing a thin glove with the symbols drawn on positions associated with their targets. The mapping may also be displayed to the user by illustrating the symbols on their tap targets on the image of a hand on a display controlled by the external processing device that data is being entered into.
  • the image of the hand(s) with marked targets may be semi-transparent, overlaying the user interface of the underlying application. Such a display could be enabled, disabled or minimized when not needed by entering a special "help" symbol.
  • the mapping of targets to symbols may be designed by analyzing the relative frequency of symbols used. For example statistical analysis of a collection of texts may be conducted to determine which letters and letter sequences are most commonly used. The most commonly occurring symbols may then be mapped to targets located close together and in the positions that are most easily accessible to the thumbs. Common sequences of symbols may have all their symbols assigned to targets that may be tapped in quick succession. In this manner, an interface may be optimized for particular languages or applications.
  • the mapping of tap targets to events may be custom-configured by the users.
  • FIG. 11 is a flowchart of an example process 1100 for interpreting signals from a user computing interface.
  • the process 1100 may be performed by executing driver software for the computing interface on a computing device (e.g., a smart-phone, a tablet device, laptop computer, automobile environmental control system, or a television) that a user seeks to control by making hand gestures while wearing the computing interface.
  • a computing device may include a microprocessor and a data storage device (e.g., flash memory) storing instructions for causing the computing device to perform process 1100.
  • the process 1100 may be performed by a data processing device (e.g., a micro-controller or microprocessor) attached to the wrist of a user and symbols derived from the signal processing may be transmitted to a computing device that the user seeks to control.
  • a data storage device e.g., flash memory
  • instructions that cause a data processing device to perform process 1100 may also be attached to the wrist.
  • the process 1100 may begin by receiving 1110 measurements from sensors of the interface.
  • the measurements may include acceleration measurements from an accelerometer that is attached to a thumb of a user.
  • the measurements also include acceleration measurements from a second accelerometer that is attached to a wrist of the user.
  • the measurements also include magnetic flux measurements from a magnetometer that is attached to the thumb of the user and magnetic flux measurements from a magnetometer that is attached to the wrist of the user.
  • the measurements also include angular rate measurements from a gyroscope that is attached to the thumb of the user and angular rate measurements from a gyroscope that is attached to the wrist of the user.
  • the measurements from the sensors may be received as a time series of samples (e.g., sampled at 250 Hz, 500 Hz, 1 KHz, or 2 KHz) from each sensor output.
  • a time series of samples e.g., sampled at 250 Hz, 500 Hz, 1 KHz, or 2 KHz
  • one or more sensors may be sampled using a co-located micro-controller and resulting samples may be transmitted through one or more communications links (e.g., Bluetooth wireless links and/or a serial port link) to processing device for further analysis.
  • the time series of samples for each sensor is time synchronized with the time series of samples for a reference sensor (e.g., the accelerometer attached to the thumb or to the accelerometer attached to the wrist may dictate timing for the other sensor signals).
  • a phase locked loop may be implemented to compensate for clock skew and maintain synchronization with a reference signal from a sensor that is sampled with different clock.
  • a processing device receiving the measurements from the sensors may operate as a master in a master-slave configuration to enforce a sample timing for measurements received from multiple sensor modules of the interface.
  • a training sequence that causes simultaneous excitations in two sensors may be used to establish an initial phase synchronization between the signals from the two sensors. For example, an arm on which accelerometers are worn on the thumb and the wrist may be swung at the shoulder joint to induce an approximately simultaneous change in quantities measured by sensors at both locations on the arm.
  • the measurements from two different sensors of the interface may be received asynchronously. For example, sensors worn on the right and left hands may be received asynchronously.
  • the measurements may have been filtered or otherwise processed prior to receiving 1110 the measurements for the sensors.
  • a sequence samples of measurements from an accelerometer and a co-located gyroscope may be filtered and/or converted to a sequence of measurements of an orientation (e.g., encoded as Euler angles or a quaternion representation) of the co-located sensors by a co-located micro-controller before the measurements are received 1110 by an external processing device.
  • an orientation e.g., encoded as Euler angles or a quaternion representation
  • the measurements from the sensors of the interface may be received 1110 through a wireless network interface (e.g., a Bluetooth interface) of a processing device that will interpret the measurements.
  • the measurements may be received by a processing device that is co-located with some of the sensors of the interface (e.g., attached to the wrist of the user).
  • the measurements from co-located sensors may be received 1110 through a bus or other short range data transfer channel while measurements from sensors located further from the processing device may be received 1110 through a wireless communication channel (e.g., a Bluetooth link) or though a two or more wires (e.g., a serial port cable) connecting sensor modules of the interface.
  • a wireless communication channel e.g., a Bluetooth link
  • a two or more wires e.g., a serial port cable
  • a tap of the thumb on a surface may be detected 1120 as an event based on the received sensor measurements.
  • a tap event may be detected by filtering a sequence of acceleration measurements and/or angular rate measurements from an accelerometer and/or a gyroscope attached to the thumb of the user. Large fast changes in these measurements may be associated with a tap event. For example, the difference between consecutive samples of these measurements may be compared to a threshold (e.g., 1.2 times the acceleration due to Earth's gravity for the linear acceleration) and when the threshold is exceeded a tap event may be detected 1120.
  • a tap detection module may also implement debouncing logic to ignore fast changes in these measurements for a short configurable period of time (e.g.
  • tap events may be detected 1120 by a tap detection module of device driver running on a computing device.
  • tap events may be detected 1120 by a tap detection module running on a processing device that is attached to the wrist of the user.
  • An orientation of the thumb during the tap event may be determined 1130.
  • signals e.g., sequences of measurements
  • sensors of the interface may be windowed and/or otherwise filtered in the neighborhood of the detected tap event to estimate characteristics of the tap event. For example, a window of sensor measurements (e.g., a 5, 10, or 20 millisecond long window) just after a large deceleration associated with the onset of a tap event may be averaged to estimate characteristics of the tap event during a brief period while the thumb is at rest (relative to the rest of the hand) and in contact with a tap target.
  • the deceleration associated with the impact is itself considered as a characteristic of the tap event.
  • acceleration measurements from an accelerometer attached to the thumb may be filtered to determine an estimate of a deceleration vector caused by the impact of the thumb with the tap target.
  • the orientation of the estimated deceleration vector relative to the axes of one or more sensors attached to the thumb may be a characteristic considered for classification of a tap event.
  • an orientation of one or more sensors attached to the thumb of the user is determined 1130 relative to the an orientation of one or more sensors attached to the wrist of the user. For example, an estimate of the acceleration experienced by an accelerometer attached to the thumb while the thumb was at rest on the tap target (e.g., acceleration due to the Earth's gravitational force, the acceleration of a vehicle the user is riding in, and/or other exogenous forces) may be compared to an estimate of the acceleration experienced by an accelerometer attached to the wrist of the user during the same period of time (e.g., a time window just after the deceleration marking the start of the tap event) to compare the relative orientations of these accelerations as experienced at each location on the hand or arm of the user. These estimates of acceleration may be determined based in part on acceleration measurements from the respective accelerometers attached to the thumb and the wrist or some other reference location.
  • the relative orientation of the thumb and the wrist is determined 1130 based in part on magnetic flux measurements from a magnetometer attached to the thumb and magnetic flux measurements from a magnetometer attached to the wrist. For example, an estimate of the magnetic flux experienced by the magnetometer attached to the thumb while the thumb was at rest on the tap target (e.g., due to the Earth's magnetic field, magnetic field from a nearby transformer or power line, and/or other sources of magnetic flux) may be compared to an estimate of the magnetic flux experienced by the magnetometer attached to the wrist of the user during the same period of time (e.g., a time window just after the deceleration marking the start of the tap event) to compare the relative orientations of these magnetic flux vectors as experienced at each location on the hand or arm of the user. Where the magnetic flux is approximately uniform in the region of space around the thumb and wrist locations, the orientations of the magnetic flux vectors, as experienced by the respective magnetometers, may provide information about the relative orientation of the two sensors.
  • an orientation of the thumb relative to the wrist may be determined 1130 by combining information about the acceleration and magnetic flux experienced at the two locations. For example, as described above in relation to Equations 1 through 16, a rotation that approximately aligns the acceleration vectors and the magnetic flux vectors estimated for the two locations may be determined that specifies an estimated orientation of the thumb relative to the wrist. The estimates of the accelerations and magnetic flux experienced at each location may be determined by filtering measurements from the respective accelerometers and magnetometers at the locations.
  • the measurements for each sensor may be similarly windowed and averaged (e.g., by applying a Hamming window lagged with respect to a large deceleration that triggered the tap event) in a period corresponding to the thumb being at rest relative to the rest of the hand on the tap target.
  • an orientation of the thumb relative to the wrist is determined 1130 based in part on angular rate measurements from a gyroscope attached to the thumb and angular rate measurements from a gyroscope attached to the wrist.
  • the angular rate measurements from the gyroscope attached to the thumb may be integrated over a period of time ending during the detected tap event to determine an estimate of an orientation of the thumb during the tap event with respect to a reference position (e.g., a rest position of the thumb).
  • the angular rate measurements from the gyroscope attached to the wrist may be integrated over the same period of time ending during the detected tap event to determine an estimate of an orientation of the wrist during the tap event with respect to a reference position corresponding to the reference position for the thumb.
  • the reference positions for the thumb and wrist may be synchronously reset periodically (e.g., every minute) or upon prompting from a user.
  • the estimate of orientation of the thumb may be compared to the estimate of the orientation of the wrist to determine 1130 an orientation of the thumb relative to the wrist at a time associated with the tap event. For example, a rotation may be determined that relates to the two respective estimates of orientation.
  • an orientation of the thumb relative to the wrist may be determined 1130 by combining information about the linear acceleration and angular rate experienced at the two locations. For example, acceleration and angular rate measurements for the thumb may be integrated over a period of time that ends during the tap event to determine an estimate of the position and/or orientation of the thumb during the tap event. Similarly, acceleration and angular rate measurements for the wrist may be integrated over the same period of time to determine an estimate of a position and/or orientation of the wrist during the tap event. The position and/or orientation estimates for the thumb and wrist may be compared to determine 1130 a orientation of the thumb relative to the wrist. For example, a rotation may be determined that relates to the two respective estimates of orientation and a displacement vector may be determined that relates the two estimates respective estimates of position.
  • an orientation of the thumb may be determined 1130 by a tap classification module of device driver running on a computing device.
  • an orientation of the thumb may be determined 1130 by a tap classification module running on a processing device that is attached to the wrist of the user.
  • a tap target that was touched during a tap event is identified 1140.
  • a set of characteristics of the tap event may be analyzed to identify 1140 which tap target from among a set of configured tap targets has been tapped by the thumb of the user.
  • the tap targets may be configured to be located on other fingers of the user (e.g., as described in relation to FIG. 4 ).
  • the characteristics of a tap event may be represented as a vector in a feature space and the tap targets may be configured by partitioning the feature space into regions associated with one or none of the tap targets.
  • the tap characteristics include an orientation of the thumb (e.g., represented as a quaternion, a Euler angles triple, or an angle weighted axis of rotation).
  • the feature space for orientations may be a three-dimensional or four-dimensional space.
  • the tap characteristics include a displacement vector describing the position of the thumb relative to the wrist.
  • the tap characteristics include an estimate of a deceleration vector associated with the impact of the thumb on the tap target.
  • different characteristics of the tap may be combined to form a larger vector in a higher dimensional feature space.
  • feature vector may include elements of a quaternion representation of a thumb orientation and a three element representation of a displacement vector describing the position of the thumb relative to the wrist. In this case, the feature space may have seven dimensions.
  • the feature space may have been previously partitioned based on training data associated with each configured tap target location. For example, the partition may be determined using a nearest neighbor rule applied to a set of cluster centroids for each tap target. In some implementations, the feature space is partitioned based on training data for a large group of users. In some implementations, the feature space is partitioned based on training data for a particular user. The partition of the feature space may implemented as slicer that maps orientation data to an identification of one of the configured tap targets or an error/ignore result.
  • a tap target may be identified 1140 by a tap classification module of device driver running on a computing device.
  • a tap target may be identified 1140 by a tap classification module running on a processing device that is attached to the wrist of the user.
  • An orientation of the wrist is determined 1150.
  • an orientation of the wrist relative to the Earth's gravitational field is used to distinguish between multiple symbols associated with a tap target.
  • An estimate of an orientation of an acceleration experienced by the accelerometer attached to the wrist with respect to the axes of that accelerometer during the tap event may be determined based on acceleration measurements from that accelerometer.
  • the acceleration experienced at the wrist during a tap event may be dominated by acceleration caused by the gravitational force of the Earth.
  • measurements from the accelerometer attached to the wrist may be windowed and averaged in a time period corresponding to the tap event to determine an estimate of the acceleration due to gravity as a vector represented in the basis of the axes of the accelerometer.
  • Estimates of angles between this gravity vector and the axes of the accelerometer may be determined as needed to classify the orientation of the wrist with respect to the gravity vector.
  • one axis of the accelerometer may be assumed to be approximately parallel to the forearm of the user when the user wears the interface, while the other two axes are perpendicular to the first axis.
  • an orientation of the wrist relative to a magnetic field is used to distinguish between multiple symbols associated with a tap target.
  • An estimate of an orientation of magnetic flux experienced by a magnetometer attached to the wrist with respect to the axes of that magnetometer during the tap event may be determined based on magnetic flux measurements from that magnetometer.
  • the magnetic flux experienced at the wrist during a tap event may be dominated by magnetic flux caused by the magnetic field of the Earth.
  • measurements from the magnetometer attached to the wrist may be windowed and averaged in a time period corresponding to the tap event to determine an estimate of the magnetic flux due to the magnetic field as a vector represented in the basis of the axes of the magnetometer
  • Estimates of angles between this magnetic flux vector and the axes of the magnetometer may be determined as needed to classify the orientation of the wrist with respect to the magnetic flux vector.
  • one axis of the magnetometer may be assumed to be approximately parallel to the forearm of the user when the user wears the interface, while the other two axes are perpendicular to the first axis.
  • a wrist orientation may be determined 1150 by a tap classification module of device driver running on a computing device.
  • a wrist orientation may be determined 1150 by a tap classification module running on a processing device that is attached to the wrist of the user.
  • a symbol is assigned 1160 to the tap event.
  • a configured mapping e.g., the mapping illustrated in FIGS. 10A-10C ) of tap targets to one or more symbols may be retrieved and used to assign 1160 a symbol to the detected tap event.
  • multiple symbols are associated with a tap target and a symbol is selected from among the multiple symbols associated with the tap target based on the orientation of an acceleration experienced by the accelerometer attached to the wrist of the user. This acceleration may be dominated by an acceleration due to gravity and may provide an estimate of the orientation of the wrist with respect to gravitational field of the Earth.
  • an estimate of the angle between this acceleration and an axis parallel to the forearm of the user may be used to select a symbol (e.g., as described above in relation to FIGS. 5 , 6 and 10A-10C ).
  • estimates of one or more angles between this acceleration and one or more axes that are perpendicular to length of the forearm may be used to select a symbol (e.g., as described above in relation to FIGS. 7 , 8 and 10A-10C ).
  • a user may be enabled to indicate a choice from among the plurality of the symbols associated with the tap target, by adjusting the angle of wrist and/or the angle of a forearm of the user with respect to the gravitational force of the Earth during the tap.
  • Examples of symbols that may be assigned to a tap include an alpha-numeric character, a Chinese character, a command for a computing device that will cause the computing device to execute an action (e.g., send a text message or e-mail, answer a call, initiate a call dialing sequence, change slides in a presentation, turn on a radio or an air conditioner, etc.), and meta-keys (e.g., 'shift') that change the interpretation of a concurrent or subsequent tap, among others.
  • an action e.g., send a text message or e-mail, answer a call, initiate a call dialing sequence, change slides in a presentation, turn on a radio or an air conditioner, etc.
  • meta-keys e.g., 'shift'
  • a symbol may be assigned 1160 by a tap classification module of device driver running on a computing device.
  • a symbol may be determined 1150 by a tap classification module running on a processing device that is attached to the wrist of the user.
  • the symbol may be transmitted, stored, and/or displayed 1170.
  • the symbol may be transmitted 1170 to another device.
  • a processing device attached to the wrist of the user that performs process 1100 may transmit (e.g., via a wireless communications link) the symbol assigned to a detected tap to a computing device.
  • the symbol may be stored 1170.
  • computing device may buffer a sequence of symbols for later access by an application or some other thread running on the computing device.
  • the symbols may be stored in non-volatile memory (e.g., written to a file on a hard-drive when a text file is edited using the interface.
  • the symbol may be displayed 1170 through a display device controlled by the device performing process 1100.
  • a symbol e.g., an alpha-numeric character assigned to the tap may be displayed by a projector, or an LCD display on a mobile device (e.g., a smart-phone or tablet), among other types of displays.
  • process 1100 may be repeated in a loop to process sequences of tap events while the interface is in an active mode.
  • a user is enabled to turn a palm side of the wrist down to face towards the Earth to enter a cursor manipulation mode, in which acceleration measurements from an accelerometer of the interface (e.g., an accelerometer attached to the wrist of a user) is used to move a cursor in a virtual space.
  • acceleration measurements from an accelerometer of the interface e.g., an accelerometer attached to the wrist of a user
  • some thumb tap events may be used to interact with object in the virtual space.
  • certain tap targets may be mapped to mouse clicks while the user has their palm facing down toward the Earth.
  • a user may execute special gestures (e.g., a particular thumb tap or another type of gesture).
  • an interface is used to track three-dimensional spatial location of a user's hand and sensor data may be used to determine two-dimensional or three-dimensional cursor location.
  • thumb orientation and/or position in relation to the reference frame on the wrist may be used to determine whether the cursor is engaged or not, so that the cursor can continue to be moved in a direction beyond the reach of the user.
  • the cursor may be disengaged when the thumb is oriented approximately perpendicular to the length of the forearm of the user (e.g., held in a thumb-up gesture) and the cursor may be engaged when the thumb is closer to parallel with the length of the forearm.
  • Taps and other spatial gestures can then be used to interact with objects in a virtual space (e.g., replacing mouse clicks or joystick commands).
  • angular rate measurements from a gyroscope in an interface may be interpreted to enable a user to rotate objects in virtual space that have been selected with a cursor while operating in a cursor manipulation mode.
  • a box in three-dimensional virtual space may be selected with the interface by using a gesture (e.g., a sustained thumb tap) to 'grab' the object.
  • the box may then be turned or rotated to a desired orientation based on angular rate measurements from a gyroscope attached to a hand (e.g., to a thumb or wrist) of the user as the user rotates their hand.
  • Another gesture e.g., removing the thumb from a sustained tap target location
  • a user may be enable to reorient objects in a virtual space based on measurements from an accelerometer and a magnetometer in an interface.
  • the orientation of the hand may be determined at different times as a user turns the hand to manipulate an object by analyzing accelerometer and magnetometer measurements to estimate the orientation of the hand in relation to the background gravitational and magnetic fields (e.g., the Earth's gravity and the Earth's magnetic field). Differences in the estimated orientations with respect to the background fields at two more times may be used to update the orientation of an object in the virtual space in a corresponding manner.
  • the background gravitational and magnetic fields e.g., the Earth's gravity and the Earth's magnetic field.
  • a computing interface described above is paired with a display to facilitate user data entry in various ergonomic positions.
  • the interface e.g., including one or more sensors worn on the thumb and one or more sensors worn on the wrist
  • a processing device such as a computer
  • a display device such as a projector.
  • a user's body is oriented in a comfortable position and the display is positioned in the user's natural line of sight. For example, a user may lie in a bed or in a reclining chair and view a display projected onto the ceiling. From this position, the user may enter data via the interface and receive visual feedback via the display.
  • Data entry mode may be started and stopped by distinctive motions of the hand that are detected with one or more accelerometers in the interface. For example movement of the hand quickly in a circle may be used to indicate the start or stop of a data entry session.
  • the orientation of a plane in which this circular motion occurs may be used to set a reference earth radius angle for the session. Earth radius angles estimated during the session may be rotated by an amount determined by the orientation of the plane of the circular motion.
  • the measured wrist orientations may be left unadjusted, while, if the plane of the circular motion is orthogonal to the detected gravitational acceleration (e.g., because the user is lying on their back while making the circular motion in front of them), the measured wrist orientations may be rotated by 90 degrees to recover orientations in with respect to the user's body. In this manner a user's training with one set of wrist orientations may be used while the body is in different positions.
  • the computing interface may be used to control a wide variety of computing devices in different contexts.
  • a computing interface including a ring may be used to control one or more processing devices integrated in an automobile.
  • Gestures e.g., thumb taps
  • a tap target may be mapped to a command for turning an air conditioner on.
  • Another tap target may be mapped to a command for turning a radio on.
  • Another tap target may be mapped to a command for seeking or selecting a radio station.
  • Another tap target may be mapped to a command for unlocking a door, and so on.
  • measurement data from sensors in an interface or other information (e.g., symbols) based on the sensor measurements may be transmitted to multiple target processing devices.
  • an interface may be used to broadcast symbols derived from sensor measurements reflecting user hand motions for display on multiple displays.
  • a interface described herein is paired with a visual gesture recognition system.
  • the position tracking capability of the interface may be used in conjunction with data from the visual sensors (e.g., camera(s)) to enhance detection of gestures. For example, when the line of sight between the visual sensor and the thumb or the entire hand is obscured, data from the interface may be used to interpolate gestures.
  • an interface includes sensor modules or housings that are detachable from a corresponding fastening article (e.g., a thumb ring or a wrist band).
  • fastening articles may be interchangeable.
  • a user may own multiple fastening articles and switch between them for various reasons, such as aesthetics or comfort.
  • alternative fastening articles may be different colors or some fastening articles may include jewels or other aspects of traditional jewelery.
  • Any processes described herein, are not limited to the hardware and software described above. All or part of the processes can be implemented as special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) and/or an ASIC (Application Specific Integrated Circuit). All or part of the processes can be implemented, at least in part, via a computer program product tangibly embodied in non-transient computer-readable media, for execution by or to control the operation of one or more data processing apparatus, such as a computer, special purpose microprocessor, or programmable logic components.
  • a computer program can be written in any programming language, including compiled or interpreted languages.
  • a computer program can be implemented as a stand-alone program or as portion, such as a module or subroutine, of a larger program.
  • a computer program can be deployed to be executed on a single data processing device or on multiple data processing devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Claims (15)

  1. Ein System, umfassend:
    ersten Beschleunigungssensor;
    erstes Befestigungselement (110), angebracht am ersten Beschleunigungssensor, das erste Befestigungselement in der Lage dazu, den ersten Beschleunigungssensor auf einem Teil eines Daumens eines Benutzers an Ort und Stelle zu halten;
    zweiten Beschleunigungssensor;
    zweites Befestigungselement (210), angebracht am zweiten Beschleunigungssensor, das zweite Befestigungselement in der Lage dazu, den zweiten Beschleunigungssensor auf einem Handgelenk des Benutzers an Ort und Stelle zu halten, wobei Handgelenk und Daumen zur selben Hand (310) des Benutzers gehören; und
    Prozessor-Gerät, konfiguriert, um Operationen durchzuführen, umfassend:
    Empfangen (1110) einer ersten Menge an Beschleunigungsmessungen vom ersten Beschleunigungssensor;
    Erkennen (1120), basierend zumindest zum Teil auf der ersten Menge an Beschleunigungsmessungen, wenn der Daumen des Benutzers, der das erste Befestigungselement und das zweite Befestigungselement trägt, gegen eine Oberfläche anschlägt;
    Empfangen einer zweiten Menge an Beschleunigungsmessungen vom zweiten Beschleunigungssensor;
    wenn eine Berührung des Daumens erkannt wird, Ermitteln, basierend zumindest zum Teil auf der ersten Menge an Beschleunigungsmessungen und der zweiten Menge an Beschleunigungsmessungen, welches Anschlagsziel aus einer Menge an Anschlagszielen, befindlich an Fingern der Hand des Benutzers, durch den Daumen angeschlagen wurde;
    Zuordnen des Anschlags des Anschlagsziels zu einem Symbol, das mit dem Anschlagsziel assoziiert wird; und
    Übertragen, Speichern oder Anzeigen (1170) des Symbols.
  2. Verfahren, umfassend:
    Empfangen (1110) einer ersten Menge an Beschleunigungsmessungen von einem ersten Beschleunigungssensor, der an einem Daumen eines Benutzers angebracht ist;
    Empfangen (1110) einer zweiten Menge an Beschleunigungsmessungen von einem zweiten Beschleunigungssensor, der am Handgelenk des Benutzers angebracht ist, wobei das Handgelenk und der Daumen zur selben Hand des Benutzers gehören;
    Erkennen (1120), basierend zumindest zum Teil auf der ersten Menge an Beschleunigungsmessungen, wenn der Daumen eine Oberfläche anschlägt;
    wenn ein Anschlag des Daumens erkannt wird, Ermitteln, basierend zumindest zum Teil auf der ersten Menge an Beschleunigungsmessungen und der zweiten Menge an Beschleunigungsmessungen, einer Ausrichtung des ersten Beschleunigungssensors relativ zum zweiten Beschleunigungssensor;
    Identifizieren, basierend auf der Ausrichtung, eines Anschlagsziels aus einer Vielzahl an Anschlagszielen, befindlich an Fingern der Hand des Benutzers, das während des Anschlags berührt wurde;
    Zuweisen (1160) eines Symbols, das mit dem Anschlagsziel assoziiert wird, zum Anschlag; und
    Übertragen, Speichern oder Anzeigen (1170) des Symbols.
  3. Verfahren nach Anspruch 2, umfassend:
    Empfangen einer ersten Menge an magnetischen Flussmessungen von einem ersten Magnetometer, das am Daumen des Benutzers angebracht ist; und
    Empfangen einer zweiten Menge an magnetischen Flussmessungen von einem zweiten Magnetometer, das am Handgelenk des Benutzers angebracht ist; und
    wobei die Ausrichtung bestimmt wird, basierend zumindest zum Teil auf der ersten Menge an magnetischen Flussmessungen und der zweiten Menge an magnetischen Flussmessungen.
  4. Verfahren nach Anspruch 3, nach welchem das Ermitteln der Ausrichtung Folgendes umfasst:
    Filtern der ersten Menge an Beschleunigungsmessungen, um eine erste Schätzung einer Beschleunigung zu ermitteln, die durch den ersten Beschleunigungssensor erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand;
    Filtern der zweiten Menge an Beschleunigungsmessungen, um eine zweite Schätzung einer Beschleunigung zu ermitteln, die durch den zweiten Beschleunigungssensor erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand;
    Filtern der ersten Menge an Messungen des magnetischen Flusses, um eine erste Schätzung eines magnetischen Flusses zu ermitteln, der durch das erste Magnetometer erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand;
    Filtern der zweiten Menge an Messungen des magnetischen Flusses, um eine zweite Schätzung eines magnetischen Flusses zu ermitteln, der durch das zweite Magnetometer erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand; und
    Ermitteln einer Rotation, welche annähernd die erste Schätzung der Beschleunigung mit der zweiten Schätzung der Beschleunigung einander angleicht, und welche außerdem annähernd die erste Schätzung des magnetischen Flusses mit der zweiten Schätzung des magnetischen Flusses einander angleicht.
  5. Verfahren nach Anspruch 2, umfassend:
    Ermitteln, basierend auf der zweiten Menge an Beschleunigungsmessungen, einer Schätzung einer Ausrichtung einer Beschleunigung, die durch den zweiten Beschleunigungssensor hinsichtlich der Achsen des zweiten Beschleunigungssensors während des Anschlags erfahren wurde; und
    wobei Zuweisen des Symbols zum Anschlag Auswählen des Symbols aus einer Vielzahl von Symbolen umfasst, die mit dem Anschlagsziel assoziiert sind, basierend auf der Ausrichtung der Beschleunigung, die durch den zweiten Beschleunigungssensor erfahren wurde.
  6. Verfahren nach Anspruch 5, nach welchem die Beschleunigung, die durch den zweiten Beschleunigungssensor erfahren wurde, maßgeblich durch die Beschleunigung bestimmt wird, die durch die Schwerkraft der Erde verursacht wird, und die Auswahl basierend zumindest zum Teil auf einer Schätzung eines Winkels zwischen der Beschleunigung durch die Schwerkraft und einer Achse erfolgt, welche annähernd parallel zum Unterarm des Benutzers verläuft.
  7. Verfahren nach Anspruch 5, nach welchem der Benutzer in die Lage versetzt wird, eine Auswahl unter der Vielzahl der Symbole zu treffen, die mit dem Anschlagsziel assoziiert sind, durch das Anpassen eines Winkels des Handgelenks oder eines Winkels des Unterarms des Benutzers hinsichtlich der Schwerkraft der Erde während des Anschlags.
  8. Verfahren nach Anspruch 5, nach welchem der Benutzer in die Lage versetzt wird, eine Handflächenseite des Handgelenks abwärts zur Erde gerichtet zu drehen, um in einen Zeiger-Manipulationsmodus einzutreten, in welchem Beschleunigungsmessungen vom zweiten Beschleunigungssensor verwendet werden, um einen Zeiger in einem virtuellen Raum zu bewegen.
  9. Verfahren nach Anspruch 2, nach welchem das Ermitteln der Ausrichtung Folgendes umfasst:
    Filtern der ersten Menge an Beschleunigungsmessungen, um eine Schätzung eines Entschleunigungsvektors zu ermitteln, verursacht durch die Auswirkung des Daumens auf das Anschlagsziel.
  10. Verfahren nach Anspruch 1, nach welchem das Ermitteln, basierend zumindest zum Teil auf der ersten Menge an Beschleunigungsmessungen und der zweiten Menge an Beschleunigungsmessungen, welches Anschlagsziel aus einer Menge an Anschlagszielen, befindlich an Fingern der Hand des Benutzers, durch den Daumen angeschlagen wurde, Folgendes umfasst:
    Ermitteln, basierend zumindest zum Teil auf der ersten Menge an Beschleunigungsmessungen und der zweiten Menge an Beschleunigungsmessungen, einer Ausrichtung des ersten Beschleunigungssensors relativ zum zweiten Beschleunigungssensor; und
    Identifizieren, basierend auf der Ausrichtung, eines Anschlagsziels aus der Menge an Anschlagszielen, das während des Anschlags berührt wurde.
  11. System nach Anspruch 10, nach welchem die Operationen umfassen:
    Empfangen einer ersten Menge an magnetischen Flussmessungen von einem ersten Magnetometer, das am Daumen des Benutzers angebracht ist; und
    Empfangen einer zweiten Menge an magnetischen Flussmessungen von einem zweiten Magnetometer, das am Handgelenk des Benutzers angebracht ist; und
    wobei die Ausrichtung bestimmt wird, basierend zumindest zum Teil auf der ersten Menge an magnetischen Flussmessungen und der zweiten Menge an magnetischen Flussmessungen.
  12. System nach Anspruch 11, nach welchem das Ermitteln der Ausrichtung Folgendes umfasst:
    Filtern der ersten Menge an Beschleunigungsmessungen, um eine erste Schätzung einer Beschleunigung zu ermitteln, die durch den ersten Beschleunigungssensor erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich einer Hand des Benutzers auf dem Anschlagsziel befand;
    Filtern der zweiten Menge an Beschleunigungsmessungen, um eine zweite Schätzung einer Beschleunigung zu ermitteln, die durch den zweiten Beschleunigungssensor erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand;
    Filtern der ersten Menge an Messungen des magnetischen Flusses, um eine erste Schätzung eines magnetischen Flusses zu ermitteln, der durch das erste Magnetometer erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand;
    Filtern der zweiten Menge an Messungen des magnetischen Flusses, um eine zweite Schätzung eines magnetischen Flusses zu ermitteln, der durch das zweite Magnetometer erfahren wurde, während der Daumen sich im Ruhezustand hinsichtlich der Hand des Benutzers auf dem Anschlagsziel befand; und
    Ermitteln einer Rotation, welche annähernd die erste Schätzung der Beschleunigung mit der zweiten Schätzung der Beschleunigung einander angleicht, und welche außerdem annähernd die erste Schätzung des magnetischen Flusses mit der zweiten Schätzung des magnetischen Flusses einander angleicht.
  13. System nach Anspruch 10, nach welchem die Operationen umfassen:
    Ermitteln, basierend auf der zweiten Menge an Beschleunigungsmessungen, einer Schätzung einer Ausrichtung einer Beschleunigung, die durch den zweiten Beschleunigungssensor hinsichtlich der Achsen des zweiten Beschleunigungssensors während des Anschlags erfahren wurde; und
    wobei Zuweisen des Symbols zum Anschlag Auswählen des Symbols aus einer Vielzahl von Symbolen umfasst, die mit dem Anschlagsziel assoziiert sind, basierend auf der Ausrichtung der Beschleunigung, die durch den zweiten Beschleunigungssensor erfahren wurde.
  14. System nach Anspruch 13, nach welchem der Benutzer in die Lage versetzt wird, eine Handflächenseite des Handgelenks abwärts zur Erde gerichtet zu drehen, um in einen Zeiger-Manipulationsmodus einzutreten, in welchem Beschleunigungsmessungen vom zweiten Beschleunigungssensor verwendet werden, um einen Zeiger in einem virtuellen Raum zu bewegen.
  15. System nach Anspruch 10, nach welchem das Ermitteln der Ausrichtung Folgendes umfasst:
    Filtern der ersten Menge an Beschleunigungsmessungen, um eine Schätzung eines Entschleunigungsvektors zu ermitteln, verursacht durch die Auswirkung des Daumens auf das Anschlagsziel.
EP13857223.5A 2012-11-24 2013-11-21 Computerschnittstellensystem Active EP2941678B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201261729573P 2012-11-24 2012-11-24
US13/804,871 US8743052B1 (en) 2012-11-24 2013-03-14 Computing interface system
PCT/US2013/071215 WO2014081927A1 (en) 2012-11-24 2013-11-21 Computing interface system

Publications (3)

Publication Number Publication Date
EP2941678A1 EP2941678A1 (de) 2015-11-11
EP2941678A4 EP2941678A4 (de) 2016-10-19
EP2941678B1 true EP2941678B1 (de) 2019-04-10

Family

ID=50441436

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13857223.5A Active EP2941678B1 (de) 2012-11-24 2013-11-21 Computerschnittstellensystem

Country Status (7)

Country Link
US (3) US8743052B1 (de)
EP (1) EP2941678B1 (de)
CN (2) CN104919391B (de)
AU (1) AU2013347935B2 (de)
CA (1) CA2931364A1 (de)
HK (1) HK1215087A1 (de)
WO (1) WO2014081927A1 (de)

Families Citing this family (147)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9158116B1 (en) 2014-04-25 2015-10-13 Osterhout Group, Inc. Temple and ear horn assembly for headworn computer
US9715112B2 (en) 2014-01-21 2017-07-25 Osterhout Group, Inc. Suppression of stray light in head worn computing
US9952664B2 (en) 2014-01-21 2018-04-24 Osterhout Group, Inc. Eye imaging in head worn computing
US9965681B2 (en) 2008-12-16 2018-05-08 Osterhout Group, Inc. Eye imaging in head worn computing
US9229233B2 (en) 2014-02-11 2016-01-05 Osterhout Group, Inc. Micro Doppler presentations in head worn computing
US9400390B2 (en) 2014-01-24 2016-07-26 Osterhout Group, Inc. Peripheral lighting for head worn computing
US9298007B2 (en) 2014-01-21 2016-03-29 Osterhout Group, Inc. Eye imaging in head worn computing
US9366867B2 (en) 2014-07-08 2016-06-14 Osterhout Group, Inc. Optical systems for see-through displays
US20150205111A1 (en) 2014-01-21 2015-07-23 Osterhout Group, Inc. Optical configurations for head worn computing
US9671566B2 (en) 2012-06-11 2017-06-06 Magic Leap, Inc. Planar waveguide apparatus with diffraction element(s) and system employing same
US10314492B2 (en) 2013-05-23 2019-06-11 Medibotics Llc Wearable spectroscopic sensor to measure food consumption based on interaction between light and the human body
US9582035B2 (en) 2014-02-25 2017-02-28 Medibotics Llc Wearable computing devices and methods for the wrist and/or forearm
US9891718B2 (en) 2015-04-22 2018-02-13 Medibotics Llc Devices for measuring finger motion and recognizing hand gestures
EP2698686B1 (de) * 2012-07-27 2018-10-10 LG Electronics Inc. Am Handgelenk tragbares Endgerät und zugehöriges Steuerungsverfahren
US11287897B2 (en) * 2012-12-14 2022-03-29 Pixart Imaging Inc. Motion detecting system having multiple sensors
TW201423484A (zh) * 2012-12-14 2014-06-16 Pixart Imaging Inc 動態偵測系統
US9696802B2 (en) * 2013-03-20 2017-07-04 Microsoft Technology Licensing, Llc Short range wireless powered ring for user interaction and sensing
WO2015006784A2 (en) 2013-07-12 2015-01-15 Magic Leap, Inc. Planar waveguide apparatus with diffraction element(s) and system employing same
US9952042B2 (en) 2013-07-12 2018-04-24 Magic Leap, Inc. Method and system for identifying a user location
US20150124566A1 (en) 2013-10-04 2015-05-07 Thalmic Labs Inc. Systems, articles and methods for wearable electronic devices employing contact sensors
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source
US10188309B2 (en) 2013-11-27 2019-01-29 North Inc. Systems, articles, and methods for electromyography sensors
US20160313806A1 (en) * 2013-12-06 2016-10-27 Nokia Technologies Oy Apparatus and method for user input
JP2015121979A (ja) * 2013-12-24 2015-07-02 株式会社東芝 装着型情報入力装置、情報入力システム、及び情報入力方法
US20150185838A1 (en) * 2013-12-27 2015-07-02 Jose R. Camacho-Perez Wrist based wearable virtual keyboard
US20150185852A1 (en) * 2013-12-30 2015-07-02 Dau-Ning GUO Ring mobile device and operation method of the same
US10725550B2 (en) * 2014-01-07 2020-07-28 Nod, Inc. Methods and apparatus for recognition of a plurality of gestures using roll pitch yaw data
US10338678B2 (en) 2014-01-07 2019-07-02 Nod, Inc. Methods and apparatus for recognition of start and/or stop portions of a gesture using an auxiliary sensor
US11103122B2 (en) 2014-07-15 2021-08-31 Mentor Acquisition One, Llc Content presentation in head worn computing
US9829707B2 (en) 2014-08-12 2017-11-28 Osterhout Group, Inc. Measuring content brightness in head worn computing
US10684687B2 (en) 2014-12-03 2020-06-16 Mentor Acquisition One, Llc See-through computer display systems
US10649220B2 (en) 2014-06-09 2020-05-12 Mentor Acquisition One, Llc Content presentation in head worn computing
US10191279B2 (en) 2014-03-17 2019-01-29 Osterhout Group, Inc. Eye imaging in head worn computing
US11227294B2 (en) 2014-04-03 2022-01-18 Mentor Acquisition One, Llc Sight information collection in head worn computing
US9746686B2 (en) 2014-05-19 2017-08-29 Osterhout Group, Inc. Content position calibration in head worn computing
US9448409B2 (en) 2014-11-26 2016-09-20 Osterhout Group, Inc. See-through computer display systems
US9841599B2 (en) 2014-06-05 2017-12-12 Osterhout Group, Inc. Optical configurations for head-worn see-through displays
US9671613B2 (en) 2014-09-26 2017-06-06 Osterhout Group, Inc. See-through computer display systems
US9299194B2 (en) 2014-02-14 2016-03-29 Osterhout Group, Inc. Secure sharing in head worn computing
US9575321B2 (en) 2014-06-09 2017-02-21 Osterhout Group, Inc. Content presentation in head worn computing
US9810906B2 (en) 2014-06-17 2017-11-07 Osterhout Group, Inc. External user interface for head worn computing
US10254856B2 (en) 2014-01-17 2019-04-09 Osterhout Group, Inc. External user interface for head worn computing
US9529195B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. See-through computer display systems
US9366868B2 (en) 2014-09-26 2016-06-14 Osterhout Group, Inc. See-through computer display systems
US20160019715A1 (en) 2014-07-15 2016-01-21 Osterhout Group, Inc. Content presentation in head worn computing
US9939934B2 (en) 2014-01-17 2018-04-10 Osterhout Group, Inc. External user interface for head worn computing
US20150277118A1 (en) 2014-03-28 2015-10-01 Osterhout Group, Inc. Sensor dependent content position in head worn computing
US9594246B2 (en) 2014-01-21 2017-03-14 Osterhout Group, Inc. See-through computer display systems
US12105281B2 (en) 2014-01-21 2024-10-01 Mentor Acquisition One, Llc See-through computer display systems
US9529199B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. See-through computer display systems
US11737666B2 (en) 2014-01-21 2023-08-29 Mentor Acquisition One, Llc Eye imaging in head worn computing
US20150206173A1 (en) 2014-01-21 2015-07-23 Osterhout Group, Inc. Eye imaging in head worn computing
US9836122B2 (en) 2014-01-21 2017-12-05 Osterhout Group, Inc. Eye glint imaging in see-through computer display systems
US9494800B2 (en) 2014-01-21 2016-11-15 Osterhout Group, Inc. See-through computer display systems
US11669163B2 (en) 2014-01-21 2023-06-06 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US11892644B2 (en) 2014-01-21 2024-02-06 Mentor Acquisition One, Llc See-through computer display systems
US11487110B2 (en) 2014-01-21 2022-11-01 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9651784B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US12093453B2 (en) 2014-01-21 2024-09-17 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US9615742B2 (en) 2014-01-21 2017-04-11 Osterhout Group, Inc. Eye imaging in head worn computing
US9753288B2 (en) 2014-01-21 2017-09-05 Osterhout Group, Inc. See-through computer display systems
US9766463B2 (en) 2014-01-21 2017-09-19 Osterhout Group, Inc. See-through computer display systems
US20150205135A1 (en) 2014-01-21 2015-07-23 Osterhout Group, Inc. See-through computer display systems
US9846308B2 (en) 2014-01-24 2017-12-19 Osterhout Group, Inc. Haptic systems for head-worn computers
CN103793057B (zh) * 2014-01-26 2017-02-01 华为终端有限公司 信息处理方法、装置及设备
US9401540B2 (en) 2014-02-11 2016-07-26 Osterhout Group, Inc. Spatial location presentation in head worn computing
US20150241963A1 (en) 2014-02-11 2015-08-27 Osterhout Group, Inc. Eye imaging in head worn computing
US10429888B2 (en) 2014-02-25 2019-10-01 Medibotics Llc Wearable computer display devices for the forearm, wrist, and/or hand
US20160187651A1 (en) 2014-03-28 2016-06-30 Osterhout Group, Inc. Safety for a vehicle operator with an hmd
US20150309534A1 (en) 2014-04-25 2015-10-29 Osterhout Group, Inc. Ear horn assembly for headworn computer
US9651787B2 (en) 2014-04-25 2017-05-16 Osterhout Group, Inc. Speaker assembly for headworn computer
US9423842B2 (en) 2014-09-18 2016-08-23 Osterhout Group, Inc. Thermal management for head-worn computer
US10853589B2 (en) 2014-04-25 2020-12-01 Mentor Acquisition One, Llc Language translation with head-worn computing
US9672210B2 (en) 2014-04-25 2017-06-06 Osterhout Group, Inc. Language translation with head-worn computing
US9594427B2 (en) 2014-05-23 2017-03-14 Microsoft Technology Licensing, Llc Finger tracking
US10663740B2 (en) 2014-06-09 2020-05-26 Mentor Acquisition One, Llc Content presentation in head worn computing
CN105204611B (zh) * 2014-06-24 2018-01-09 北京创思博德科技有限公司 一种基于手指运动模式的莫尔斯码字符输入系统和方法
CN105446461B (zh) * 2014-06-24 2019-03-26 北京创思博德科技有限公司 手势识别方法及使用该方法的手掌虚拟键盘及其输入方法
US9619024B2 (en) * 2014-06-24 2017-04-11 Beijing TransBorder Information Technology Co., Ltd. Virtual input device and virtual input method
WO2016003365A1 (en) * 2014-07-04 2016-01-07 Loke Kar Kit Bernard A wearable input device
US9582076B2 (en) 2014-09-17 2017-02-28 Microsoft Technology Licensing, Llc Smart ring
JP6329469B2 (ja) 2014-09-17 2018-05-23 株式会社東芝 認識装置、認識方法及び認識プログラム
US20180101277A9 (en) * 2014-09-24 2018-04-12 Evan John Kaye Skin Touchpad
CN105511748A (zh) 2014-09-25 2016-04-20 阿里巴巴集团控股有限公司 自适应调整智能终端的用户界面的方法及装置
CN104394312B (zh) * 2014-10-23 2017-08-22 小米科技有限责任公司 拍摄控制方法及装置
US9684172B2 (en) 2014-12-03 2017-06-20 Osterhout Group, Inc. Head worn computer display systems
US10191544B2 (en) * 2014-12-06 2019-01-29 Horsemoon Llc Hand gesture recognition system for controlling electronically controlled devices
USD743963S1 (en) 2014-12-22 2015-11-24 Osterhout Group, Inc. Air mouse
USD751552S1 (en) 2014-12-31 2016-03-15 Osterhout Group, Inc. Computer glasses
USD753114S1 (en) 2015-01-05 2016-04-05 Osterhout Group, Inc. Air mouse
US10878775B2 (en) 2015-02-17 2020-12-29 Mentor Acquisition One, Llc See-through computer display systems
US20160239985A1 (en) 2015-02-17 2016-08-18 Osterhout Group, Inc. See-through computer display systems
US9652038B2 (en) 2015-02-20 2017-05-16 Sony Interactive Entertainment Inc. Magnetic tracking of glove fingertips
US9983680B2 (en) 2015-03-23 2018-05-29 Intel Corporation Gesture recognition mechanism
US9921651B2 (en) * 2015-04-29 2018-03-20 International Business Machines Company Video display for visually impaired people
US9952676B2 (en) * 2015-06-25 2018-04-24 Intel Corporation Wearable device with gesture recognition mechanism
US10139966B2 (en) 2015-07-22 2018-11-27 Osterhout Group, Inc. External user interface for head worn computing
US11003246B2 (en) 2015-07-22 2021-05-11 Mentor Acquisition One, Llc External user interface for head worn computing
US20170045948A1 (en) * 2015-08-12 2017-02-16 Fin Robotics, Inc. Controlling target devices
CN105159117A (zh) * 2015-09-02 2015-12-16 王盘铭 运动模式悬空识别系统
US10324494B2 (en) 2015-11-25 2019-06-18 Intel Corporation Apparatus for detecting electromagnetic field change in response to gesture
US20170193289A1 (en) * 2015-12-31 2017-07-06 Microsoft Technology Licensing, Llc Transform lightweight skeleton and using inverse kinematics to produce articulate skeleton
JP6709627B2 (ja) * 2016-01-29 2020-06-17 東芝テック株式会社 情報処理装置およびプログラム
US10850116B2 (en) 2016-12-30 2020-12-01 Mentor Acquisition One, Llc Head-worn therapy device
US10591728B2 (en) 2016-03-02 2020-03-17 Mentor Acquisition One, Llc Optical systems for head-worn computers
US10151606B1 (en) 2016-02-24 2018-12-11 Ommo Technologies, Inc. Tracking position and movement using a magnetic field
US10667981B2 (en) 2016-02-29 2020-06-02 Mentor Acquisition One, Llc Reading assistance system for visually impaired
US9880441B1 (en) 2016-09-08 2018-01-30 Osterhout Group, Inc. Electrochromic systems for head-worn computer systems
US9826299B1 (en) 2016-08-22 2017-11-21 Osterhout Group, Inc. Speaker systems for head-worn computer systems
US9864434B2 (en) 2016-03-30 2018-01-09 Huami Inc. Gesture control of interactive events using multiple wearable devices
US10684478B2 (en) 2016-05-09 2020-06-16 Mentor Acquisition One, Llc User interface systems for head-worn computers
US10466491B2 (en) 2016-06-01 2019-11-05 Mentor Acquisition One, Llc Modular systems for head-worn computers
US10824253B2 (en) 2016-05-09 2020-11-03 Mentor Acquisition One, Llc User interface systems for head-worn computers
CN105824431B (zh) * 2016-06-12 2018-12-04 齐向前 信息输入装置与方法
US10298282B2 (en) 2016-06-16 2019-05-21 Intel Corporation Multi-modal sensing wearable device for physiological context measurement
WO2020112986A1 (en) 2018-11-27 2020-06-04 Facebook Technologies, Inc. Methods and apparatus for autocalibration of a wearable electrode sensor system
US10168776B2 (en) 2016-08-23 2019-01-01 International Business Machines Corporation Remote control via proximity data
CN107995965A (zh) * 2016-09-29 2018-05-04 深圳市柔宇科技有限公司 控制方法、控制装置及智能穿戴设备
USD864959S1 (en) 2017-01-04 2019-10-29 Mentor Acquisition One, Llc Computer glasses
US10578869B2 (en) 2017-07-24 2020-03-03 Mentor Acquisition One, Llc See-through computer display systems with adjustable zoom cameras
US10422995B2 (en) 2017-07-24 2019-09-24 Mentor Acquisition One, Llc See-through computer display systems with stray light management
US11409105B2 (en) 2017-07-24 2022-08-09 Mentor Acquisition One, Llc See-through computer display systems
US10152141B1 (en) 2017-08-18 2018-12-11 Osterhout Group, Inc. Controller movement tracking with light emitters
WO2019079757A1 (en) 2017-10-19 2019-04-25 Ctrl-Labs Corporation SYSTEMS AND METHODS FOR IDENTIFYING BIOLOGICAL STRUCTURES ASSOCIATED WITH NEUROMUSCULAR SOURCE SIGNALS
US10866652B2 (en) 2017-11-13 2020-12-15 Samsung Electronics Co., Ltd. System and method for distributed device tracking
CN109992093B (zh) * 2017-12-29 2024-05-03 博世汽车部件(苏州)有限公司 一种手势比较方法以及手势比较系统
US11961494B1 (en) 2019-03-29 2024-04-16 Meta Platforms Technologies, Llc Electromagnetic interference reduction in extended reality environments
US11493993B2 (en) * 2019-09-04 2022-11-08 Meta Platforms Technologies, Llc Systems, methods, and interfaces for performing inputs based on neuromuscular control
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
US11481030B2 (en) 2019-03-29 2022-10-25 Meta Platforms Technologies, Llc Methods and apparatus for gesture detection and classification
US11150730B1 (en) 2019-04-30 2021-10-19 Facebook Technologies, Llc Devices, systems, and methods for controlling computing devices via neuromuscular signals of users
CN110134225B (zh) * 2018-02-09 2022-12-06 景俊年 一种智能手环及其控制方法
US11614793B2 (en) * 2018-02-28 2023-03-28 Logitech Europe S.A. Precision tracking of user interaction with a virtual input device
US10276289B1 (en) 2018-06-01 2019-04-30 Ommo Technologies, Inc. Rotating a permanent magnet in a position detection system
WO2019236096A1 (en) * 2018-06-08 2019-12-12 Hewlett-Packard Development Company, L.P. Computing input devices with sensors concealed in articles of clothing
US11027822B2 (en) * 2018-06-21 2021-06-08 Rockwell Collins, Inc. Control system for touchless operation of mechanical input devices
EP3853698A4 (de) 2018-09-20 2021-11-17 Facebook Technologies, LLC Neuromuskuläre texteingabe, schreiben und zeichnen in systemen mit erweiterter realität
DE102018220660A1 (de) * 2018-11-30 2020-06-04 Zf Friedrichshafen Ag Bedienanordnung für ein Fahrzeug, Fahrzeug mit der Bedienanordnung sowie Verfahren zum Bedienen mit der Bedienanordnung
WO2020239202A1 (de) * 2019-05-28 2020-12-03 Asset Green Holding Ug Tragbare mensch-maschine schnittstelle und mit dieser ausführbares verfahren
EP3660633A1 (de) 2019-07-31 2020-06-03 Taurum Technologies SL Handgetragene dateneingabevorrichtung
GB201914638D0 (en) * 2019-10-10 2019-11-27 Rolls Royce Plc Inspection system
US11531392B2 (en) * 2019-12-02 2022-12-20 Finchxr Ltd. Tracking upper arm movements using sensor modules attached to the hand and forearm
US11003247B1 (en) * 2020-03-20 2021-05-11 Microsoft Technology Licensing, Llc Deployable controller
US11340707B2 (en) * 2020-05-29 2022-05-24 Microsoft Technology Licensing, Llc Hand gesture-based emojis
CN114639158A (zh) * 2020-11-30 2022-06-17 伊姆西Ip控股有限责任公司 计算机交互方法、设备和程序产品
US11573642B2 (en) * 2020-12-30 2023-02-07 Huawei Technologies Co., Ltd. Wearable devices, methods and media for multi-finger mid-air gesture recognition
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3972038A (en) 1975-03-28 1976-07-27 Nasa Accelerometer telemetry system
JP2698320B2 (ja) * 1993-08-31 1998-01-19 日本電信電話株式会社 常装着型入力システム、常装着型意図伝達システム、常装着型音楽用キーボードシステム及び常装着型点字入出力システム
JP3630712B2 (ja) 1994-02-03 2005-03-23 キヤノン株式会社 ジェスチャー入力方法及びその装置
JP2000132305A (ja) * 1998-10-23 2000-05-12 Olympus Optical Co Ltd 操作入力装置
US20030051836A1 (en) * 2001-05-21 2003-03-20 Novozymes A/S Enzymatic hydrolysis of a polymer comprising vinyl acetate monomer
US20030142065A1 (en) 2002-01-28 2003-07-31 Kourosh Pahlavan Ring pointer device with inertial sensors
US20040001255A1 (en) * 2002-06-28 2004-01-01 Integrated Photonics, Inc. Polarization compensators and optical devices and systems incorporating polarization compensators
JP4119187B2 (ja) * 2002-07-17 2008-07-16 国立大学法人金沢大学 入力装置
KR100486739B1 (ko) * 2003-06-27 2005-05-03 삼성전자주식회사 착용형 휴대폰 및 그 사용방법
US7565295B1 (en) 2003-08-28 2009-07-21 The George Washington University Method and apparatus for translating hand gestures
JP4559140B2 (ja) * 2004-07-05 2010-10-06 ソフトバンクモバイル株式会社 電子機器
WO2007127537A1 (en) 2006-03-15 2007-11-08 Qualcomm Incorporated Sensor-based orientation systeim
US20100023314A1 (en) 2006-08-13 2010-01-28 Jose Hernandez-Rebollar ASL Glove with 3-Axis Accelerometers
US8881049B2 (en) 2007-12-14 2014-11-04 Apple Inc. Scrolling displayed objects using a 3D remote controller in a media system
US20100090949A1 (en) * 2008-07-22 2010-04-15 Shanda Computer (Shanghai) Co., Ltd. Method and Apparatus for Input Device
EP2347321B1 (de) * 2008-09-25 2013-09-18 Movea S.A Befehl durch gestikschnittstelle
CN101751126A (zh) * 2008-12-17 2010-06-23 孙骏恭 使用多个传感器空间的免提的基于手势的接口
FR2944461B1 (fr) * 2009-04-15 2011-05-20 Air Liquide Procede et installation de traitement de surface par jets de fluide cryogenique.
US9596999B2 (en) 2009-06-17 2017-03-21 Sotera Wireless, Inc. Body-worn pulse oximeter
US9008973B2 (en) 2009-11-09 2015-04-14 Barry French Wearable sensor system with gesture recognition for measuring physical performance
US9110505B2 (en) * 2010-04-16 2015-08-18 Innovative Devices Inc. Wearable motion sensing computing interface
US20110260963A1 (en) * 2010-04-27 2011-10-27 Gm Global Technology Operations, Inc. Symbolic input via mid-air finger/thumb motions
EP2418562B1 (de) 2010-08-13 2013-04-17 Deutsches Primatenzentrum GmbH (DPZ) Modellierung der Hand- und Armposition und -ausrichtung
CN102478959A (zh) * 2010-11-28 2012-05-30 蒋霞 一种电子设备的控制系统和方法
US20120139708A1 (en) * 2010-12-06 2012-06-07 Massachusetts Institute Of Technology Wireless Hand Gesture Capture
US8884877B2 (en) * 2011-04-29 2014-11-11 Movea Pointing device
US9218058B2 (en) * 2011-06-16 2015-12-22 Daniel Bress Wearable digital input device for multipoint free space data collection and analysis
US8228315B1 (en) 2011-07-12 2012-07-24 Google Inc. Methods and systems for a virtual input device
EP2600109A3 (de) * 2011-11-30 2015-03-25 Sony Ericsson Mobile Communications AB Verfahren zur Kalibrierung einer Sensoreinheit und diese umfassendes Zubehörteil
TW201325101A (zh) * 2011-12-13 2013-06-16 Askey Technology Jiangsu Ltd 遠端多點遙控裝置及系統
US10509469B2 (en) * 2016-04-21 2019-12-17 Finch Technologies Ltd. Devices for controlling computers based on motions and positions of hands

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
AU2013347935A1 (en) 2015-07-09
CN104919391A (zh) 2015-09-16
WO2014081927A1 (en) 2014-05-30
CA2931364A1 (en) 2014-05-30
HK1215087A1 (zh) 2016-08-12
AU2013347935B2 (en) 2019-09-19
CN108829243B (zh) 2021-10-15
US10503275B2 (en) 2019-12-10
US20140176439A1 (en) 2014-06-26
US20190346940A1 (en) 2019-11-14
CN104919391B (zh) 2018-06-22
US8743052B1 (en) 2014-06-03
EP2941678A4 (de) 2016-10-19
CN108829243A (zh) 2018-11-16
EP2941678A1 (de) 2015-11-11

Similar Documents

Publication Publication Date Title
US20190346940A1 (en) Computing interface system
US20220083149A1 (en) Computing interface system
US11231786B1 (en) Methods and apparatus for using the human body as an input device
US10649549B2 (en) Device, method, and system to recognize motion using gripped object
US11009951B2 (en) Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
JP6545258B2 (ja) スマートリング
KR101284797B1 (ko) 착용형 컴퓨팅 환경 기반의 사용자 인터페이스 장치 및 그 방법
US20100103104A1 (en) Apparatus for user interface based on wearable computing environment and method thereof
US10928929B2 (en) Method for displaying a virtual interaction on at least one screen and input device, system and method for a virtual application by means of a computing unit
US10120444B2 (en) Wearable device
US20220155866A1 (en) Ring device having an antenna, a touch pad, and/or a charging pad to control a computing device based on user motions
CN103823548A (zh) 电子设备、穿戴式设备、控制系统和方法
US20230031200A1 (en) Touchless, Gesture-Based Human Interface Device
US10082885B2 (en) Information input and output apparatus and information input and output method
JP2024048680A (ja) 制御装置、制御方法、プログラム
Henschke et al. Extending the index finger is worse than sitting: Posture and minimal objects in 3D pointing interfaces

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150618

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602013053852

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G06F0003000000

Ipc: G06F0003010000

A4 Supplementary search report drawn up and despatched

Effective date: 20160921

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 3/023 20060101ALI20160915BHEP

Ipc: G06F 3/0346 20130101ALI20160915BHEP

Ipc: G06F 3/01 20060101AFI20160915BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20181022

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1119571

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190415

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013053852

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190410

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1119571

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190910

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190710

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190711

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190710

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190810

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013053852

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

26N No opposition filed

Effective date: 20200113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191121

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20191130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20131121

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190410

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231116

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IE

Payment date: 20231127

Year of fee payment: 11

Ref country code: FR

Payment date: 20231117

Year of fee payment: 11

Ref country code: DE

Payment date: 20231121

Year of fee payment: 11

Ref country code: CH

Payment date: 20231201

Year of fee payment: 11