US20150057770A1 - Systems, articles, and methods for human-electronics interfaces - Google Patents

Systems, articles, and methods for human-electronics interfaces Download PDF

Info

Publication number
US20150057770A1
US20150057770A1 US14/465,194 US201414465194A US2015057770A1 US 20150057770 A1 US20150057770 A1 US 20150057770A1 US 201414465194 A US201414465194 A US 201414465194A US 2015057770 A1 US2015057770 A1 US 2015057770A1
Authority
US
United States
Prior art keywords
processor
emg
gesture identification
identification flag
signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/465,194
Inventor
Matthew Bailey
Stephen Lake
Aaron Grant
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thaimic Labs Inc
Facebook Technologies LLC
Original Assignee
Thaimic Labs Inc
Thalmic Labs Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US201361869526P priority Critical
Application filed by Thaimic Labs Inc, Thalmic Labs Inc filed Critical Thaimic Labs Inc
Priority to US14/465,194 priority patent/US20150057770A1/en
Publication of US20150057770A1 publication Critical patent/US20150057770A1/en
Assigned to Thalmic Labs Inc. reassignment Thalmic Labs Inc. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LAKE, STEPHEN, BAILEY, MATTHEW, GRANT, AARON
Assigned to NORTH INC. reassignment NORTH INC. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: Thalmic Labs Inc.
Assigned to CTRL-LABS CORPORATION reassignment CTRL-LABS CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NORTH INC.
Assigned to FACEBOOK TECHNOLOGIES, INC. reassignment FACEBOOK TECHNOLOGIES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CTRL-LABS CORPORATION
Assigned to FACEBOOK TECHNOLOGIES, LLC reassignment FACEBOOK TECHNOLOGIES, LLC CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY DATA PREVIOUSLY RECORDED AT REEL: 051649 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Assignors: CTRL-LABS CORPORATION
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A61B5/0488
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/389Electromyography [EMG]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6824Arm or wrist
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6829Foot or ankle
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1121Determining geometric values, e.g. centre of rotation or angular range of movement
    • A61B5/1122Determining geometric values, e.g. centre of rotation or angular range of movement of movement trajectories

Abstract

Human-electronics interfaces in which a wearable electromyography (“EMG”) device is operated to control virtually any electronic device are described. In response to detected muscle activity and/or motions of a user, the wearable EMG device transmits generic gesture identification flags that are not specific to the particular electronic device(s) being controlled. An electronic device being controlled is programmed with user-definable instructions for how to interpret and respond to the gesture identification flags.

Description

    BACKGROUND
  • 1. Technical Field
  • The present systems, articles, and methods generally relate to human-electronics interfaces and particularly relate to electromyographic control of electronic devices.
  • 2. Description of the Related Art
  • Wearable Electronic Devices
  • Electronic devices are commonplace throughout most of the world today. Advancements in integrated circuit technology have enabled the development of electronic devices that are sufficiently small and lightweight to be carried by the user. Such “portable” electronic devices may include on-board power supplies (such as batteries or other power storage systems) and may be designed to operate without any wire-connections to other electronic systems; however, a small and lightweight electronic device may still be considered portable even if it includes a wire-connection to another electronic system. For example, a microphone may be considered a portable electronic device whether it is operated wirelessly or through a wire-connection.
  • The convenience afforded by the portability of electronic devices has fostered a huge industry. Smartphones, audio players, laptop computers, tablet computers, and ebook readers are all examples of portable electronic devices. However, the convenience of being able to carry a portable electronic device has also introduced the inconvenience of having one's hand(s) encumbered by the device itself. This problem is addressed by making an electronic device not only portable, but wearable.
  • A wearable electronic device is any portable electronic device that a user can carry without physically grasping, clutching, or otherwise holding onto the device with their hand(s). For example, a wearable electronic device may be attached or coupled to the user by a strap or straps, a band or bands, a clip or clips, an adhesive, a pin and clasp, an article of clothing, tension or elastic support, an interference fit, an ergonomic form, etc. Examples of wearable electronic devices include digital wristwatches, electronic armbands, electronic rings, electronic ankle-bracelets or “anklets,” head-mounted electronic display units, hearing aids, and so on.
  • Human-Electronics Interfaces
  • A wearable electronic device may provide direct functionality for a user (such as audio playback, data display, computing functions, etc.) or it may provide electronics to interact with, receive information from, or control another electronic device. For example, a wearable electronic device may include sensors that detect inputs effected by a user and transmit signals to another electronic device based on those inputs. Sensor-types and input-types may each take on a variety of forms, including but not limited to: tactile sensors (e.g., buttons, switches, touchpads, or keys) providing manual control, acoustic sensors providing voice-control, electromyography sensors providing gesture control, and/or accelerometers providing gesture control.
  • A human-computer interface (“HCI”) is an example of a human-electronics interface. The present systems, articles, and methods may be applied to HCIs, but may also be applied to any other form of human-electronics interface.
  • Electromyography Devices
  • Electromyography (“EMG”) is a process for detecting and processing the electrical signals generated by muscle activity. EMG devices employ EMG sensors that are responsive to the range of electrical potentials (typically μV-mV) involved in muscle activity. EMG signals may be used in a wide variety of applications, including: medical monitoring and diagnosis, muscle rehabilitation, exercise and training, prosthetic control, and even in controlling functions of electronic devices.
  • Human-electronics interfaces that employ EMG have been proposed in the art. For example, U.S. Pat. No. 6,244,873 and U.S. Pat. No. 8,170,656 describe such systems. Characteristics that are common to these known proposals will now be described.
  • Typically, such systems (including the two examples listed above) employ a wearable EMG device that exclusively controls specific, pre-defined functions of a specific, pre-defined “receiving” electronic device. The term “pre-defined” here refers to information that is programmed into the wearable EMG device (or with which the wearable EMG device is programmed) in advance of a following interaction with a receiving device. The wearable EMG device typically includes built-in EMG sensors that detect muscle activity of a user and an on-board processor that determines when the detected muscle activity corresponds to a pre-defined gesture. The on-board processor maps each pre-defined gesture to a particular pre-defined function of the pre-defined receiving device. In other words, the wearable EMG device stores and executes pre-defined mappings between detected gestures and receiving device functions. The receiving device function(s) is/are then controlled by one or more “command(s)” that is/are output by the wearable EMG device. Each command that is output by the wearable EMG device has already been formulated to control (and is therefore limited to exclusively controlling) a specific function of a specific receiving device prior to being transmitted by the wearable EMG device.
  • The wearable EMG devices proposed in the art are hard-coded to map pre-defined gestures to specific, pre-defined commands controlling specific, pre-defined functions of a specific, pre-defined receiving device. The wearable EMG devices proposed in the art are programmed with information about the specific receiving device (and/or about a specific application within the specific receiving device) under their control such that the wearable EMG devices proposed in the art output commands that include instructions that are specifically formulated for the specific receiving device (and/or the specific application within the specific receiving device). Thus, existing proposals for human-electronics interfaces that employ EMG are limited in their versatility because they employ a wearable EMG device that is hard-coded to control a specific electronic device (and/or a specific application within a specific electronic device). For such systems, the wearable EMG device needs to be modified/adapted for each distinct use (e.g., the wearable EMG device needs to be programmed with command signals that are specific to the receiving device and/or specific to the application within the receiving device). Because the outputs (i.e., commands) provided by such wearable EMG devices are hard-coded with information about the function(s) of the receiving device(s), a user cannot use such a wearable EMG device to control any generic electronic device (or any generic application within an electronic device) without reprogramming/reconfiguring the wearable EMG device itself. A user who wishes to control multiple electronic devices (or multiple applications within a single electronic device, either simultaneously or in sequence) must use multiple such wearable EMG devices with each wearable EMG device separately controlling a different electronic device, or the user must re-program a single such wearable EMG device in between uses. There is a need in the art for a human-electronics interface employing EMG that overcomes these limitations.
  • BRIEF SUMMARY
  • A wearable electromyography (“EMG”) device may be summarized as including: at least one EMG sensor to in use detect muscle activity of a user of the wearable EMG device and provide at least one signal in response to the detected muscle activity; a processor communicatively coupled to the at least one EMG sensor, the processor to in use determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor; and an output terminal communicatively coupled to the processor to in use transmit the gesture identification flag. The gesture identification flag may be independent of any downstream processor-based device and generic to a variety of end user applications executable by a variety of downstream processor-based devices useable with the wearable EMG device.
  • The wearable EMG device may further include a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags. The non-transitory processor-readable storage medium may store processor-executable instructions that embody and/or produce/effect a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the processor, the processor-executable instructions may cause the processor to determine a gesture identification flag in accordance with the mapping. The non-transitory processor-readable storage medium may store processor-executable instructions that, when executed by the processor, cause the processor to determine a gesture identification flag based at least in part on at least one signal provided by the at least one EMG sensor.
  • The wearable EMG device may further include at least one accelerometer communicatively coupled to the processor, the at least one accelerometer to in use detect motion effected by the user of the wearable EMG device and provide at least one signal in response to the detected motion, and the processor may in use determine the gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least one signal provided by the at least one accelerometer.
  • The processor may be selected from the group consisting of: a digital microprocessor, a digital microcontroller, a digital signal processor, a graphics processing unit, an application specific integrated circuit, a programmable gate array, and a programmable logic unit. The at least one EMG sensor may include a plurality of EMG sensors, and the wearable EMG device may further include a set of communicative pathways to route signals provided by the plurality of EMG sensors to the processor, wherein each EMG sensor in the plurality of EMG sensors is communicatively coupled to the processor by at least one communicative pathway from the set of communicative pathways. The wearable EMG device may further include a set of pod structures that form physically coupled links of the wearable EMG device, wherein each pod structure in the set of pod structures is positioned adjacent and physically coupled to at least one other pod structure in the set of pod structures, and wherein the set of pod structures comprises at least two sensor pods and a processor pod, each of the at least two sensor pods comprising a respective EMG sensor from the plurality of EMG sensors and the processor pod comprising the processor. Each pod structure in the set of pod structures may be positioned adjacent and in between two other pod structures in the set of pod structures and physically coupled to the two other pod structures in the set of pod structures, and the set of pod structures may form a perimeter of an annular configuration. The wearable EMG device may further include at least one adaptive coupler, wherein each respective pod structure in the set of pod structures is adaptively physically coupled to at least one adjacent pod structure in the set of pod structures by at least one adaptive coupler.
  • The output terminal of the wearable EMG device may include at least one of a wireless transmitter and/or a tethered connector port. The at least one EMG sensor may include at least one capacitive EMG sensor.
  • A method of operating a wearable electromyography (“EMG”) device to provide electromyographic control of an electronic device, wherein the wearable EMG device includes at least one EMG sensor, a processor, and an output terminal, may be summarized as including: detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor; providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity; determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor, wherein the gesture identification flag is independent of the electronic device; and transmitting the gesture identification flag to the electronic device by the output terminal. Detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor may include detecting muscle activity of the user of the wearable EMG device by a first EMG sensor and by at least a second EMG sensor. Providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity may include providing at least a first signal from the first EMG sensor to the processor in response to the detected muscle activity and providing at least a second signal from the second EMG sensor to the processor in response to the detected muscle activity. Determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor may include determining, by the processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the processor and the at least a second signal provided from the at least a second EMG sensor to the processor.
  • The wearable EMG device may further include a non-transitory processor-readable storage medium that stores processor-executable instructions, and determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor may include executing the processor-executable instructions by the processor to cause the processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor.
  • The wearable EMG device may further include at least one accelerometer, and the method may further include: detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and providing at least one signal from the at least one accelerometer to the processor in response to the detected motion. Determining a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor may include determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor. The wearable EMG device may include a non-transitory processor-readable storage medium that stores processor-executable instructions, and determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor may includes executing the processor-executable instructions by the processor to cause the processor to determine the gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor.
  • The output terminal of the wearable EMG device may include a wireless transmitter, and transmitting the gesture identification flag to the electronic device by the output terminal may include wirelessly transmitting the gesture identification flag to the electronic device by the wireless transmitter.
  • A system that enables electromyographic control of an electronic device may be summarized as including: a wearable electromyography (“EMG”) device comprising: at least one EMG sensor to in use detect muscle activity of a user of the wearable EMG device and provide at least one signal in response to the detected muscle activity, a first processor communicatively coupled to the at least one EMG sensor, the first processor to in use determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor, and an output terminal communicatively coupled to the first processor, the output terminal to in use transmit the gesture identification flag; and an electronic device comprising: an input terminal to in use receive the gesture identification flag, and a second processor communicatively coupled to the input terminal, the second processor to in use determine a function of the electronic device based at least in part on the gesture identification flag. The gesture identification flag may be independent of the electronic device and generic to a variety of end user applications executable by the electronic device.
  • The wearable EMG device of the system may further include a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags. The non-transitory processor-readable storage medium of the wearable EMG device may store processor-executable instructions that embody and/or produce/effect a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the first processor, the processor-executable instructions may cause the first processor to determine a gesture identification flag in accordance with the mapping.
  • The wearable EMG device of the system may include a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores processor-executable instructions that, when executed by the first processor, cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor.
  • The wearable EMG device of the system may include at least one accelerometer communicatively coupled to the first processor, the at least one accelerometer to in use detect motion effected by the user of the wearable EMG device and provide at least one signal in response to the detected motion, and the first processor may in use determine a gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least on signal provided by the at least one accelerometer.
  • The electronic device of the system may include a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores at least a set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
  • The electronic device of the system may include a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores: a first application executable by the electronic device; at least a second application executable by the electronic device; a first set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the first application based at least in part on a gesture identification flag; and a second set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the second application based at least in part on a gesture identification flag.
  • The output terminal of the wearable EMG device may include a first tethered connector port, the input terminal of the electronic device may include a second tethered connector port, and the system may further include a communicative pathway to in use communicatively couple the first tethered connector port to the second tethered connector port and to route the gesture identification flag from the output terminal of the wearable EMG device to the input terminal of the electronic device.
  • The output terminal of the wearable EMG device may include a wireless transmitter to in use wirelessly transmit the gesture identification flag, the input terminal of the electronic device may include a tethered connector port, and the system may include a wireless receiver to in use communicatively couple to the tethered connector port of the electronic device and to in use wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
  • The output terminal of the wearable EMG device may include a wireless transmitter to in use wirelessly transmit the gesture identification flag and the input terminal of the electronic device may include a wireless receiver to in use wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
  • The electronic device may be selected from the group consisting of: a computer, a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone, a portable electronic device, an audio player, a television, a video player, a video game console, a robot, a light switch, and a vehicle.
  • A method of electromyographically controlling at least one function of an electronic device by a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes at least one EMG sensor, a first processor, and an output terminal and the electronic device includes an input terminal and a second processor, may be summarized as including: detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor; providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity; determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor, wherein the gesture identification flag is independent of the electronic device; transmitting the gesture identification flag by the output terminal of the wearable EMG device; receiving the gesture identification flag by the input terminal of the electronic device; determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag; and performing the function by the electronic device. Detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor may include detecting muscle activity of the user of the wearable EMG device by a first EMG sensor of the wearable EMG device and by at least a second EMG sensor of the wearable EMG device. Providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity may include providing at least a first signal from the first EMG sensor to the first processor in response to the detected muscle activity and providing at least a second signal from the send EMG sensor to the first processor in response to the detected muscle activity. Determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor may include determining, by the first processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the first processor and the at least a second signal provided from the at least a second EMG sensor to the first processor.
  • The wearable EMG device may include a non-transitory processor-readable medium that stores processor-executable instructions, and determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor may include executing the processor-executable instructions by the first processor to cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor.
  • The wearable EMG device may include at least one accelerometer, and the method may include: detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and providing at least one signal from the at least one accelerometer to the first processor in response to the detected motion. Determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor may include determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor and the at least one signal provided by the at least one accelerometer to the first processor.
  • The output terminal of the wearable EMG device may include a wireless transmitter and the input terminal of the electronic device may include a wireless receiver. Transmitting the gesture identification flag by the output terminal of the wearable EMG device may include wirelessly transmitting the gesture identification flag by the wireless transmitter of the wearable EMG device, and receiving the gesture identification flag by the input terminal of the electronic device may include wirelessly receiving the gesture identification flag by the wireless receiver of the electronic device.
  • The electronic device may include a non-transitory processor-readable storage medium that stores processor-executable instructions, and determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag may include executing the processor-executable instructions by the second processor to cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
  • BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
  • In the drawings, identical reference numbers identify similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes of various elements and angles are not drawn to scale, and some of these elements are arbitrarily enlarged and positioned to improve drawing legibility. Further, the particular shapes of the elements as drawn are not intended to convey any information regarding the actual shape of the particular elements, and have been solely selected for ease of recognition in the drawings.
  • FIG. 1 is a perspective view of an exemplary wearable electromyography device that forms part of a human-electronics interface in accordance with the present systems, articles and methods.
  • FIG. 2 is an illustrative diagram of a system that enables electromyographic control of an electronic device in accordance with the present systems, articles, and methods.
  • FIG. 3 is a flow-diagram showing a method of operating a wearable electromyography device to provide electromyographic control of an electronic device in accordance with the present systems, articles, and methods.
  • FIG. 4 is a flow-diagram showing a method of operating a wearable electromyography device to provide both electromyographic and motion control of an electronic device in accordance with the present systems, articles, and methods.
  • FIG. 5 is a schematic illustration that shows an exemplary mapping between a set of exemplary gestures and a set of exemplary gesture identification flags in accordance with the present systems, articles, and methods.
  • FIG. 6 is a flow-diagram showing a method of electromyographically controlling at least one function of an electronic device by a wearable electromyography device in accordance with the present systems, articles, and methods.
  • FIG. 7 is a schematic illustration that shows an exemplary mapping between a set of exemplary gesture identification flags and a set of exemplary functions of an electronic device in accordance with the present systems, articles, and methods.
  • DETAILED DESCRIPTION
  • In the following description, certain specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. However, one skilled in the relevant art will recognize that embodiments may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with electronic devices, and in particular portable electronic devices such as wearable electronic devices, have not been shown or described in detail to avoid unnecessarily obscuring descriptions of the embodiments.
  • Unless the context requires otherwise, throughout the specification and claims which follow, the word “comprise” and variations thereof, such as, “comprises” and “comprising” are to be construed in an open, inclusive sense, that is as “including, but not limited to.”
  • Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
  • As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the content clearly dictates otherwise. It should also be noted that the term “or” is generally employed in its broadest sense, that is as meaning “and/or” unless the content clearly dictates otherwise.
  • The headings and Abstract of the Disclosure provided herein are for convenience only and do not interpret the scope or meaning of the embodiments.
  • The various embodiments described herein provide systems, articles, and methods for human-electronics interfaces employing a generalized wearable EMG device that may be readily implemented in a wide range of applications. The human-electronics interfaces described herein employ a wearable EMG device that controls functions of another electronic device not by outputting “commands” as in the known proposals previously described, but by outputting generic gesture identification signals, or “flags,” that are not specific to the particular electronic device being controlled. In this way, the wearable EMG device may be used to control virtually any other electronic device if, for example, the other electronic device (or multiple other electronic devices) is (are) programmed with instructions for how to respond to the gesture identification flags.
  • Throughout this specification and the appended claims, the term “gesture identification flag” is used to refer to at least a portion of a data signal (e.g., a bit string) that is defined by and transmitted from a wearable EMG device in response to the wearable EMG device identifying that a user thereof has performed a particular gesture. The gesture identification flag may be received by a “receiving” electronic device, but the “gesture identification flag” portion of the data signal does not contain any information that is specific to the receiving electronic device. A gesture identification flag is a general, universal, and/or ambiguous signal that is substantially independent of the receiving electronic device (e.g., independent of any downstream processor-based device) and/or generic to a variety of applications run on any number of receiving electronic devices (e.g., generic to a variety of end user applications executable by one or more downstream processor-based device(s) useable with the wearable EMG device). A gesture identification flag may carry no more information than the definition/identity of the flag itself. For example, a set of three gesture identification flags may include a first flag simply defined as “A,” a second flag simply defined as “B,” and a third flag simply defined as “C.” Similarly, a set of four binary gesture identification flags may include a 00 flag, a 01 flag, a 10 flag, and a 11 flag. In accordance with the present systems, articles, and methods, a gesture identification flag may be defined and output by a wearable EMG device with little to no regard for the nature or functions of the receiving electronic device. The receiving electronic device may be programmed with specific instructions for how to interpret and/or respond to one or more gesture identification flag(s). As will be understood by a person of skill in the art, in some applications a gesture identification flag may be combined with authentication data, encryption data, device ID data (i.e., transmitting electronic device ID data and/or receiving electronic device ID data), pairing data, and/or any other data to enable and/or facilitate telecommunications between the wearable EMG device and the receiving electronic device in accordance with known telecommunications protocols (e.g., Bluetooth®). For greater certainty, throughout this specification and the appended claims, the term “gesture identification flag” refers to at least a portion of a data signal that is defined by a wearable EMG device based (at least in part) on EMG and/or accelerometer data and is substantially independent of the receiving electronic device. For the purposes of transmission, a gesture identification flag may be combined with other data that is at least partially dependent on the receiving electronic device. For example, a gesture identification flag may be a 2-bit component of an 8-bit data byte, where the remaining 6 bits are used for telecommunication purposes, as in: 00101101, where the exemplary first six bits “001011” may correspond to telecommunications information such as transmitting/receiving device IDs, encryption data, pairing data, and/or the like, and the exemplary last two bits “01” may correspond to a gesture identification flag. While a bit-length of two bits is used to represent a gesture identification flag in this example, in practice a gesture identification flag may comprise any number of bits (or other measure of signal length of a scheme not based on bits is employed).
  • FIG. 1 is a perspective view of an exemplary wearable EMG device 100 that may form part of a human-electronics interface in accordance with the present systems, articles, and methods. Exemplary device 100 is an armband designed to be worn on the wrist, forearm, or upper arm of a user, though a person of skill in the art will appreciate that the teachings described herein may readily be applied in wearable EMG devices designed to be worn elsewhere on the body of the user (such as on the finger, leg, ankle, neck, and/or torso of the user). Exemplary details that may be included in exemplary wearable EMG device 100 are described in at least U.S. Provisional Patent Application Ser. No. 61/752,226 (now U.S. Non-Provisional patent application Ser. No. 14/155,107), U.S. Provisional Patent Application Ser. No. 61/768,322 (now U.S. Non-Provisional patent application Ser. No. 14/186,889), and U.S. Provisional Patent Application Ser. No. 61/771,500 (now U.S. Non-Provisional patent application Ser. No. 14/194,252), each of which is incorporated herein by reference in its entirety.
  • Device 100 includes a set of eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 that form physically coupled links of the wearable EMG device 100. Each pod structure in the set of eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 is positioned adjacent and in between two other pod structures in the set of eight pod structures and the set of pod structures forms a perimeter of an annular or closed loop configuration. For example, pod structure 101 is positioned adjacent and in between pod structures 102 and 108 at least approximately on a perimeter of the annular or closed loop configuration of pod structures, pod structure 102 is positioned adjacent and in between pod structures 101 and 103 at least approximately on the perimeter of the annular or closed loop configuration, pod structure 103 is positioned adjacent and in between pod structures 102 and 104 at least approximately on the perimeter of the annular or closed loop configuration, and so on. Each of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 is physically coupled to the two adjacent pod structures by at least one adaptive coupler (not visible in FIG. 1). For example, pod structure 101 is physically coupled to pod structure 108 by an adaptive coupler and to pod structure 102 by an adaptive coupler. The term “adaptive coupler” is used throughout this specification and the appended claims to denote a system, article or device that provides flexible, adjustable, modifiable, extendable, extensible, or otherwise “adaptive” physical coupling. Adaptive coupling is physical coupling between two objects that permits limited motion of the two objects relative to one another. An example of an adaptive coupler is an elastic material such as an elastic band. Thus, each of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 in the set of eight pod structures may be adaptively physically coupled to the two adjacent pod structures by at least one elastic band. The set of eight pod structures may be physically bound in the annular or closed loop configuration by a single elastic band that couples over or through all pod structures or by multiple separate elastic bands that couple between adjacent pairs of pod structures or between groups of adjacent pairs of pod structures. Device 100 is depicted in FIG. 1 with the at least one adaptive coupler completely retracted and contained within the eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 (and therefore the at least one adaptive coupler is not visible in FIG. 1). Further details of adaptive coupling in wearable electronic devices are described in, for example, U.S. Provisional Application Ser. No. 61/860,063 (now U.S. Non-Provisional patent application Ser. No. 14/276,575), which is incorporated herein by reference in its entirety.
  • Throughout this specification and the appended claims, the term “pod structure” is used to refer to an individual link, segment, pod, section, structure, component, etc. of a wearable EMG device. For the purposes of the present systems, articles, and methods, an “individual link, segment, pod, section, structure, component, etc.” (i.e., a “pod structure”) of a wearable EMG device is characterized by its ability to be moved or displaced relative to another link, segment, pod, section, structure component, etc. of the wearable EMG device. For example, pod structures 101 and 102 of device 100 can each be moved or displaced relative to one another within the constraints imposed by the adaptive coupler providing adaptive physical coupling therebetween. The desire for pod structures 101 and 102 to be movable/displaceable relative to one another specifically arises because device 100 is a wearable EMG device that advantageously accommodates the movements of a user and/or different user forms.
  • Device 100 includes eight pod structures 101, 102, 103, 104, 105, 106, 107, and 108 that form physically coupled links thereof. Wearable EMG devices employing pod structures (e.g., device 100) are used herein as exemplary wearable EMG device designs, while the present systems, articles, and methods may be applied to wearable EMG devices that do not employ pod structures (or that employ any number of pod structures). Thus, throughout this specification, descriptions relating to pod structures (e.g., functions and/or components of pod structures) should be interpreted as being applicable to any wearable EMG device design, even wearable EMG device designs that do not employ pod structures (except in cases where a pod structure is specifically recited in a claim).
  • In exemplary device 100 of FIG. 1, each of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 comprises a respective housing having a respective inner volume. Each housing may be formed of substantially rigid material and may be optically opaque. Throughout this specification and the appended claims, the term “rigid” as in, for example, “substantially rigid material,” is used to describe a material that has an inherent tendency to maintain its shape and resist malformation/deformation under the moderate stresses and strains typically encountered by a wearable electronic device.
  • Details of the components contained within the housings (i.e., within the inner volumes of the housings) of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 are not visible in FIG. 1. To facilitate descriptions of exemplary device 100, some internal components are depicted by dashed lines in FIG. 1 to indicate that these components are contained in the inner volume(s) of housings and may not normally be actually visible in the view depicted in FIG. 1, unless a transparent or translucent material is employed to form the housings. For example, any or all of pod structures 101, 102, 103, 104, 105, 106, 107, and/or 108 may include electric circuitry. In FIG. 1, a first pod structure 101 is shown containing electric circuitry 111 (i.e., electric circuitry 111 is contained in the inner volume of the housing of pod structure 101), a second pod structure 102 is shown containing electric circuitry 112, and a third pod structure 108 is shown containing electric circuitry 118. The electric circuitry in any or all pod structures may be communicatively coupled to the electric circuitry in at least one other pod structure by at least one respective communicative pathway (e.g., by at least one electrically conductive pathway and/or by at least one optical pathway). For example, FIG. 1 shows a first set of communicative pathways 121 providing communicative coupling between electric circuitry 118 of pod structure 108 and electric circuitry 111 of pod structure 101, and a second set of communicative pathways 122 providing communicative coupling between electric circuitry 111 of pod structure 101 and electric circuitry 112 of pod structure 102. Communicative coupling between electric circuitries of pod structures in device 100 may advantageously include systems, articles, and methods for signal routing as described in U.S. Provisional Patent Application Ser. No. 61/866,960 (now U.S. Non-Provisional patent application Ser. No. 14/461,044) and/or systems, articles, and methods for strain mitigation as described in U.S. Provisional Patent Application Ser. No. 61/857,105 (now U.S. Non-Provisional patent application Ser. No. 14/335,668), both of which are incorporated by reference herein in their entirety.
  • Throughout this specification and the appended claims the term “communicative” as in “communicative pathway,” “communicative coupling,” and in variants such as “communicatively coupled,” is generally used to refer to an engineered arrangement for transferring and/or exchanging information. Exemplary communicative pathways include, but are not limited to, electrically conductive pathways (e.g., electrically conductive wires, electrically conductive traces), magnetic pathways (e.g., magnetic media), and/or optical pathways (e.g., optical fiber), and exemplary communicative couplings include, but are not limited to, electrical couplings and/or optical couplings.
  • Each individual pod structure within a wearable EMG device may perform a particular function, or particular functions. For example, in device 100, each of pod structures 101, 102, 103, 104, 105, 106, and 107 includes a respective EMG sensor 110 (only one called out in FIG. 1 to reduce clutter) to in use detect muscle activity of a user and to in use provide electrical signals in response to the detected muscle activity. Thus, each of pod structures 101, 102, 103, 104, 105, 106, and 107 may be referred to as a respective “sensor pod.” Throughout this specification and the appended claims, the term “sensor pod” is used to denote an individual pod structure that includes at least one sensor to detect muscle activity of a user. Each EMG sensor may be, for example, a respective capacitive EMG sensor that detects electrical signals generated by muscle activity through capacitive coupling, such as for example the capacitive EMG sensors described in U.S. Provisional Patent Application Ser. No. 61/771,500 (now U.S. Non-Provisional patent application Ser. No. 14/194,252).
  • Pod structure 108 of device 100 includes a processor 140 that in use processes the signals provided by the EMG sensors 110 of sensor pods 101, 102, 103, 104, 105, 106, and 107 in response to detected muscle activity. Pod structure 108 may therefore be referred to as a “processor pod.” Throughout this specification and the appended claims, the term “processor pod” is used to denote an individual pod structure that includes at least one processor to process signals. The processor may be any type of processor, including but not limited to: a digital microprocessor or microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a graphics processing unit (GPU), a programmable gate array (PGA), a programmable logic unit (PLU), or the like, that in use analyzes the signals to determine at least one output, action, or function based on the signals.
  • As used throughout this specification and the appended claims, the terms “sensor pod” and “processor pod” are not necessarily exclusive. A single pod structure may satisfy the definitions of both a “sensor pod” and a “processor pod” and may be referred to as either type of pod structure. For greater clarity, the term “sensor pod” is used to refer to any pod structure that includes a sensor and performs at least the function(s) of a sensor pod, and the term processor pod is used to refer to any pod structure that includes a processor and performs at least the function(s) of a processor pod. In device 100, processor pod 108 includes an EMG sensor 110 (not visible in FIG. 1) to sense, measure, transduce or otherwise detect muscle activity of a user, so processor pod 108 could be referred to as a sensor pod. However, in exemplary device 100, processor pod 108 is the only pod structure that includes a processor 140, thus processor pod 108 is the only pod structure in exemplary device 100 that can be referred to as a processor pod. In alternative embodiments of device 100, multiple pod structures may include processors, and thus multiple pod structures may serve as processor pods. Similarly, some pod structures may not include sensors, and/or some sensors and/or processors may be laid out in other configurations that do not involve pod structures.
  • Processor 140 includes and/or is communicatively coupled to a non-transitory processor-readable storage medium or memory 141. As will be described in more detail later, memory 141 may store, for example, a set of gesture identification flags to be transmitted by device 100 and/or, for example, processor-executable instructions to be executed by processor 140. For transmitting gesture identification flags, a wearable EMG device may include at least one output terminal communicatively coupled to processor 140. Throughout this specification and the appended claims, the term “terminal” is generally used to refer to any physical structure that provides a telecommunications link through which a data signal may enter and/or leave a device. The term “output terminal” is used to describe a terminal that provides at least a signal output link and the term “input terminal” is used to describe a terminal that provides at least a signal input link; however unless the specific context requires otherwise, an output terminal may also provide the functionality of an input terminal and an input terminal may also provide the functionality of an output terminal. In general, a “communication terminal” represents the end (or “terminus”) of communicative signal transfer within a device and the beginning of communicative signal transfer to/from an external device (or external devices). As examples, communication terminal 151 of device 100 may include a wireless transmitter that implements a known wireless communication protocol, such as Bluetooth®, WiFi®, or Zigbee®, while communication terminal 152 may include a tethered communication port such as Universal Serial Bus (USB) port, a micro-USB port, a Thunderbolt® port, and/or the like.
  • For some applications, device 100 may also include at least one accelerometer 160 (e.g., an inertial measurement unit, or “IMU,” that includes at least one accelerometer and/or at least one gyroscope) communicatively coupled to processor 140. In use, the at least one accelerometer may detect, sense, and/or measure motion effected by a user and provide signals in response to the detected motion. As will be described in more detail later, signals provided by accelerometer 160 may be processed together with signals provided by EMG sensors 110 by processor 140.
  • Throughout this specification and the appended claims, the term “accelerometer” is used as a general example of an inertial sensor and is not intended to limit (nor exclude) the scope of any description or implementation to “linear acceleration.”
  • Throughout this specification and the appended claims, the term “provide” and variants such as “provided” and “providing” are frequently used in the context of signals. For example, an EMG sensor is described as “providing at least one signal” and an accelerometer is described as “providing at least one signal.” Unless the specific context requires otherwise, the term “provide” is used in a most general sense to cover any form of providing a signal, including but not limited to: relaying a signal, outputting a signal, generating a signal, routing a signal, creating a signal, transducing a signal, and so on. For example, a capacitive EMG sensor may include at least one electrode that capacitively couples to electrical signals from muscle activity. This capacitive coupling induces a change in a charge or electrical potential of the at least one electrode which is then relayed through the sensor circuitry and output, or “provided,” by the sensor. Thus, the capacitive EMG sensor may “provide” an electrical signal by relaying an electrical signal from a muscle (or muscles) to an output (or outputs). In contrast, an accelerometer may include components (e.g., piezoelectric, piezoresistive, capacitive, etc.) that are used to convert physical motion into electrical signals. The accelerometer may “provide” an electrical signal by detecting motion and generating an electrical signal in response to the motion.
  • As previously described, each of pod structures 101, 102, 103, 104, 105, 106, 107, and 108 may include electric circuitry. FIG. 1 depicts electric circuitry 111 inside the inner volume of sensor pod 101, electric circuitry 112 inside the inner volume of sensor pod 102, and electric circuitry 118 inside the inner volume of processor pod 118. The electric circuitry in any or all of pod structures 101, 102, 103, 104, 105, 106, 107 and 108 (including electric circuitries 121, 122, and 128) may include any or all of: an amplification circuit to in use amplify electrical signals provided by at least one EMG sensor 110, a filtering circuit to in use remove unwanted signal frequencies from the signals provided by at least one EMG sensor 110, and/or an analog-to-digital conversion circuit to in use convert analog signals into digital signals. Device 100 may also include a battery (not shown in FIG. 1) to in use provide a portable power source for device 100.
  • Signals that are provided by EMG sensors 110 in device 100 are routed to processor pod 108 for processing by processor 140. To this end, device 100 employs a set of communicative pathways (e.g., 121 and 122) to route the signals that are provided by sensor pods 101, 102, 103, 104, 105, 106, and 107 to processor pod 108. Each respective pod structure 101, 102, 103, 104, 105, 106, 107, and 108 in device 100 is communicatively coupled to at least one other pod structure by at least one respective communicative pathway from the set of communicative pathways. Each communicative pathway (e.g., 121 and 122) may be realized in any communicative form, including but not limited to: electrically conductive wires or cables, ribbon cables, fiber-optic cables, optical/photonic waveguides, electrically conductive traces carried by a rigid printed circuit board, and/or electrically conductive traces carried by a flexible printed circuit board.
  • The present systems, articles, and methods describe a human-electronics interface in which a wearable EMG device (e.g., device 100) is used to control another electronic device. The human-electronics interface may be characterized as a system that enables electromyographic control of an electronic device.
  • FIG. 2 is an illustrative diagram of a system 200 that enables electromyographic control of an electronic device in accordance with the present systems, articles, and methods. System 200 includes a wearable EMG device 270 and an unspecified electronic device 280. Wearable EMG device 270 may be, as an illustrative example, substantially similar to wearable EMG device 100 from FIG. 1. That is, exemplary wearable EMG device 270 includes a set of pod structures 201 (only one called out in FIG. 2 to reduce clutter) that form physically coupled links of device 270, where each pod structure 201 includes a respective EMG sensor 210 (e.g., a respective capacitive EMG sensor) to in use sense, measure, transduce or otherwise detect muscle activity of a user and provide electrical signals in response to the muscle activity. As previously described, however, the present systems, articles, and methods may be implemented using wearable EMG devices that do not employ pod structures.
  • Each pod structure 201 is electrically coupled to at least one adjacent pod structure by at least one respective communicative pathway 220 to route signals in between pod structures (e.g., to route signals from sensor pods to a processor pod). Each pod structure 201 is also physically coupled to two adjacent pod structures 201 by at least one adaptive coupler 260 and the set of pod structures forms a perimeter of an annular or closed loop configuration. FIG. 2 shows device 270 in an expanded annular or closed loop configuration adapted to fit the arm of a larger user than the contracted annular or closed loop configuration of device 100 from FIG. 1. As a result, adaptive couplers 260 (only one called out in FIG. 2) providing adaptive physical coupling between adjacent pairs of pod structures 201 are visible in FIG. 2, whereas such adaptive couplers 260 are not visible in FIG. 1.
  • Each pod structure 201 includes respective electric circuitry 230 and at least one electric circuitry 230 includes a first processor 240 (e.g., akin to processor 140 in device 100 of FIG. 1). At least one electric circuitry 230 may include an IMU and/or at least one accelerometer. Device 270 also includes an output terminal 250 to in use interface with unspecified electronic device 280. For example, device 270 is operative to in use send gesture identification flags to unspecified electronic device 280 through output terminal 250.
  • Unspecified electronic device 280 may be any electronic device, including but not limited to: a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone, a portable electronic device, an audio player, a television, a video player, a video game console, a robot, a light switch, and/or a vehicle. Electronic device 280 is denominated as “unspecified” herein to emphasize the fact that the gesture identification flags output by wearable EMG device 270 are generic to a variety of electronic devices and/or applications executed by the electronic devices. The electronic device 280, its operating characteristics and/or the operating characteristics of applications executed by the electronic device 280 may not be a priori known by the EMG device 270 during use, or even prior to use when a mapping between signals, gesture flags, and/or gestures is initially defined or established. As previously described, a data signal output by device 270 through output terminal 250 may include a gesture identification flag as a first portion thereof and may also include at least a second portion to implement known telecommunications protocols (e.g., Bluetooth®). Thus, electronic device 280 may remain “unspecified” with respect to the gesture identification flag portion(s) of signals output by EMG device 270 but electronic device 280 may be “specified” by the telecommunications portion(s) of signals output by EMG device 270 (if such specification is necessary for signal transfer, e.g., to communicatively “pair” device 270 and device 280 if required by the telecommunications protocol being implemented). For example, electronic device 280 may be and remain “unspecified” while muscle activity is detected by EMG device 270 and while the processor in EMG device 270 determines a gesture identification flag based, at least in part, on the detected muscle activity. After a gesture identification flag is determined by the processor in EMG device 270, electronic device 280 may become “specified” when the gesture identification flag is combined with telecommunication data and transmitted to electronic device 280. In this scenario, the gesture identification flag itself does not include any information that is specific to electronic device 280 and therefore electronic device 280 is “unspecified” in relation to the gesture identification flag.
  • Electronic device 280 includes an input terminal 281 to in use interface with wearable EMG device 270. For example, device 280 may receive gesture identification flags from device 270 through input terminal 281. Device 280 also includes a second processor 283 to in use process gesture identification flags received from device 270. Second processor 283 may include or be communicatively coupled to a non-transitory processor-readable storage medium or memory 284 that stores processor-executable instructions to be executed by second processor 283.
  • Wearable EMG device 270 and electronic device 280 are, in use, communicatively coupled by communicative link 290. More specifically, output terminal 250 of wearable EMG device 270 is, in use, communicatively coupled to input terminal 281 of electronic device 280 by communicative link 290. Communicative link 290 may be used to route gesture identification flags from wearable EMG device 270 to electronic device 280. Communicative link 290 may be established in variety of different ways. For example, output terminal 250 of wearable EMG device 270 may include a first tethered connector port (e.g., a USB port, or the like), input terminal 281 of electronic device 280 may include a second tethered connector port, and communicative link 290 may be established through a communicative pathway (e.g., an electrical or optical cable, wire, circuit board, or the like) that communicatively couples the first connector port to the second connector port to route gesture identification flags from output terminal 250 to input terminal 281. Alternatively, output terminal 250 of wearable EMG device 270 may include a wireless transmitter and communicative link 290 may be representative of wireless communication between wearable EMG device 270 and electronic device 280. In this case, input terminal 281 of electronic device 280 may include a wireless receiver to in use wirelessly receive gesture identification flags from the wireless transmitter of wearable EMG device 270 (using, for example, established wireless telecommunication protocols, such as Bluetooth®); or, input terminal 281 may be communicatively coupled to a wireless receiver 282 (such as a USB dongle communicatively coupled to a tethered connector port of input terminal 281) to in use wirelessly receive gesture identification flags from the wireless transmitter of wearable EMG device 270.
  • As previously described, known proposals for human-electronics interfaces that employ a wearable EMG device are limited in their versatility because they involve mapping gestures to functions on-board the wearable EMG device itself. Thus, in known proposals, the wearable EMG device outputs control signals (i.e., “commands”) that embody pre-defined instructions to effect pre-defined functions that are specific to a pre-defined receiving device. If a user wishes to use such a wearable EMG device for a different purpose (i.e., to control a different receiving device, or a different application within the same receiving device), then the definitions of the commands themselves must be re-programmed within the wearable EMG device. Conversely, the various embodiments described herein provide systems, articles, and methods for human-electronics interfaces that employ a wearable EMG device that controls functions of another electronic device by outputting generic gesture identification flags that are not specific to the particular electronic device being controlled. The electronic device being controlled may include or may access an Application Programming Interface (i.e., an “API” including instructions and/or data or information (e.g., library) stored in a non-transitory processor-readable storage medium or memory) through which a user may define how gesture identification flags are to be interpreted by the electronic device being controlled (i.e., where the user may define how the electronic device responds to gesture identification flags). The present systems, articles, and methods greatly enhance the versatility of human-electronics interfaces by employing a wearable EMG device that outputs the same gesture identification flags regardless of what it is being used to control, and may therefore be used to control virtually any electronic receiving device. The functions or operations that are controlled by the wearable EMG devices described herein are defined within the receiving device (or within the applications within the receiving device) rather than within the wearable EMG device.
  • FIG. 3 is a flow-diagram showing a method 300 of operating a wearable EMG device to provide electromyographic control of an electronic device in accordance with the present systems, articles, and methods. The electronic device may be any “unspecified” electronic device as described previously. For example, the electronic device may be any downstream processor-based device. The wearable EMG device may include at least one EMG sensor, a processor, and an output terminal (i.e., the wearable EMG device may be substantially similar to wearable EMG device 100 from FIG. 1 and wearable EMG device 270 from FIG. 2). Method 300 includes four acts 301, 302, 303, and 304, though those of skill in the art will appreciate that in alternative embodiments certain acts may be omitted and/or additional acts may be added. Those of skill in the art will also appreciate that the illustrated order of the acts is shown for exemplary purposes only and may change in alternative embodiments.
  • At 301, muscle activity of a user (i.e., a wearer of the wearable EMG device) is sensed, measured, transduced or otherwise detected by at least one EMG sensor of the wearable EMG device. As previously described, the at least one EMG sensor may be, for example, a capacitive EMG sensor and sensing, measuring, transducing or otherwise detecting muscle activity of the user may include, for example, capacitively coupling to electrical signals generated by muscle activity of the user.
  • At 302, at least one signal is provided from the at least one EMG sensor to the processor of the wearable EMG device in response to the sensed, measured, transduced or otherwise detected muscle activity. The at least one signal may be an analog signal that is amplified, filtered, and converted to digital form by electric circuitry within the wearable EMG device. Providing the at least one signal from the at least one EMG sensor to the processor may include routing the at least one signal to the processor through one or more communicative pathway(s) as described previously.
  • At 303, a gesture identification flag is determined by the processor of the wearable EMG device, based at least in part on the at least one signal provided from the at least one EMG sensor to the processor. The gesture identification flag is substantially independent of the downstream electronic device. As will be described in more detail later (e.g., with reference to FIG. 5), determining a gesture identification flag by the processor may implement a range of different algorithms, including but not limited to: a look-up table, a mapping, a machine learning algorithm, a pattern recognition algorithm, and the like. In some applications, the wearable EMG device may include a non-transitory processor-readable medium that stores a set of gesture identification flags and/or stores processor-executable instructions that, when executed by the processor of the wearable EMG device, cause the processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor. In such a case, act 303 may include executing the processor-executable instructions by the processor to cause the processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor.
  • At 304, the gesture identification flag is transmitted to the electronic device by the output terminal of the wearable EMG device. As previously described, the output terminal of the wearable EMG device may include a wireless transmitter, and transmitting the gesture identification flag to the electronic device may include wirelessly transmitting the gesture identification flag to the electronic device by the wireless transmitter.
  • As an example, the at least one EMG sensor may include a first EMG sensor and at least a second EMG sensor, and muscle activity of the user may be sensed, measured, transduced or otherwise detected by the first EMG sensor and by at least the second EMG sensor (at 301). In this case at least a first signal is provided from the first EMG sensor to the processor of the wearable EMG device in response to the detected muscle activity (at 302) and at least a second signal is provided from at least the second EMG sensor to the processor of the wearable EMG device in response to the detected muscle activity (at 302). The processer of the wearable EMG device may then determine (at 303) a gesture identification flag based at least in part on both the at least a first signal provided from the first EMG sensor to the processor and the at least a second signal provided from at least the second EMG sensor to the processor.
  • As previously described, in some applications it may be advantageous to combine or otherwise make use of both EMG signals and motion signals sensed, measured or otherwise detected, for example, by an accelerometer. To this end, the wearable EMG device may include at least one accelerometer, and an additional method employing further acts may be combined with acts 301-304 of method 300 to detect and process motion signals.
  • FIG. 4 is a flow-diagram showing a method 400 of operating a wearable EMG device to provide both electromyographic and motion control of an electronic device in accordance with the present systems, articles, and methods. Method 400 includes three acts 401, 402, and 403, though those of skill in the art will appreciate that in alternative embodiments certain acts may be omitted and/or additional acts may be added. Those of skill in the art will also appreciate that the illustrated order of the acts is shown for exemplary purposes only and may change in alternative embodiments. Method 400 is optionally performed in conjunction with method 300 from FIG. 3 and, if performed, performed using the same wearable EMG device as that used to perform method 300. For example, while acts 301 and 302 of method 300 are performed by EMG sensors of the wearable EMG device, acts 401 and 402 of method 400 may optionally be performed by at least one accelerometer of the wearable EMG device.
  • At 401, motion effected by the user of the wearable EMG device is sensed, measured, transduced or otherwise detected by at least one accelerometer in the wearable EMG device. The at least once accelerometer may be part of an IMU that includes multiple accelerometers (such as an MPU-9150 Nine-Axis MEMS MotionTracking™ Device from InvenSense). The motion effected by the user that may be detected and/or measured may include, e.g., translation in one or multiple spatial directions and/or rotation about one or more axes in one or more spatial directions. The motion(s) may be detected in terms of a presence or absence of translation and/or rotation, and/or measured in terms of a speed of translation and/or rotation and/or acceleration of translation and/or rotation.
  • At 402, at least one signal is provided from the at least one accelerometer to the processor in response to the sensed, measured, transduced or otherwise detected motion. The at least one signal may be an analog signal that is amplified, filtered, and converted to digital form by electric circuitry within the wearable EMG device. The at least one signal may be routed to the processor in the wearable EMG device via one or more communicative pathway(s) as described previously.
  • As previously described, act 303 of method 300 involves determining, by a processor of the wearable EMG device, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor in response to detected muscle activity. In applications where the wearable EMG device further includes at least one accelerometer and acts 401 and 402 of method 400 are performed, act 303 of method 300 may be replaced by act 403 of method 400.
  • At 403, a gesture identification flag is determined by the processor, based at least in part on the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor. The wearable EMG device may include a non-transitory processor-readable medium (e.g., memory 284 of device 280 from FIG. 2) that stores processor-executable instructions that, when executed by the processor, cause the processor to determine a gesture identification flag based on the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor (i.e., to perform act 403). Thus, act 403 may include executing the processor-executable instructions stored in the non-transitory processor-readable medium.
  • In some implementations, the at least one signal provided from the at least one accelerometer to the processor (i.e., at act 402) may be combined with at least one signal provided from at least one EMG sensor to the processor (i.e., at act 302 of method 300 from FIG. 3) by the processor of the wearable EMG device. Thus, act 403 requires that acts 401 and 402 from method 400 and acts 301 and 302 from method 300 all be completed. The at least one signal from the at least one accelerometer and the at least one signal from the at least one EMG sensor may be summed, concatenated, overlaid, or otherwise combined in any way by the processor to produce, provide or output any number of signals, operations, and/or results.
  • After act 403, the gesture identification flag may be transmitted or output by an output terminal of the wearable EMG device (i.e., according to act 304 of method 300) to any downstream electronic device and interpreted or otherwise processed by the downstream electronic device to cause the downstream electronic device to perform some function(s) or operation(s), or otherwise effect an interaction with or response from the downstream electronic device, in response to the gesture identification flag.
  • In accordance with the present systems, articles, and methods, at least one signal provided by at least one EMG sensor (either alone or together with one or more signals provided by one or more transducers such as an accelerometer or other motion or acceleration responsive transducers) may represent or be indicative of a gesture performed by a user of a wearable EMG device. Determining a gesture identification flag corresponding to that at least one signal may involve identifying, by a processor, the gesture performed by the user based at least in part on the at least one signal(s) from the EMG and/or other sensors or transducers, and determining, by the processor, a gesture identification flag that corresponds to that determined gesture. Unless the specific context requires otherwise, throughout this specification and the appended claims “a” gesture identification flag should be interpreted in a general, inclusive sense as “at least one” gesture identification flag with the understanding that determining any number of gesture identification flags (e.g., determining one gesture identification flag, or determining multiple gesture identification flags) includes determining “a” gesture identification flag. Each gesture identification flag may include, or be represented by, one or more bits of information. Furthermore, “determining” a gesture identification flag by a processor may be achieved through a wide variety of different techniques. For example, a processor may determine a gesture identification flag by performing or otherwise effecting a mapping between gestures (e.g., between EMG and/or accelerometer signals representative of gestures) and gesture identification flags (e.g., by invoking a stored look-up table or other form of stored processor-executable instructions providing and/or effecting mappings between gestures and gesture identification flags), or a processor may determine a gesture identification flag by performing an algorithm or sequence of data processing acts (e.g., by executing stored processor-executable instructions dictating how to determine a gesture identification flag based at least in part on one or more signal(s) provided by at least one EMG sensor and/or at least one accelerometer).
  • FIG. 5 is a schematic illustration showing an exemplary mapping 500 between a set of exemplary gestures and a set of exemplary gesture identification flags in accordance with the present systems, articles, and methods. Mapping 500 may be representative of processor-executable instructions that are defined in advance of determining gesture identification flags based at least in part on at least one EMG signal (and, e.g., executed by a processor to perform the act of determining gesture identification flags based at least in part on at least one EMG signal), or mapping 500 may be representative of the results (i.e., the mapping that is effected) when gesture identification flags are determined based at least in part on at least one EMG signal. In other words, mapping 500 characterizes: i) a prescription, embodied in processor-executable instructions, for or definition of how gestures (e.g., EMG and/or accelerometer signals that are representative of gestures) are to be mapped to gesture identification flags by a processor when determining a gesture identification flag based at least in part on at least one signal provided from at least one EMG sensor to the processor; or ii) the end results when a processor performs an algorithm or series of data processing steps to determine a gesture identification flag based at least in part on at least one signal provided from at least one EMG sensor to the processor. In the former characterization (i.e., characterization i)), mapping 500 may be stored as a look-up table or set of defined processor-executable “mapping instructions” in a non-transitory processor-readable storage medium and invoked/executed by the processor when determining a gesture identification flag. In the latter characterization (i.e., characterization ii)), mapping 500 may not be stored in a non-transitory processor-readable storage medium itself, but instead processor-executable instructions to perform an algorithm or series of data processing acts may be stored in the non-transitory processor-readable storage medium and mapping 500 may represent the results of executing the stored processor-executable instructions by the processor when determining a gesture identification flag. In either case, the present systems, articles, and methods provide a framework in which a wearable EMG device is programmed with processor-executable instructions that embody (i.e., in accordance with characterization i)) and/or produce/effect (i.e., in accordance with characterization ii)) a mapping from gestures to gesture identification flags, such as exemplary mapping 500 from FIG. 5.
  • As shown in mapping 500, each gesture identification flag may, for example, comprise a bit string (e.g., an 8-bit data byte as illustrated) that uniquely maps to a corresponding gesture performed by a user. For example, a “gun” or “point” hand gesture may correspond/map to gesture identification flag 00000001 as illustrated, a “thumbs up” gesture may correspond/map to gesture identification flag 00000010 as illustrated, a “fist” gesture may correspond/map to gesture identification flag 00000011 as illustrated, and a “rock on” gesture may correspond/map to gesture identification flag 00000100 as illustrated. A person of skill in the art will appreciate that an 8-bit data byte can be used to represent 256 unique gesture identification flags (corresponding to 256 unique gestures). In practice, gesture identification flags having any number of bits may be used, and if desired, multiple gestures may map to the same gesture identification flag and/or the same gesture may map to multiple gesture identification flags. In accordance with the present systems, articles, and methods, a gesture identification flag contains only information that identifies (i.e., maps to) a gesture performed by a user of a wearable EMG device. A gesture identification flag does not contain any information about a function or operation that the corresponding gesture may be used to control. A gesture identification flag does not contain any information about any downstream electronic device and/or application that the corresponding gesture may be used to control. A gesture identification flag may be appended, adjoined, supplemented, or otherwise combined with additional data bits as needed for, e.g., the purposes of telecommunications.
  • Mapping 500 represents gestures with actual illustrations of hands solely for ease of illustration and description. In practice, a gesture may be represented by any corresponding configuration of signals provided by at least one EMG sensor and/or at least one accelerometer. For example, a gesture may be represented by a particular signal waveform, a particular signal value, or a particular configuration/arrangement/permutation/combination of signal waveforms/values.
  • The present systems, articles, and methods describe human-electronics interfaces. Methods 300 and 400 provide methods of operating a wearable EMG device to control an unspecified electronic device (e.g., methods of operating device 100 from FIG. 1 or device 270 from FIG. 2). A complete human-electronics interface may involve acts performed by both the controller and the receiver (e.g., methods of operating system 200 from FIG. 2).
  • FIG. 6 is a flow-diagram showing a method 600 of electromyographically controlling at least one function of an electronic device by a wearable EMG device in accordance with the present systems, articles, and methods. The wearable EMG device includes at least one EMG sensor, a first processor, and an output terminal (with the at least one EMG sensor and the output terminal each communicatively coupled to the first processor) and the electronic device includes an input terminal and a second processor (with the input terminal communicatively coupled to the second processor). Method 600 includes seven acts 601, 602, 603, 604, 611, 612, and 613, though those of skill in the art will appreciate that in alternative embodiments certain acts may be omitted and/or additional acts may be added. Those of skill in the art will also appreciate that the illustrated order of the acts is shown for exemplary purposes only and may change in alternative embodiments. Acts 601, 602, 603, and 604 are performed by the wearable EMG device to produce and transmit signals and acts 611, 612, and 613 are performed by the electronic device to receive and respond to the transmitted signals.
  • Acts 601, 602, 603, and 604 are substantially similar to acts 301, 302, 303, and 304 (respectively) of method 300 from FIG. 3. At 601, muscle activity of a user is sensed, measured, transduced or otherwise detected by at least one EMG sensor of the wearable EMG device. At 602, at least one signal is provided from the at least one EMG sensor to a first processor on-board the wearable EMG device in response to the detected muscle activity. At 603, the first processor determines a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor. At 604, the gesture identification flag is transmitted by the output terminal of the wearable EMG device. In some applications, the wearable EMG device may include at least one accelerometer and the wearable EMG device may be used to perform method 400 from FIG. 4. Therefore, act 603 may comprise determining a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the first processor and the at least one signal provided from the at least one accelerometer to the first processor.
  • At 611, the gesture identification flag that is transmitted or output by the output terminal of the wearable EMG device at 604 is received by the input terminal of the electronic device. As previously described, transmission of gesture identification flags between the wearable EMG device and the electronic device may be through a wired or wireless communicative link (e.g. communicative link 290 from FIG. 2).
  • At 612, a second processor on-board the electronic device determines a function of the electronic device based at least in part on the gesture identification flag received by the input terminal of the electronic device at 611. As described previously, the electronic device may include a non-transitory processor-readable storage medium or memory that stores an API or other information or data structures (e.g., implemented as one or library(ies)) through which a user may define mappings (i.e., processor-executable instructions that embody and/or produce/effect mappings) between gesture identification flags and functions of the electronic device, and/or the non-transitory processor-readable storage medium may store processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
  • At 613, the function determined at 612 is performed by the electronic device. The function may be any function or operation of the electronic device. For example, if the electronic device is an audio and/or video player (or a computer running an application that performs audio and/or video playback), then the corresponding function may be a PLAY function that causes the audio/video to play, a STOP function that causes the audio/video to stop, a REWIND function that causes the audio/video to rewind, a FAST FORWARD function that causes the audio/video to fast forward, and so on.
  • Throughout this specification and the appended claims, reference is often made to “determining a function of an electronic device based at least in part on a gesture identification flag.” Unless the specific context requires otherwise, throughout this specification and the appended claims “a” function should be interpreted in a general, inclusive sense as “at least one” function with the understanding that determining any number of functions (e.g., determining one function, or determining multiple functions) includes determining “a” function. Furthermore, “determining” a function by a processor may be achieved through a wide variety of different techniques. For example, a processor may determine a function by employing a defined mapping between gesture identification flags and functions (e.g., by invoking a stored look-up table or other form of stored processor-executable instructions providing defined mappings between gesture identification flags and functions), or a processor may determine a function by performing an algorithm or sequence of data processing steps (e.g., by executing stored processor-executable instructions dictating how to determine a function based at least in part on one or more gesture identification flag(s)).
  • FIG. 7 is a schematic illustration showing an exemplary mapping 700 between a set of exemplary gesture identification flags and a set of exemplary functions of an electronic device in accordance with the present systems, articles, and methods. Similar to mapping 500 from FIG. 5, mapping 700 may be characterized as: i) a prescription for how gesture identification flags are to be mapped to functions by a processor when determining a function based at least in part on a gesture identification flag received from a wearable EMG device; or ii) the end results when a processor performs an algorithm or series of data processing acts to determine a function based at least in part on a gesture identification flag received from a wearable EMG device. In the former characterization (i.e., characterization i)), mapping 700 may be stored as a look-up table or set of defined processor-executable “mapping instructions” in a non-transitory processor-readable storage medium and invoked by the processor when determining a function of the electronic device. In the latter characterization (i.e., characterization ii)), mapping 700 may not be stored in a non-transitory processor-readable storage medium itself, but instead processor-executable instructions to perform an algorithm or series of data processing acts may be stored in the non-transitory processor-readable storage medium and mapping 700 may represent the results of executing the stored processor-executable instructions by the processor to determine a function of the electronic device. In either case, the present systems, articles, and methods provide a framework in which generic gesture identification flags are output by a wearable EMG device and a receiving device is programmed (and/or programmable through, e.g., an API or other information or data or calls) with processor-executable instructions that embody and/or produce/effect a mapping from gesture identification flags to functions of the electronic device, such as exemplary mapping 700 from FIG. 7.
  • For the illustrative example of FIG. 7, the electronic device is an audio player; however, any electronic device may include or be communicatively coupled to (or be adapted to include or be communicatively coupled to) a non-transitory processor-readable storage medium or memory that stores processor-executable instructions that embody and/or produce/effect a mapping such as mapping 700. As shown in mapping 700, each gesture identification flag may, for example, be a bit string (e.g., an 8-bit data byte as illustrated) that uniquely maps to a corresponding function of the electronic device. For example, a 00000001 gesture identification flag may map/correspond to a REWIND function of an audio player as illustrated, a 00000010 gesture identification flag may map/correspond to a PLAY function of an audio player as illustrated, a 00000011 gesture identification flag may map/correspond to a STOP function of an audio player as illustrated, and a 00000100 gesture identification flag may map/correspond to a FAST FORWARD function of an audio player as illustrated. A person of skill in the art will appreciate that an 8-bit data byte can be used to represent 256 unique gesture identification flags (corresponding to 256 unique functions). In practice, gesture identification flags having any number of bits may be used, multiple gesture identification flags may be mapped to the same function, and/or a single gesture identification flag may map to multiple functions.
  • In accordance with the present systems, articles, and methods, processor-executable instructions that embody and/or produce/effect a mapping from gestures to gesture identification flags (e.g., mapping 500 from FIG. 5) may be stored in a non-transitory processor-readable storage medium or memory on-board a wearable EMG device (e.g., memory 141 of device 100 from FIG. 1) and communicatively coupled to a first processor (e.g., processor 140 of device 100), and processor-executable instructions that embody and/or produce/effect a mapping from gesture identification flags to functions of an electronic device (e.g., mapping 700 from FIG. 7) may be stored in a non-transitory processor-readable storage medium or memory on-board an electronic device (e.g., memory 282 of device 280 from FIG. 2) and communicatively coupled to a second processor (e.g., processor 283 of device 280 in FIG. 2). In this way, gesture identification flags may be determined by the first processor on-board the wearable EMG device based on signals from one or more sensor(s) (e.g., EMG sensors and/or inertial sensors) in accordance with, e.g., mapping 500 of FIG. 5; the gesture identification flags may be transmitted or output to a receiving device; and then functions of the receiving device may be determined by the second processor on-board the receiving device based on the gesture identification flags in accordance with, e.g., mapping 700 from FIG. 7. For example, signals corresponding to a “gun” or “point” gesture (e.g., outwardly extended index finger with other fingers curled upon themselves) may be processed by the first processor of the wearable EMG device to determine gesture identification flag 00000001 according to mapping 500 from FIG. 5, the 00000001 flag may be transmitted to the electronic device (through a wired or wireless communicative link), and the 00000001 flag may be processed by the second processor of the electronic device to determine a REWIND function in accordance with mapping 700.
  • In accordance with the present systems, articles, and methods, an electronic device may store multiple mappings (e.g., multiple sets of processor-executable instructions that embody and/or produce/effect mappings) between gesture identification flags and functions of the electronic device, and when the electronic device receives a gesture identification flag it may perform a corresponding function based on the implementation of one of the multiple stored mappings (e.g., one or more of the multiple sets of processor-executable instructions). For example, the electronic device may be a computer such as a desktop computer, a laptop computer, a tablet computer, or the like. The computer may include a non-transitory processor-readable storage medium or memory that stores multiple mappings (e.g., multiple sets of processor-executable instructions that embody and/or produce/effect mappings) between gesture identification flags and functions of the computer (e.g., multiple variants of mapping 700 from FIG. 7), with each mapping corresponding to and invoked by a different application executed by the computer. For example, the non-transitory processor-readable storage medium may store a first mapping (e.g., a first set of processor-executable instructions that embody and/or produce/effect a first mapping) between gesture identification flags and functions (e.g., a first variant of mapping 700 from FIG. 7) to be invoked by a first application run on the computer, a second mapping (e.g., a second set of processor-executable instructions that embody and/or produce/effect a second mapping) between gesture identification flags and functions (e.g., a second variant of mapping 700 from FIG. 7) to be invoked by a second application run on the computer, a third mapping (e.g., a third set of processor-executable instructions that embody and/or produce/effect a third mapping) between gesture identification flags and functions (e.g., a third variant of mapping 700 from FIG. 7) to be invoked by a third application run on the computer, and so on. Each of the first, second, and third applications may be any application, including but not limited to: an audio/video playback application, a video game application, a drawing or modeling application, a control application, a communication application, a browsing or navigating applications, and so on. As previously described, the non-transitory processor-readable medium of the computer may store an API or other data or information through which a user may program processor-executable instructions that embody and/or produce/effect any mapping(s) between gesture identification flags and functions of any electronic device (including but not limited to the computer itself). For example, a user may use an API executed by a computer to define processor-executable instructions that embody and/or produce/effect mappings between gesture identification flags and functions of the computer itself (e.g., functions of one or multiple applications executed by the computer itself), or the user may use an API executed by a computer to define processor-executable instructions (such as firmware or embedded software instructions) that are then ported to, installed on, loaded in, or otherwise received by a separate electronic device, where the processor-executable instructions embody and/or produce/effect mappings between gesture identification flags and functions of the separate electronic device. In accordance with the present systems, articles, and methods, virtually any application run on a computer or any other electronic device may be adapted to respond to generic gesture identification flags output by a wearable EMG device. Thus, in some cases, method 600 may include an additional act performed by the electronic device, the additional act being selecting and/or initializing a specific application of the electronic device (e.g., stored in and/or to be executed by the electronic device) to be controlled by the wearable EMG device. Selecting and/or initializing a specific application of the electronic device may include selecting/initializing a first set of processor-executable instructions that embody and/or produce/effect a first mapping from multiple sets of processor-executable instructions that embody and/or produce/effect multiple mappings (e.g., one set of processor-executable instructions that embody and/or produce/effect a particular mapping from a plurality of sets of processor-executable instructions that embody and/or produce/effect a plurality of respective mappings).
  • In accordance with the present systems, articles, and methods, a wearable EMG device may be used to control multiple electronic devices, or multiple applications within a single electronic device. Such is distinct from known proposals for human-electronics interfaces that employ a wearable EMG device, at least because the known proposals typically store a direct mapping from gestures to functions within the wearable EMG device itself, whereas the present systems, articles, and methods describe an intermediate mapping from gestures (e.g., from EMG and/or accelerometer signals representative of gestures) to gesture identification flags that are stored and executed by the wearable EMG device and then mappings from gesture identification flags to functions that are stored and executed by the downstream electronic device. In accordance with the present systems, articles, and methods, the mapping from gestures to gesture identification flags stored and executed by the wearable EMG device is independent of the downstream electronic device and the same mapping from gestures to gesture identification flags may be stored and executed by the wearable EMG device regardless of the nature and/or function(s) of the downstream electronic device.
  • The implementation of gesture identification flags as described herein enables users to employ the same wearable EMG device to control a wide range of electronic devices and/or a wide range of applications within a single electronic device. Since the gesture identification flags output by the wearable EMG device are not tied to any specific functions or commands, a user may define their own mappings (including their own techniques for performing mappings) between gesture identification flags and electronic device functions. For example, a user may adapt the human-electronics interfaces described herein to control virtually any functions of virtually any electronic device (e.g., to control virtually any application executed by a computer) by defining processor-executable instructions that embody and/or produce a corresponding mapping between gesture identification flags and electronic device functions (such as mapping 700 from FIG. 7) and establishing automatic execution of the processor-executable instructions by the electronic device in response to receiving gesture identification flags. The processor-executable instructions may be defined for/within the electronic device itself without making any modifications to the wearable EMG device.
  • The various embodiments described herein provide human-electronics interfaces in which a wearable EMG device (i.e., a controller) provides generic signal “flags” and a downstream receiving device interprets and responds to the generic flags. The flags provided by the wearable EMG device are substantially independent of any downstream receiving device. In accordance with the present systems, articles, and methods, other forms of controllers (i.e., controllers that are not wearable and/or controllers that do not employ EMG sensors) may similarly be configured to provide generic flags in this way. For example, instead of or in addition to employing EMG sensors and/or accelerometers providing gesture control, a controller that operates in accordance with the present systems, articles, and methods may employ, for example, tactile sensors (e.g., buttons, switches, touchpads, or keys) providing manual control, acoustic sensors providing voice-control, optical/photonic sensors providing gesture control, or any other type(s) of user-activated sensors providing any other type(s) of user-activated control. Thus, the teachings of the present systems, articles, and methods may be applied using virtually any type of controller employing sensors (including gesture-based control devices that do not make use of electromyography or EMG sensors), with the acts described herein as being performed by “at least one EMG sensor” and/or “at least one accelerometer” being more generally performed by “at least one sensor.”
  • The above description of illustrated embodiments, including what is described in the Abstract, is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Although specific embodiments of and examples are described herein for illustrative purposes, various equivalent modifications can be made without departing from the spirit and scope of the disclosure, as will be recognized by those skilled in the relevant art. The teachings provided herein of the various embodiments can be applied to other portable and/or wearable electronic devices, not necessarily the exemplary wearable electronic devices generally described above.
  • For instance, the foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, schematics, and examples. Insofar as such block diagrams, schematics, and examples contain one or more functions and/or operations, it will be understood by those skilled in the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, the present subject matter may be implemented via Application Specific Integrated Circuits (ASICs). However, those skilled in the art will recognize that the embodiments disclosed herein, in whole or in part, can be equivalently implemented in standard integrated circuits, as one or more computer programs executed by one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs executed by on one or more controllers (e.g., microcontrollers) as one or more programs executed by one or more processors (e.g., microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of ordinary skill in the art in light of the teachings of this disclosure.
  • When logic is implemented as software and stored in memory, logic or information can be stored on any computer-readable medium for use by or in connection with any processor-related system or method. In the context of this disclosure, a memory is a computer-readable medium that is an electronic, magnetic, optical, or other physical device or means that contains or stores a computer and/or processor program. Logic and/or the information can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions associated with logic and/or information.
  • In the context of this specification, a “non-transitory computer-readable medium” can be any element that can store the program associated with logic and/or information for use by or in connection with the instruction execution system, apparatus, and/or device. The computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device. More specific examples (a non-exhaustive list) of the computer readable medium would include the following: a portable computer diskette (magnetic, compact flash card, secure digital, or the like), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), a portable compact disc read-only memory (CDROM), digital tape, and other non-transitory media.
  • The various embodiments described above can be combined to provide further embodiments. To the extent that they are not inconsistent with the specific teachings and definitions herein, all of the U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications and non-patent publications referred to in this specification and/or listed in the Application Data Sheet, including but not limited to: U.S. Provisional Patent Application Ser. No. 61/869,526; U.S. Provisional Patent Application Ser. No. 61/857,105 (now U.S. Non-Provisional patent application Ser. No. 14/335,668); U.S. Provisional Patent Application Ser. No. 61/752,226 (now U.S. Non-Provisional patent application Ser. No. 14/155,107); U.S. Provisional Patent Application Ser. No. 61/768,322 (now U.S. Non-Provisional patent application Ser. No. 14/186,889); U.S. Provisional Patent Application Ser. No. 61/771,500 (now U.S. Non-Provisional patent application Ser. No. 14/194,252); U.S. Provisional Application Ser. No. 61/860,063 (now U.S. Non-Provisional patent application Ser. No. 14/276,575), and U.S. Provisional Application Ser. No. 61/866,960 (now U.S. Non-Provisional patent application Ser. No. 14/461,044), are incorporated herein by reference, in their entirety. Aspects of the embodiments can be modified, if necessary, to employ systems, circuits and concepts of the various patents, applications and publications to provide yet further embodiments.
  • These and other changes can be made to the embodiments in light of the above-detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the disclosure.

Claims (37)

1. A wearable electromyography (“EMG”) device comprising:
at least one EMG sensor to in use detect muscle activity of a user of the wearable EMG device and provide at least one signal in response to the detected muscle activity;
a processor communicatively coupled to the at least one EMG sensor, the processor to in use determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor; and
an output terminal communicatively coupled to the processor to in use transmit the gesture identification flag.
2. The wearable EMG device of claim 1 wherein the gesture identification flag is independent of any downstream processor-based device and generic to a variety of end user applications executable by a variety of downstream processor-based devices useable with the wearable EMG device.
3. The wearable EMG device of claim 1, further comprising:
a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags.
4. The wearable EMG device of claim 3 wherein the non-transitory processor-readable storage medium stores processor-executable instructions that embody and/or produce a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the processor, the processor-executable instructions cause the processor to determine a gesture identification flag in accordance with the mapping.
5. The wearable EMG device of claim 3 wherein the non-transitory processor-readable storage medium stores processor-executable instructions that, when executed by the processor, cause the processor to determine a gesture identification flag based at least in part on at least one signal provided by the at least one EMG sensor.
6. The wearable EMG device of claim 1, further comprising:
at least one accelerometer responsive to motion effected by the user of the wearable EMG device and communicatively coupled to the processor to provide at least one signal in response to the detected motion, and wherein the processor determines the gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least one signal provided by the at least one accelerometer.
7. The wearable EMG device of claim 1 wherein the processor is selected from the group consisting of: a digital microprocessor, a digital microcontroller, a digital signal processor, a graphics processing unit, an application specific integrated circuit, a programmable gate array, and a programmable logic unit.
8. The wearable EMG device of claim 1 wherein the at least one EMG sensor includes a plurality of EMG sensors, and wherein the wearable EMG device further comprises:
a set of communicative pathways to route signals provided by the plurality of EMG sensors to the processor, wherein each EMG sensor in the plurality of EMG sensors is communicatively coupled to the processor by at least one communicative pathway from the set of communicative pathways.
9. The wearable EMG device of claim 8, further comprising:
a set of pod structures that form physically coupled links of the wearable EMG device, wherein each pod structure in the set of pod structures is positioned adjacent and physically coupled to at least one other pod structure in the set of pod structures, and wherein the set of pod structures comprises at least two sensor pods and a processor pod, each of the at least two sensor pods comprising a respective EMG sensor from the plurality of EMG sensors and the processor pod comprising the processor.
10. The wearable EMG device of claim 9 wherein each pod structure in the set of pod structures is positioned adjacent and in between two other pod structures in the set of pod structures and physically coupled to the two other pod structures in the set of pod structures, and wherein the set of pod structures forms a perimeter of an annular configuration.
11. The wearable EMG device of claim 9, further comprising:
at least one adaptive coupler, wherein each respective pod structure in the set of pod structures is adaptively physically coupled to at least one adjacent pod structure in the set of pod structures by at least one adaptive coupler.
12. The wearable EMG device of claim 1 wherein the output terminal includes at least one of a wireless transmitter and/or a tethered connector port.
13. The wearable EMG device of claim 1 wherein the at least one EMG sensor includes at least one capacitive EMG sensor.
14. A method of operating a wearable electromyography (“EMG”) device to provide electromyographic control of an electronic device, wherein the wearable EMG device includes at least one EMG sensor, a processor, and an output terminal, the method comprising:
detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor;
providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity;
determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor, wherein the gesture identification flag is independent of the electronic device; and
transmitting the gesture identification flag to the electronic device by the output terminal.
15. The method of claim 14 wherein:
detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor includes detecting muscle activity of the user of the wearable EMG device by a first EMG sensor and by at least a second EMG sensor,
providing at least one signal from the at least one EMG sensor to the processor in response to the detected muscle activity includes providing at least a first signal from the first EMG sensor to the processor in response to the detected muscle activity and providing at least a second signal from the second EMG sensor to the processor in response to the detected muscle activity, and
determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor includes determining, by the processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the processor and the at least a second signal provided from the at least a second EMG sensor to the processor.
16. The method of claim 14 wherein the wearable EMG device further comprises a non-transitory processor-readable storage medium that stores processor-executable instructions, and wherein determining, by the processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor includes executing the processor-executable instructions by the processor to cause the processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor.
17. The method of claim 14 wherein the wearable EMG device further comprises at least one accelerometer, and wherein the method further comprises:
detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and
providing at least one signal from the at least one accelerometer to the processor in response to the detected motion,
and wherein determining a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the processor includes:
determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor.
18. The method of claim 17 wherein the wearable EMG device further comprises a non-transitory processor-readable storage medium that stores processor-executable instructions, and wherein determining, by the processor, a gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor includes executing the processor-executable instructions by the processor to cause the processor to determine the gesture identification flag based at least in part on both the at least one signal provided from the at least one EMG sensor to the processor and the at least one signal provided from the at least one accelerometer to the processor.
19. The method of claim 14 wherein the output terminal includes a wireless transmitter, and wherein transmitting the gesture identification flag to the electronic device by the output terminal includes wirelessly transmitting the gesture identification flag to the electronic device by the wireless transmitter.
20. A system that enables electromyographic control of an electronic device, the system comprising:
a wearable electromyography (“EMG”) device comprising:
at least one EMG sensor responsive to muscle activity of a user of the wearable EMG device and provide at least one signal in response to a detected muscle activity,
a first processor communicatively coupled to the at least one EMG sensor, the first processor which in use determines a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor, and
an output terminal communicatively coupled to the first processor to transmit the gesture identification flag;
and
an electronic device comprising:
an input terminal to receive the gesture identification flag, and
a second processor communicatively coupled to the input terminal, the second processor which in use determines a function of the electronic device based at least in part on the gesture identification flag.
21. The system of claim 20 wherein the gesture identification flag is independent of the electronic device and generic to a variety of end user applications executable by the electronic device.
22. The system of claim 20 wherein the wearable EMG device of the system further comprises:
a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores at least a set of gesture identification flags.
23. The system of claim 22 wherein the non-transitory processor-readable storage medium of the wearable EMG device stores processor-executable instructions that embody and/or produce a mapping between at least one signal provided by the at least one EMG sensor and at least one gesture identification flag and, when executed by the first processor, the processor-executable instructions cause the first processor to determine a gesture identification flag in accordance with the mapping.
24. The system of claim 20 wherein the wearable EMG device of the system further comprises:
a non-transitory processor-readable storage medium communicatively coupled to the first processor, wherein the non-transitory processor-readable storage medium stores processor-executable instructions that, when executed by the first processor, cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided by the at least one EMG sensor.
25. The system of claim 20 wherein the wearable EMG device of the system further comprises:
at least one accelerometer communicatively coupled to the first processor, the at least one accelerometer responsive to motion effected by the user of the wearable EMG device and which provides at least one signal in response to a detected motion, and wherein the first processor determines a gesture identification flag based at least in part on both the at least one signal provided by the at least one EMG sensor and the at least on signal provided by the at least one accelerometer.
26. The system of claim 20 wherein the electronic device of the system further comprises:
a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores at least a set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
27. The system of claim 20 wherein the electronic device of the system further comprises:
a non-transitory processor-readable storage medium communicatively coupled to the second processor, wherein the non-transitory processor-readable storage medium stores:
a first application executable by the electronic device;
at least a second application executable by the electronic device;
a first set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the first application based at least in part on a gesture identification flag; and
a second set of processor-executable instructions that, when executed by the second processor, cause the second processor to determine a function of the second application based at least in part on a gesture identification flag.
28. The system of claim 20 wherein the output terminal of the wearable EMG device includes a first tethered connector port, the input terminal of the electronic device includes a second tethered connector port, and further comprising:
a communicative pathway that in use communicatively couples the first tethered connector port to the second tethered connector port and to route the gesture identification flag from the output terminal of the wearable EMG device to the input terminal of the electronic device.
29. The system of claim 20 wherein the output terminal of the wearable EMG device includes a wireless transmitter that in use wirelessly transmits the gesture identification flag and the input terminal of the electronic device includes a tethered connector port, and wherein the system further comprises:
a wireless receiver that in use communicatively couples to the tethered connector port of the electronic device and to in use wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
30. The system of claim 20 wherein the output terminal of the wearable EMG device includes a wireless transmitter to wirelessly transmit the gesture identification flag, and wherein the input terminal of the electronic device includes a wireless receiver to wirelessly receive the gesture identification flag from the wireless transmitter of the wearable EMG device.
31. The system of claim 20 wherein the electronic device is selected from the group consisting of: a computer, a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone, a portable electronic device, an audio player, a television, a video player, a video game console, a robot, a light switch, and a vehicle.
32. A method of electromyographically controlling at least one function of an electronic device by a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes at least one EMG sensor, a first processor, and an output terminal and the electronic device includes an input terminal and a second processor, the method comprising:
detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor;
providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity;
determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor, wherein the gesture identification flag is independent of the electronic device;
transmitting the gesture identification flag by the output terminal of the wearable EMG device;
receiving the gesture identification flag by the input terminal of the electronic device;
determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag; and
performing the function by the electronic device.
33. The method of claim 32 wherein:
detecting muscle activity of a user of the wearable EMG device by the at least one EMG sensor includes detecting muscle activity of the user of the wearable EMG device by a first EMG sensor of the wearable EMG device and by at least a second EMG sensor of the wearable EMG device,
providing at least one signal from the at least one EMG sensor to the first processor in response to the detected muscle activity includes providing at least a first signal from the first EMG sensor to the first processor in response to the detected muscle activity and providing at least a second signal from the send EMG sensor to the first processor in response to the detected muscle activity, and
determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor includes determining, by the first processor, a gesture identification flag based at least in part on the at least a first signal provided from the first EMG sensor to the first processor and the at least a second signal provided from the at least a second EMG sensor to the first processor.
34. The method of claim 32 wherein the wearable EMG device further comprises a non-transitory processor-readable medium that stores processor-executable instructions, and wherein determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor includes executing the processor-executable instructions by the first processor to cause the first processor to determine a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor.
35. The method of claim 32 wherein the wearable EMG device further comprises at least one accelerometer, and wherein the method further comprises:
detecting motion effected by the user of the wearable EMG device by the at least one accelerometer; and
providing at least one signal from the at least one accelerometer to the first processor in response to the detected motion,
and wherein determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor includes:
determining, by the first processor, a gesture identification flag based at least in part on the at least one signal provided from the at least one EMG sensor to the first processor and the at least one signal provided by the at least one accelerometer to the first processor.
36. The method of claim 32 wherein the output terminal of the wearable EMG device includes a wireless transmitter and the input terminal of the electronic device includes a wireless receiver, and wherein:
transmitting the gesture identification flag by the output terminal of the wearable EMG device includes wirelessly transmitting the gesture identification flag by the wireless transmitter of the wearable EMG device, and
receiving the gesture identification flag by the input terminal of the electronic device includes wirelessly receiving the gesture identification flag by the wireless receiver of the electronic device.
37. The method of claim 32 wherein the electronic device further comprises a non-transitory processor-readable storage medium that stores processor-executable instructions, and wherein determining, by the second processor, a function of the electronic device based at least in part on the gesture identification flag includes executing the processor-executable instructions by the second processor to cause the second processor to determine a function of the electronic device based at least in part on the gesture identification flag.
US14/465,194 2013-08-23 2014-08-21 Systems, articles, and methods for human-electronics interfaces Abandoned US20150057770A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US201361869526P true 2013-08-23 2013-08-23
US14/465,194 US20150057770A1 (en) 2013-08-23 2014-08-21 Systems, articles, and methods for human-electronics interfaces

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US14/465,194 US20150057770A1 (en) 2013-08-23 2014-08-21 Systems, articles, and methods for human-electronics interfaces

Publications (1)

Publication Number Publication Date
US20150057770A1 true US20150057770A1 (en) 2015-02-26

Family

ID=52481060

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/465,194 Abandoned US20150057770A1 (en) 2013-08-23 2014-08-21 Systems, articles, and methods for human-electronics interfaces

Country Status (3)

Country Link
US (1) US20150057770A1 (en)
CA (1) CA2921954A1 (en)
WO (1) WO2015027089A1 (en)

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150124566A1 (en) * 2013-10-04 2015-05-07 Thalmic Labs Inc. Systems, articles and methods for wearable electronic devices employing contact sensors
CN105005383A (en) * 2015-07-10 2015-10-28 昆山美莱来工业设备有限公司 Wearable arm band that manipulates mobile robot by using hand gesture
CN105014676A (en) * 2015-07-03 2015-11-04 浙江大学 Robot motion control method
US20150363631A1 (en) * 2014-06-12 2015-12-17 Yahoo! Inc. User identification on a per touch basis on touch sensitive devices
US20160092668A1 (en) * 2014-09-29 2016-03-31 Xiaomi Inc. Methods and devices for authorizing operation
USD756359S1 (en) * 2014-05-15 2016-05-17 Thalmic Labs Inc. Expandable armband device
US9600030B2 (en) 2014-02-14 2017-03-21 Thalmic Labs Inc. Systems, articles, and methods for elastic electrical cables and wearable electronic devices employing same
US9788789B2 (en) 2013-08-30 2017-10-17 Thalmic Labs Inc. Systems, articles, and methods for stretchable printed circuit boards
US9807221B2 (en) 2014-11-28 2017-10-31 Thalmic Labs Inc. Systems, devices, and methods effected in response to establishing and/or terminating a physical communications link
US9809231B2 (en) 2015-10-28 2017-11-07 Honda Motor Co., Ltd. System and method for executing gesture based control of a vehicle system
US9880632B2 (en) 2014-06-19 2018-01-30 Thalmic Labs Inc. Systems, devices, and methods for gesture identification
CN107856014A (en) * 2017-11-08 2018-03-30 浙江工业大学 Mechanical arm posture control method based on gesture identification
EP3306442A1 (en) * 2016-10-10 2018-04-11 Deere & Company Control of machines through detection of gestures by optical and muscle sensors
CN108108016A (en) * 2017-12-07 2018-06-01 浙江大学 Gesture perceptron
DE102017205640B3 (en) * 2017-04-03 2018-07-12 Audi Ag Method and device for detecting an operating gesture
WO2018135692A1 (en) * 2017-01-22 2018-07-26 계명대학교 산학협력단 Wearable device for motion recognition and control, and method for motion recognition control using same
US10042422B2 (en) 2013-11-12 2018-08-07 Thalmic Labs Inc. Systems, articles, and methods for capacitive electromyography sensors
US10078435B2 (en) 2015-04-24 2018-09-18 Thalmic Labs Inc. Systems, methods, and computer program products for interacting with electronically displayed presentation materials
US10152082B2 (en) 2013-05-13 2018-12-11 North Inc. Systems, articles and methods for wearable electronic devices that accommodate different user forms
US10188309B2 (en) 2013-11-27 2019-01-29 North Inc. Systems, articles, and methods for electromyography sensors
US10199008B2 (en) 2014-03-27 2019-02-05 North Inc. Systems, devices, and methods for wearable electronic devices as state machines
US10409371B2 (en) 2016-07-25 2019-09-10 Ctrl-Labs Corporation Methods and apparatus for inferring user intent based on neuromuscular signals
US10460455B2 (en) 2018-01-25 2019-10-29 Ctrl-Labs Corporation Real-time processing of handstate representation model estimates
US10489986B2 (en) 2018-01-25 2019-11-26 Ctrl-Labs Corporation User-controlled tuning of handstate representation model parameters
US10496168B2 (en) 2018-01-25 2019-12-03 Ctrl-Labs Corporation Calibration techniques for handstate representation modeling using neuromuscular signals
US10504286B2 (en) 2018-01-25 2019-12-10 Ctrl-Labs Corporation Techniques for anonymizing neuromuscular signal data
US10528135B2 (en) 2013-01-14 2020-01-07 Ctrl-Labs Corporation Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US10592001B2 (en) 2018-05-08 2020-03-17 Facebook Technologies, Llc Systems and methods for improved speech recognition using neuromuscular information
US10687759B2 (en) 2018-05-29 2020-06-23 Facebook Technologies, Llc Shielding techniques for noise reduction in surface electromyography signal measurement and related systems and methods
US10772519B2 (en) 2018-05-25 2020-09-15 Facebook Technologies, Llc Methods and apparatus for providing sub-muscular control
US10817795B2 (en) 2018-01-25 2020-10-27 Facebook Technologies, Llc Handstate reconstruction based on multiple inputs
US10842407B2 (en) 2018-08-31 2020-11-24 Facebook Technologies, Llc Camera-guided interpretation of neuromuscular signals
US10902743B2 (en) 2017-04-14 2021-01-26 Arizona Board Of Regents On Behalf Of Arizona State University Gesture recognition and communication
US10905383B2 (en) 2019-02-28 2021-02-02 Facebook Technologies, Llc Methods and apparatus for unsupervised one-shot machine learning for classification of human gestures and estimation of applied forces
US10921764B2 (en) 2018-09-26 2021-02-16 Facebook Technologies, Llc Neuromuscular control of physical objects in an environment
US10921886B2 (en) 2012-06-14 2021-02-16 Medibotics Llc Circumferential array of electromyographic (EMG) sensors
US10937414B2 (en) 2018-05-08 2021-03-02 Facebook Technologies, Llc Systems and methods for text input using neuromuscular information
US10970936B2 (en) 2018-10-05 2021-04-06 Facebook Technologies, Llc Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment
US10970374B2 (en) 2018-06-14 2021-04-06 Facebook Technologies, Llc User identification and authentication with neuromuscular signatures
US10990174B2 (en) 2016-07-25 2021-04-27 Facebook Technologies, Llc Methods and apparatus for predicting musculo-skeletal position information using wearable autonomous sensors
US11000211B2 (en) 2016-07-25 2021-05-11 Facebook Technologies, Llc Adaptive system for deriving control signals from measurements of neuromuscular activity
US11045137B2 (en) 2018-07-19 2021-06-29 Facebook Technologies, Llc Methods and apparatus for improved signal robustness for a wearable neuromuscular recording device
US11069148B2 (en) 2018-01-25 2021-07-20 Facebook Technologies, Llc Visualization of reconstructed handstate information

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105012057B (en) * 2015-07-30 2017-04-26 沈阳工业大学 Intelligent artificial limb based on double-arm electromyogram and attitude information acquisition and motion classifying method
EP3133467A1 (en) 2015-08-17 2017-02-22 Bluemint Labs Universal contactless gesture control system
CN111752137A (en) * 2020-07-06 2020-10-09 诺百爱(杭州)科技有限责任公司 Myoelectric smart watch, myoelectric control method of smart watch and electronic equipment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8170656B2 (en) * 2008-06-26 2012-05-01 Microsoft Corporation Wearable electromyography-based controllers for human-computer interface
US20130317648A1 (en) * 2012-05-25 2013-11-28 California Institute Of Technology Biosleeve human-machine interface
US8702629B2 (en) * 2005-03-17 2014-04-22 Great Lakes Neuro Technologies Inc. Movement disorder recovery system and method for continuous monitoring
US9170674B2 (en) * 2012-04-09 2015-10-27 Qualcomm Incorporated Gesture-based device control using pressure-sensitive sensors
US9299248B2 (en) * 2013-02-22 2016-03-29 Thalmic Labs Inc. Method and apparatus for analyzing capacitive EMG and IMU sensor signals for gesture control
US9367139B2 (en) * 2013-12-12 2016-06-14 Thalmic Labs Inc. Systems, articles, and methods for gesture identification in wearable electromyography devices
US9389694B2 (en) * 2013-10-22 2016-07-12 Thalmic Labs Inc. Systems, articles, and methods for gesture identification in wearable electromyography devices

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE60215504T2 (en) * 2002-10-07 2007-09-06 Sony France S.A. Method and apparatus for analyzing gestures of a human, e.g. for controlling a machine by gestures
JP5545574B2 (en) * 2009-07-15 2014-07-09 国立大学法人 筑波大学 Classification estimation system and classification estimation program
US20130135223A1 (en) * 2009-12-13 2013-05-30 Ringbow Ltd. Finger-worn input devices and methods of use
KR101448106B1 (en) * 2011-02-17 2014-10-08 주식회사 라이프사이언스테크놀로지 Analisys Method of Rehabilitation status using Electromyogram

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8702629B2 (en) * 2005-03-17 2014-04-22 Great Lakes Neuro Technologies Inc. Movement disorder recovery system and method for continuous monitoring
US8170656B2 (en) * 2008-06-26 2012-05-01 Microsoft Corporation Wearable electromyography-based controllers for human-computer interface
US9170674B2 (en) * 2012-04-09 2015-10-27 Qualcomm Incorporated Gesture-based device control using pressure-sensitive sensors
US20130317648A1 (en) * 2012-05-25 2013-11-28 California Institute Of Technology Biosleeve human-machine interface
US9299248B2 (en) * 2013-02-22 2016-03-29 Thalmic Labs Inc. Method and apparatus for analyzing capacitive EMG and IMU sensor signals for gesture control
US9389694B2 (en) * 2013-10-22 2016-07-12 Thalmic Labs Inc. Systems, articles, and methods for gesture identification in wearable electromyography devices
US9367139B2 (en) * 2013-12-12 2016-06-14 Thalmic Labs Inc. Systems, articles, and methods for gesture identification in wearable electromyography devices

Cited By (64)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10921886B2 (en) 2012-06-14 2021-02-16 Medibotics Llc Circumferential array of electromyographic (EMG) sensors
US11009951B2 (en) 2013-01-14 2021-05-18 Facebook Technologies, Llc Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US10528135B2 (en) 2013-01-14 2020-01-07 Ctrl-Labs Corporation Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US10152082B2 (en) 2013-05-13 2018-12-11 North Inc. Systems, articles and methods for wearable electronic devices that accommodate different user forms
US9788789B2 (en) 2013-08-30 2017-10-17 Thalmic Labs Inc. Systems, articles, and methods for stretchable printed circuit boards
US20150124566A1 (en) * 2013-10-04 2015-05-07 Thalmic Labs Inc. Systems, articles and methods for wearable electronic devices employing contact sensors
US10042422B2 (en) 2013-11-12 2018-08-07 Thalmic Labs Inc. Systems, articles, and methods for capacitive electromyography sensors
US10310601B2 (en) 2013-11-12 2019-06-04 North Inc. Systems, articles, and methods for capacitive electromyography sensors
US10331210B2 (en) 2013-11-12 2019-06-25 North Inc. Systems, articles, and methods for capacitive electromyography sensors
US11079846B2 (en) 2013-11-12 2021-08-03 Facebook Technologies, Llc Systems, articles, and methods for capacitive electromyography sensors
US10101809B2 (en) 2013-11-12 2018-10-16 Thalmic Labs Inc. Systems, articles, and methods for capacitive electromyography sensors
US10188309B2 (en) 2013-11-27 2019-01-29 North Inc. Systems, articles, and methods for electromyography sensors
US10251577B2 (en) 2013-11-27 2019-04-09 North Inc. Systems, articles, and methods for electromyography sensors
US10362958B2 (en) 2013-11-27 2019-07-30 Ctrl-Labs Corporation Systems, articles, and methods for electromyography sensors
US10898101B2 (en) 2013-11-27 2021-01-26 Facebook Technologies, Llc Systems, articles, and methods for electromyography sensors
US9600030B2 (en) 2014-02-14 2017-03-21 Thalmic Labs Inc. Systems, articles, and methods for elastic electrical cables and wearable electronic devices employing same
US10199008B2 (en) 2014-03-27 2019-02-05 North Inc. Systems, devices, and methods for wearable electronic devices as state machines
USD756359S1 (en) * 2014-05-15 2016-05-17 Thalmic Labs Inc. Expandable armband device
US20150363631A1 (en) * 2014-06-12 2015-12-17 Yahoo! Inc. User identification on a per touch basis on touch sensitive devices
US9396378B2 (en) * 2014-06-12 2016-07-19 Yahoo! User identification on a per touch basis on touch sensitive devices
US9880632B2 (en) 2014-06-19 2018-01-30 Thalmic Labs Inc. Systems, devices, and methods for gesture identification
US10684692B2 (en) 2014-06-19 2020-06-16 Facebook Technologies, Llc Systems, devices, and methods for gesture identification
US20160092668A1 (en) * 2014-09-29 2016-03-31 Xiaomi Inc. Methods and devices for authorizing operation
US9892249B2 (en) * 2014-09-29 2018-02-13 Xiaomi Inc. Methods and devices for authorizing operation
US9807221B2 (en) 2014-11-28 2017-10-31 Thalmic Labs Inc. Systems, devices, and methods effected in response to establishing and/or terminating a physical communications link
US10078435B2 (en) 2015-04-24 2018-09-18 Thalmic Labs Inc. Systems, methods, and computer program products for interacting with electronically displayed presentation materials
CN105014676A (en) * 2015-07-03 2015-11-04 浙江大学 Robot motion control method
CN105005383A (en) * 2015-07-10 2015-10-28 昆山美莱来工业设备有限公司 Wearable arm band that manipulates mobile robot by using hand gesture
US20170326984A1 (en) * 2015-10-28 2017-11-16 Honda Motor Co., Ltd. System and method for executing gesture based control of a vehicle system
US9809231B2 (en) 2015-10-28 2017-11-07 Honda Motor Co., Ltd. System and method for executing gesture based control of a vehicle system
US10562394B2 (en) 2015-10-28 2020-02-18 Honda Motor Co., Ltd. System and method for executing gesture based control of a vehicle system
US10409371B2 (en) 2016-07-25 2019-09-10 Ctrl-Labs Corporation Methods and apparatus for inferring user intent based on neuromuscular signals
US10656711B2 (en) 2016-07-25 2020-05-19 Facebook Technologies, Llc Methods and apparatus for inferring user intent based on neuromuscular signals
US11000211B2 (en) 2016-07-25 2021-05-11 Facebook Technologies, Llc Adaptive system for deriving control signals from measurements of neuromuscular activity
US10990174B2 (en) 2016-07-25 2021-04-27 Facebook Technologies, Llc Methods and apparatus for predicting musculo-skeletal position information using wearable autonomous sensors
US10162422B2 (en) * 2016-10-10 2018-12-25 Deere & Company Control of machines through detection of gestures by optical and muscle sensors
US20180101235A1 (en) * 2016-10-10 2018-04-12 Deere & Company Control of machines through detection of gestures by optical and muscle sensors
EP3306442A1 (en) * 2016-10-10 2018-04-11 Deere & Company Control of machines through detection of gestures by optical and muscle sensors
KR20180086547A (en) * 2017-01-22 2018-08-01 계명대학교 산학협력단 Wearable device for gesture recognition and control and gesture recognition control method using the same
KR101963694B1 (en) * 2017-01-22 2019-03-29 계명대학교 산학협력단 Wearable device for gesture recognition and control and gesture recognition control method using the same
WO2018135692A1 (en) * 2017-01-22 2018-07-26 계명대학교 산학협력단 Wearable device for motion recognition and control, and method for motion recognition control using same
DE102017205640B3 (en) * 2017-04-03 2018-07-12 Audi Ag Method and device for detecting an operating gesture
US10902743B2 (en) 2017-04-14 2021-01-26 Arizona Board Of Regents On Behalf Of Arizona State University Gesture recognition and communication
CN107856014A (en) * 2017-11-08 2018-03-30 浙江工业大学 Mechanical arm posture control method based on gesture identification
CN108108016A (en) * 2017-12-07 2018-06-01 浙江大学 Gesture perceptron
US10460455B2 (en) 2018-01-25 2019-10-29 Ctrl-Labs Corporation Real-time processing of handstate representation model estimates
US10504286B2 (en) 2018-01-25 2019-12-10 Ctrl-Labs Corporation Techniques for anonymizing neuromuscular signal data
US10817795B2 (en) 2018-01-25 2020-10-27 Facebook Technologies, Llc Handstate reconstruction based on multiple inputs
US11069148B2 (en) 2018-01-25 2021-07-20 Facebook Technologies, Llc Visualization of reconstructed handstate information
US10496168B2 (en) 2018-01-25 2019-12-03 Ctrl-Labs Corporation Calibration techniques for handstate representation modeling using neuromuscular signals
US10950047B2 (en) 2018-01-25 2021-03-16 Facebook Technologies, Llc Techniques for anonymizing neuromuscular signal data
US10489986B2 (en) 2018-01-25 2019-11-26 Ctrl-Labs Corporation User-controlled tuning of handstate representation model parameters
US10937414B2 (en) 2018-05-08 2021-03-02 Facebook Technologies, Llc Systems and methods for text input using neuromuscular information
US11036302B1 (en) 2018-05-08 2021-06-15 Facebook Technologies, Llc Wearable devices and methods for improved speech recognition
US10592001B2 (en) 2018-05-08 2020-03-17 Facebook Technologies, Llc Systems and methods for improved speech recognition using neuromuscular information
US10772519B2 (en) 2018-05-25 2020-09-15 Facebook Technologies, Llc Methods and apparatus for providing sub-muscular control
US10687759B2 (en) 2018-05-29 2020-06-23 Facebook Technologies, Llc Shielding techniques for noise reduction in surface electromyography signal measurement and related systems and methods
US10970374B2 (en) 2018-06-14 2021-04-06 Facebook Technologies, Llc User identification and authentication with neuromuscular signatures
US11045137B2 (en) 2018-07-19 2021-06-29 Facebook Technologies, Llc Methods and apparatus for improved signal robustness for a wearable neuromuscular recording device
US10842407B2 (en) 2018-08-31 2020-11-24 Facebook Technologies, Llc Camera-guided interpretation of neuromuscular signals
US10905350B2 (en) 2018-08-31 2021-02-02 Facebook Technologies, Llc Camera-guided interpretation of neuromuscular signals
US10921764B2 (en) 2018-09-26 2021-02-16 Facebook Technologies, Llc Neuromuscular control of physical objects in an environment
US10970936B2 (en) 2018-10-05 2021-04-06 Facebook Technologies, Llc Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment
US10905383B2 (en) 2019-02-28 2021-02-02 Facebook Technologies, Llc Methods and apparatus for unsupervised one-shot machine learning for classification of human gestures and estimation of applied forces

Also Published As

Publication number Publication date
CA2921954A1 (en) 2015-02-26
WO2015027089A1 (en) 2015-02-26

Similar Documents

Publication Publication Date Title
US20150057770A1 (en) Systems, articles, and methods for human-electronics interfaces
US9372535B2 (en) Systems, articles, and methods for electromyography-based human-electronics interfaces
US9483123B2 (en) Systems, articles, and methods for gesture identification in wearable electromyography devices
US9367139B2 (en) Systems, articles, and methods for gesture identification in wearable electromyography devices
US11009951B2 (en) Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US20180150033A1 (en) Systems, articles and methods for wearable electronic devices employing contact sensors
US20150261306A1 (en) Systems, devices, and methods for selecting between multiple wireless connections
US20150051470A1 (en) Systems, articles and methods for signal routing in wearable electronic devices
US10199008B2 (en) Systems, devices, and methods for wearable electronic devices as state machines
US9389694B2 (en) Systems, articles, and methods for gesture identification in wearable electromyography devices
US10216274B2 (en) Systems, articles, and methods for wearable human-electronics interface devices
EP2959394B1 (en) Methods and devices that combine muscle activity sensor signals and inertial sensor signals for gesture-based control
KR101541082B1 (en) System and method for rehabilitation exercise of the hands
US10649549B2 (en) Device, method, and system to recognize motion using gripped object
US9788789B2 (en) Systems, articles, and methods for stretchable printed circuit boards
US10152082B2 (en) Systems, articles and methods for wearable electronic devices that accommodate different user forms
WO2015199747A1 (en) Systems, articles, and methods for wearable human-electronics interface devices
US20150035743A1 (en) Wrist Worn Platform for Sensors
JP6144743B2 (en) Wearable device
CN106814852A (en) A kind of gesture armlet controller
CN106933342A (en) Body-sensing system, motion sensing control equipment and intelligent electronic device
CN108664127B (en) Data transmission method and device based on gesture recognition input device
CN111142678A (en) Intelligent glove

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

AS Assignment

Owner name: THALMIC LABS INC., CANADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BAILEY, MATTHEW;LAKE, STEPHEN;GRANT, AARON;SIGNING DATES FROM 20131217 TO 20140110;REEL/FRAME:049307/0588

AS Assignment

Owner name: NORTH INC., CANADA

Free format text: CHANGE OF NAME;ASSIGNOR:THALMIC LABS INC.;REEL/FRAME:049548/0200

Effective date: 20180830

AS Assignment

Owner name: CTRL-LABS CORPORATION, NEW YORK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NORTH INC.;REEL/FRAME:049368/0634

Effective date: 20190502

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

AS Assignment

Owner name: FACEBOOK TECHNOLOGIES, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CTRL-LABS CORPORATION;REEL/FRAME:051649/0001

Effective date: 20200116

AS Assignment

Owner name: FACEBOOK TECHNOLOGIES, LLC, CALIFORNIA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY DATA PREVIOUSLY RECORDED AT REEL: 051649 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:CTRL-LABS CORPORATION;REEL/FRAME:051867/0136

Effective date: 20200116

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE