WO2015033327A1

WO2015033327A1 - Wearable controller for wrist

Info

Publication number: WO2015033327A1
Application number: PCT/IB2014/064348
Authority: WO
Inventors: Alfredo BELFIORI
Original assignee: Belfiori Alfredo
Priority date: 2013-09-09
Filing date: 2014-09-09
Publication date: 2015-03-12

Abstract

A human-computer interface comprises a wearable device (302, 503, 603) having a plurality of sensor nodes (103, 301, 601, 602, 501, 401, 402, 404). Each sensor node includes one or more Mechanomyography (MMG) sensors, and additionally optic sensors. A module (103, 403) automatically measures finger gestures generated signals using one or more of said sensors. A module (401, 402 and 404) automatically determines the position and the movement at the wrist level that correspond to one or more specific user finger gestures. A module (403, 405) causes one or more computing devices to automatically execute one or more specific commands corresponding to one or more of the specific user gestures.

Description

WEARABLE CONTROLLER FOR WRIST

BACKGROUND

Field of the invention

This invention relates to the field of the wearable devices, biometric controller and interfaces and biomedical engineering. More specifically this invention is directed to a human-computer interface and a system for providing the same. According to an aspect of the invention, a controller, placed around the wrist, able to read fingers movements is described. The invention is also directed to a process for sending commands to one or more computing devices by means of a wearable device.

Description of the Related Art

The technology that has been used as an interface between human and machine had so far faced two big revolutions. At first the big electronic companies produced interfaces based on the use of buttons, such as control panel, keyboard or mouse. This technology was replaced by the more innovative and fascinating touchscreen, no more physical buttons but a surface sensitive to the pressure of a part of the body. Next step in the technologic world will be the elimination of any physical contact with the device meant to be controlled. This revolution will use the biometric body signals, such as the electric, mechanical or vibrational phenomena involved in the muscles contraction, in order to control any electronic device.

BRIEF SUMMARY OF THE INVENTION

The invention provides a new human-computer interface based on the measurements of the movements of the tendons and bones at the level of the wrist and other body tissues at the level of the wrist.

This disclosure is directed to a human-computer interface (HCI) able to interpret fingers gestures and send the information to any electronic device, particularly a computing device.

According to one aspect of the invention, a human-computer interface comprises a wearable device having a plurality of sensor nodes. Each sensor node includes one or more Mechanomyography (MMG) sensors and additionally optic sensors. A module automatically measures finger gestures generated signals using one or more of said sensors. A module automatically determines the position and the movement at the wrist level that correspond to one or more specific user finger gestures. A module causes one or more computing devices to automatically execute one or more specific commands corresponding to one or more of the specific user gestures.

More specifically, a wearable controller for wrist is described which includes a plurality of sensors: sensors similar to Mechanomyography (MMG) sensors and a set of LEDs and photodiodes, in order to monitor the movements of muscles, tendons, bones and other body tissues, their combinations of movements in the wrist and their initial and final position.

Preferably, in the human-computer interface of the present invention, the one or more Mechanomuography (MMG) sensors are selected from the group comprising strain gauge sensors, force sensors, piezoresistive sensors, deformation sensors, pressure sensors and microphones. As said above, the optic sensors comprise photodiodes.

The present invention provides a wired or wireless HCI for interacting with computing systems and attached devices via electrical signals generated by specific movement of the user's fingers. The specific movements follow a fixed protocol. Following initial automated calibration process, measurement and interpretation of finger generated movement signals is accomplished by sampling signals with the MMG- based sensors and photodiodes of the wearable controller for wrist. In operation, the wearable controller for wrist is donned by the user and placed into a fixed position on the surface of the user's wrist skin. Automated cues or instructions are then provided to the user for fine-tuning control of the wearable controller for wrist. Examples of wearable controllers for wrist include articles of manufacture, such as a wristband, wristwatch, or article of clothing having a plurality of integrated MMG-based sensor nodes, one or more LEDs and photodiodes and associated electronics .

According to an aspect of the invention, a calibration phase is provided for the human- computer interface. The calibration phase automatically identifies the parameters useful to run a software installed in the module, or in one or more external computing devices. The software receives the signals and identifies the parameter for the training following a protocol of specific finger gestures.

The calibration phase of the human-computer interface which is described herein involves the user performing one or more specific finger gestures as part of a training phase for the calibration phase. This allows to precisely tune the performance of the interface to the specific biometry of the user .

More specifically, the sensors provided in the device measure the signals that are associated with one or more of the specific user gestures.

More specifically, the human-computer interface further comprises a module for automatically determining the position of the signal source on the surface of the skin of the user's wrist, in order to identify which finger moved and how.

According to another aspect of the invention, the human- computer interface of the invention is used in a process for sending commands to one or more computing devices. The process involves positioning the wearable device in contact with the surface of a user's wrist skin. Through the one or more Mechanomyography (MMG) sensors and additionally optic sensors, the state and the activity of the different body tissues at the user's wrist are then measured. The process also involves automatically evaluating gesture-generated signals of the user, which are measured via the one or more of Mechanomyography (MMG) sensors and optic sensors, in order to automatically identify one or more specific gestures of the user from a predefined set of gestures. Furthermore, the process involves automatically directing one or more computing devices to execute one or more specific commands corresponding to one or more of the identified gestures.

Preferably, the process further comprises performing an initial calibration phase which evaluates gesture generated signals associated with a subset of user finger gestures to determine expected signals during the automatic evaluation phase .

According to a preferred feature, commands associated with one or more of the gestures of the set of gestures can be user definable.

According to another aspect of the invention, a system for providing a human-computer interface (HCI) comprises a user- wearable device having one or more Mechanomyography (MMG) sensors and additionally optical sensors. The user-wearable device is adapted to be placed in use on the surface of the user's wrist skin. The system also comprises an automated calibration process which maps finger gestures generated signals corresponding to one or more specific user finger gestures to one or more specific commands. The finger gestures generated signals are measured by one or more of the Mechanomyography (MMG) sensors and optical sensors. Furthermore, the system comprises an automated process for disabling some of said sensors during rest, and an automated process for evaluating one or more user gestures associated with the signals captured by the Mechanomyography (MMG) sensors and additionally optical sensors to identify one or more commands associated with those user gestures. The system also comprises a process for transmitting specific commands associated with one or more specific user gestures to one or more computing devices.

Preferably, the user-wearable device of the above-mentioned system includes a wireless or wired interface to the one or more computing devices.

Even more preferably, in the above-mentioned system the one or more Mechanomuography (MMG) sensors are selected from the group comprising strain gauge sensors, force sensors, piezoresistive sensors, deformation sensors, pressure sensors and microphones, and where the optic sensors comprise photodiodes.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be now described in more detail, with reference to the attached drawings, given as non-limiting examples, wherein:

- Figure 1 shows a flow chart of the biometric signal analysis and output control;

- Figure 2 shows a flow chart of the biometric signal analysis and output control with the addition of an accelerometer sensor;

- Figure 3 shows the wearable device in the form of a bracelet, with a single side sensors positioned on the bottom part;

- Figure 4 shows a configuration of the sensors;

- Figure 5 shows the wearable device in the form of a watchband, with sensors in the bottom and with the watch attachments or latch on the top;

- Figure 6 shows the bracelet version of the wearable device, with double side sensors positioned on the top and on the bottom of the bracelet; and

- Figure 7 shows the piezoelectric microphone MMG-based sensors configuration in the module embodiment.

DETAILED DESCRIPTION AND BEST MODE OF IMPLEMENTATION

In the following description reference is made to the accompanying drawings, which form a part thereof, and in which is shown an example of configuration. It should be understood that other configurations may be utilized and structural changes may be made without departing from the scope of the presently claimed subject matter.

1.0 Introduction

In general, a "Wearable Controller For Wrist", as described herein, provides various techniques for measuring user muscles, tendons and bones position and activity on the wrist to interact with and control one or more computing devices. More specifically, the "Wearable Controller For Wrist" provides a wearable device having a set of Mechanomyography (MMG) - based sensor nodes and a set of LEDs and photodiodes for detecting a user's finger-generated signals, an acquisition module, a signal processing module and a module for interacting with and/or controlling general purpose computing devices, applications running on such computing devices, personal music players, physical devices coupled to a computing device, game consoles, televisions or other multimedia devices, virtual devices such as a virtual piano or virtual guitar implemented within a computing environment, etc.

The Wearable Controller For Wrist is implemented in various form factors, including sets of individual sensor nodes (wired or wireless), wearable devices including a plurality of sensor nodes, or articles of clothing including a plurality of sensor nodes. For example, in various embodiments, the Wearable Controller For Wrist is implemented as a wristband, a wristwatch, an article of clothing worn by the user covering the wrist, or any other physical device or collection of devices worn by the user that has sufficient contact with the surface of the user's wrist skin to measure the activity of one or more of the user's tendons, bones, muscles and other body tissues, and their combinations. Further, it should also be understood that a user can wear multiple Wearable Controller For Wrist, with each such Wearable Controller For Wrist being used to interact with either the same or a different computing device, application, or other attached device.

For example, in one embodiment (FIG. 5), the MMG sensor nodes of the Wearable Controller For Wrist are placed in a simple band which is worn around the users' wrist in order to sense muscle activity associated with specific finger and hand gestures, and the initial and final position of muscles, tendons and bones.

The Wearable Controller For Wrist described herein measures in different ways all the movements in the wrist caused by finger gestures using photodiodes and MMG-based sensors. The combination of measurements is analysed and classified by the device and the control command sent to any electronic device .

The MMG-based sensors record every kind of movement determined by the finger gestures. The photodiodes record the reflected LEDs' light on the wrist interior, in order to detect specific muscles, tendons, bones and other body tissues movements, and more generally, every kind of movements. The reflected light rays store information about the movements and about the position of tendons and bones.

A micro-controller or a microprocessor and the related electronics receives the signals from the sensors in order to process them and to send information, such as commands, to other devices. Due to the wide heterogeneity of the human body, preferably a calibration phase precedes the use of the device. The calibration is repeated periodically, in order to ensure the best performances.

Further, the LEDs, photodiodes, or the MMG-based sensors within the Wearable Controller For Wrist can be automatically turned off in order to save power. The micro^¬ controller can be set in rest-mode at the same time. This is particularly useful in wireless implementations of the Wearable Controller For Wrist where an onboard battery (replaceable or chargeable) , fuel cell, photovoltaic power cell, etc., is used to energize selected sensor nodes and associated circuitry. It should also be noted that in wireless implementations of the Wearable Controller For Wrist communication between the Wearable Controller For Wrist and one or more computing systems is accomplished via conventional wireless communications protocols such as, for example, radio frequency (RF) communications, infrared (IR) based communications, Bluetooth, Zigbee, etc. In this case, the Wearable Controller For Wrist includes one or more wireless transmitters, and optionally one or more receivers, for directly interfacing with one or more computing devices, or interfacing with one or more "hubs" that serve as intermediaries for interfacing the Wearable Controller For Wrist with one or more computing devices.

In addition, the gestures analysis can be stopped and enabled with a fixed user command in order to ensure not sending controls while the user is not aware. This command will be linked with the saving power modules.

It should be understood that various embodiments of the Wearable Controller For Wrist are implemented in wired embodiments, such as, for example, by including an integrated USB cable or the like that both provides the power for the sensor nodes and provides a communications pathway between the Wearable Controller For Wrist and one or more computing devices. As in the wireless case, in wired embodiments, the Wearable Controller For Wrist communicates either directly with computing devices, or with those computing devices via an intermediary hub.

In addition, given the various wired and wireless configurations of the Wearable Controller For Wrist described above, it should be understood that hybrid embodiments using various elements of both the wired and wireless configurations are enabled. For example, in one embodiment, a power cable provides operational power, while wireless communications are then enabled by one or more transmitters/receivers integrated into, or coupled to, the Wearable Controller For Wrist. For example, in these types of hybrid embodiments, the power cable (e.g., a power cable connected to a transformer or other power source, or a USB power cable connected to a computing device or transformer, etc.) provides operational power to the Wearable Controller For Wrist, while the wireless transmitters/receivers provide communications between the Wearable Controller For Wrist and one or more computing devices or intermediary hubs within wireless range of the Wearable Controller For Wrist.

Some of the advantages offered by the Wearable Controller For Wrist are that the Wearable Controller For Wrist enables finger movements generated signal to control computing devices, applications, and attached devices with little preparation and setup on the part of the user. In fact, in the simplest embodiment, the user simply generally places the Wearable Controller For Wrist on the wrist that requires no expertise or attention to specific sensor node placement. Further, the Wearable Controller For Wrist allows users to move freely as they would if they were not wearing the device .

The two greatest advantages of controlling a device using the Wearable Controller For Wrist is that it are no more needed the physical contact and the visual contact such as controlling a smartphone trough the touchscreen. The finger movement made without the eye supervision, triggers the Wearable Controller For Wrist to send a control signal or some data to other electronic device.

Additionally the sensors can measure the acceleration of the whole hand in order to track the wrist movement and orientation, useful to implement more controlling patterns.

In view of the above summarized capabilities, and in further view of the following detailed description, it should be understood that the Wearable Controller For Wrist provides users with a "universal" input mechanism that can be used to control any computing device, applications running of computing devices, electronic or mechanical devices coupled to a computing device, or any other electronic device (television, radio, appliance, light switch, etc.) having an appropriate infrastructure or interface for receiving input from a wired or wireless controller. Note also that the use of a small wearable device such as the Wearable Controller For Wrist, which may be under the user's clothes, if desired, provides a mechanism that is unobtrusive (i.e., the user can be using her hands to perform other tasks while using Wearable Controller For Wrist to provide active control of one or more devices) . Further, it should also be appreciated that the control and interface capabilities provided by the Wearable Controller For Wrist are potentially invisible in the sense that a user wearing one or more such controllers can remotely interact with various devices without anyone else being able to see or hear any overt actions by the user to indicate that the user is interacting with such devices.

1.1 System Overview

As noted above, the "Wearable Controller For Wrist" provides a unique device for measuring user' s finger activity in the wrist, monitoring muscles, tendons and bones movements and their combination of movements, for interacting with and controlling one or more computing devices. The processes summarized above are illustrated by the general system diagram of FIG. 1. In particular, the system diagram of FIG. 1 illustrates the interrelationships between functional modules for implementing the MMG-based sensors and photodiodes of the Wearable Controller For Wrist, as described herein.

Furthermore, while the system diagram of FIG. 1 illustrates a high-level view of various steps of the Wearable Controller For Wrist, FIG. 1 is not intended to provide an exhaustive or complete illustration of every possible step of the Wearable Controller For Wrist as described throughout this document.

In general, as illustrated by FIG. 1, the Wearable Controller For Wrist operates by monitoring the movement signals outgoing from the sensors after attaching the sensors of the Wearable Controller For Wrist to the user's wrist skin. As described in greater detail in Section 2.1, the sensors are integrated into a wearable article or device, such as, for example, an wristband, wristwatch, article of clothing, etc.

First part of the scheme FIG. 1 101 represents the "Protocol", a set of gestures recognizable by the device. The user is previously informed about which gestures are associated with control commands. Note that this protocol is described only generally herein. However, it should be understood that in various embodiments, the "Protocol" can change. It can include specific finger or additionally some hand movements.

Once the user performs one of the gestures described in the "Protocol", his muscles, tendons, bones and other body tissues movements become the control signal source 102. Note that the source can be considered singularly or together, it is described in greater details in Section 2.1.

Following the finger movements, the Wearable Controller For Wrist captures the signals generated by the user's finger and to then use those signals to form the basis of a human- computer interface. However, in various embodiments, additional features of the Wearable Controller For Wrist are implemented in order to improve the overall efficiency and utility of the Wearable Controller For Wrist. For example, in one embodiment, FIG. 2, an accelerometer 206 is used to capture the hand movement in the space .

These signals are then sampled, analyzed and interpreted by a control module 104 that interfaces through the sending module 105 with computing devices, or "hubs" acting as intermediaries between the Wearable Controller For Wrist and one or more computing devices. The device Wearable Controller For Wrist then provides HCI for interacting with and controlling one or more computing devices, or devices coupled to or otherwise controlled by such computing devices. Note that once the gesture-generated signals have been interpreted, the device Wearable Controller For Wrist operates like any other HCI device, such as a keyboard, mouse, etc., to interface with various computing devices.

The above-described functional modules and components are employed for implementing various embodiments of the Wearable Controller For Wrist. As summarized above, the Wearable Controller For Wrist provides a unique device for measuring user hand activity associated with particular user gestures or motions for interacting with and controlling one or more computing devices.

The following sections provide a detailed discussion of the operation of various embodiments of the Wearable Controller For Wrist, and of exemplary methods for implementing the functional modules and components described in Section 1 with respect to FIG. 1. In particular, the following sections examples and operational details of various embodiments of the Wearable Controller For Wrist, including: sensing finger movement activity in the wrist using different sensors; wearable devices with different sensors; initial calibration; determination of when to engage or disengage active control using the Wearable Controller For Wrist; and additional embodiments and considerations.

2.0 Sensing Finger Activity at the Level of the Wrist: In general, human skeletal muscles are made up of muscle fibers attached to bone by tendons. These muscles contract to create skeletal movement. To contract a muscle, the brain sends an electrical signal through the nervous system to motor neurons. These motor neurons then transmit electrical impulses known as action potentials to the adjoining muscle fibers, causing the muscle fibers to contract. The combination of a motor neuron and the attached muscle fibers are known as a motor unit. Each muscle is made up of many motor units. During muscle contraction, some subset of the muscle's motor units is activated. The contractions of a motor unit cause a characteristic vibration. The mechanomyographic techniques measure the vibrational response to the contraction measuring the movements of the skin just above the muscle tissues. There are no muscle tissues on the wrist, hence generally the MMG is performed on the arm. The wrist has a considerable number of tendons and bones connected to the muscles. Considering muscles tendons and bones as a linked system, the vibration propagates into the tendons and the bones, such as the tendons and the bones of the wrist. Wearable Controller For Wrist is addressed to this variation of the MMG.

An MMG sensor can measure muscular vibrational signals from the surface of the skin. While easy to apply, an MMG sensor can record a somewhat noisy signal as the vibrational signals must pass through body tissues such as fat and skin before they can be captured at the surface of the skin. Due to the sensitivity of MMG-based sensors required to detect these signals, they also typically detect other vibrational phenomena such as activity from other muscles, skin movement at the wrist, and environmental phenomena. The wearable controller for wrist takes advantege of the me- chanomyograpic sensor sensibility in order to monitor all the static and dynamic phenome- na triggered by finger movements. The dynamic and static phenomena consist in every tendons, bones, muscles and other body tissues movements related to the movements performed by the hand fingers. The Wearable Electronic Controller for Wrist provides a finger movement based HCI .

Such MMG sensors of point are geared inside the Wearable Controller For Wrist in order to measure not just the muscle contraction vibration such the classical MMG does, but all the possible movements and vibration of tendons, muscles, bones and other body tissues, caused by some fixed finger gestures. The sensor used for this purpose are the same usually used for the MMG as described herein, but with a different application. For this reason this kind os sensors in this patent are called MMS-based sensors.

Some particular finger gestures produce particular bones, tendon, muscles and other body tissues movements. These movements produce a signal that can be used as a control signal to put the device in rest-mode. For example if the thumb is moved behind the palm, is easy to be read by the sensors .

2.1 Wearable Devices

As discussed above, the Wearable Controller For Wrist may be implemented in a number of form factors. For example, in one embodiment, as illustrated by FIG. 3, the Wearable Controller For Wrist is implemented as a wearable wristband 302 that is worn on the user's wrist. In general, the wristband 302 has a plurality of MMG-based sensors on the block 301 (FIG. 4, 404) and an array of LEDs and one of photodiodes (FIG.4, 401 and 402) . In general the Wearable Controller For Wrist will firmly contact the skin of the wrist when the band is worn. One configuration consist in having a domed block with all the sensors or part of the sensors inside. It gives stability to the sensor placement thanks to the help given by the wrist bone shape.

In the example of the Wearable Controller For Wrist illustrated by FIG. 3, a wired or wireless interface (not shown) is used to communicate with a computing device that uses the control signals generated by the sensors (401, 402 and 404) . Any conventional means of communication, RF, IR, wired connections, etc., may be used.

Once the wristband has been placed, as fingers perform the gesture, the MMG-based sensors and photodiodes (401 and 404) read the movement and the microprocessor will detect the corresponding activity and transmit the signals. The wristband 302 may transmit raw signals or it may have a processor to perform some initial signal processing, for example rectification, noise reduction, filtering, etc. In yet another embodiment, the wristband may be provided with sufficient processor and storage capacity to perform the computational tasks described below, in which case the wristband 302 recognizes particular user gestures or motions and transmits signals indicating that particular actions corresponding to those gestures of motions are to be performed .

Other wearable device configurations may also be used. For example, the Wearable Controller For Wrist can be implemented as an armband (as illustrated by FIG. 3), a wristwatch (FIG. 5), an article of clothing worn on the wrist by the user, as a module FIG.7 meant to be inserted between a watchband and the wrist, or as any other physical device or collection of devices worn by the user that is sufficient to ensure that one or more sets of sensors are in contact with approximate positions on the user's wrist skin.

It should also be noted that the techniques described herein can also be used with one or more sets of individual MMG- based sensors, such that in FIG. 6 with two blocks of sensors, one on the top 602 and one on the bottom 603.

Note that in general, individual sensor block are either wireless, in the sense that they can communicate wirelessly (RF, IR, etc.) with one or more computing devices or "hubs" as discussed in further detail below, or are wired such that they are coupled to a computing device or hub in order to provide the desired HCI . Similarly, where multiple sensors are integrated into a wearable device such as a wristband, wristwatch, or an article of clothing, for example, those sensors may be coupled to one or more wireless transmitters contained within or coupled to the wearable device such that the overall Wearable Controller For Wrist is wireless in the sense that it provides wireless HCI capabilities to the user based on electrical signals generated by the user's muscles, tendons, bones and other body tissues movements and position related to finger gestures.

2.1.1 Configurations of Sensors

The Wearable Controller For Wrist can take advantage of all the different sensors used in MMG and in all the cases where is needed to measure a mechanical variation. The MMG-based sensors are different kind of sensors, such as force and deformation sensors, accelerometer , pressure sensor or microphone. For example the accelerometer can collect the mechanical stimulation generated by the finger gestures, by putting the accelerometer close to the muscles, tendons, bones and other body tissues, as shown in FIG.3.

The embodiments can be geared with an accelerometer on the external surface of the device in order to measure the hand movements and the vibrational noise. In particular the vibrational noise signal is fundamental in the vibrational signal analysis made by the microprocessor, to detect just the vibrations belonging to the finger and hand movements.

The Wearable Controller For Wrist can measure the mechanical signals of tendons, bones and muscles with a microphone placed attached to the skin of the wrist or in any other way that makes the mechanical signal pass through and arrives to the microphone. The microphone can be a condenser a piezoelectric or a MEMS microphone or any other kind of microphone suitable for the form factors and power characteristic of the device. The microphone can be coupled with other sensors, for example an accelerometer in order to obtain the best performances in terms of signal acquisition.

The pressure sensors work in the same way as the above- mentioned microphone, and can be coupled with other sensors to obtain the best performances.

The photodiodes configuration depends on the LEDs configuration. For example one configuration (FIG.3) consists in having an array of LEDs pointing collimated light rays to the tendons and bones of the wrist. The light emitted by the LEDs is reflected by the tendons and bones of the wrist. The photodiodes receive the reflected light. The light levels in each photodiodes can give clues about static and dynamic behavior of tendons, muscles and bones.

Further, another configuration for the photodiodes and LEDs is to put the photodiodes in the way they receive the transmitted light through the tendons. For this configuration can be used just one LED and one photodiode. The Wearable Controller For Wrist can use all of the sensors configuration mentioned in previous paragraphs, or a combination thereof.

Additionally, the Wearable Controller For Wrist can be provided with an accelerometer for the arm and hand movement monitoring in order to increase the possible movements for the control of other devices. For example the finger can be used in the same way the buttons in the mouse are used while moving the arm can be like moving the mouse.

Finally, each sensor node needs a power source to operate. In various embodiments, the power source is integrated into the sensor node, such as by using a battery, fuel cell, photovoltaic power cell, etc. In related embodiments FIG. 4, the power source is provided via a direct power connection from an external power source 405. Note also that in the case that multiple sensor nodes are integrated into a device such as an armband, wristwatch, or article of clothing, the "external" power source can be integrated into that device. For example, in the case of an armband type Wearable Controller For Wrist, one or more onboard batteries can be included in the armband itself to apply power to the sensor nodes contained in that armband. In addition, as noted above, and as discussed in further detail in Section 2.5, in various embodiments, unneeded sensor nodes (or individual electrode pairs) are selectively enabled or disabled via a power management nodule in order to conserve power. Such embodiments are especially valuable in the case where the onboard power source is limited.

2.1.2 Configuration of the Calculation Unit

The Wearable Controller For Wrist is equipped with a calculation unit, embodied for example by a micro-controller able to perform the signals acquisition, signal processing and sending the control with an appropriate internal or external module. The micro-controller is provided of all the electronics needed to make it work.

The micro-controller can be replaced by a microprocessor with all the missing part of the microprocessor needed to let the microprocessor acquire, process and send the signals.

The calculation unit provides an ADC depending on the sensors output. If the sensors output is an analog signal then the ADC is provided, otherwise a port capable to receive the digital output of the sensors is provided.

The calculation unit provides or it is connected to a sending module able to send the control to an external device via wired or wireless connection.

2.2 Calibration

In general, it is assumed that users of the Wearable Controller For Wrist will not place the device (or individual sensor nodes) in exactly the same place relative to specific bones and tendons each time that the user wears the Wearable Controller For Wrist, further each person has a different physiognomy . Consequently, one of the advantages of the Wearable Controller For Wrist is the capability to rapidly calibrate.

Calibration can be accomplished in various ways. For example, in one embodiment, calibration is accomplished by connecting the Wearable Controller For Wrist to a main station such as a computer or a smartphone, with the calibration software installed. The software ask to the user to make some finger gestures wearing the Wearable Controller For Wrist, and collect the parameter useful for the device to recognize the gestures. Once finished the calibration, the Wearable Controller For Wrist receives the parameters and thus is ready to work.

Note that in training or retraining the classification system, given the limited number of muscles involved in such gestures, in various embodiments, the classification system is trained or calibrated by using only a subset of recognized gestures or motions in order to find matching points from previously built models.

Further, in various embodiments, this calibration is continually or periodically performed as the system observes the user's actions. Note that periodically or continuously performing the calibration serves at least two purposes. First, repeating the calibration process may help to further refine the gesture model, and second, repeating the calibration process will help to adjust for minor positional movements of the Wearable Controller For Wrist on the user's body .

In addition, since the Wearable Controller For Wrist is worn by the user, calibration data can be collected even when the user is not actively engaged in using the Wearable Controller For Wrist for HCI purposes. This additional calibration data collection allows the system to statistically model likely gestures or movements, and given enough time, the system can infer the gestures or movements that the user is performing.

Further, in various embodiments, the user is provided with various mechanisms for performing user-defined gestures or sequences of gestures, and then assigning particular actions to those gestures, either from a pre-defined list of actions, or from user defined actions or macros. In this case, the training described above is the same, with the difference simply being the particular command or macro that is being mapped to the predefined or user-defined gesture.

2.3 Fingers Gestures

In various embodiments, an important finger gesture such as the action of tapping a finger against the thumb will be provided by protocol. Such gesture is recognized by the two sets of sensors, MMG and photodiode. Together they provide the information that the gesture has been performed, and the position of the tendon involved in the action, in order to identify the linked finger. These pieces of information are given by a trained classifier after a brief signal filtering. The classifier is trained during the calibration.

In some embodiments can be used the four finger pushing against the thumb as four buttons ON/OFF. It can be seen as having for bits to work with. The control commands can be one or a combination of more buttons. The possible commands are therefore infinite, limited just by the usability. 2.4 On-Board Analysis

The Wearable Controller For Wrist presents a micro^¬ controller and related electronics able to read the sensors signal, filter it and analyze it in order to perform the gesture recognition and classification. The micro-controller receives the parameter for the classification during the calibration. The calibration can be made by the microcontroller itself or by another computer device connected to the micro-controller.

Further, The Wearable Controller For Wrist presents a micro^¬ controller and a module for the communication with other electronic devices. The micro-controller controls the communication module.

As described above, when using the Wearable Controller For Wrist large numbers of individual MMG-based sensors or sensor nods and with LEDs and photodiodes are integrated into a wearable device and are placed on the wrist skin. During resting periods, some of the components can be switched off to save battery energy. Consequently, in various embodiments, a process is useful to determine which sensor nodes should be enabled or active, and which sensor nodes should be disabled or inactive.

Signal filtering and classification are made by the micro^¬ controller inside the sensors block 403 FIG.4. It can be update and programmed by a computer during a first calibration phase, or by itself.

The energy for the micro-controller is provided by an internal battery or by a wired connection. The downstream receiver can also direct that the initial calibration phase is repeated to reselect the appropriate individual sensors or sensor nodes for amplification and transmission .

However, it should also be understood that the overall decision can be made using relatively simple logic that does not require significant computing power.

In general, various embodiments of the Wearable Controller For Wrist use different techniques for deciding whether a particular signal is sufficiently important to transmit. These techniques include an evaluation of sensor or sensor node positional information, and available signal information. In general, each of these techniques can be used either separately, or in any desired combination

2.5 Raw Signal Analysis

Some analyses can be performed on the raw sensor signals to determine gesture-recognition system. These analyses are generally computationally simple, and are thus suited to being performed on the microprocessors that are built in to each wireless sensor node or into an integrated device such as the aforementioned wristband. However, as discussed above, such processing of raw signals can also be on a downstream receiver or processor during an initial high- power calibration phase. Examples of raw-signal analyses that can provide indications of signal relevance include measures of RMS amplitude of the finger-generated signals and measured power bands .

Signals within a known range of amplitude are most likely to be informative. In this case, a very simple logic test to determine whether a measured signal is within a simple range can be included in the individual sensor nodes, such as, for example, by adding a simple logic gate to the analog to digital converter or to the digital signal processing module. Similarly, an analysis of individual frequency bands of measured signals can also be performed using very simple computational capabilities. For example, in the case that a particular signal where one or more individual frequency bands falls outside a "reasonable" or expected range in known frequency bands are unlikely to be informative.

2.6 Automated Feedback for Placement and Setup

As discussed herein, the sensors of the Wearable Controller For Wrist are applied coarsely, without an expert present to ensure precise placement. For example, in the aforementioned wristband configuration, an end-user attaches the armband on the wrist, such that sensors are quite precisely located over the wrist skin. Therefore, in various embodiments, in order to allow rapid placement of electrodes by individual users while still ensuring adequate signal quality, automated feedback is provided the user to assist him in quickly adjusting the Wearable Controller For Wrist (or individual sensor nodes) to ensure adequate signal strength and quality for gesture-based HCI purposes.

Given this approach, the basic process of "installing" the Wearable Controller For Wrist can be implemented in a number of user- friendly ways. For example, initial positioning of the Wearable Controller For Wrist can be accomplished using a process such as the simple three step process illustrated below :

1) The user puts the wristband, wristwatch, or other Wearable Controller For Wrist in a coarsely approximate location where the device is intended to be placed. For example, would be coarsely placed somewhere on the users wrist. The system would then be activated or turned on (unless the system was already activated or turned on) ;

2) The user would then make coarse manipulations to the initial positioning of the device, such as, for example, rotating the wristband, while receiving simple feedback about signal quality (such as a simple "meter" on a computer screen, a sound emanating from the device, or speech cues to direct the user with respect to specific motions);

3) Finally, the user would make fine adjustments to the position or orientation of the device (e.g. rotate and/or move the position of the Wearable Controller For Wrist) until a simple goal is achieved, such as "meter goes above level 5," "sound stops", "vibration stops", etc.

In various embodiments, the feedback provided to the user during this simple adjustment process is visual (e.g., a bar or meter on a computer screen, on a portable music player, or on a small on-board LCD or series of one or more LEDs or lights), auditory (e.g., a noise that gets quieter as signal quality increases, or a voice saying "keep turning, keep turning, perfect!") , or haptic (e.g., the Wearable Controller For Wrist vibrates or electrically stimulates one or more areas of the user's skin while the user should continue to adjust the device and stops vibrating or electrically stimulating the user's skin when the signal quality is adequate.

Further the sensors block will be designed with a shape that helps the user to find the right position, and to keep the wristband in that position during time. For example the sensor block can be as FIG. 3, 301 illustrates. All of these feedback modalities require some measure of "how good" the placement of the system is at any given time. Therefore, in various embodiments, two general categories of metrics for determining signal quality during adjustment are provided. These general categories include raw signal analysis and decoding power, as described above with respect to multiplexing in Sections 2.6

In general, these same metrics can be used to provide real^¬ time feedback about placement of the Wearable Controller For Wrist on the user's body. For example, the overall system can direct the user to rotate the wristband until the RMS amplitude of all electrodes is maximized. Similarly, the overall system can direct the user to rotate his wristband while wiggling his fingers, until the gesture-recognizer's confidence in decoding the gestures is maximized. Again, any of the visual, audible, or haptic techniques described herein can be used in directing the user to continue positioning the Wearable Controller For Wrist (or individual sensor nodes) .

2.7 Determination of When to Engage or Disengage Active Control

One of the problems of a system that is always available and always recognizing user movements through vibrational signals is that system needs to be able to differentiate between normal everyday gestures performed not for interacting with the system (i.e., using the hands for everyday tasks) and explicit commands issued for the system to recognize. In conventional HCI literature, this problem is sometimes referred to "avoiding the Midas touch" where every user action is interpreted as an intentional command. In order to avoid this problem, in various embodiments, the Wearable Controller For Wrist uses gross movements that are robustly recognized, and are unlikely to be confused with everyday gestures, to engage and disengage the system, or to inform the system that a subsequent gesture will correspond to a command. With respect to the use of an wristband-based Wearable Controller For Wrist, examples of such movements include thumb folding, or a series of actions that are not likely to be performed normally, such as tapping the thumb against the palm twice, or any other predefined or user- defined motion or gesture desired.

2.8 Additional Embodiments and Considerations

As summarized in Section 1.1 with respect to FIG. 1, the Wearable Electromyography-Based Controller provides HCI capabilities based on signals generated by the body in response to the contraction of one or more muscles connected to the fingers and of the hand. As such, it should be clear that the Wearable Controller For Wrist is capable of being used for any of a number of purposes. For example, these purposes include interaction with conventional application such as interacting with a computer operating system by moving a cursor and directing simple object selection operations (similar to using a computer mouse to select an object), wired or wireless game controllers for interacting with game consoles or with video games operating on such consoles, control of pan-tilt- zoom cameras, interaction with home automation systems such as audio, video, or lighting controls, etc.

Other obvious uses for the Wearable Controller For Wrist include local or remote control of robots or robotic devices, such as, for example, using a glove with embedded sensor nodes on the wrist to control a remote robotic hand wielding tools or medical instruments.

Additionally the couple LED and photodiode can be used to get information about the user heart rate. It is possible to monitor the different oxygenations of the blood by analysing the photodiode signal.

As already mentioned before, the Wearable Controller For Wrist can be geared with an additional accelerometer in order to measure the movements of the whole hand in the space, in order to have more information to send.

3.0 Exemplary Operating Environments

The Wearable Controller For Wrist described herein is operational for interfacing with, controlling, or otherwise interacting with numerous types of general purpose or special purpose computing system environments or configurations, or with devices attached or coupled to such computing devices. For example the wristwatch can act as a "hub" in the case, as a wireless intermediary between one or more of the sensor nodes and a second device.

For example, Wearable Controller For Wrist can work communicating with a simplified computing device. Such computing devices can be typically be found in devices having at least some minimum computational capability, including, but not limited to, personal computers, server computers, hand-held computing devices, laptop or mobile computers, communications devices such as cell phones and PDA's, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, video media players, in-vehicle computing systems (e.g., automotive computer system), etc.

As noted above, computing devices such as those described herein operate either in response to user gestures recognized via one or more Wearable Controller For Wrist. However, in various embodiments, such computing devices also provide computing power for operations such as the initial calibration described in Section 2.2. In addition, such computing devices may also act as hubs or intermediaries to facilitate communications between the Wearable Controller For Wrist and one or more other computing devices or attached mechanisms.

In general, such computing devices include at least some minimum computational capability along with some way to send and receive data. In particular, the computational capability is generally given by one or more processing unit(s), and may also include one or more GPUs . Note that that the processing unit(s) of the general computing device of may be specialized microprocessors, such as a DSP, a VLIW, or other micro-controller, or can be conventional CPUs having one or more processing cores, including specialized GPU-based cores in a multi-core CPU.

In addition, the computing device may also include other components, such as, for example, a communications interface. The computing device may also include one or more conventional computer input devices (such as a microphone or microphone array for receiving voice inputs) . The simplified computing device may also include other optional components, such as, for example one or more conventional computer output devices (such as audio and/or video output devices) . Finally, the computing device may also include storage that is either removable and/or non-removable. Note that typical communications interfaces, input devices, output devices, and storage devices for general-purpose computers are well known to those skilled in the art, and will not be described in detail herein.

4.0 Preferred Configurations and Improvements

A preferred configuration involves the use of piezoelectric microphone MMG-based sensors 701, 702. This kind of sensors can be used for detecting skin deformation caused by tendons movement. In FIG. 7 a preferred configuration of the sensors is shown, which comprises a set of piezoelectric microphone MMG-based sensors. Each sensor is inclined with respect to the module 703 , 704. With this configuration, each microphone focuses on a smaller region of the wrist skin and is more sensible to the deformation, with respect to a configuration where each sensor is laying parallel to the skin and to the module (703, 704) . Therefore this slanted or inclined configuration of the piezoelectric microphone MMG- based sensors improves the spatial resolution and the sensibility of the device.

A preferred configuration for using the optical sensors is the one shown in FIG. 4. The received optical signal is proportional to the tendons distance. The amplitude of this signal gives precise information about the static position of the tendons. In case the tendons are close to the optical sensors the skin in-between results compressed, and this results in a small signal. In case the tendons are far from the optical sensors the optical signal is higher.

A preferred signal analysis considers all the signals coming from the sensors during each finger movement and gesture. After band-pass filtering the signals in case the amount of data is too great, it can be analyzed by a real-time PCA. The PCA permits to reduce the amount of data and focus on the relevant signals. The signals are then analyzed by a feature extractor. The feature extractor analyzes the signals in order to obtain a set of features that robustly describe the signals and that can be compared with other signal features coming from other finger movements and gestures. The comparison is usually made in order to classify the signal and recognize the associated finger gesture. The feature can be a time domain feature

(amplitude, ratio between the signal amplitude and other prerecorded signals amplitudes, number of lobes, number of zero-crossings, time length of each lobe, time length of each movement, correlation with other pre-recorded signals, difference between the signal and other pre-recorded signals) . The feature can be a frequency domain feature

(power of the spectrum, power of a range of frequencies, ratio between amplitude of certain range of frequencies, wavelet features) .

A preferred system for power management is described as follows. The embodiment is normally set in sleeping mode, the signal acquisition is not active, the microcontroller is set in low power consumption. The microcontroller wakes up from the sleeping mode thanks to an external signal triggered by a mechanical button, which is preferably placed in the part of the device which is in contact with the wrist. When the user's wrist flexes, said button is pressed and activates the signal. The flexion movement of the user's wrist increases the pressure of the device itself onto the wrist skin, triggering the activation of the button. With said power management system, two problems are prevented: high power consumption and accidental gestures that the user might otherwise perform involuntarily which could cause wrong commands .

A preferred embodiment comprises a module containing the electronic parts and the sensors. Said module is shown in FIG. 7 and can be covered by a layer (not shown in the figure) made of silicon or fabric that highly improves the comfortability . In order for a user to wear the module, it can be attached for example to an existing watchband or bracelet. This improves the usability of the Wearable Controller For Wrist, because a user is not requested to replace his wristwatch, since he can just attach the module to its own wristwatch and hide it under its watchband.

The foregoing description of the Wearable Controller For Wrist has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the claimed subject matter to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. Further, it should be noted that any or all of the aforementioned alternate embodiments may be used in any combination desired to form additional hybrid embodiments of the Wearable Controller For Wrist. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.

Claims

1. A human-computer interface, comprising: a wearable device (302, 503, 603) having a plurality of sensor nodes (103, 301, 601, 602, 501, 401, 402, 404), wherein each sensor node includes one or more Mechanomyography (MMG) sensors, and additionally optic sensors; a module (103, 403) for automatically measuring finger gestures generated signals using one or more of said sensors; a module (401, 402 and 404) for automatically determining the position and the movement at the wrist level that correspond to one or more specific user finger gestures; and a module (403, 405) for causing one or more computing devices to automatically execute one or more specific commands corresponding to one or more of the specific user gestures.

2. The human-computer interface of claim 1, wherein the one or more Mechanomyography (MMG) sensors are selected from the group comprising strain gauge sensors, force sensors, piezoresistive sensors, deformation sensors, pressure sensors and microphones, and where the optic sensors comprise photodiodes.

3. The human-computer interface of claim 1 or 2, comprising a set of piezoelectric microphone MMG-based sensors (701, 702), each mounted inclined with respect to a module (703, 704) .

4. The human-computer interface of any one of claims 1 to 3, wherein a calibration phase is provided which automatically identifies the parameters useful to run a software installed in the module (104, 601, 301, 501, 403) or in one or more external computing devices, the software receiving the signals and identifying the parameter for the training following a protocol of specific finger gestures.

5. The human-computer interface of claim 4, wherein the calibration phase involves the user performing one or more specific finger gestures (101) as part of a training phase for the calibration phase.

6. The human-computer interface of any one of claims 1 to 5, wherein the sensors measure the signals that are associated with one or more of the specific user gestures.

7. The human-computer interface of any one of claims 1 to 6, further comprising a module for automatically determining the position of the signal source on the surface of the skin of the user's wrist in order to identify which finger moved and how.

8. The human-computer interface of any one of claims 1 to 7, wherein the wearable device communicates wirelessly or wiredly (105) with one or more computing devices.

9. The human-computer interface of any one of the preceding claims, comprising a button preferably placed in the part of the device destined to contact a user's wrist so as to be triggered by the user flexing the wrist and cause the activation of the device from a sleeping, power-saving mode to an active acquisition mode.

10. A process for sending commands to one or more computing devices, comprising:

- providing a human-computer interface according to any one of claims 1 to 9;

positioning the wearable device (302,503, 603) in contact with the surface of a user's wrist skin, the one or more Mechanomyography (MMG) sensors (103, 205, 301, 601, 602, 501, 401, 402, 404 and additionally optic sensors measuring the state and the activity of the different body tissues at the user's wrist;

automatically evaluating (103) gesture-generated signals of the user measured via the one or more of Mechanomyography (MMG) sensors and optic sensors to automatically identify (104) one or more specific gestures of the user from a predefined set of gestures; - automatically directing (105) one or more computing devices to execute one or more specific commands corresponding to one or more of the identified gestures.

11. The process of claim 10, further comprising performing an initial calibration phase which evaluates gesture generated signals associated with a subset of user finger gestures to determine expected signals during the automatic evaluation phase.

12. The process of claim 10 or 11, wherein commands associated with one or more of the gestures of the set of gestures can be user definable.

13. The process of any one of claims 10 to 12, comprising a signal analysis considering all signals coming from the sensors during each finger movement and gesture, band-pass filtering said signals to limit the data to a predetermined amount, and analyzing the signals by means of a feature extractor .

14. The process of claim 13, wherein the feature extractor analyzes the signals in order to obtain a set of features describing the signals to be compared with other signal features coming from other finger movements and gestures.

15. The process of claim 14, wherein a feature of the set of features is a time domain feature or a frequency domain feature .

16. A system for providing a human-computer interface (HCI), comprising :

- a user-wearable device (302,503 and 603) having one or more Mechanomyography (MMG) sensors (103, 205, 301, 601, 602, 501, 401, 402, 404) and additionally optical sensors, said user-wearable device being adapted to be placed in use on the surface of the user's wrist skin; an automated calibration process which maps finger gestures generated signals corresponding to one or more specific user finger gestures to one or more specific commands, said finger gestures generated signals being measured by one or more of the Mechanomyography (MMG)— sensors and optical sensors;

an automated process for disabling some of said sensors during rest;

- an automated process (104) for evaluating one or more user gestures associated with the signals captured by the Mechanomyography (MMG) sensors (103, 205, 301, 601, 602, 501, 401, 402, 404) and additionally optical sensors to identify one or more commands associated with those user gestures; and

a process (105) for transmitting specific commands associated with one or more specific user gestures to one or more computing devices.

17. The system of claim 16, wherein the user-wearable device includes a wireless or wired interface to the one or more computing devices.

18. The system of claim 16 or 17, wherein the one or more Mechanomyography (MMG) sensors are selected from the group comprising strain gauge sensors, force sensors, piezoresistive sensors, deformation sensors, pressure sensors and microphones, and where the optic sensors comprise photodiodes.