WO2023239361A1 - Angle prévu basé sur des valeurs d'emg du muscle - Google Patents

Angle prévu basé sur des valeurs d'emg du muscle Download PDF

Info

Publication number
WO2023239361A1
WO2023239361A1 PCT/US2022/032745 US2022032745W WO2023239361A1 WO 2023239361 A1 WO2023239361 A1 WO 2023239361A1 US 2022032745 W US2022032745 W US 2022032745W WO 2023239361 A1 WO2023239361 A1 WO 2023239361A1
Authority
WO
WIPO (PCT)
Prior art keywords
forearm
muscle
sensor
emg
arm
Prior art date
Application number
PCT/US2022/032745
Other languages
English (en)
Inventor
Hsiang-Ta KE
Yenting KUO
Yi-Hsien Lin
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to PCT/US2022/032745 priority Critical patent/WO2023239361A1/fr
Publication of WO2023239361A1 publication Critical patent/WO2023239361A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/296—Bioelectric electrodes therefor specially adapted for particular uses for electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/389—Electromyography [EMG]
    • A61B5/397—Analysis of electromyograms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/486—Bio-feedback
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813—Specially adapted to be attached to a specific body part
    • A61B5/6824—Arm or wrist
    • D—TEXTILES; PAPER
    • D04—BRAIDING; LACE-MAKING; KNITTING; TRIMMINGS; NON-WOVEN FABRICS
    • D04H—MAKING TEXTILE FABRICS, e.g. FROM FIBRES OR FILAMENTARY MATERIAL; FABRICS MADE BY SUCH PROCESSES OR APPARATUS, e.g. FELTS, NON-WOVEN FABRICS; COTTON-WOOL; WADDING ; NON-WOVEN FABRICS FROM STAPLE FIBRES, FILAMENTS OR YARNS, BONDED WITH AT LEAST ONE WEB-LIKE MATERIAL DURING THEIR CONSOLIDATION
    • D04H1/00—Non-woven fabrics formed wholly or mainly of staple fibres or like relatively short fibres
    • D04H1/70—Non-woven fabrics formed wholly or mainly of staple fibres or like relatively short fibres characterised by the method of forming fleeces or layers, e.g. reorientation of fibres
    • D04H1/72—Non-woven fabrics formed wholly or mainly of staple fibres or like relatively short fibres characterised by the method of forming fleeces or layers, e.g. reorientation of fibres the fibres being randomly arranged
    • D04H1/728—Non-woven fabrics formed wholly or mainly of staple fibres or like relatively short fibres characterised by the method of forming fleeces or layers, e.g. reorientation of fibres the fibres being randomly arranged by electro-spinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • D—TEXTILES; PAPER
    • D01—NATURAL OR MAN-MADE THREADS OR FIBRES; SPINNING
    • D01F—CHEMICAL FEATURES IN THE MANUFACTURE OF ARTIFICIAL FILAMENTS, THREADS, FIBRES, BRISTLES OR RIBBONS; APPARATUS SPECIALLY ADAPTED FOR THE MANUFACTURE OF CARBON FILAMENTS
    • D01F8/00—Conjugated, i.e. bi- or multicomponent, artificial filaments or the like; Manufacture thereof
    • D01F8/04—Conjugated, i.e. bi- or multicomponent, artificial filaments or the like; Manufacture thereof from synthetic polymers

Definitions

  • Extended reality (XR) technologies include virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies, and quite literally extend the reality that users experience.
  • Some types of XR technologies can employ head-mountable displays (HMDs), which are display devices that can be worn on the head.
  • HMDs head-mountable displays
  • VR technologies a user is immersed in an entirely virtual world
  • AR technologies a user’s direct or indirect view of the physical, real-world environment is augmented.
  • MR, or hybrid reality technologies, a user experiences the merging of real and virtual worlds.
  • FIGs. 1 and 2A-2B are diagrams of an example muscle sensor and haptic feedback sleeve.
  • FIGs. 3A-3H are diagrams of example placement of muscle electromyographic (EMG) sensors and haptic actuators of a muscle sensor and haptic feedback sleeve in relation to different muscles.
  • EMG muscle electromyographic
  • FIGs. 4A-4C are diagrams of example angles that can be calculated from muscle EMG sensor values.
  • FIGs. 5A and 5B are diagrams of example arm avatars that can be rendered and displayed using angles calculated from muscle EMG sensor values.
  • FIGs. 6A-6C are flowcharts of example methods for calculating angles from muscle EMG sensor values.
  • FIGs. 7 is a diagram of an example non-transitory computer- readable storage medium storing program code for calculating angles from muscle EMG sensor values for rendering and display of an arm avatar.
  • FIG. 8A is a diagram of an example muscle sensor and haptic feedback sleeve as to its constituent layers.
  • FIG. 8B is a diagram of an example nanofiber layer that can implement each constituent layer.
  • FIG. 9A is a diagram of an example nanofiber yarn of a nanofiber layer in which muscle EMG sensors and/or haptic actuators of a muscle sensor and haptic feedback sleeve are disposed.
  • FIG. 9B is a diagram of an example nanofiber array that can realize the EMG sensors and/or the haptic actuators.
  • FIG. 10 is a diagram of an example nanofiber yarn of a nanofiber layer that that implements a astatic electric and/or thermoelectric layer of a muscle sensor and haptic feedback sleeve.
  • FIGs. 11 A-11 D are diagrams depicting how an example static electric layer of a muscle sensor and haptic feedback sleeve can generate electricity to power the sleeve.
  • FIG. 12 is a diagram depicting how an example thermoelectric layer of a muscle sensor and haptic feedback sleeve can generate electricity to power the sleeve.
  • an extended reality (XR) technology to extend the reality experienced by a user.
  • Images may be displayed, such as on a head-mountable display (HMD) worn by a user, to immerse the user within an XR environment, be it a virtual reality (VR), augmented reality (AR), a mixed reality (MR), or another type of XR environment.
  • VR virtual reality
  • AR augmented reality
  • MR mixed reality
  • an arm avatar may be displayed within an VR or other XR environment, in correspondence with a user’s actual arm.
  • Cameras or sensors may be employed in this respect to detect movement and positioning of the user’s actual arm, including the forearm, hand, and fingers (including the thumb). Based on this information, the arm avatar may then be displayed within an XR environment in correspondence with the actual movement and positioning of the user’s arm in the real (i.e., physical) world.
  • the arm avatar may be displayed from a first-person perspective for viewing by the user him or herself, or may be displayed from a third-person perspective for viewing by the user and/or other users.
  • Some users may be differently physically abled, preventing them from having arm avatars rendered in correspondence with actual motion and positioning of their arms. For instance, some users may have been born without a hand, or may have been born without a forearm and hand. As another example, users may have been amputated at the wrist, such that they do not have a hand, or may have been amputated at the elbow, such that they do not have a forearm nor a hand. Users may have been born without one or multiple fingers (including the thumb), or have had one or multiple fingers amputated. Techniques that detect movement and positioning of an actual hand and/or forearm for rendering and displaying a corresponding arm avatar therefore cannot be used for such users.
  • a muscle sensor and haptic feedback sleeve includes various muscle EMG sensors and haptic actuators.
  • the sleeve can include an upper arm portion for wearing on the upper arm between the shoulder and elbow, and a forearm (i.e., lower arm) portion for wearing on the forearm between the elbow and wrist.
  • a user having a forearm but not a hand, or a user having a forearm and a hand (with or without fingers) may wear the upper arm portion or both the upper arm and forearm portions, whereas a user having neither a forearm nor a hand wears just the upper arm portion and not the forearm portion.
  • the calculated angle is an intended angle in that, in the case in which the user does not have a forearm, the angle is not an actual angle of the forearm in the real world. In the case in which the user has a forearm, the angle is not an actual angle in that the forearm itself is not detected.
  • EMG sensor values from the upper arm portion can therefore be used to calculate the intended angle of the forearm that the user may or may not in actuality have.
  • An arm avatar including a forearm then can still be rendered and displayed.
  • the haptic actuators in the upper arm portion permit feedback to the muscles of the upper arm in correspondence with virtual movement and positioning of the forearm in the XR environment.
  • the muscle EMG sensors and the haptic actuators in the forearm portion permit an intended angle of the hand, an intended rotational angle of the wrist (and thus the hand), and an intended angle of each finger to be calculated.
  • angles are intended angles in that, in the case in which the user does not have a hand or one or multiple fingers, the angles are not actual angles of the hand or the fingers in the real world. In the case in which the user has a hand and all fingers, the angles are actual angles in that the hand and the fingers themselves are not detected.
  • FIG. 1 shows a block diagram of an example muscle sensor and haptic feedback sleeve 100.
  • the sleeve 100 includes an upper arm flexible material portion 102 and a lower arm (i.e., forearm) flexible material portion 104.
  • the flexible material portions 102 and 104 may be removably attached to one another, using a zipper, hook-and-loop fasteners, snaps, or in another manner. Specifics regarding an implementation of the makeup of layers the material portions 102 and 104 are described later in the detailed description.
  • the upper arm flexible material portion 102 has a shape to fit or wrap around the arm between the shoulder and the elbow of the arm.
  • the lower arm flexible material portion 104 has a shape to fit or wrap around the arm between the elbow and the wrist of the arm, and thus to fit or wrap around the forearm.
  • the flexible material portions 102 and 104 may be independently worn and used. That is, just the material portion 102 may be worn and used, just the material portion 104 may be worn and used, or both portions 102 and 104 may be worn and used.
  • the upper arm flexible material portion 102 includes respective bicep and tricep muscle EMG sensors 106A and 108A, respective bicep and tricep haptic actuators 110A and 112A, and a tilt sensor 114.
  • the EMG sensors 106A and 108B are disposed within the flexible material portion 102 such that they are adjacent to the bicep and tricep muscles, respectively, when the material portion 102 is being worn.
  • the haptic actuators 110A and 112A are similarly disposed within the material portion 102 such that they are adjacent to the bicep and tricep muscles, respectively, when the portion 102 is being worn.
  • the tilt sensor 114 is also disposed within the material portion 102.
  • the lower arm flexible material portion 104 includes respective upper forearm flexor and extensor muscle EMG sensors 106B and 108B and respective upper forearm flexor and extensor haptic actuators 110B and 112B.
  • the EMG sensors 106B and 108B are disposed within the flexible material portion 104 such that they are adjacent to the upper forearm flexor and extensor muscles, respectively, when the material portion 104 is being worn.
  • the haptic actuators 11 OB and 112B are similarly disposed within the material portion 104 such that they are adjacent to the upper forearm flexor and extensor muscles, respectively, when the portion 104 is being worn.
  • the lower arm flexible material portion 104 includes respective forearm pronator and supinator muscle EMG sensors 106C and 108C and respective forearm pronator and supinator haptic actuators 110C and 112C.
  • the EMG sensors 106C and 108C are disposed within the flexible material portion 104 such that they are adjacent to the upper forearm pronator and supinator muscles, respectively, when the material portion 104 is being worn.
  • the haptic actuators 110C and 112C are similarly disposed within the material portion 104 such that they are adjacent to the forearm pronator and supinator muscles, respectively, when the portion 104 is being worn.
  • the lower arm flexible material portion 104 includes, for each finger (including the thumb) respective lower forearm flexor and extensor muscle EMG sensors 106D and 108D and respective lower forearm flexor and extensor haptic actuators 110D and 112D. Therefore, there are five sets of EMG sensors 106D and 108D and haptic actuators 110D and 112D.
  • the EMG sensors 106D and 108D are disposed within the flexible material portion 104 such that they are adjacent to the lower forearm flexor and extensor muscles, respectively, when the material portion 104 is being worn.
  • FIG. 2A shows example wearing of the muscle sensor and haptic feedback sleeve 100 by a user who does not have a hand and thus also does not have any fingers.
  • the upper arm flexible material portion 102 is fitted around the user’s upper arm 202, and the lower arm flexible material portion 104 is fitted around the user’s forearm 204.
  • the flexible material portions 102 and 104 can be removably attached to each another as depicted in the example.
  • FIG. 2B shows example wearing of the muscle sensor and haptic feedback sleeve 100 by a user who does not a forearm, and thus also does not have a hand or any fingers.
  • the upper arm flexible material portion 102 is fitted around the user’s upper arm 202.
  • the lower arm flexible material portion 104 is not used, and therefore is removed from the upper arm flexible material portion 102 and is not depicted.
  • FIG. 3A shows example placement of the EMG sensors 106A and 108A, the haptic actuators 110A and 112A, and the tilt sensor 114 in relation to muscles 302A, 304A, and 306.
  • the bicep muscle EMG sensor 106A and the bicep haptic actuator 110A are adjacent to the bicep muscle 302A, whereas the tricep muscle EMG sensor 108A and the tricep haptic actuator 112A are adjacent to the tricep muscle 304A.
  • the tilt sensor 114 is depicted as being adjacent to the bicep muscle 302A, but can instead be adjacent to the tricep muscle 304A or the brachialis muscle 306 between the bicep and tricep muscles 302A and 304A.
  • a user uses his or her bicep and tricep muscles 302A and 304A in order to raise (flex) or lower (extend) the forearm 204 relative to the upper arm 202. Even if the user does not have a forearm 204, the muscles 302A and 304A are still engaged if the user intends or tries to position the forearm 204 in such a phantom motion.
  • a bicep muscle EMG value from the bicep muscle EMG sensor 106A can be used to calculate an intended flexion angle of the forearm 204 relative to the upper arm 202
  • a tricep muscle EMG value from the tricep muscle EMG sensor 108A can be used to calculate an intended extension angle of the forearm 204 relative to the upper arm 202.
  • the haptic actuators 110A and 112A can be used in the reverse manner, to convey to the user the raising (flexing) or lowering (extending) of the forearm 204 relative to the upper arm 202. Such haptic actuators 110A and 112A may not be used if the user actually has a forearm 204. If the virtual angle of the forearm of the corresponding arm avatar relative to the upper arm 202 in an XR environment changes, the haptic actuators 110A and 112A can stimulate the muscles 302A and 304A to convey such movement even though the user lacks a forearm 204. The angle is a virtual angle because it originates within the XR environment, such as with respect to the arm avatar.
  • the bicep haptic actuator 110A is used to convey a virtual flexion angle of the forearm 204 relative to the upper arm 202.
  • the tricep haptic actuator 112A is used to convey a virtual extension angle of the forearm 204 relative to the upper arm 202.
  • the tilt sensor 114 may be an inclinometer or other type of tilt sensor.
  • the tilt sensor 114 provides a tilt sensor value depending on the degree to which a user has raised or lowered his or her upper arm 202.
  • the tilt sensor value from the tilt sensor 114 can be thus be used to calculate the angle of the upper arm 202, such as relative to the side of the user’s body or the user’s shoulder.
  • FIG. 3B shows example placement of the EMG sensors 106B and 108B and the haptic actuators 110B and 112B in relation to muscles 302B and 304B.
  • the upper forearm flexor muscle EMG sensor 106B and the upper forearm flexor haptic actuator 110B are adjacent to an upper forearm flexor muscle 302B, such as the flexor carpi ulnaris muscle in the example.
  • the upper forearm extensor muscle EMG sensor 108B and the upper forearm extensor actuator 112B are adjacent to the upper forearm extensor muscle 304B, such as the extensor carpi ulnaris muscle in the example.
  • a user uses his or her upper forearm flexor and extensor muscles 302B and 304B in order to raise (flex) or lower (extend) the hand relative to the forearm 204. Even if the user does not have a hand, the muscles 302B and 304B are still engaged if the user intends or tries to position the hand in such a phantom motion.
  • an upper forearm flexor EMG value from the upper forearm flexor muscle EMG sensor 106B can be used to calculate an intended flexion angle of the hand relative to the forearm 204
  • an upper forearm extensor EMG value from the upper forearm extensor muscle EMG sensor 108B can be used to calculate an intended extension angle of the hand relative to the forearm 204.
  • the haptic actuators 110B and 112B can be used to convey to the user the raising (flexing) or lowering (extending) of the hand relative to the forearm 204. Such haptic actuators 110B and 112B may not be used if the user actually has a hand. If the virtual angle of the hand of the corresponding arm avatar relative to the forearm 204 in an XR environment changes, the haptic actuators 11 OB and 112B can stimulate the muscles 302B and 304B to convey such movement even though the user lacks a hand.
  • the haptic actuator 110B is used to convey a virtual flexion angle of the hand relative to the forearm 204, and the haptic actuator 112B is used to convey a virtual extension angle of the hand relative to the forearm 204.
  • FIG. 3C shows example placement of the EMG sensor 106C and the haptic actuator 110C in relation to muscle 302C
  • FIG. 3D shows example placement of the EMG sensor 108C and the haptic actuator 112C in relation to muscle 304C
  • the bicep muscle 302A is called out in FIG. 3C and the brachialis muscle 306 is called out in FIG. 3D for reference purposes.
  • Bone is shown in shaded manner in FIGs. 3C and 3D for illustrative clarity.
  • the forearm pronator muscle EMG sensor 106C and the forearm pronator haptic actuator 110C are adjacent to the forearm pronator muscle 302C.
  • the muscle 302C is specifically the pronator teres muscle in the example.
  • the forearm supinator muscle EMG sensor 108C and the forearm supinator haptic actuator 110C are adjacent to the forearm supinator muscle 304C.
  • a user uses his or her forearm pronator and supinator muscles 302C and 304C in order to rotate the wrist and thus the hand inwards (pronation) or outwards (supination). Even if the user does not have a hand, the muscles 30C and 304C are still engaged if the user intends or tries to rotate the hand in such a phantom motion.
  • a forearm pronator muscle EMG value from the forearm pronator muscle EMG sensor 106C can be used to calculate an intended pronation rotational angle of the hand
  • a forearm supinator muscle EMG value from the forearm supinator muscle EMG sensor 108C can be used to calculate an intended supination rotational angle of the hand.
  • the haptic actuators 110C and 112C can be used to convey to the user the inwards (pronating) or outwards (supinating) rotation of the hand. Such haptic actuators 110C and 112C may not be used if the user actually has a hand. If the virtual rotational angle of the hand of the corresponding arm avatar in an XR environment changes, the haptic actuators 110B and 112B can stimulate the muscles 302C and 304C to convey such rotation even though the user lacks a hand.
  • the haptic actuator 110B is used to convey a virtual pronation rotational angle of the hand, and the haptic actuator 112B is used to convey a virtual supination rotational angle of the hand.
  • FIG. 3E shows muscles 302D of the forearm 204
  • FIG. 3F shows example placement of the EMG sensor 106D and the haptic actuator 110D in relation to one such muscle 302D.
  • FIG. 3G shows muscles 304D of the forearm 204
  • FIG. 3H shows example placement of the EMG sensor 108D and the haptic actuator 112D in relation to one such muscle 304D.
  • Skin is shaded in FIGs. 3E and 3G for illustrative clarity.
  • EMG sensor 106D and the corresponding lower forearm flexor haptic actuator 110D are adjacent to a corresponding lower forearm flexor muscle 302D.
  • Examples of a lower forearm flexor muscle 302D include the flexor carpi ulnaris and radialis muscles.
  • the corresponding lower forearm extensor muscle EMG sensor 108D and the corresponding lower forearm extensor actuator 112D are adjacent to a corresponding lower forearm extensor muscle 304D.
  • Examples of a lower forearm extensor muscle 304D include the extensor carpi ulnaris and radialis muscles.
  • a user uses his or her corresponding lower forearm flexor and extensor muscles 302D and 304D in order to lower (flex) or raise (extend) a finger of the hand. Even if the user does not have a particular finger, the corresponding muscles 302D and 304D are still engaged if the user intends or tries to position the hand in such a phantom motion.
  • a lower forearm flexor EMG value from the corresponding lower forearm flexor muscle EMG sensor 106D can be used to calculate an intended flexion angle of the finger relative to a baseline angle
  • a lower forearm extensor EMG value from the corresponding lower forearm extensor muscle EMG sensor 108D can be used to calculate an intended extension angle of the hand relative to the baseline angle.
  • the corresponding haptic actuators 110D and 112D can be used to convey to the user the lowering (flexing) or raising (extending) of a finger. Such corresponding haptic actuators 110D and 112D may not be used if the user actually has a particular finger. If the virtual angle of the finger of the corresponding arm avatar relative to a baseline angle in an XR environment changes, the haptic actuators 110D and 112D can stimulate the muscles 302B and 304B to convey such movement even though the user lacks the finger.
  • the corresponding haptic actuator 110D is used to convey a virtual flexion angle of a finger, and the haptic actuator 112D is used to convey a virtual extension angle of the finger.
  • FIGs. 4A, 4B, and 4C show the various angles that can be calculated from different sensor values.
  • an angle 402 of the upper arm 202 relative to the side or shoulder can be calculated based on a tilt sensor value from the tilt sensor 114.
  • the baseline angle of the upper arm 202 relative to the side or shoulder may be straight down, where the arm 202 rests against the user’s side and has been rotated neither upwards nor backwards.
  • an angle 406 of the forearm 204 relative to the upper arm 202 can be calculated based on the EMG values from the bicep and tricep muscle EMG sensors 106A and 108A. If the user flexes the forearm 204 inwards from a baseline right angle of the forearm 204 relative to the upper arm 202 per arrow 408A, an intended flexion angle of the forearm 204 relative to the upper arm 202 can be calculated based on the bicep muscle EMG value from the EMG sensor 106A.
  • an intended extension angle of the forearm 204 relative to the upper arm 202 can be calculated based on the tricep EMG value from the EMG sensor 108A.
  • the angle of the forearm 204 relative to the side or shoulder is the sum of or is otherwise based on whichever of the flexion or extension angle that has been calculated and the angle of the upper arm 202 relative to the side or shoulder.
  • an angle 410 of the hand 206 relative to the forearm 204 can be calculated based on the EMG values from the upper forearm extensor and flexor muscle EMG sensors 106B and 108B. If the user flexes the hand 206 inwards from a baseline zero-degree angle of the hand 206 relative to the forearm 204 per arrow 412A, an intended flexion angle of the hand 206 relative to the forearm 204 can be calculated based on the upper forearm flexor muscle EMG value from the EMG sensor 106B.
  • an intended extension angle of the hand 206 relative to the forearm 204 can be calculated based on the upper forearm extensor muscle EMG value from the EMG sensor 108B.
  • the angle of the hand 206 relative to the upper arm 202 is the sum or is otherwise based on whichever of the flexion or extension angle that has been calculated, and whichever of the flexion angle of the forearm 204 relative to the upper arm 202 that has been calculated.
  • the angle of the hand 206 relative to the side or shoulder is the sum of or is otherwise based on the angle of the hand 206 relative to the upper arm 202, and the angle of the upper arm 202 relative to the side or shoulder.
  • a rotational angle of the hand 206 relative to a baseline rotational angle can be calculated based on the EMG values from the forearm pronator and supinator muscle EMG sensors 106C and 108C.
  • the baseline rotational angle of the hand 206 corresponds to no rotation of the wrist. If the user rotates the hand inwards from the baseline rotational angle per arrow 414A, an intended pronation angle of the hand 206 relative to the baseline rotational angle can be calculated based on the forearm pronator muscle EMG value from the EMG sensor 106C.
  • an intended supination angle of the hand 206 relative to the baseline rotational angle can be calculated based on the forearm supinator muscle EMG value from the EMG sensor 108C.
  • an angle of each finger of the hand 206 relative to a baseline angle can be calculated based on the EMG values from corresponding lower forearm extensor and flexor muscles EMG sensors 106D and 108D.
  • the baseline angle of each finger corresponds to the finger having been neither raised nor lowered.
  • An intended flexion angle or an intended extension angle of a finger is specifically calculated depending on whether the finger is being flexed inwards towards the palm of the hand 206 or is being extended outwards away from the palm.
  • the user is flexing the fourth finger 422B inwards, and therefore an intended flexion angle 424 of the finger 422B relative to the baseline angle can be calculated based on the lower forearm flexor muscle EMG value from the EMG sensor 106D corresponding to the flexor muscle 302D for this finger 422B.
  • the user is also extending the index finger 422A outwards, and therefore an intended extension angle 426 of the finger 422A relative to the baseline angle be calculated based on the lower forearm extension muscle EMG value from the EMG sensor 108D corresponding to the extensor muscle 304D for this finger 422A.
  • FIGs. 5A and 5B show example images 500 and 500 of arm avatars that can be rendered and displayed based on the intended angles that have been calculated.
  • the image 500 of FIG. 5A includes an avatar 502 corresponding to the user wearing the muscle sensor and haptic feedback sleeve 100.
  • the arm of the avatar 502 is referred to herein as an arm avatar 504.
  • the image 500 of the avatar 502 and the arm avatar 504 is from a third-person perspective.
  • the image 550 of FIG. 5B similarly includes an arm avatar 552 corresponding to the user wearing the muscle sensor and haptic feedback sleeve 100.
  • the image 550 of the arm avatar 552 is from a first-person perspective, however.
  • the first-person perspective arm avatar 552 may be displayed at the user’s HMD, and the third- person perspective avatar 502 and arm avatar 504 may be displayed at the other users’ HMDs.
  • FIGs. 6A, 6B, and 6C show respective example methods 600, 620, and 650 for calculating an intended angle from an associated pair of muscle EMG sensors.
  • the methods 600, 620, and 650 can be implemented as program code stored on a non-transitory computer-readable data storage medium and executable by a processor.
  • the processor may be that of the muscle sensor and haptic feedback sleeve 100, or that of an HMD worn by the user or of a host computing device communicatively connected to the sleeve 100.
  • the calculated angle is the flexion or extension angle of the forearm 204 relative to the upper arm 202.
  • the calculated angle is the flexion or extension angle of the hand 206 relative to the forearm 204.
  • the calculated angle is the pronation or supination rotational angle of the hand 206 relative to a baseline rotational angle.
  • the calculated angle is the flexion or extension angle of the finger relative to a baseline angle.
  • the method 600 of FIG. 6A shows how a muscle EMG sensor value is calculated from any muscle EMG sensor 106A, 106B, 106C, 106D, 108A, 108B, 108C, or 108D in the case in which such an EMG sensor provides an analog muscle EMG sensor signal indicative of usage (e.g., flexion, extension, pronation, or supination) of a corresponding muscle.
  • the analog muscle EMG sensor signal is read from the sensor (602), and transformed into a corresponding digital muscle EMG sensor signal (604), such as by using an analog-to-digital converter.
  • the resulting digital muscle EMG sensor signal may be filtered to remove any background noise within the signal (606).
  • the digital muscle EMG sensor signal is a continuing series of values spaced apart by time intervals corresponding to the sample rate at which the analog EMG sensor signal is read.
  • a muscle EMG sensor value is calculated from the values sampled (i.e., read) over or within a given sample time period. For instance, the root mean square of these sampled or read values over the sample period can be used as the muscle EMG sensor value for that period (608). That is, it can be stated that the root mean square of the digital muscle EMG sensor signal is calculated to yield the muscle EMG sensor value.
  • the method 620 of FIG. 6B shows how calibrated minimum and maximum values for each muscle EMG sensor of an associated pair is calculated.
  • the user is instructed to maintain a neutral position for the muscles to which the associated pair of muscle EMG sensors correspond (622).
  • the neutral position for the bicep and tricep muscles 302A and 304A is for the forearm 204 to be at a right angle to the upper arm 202.
  • the neutral position for the upper forearm flexor and extensor muscles 302B and 304B is for the hand 206 to be neither raised nor lowered relative to the forearm 204.
  • the neutral position for the forearm pronator and supinator muscles 302C and 304C is for the wrist (and thus the hand 206) not to rotated.
  • the neutral position for the lower forearm flexor and extensor muscles 302D and 304D is for the finger to neither be raised nor lowered relative to the hand 206.
  • the EMG sensor value from each muscle EMG sensor of the associated pair is received (624), per the method 600.
  • the EMG sensor value received from each muscle EMG sensor is then set as the calibrated baseline value for that sensor (626).
  • the EMG sensor value received from the sensor 106A is set as the calibrated baseline bicep muscle EMG value corresponding to no flexion of the bicep muscle 302A.
  • the EMG sensor value received from the sensor 106B is set as the calibrated baseline tricep EMG value corresponding to no extension of the tricep muscle 302B.
  • the user is then instructed to maintain a maximum angle corresponding to the first muscle EMG sensor of the associated pair (628).
  • the bicep muscle EMG sensor 106A is the first sensor of the pair, and has a maximum flexion angle of nominally 90 degrees when the user is maximally flexing the bicep muscle 302A to rotate the forearm 204 against the upper arm 202.
  • the upper forearm flexor EMG sensor 106B is the first sensor of the pair, and has a maximum flexion angle of nominally 90 degrees when the user is maximally flexing the upper forearm flexor muscle 302B to rotate the hand 206 downwards or inwards towards the wrist as much as possible.
  • the forearm pronator muscle EMG sensor 106C is the first sensor of the pair, and has a maximum pronation rotational angle of nominally 180 degrees side or shoulder the user maximally pronates the forearm pronator muscle 302C to rotate wrist (and thus the hand 206) inwards as much as possible.
  • the lower forearm flexor muscle EMG sensor 106D is the first sensor of the pair, and has a maximum flexion angle of nominally 90 degrees when the user maximally flexes the lower forearm flexor muscle 302D to rotate the finger downwards towards the palm of the hand 206 as much as possible.
  • the EMG sensor value from the first muscle EMG sensor of the associated pair is received (630), per the method 600.
  • the EMG sensor value received from the first muscle EMG sensor is then set as the calibrated maximum value for that sensor (632).
  • the EMG sensor value received from the sensor 106A is set as the calibrated maximum bicep muscle EMG value corresponding to maximum intended flexion of the bicep muscle 302A.
  • the user is then instructed to maintain a maximum angle corresponding to the second muscle EMG sensor of the associated pair (634).
  • the tricep muscle EMG sensor 108A is the second sensor of the pair, and has a maximum extension angle of nominally 90 degrees when the user is maximally extending the tricep muscle 304A to rotate the forearm 204 inline with the upper arm 202.
  • the upper forearm extensor EMG sensor 108B is the second sensor of the pair, and has a maximum extension angle of nominally 90 degrees when the user is maximally extending the upper forearm extensor muscle 304B to rotate the hand 206 upwards or outwards away from the wrist as much as possible.
  • the forearm supinator muscle EMG sensor 108C is the second sensor of the pair, and has a maximum supination rotational angle of nominally 180 degrees when the user maximally supinates the forearm supinator muscle 304C to rotate wrist (and thus the hand 206) outwards as much as possible.
  • the lower forearm extensor muscle EMG sensor 108D is the second sensor of the pair, and has a maximum extension angle of nominally 90 degrees when the user maximally extends the lower forearm extensor muscle 304D to rotate the finger upwards away from the palm of the hand 206 as much as possible.
  • the EMG sensor value from the second muscle EMG sensor of the associated pair is received (636), per the method 600.
  • the EMG sensor value received from the second muscle EMG sensor is then set as the calibrated maximum value for that sensor (638).
  • the EMG sensor value received from the sensor 108A is set as the calibrated maximum tricep muscle EMG value corresponding to maximum intended extension of the tricep muscle 304A.
  • the method 650 of FIG. 6C shows how an intended angle for an associated pair of muscle EMG sensors is calculated.
  • the angle is the intended flexion or extension angle of the forearm 204 relative to the upper arm 202.
  • the angle is the intended flexion or extension angle of the hand 206 relative to the forearm 204.
  • the angle is the intended pronation or supination rotational angle of the wrist (and thus of the hand 206).
  • the angle is the intended flexion or extension angle of the finger (relative to a baseline angle).
  • a first muscle EMG sensor value is received from the first EMG muscle sensor of the associated pair, and a second muscle EMG sensor value is received from the second EMG muscle sensor of the associated pair (652).
  • bicep and tricep muscle EMG sensor values may be respectively received from the bicep and tricep muscle EMG sensors 106A and 108A. Then, whether the first muscle EMG sensor value is greater than the calibrated baseline value for the first EMG muscle sensor or whether the second muscle EMG sensor value is greater than the calibrated baseline value for the second EMG muscle sensor is determined (654).
  • the bicep muscle EMG sensor value can be greater than the calibrated baseline bicep muscle EMG value, or the tricep muscle EMG sensor value can be greater than the calibrated baseline tricep muscle EMG value. That is, both the muscle EMG sensor values cannot be greater than their respective calibrated baseline EMG values at the same time.
  • the angle for the associated pair of muscle EMG sensors is set based on the maximum angle for the first sensor multiplied by a ratio of the first muscle EMG sensor value and a difference between the calibrated maximum and baseline values for the first sensor (656). For example, if the bicep muscle EMG sensor value is greater than the calibrated bicep muscle EMG sensor value, then the angle of the forearm 204 relative to the upper arm 202 is set to a flexion angle. This flexion angle is equal to the maximum flexion angle of the forearm 204 relative to the upper arm 202 (e.g., nominally 90 degrees), multiplied by a ratio. The ratio is the bicep muscle EMG sensor value divided by the difference between the calibrated maximum bicep muscle EMG value and the calibrated baseline bicep muscle EMG value.
  • the angle for the associated pair of muscle EMG sensors is set based on the maximum angle for the second sensor multiplied by a ratio of the second value and a difference between the calibrated maximum and baseline values for the second sensor (658). For example, if the tricep muscle EMG sensor value is greater than the calibrated tricep muscle EMG sensor value, then the angle of the forearm 204 relative to the upper arm 202 is set to an extension angle. This extension angle is equal to the maximum extension angle of the forearm 204 relative to the upper arm 202 (e.g., nominally 90 degrees), multiplied by a ratio. The ratio is the tricep muscle EMG sensor value divided by the difference between the calibrated maximum tricep muscle EMG value and the calibrated baseline tricep muscle EMG value.
  • FIG. 7 shows an example non-transitory computer-readable data storage medium 700 storing program code 702 executable by a processor to perform processing for calculating angles from muscle EMG sensor values for rendering and display of an arm avatar.
  • the processor may be that of the muscle sensor and haptic feedback sleeve 100, or that of an HMD worn by the user or of a host computing device communicatively connected to the sleeve 100.
  • the processing can include receiving a tilt sensor value from the tilt sensor 114 (704), and calculating an angle of the upper arm 202 of the user relative to the user’s side or shoulder (706).
  • the angle may be calculated in that the tilt sensor 114 itself provides this angle, or provides a raw value from which the angle may be calculated.
  • the angle of the upper arm 202 relative to the side or shoulder may be calculated from such a raw value in a similar manner to that provided in the methods 620 and 650.
  • a user may be requested to place the upper arm 202 against his or her side for recording a calibrated baseline tilt sensor value, and then requested to raise the arm for recording a calibrated maximum tilt sensor value.
  • the angle of the upper arm 202 relative to the user user’s side may then be calculated based on the maximum angle that the user’s upper arm 202 can be raised (nominally 180 degrees) multiplied by a ratio.
  • the ratio is the tilt sensor value divided by the difference between the calibrated maximum and baseline tilt sensor values.
  • the processing can include receiving bicep and tricep muscle EMG sensor values from the EMG sensors 106A and 108A (708), and calculating an intended flexion or extension angle of the forearm 204 relative to the upper arm 202 based on these values (710), per the method 650.
  • the angle of the forearm 204 relative to the side or shoulder may also be calculated (712). This angle may be the sum of, or otherwise based on, the intended flexion or extension angle of the forearm 204 relative to the upper arm 202 and the angle of the upper arm 202 relative to the side or shoulder.
  • the processing can include receiving upper forearm flexor and extensor EMG sensor values from the EMG sensors 106B and 108B (714), and calculating an intended flexion or extension angle of the hand 206 relative to the forearm 204 based on these values (716), per the method 650.
  • the angle of the hand 206 relative to the upper arm 202, shoulder, and/or side may also be calculated (718).
  • the angle of the hand 206 relative to the upper arm 202 may be the sum of, or otherwise based on, the intended flexion angle or extension angle of the hand 206 relative to the forearm 204 and the intended flexion or extension angle of the forearm 204 relative to the upper arm 202.
  • the angle of the hand 206 relative to the side or shoulder may be the sum of, or otherwise based on, the angle of the hand 206 relative to the upper arm 202 and the angle of the upper arm 202 relative to the side or shoulder.
  • the processing can include receiving forearm pronator and supinator EMG sensor values from the EMG sensors 106C and 108C (720), and calculating an intended pronation or supination rotational angle of the hand 206 based on these values (722).
  • the processing can include, for each of one or multiple fingers (724), receiving corresponding lower forearm flexor and extensor EMG sensor values (726), and calculating an intended flexion or extension angle of the finger based on these values (728).
  • the processing can include then rendering and displaying an arm avatar based on one or multiple of the intended angles that have been calculated (730). Since the intended movement or position of the arm is known from these calculated angles, an arm avatar can be rendered even though the user may be missing a forearm 204, a hand 206, and/or one or multiple fingers on the hand 206.
  • the arm avatar may be rendered at one device and displayed on another device. For example, a computing device of the user may render the arm avatar, and transmit the avatar to the device of another user for display by that device.
  • FIG. 8A shows an example implementation of the muscle sensor and haptic feedback sleeve 100.
  • the sleeve 100 can include a first layer 802 and a second layer 804.
  • the second layer 804 is the layer that is adjacent to the skin of the user’s upper arm 202 or forearm 204 when the sleeve 100 is being worn, whereas the first layer 802 is the layer that is exposed (i.e., that is visible).
  • the muscle EMG sensors 106A-106D and 108A-108D and the haptic actuators 110A-110D and 112A-112D may be disposed within the first layer 802.
  • the second layer 804 may be composed of two layers 806 and 808, which can be electric layers that generate electricity to self-power the muscle sensor and haptic feedback sleeve 100.
  • the layer 806 may be a thermoelectric layer 806 that thermoelectrically generates electricity.
  • the layer 808 may be a static electric layer 808 that generates electricity in a static electric manner.
  • the layers 806 and 808 are shown as one on top of the other in the example, but can instead be combined or integrated with one another throughout or within the second layer 804.
  • FIG. 8B shows an example electrospun polymer nanofiber layer 810 that can realize each of the first and second layers 802 and 804.
  • the layer 810 includes polymer nanofibers 812 and 814 that are interwoven with one another.
  • the polymer nanofibers 812 running in one direction are separated from the polymer nanofibers 814 running in a perpendicular direction via spacers 816. Electrospinning may be used to generate the nanofibers 812 and 814, and to interweave them.
  • the nanofibers 812 may be nanofiber sensing yarns that form a nanofiber array to realize the muscle EMG sensor 106A-106D and 108A-108D.
  • the nanofibers 814 may in turn be nanofiber actuator yarns that form a nanofiber array to realize the haptic actuators 110A-110D and 112A-112D.
  • the nanofibers 812 may be nanofiber yarns that realize the thermoelectric layer 806, and the nanofibers 814 may be nanofiber yarns that realize the static electric layer 808.
  • both the nanofibers 812 and 814 of the corresponding nanofiber layer 810 for the thermoelectric layer 806 realize the thermoelectric layer 806.
  • both the nanofibers 812 and 814 of the corresponding nanofiber layer 810 realize the static electric layer 808.
  • FIG. 9A shows an example nanofiber yarn 900 that is used within the electrospun polymer nanofiber layer 810 (i.e., as the nanofibers 812 and/or 814) to realize the muscle EMG sensors 106A-106D and 108A-108D and the haptic actuators 110A-110D and 112A-112D over corresponding nanofiber arrays.
  • a stretchable core fiber 902 is an elastic nanofiber 904.
  • the elastic nanofiber 904 is disposed between carbon nanotube layers 906 and 908.
  • the nanofiber yarn 900 is a nanofiber sensing yarn
  • the elastic nanofiber 904 is a piezoresistive elastic nanofiber.
  • the nanofiber 904 has a resistance corresponding to the pressure imparted by an adjacent muscle. The resistance can be measured between the nanotube layers 906 and 908 to detect the muscle EMG signal of that muscle.
  • the nanofiber yarn 900 is a nanofiber actuating yarn
  • the elastic nanofiber 904 is a piezoelectric elastic nanofiber.
  • Power applied between the nanotube layers 906 and 908 cause the nanofiber 904 to impart a pressure against an adjacent muscle.
  • the amount of power that is applied is based on a virtual angle. As one example, to impart a virtual flexion angle of the forearm 204 relative to the upper arm 202, a corresponding amount of power is applied between the nanotube layers 906 and 908 of the portion of each elastic nanofiber 904 that realizes the bicep haptic actuator 110A.
  • FIG. 9B shows a nanofiber array 950 that can realize the muscle EMG sensors 106A-106D and 108A-108D and the haptic actuators 110A-110D and 112A-112D.
  • the nanofiber array 950 represents the layer 802 of the muscle and haptic feedback sleeve 100 as implemented as the nanofiber layer 810 having nanofibers 812 and 814 that are each implemented as the nanofiber yarn 900.
  • the layer 802 is shown in planar fashion in the example for illustrative clarity. However, in actuality, the layer 802 is wrapped around the upper arm 202 and forearm 204 of the user.
  • 106D and 108A-108D can be integrated with the array 950 implementing the haptic actuators 110A-110D and 112A-112D in the case in which the nanofibers 812 correspond to the sensors 106A-106D and 108A-108D and the nanofibers 814 correspond to the actuators 110A-110D and 112A-112D (or vice-versa).
  • the nanofiber array 950 implementing the EMG sensors 106A-106D and 108A-108D can be separate from the array 950 implementing the haptic actuators 110A- 110D and 112A-112D in the case in which there are nanofibers 812 and 814 implementing the sensors 106A-106D and 108A-108D and different nanofibers 812 and 814 implementing the actuators 110A-110D and 112A-112D.
  • Different portions of the nanofiber array 950 may be adjacent to various muscles on a per-user basis and a per-usage session basis. On a peruser basis, differences in user arm size and user anatomy can result in different portions of the array 950 being adjacent to various muscles. On a per-usage basis, different portions of the array 950 can be adjacent to various muscles depending on how a particular user puts on the muscle and haptic feedback sleeve 100 in a given usage session.
  • the nanofiber array 950 implements the EMG sensors 106A-106D and 108A-108D
  • the portions of the array 950 are then fired (i.e., change in sensor value) are identified. In this way, which portion of the array 950 corresponds to each muscle can be identified.
  • a portion 956A of the array 950 may fire, meaning the portion 956A is adjacent to the bicep muscle
  • a portion 958A of the array 950 may fire, meaning the portion 958A is adjacent to the tricep muscle 304A and such that the portion 958A is identified as the tricep muscle EMG sensor 108A.
  • a similar process can be used to identify portions 956B and 958B as the upper forearm flexor and extensor muscle EMG sensors 106B and 108B, respectively, and to identify portions 956C and 958C as the forearm pronator and supinator muscle EMG sensors 106C and 108C, respectively.
  • the portions 956D can be similarly identified as the lower forearm flexor muscle EMG sensors 106D,and the portions 958D can be identified as the lower forearm extensor muscle EMG sensors 108D.
  • the nanofiber array 950 implements the haptic actuators 110A-110D and 112A-112D
  • a similar calibration process is used to identify the portions that corresponds to different muscles for a user in a given usage session. Specifically, different portions of the array 950 are fired (e.g., different portions of the array 950 have power applied to them), and the user is requested to identify which virtual angle, if any, has been correspondingly conveyed.
  • the portion 956A when the portion 956A is fired, the user may receive feedback that the virtual angle between forearm 204 relative to the upper arm 202 is at a maximum flexion angle, which means the portion 956A is adjacent to the bicep muscle 302A.
  • the portion 956A is identified as the bicep haptic actuator 110A.
  • the user may receive feedback that the virtual angle between the forearm 204 relative to the upper arm 202 is at a maximum extension angle, which means the portion 958A is adjacent to the tricep muscle 304A.
  • the portion 958A is identified as the tricep haptic actuator 112A.
  • a similar process can be used to identify portions 956B and 958B as the upper forearm flexor and extensor haptic actuators 110B and 112B, respectively, and to identify portions 956C and 958C as the forearm pronator and supinator haptic actuators 110C and 112C, respectively.
  • the portions 956D can be similarly identified as the lower forearm flexor haptic actuators 110D,and the portions 958D can be identified as the lower forearm extensor haptic actuators 112D.
  • FIG. 10 shows an example nanofiber yarn 1000 that is used within the electrospun polymer nanofiber layer 810 (e.g., as the nanofibers 812 and/or 814) to realize the thermoelectric layer 806 and the static electric layer 808.
  • a stretchable core fiber 1002 are layers 1004 and 1006 disposed between carbon nanotube layers 1003 and 1008.
  • the layer 1004 is a dielectric polyimide layer and the layer 1006 is an acrylic layer.
  • the layers 1006 is a dielectric polyimide layer and the layer 1006 is an acrylic layer.
  • 1004 and 1006 are different conductor or semiconductor layers.
  • FIGs. 11A-11D show example operation of the nanofiber yarn 1000 in the case in which the yarn 1000 is used within the electrospun polymer nanofiber layer 810 to realize the static electric layer 808.
  • An ammeter 1102 is depicted in the figures, between nanotube layers 1003 and 1008, to indicate the direction of electricity that is generated.
  • FIG. 11A when the layers 1004 and 1006 are brought into contact with one another due to movement of the upper arm 202 or forearm 204 within the muscle sensor and haptic feedback sleeve 100, the boundary between the layers 1004 and 1006 generates positive and negative charges.
  • the flow of electrons from the nanotube layer 1008 to the nanotube layer 1003 creates a reverse potential difference to balance the potential difference at the boundary between the layers 1004 and 1006.
  • FIG. 12 shows example operation of the nanofiber yarn 1000 in the case in which the yarn 1000 is used within the electrospun polymer nanofiber layer 810 to realize the thermoelectric layer 806.
  • An ammeter 1102 is again depicted, between nanotube layers 1008 and 1003, to indicate the direction of electricity that is generated.
  • the layers 1004 and 1006 are subjected to a heat source, such as the body heat from the upper arm 202 or forearm 204 when the user is wearing the muscle sensor and haptic feedback sleeve 100.
  • the layers 1004 and 1006 are different conductive of semiconductive layers, and therefore heat and cool at different rates. This means that the layers 1004 and 1006 can be at different temperatures even though both are subjected to the same heat source.
  • current is generated in accordance with the Seebeck effect, such as from the layer 1006 to the layer 1004 in the example. Therefore, electricity flows from the nanotube layer 1008 to the nanotube layer 1003, and this generated electricity can be used to self-power the sleeve 100.
  • Techniques have been described for detection of an intended forearm-to-upper arm angle even if a user does not have a forearm, and can similarly provide for conveyance of a virtual forearm-to-upper arm angle.
  • the techniques provide for detection of an intended hand-to-forearm angle and an intended rotational hand (i.e., wrist) angle even if a user does not have a hand, and similarly provide for conveyance of a virtual hand-to-forearm angle and a virtual rotational hand angle.
  • the techniques also provide for detection of an intended finger-to-hand angle, and similarly provide for conveyance of a virtual finger-to-hand angle.
  • muscle EMG sensors and haptic actuators can be realized within arrays, and in the same or other implementations, the sleeve can self-generate electricity for powering the sleeve.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Textile Engineering (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne une valeur électromyographique (EMG) du muscle biceps qui est reçue en provenance d'un capteur EMG du muscle biceps et une valeur EMG du muscle triceps qui est reçue en provenance d'un capteur EMG du muscle triceps. Un angle de flexion ou d'extension prévu d'un avant-bras par rapport à un bras supérieur est calculé sur la base des valeurs d'EMG du muscle biceps et triceps. Un avatar de bras est rendu selon l'angle de flexion ou d'extension prévu de l'avant-bras.
PCT/US2022/032745 2022-06-08 2022-06-08 Angle prévu basé sur des valeurs d'emg du muscle WO2023239361A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2022/032745 WO2023239361A1 (fr) 2022-06-08 2022-06-08 Angle prévu basé sur des valeurs d'emg du muscle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2022/032745 WO2023239361A1 (fr) 2022-06-08 2022-06-08 Angle prévu basé sur des valeurs d'emg du muscle

Publications (1)

Publication Number Publication Date
WO2023239361A1 true WO2023239361A1 (fr) 2023-12-14

Family

ID=82404414

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/032745 WO2023239361A1 (fr) 2022-06-08 2022-06-08 Angle prévu basé sur des valeurs d'emg du muscle

Country Status (1)

Country Link
WO (1) WO2023239361A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180239430A1 (en) * 2015-03-02 2018-08-23 Mindmaze Holding Sa Brain activity measurement and feedback system
US20200323460A1 (en) * 2019-04-11 2020-10-15 University Of Rochester System And Method For Post-Stroke Rehabilitation And Recovery Using Adaptive Surface Electromyographic Sensing And Visualization
WO2021180945A1 (fr) * 2020-03-12 2021-09-16 Université De Bordeaux Méthode de contrôle d'un membre d'un avatar virtuel par les activités myoélectriques d'un membre d'un sujet et système associé

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180239430A1 (en) * 2015-03-02 2018-08-23 Mindmaze Holding Sa Brain activity measurement and feedback system
US20200323460A1 (en) * 2019-04-11 2020-10-15 University Of Rochester System And Method For Post-Stroke Rehabilitation And Recovery Using Adaptive Surface Electromyographic Sensing And Visualization
WO2021180945A1 (fr) * 2020-03-12 2021-09-16 Université De Bordeaux Méthode de contrôle d'un membre d'un avatar virtuel par les activités myoélectriques d'un membre d'un sujet et système associé

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
NIIJIMA ARINOBU ARINOBU NIIJIMA HW@HCO NTT CO JP ET AL: "Reducing Muscle Activity when Playing Tremolo by Using Electrical Muscle Stimulation to Learn Efficient Motor Skills", PROCEEDINGS OF THE ACM ON INTERACTIVE, MOBILE, WEARABLE AND UBIQUITOUS TECHNOLOGIES, ACMPUB27, NEW YORK, NY, USA, vol. 5, no. 3, 14 September 2021 (2021-09-14), pages 1 - 17, XP058621649, DOI: 10.1145/3478110 *

Similar Documents

Publication Publication Date Title
Li et al. Towards the sEMG hand: internet of things sensors and haptic feedback application
US10602965B2 (en) Wearable deformable conductive sensors for human motion capture including trans-joint pitch, yaw, and roll
US10716510B2 (en) Smart clothing with converging/diverging bend or stretch sensors for measuring body motion or configuration
Demolder et al. Recent advances in wearable biosensing gloves and sensory feedback biosystems for enhancing rehabilitation, prostheses, healthcare, and virtual reality
US9582072B2 (en) Motion recognition clothing [TM] with flexible electromagnetic, light, or sonic energy pathways
Tognetti et al. Wearable kinesthetic system for capturing and classifying upper limb gesture in post-stroke rehabilitation
Alirezaei et al. A tactile distribution sensor which enables stable measurement under high and dynamic stretch
Gentner et al. Development and evaluation of a low-cost sensor glove for assessment of human finger movements in neurophysiological settings
Rossi et al. Electroactive fabrics and wearable biomonitoring devices
US6049327A (en) System for data management based onhand gestures
An et al. Deep learning enabled neck motion detection using a triboelectric nanogenerator
Fatema et al. A low-cost pressure sensor matrix for activity monitoring in stroke patients using artificial intelligence
Chappell Making sense of artificial hands
CN112230763A (zh) 一种多参数测量的智能手套
Tan et al. Force-direction discrimination is not influenced by reference force direction (short paper)
Yeo et al. Tactile sensorized glove for force and motion sensing
Loeb Estimating point of contact, force and torque in a biomimetic tactile sensor with deformable skin
Simone et al. A low cost method to measure finger flexion in individuals with reduced hand and finger range of motion
O’Meara et al. Comparing two different cursor control methods which use single-site surface electromyography
De Pasquale et al. Wearable system with embedded force sensors for neurologic rehabilitation trainings
WO2023239361A1 (fr) Angle prévu basé sur des valeurs d'emg du muscle
Saggio et al. Sensory systems for human body gesture recognition and motion capture
Teng et al. Haptic Permeability: Adding Holes to Tactile Devices Improves Dexterity
Tian et al. Ultrathin epidermal P (VDF-TrFE) piezoelectric film for wearable electronics
Moreno et al. Wearable robot technologies

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22738137

Country of ref document: EP

Kind code of ref document: A1