CN115515477A - Method for controlling limbs of virtual avatar through myoelectric activity of individual limbs and system thereof - Google Patents

Method for controlling limbs of virtual avatar through myoelectric activity of individual limbs and system thereof Download PDF

Info

Publication number
CN115515477A
CN115515477A CN202180032805.2A CN202180032805A CN115515477A CN 115515477 A CN115515477 A CN 115515477A CN 202180032805 A CN202180032805 A CN 202180032805A CN 115515477 A CN115515477 A CN 115515477A
Authority
CN
China
Prior art keywords
activity
limb
signal
virtual avatar
avatar
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202180032805.2A
Other languages
Chinese (zh)
Inventor
克里斯多夫·阿尔冈
马修·格曼
艾马·古利特德鲁吉
丹尼尔·卡塔尔特
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Centre National de la Recherche Scientifique CNRS
Universite de Bordeaux
Original Assignee
Centre National de la Recherche Scientifique CNRS
Universite de Bordeaux
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Centre National de la Recherche Scientifique CNRS, Universite de Bordeaux filed Critical Centre National de la Recherche Scientifique CNRS
Publication of CN115515477A publication Critical patent/CN115515477A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/389Electromyography [EMG]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/389Electromyography [EMG]
    • A61B5/397Analysis of electromyograms
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6824Arm or wrist
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/742Details of notification to user or communication with user or patient ; user input means using visual displays
    • A61B5/744Displaying an avatar, e.g. an animated cartoon character
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/7455Details of notification to user or communication with user or patient ; user input means characterised by tactile indication, e.g. vibration or electrical stimulation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/02Operational features
    • A61B2560/0223Operational features of calibration, e.g. protocols for calibrating sensors
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1107Measuring contraction of parts of the body, e.g. organ, muscle
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25Bioelectric electrodes therefor
    • A61B5/279Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/296Bioelectric electrodes therefor specially adapted for particular uses for electromyography [EMG]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61FFILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50Prostheses not implantable in the body
    • A61F2/68Operating or control means
    • A61F2/70Operating or control means electrical
    • A61F2/72Bioelectric control, e.g. myoelectric

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Dermatology (AREA)
  • Neurosurgery (AREA)
  • Neurology (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention relates to a method for controlling a limb of an avatar (24) by myoelectrical activity of a limb (8) of an individual (10), comprising a first step of calibration and a second step of moving the limb of the avatar (24). The invention also relates to a system suitable for implementing the method according to the invention for controlling a limb of a virtual avatar (24) by myoelectrical activity of a limb (8) of an individual (10).

Description

Method for controlling limbs of virtual avatar through myoelectric activity of individual limbs and system thereof
The invention relates to a method for controlling a limb of a virtual avatar by myoelectrical activity of the limb of a subject.
The invention is particularly relevant for the population who are victims of amputations, in particular upper limb amputations.
Some amputees have myoelectric prostheses, i.e. contact prostheses, the movement of which is controlled by means of electrical signals from the muscles of the user.
One of the major problems encountered by patients wearing myoelectric prostheses is the cognitive load associated with control and the permanent visual fate required to use them.
The myoelectric control necessary to control the prosthesis is based on the activation of two antagonistic muscles and the quality of dissociation of these muscle controls. Such control is tedious and requires a lot of training.
Since the patient has no information about the state of the prosthesis other than that provided by visual feedback, the actions become more difficult. They must remain focused on the object to be reached and monitor the movement of the prosthesis in the process. This is extremely costly in terms of cognitive resources (e.g., attention and muscle control). Because of this, many patients abandon their prostheses, which are known as "cabinet prostheses".
For these different reasons, prostheses are currently underutilized compared to their capabilities.
Tabor et al, in Design and Cognitive Impatiention, 2017: "design Game-Based Myoelectric prosthetics Training" relates to games developed for the purpose of Training amputees to use their Myoelectric prostheses. This document discloses The development of a visualization interface ("The Falling of Momo"). The character in the game is controlled by a bracelet control with electrodes placed on the amputated limb according to the same proportional control as that controlling the myoelectric prosthesis. The intensity of muscle contraction sensed by the electrodes is proportional to the movement of the character in the game to provide visual feedback to the user.
US 2014/198034 discloses a bracelet with electrodes for use with integrated screen glasses operated by recognizing gestures. Sensing muscle contractions (e.g., stretching the index finger and bending or stretching the wrist) by the electrodes during a predetermined gesture enables actions to be performed on the glasses screen, such as moving a scroll down menu. Recognition of the gesture may be confirmed by tactile feedback, by applying a discrete two-state signal at the end of the gesture).
Patel et al, published in Neural Engineering, volume 14, phase 5 (2017): "Context-dependent adaptation events robustness of myoelectric control for upper-limb prostheses" describes a protocol for improving the control of a forearm myoelectric prosthesis by means of a bracelet with electrodes and according to a learning model. The signals obtained by the electrodes are processed to obtain their envelopes. The envelope of the muscle activity signal is used for the gesture recognition algorithm. The output signal of the algorithm is an estimate of the normalized activation level associated with the function of the prosthesis (e.g., opening the hand or rotating the wrist). Based on the output signal, an activity threshold and a gain are calculated to define control information that is sent to the prosthesis to move the prosthesis. The activity threshold is modified to avoid parasitic activity of other functions. The gain is fixed by a function.
It is an object of the invention to propose a method for improving the learning of muscle separation and the fine work of muscle contraction.
To this end, the object of the invention is a method for controlling a limb of a virtual avatar through electromyographic activity of the limb of a subject, comprising a first calibration step comprising:
-acquiring, by a device for measuring electromyographic activity signals, a first and a second raw calibrated electromyographic activity signal resulting from a variable-strength contraction of a first and a second antagonistic muscle, respectively, of a limb of the subject during a given period of time, and then, for each of the two antagonistic muscles:
-determining an envelope of the signal,
-determining a minimum myoelectric activity and a maximum myoelectric activity from the envelope of the signal,
-determining a myoelectric activity threshold value,
-normalizing the maximum myoelectric activity and the myoelectric activity threshold,
-determining a coefficient for converting the normalized myoelectric activity into a movement velocity component of the limb of the avatar, and
the method for controlling includes a second step of moving a limb of the virtual avatar, the second step including:
-acquiring, by a device for measuring an electromyographic activity signal, a first and a second raw electromyographic activity signal resulting from a contraction of two antagonistic muscles of a limb of the subject during a given period, and then, for each of the first and second raw electromyographic activity signals:
-normalizing the myoelectrical activity resulting from the signal,
-converting the normalized myoelectric activity into a movement velocity component of the limb of the avatar by applying a conversion coefficient,
-determining the movement speed and the movement direction of the limb of the virtual avatar by subtracting the movement speed component obtained for each muscle, and
-moving a limb of the virtual avatar with the determined movement speed and movement direction.
According to other advantageous aspects of the invention, the method comprises one or more of the following features taken alone or according to any technically possible combination:
-while moving the limb of the virtual avatar at the determined movement speed and movement direction, the method comprises activating a sensory feedback device provided on the limb of the subject, the sensory feedback device being selectively and discretely activated in accordance with a predetermined value range of the sensory parameter;
-the sensory parameter is selected from the bending angle of the elbow of the virtual avatar, the forward-facing angle of the shoulder of the virtual avatar, the abduction/adduction angle of the shoulder of the virtual avatar, the medial or lateral rotation angle of the shoulder of the virtual avatar, the bending angle of the wrist of the virtual avatar, the radial or ulnar rotation angle of the virtual avatar, the degree of opening of the hand of the virtual avatar, the degree of opening of the finger grip of the hand of the virtual avatar involving the thumb and one or more other fingers, the grip force of the hand of the virtual avatar, the bending angle of the knee of the virtual avatar, the bending angle of the ankle of the virtual avatar, the rotation speed of one of the motions mentioned in this list, or any other sensory means that can be replaced by vibration, such as the temperature or configuration of a prosthetic limb.
The device for measuring electromyographic activity signals comprises a plurality of electrodes, the method comprising selecting, from the plurality of electrodes, two electrodes enabling maximum acquisition of an electromyographic activity signal of each of the two antagonistic muscles;
-determining a minimum electromyographic activity and a maximum electromyographic activity from the acquired raw calibrated electromyographic activity signals, determining an electromyographic activity threshold, normalizing the maximum electromyographic activity and the electromyographic activity threshold, determining a coefficient for converting the normalized electromyographic activity into a movement velocity component of a limb of the virtual avatar, normalizing the electromyographic activity produced by each of the first signal and the second signal, converting the normalized electromyographic activity into a movement velocity component of a limb of the virtual avatar by applying the conversion coefficient, determining a movement velocity and a movement direction of the limb of the virtual avatar by subtracting the movement velocity component obtained for each muscle, and the step of moving the limb of the virtual avatar at the determined movement velocity and movement direction is carried out by the processing device;
-determining the minimum myoelectrical activity from the envelope of the signal comprises:
-setting the raw calibration electromyographic activity signal to zero mean,
-determining an absolute value of the raw calibration myoelectric activity signal set to zero mean,
-filtering the signal at absolute value to obtain its envelope, an
-determining a minimum value and a maximum value of the envelope of the filtered signal from the envelope of the filtered signal.
According to another aspect, the object of the invention is a method for controlling a limb of a virtual avatar through myoelectrical activity of the limb of a subject, comprising a first calibration step comprising:
-acquiring, by a device for measuring an electromyographic activity signal, a first and a second raw calibrated electromyographic activity signal resulting from a contraction of variable intensity of a first and a second antagonistic muscle, respectively, of a limb of the subject during a given time period, then:
determining, by a processing device, an envelope of each of the first signal and the second signal,
determining, by a device for processing signals, a minimum myoelectric activity and a maximum myoelectric activity from the envelope of each of the first signal and the second signal, and then, for each of the two antagonistic muscles,
-determining, by a processing device, a myoelectric activity threshold,
-normalizing by the processing device the maximal myoelectric activity and the myoelectric activity threshold,
-determining, by the processing device, a coefficient for converting the normalized myoelectric activity into a movement velocity component of the limb of the avatar, and
the method for controlling includes a second step of moving a limb of the virtual avatar, the second step including:
-acquiring, by means of a device for measuring electromyographic activity signals, a first and a second raw electromyographic activity signal resulting from a contraction of two antagonistic muscles of a limb of the subject during a given period and then, for each of the first and second raw electromyographic activity signals,
-normalizing the myoelectrical activity produced by the signal by means of a processing device,
-converting, by the processing device, the normalized electromyographic activity into a movement velocity component of the limb of the avatar by applying a conversion coefficient,
-determining, by the processing device, a movement speed and a movement direction of a limb of the virtual avatar by subtracting the movement speed component obtained for each muscle, and
-moving the limb of the virtual avatar with the movement speed and movement direction determined by the processing device.
Another object of the invention is a system adapted to implement the method of controlling a limb of a virtual avatar by myoelectrical activity of the limb of a subject according to the invention, the system comprising:
-a device for measuring electromyographic activity signals produced by contractions of a first antagonistic muscle and a second antagonistic muscle of a limb of the subject, the device being adapted to acquire a first raw calibration electromyographic activity signal and a second raw calibration electromyographic activity signal produced by contractions of a variable strength of the first antagonistic muscle and the second antagonistic muscle, respectively, of the limb of the subject during a given time period, and a first raw electromyographic activity signal and a second raw electromyographic activity signal produced by contractions of the two antagonistic muscles of the limb of the subject during the given time period,
a processing device for processing myoelectrical activity, the processing device comprising at least one display device configured for displaying a virtual avatar,
-the processing device is configured for determining from the acquired raw calibrated electromyographic activity signals a minimum electromyographic activity and a maximum electromyographic activity, determining an electromyographic activity threshold, normalizing the maximum electromyographic activity and the electromyographic activity threshold, determining a coefficient for converting the normalized electromyographic activity into a movement velocity component of the limb of the avatar, normalizing the electromyographic activity produced by each of the first signal and the second signal, converting the normalized electromyographic activity into a movement velocity component of the limb of the avatar by applying the conversion coefficient, determining a movement velocity and a movement direction of the limb of the avatar by subtracting the movement velocity component obtained for each muscle, and moving the limb of the avatar at the determined movement velocity and in the movement direction.
According to other advantageous aspects of the invention, the system comprises one or more of the following features, taken alone or according to any technically possible combination:
the device for measuring comprises a plurality of electrodes and is in the form of a bracelet;
-the system further comprises a sensory feedback device;
the sensory feedback device comprises a plurality of vibratory elements;
-the processing device is configured for selectively and discretely activating the sensory feedback device in accordance with a predetermined range of values of the sensory parameter;
the device for measuring electromyographic activity signals comprises a remote communication device configured for remote communication with the processing device.
Another object of the invention is a system adapted to implement the method of controlling a limb of a virtual avatar by myoelectrical activity of the limb of a subject according to the invention, the system comprising:
-a device for measuring electromyographic activity signals produced by contractions of a first antagonistic muscle and a second antagonistic muscle of a limb of the subject, the device being adapted to acquire a first raw calibration electromyographic activity signal and a second raw calibration electromyographic activity signal produced by contractions of the first antagonistic muscle and the second antagonistic muscle of the limb of the subject of variable intensity, respectively, during a given period of time, and a first raw electromyographic activity signal and a second raw electromyographic activity signal produced by contractions of the two antagonistic muscles of the limb of the subject during the given period of time,
-a processing device for processing myoelectrical activity, the processing device comprising at least one display device configured for displaying a virtual avatar,
-the processing device is configured for, in a first calibration step, determining an envelope of each of the first and second signals, determining a minimum and a maximum electromyographic activity from the envelope of each of the first and second signals, determining an electromyographic activity threshold, normalizing the maximum and electromyographic activity threshold, determining a coefficient for converting the normalized electromyographic activity into a movement velocity component of the limb of the avatar, normalizing the electromyographic activity produced by each of the first and second signals, converting the normalized electromyographic activity into a movement velocity component of the limb of the avatar by applying the conversion coefficient, and, in a second step of moving the limb of the avatar, for each of the first and second raw electromyographic activity signals: normalizing myoelectric activity resulting from the signal, converting the normalized myoelectric activity into a movement velocity component of a limb of the avatar by applying a conversion coefficient, determining a movement velocity and a movement direction of the limb of the avatar by subtracting the movement velocity component obtained for each muscle, and moving the limb of the avatar at the determined movement velocity and movement direction.
These characteristics and advantages of the invention will emerge from a reading of the following description, given purely by way of non-limiting example and made with reference to the accompanying drawings, on which:
figure 1 is a schematic view of a system according to an embodiment of the invention;
figure 2 is a schematic view of a subject wearing a myoelectric prosthesis; and
figure 3 is the following graph: configurations without any sensory feedback for the three types of sensory feedback and non-amputated subjects, the distribution of deviation from the target is shown for each movement during the test that is reciprocated between two targets by the avatar.
In fig. 1, a system 1 suitable for implementing the method according to the invention is shown.
The system 1 comprises a device 4 for measuring myoelectrical activity signals. The myoelectrical activity signal comes from a surge of electricity that is transmitted by nerve fibers to the muscle and causes the muscle to contract.
The device 4 for measuring comprises a plurality of electrodes 6. The electrodes 6 are, for example, three-phase surface electrodes adapted to be placed on a limb 8 of a subject 10 and to receive electromyographic activity signals from muscles of the limb 8 of said subject 10.
The electrodes 6 are typically configured for acquiring myoelectrical activity signals.
In particular, the electrodes 6 are configured for acquiring a first and a second raw calibrated electromyographic activity signal resulting from a contraction of variable intensity of a first and a second antagonistic muscle (for example biceps and triceps).
The electrodes 6 are typically also configured for acquiring a first and a second raw electromyographic activity signal resulting from contraction of a first and a second antagonistic muscle of a limb 8 of the subject 10 during a given period of time.
In particular, the device 4 for measuring myoelectrical activity signals comprises eight electrodes 6.
The acquisition frequency of the electrodes 6 is, for example, 200Hz. Such frequencies are particularly suitable for obtaining the envelope of the signal.
However, other frequency values are contemplated, including for example frequencies of 100Hz to 2000Hz in order to obtain a more accurate signal.
The device 4 for measuring is typically in the form of a flexible bracelet. This form makes it possible to place the measuring device 4 around a limb 8, the limb 8 being for example the arm, the forearm or the remainder of the stump, or the remainder of the arm. Then, the electrode 6 is arranged along the circle and toward the inside of the circle.
The electrodes 6 are connected together by any material suitable for forming a bracelet. For example, the bracelet is extendable. According to a variant, the bracelet has several sizes adapted to different modalities and for adapting to the upper and lower limbs. The bracelet may also include a closure device, such as a self-clamping system, a buckle, a snap, or any other option that may be envisaged.
Other forms of the device 4 for measuring are conceivable, for example the electrodes are not connected together, or the device 4 for measuring is integrated in a myoelectric prosthesis 11 as shown in fig. 2. An electromyographic prosthesis refers to a contact prosthesis operated by an electromyographic activity signal and controlled by its user.
The system 1 also includes a remote communication device 12.
The remote communication device 12 is typically a radio communication device.
For example, the remote communication device 12 is a Bluetooth or Wi-Fi communication device.
The telecommunication device 12 is configured for transmitting the electromyographic activity read by the electrodes 6 to a device 14 for processing the electromyographic activity.
According to an advantageous embodiment, the system 1 further comprises a sensory feedback device 16.
The sensory-feedback device 16 includes, for example, a plurality of vibratory elements 18.
Sensory-feedback device 16 is typically in the form of a bracelet. For example, sensory-feedback device 16 is in the form of an elastic bracelet. The sensory feedback device 16 and the device 4 for measuring are preferably fixed together.
For example, sensory-feedback device 16 includes sixteen vibratory elements 18. In a variation, sensory-feedback device 16 includes a different number of vibratory elements 18, such as six vibratory elements.
Sensory feedback means 16 enables sensory feedback to be provided to subject 10 after muscle contraction causes myoelectrical activity to be detected by electrodes 6. The operation thereof will be described later.
The processing device 14 includes at least one processor. The processing device 14 further comprises a memory adapted to record the read electromyographic activity.
The processing device 14 shown in fig. 1 comprises three computers 20, 21, 22.
The processing device 14 preferably includes at least one display device 23.
In fig. 1, each computer 20, 21, 22 comprises a display device 23. The display device 23 is typically a screen.
The three computers 20, 21, 22 are configured to communicate with each other, for example through switches configured to interact in a local area network.
The device 4 for measuring electromyographic activity signals is configured for remote communication with a processing device 14. In the example of fig. 1, the device for measuring 4 is configured for remote communication with a first computer 20.
The first host computer 20 is configured for receiving the raw electromyographic activity signals collected by the electrodes 6 and transmitted by the telecommunication device 12, processing them and transmitting the processed electromyographic activity signals, in particular to the second computer 21 and to the third computer 22.
Advantageously, the first computer 20 is configured for identifying the two electrodes 6 of the measuring device 4 that enable maximum acquisition of the electromyographic activity signals from each of the two antagonistic muscles.
The first computer 20 is configured for determining a minimum electromyographic activity and a maximum electromyographic activity from the acquired raw calibration electromyographic activity signal for each of the two antagonistic muscles.
For this purpose, the first computer 20 is further adapted to set the raw calibration myoelectrical activity signal acquired by the electrodes 6 to a zero mean value.
The first computer 20 is further configured for determining an absolute value of the raw calibration electromyographic activity signal set to a zero mean value.
The first computer 20 is configured to filter the signal at absolute value to obtain its envelope.
For example, the first computer 20 comprises a second order low-pass Butterworth (Butterworth) filter with a cut-off frequency of 1.5Hz, configured to filter the obtained signal to obtain its envelope.
The first computer 20 is adapted to determine a electromyographic activity threshold, called EMGx offset, below which the contraction of a muscle is not mandatory and must not be taken into account in order to avoid disturbing the resulting movement.
The first computer 20 is configured to consider only the values recorded between a given threshold and the maximum value of the envelope of the filtered signal.
The first computer 20 is configured for normalizing the value of the envelope of the filtered signal.
The first computer 20 is generally configured for assigning a value of zero to the previously calculated myoelectrical activity threshold and for assigning a value of 1 to the maximum value of the envelope of the filtered signal.
In particular, the first computer 20 is configured to determine coefficients for converting the normalized electromyographic activity into a movement velocity component of a limb of the virtual avatar 24.
For example, the conversion coefficient is determined in the following manner for the motion of the arm of the avatar. Conventionally, the extremely fast rotation of the elbow is 4rad/s. This value is taken as a starting point and the subject 10 gives his perception of this velocity. The conversion factor is increased or decreased so that the perception of velocity is consistent with the muscle activity of the subject 10. This coefficient is specific to each muscle.
The first computer 20 is generally configured for: for each muscle, a zero velocity component is assigned when the normalized value assigned to the value of the envelope of the filtered signal is 0, and a maximum velocity component is assigned when the normalized value assigned to the value of the envelope of the filtered signal is 1.
In particular, for each muscle, the first computer 20 is able to assign to each normalized value of the envelope of the filtered signal a velocity component lying between the zero velocity component and the maximum velocity component.
The first computer 20 is further configured to convert the normalized myoelectrical activity into a movement velocity component of the limb of the avatar 24 by applying the conversion coefficient.
The first computer 20 is configured to determine the movement speed and direction of movement of the limb of the virtual avatar 24 displayed by the second computer 21 by subtracting the velocity component obtained for each muscle at a given time. For example, in the case of elbow rotation, subtracting the velocity value makes it possible to determine the speed of flexion or extension of the elbow of the avatar 24 displayed by the display device 23 of the second computer 21.
The display device 23 of the second computer 21 is configured to display a virtual avatar 24, the virtual avatar 24 showing at least one limb corresponding to a movement that has to be controlled by myoelectrical activity. This may be the case for an arm or limb, a full body or half-body, depending on the application chosen.
Stopping muscle contraction causes, for example, motion to stop, and the elbow of virtual avatar 24 will then be positioned at a fixed location corresponding to the last position reached.
According to an alternative, the calculated speed is the speed at which the hand of the avatar opens or closes, or the rotational speed of the wrist of the avatar, or the flexion/extension speed of the knee or ankle of the avatar, or any other conceivable option.
The first computer 20 is configured to transmit movement instructions to the second computer 21.
The movement instructions typically include a determined speed of movement and a direction of movement.
Preferably, the first computer 20 is configured to determine a sensory parameter of a limb of the virtual avatar 24 that will be associated with the at least one vibratory element 18 and will be retransmitted in the vibrations.
The sensory parameters are: for example, selected from the group consisting of a bend angle of an elbow of virtual avatar 24, a forward tilt angle of a shoulder of virtual avatar 24, a abduction/adduction angle of a shoulder of virtual avatar 24, an inside or outside rotation angle of a shoulder of virtual avatar 24, a bend angle of a wrist of virtual avatar 24, a radius or ulna rotation angle of virtual avatar 24, a degree of opening of a hand of virtual avatar 24, a degree of opening of a finger grip of a hand of virtual avatar 24 involving a thumb and one or more other fingers, a grip force of a hand of virtual avatar 24, a bend angle of a knee of virtual avatar 24, a bend angle of an ankle of virtual avatar 24, a rotation speed of one of the motions mentioned in this list, or any other sensory manner that can be replaced by vibration, such as a temperature or configuration of myoelectric prosthesis 11 of subject 10.
Configuring the myoelectric prosthesis 11 corresponds to selecting one of the modes of the myoelectric prosthesis 11 during use in order to know on which joint the muscle activation will act. For example, if the prosthesis is in a mode called "pronation and supination (pronation) of the wrist," the vibration will signal the pronation and supination movements of the wrist. Also, in a mode called "elbow," the vibration will signal an elbow motion.
The second computer 21 is dedicated to a visual interface with the subject 10.
According to an alternative not shown, the second computer is integrated directly in the first computer 20. This makes it possible to reduce the number of interfaces.
The second computer 21 is configured to receive the movement instructions transmitted by the first computer 20.
In an advantageous embodiment, third computer 22 is configured to activate sensory feedback device 16.
According to an alternative not shown, the third computer is integrated directly in the electrode bracelet 6 by means of a microcontroller and is integrated with the control system by means of pulse width modulation.
The first computer 20 is configured for transmitting the processed myoelectrical activity converted into instantaneous values of sensory parameters by applying conversion coefficients to the third computer 22.
The third computer 22 is generally configured to determine a range of values of a sensory parameter to be associated with the at least one vibratory element 18.
The third computer 22 is typically configured for activating at least one element 18 corresponding to a previously determined range of values of the sensory parameter.
The third computer 22 comprises, for example, a reading program configured for matching between the instantaneous value of the sensory parameter and the activation of the corresponding vibrating element 18.
The third computer 22 is then configured to activate the corresponding vibratory element 18.
Sensory-feedback device 16 is connected to a third computer 22, such as by a cable to third computer 22.
In the alternative where the third computer 22 is integrated directly into the electrode bracelet 6, the electrodes 6, sensory feedback device 16 and third computer 22 are combined.
Preferably, the third computer 22 is configured for selectively and discretely activating the vibratory element 18.
For example, the third computer 22 can determine the number and location of the vibratory elements 18 to be activated based on the selected range of values of the sensory parameter.
Preferably, a single determined vibratory element 18 is activated within each value range of the sensory parameter.
Again preferably, a single vibration element 18, which is different for each range of values, is assigned to a range of values of the sensory parameter.
According to an example in which the sensory parameter is the angle of flexion of the elbow, when the subject 10 contracts the biceps and/or triceps, the electrodes 6 are configured for acquiring a first and a second raw electromyographic activity signal respectively generated by the contraction of the biceps or triceps; within a given period, the remote communication device 12 is configured for transmitting a first and a second raw electromyographic activity signal to the first computer 20, the first computer 20 is configured for processing the first and the second raw electromyographic activity signals and for transmitting a command to the second computer 21 for moving the arm of the virtual avatar 24 and for transmitting the instantaneous position of the arm of the virtual avatar 24 to the third computer 22, the second computer 21 is configured for displaying the movement of the arm of the virtual avatar 24, and the third computer 22 is configured for activating each vibration element 18 in steps with an elbow bending angle of 20 °. Naturally, the step size may have a value different from 20 °, e.g. 10 °, 15 °, 25 °, 30 °.
The third computer 22 is preferably configured to determine the duration and sequence of vibrations.
For example, the third computer 22 is configured to activate a given vibratory element 18 during a period of 100ms, alternating with stopping for a period of 100 ms.
The intensity of the vibration is preferably adapted to improve the comfort of the subject 10.
In the alternative, the processing device 14 comprises a single computer, or a smartphone, television, or tablet, or any combination of these elements as may be contemplated. For example, all of the functions of the first computer, the second computer, and the third computer described above can be implemented by the single computer.
According to another alternative, the processing device is at least partially incorporated in the device for measuring. For example, the incorporated portion of the processing device is configured to implement all of the functionality of the first computer and the second computer described above.
A method for controlling a limb of a virtual avatar through myoelectrical activity of the limb of a subject will now be described. The method is implemented by the system 1.
The device 4 for measuring electromyographic activity signals is placed on a limb 8 of a subject 10 such that the electrodes 6 are in contact with a first antagonistic muscle and a second antagonistic muscle of the limb.
For example, subject 10 is an amputated human. Subject 10 has been amputated, e.g., with an arm and with a humerus (i.e., above the elbow). The device 4 for measuring electromyographic activity signals is placed around the amputated arm of the subject 10 so that the electrodes 6 are in contact with the biceps and triceps muscles (two antagonistic muscles).
In the alternative, the subject is not amputated and the device for measuring electromyographic activity signals is placed around the subject's arm such that the electrodes are in contact with the biceps and triceps muscles. The arm of a non-amputated subject is fixed in the device. This makes it possible to reproduce the muscular condition of the amputated subject, i.e. the isometric contraction of the biceps and triceps, characterized by the absence of movement.
The method comprises a first calibration step.
The electrodes 6 acquire a first and a second raw calibrated electromyographic signal, generated by respectively variable-strength contractions of a first and a second antagonistic muscle of a limb 8 of the subject 10 during a given period.
The device 4 for measuring electromyographic activity signals is in remote communication with the processing device 14 via the remote communication device 12. In the example of fig. 1, the device for measuring 4 is in remote communication with a first computer 20.
Advantageously, the method further comprises identifying the two electrodes 6 of the measuring device 4 that enable maximum acquisition of the myoelectric activity signals of each of the two antagonistic muscles. To this end, the subject 10 separately contracts two antagonistic muscles (e.g. biceps and triceps) wearing the measurement device 4.
Each electrode 6 acquires a raw calibrated myoelectrical activity signal transmitted to the processing device 14 by the remote communication device 12.
The acquisition frequency of the electrodes 6 is for example 200Hz.
Once the contraction has been performed, the acquisition is stopped and the processing device 14 selects two electrodes 6, each of these two electrodes 6 acquiring its own significant activity during the activation of one of the antagonistic muscles and not acquiring or only slightly acquiring the co-contraction activity.
The co-contractile activity corresponds to simultaneous contraction of the two antagonistic muscles.
In particular, the first computer 20 selects the two electrodes 6 that enable the maximum acquisition of the raw calibrated electromyographic activity signals of each of the two antagonistic muscles.
After identifying the two electrodes 6 which allow maximum acquisition, the method comprises determining a minimum electromyographic activity and a maximum electromyographic activity from the acquired raw calibrated electromyographic activity signal for each of the two antagonistic muscles, in particular by means of the processing device 14.
In particular, the first computer 20 determines, for each of the two antagonistic muscles, a minimum electromechanical activity and a maximum electromyographic activity from the acquired raw calibrated electromyographic activity signal.
To this end, subject 10 may, for example, have a first muscle (e.g., the biceps) undergo the strongest muscle contraction for a given period of time, and then relax. The given period of time is one to three seconds, preferably equal to two seconds. Subject 10 then allows the second muscle (e.g., the triceps) to undergo the strongest muscle contraction for a given period of time, and then relax. The given period is, for example, two seconds.
The first and second raw calibrated myoelectrical activity signals generated by these contractions are acquired by the electrodes 6.
The remote communication device 12 transmits the raw calibrated myoelectrical activity signal to the processing device 14.
In particular, the remote communication device 12 transmits the raw calibrated myoelectrical activity signal to the first computer 20.
The processing device 14 then sets the raw calibration myoelectrical activity signal to zero mean.
In particular, the first computer 20 sets the raw calibrated electromyographic activity signal to a zero mean.
The processing device 14 determines the absolute value of the raw calibrated electromyographic activity signal set to zero mean.
In particular, the first computer 20 determines the absolute value of the raw calibration electromyographic activity signal set to a zero mean.
The processing device 14 filters the signal at absolute value to obtain its envelope.
In particular, the first computer 20 filters the signal at absolute value to obtain its envelope.
For example, the filter used is a second-order low-pass butterworth filter with a cutoff frequency of 1.5 Hz.
From the envelope of the filtered signal, the processing device 14 determines, for each of the two antagonistic muscles, a minimum value and a maximum value of the envelope of the filtered signal.
In particular, the first computer 20 determines, for each of the two antagonistic muscles, a minimum value and a maximum value of the envelope of the filtered signal.
The minimum and maximum values of the envelope of the filtered signal correspond to the minimum myoelectric activity and the maximum myoelectric activity.
Advantageously, the processing device 14 stores the minimum and maximum values of the envelope of the filtered signal.
The processing device 14 then determines a myoelectric activity threshold, called EMGx offset, below which the contraction of the muscle is not forced and must not be taken into account in order to avoid interfering with the acquired movement. This is because values below this threshold correspond to the remaining activity of the muscle after muscle fatigue and may interfere with the movement.
In particular, in determining the threshold, the perception of subject 10 is taken into account and calculated for each of the two antagonistic muscles as follows:
offset EMG x =perc x *(EMG x max -EMG x min )+EMG x min Wherein
Offset EMG x Is a threshold value of myoelectrical activity;
perc x as a percentage of the degree of muscle activity;
EMG x max is the maximum value of the envelope of the filtered signal, an
EMG x min Is the minimum value of the envelope of the filtered signal.
The percentage of muscle activity level of the subject 10 is determined such that the speed of movement of the limb of the virtual avatar is zero when the subject feels relaxed.
Typically, the percentage is between 4% and 10%.
This percentage may have an effect when the subject wishes to generate the speed of movement of the limb of the avatar but also to counteract that speed.
This percentage is refined for each muscle.
The myoelectric activity threshold is typically determined by default to be 5% of the muscle activity level.
This electromyographic activity threshold is then typically tested and adjusted by subject 10 to assess whether the threshold interferes with the control of virtual avatar 24.
Preferably, the value of the myoelectric activity threshold can be adjusted at any time, in particular after a few minutes of working, after which the muscle may feel tired and the remaining muscle activity becomes a problem.
In particular, the first computer 20 determines a myoelectrical activity threshold.
Upon determination of the myoelectrical activity threshold, the first computer 20 will only consider values recorded between the given threshold to the maximum of the envelope of the filtered signal.
Advantageously, the processing device 14 stores the myoelectrical activity threshold.
The processing device 14 then normalizes the determined threshold value and the maximum value of the envelope of the filtered signal.
In particular, the first computer 20 normalizes the determined threshold and the maximum of the envelope of the filtered signal.
The first computer 20 typically assigns a value of 0 to the previously calculated myoelectric activity threshold and a value of 1 to the maximum value of the envelope of the filtered signal.
The processing device 14 determines a conversion coefficient for each muscle such that any normalized value of the envelope of the filtered signal lying between the determined threshold and the maximum value of the envelope of the filtered signal can be converted into a velocity component. The conversion coefficients are adapted to the subject for each muscle so that the perception of motion is as faithful as possible. The transformation coefficients are typically such that a zero velocity component can be assigned for each muscle when the normalized value of the envelope of the filtered signal is 0 and a maximum velocity component can be assigned when the normalized value of the envelope of the filtered signal is 1.
Advantageously, the processing device 14 stores the conversion coefficients.
This calibration of the recorded electromyographic activity signals allows for fine tuning of the control of a virtual avatar 24 adapted to subject 10, particularly when the electromyographic activity is very uneven between the two antagonistic muscles.
After the calibration phase, the method includes the step of moving a limb of the virtual avatar 24.
The system 1 is ready to acquire and process new myoelectrical activity signals.
Two selected electrodes 6 acquire a first and a second raw electromyographic activity signal generated by the contraction of two antagonistic muscles of a limb 8 of the subject 10 within a given period of time.
For each of the first and second raw electromyographic activity signals, the remote communication device 12 transmits the raw electromyographic activity signal to the first computer 20.
The first computer 20 sets the raw myoelectrical activity signal to zero mean.
The first computer 20 determines the absolute value of the raw electromyographic activity signal set to zero mean.
The first computer 20 filters the signal at absolute value to obtain its envelope.
The first computer 20 normalizes the value of the envelope of the filtered signal, for example by means of a rule of three (a rule of three), according to the previously calculated and stored electromyographic activity threshold and to the previously determined and stored maximum value of the envelope of the filtered signal, so as to obtain values from 0 to 1.
For each muscle, the first computer 20 assigns to each normalized value of the envelope of the filtered signal a velocity component lying between the zero velocity component and the maximum velocity component, generally by applying coefficients previously determined and stored.
The first computer 20 determines the moving speed and moving direction of the limb of the virtual avatar 24 displayed by the second computer 21 by subtracting the speed component obtained for each muscle.
For example, in the case of elbow rotation, subtracting the velocity value makes it possible to determine the bending or stretching velocity of the elbow of the virtual avatar 24 displayed by the second computer 21.
The first computer 20 transmits the movement instruction to the second computer 21.
The movement instructions typically include a determined movement speed and movement direction of the limb of the virtual avatar 24.
The second computer 21 receives the movement instruction transmitted by the first computer 20.
The display device 23 of the second computer 21 displays a virtual avatar 24, the virtual avatar 24 showing at least one limb corresponding to movements that must be controlled by myoelectrical activity. This may be the case for arms or legs, a full body or half-length, depending on the application chosen.
The limbs of the virtual avatar 24 move according to the movement instructions transmitted by the first computer 20.
For example, stopping muscle contraction causes movement to stop, and then the elbow of the virtual avatar 24 will be positioned at a fixed location corresponding to the last position reached.
Thus, when the subject 10 contracts at least one muscle on which the electrodes 6 are placed, the virtual avatar 24 bends or stretches his elbow according to the adjustments appropriate for the subject.
According to an alternative, the calculated speed is the speed of the opening or closing of the hand of the virtual avatar, or the rotational speed of the wrist of the virtual avatar, or the speed of the flexion/extension of the knee or ankle of the virtual avatar, or any other option that may be envisaged.
The subject 10 thus has visual feedback through the avatar 24 of contraction through the subject through the two antagonistic muscles, the movement of the avatar 24 being adapted to the subject's muscle contraction capacity. The learning of muscle separation and the delicate work of muscle contraction are thus improved.
In an advantageous embodiment of the method, after determining the movement speed and the movement direction of the limb of the virtual avatar 24 by acquiring the first electromyographic activity signal and the second electromyographic activity signal, the method comprises activating the sensory feedback device 16, in particular activating the sensory feedback device 16 by the processing device 14.
The vibratory element 18 of the sensory feedback device 16 is disposed on a limb 8 of the subject 10, the limb 8 including two antagonistic muscles whose electromyographic activity is measured.
The number of vibrating elements 18 is sixteen, for example, and is arranged in two strips on either side of the electrode 6, each strip being provided with eight vibrating elements 18.
The vibration element 18 is activated, for example, in the following manner.
For each myoelectric activity signal acquired by the electrodes 6, the first computer 20 determines an instantaneous value of a sensory parameter, such as the instantaneous position of a limb of the virtual avatar 24, from the previously determined movement speed and movement direction.
The first computer 20 transmits instantaneous values of the sensory parameters of the limbs of the virtual avatar 24 to the third computer 22.
The third computer 22 determines a range of values for a sensory parameter of a limb of the virtual avatar 24 to be associated with the at least one vibratory element 18.
The third computer 22 typically activates at least one element 18 corresponding to a previously determined range of values of the sensory parameter.
In particular, the reading program of the third computer 22 matches the instantaneous value of the first sensory parameter with the activation of the corresponding vibrating element 18.
Preferably, the third computer 22 selectively and discretely activates the vibratory element 18.
For example, the third computer 22 determines the number and location of vibratory elements 18 to be activated based on the range of values of the sensory parameter.
Preferably within each value range of the sensory parameter.
Again preferably, a single vibration element 18, which is different for each range of values of the sensory parameter, is assigned to the range of values of the sensory parameter.
According to an example in which the sensory parameter is a bending angle of an elbow, when the subject 10 contracts the biceps and/or triceps, the electrodes 6 acquire a first raw electromyographic activity signal and a second raw electromyographic activity signal generated by contraction of the biceps and the triceps, respectively, the remote communication device 12 transmits the first electromyographic activity signal and the second electromyographic activity signal to the first computer 20, the first computer 20 processes the first electromyographic activity signal and the second electromyographic activity signal and transmits a command to move the arm of the virtual avatar 24 to the second computer 21, and transmits an instantaneous position of the arm of the virtual avatar 24 to the third computer 22, the second computer 21 displays a motion of the arm of the virtual avatar 24, and the third computer 22 activates each of the vibration elements 18 in steps with a bending angle of 20 °. Obviously, the step size may have a value different from 20 °, for example 10 °, 15 °, 25 °, 30 °.
This allows the most natural control to be achieved, i.e. both to perform the movement quickly and to allow stopping precisely on the target without oscillation.
The third computer 22 preferably determines the duration and sequence of the vibrations for each instantaneous position.
For example, for a given instantaneous position, the third computer 22 activates a given vibratory element 18 for a period of 100ms, alternating with stopping for a period of 100 ms.
The intensity of the vibration is preferably adapted to improve the comfort of the subject 10.
For example, the first intensity level is defined by the subject's perception of the first vibration when the vibration intensity is never increased by the presence of the vibration. The second intensity level is defined by a change from the subject's perception of vibration to the absence of perception of vibration when the intensity of vibration is reduced from high vibration.
The level of vibration intensity sufficient to stimulate the subject is calculated by the average between the first level of vibration intensity and the second level of vibration intensity.
According to another example, the vibration intensity varies according to the recorded myoelectrical activity. In the case of the grip force of the hand of the virtual avatar 24, the greater the grip force, the greater the intensity of the applied vibration.
The sensory feedback caused by the vibration makes it possible to visually add consistent sensory information while reducing the attention load of the subject, as shown in the examples below.
An implementation example of the method according to the invention will now be described.
Examples of the invention
Device and participant
Participants included subjects with arm amputation via the humerus (i.e., above the elbow), as well as non-amputated subjects referred to as healthy subjects.
The tool for restoring electrical activity of biceps and triceps is MyoArm from Thalmic
Figure BDA0003924687290000211
This is a bracelet comprising eight surface electrodes, originally designed to run through an algorithm of patterns of muscle activity at the forearm. It is used here to restore the electrical muscular activity of the biceps and triceps muscles on the arms of the participants。
A bracelet of 6 vibrators with a diameter of 7mm and a thickness of 2mm was used to send the vibrotactile sensory feedback to the participants.
Three computers are connected together and there is a very precise role for each computer. The main computer, i.e. the main control computer, is the carrier for running the whole protocol program for various exercises, and also has the following functions: the method includes (1) receiving myoelectric activity transmitted from MyoArm Band through bluetooth, (2) processing the myoelectric activity, (3) transmitting a movement instruction converted into a velocity (resulting from the processing) to the second computer so that an elbow of a virtual avatar displayed on a screen of the second computer moves, and (4) transmitting an instantaneous position calculated from the myoelectric activity to a third computer responsible for activating a vibrator. The exchange between the master and the other two computers is accomplished through a switch configured to interact in a local area network.
The second computer is dedicated to the visualization interface and is placed facing the participants. The motion of the avatar is displayed in real time on the computer using Animatlab software. In addition, all instructions are displayed on the computer during the exercise. The participant need only focus on the screen and follow various instructions.
Finally, a third computer (pi-top) has the role of managing the activation of the vibrator. The third computer receives instructions from the master control computer which indicate thereto an indication of a position calculated from myoelectrical activity. The read program on the pi-top matches the location with the activation of the corresponding vibrator. The computer consists of a Raspberry Pi3 to which a bracelet of vibrators is connected.
Calibration
Each participant goes through the necessary calibration phase before any testing begins. During this phase, muscle activity from the eight electrodes of the bracelet is displayed on the graphical interface. It is sufficient to wear the bracelet and directly observe the muscle activity recorded by the electrodes. The frequency of acquisition of the MyoArm Band is 200Hz. The original activity is visible as well as the traces of the filtered signal. During this phase, the participants were asked to perform separate contractions of the biceps and triceps muscles.
This makes it possible to identify two electrodes with which the signal is most clear in the rest of the experiment. Once contraction is complete, acquisition is stopped and two electrodes are selected that (i) have intrinsic and significant activity during activation of the biceps or triceps, and (ii) have little or no co-contraction activity.
Once the electrodes are selected, the minimum myoelectrical activity and the maximum myoelectrical activity on each electrode channel is determined. For this purpose, the participants were asked to contract the biceps almost maximally for 2 seconds, and then relax and contract the triceps for 2 seconds.
Depending on the laterality of the participant's amputation, an avatar with either a right arm or a left arm may be obtained. Next, the signal is normalized. A muscle activity threshold (typically 5% to 7.5% of the normalized signal range) is defined below which no movement will be induced. This makes it possible to prevent residual activity of the muscle from occurring after muscle fatigue, which would cause the avatar to move without any real intention of the subject. The threshold may be adjusted during the experiment to provide a certain mobility margin for adjusting the threshold depending on the participant's situation. Finally, a gain is established between the muscle activity and the elbow rotation speed. This allows fine adjustment of the avatar's control, especially when muscle activity is very unbalanced.
A spatial reference is defined to activate the vibration by a rapid stimulus that alternates (100 ms activation, then 100ms rest, and so on). After the occurrence of one of these references (in the exact case of more than 2 seconds), the rest time of the stimulation is increased to reduce stress on the participant, while the participant continues to be notified every 500ms by 100ms activation.
When the participants contract their muscles, the arm of the avatar moves based on the data from the calibration, and the vibrator is activated in steps of 20 ° in angle.
Experiment of
Participants were asked to reciprocate between two targets to assess control stability over time. The second computer's screen displays two targets representing the magnitude of the motion, with the avatar's arm moving toward the starting target. The amplitude of motion ranges from 20 ° to 100 °. The training exercise included a series of 10 movements with visual sensory feedback, followed by 10 movements with visual and vibratory sensory feedback, and 1 movement with vibration only. During the execution of the movement, the subject is asked to mark a pause time on the target to be reached before starting again towards the other targets. When the avatar has no motion within 500ms, a stable characteristic of the position of the avatar is notified with sound. However, the sound does not inform the subject of the fact that it is on the correct target. Thus, the target can only be verified visually and/or by vibration. At the end of each test, the participants were shown a motion profile. This enables them to judge the speed of execution and the accuracy of the movement throughout the test. The duration of the test performed is proportional to the amplitude of the reciprocating movement to be performed and extends from about 20 to 45 seconds for a test with an amplitude of 100 °. Once every 3 series of 10 movements have ended, the participants enter the testing phase. The duration of the exercise is approximately 45 minutes.
And (3) a testing stage: the testing phase comprises a total of 8 blocks, each block comprising 3 tests, one for each type of sensory feedback. The test without sensory feedback was performed every two blocks (i.e. every 6 movements). This test serves as a baseline to evaluate performance in the case of other types of sensory feedback. A total of 28 movements were performed during this test phase. In a similar way to the training phase, a pause time on the target to be reached is required. The aim is not to reciprocate as much as possible but to be precise and directed to the correct target. At the end of the test, the motion trajectory will be shown on the screen to show the participants the quality of their execution. The duration of this test phase is about 40 minutes.
Results
Figure 3 illustrates the behavior of a non-amputated subject and shows the distribution of deviation of each movement from the target during the reciprocating test, shown for four types of sensory feedback. The X-axis represents the number of stabilizations performed by the subject during the reciprocating motion. The Y-axis represents deviation from the target. One reciprocation is equivalent to two iterations. The solid or dashed lines represent the average of the deviation from the target for all tests using the same type of sensory feedback.
The traces show that for sensory feedback including vision, the subject remained very accurate on average over 3 reciprocations (deviation less than 10%). When vibrotactile sensory feedback alone is used to observe the trajectory of a condition, it is stable over time, with values better than the constraint of including a visual trajectory but lower than 20 °, which corresponds to a discretization of 2 vibrators. It can be clearly observed that the last condition without sensory feedback is different from the other 3 conditions in which the error increases over time. This demonstrates the following fact: subjects with no sensory feedback on the performance of their actions lose their reference and drift away from the goal over time. Thus, this trajectory illustrates a possible effect of vibrotactile sensory feedback in that the subject is enabled to locate himself in space. Thus, the information transmitted by the vibration is correctly used and is effective.

Claims (11)

1. A method for controlling a limb of a virtual avatar (24) by myoelectrical activity of a limb (8) of a subject (10), comprising a first calibration step comprising:
-acquiring, by means of a device (4) for measuring electromyographic activity signals, a first and a second raw calibrated electromyographic activity signal resulting from respectively a contraction of variable intensity of a first and a second antagonistic muscle of a limb (8) of a subject (10) during a given period, and then:
-determining, by a processing device (14), an envelope of each of the first signal and the second signal,
-determining, by a processing device (14), a minimum myoelectric activity and a maximum myoelectric activity from the envelope of each of the first signal and the second signal, then, for each of the two antagonistic muscles:
-determining, by a processing device (14), a myoelectric activity threshold,
-normalizing by the processing device (14) the maximal myoelectric activity and the myoelectric activity threshold,
-determining, by the processing device (14), a coefficient converting the normalized myoelectrical activity into a movement velocity component of a limb of the avatar (24), and
the method for controlling comprises a second step of moving a limb of a virtual avatar (24), the second step comprising:
-acquiring, by a device (4) for measuring an electromyographic activity signal, a first and a second raw electromyographic activity signal resulting from a contraction of two antagonistic muscles of a limb (8) of a subject (10) during a given period, and then, for each of the first and second raw electromyographic activity signals:
-normalizing the myoelectrical activity produced by said signal by means of a processing device (14),
-converting, by the processing device (14), the normalized myoelectric activity into a movement velocity component of a limb of the avatar (24) by applying a conversion coefficient,
-determining, by the processing device (14), a movement speed and a movement direction of a limb of the virtual avatar (24) by subtracting a movement speed component obtained for each muscle, and
-moving, by the processing device (14), a limb of the virtual avatar (24) at the determined movement speed and movement direction.
2. A method for controlling according to claim 1, wherein, simultaneously with the step of moving the limb of the virtual avatar (24) with the determined movement speed and movement direction, the method comprises activating a sensory feedback device (16) provided on the limb (8) of the subject (10), the sensory feedback device (16) being selectively and discretely activated in accordance with a predetermined value range of sensory parameters.
3. The method for controlling according to claim 2, wherein the sensory parameter is selected from a bending angle of an elbow of the virtual avatar (24), a forward facing angle of a shoulder of the virtual avatar (24) and a abduction/adduction angle of a shoulder of the virtual avatar (24), an inside or outside rotation angle of a shoulder of the virtual avatar (24), a bending angle of a wrist of the virtual avatar (24), a radius or ulna rotation angle of the virtual avatar (24), a degree of opening of a hand of the virtual avatar (24), a degree of opening of a finger grip of a hand of the virtual avatar (24) involving a thumb and one or more other fingers, a grip force of a hand of the virtual avatar (24), a bending angle of a knee of the virtual avatar (24), a bending angle of an ankle of the virtual avatar (24), a rotation speed of one of the motions mentioned in the present list, or any other sensory manner that can be replaced by vibration, such as a temperature or configuration of an electromyographic prosthetic (11) of the subject (10).
4. A method for controlling according to any one of claims 1 to 3, wherein the device (4) for measuring electromyographic activity signals comprises a plurality of electrodes (6), the method comprising selecting, from said plurality of electrodes (6), two electrodes (6) that enable maximum acquisition of the electromyographic activity signal of each of the two antagonistic muscles.
5. A method for control according to any one of claims 1 to 4 wherein determining minimum and maximum myoelectric activity from the envelope of the signal comprises:
-setting the raw calibration electromyographic activity signal to zero mean,
-determining an absolute value of the raw calibration myoelectric activity signal set to zero mean,
-filtering the signal at absolute value to obtain an envelope therefrom, and
-determining a minimum value and a maximum value of the envelope of the filtered signal from the envelope of the filtered signal.
6. A system adapted to implement the method for controlling a limb of a virtual avatar (24) by myoelectrical activity of a limb (8) of a subject (10) according to any of the preceding claims, the system comprising:
-a device (4) for measuring electromyographic activity signals produced by contractions of a first antagonistic muscle and a second antagonistic muscle of a limb (8) of a subject (10), the device (4) being adapted to acquire a first raw calibration electromyographic activity signal and a second raw calibration electromyographic activity signal respectively produced by contractions of variable intensity of the first antagonistic muscle and of the second antagonistic muscle of the limb (8) of the subject (10) during a given time period, and a first raw electromyographic activity signal and a second raw electromyographic activity signal produced by contractions of the two antagonistic muscles of the limb (8) of the subject (10) during the given time period,
a processing device (14) for processing myoelectrical activity, the processing device (14) comprising at least one display device (23) configured for displaying a virtual avatar (24),
-the processing device (14) is configured for determining in a first calibration step an envelope of each of the first and second signals, determining a minimum myoelectric activity and a maximum myoelectric activity from the envelope of each of the first and second signals, determining a myoelectric activity threshold, normalizing the maximum myoelectric activity and the myoelectric activity threshold, determining a coefficient to convert the normalized myoelectric activity into a movement velocity component of the limb of the avatar (24), normalizing the myoelectric activity produced by each of the first and second signals, converting the normalized myoelectric activity into a movement velocity component of the limb of the avatar (24) by applying the conversion coefficient, and
in a second step of moving the limb of the virtual avatar (24), for each of the first and second raw electromyographic activity signals:
normalizing myoelectric activity resulting from the signal, converting the normalized myoelectric activity into a movement velocity component of a limb of the avatar (24) by applying a conversion coefficient, determining a movement velocity and a movement direction of the limb of the avatar (24) by subtracting the movement velocity component obtained for each muscle, and moving the limb of the avatar (24) at the determined movement velocity and movement direction.
7. The system according to claim 6, wherein the device (4) for measuring comprises a plurality of electrodes (6) and is in the form of a bracelet.
8. The system according to claim 6 or 7, further comprising a sensory feedback device (16).
9. The system according to claim 8, wherein the sensory-feedback device (16) includes a plurality of vibratory elements (18).
10. The system according to claim 8 or 9, wherein the processing device (14) is configured for selectively and discretely activating the sensory feedback device (16) according to a predetermined range of values of a sensory parameter.
11. System according to any one of claims 6 to 10, wherein the device (4) for measuring electromyographic activity signals comprises a telecommunication device (12) configured for telecommunication with the processing device (14).
CN202180032805.2A 2020-03-12 2021-03-12 Method for controlling limbs of virtual avatar through myoelectric activity of individual limbs and system thereof Pending CN115515477A (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
FRFR2002472 2020-03-12
FR2002472A FR3108025B1 (en) 2020-03-12 2020-03-12 Method of controlling a member of a virtual avatar by the myoelectric activities of a member of a subject and related system
PCT/EP2021/056373 WO2021180945A1 (en) 2020-03-12 2021-03-12 Method for controlling a limb of a virtual avatar by means of the myoelectric activities of a limb of an individual and system thereof

Publications (1)

Publication Number Publication Date
CN115515477A true CN115515477A (en) 2022-12-23

Family

ID=71784157

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180032805.2A Pending CN115515477A (en) 2020-03-12 2021-03-12 Method for controlling limbs of virtual avatar through myoelectric activity of individual limbs and system thereof

Country Status (5)

Country Link
US (1) US11809625B2 (en)
EP (1) EP4117508A1 (en)
CN (1) CN115515477A (en)
FR (1) FR3108025B1 (en)
WO (1) WO2021180945A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023239361A1 (en) * 2022-06-08 2023-12-14 Hewlett-Packard Development Company, L.P. Intended angle based on muscle emg values

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140198034A1 (en) * 2013-01-14 2014-07-17 Thalmic Labs Inc. Muscle interface device and method for interacting with content displayed on wearable head mounted displays
US20180070864A1 (en) * 2016-06-02 2018-03-15 Matthew Schuster Methods and devices for assessing a captured motion
US20190247650A1 (en) * 2018-02-14 2019-08-15 Bao Tran Systems and methods for augmenting human muscle controls
WO2021226445A1 (en) * 2020-05-08 2021-11-11 Mvi Health Inc. Avatar tracking and rendering in virtual reality
US20230023609A1 (en) * 2021-07-22 2023-01-26 Penumbra, Inc. Systems and methods for animating a simulated full limb for an amputee in virtual reality

Also Published As

Publication number Publication date
EP4117508A1 (en) 2023-01-18
FR3108025B1 (en) 2022-02-18
US20230147243A1 (en) 2023-05-11
US11809625B2 (en) 2023-11-07
WO2021180945A1 (en) 2021-09-16
FR3108025A1 (en) 2021-09-17

Similar Documents

Publication Publication Date Title
Li Electromyography pattern-recognition-based control of powered multifunctional upper-limb prostheses
US10206793B2 (en) System and method for conscious sensory feedback
Pan et al. Myoelectric control based on a generic musculoskeletal model: Toward a multi-user neural-machine interface
Bunderson et al. Quantification of feature space changes with experience during electromyogram pattern recognition control
Scheme et al. Improving myoelectric pattern recognition positional robustness using advanced training protocols
US20170061828A1 (en) Functional prosthetic device training using an implicit motor control training system
Williams III Guest Editorial. Progress on stabilizing and controlling powered upper-limb prostheses.
Wilson et al. A third arm—Design of a bypass prosthesis enabling incorporation
CN115515477A (en) Method for controlling limbs of virtual avatar through myoelectric activity of individual limbs and system thereof
Johansen et al. A novel hand prosthesis control scheme implementing a tongue control system
Chatterjee et al. Quantifying prosthesis control improvements using a vibrotactile representation of grip force
Vargas et al. Static and dynamic proprioceptive recognition through vibrotactile stimulation
Mamidanna et al. Estimating speed-accuracy trade-offs to evaluate and understand closed-loop prosthesis interfaces
Thomas et al. Sensorimotor-inspired tactile feedback and control improve consistency of prosthesis manipulation in the absence of direct vision
CN113713252B (en) Bionic type body sense reconstruction method for prosthetic wrist and elbow joint
Alshammary et al. Assessment of a multigrasp myoelectric control approach for use by transhumeral amputees
CN114652958A (en) Replacement type wrist elbow joint proprioception reconstruction method based on nerve electrical stimulation system
Pilarski et al. Upper and lower limb robotic prostheses
JP4926042B2 (en) Neuromuscular stimulation
Kodali et al. Wearable Sensory Substitution for Proprioception via Deep Pressure
Losier et al. A control system for a powered prosthesis using positional and myoelectric inputs from the shoulder complex
US20230255802A1 (en) Method for controlling an orthopedic device and orthopedic device
Hussaini Independent pro-supination control in transradial myoelectric prostheses
Segil et al. Comparison of myoelectric control schemes for simultaneous hand and wrist movement using chronically implanted electromyography: a case series
Fougner Robust, Coordinated and Proportional Myoelectric Control of Upper-Limb Prostheses

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination