DE102020122551A1 - Method for feedback of exogenous forces to a user using electrical muscle stimulation, controller for controlling a force feedback device, force feedback device and use of a controller or a force feedback device - Google Patents

Abstract

Method for feedback of exogenous forces to a user by means of electrical muscle stimulation via a force feedback device with a stimulation device, a processor and an input device; the method comprising the steps:a) specification of a position to be assumed via the input device by the user as a task coordinate vector p for an actuating device;b) calculation of a generalized force vector Fresals force feedback for the user on the basis of the specified position p to be assumed for the actuating device;c) transformation of the generalized force vector Fresin a user-specific force moment vector T in the joint space of the user;d) assigning the joint-related force moment components Tides force moment vector T to at least one joint-actuating muscle of the user depending on the sign of the joint-related force moment components Ti;e) calculating a stimulation vector s for the muscles of the user; andf) outputting force feedback to the user via electrical muscle stimulation by outputting the stimulation currents via the stimulation device, method steps b) to e) being carried out by the processor.

Description

The present invention relates to a method for feeding back exogenous forces to a user by means of electrical muscle stimulation via a force feedback device, a control unit for controlling a force feedback device, a force feedback device and the use of a control unit according to the invention or a force feedback device according to the invention.

Both methods and devices are already known from the prior art, by means of which direct feedback is given to a user of devices when activities are carried out.

In particular, it is known to provide users, for example when using an input device, with direct feedback based on predefined movements on an input device, for example as direct force feedback. Such systems, for example in the form of a joystick for operating a computer device, have actuators for generating force feedback, for example.

The aforementioned input devices are used in particular for simulating a virtual reality in order to provide the user with haptic feedback on the interactions he has carried out in the virtual reality by means of force feedback. Virtual forces occur in virtual reality when actions are triggered by the user, for example if the user moves and/or lifts an object in virtual reality or collides with objects in virtual reality; these purely virtual forces are presented to the user as exogenous forces output as virtual reality feedback.

It is also known from the prior art to give a user of a virtual reality feedback with regard to actions carried out in the virtual reality by means of electrical muscle stimulation.

A disadvantage of the aforementioned prior art is that additional control devices such as actuators, by means of which the haptic feedback can be output to the user, often have to be provided on the input devices via which the user interacts with the virtual reality. This significantly increases the complexity of the input devices and their weight. The devices known from the prior art for feedback of a user interaction via electrical muscle stimulation have so far only been known in applications for controlling a virtual reality or simulation. Furthermore, feedback to be output is only output qualitatively and not quantitatively to the user through the electrical muscle stimulation.

Based on the aforementioned disadvantages of the prior art, it is the object of the present invention to improve the methods and devices from the prior art for feedback of exogenous forces in such a way that their structure and thus their weight are significantly reduced and at the same time the quantitative accuracy of the to increase the forces fed back to the user.

According to the invention, the object is achieved by a method for feedback of exogenous forces to a user according to claim 1, a control unit for controlling a force feedback device according to claim 13 and a force feedback device according to claim 14 and the use of a control unit or a force feedback device according to claim 15.

1. a) Vorgeben einer einzunehmenden Position über die Eingabeeinrichtung durch den Nutzer als Aufgabenkoordinatenvektor p für eine Aktuationseinrichtung;
2. b) Berechnen eines generalisierten Kraftvektors Fres als Kraftrückkopplung für den Nutzer aufgrund der vorgegebenen einzunehmenden Position p für die Aktuationseinrichtung;
3. c) Transformieren des generalisierten Kraftvektors Fres in einem nutzerspezifischen KraftmomentenvektorT im Gelenkraum q des Nutzers;
4. d) Zuordnen der gelenkbezogenen Kraftmomentkomponenten T_i des Kaftmomentenvektors T zu mindestens einem gelenkstellenden Muskel des Nutzers in Abhängigkeit von dem Vorzeichen der gelenkbezogenen Kraftmomentkomponenten T_i;
5. e) Berechnen eines Stimulationsvektors s für die Muskeln des Nutzers; und
6. f) Ausgeben einer Kraftrückkopplung an den Nutzer über eine elektrische Muskelstimulation durch Ausgabe der Stimulationsströme s_i über die Stimulationseinrichtung, wobei die Verfahrensschritte b) bis e) durch den Prozessor durchgeführt werden.

According to a first aspect, the present invention relates to a method for feeding back exogenous forces to a user by means of electrical muscle stimulation via a force feedback device with a stimulation device, a processor and an input device. The method according to the invention comprises the steps:

1. a) Predetermining a position to be assumed via the input device by the user as a task coordinate vector p for an actuating device;
2. b) calculating a generalized force vector Fres as force feedback for the user based on the predetermined position p to be assumed for the actuating device;
3. c) transforming the generalized force vector Fres into a user-specific force moment vector T in joint space q of the user;
4. d) assigning the joint-related force moment components T _i of the force moment vector T to at least one joint-actuating muscle of the user depending on the sign of the joint-related force moment components T _i ;
5. e) calculating a stimulation vector s for the user's muscles; and
6. f) Outputting force feedback to the user via electrical muscle stimulation by outputting the stimulation currents s _i via the stimulation device, method steps b) to e) being carried out by the processor.

Within the scope of the present invention, the term exogenous forces is understood to mean any forces which are to be specified within a force feedback system to a user who in particular specifies a change in position for a system on the basis of the position changes made. In particular, the exogenous forces can be, for example, frictional forces of a real handling device if a person to be taken is specified by the user for the handling device via the input device and the forces occurring within the handling device while taking the given position as force feedback to the user be issued.

Furthermore, the exogenous forces can also convey forces, for example, which occur when the handling device comes into contact with an object, in particular in the context of a collision and/or when a force is permanently applied to an object, for example in the form of a gripping device, and are returned to the user should be.

Joint adjusting muscles are understood to mean the muscle pairs that bring about the joint position and thus the position of the limbs of a user or of the user's human body.

In particular, this refers to the pair of muscles of the agonist and antagonist, but within the scope of the present invention, electrical muscle stimulation can basically be applied to all muscles of the user, particularly preferably to those that can influence the position of the limbs of a user.

wobei q_i die Gelenkwinkel bzw. Gelenkfreiheitsgrade des Nutzers und N_Gelenke die Anzahl der Gelenkfreiheitsgrade des Nutzers darstellen.The user uses the input device to specify a position to be assumed for an actuating device. The position to be taken is introduced as a task coordinate vector p in a task coordinate system. Furthermore, a joint space q of the user is introduced, with $q\in R^{N_{Gelenke}},$

where q _i represents the joint angle or joint degrees of freedom of the user and N _joints represent the number of joint degrees of freedom of the user.

der Kraftmomentvektor weist die gelenkbezogenen Kraftmomentkomponenten T_i auf welche den jeweiligen Gelenkfreiheitsgraden zugeordnet sind. Die Kraftmomentkomponenten T_i werden wiederum anhand ihres Vorzeichens mindestens einem Muskel des Nutzers zugeordnet, welcher geeignet ist die Gelenkstellung des Gelenkfreiheitsgrades zu beeinflussen. Üblicherweise wird die gelenkbezogene Kraftmomentenkomponente T_i wiederum auf mehrere Einzelmuskeln aufgeteilt, die gemeinsam die Gelenkstellung des jeweiligen Gelenkfreiheitsgrades beeinflussen können, die Kraftmomentenkomponente T_i wird damit wiederum selbst in die Anteile τ_i für die Einzelmuskeln aufgeteilt. Die Anteile τ_i können aus den geometrischen Verhältnissen und Abmessungen der Einzelmuskeln relativ zu dem Gelenk in die gelenkbezogene Kraftmomentenkomponente T_i umgerechnet werden.The force moment vector T is defined in the joint space q of the user, with q ∈ $R^{N_{Gelenke}},$

the force moment vector has the joint-related force moment components T _i which are assigned to the respective joint degrees of freedom. The force moment components T _i are in turn assigned on the basis of their sign to at least one muscle of the user which is suitable for influencing the joint position of the degree of freedom of the joint. Usually, the joint-related force moment component T _i is in turn divided among several individual muscles, which together can influence the joint position of the respective joint degree of freedom. The force moment component T _i is thus in turn divided into the portions τ _i for the individual muscles. The proportions τ _i can be converted from the geometric relationships and dimensions of the individual muscles relative to the joint into the joint-related torque component T _i .

The generalized force vector Fres is specified in the task space.

In the context of the present invention, the actuating device is preferably a handling device, such as a robot arm. The robotic arm can be attached at a first end to a static attachment position by means of appropriate attachment means. Alternatively, the robot arm can also be connected to a mobile platform at the aforementioned end. At a second end opposite the first fastening end, the handling device can in particular have a manipulation device, such as a gripper. The actuating device also has at least one joint with at least one degree of freedom, with the respective degrees of freedom being assigned adjusting devices, for example in the form of actuators, via which the actuating device can assume a predetermined position. The actuating device particularly preferably has the same number of degrees of freedom as the input device.

According to the invention, the user can use the input device to specify a position to be assumed or a position sequence to be assumed for the actuating device, in particular in the form of the handling device.

The input device preferably has the same number of degrees of freedom as the handling device or actuating device, such that a user can directly set the alignment of the joint degrees of freedom of the handling device by setting the joint degrees of freedom of the input device. This is particularly the case when the user continuously specifies a position sequence over a certain period of time by repeating the method according to the invention, which is continuously followed by the handling device, and force feedback is continuously returned to the user via the electrical stimulation device.

mit K_p als virtuelle Steifigkeitsmatrix und p als Aufgabenkoordinatenvektor.The generalized force vector Fres is preferably calculated in method step b) using the following relationship: $$f_{\mathrm{right}es}^T=f_p^T=-K_{p}p$$

with K _p as the virtual stiffness matrix and p as the task coordinate vector.

In its most general case, the virtual stiffness matrix K _p represents a non-physically motivated stiffness matrix. Rather, it conveys a strength of the feedback that is output to the user when making movements of the input or when taking certain positions of the input device shall be. The stiffness matrix can be represented, for example, as a stiffness matrix of an actuator device to be controlled by the body movements of the user, such as a robot arm, with a manipulation device, with the entries of the stiffness matrix determined for the real actuator device preferably being adapted by multiplying at least one scaling factor. The adaptation of the scaling factor can be specified or selected in a user- and task-specific manner in order to adapt the strength of the force feedback in a user- and task-specific manner.

mit J_pq(q) als Jacobi-Matrix des Körpers des Nutzers.Particularly preferably, in method step c), the force moment vector T can be calculated from the generalized force vector F _res using the following relationship: $$T_{\mathrm{right}es}^T=f_{\mathrm{right}es}^T{J}_{pq}\left(q\right)$$

with J _pq (q) as the Jacobian matrix of the user's body.

genutzt werden.When calculating the generalized force vector F _res , a force vector can preferably also be used to take into account dissipative forces and/or integrative forces using the calculation formula: $$f_{\mathrm{right}es}^T=f_p^T+f_{\dot{p}}^T+f_{\overline{p}}^T=-K_{p}p-D_{\dot{p}}\dot{p}-I_{\overline{p}}\overline{p}.$$

be used.

According to a preferred development, when calculating the generalized force vector Fres, a force vector F _external can also be included to take account of forces acting externally on the actuating device, using the following calculation formula: $$f_{\mathrm{right}es}^T=f_p^T+f_{exte\mathrm{right}n}^T=-K_{p}p+f_{exte\mathrm{right}n}^T.$$

Preferably, in step d) after the assignment of the joint-related force moment components T _i to at least one joint-actuating muscle of the user, a feedback force M _i to be generated in the at least one muscle can be calculated on the basis of the force moment component τ _i of each muscle.

The sum of all force moment components τ _i of the joint-actuating individual muscles of the joint result in turn in the desired or to be specified joint force moment T _L . The force moment components τ _i can in turn be calculated from the points of application and the positions of the respective individual muscles in relation to the joints to be positioned or divided using the geometric relationships.

The forces M _i to be generated in the muscles of the user can preferably be calculated using the following relationship, taking into account the respective position d _i of the point of application of the muscle in relation to the respective joint: $$M_i=\frac{{\tau }_i}{{\mathrm{i.e}}_i}.$$

With d _i as the length of the lever arm of the muscle relative to the respective joint, the lever arm being dependent on the respective joint angle and the electrode position.

The force to be generated within the muscle can preferably be determined taking into account the respective electrode position in relation to the muscle group. The position of the electrodes has an influence on the position of the point of application of the stimulated muscle groups and on the effect of the stimulation currents on the resulting muscle power output.

The simulation currents to be output to the user can generally be calculated using the following formula, with ◦ as a symbol for the chain of functions: $$s_i={ƒ}^{-1}\left(M_i\left({\tau }_i\right)\right)={ƒ}^{-1}\circ M_i\left({\tau }_i\right).$$

mit zugehörigem Simulationsstrom $s_i^{max}$

und einer minimal zu erzeugenden Muskelkraft $M_i^{min}$

mit zugehörigem Simulationsstrom $s_i^{min}$

anhand eines linearen Verlaufes abgebildet werden.The value of the simulation current can preferably be between a maximum muscle force to be generated $M_i^{max}$

with associated simulation stream $s_i^{max}$

and a minimum muscle power to be generated $M_i^{min}$

with associated simulation stream $s_i^{min}$

be mapped using a linear progression.

In method step a), the position to be taken can preferably be changed by changing the position of an input device by the user and Detecting the change in position of the input device via a sensor device.

According to the invention, it can also be provided that the method according to the invention is carried out repeatedly over a period of time and with a repetition frequency such that in method step a) instead of an individual position to be assumed, a sequence of positions to be assumed over a certain period of time and thus a movement sequence to be completed for the actuating device can be specified by the user, in which case the method steps a) to f) of the method according to the invention are repeatedly carried out again over the course of time at a defined repetition rate.

The repetition rate can preferably be variable over the course of the movement; in particular, the repetition rate can be increased when movements are carried out at high speeds.

The sensor device can be designed, for example, as an inertial measurement unit (IMU) or as an optical body tracking system, which can be used to specify the change in position, speed and/or acceleration to define a position to be assumed or a position sequence to be tracked.

According to a second aspect, the present invention relates to a control of a force feedback device, which comprises a memory unit and at least one processor, wherein the processor is adapted to carry out the inventive method according to the first aspect of the present invention.

According to a third aspect of the present invention, this relates to a force feedback device comprising at least one stimulation device, a control unit according to the second aspect of the invention and at least one input device for specifying a position to be assumed for an actuating device.

According to the fourth aspect of the present invention, this relates to the use of a control unit according to the second aspect of the invention or a force feedback device according to the third aspect of the invention for carrying out the method for feedback of exogenous forces to a user according to the first aspect of the invention.

Details and examples of the present invention are described below with reference to the attached figures.

• 1: die schematische Umsetzung einer Eingabevorrichtung sowie einer Kraftrückkopplungsvorrichtung für einen Nutzer;
• 2: eine beispielhafte elektrische Schaltung einer Kraftrückkopplungsvorrichtung;
• 3: schematisch die Umwandlung einer auf eine Aktuationseinrichtung einwirkende Kraft in gelenkbezogene Kraftmomente T₁ bis T₃;
• 4: die im Rahmen der vorliegenden Erfindung verwendete Modellierung der Körperanatomie des Nutzers zur Berechnung der rückzukoppelnden Muskelkräfte M₁ und M₂;
• 5 die Abhängigkeit der Muskelkraft von dem angelegten Stimulationsstrom mit einer beispielhaft angenommenen linearen Approximation der Abhängigkeitsfunktion;
• 6 die Umrechnung eines an den Nutzer rückzukoppelnden Kraftmomentes in einen Stimulationsstrom des Agonisten- bzw. Antagonistenmuskels.

Show it:

• 1 : the schematic implementation of an input device and a force feedback device for a user;
• 2 : an exemplary electrical circuit of a force feedback device;
• 3 : schematically the conversion of a force acting on an actuating device into joint-related force moments T ₁ to T ₃ ;
• 4 : the modeling of the user's body anatomy used within the scope of the present invention for calculating the muscle forces M ₁ and M ₂ to be fed back;
• 5 the dependence of the muscular strength on the applied stimulation current with a linear approximation of the dependence function assumed as an example;
• 6 the conversion of a moment of force to be fed back to the user into a stimulation current of the agonist or antagonist muscle.

the 1 shows the schematic implementation of an input device 3 according to the invention in combination with a force feedback device 2 for electrical muscle stimulation.

The input device 3 comprises according to the embodiment of FIG 1 three inertial measurement units IMU1, IMU2 and IMU3, with a position to be taken directly by the position of the extremities, in the example shown according to 1 in particular by the right arm and torso of a user 1 is specified. The position or body posture currently assumed by the user 1 is detected continuously or at specific times by means of the IMU1 to IMU3 and forwarded to a host PC via appropriate data connections. The force feedback device 2 is according to 1 designed as an electrical stimulation device 21 in the form of four electrical electrodes for muscle stimulation in the area of the upper arm of the user 1. In the embodiment shown, the electrical stimulation device 21 comprises according to FIG 1 two stimulation channels Ch1 and Ch2, each having a positive muscle stimulation electrode Ch1+ and Ch2+ and a negative electrode Ch1- and Ch2-. The signals of the IMU1 to IMU3 are converted in the host PC via a processor 41 into a task coordinate p for an actuation device 6 (in 1 not shown) converted. The processor 41 of the host PC continues to carry out method steps b) to f) according to the invention. The electrical stimulation signals to be output are transmitted by the host PC via a Bluetooth interface to an RN 41 Bluetooth module and via an ESP32 Passed module (ESP 32 Thing) to an electrical stimulation device 21 via a pulse width modulated signal (PWM). The electrical stimulation device 21 has, in the exemplary embodiment according to FIG 1 a control unit (control electronics) with an electrode interface for connecting the muscle stimulation electrodes (Ch1+, Chi-, Ch2+ and Ch2-). The electrode interface is in turn connected to a transcutaneous electrical muscle nerve stimulator (TENS muscle stimulator), which emits electrical muscle stimulation signals.

the 2 12 continues to refer to the exemplary embodiment of FIG 1 the electrical wiring of the exemplary electrical stimulation device 21 according to FIG 1 .

3 shows schematically the conversion of exogenous forces acting on an actuation device 6 into user-specific joint-associated force moments T ₁ to T ₃ for force feedback to the user 1. The actuation device 6 is according to FIG 3 designed in the form of a humanoid robot with a torso and associated arms. An exogenous force F acts on the actuating device 6, for example it is an external force which acts on the actuating device 6 due to the assumption of a position to be assumed by collision with an object. According to the invention, the exogenous force F is now converted into a generalized force vector F _res for force feedback. Furthermore, the generalized force vector Fres is converted into a user-specific force moment vector T in the joint space of the user 1 . The force moment vector T contains the force moments to be applied in the joints as part of the force feedback, for example the force moments T ₁ to T ₃ .

the 4 12 illustrates the idealization of the muscle anatomy of the user 1 carried out according to the invention using the example of a user's arm, the arm flexor muscle being modeled and designated as M1 and the arm extensor muscle as M2. The distance of the attachment point of the arm flexor muscle M1 relative to the forearm joint is defined as distance d ₁ and the distance of the arm extensor muscle M2 relative to the aforementioned joint is defined as distance d ₂ . The sum of the moments of force generated by the muscles M1 and M2 gives the moment of force T ₂ associated with the joint, as also in 3 shown.

und einer maximal aufzubringenden Muskelkraft $M_i^{max}$

wird der Verlauf der Abhängigkeit des Stimulationsstroms s_i relativ zu der Muskelkraft M_i erfindungsgemäß vereinfacht als Gerade und damit eine lineare Abhängigkeit zwischen den beiden Größen angenommen, welche als gestrichelte Linie in 5 wiedergegeben ist.the 5 shows the dependency of a generated muscle force M _i due to a stimulation current s _i applied to the user muscle. The curve between the stimulation current s _i and the muscle force M _i runs approximately based on the curve of a sigmoid function, which is shown by the solid line. In the range between a minimum muscle force to be applied $M_i^{min}$

and a maximum muscle force that can be applied $M_i^{max}$

the course of the dependency of the stimulation current s _i relative to the muscle force M _i is simplified according to the invention as a straight line and thus a linear dependency between the two variables is assumed, which is shown as a dashed line in 5 is reproduced.

the 6 clarifies the inventive assignment of a joint-associated moment of force T _i to be applied as a function of its sign to the agonist or antagonist muscle as a joint-associated moment of force T _i of a user 1 and the dependence of the stimulation current s _i output to the user 1 via the electrical stimulation device 21.

Claims (15)

Method for feedback of exogenous forces to a user (1) by means of electrical muscle stimulation via a force feedback device (2) with a stimulation device (21), a processor (41) and an input device (3); the method comprising the steps: a) specifying a position to be assumed via the input device (3) by the user (1) as a task coordinate vector p for an actuating device (6); b) calculating a generalized force vector Fres as force feedback for the user (1) on the basis of the predetermined position p to be assumed for the actuating device (6); c) transforming the generalized force vector Fres into a user-specific force moment vector T in the joint space of the user (1); d) assigning the joint-related force moment components T _i of the force moment vector T to at least one joint-actuating muscle of the user (1) as a function of the sign of the joint-related force moment components T _i ; e) calculating a stimulation vector s for the muscles of the user (1); and f) outputting force feedback to the user (1) via electrical muscle stimulation by outputting the stimulation currents s _i via the stimulation device (21), method steps b) to e) being carried out by the processor (41). procedure after claim 1 , where the generalized force vector Fres is calculated in method step b) using the following relationship: $$f_{\mathrm{right}es}^T=f_p^T=-K_{p}p$$

with K _p as the virtual stiffness matrix and p as the task coordinate vector. Method according to one of the preceding claims, wherein in method step c) the force moment vector T is calculated from the generalized force vector F _res using the following relationship: $$T_{\mathrm{right}es}^T=f_{\mathrm{right}es}^T{J}_{pq}\left(q\right)$$

with J _pq (q) as the Jacobian matrix of the user's body (1). Method according to one of the preceding claims, wherein in the calculation of the generalized force vector F _res a force vector for considering dissipative forces and/or integrative forces is also taken into account using the calculation formula: $$f_{\mathrm{right}es}^T=f_p^T+f_{\dot{p}}^T+f_{\overline{p}}^T=-K_{p}p-D_{\dot{p}}\dot{p}-I_{\overline{p}}\overline{p}.$$

Method according to one of the preceding claims, wherein a force vector F _external is also included in the calculation of the generalized force vector Fres to take account of externally acting forces, using the following calculation formula: $$f_{\mathrm{right}es}^T=f_p^T+f_{exte\mathrm{right}n}^T=-K_{p}p+f_{exte\mathrm{right}n}^T.$$

Method according to one of the preceding claims, wherein in method step d) after the assignment of the joint-related force moment components T _i to at least one joint-actuating muscle of the user (1), a feedback force M _i to be generated in the at least one muscle is calculated on the basis of the force moment component τ _i of each muscle . procedure after claim 6 , whereby the forces M _i to be generated in the muscles of the user (1) are calculated using the following relationship, taking into account the respective distance d _i of the point of application of the muscle to the respective joint: $$M_i=\frac{{\tau }_i}{{\mathrm{i.e}}_i}.$$

Procedure according to one of Claims 6 or 7 , where the force to be generated is determined taking into account the respective electrode position in relation to the muscle group, with the positions of the muscle application points and thus the lever arms d _i being approximated in a calibration phase based on the measurement of the movements stimulated by a defined stimulation current on the user. Method according to one of the preceding claims, wherein the simulation currents s _i to be output to the user (1) are calculated via the following relationship: $$s_i={ƒ}^{-1}\left(M_i\left({\tau }_i\right)\right)={ƒ}^{-1}\cdot M_i\left({\tau }_i\right).$$

procedure after claim 9 , where the value of the simulation current is between a maximum muscle force to be generated $M_i^{max}$

with associated simulation stream $s_i^{max}$

and a minimum muscle power to be generated $M_i^{min}$

with associated simulation stream $s_i^{min}$

is mapped using a linear progression. Method according to one of the preceding claims, wherein in method step a) the position to be assumed takes place by changing the position of an input device (3) by the user (1) and detecting the change in position of the input device (3) via a sensor device. procedure after claim 11 , wherein the sensor device comprises at least one inertial measurement unit (IMU). Control unit (4) for controlling a force feedback device (2), comprising a memory unit (43) and at least one processor (41), wherein the processor (41) is adapted to a method according to any one of Claims 1 until 12 to execute. A force feedback device (2) comprising: - at least one stimulation device (21); - A control unit (4) after Claim 13 and - at least one input device (3) for specifying a position to be assumed for an actuating device (6). Using a control unit (4) according to Claim 13 or a force feedback device (2). Claim 14 for carrying out the method for feedback of exogenous forces to a user (1) according to one of Claims 1 until 12 .

Images