CN118061200B - Force feedback method and device for teleoperation system based on vibration induction - Google Patents

Force feedback method and device for teleoperation system based on vibration induction Download PDF

Info

Publication number
CN118061200B
CN118061200B CN202410471429.2A CN202410471429A CN118061200B CN 118061200 B CN118061200 B CN 118061200B CN 202410471429 A CN202410471429 A CN 202410471429A CN 118061200 B CN118061200 B CN 118061200B
Authority
CN
China
Prior art keywords
arm
vibration
joint
muscle
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410471429.2A
Other languages
Chinese (zh)
Other versions
CN118061200A (en
Inventor
张世武
朱阅微
王腾达
高纬
张晓虎
谢远哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202410471429.2A priority Critical patent/CN118061200B/en
Publication of CN118061200A publication Critical patent/CN118061200A/en
Application granted granted Critical
Publication of CN118061200B publication Critical patent/CN118061200B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The invention discloses a force feedback method and a force feedback device for a teleoperation system based on vibration induction, which are used for acquiring arm information of an operator and constructing an arm end model according to the arm information; controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture; determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data; determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment; and determining the vibration intensity of a vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity. The invention gives the subcutaneous susceptor a certain degree of mechanical vibration through the vibration motor, so as to trigger the illusion of human actions and realize good in-situ feeling.

Description

Force feedback method and device for teleoperation system based on vibration induction
Technical Field
The invention relates to the technical field of teleoperation system control, in particular to a force feedback method and device for a teleoperation system based on vibration induction.
Background
With the continuous progress of robotics, a wide variety of robots have been widely used in human production activities. When there are work tasks to be performed at a particular location, robotic systems are better suited than humans for performing the corresponding work tasks, such as remote processing of hazardous materials, aerospace exploration, and underwater tasks, for accessibility, cost, and/or safety reasons. Due to the complex variability of the operating environment and tasks, the robot needs to flexibly process task content in task execution work, but is limited by the development of software and hardware of the robot, and the existing robot system cannot effectively and flexibly work in any environment. In this case, the robot can effectively combine the human intelligence and decision-making ability with the robot's executive ability to complete the corresponding work task through the teleoperation system.
Teleoperation systems need to achieve a good feeling of presence. Ideally, the presence can improve the quality of the operator's performance of tasks if the robotic system transmits enough information to the operator and naturally displays it. Currently, teleoperational systems generally achieve a realistic sensation through two main haptic channels (kinesthesia and skin feel) of humans, so there are different types of feedback methods: skin sensation is the reception of sensory input from receptors embedded in the skin, while kinesthetic sensation is the reception of sensory input from receptors within muscles, tendons and joints. Among them, the feedback of skin sensation is tactile feedback by electric or thermal stimulation on the cutaneous epidermal nerve, several tactile interfaces have been currently developed to trigger this type of tactile feedback, but these approaches present the risk of injuring the subject. Kinesthetic feedback often requires externally applied resistance, but such devices tend to be bulky, difficult to move, and expensive to manufacture.
Therefore, the feedback methods of the existing teleoperation systems have certain limitations, and the force feedback with light weight, low cost and good in-situ effect cannot be realized.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the force feedback method and the device for the teleoperation system based on vibration induction are provided, and the force feedback with light weight, low cost and good in-situ effect is realized.
In order to solve the technical problems, the invention adopts the following technical scheme:
A vibration-induced force feedback method for a teleoperational system, comprising:
Acquiring arm information of an operator, and constructing an arm end model according to the arm information;
Controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture;
determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data;
Determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment;
and determining the vibration intensity of a vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity.
In order to solve the technical problems, the invention adopts another technical scheme that:
the force feedback device for the teleoperation system based on vibration induction comprises an inertial sensor, a vibration motor and an information processing unit; the vibration motor is fixed at the arm muscle of the operator; the inertial sensor is fixed at an arm joint of an operator;
The information processing unit is used for acquiring arm information of an operator through the inertial sensor and constructing an arm tail end model according to the arm information;
Controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture;
determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data;
determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment; and
And determining the vibration intensity of the vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity.
The invention has the beneficial effects that: and the slave end robot is controlled to execute the current arm gesture of the operator by acquiring the arm information of the operator to construct an arm end model, and arm end force data of the slave end robot are acquired under the arm gesture to determine the current arm stress condition of the slave end robot. Meanwhile, based on the current arm information of an operator and the force data of the tail end of the arm of the slave robot, the joint moment of the operator under the current arm gesture and when the tail end of the arm is stressed can be determined, and then the nerve activation degree of the arm muscle is determined through the joint moment, so that the vibration intensity of the vibration motor is calculated through the nerve activation degree to carry out force feedback. According to the invention, the vibration motor is arranged at the muscle of the human arm, and the corresponding vibration intensity is set according to the stress degree of the human arm, so that the subcutaneous susceptor is given a certain degree of mechanical vibration, the illusion of human actions is induced, good feeling of reality is realized, and the safety of the force feedback mode is high. In addition, the exoskeleton device is not required to be used for force feedback, so that the cost is effectively reduced, and the weight reduction of a force feedback mode is realized.
Drawings
FIG. 1 is a flow chart of steps of a force feedback method for a teleoperational system based on vibration induction provided by an embodiment of the present invention;
FIG. 2 is an interactive schematic diagram of a force feedback method for a teleoperation system based on vibration induction according to an embodiment of the present invention;
Fig. 3 is a coordinate system of a human arm and seven axial schematic diagrams according to an embodiment of the present invention;
fig. 4 is a schematic diagram of arm muscles corresponding to a vibration motor according to an embodiment of the present invention;
FIG. 5 is a schematic diagram showing the proportional relationship between vibration intensities according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of an inertial sensor arrangement according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of an arrangement of a vibration motor according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of another vibration motor according to an embodiment of the present invention;
Description of the reference numerals:
200. A force feedback device for a teleoperational system based on vibration induction; 201. an inertial sensor; 202. a vibration motor; 2011. a first inertial sensor; 2012. a second inertial sensor; 2013. a third inertial sensor; 2021. a first vibration motor; 2022. a second vibration motor; 2023. a third vibration motor; 2024. a fourth vibration motor; 2025. a fifth vibration motor; 2026. a sixth vibration motor; 2027. a seventh vibration motor; 2028. and an eighth vibration motor.
Detailed Description
In order to describe the technical contents, the achieved objects and effects of the present invention in detail, the following description will be made with reference to the embodiments in conjunction with the accompanying drawings.
Referring to fig. 1 to 4, a force feedback method for a teleoperation system based on vibration induction according to an embodiment of the present invention includes:
Acquiring arm information of an operator, and constructing an arm end model according to the arm information;
Controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture;
determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data;
Determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment;
and determining the vibration intensity of a vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity.
From the above description, the beneficial effects of the invention are as follows: and the slave end robot is controlled to execute the current arm gesture of the operator by acquiring the arm information of the operator to construct an arm end model, and arm end force data of the slave end robot are acquired under the arm gesture to determine the current arm stress condition of the slave end robot. Meanwhile, based on the current arm information of an operator and the force data of the tail end of the arm of the slave robot, the joint moment of the operator under the current arm gesture and when the tail end of the arm is stressed can be determined, and then the nerve activation degree of the arm muscle is determined through the joint moment, so that the vibration intensity of the vibration motor is calculated through the nerve activation degree to carry out force feedback. According to the invention, the vibration motor is arranged at the muscle of the human arm, and the corresponding vibration intensity is set according to the stress degree of the human arm, so that the subcutaneous susceptor is given a certain degree of mechanical vibration, the illusion of human actions is induced, good feeling of reality is realized, and the safety of the force feedback mode is high. In addition, the exoskeleton device is not required to be used for force feedback, so that the cost is effectively reduced, and the weight reduction of a force feedback mode is realized.
Further, the arm information includes a posture angle of each arm joint and a trunk length between adjacent arm joints; the arm tail end model comprises a standard position model and a standard posture model;
The obtaining arm information of the operator and constructing an arm end model according to the arm information comprises:
acquiring the attitude angles and the trunk lengths of the joints of the arms of an operator;
determining a first rotation matrix on each arm joint according to the attitude angle;
And constructing a standard position model and a standard posture model according to the first rotation matrix on each arm joint and the length of the trunk.
From the above description, the human arm can be approximately considered as a connecting rod structure, so that the current position and the gesture of the operator arm can be accurately determined by acquiring the gesture angle of each arm joint of the operator and the length of each section of trunk of the arm, thereby accurately controlling the slave end robot to execute the current arm gesture of the operator, realizing teleoperation, avoiding error acquisition of the tail end force data of the slave end robot arm caused by inconsistent gesture between the operator arm and the slave end robot arm, and influencing the in-situ effect generated by the vibration motor.
Further, the building of the standard position model and the standard posture model according to the rotation matrix on each arm joint and the length of the trunk is specifically as follows:
Wherein P e represents a standard position model; r e represents a standard posture model; r 1 represents the first rotation matrix of the shoulder arm joint; r 2 represents a first rotation matrix of the elbow arm joint; r 3 represents a first rotation matrix of the wrist-arm joint; l 1 denotes a large arm torso vector, L 1=[0,0,-l1],l1 denotes a large arm torso length; l 2 denotes the forearm torso vector, L 2=[0,0,-l2],l2 denotes the torso length of the forearm; l 3 denotes a palm torso vector, and L 3=[0,0,-l3],l3 denotes a palm torso length.
From the above description, it can be seen that the human arm joints include shoulders, elbows and wrists, the human arm trunk includes a large arm, a small arm and a palm, the direction of the arm trunk is determined based on the rotation angle of each arm joint, and the rotation center of the next arm joint is determined based on the length of the arm trunk, so that the accurate analysis and acquisition of the arm end position and the arm posture are realized, and the accurate arm end force data is acquired by the slave end robot controlled by an operator.
Further, the determining the virtual joint moment of the target arm gesture according to the arm information and the arm end force data includes:
determining the rotation angle values of the target arm gesture in seven axial directions according to the arm information;
Determining a second rotation matrix on each arm joint according to the rotation angle value;
constructing a trunk end position model according to the second rotation matrix and the trunk length;
and determining the virtual joint moment of the target arm gesture according to the trunk tail end position model and the arm tail end force data.
From the above description, the human arm can be modeled as a seven-degree-of-freedom link structure, and the position model of each trunk end can be accurately constructed by determining the rotation angle value in each degree of freedom and the trunk length corresponding to each degree of freedom, so as to determine the trunk posture of each trunk under the target arm posture, further analyze the moment born by the joint corresponding to the trunk currently when the arm end is stressed, and each arm joint has corresponding arm muscles to control the change of the joint angle, so that the stress condition of each arm muscle can be determined through the joint moment, the vibration intensity of the motor can be determined through the nerve excitation activity of different arm muscles subsequently, and the force feedback feeling is optimized.
Further, the determining, according to the arm information, the rotation angle value of the target arm gesture in seven axial directions is specifically:
Wherein, AndRespectively representing the attitude angles of the shoulder arm joint in the x-axis direction, the y-axis direction and the z-axis direction; And Representing the rotation angle values in seven axial directions; Element values representing the j-th column of the i-th row in matrix c; r 1 represents the first rotation matrix of the shoulder arm joint; r 2 represents a first rotation matrix of the elbow arm joint; r 3 represents a first rotation matrix of the wrist-arm joint; representing a unit vector in the z-axis direction.
From the above description, it is known that modeling the human arm as a seven-degree-of-freedom link structure, in which the overall tracking of the human arm joint position and posture is achieved, corresponds to three degrees of freedom (supination/supination, adduction/abduction, and flexion/extension) of the shoulder, two degrees of freedom of the elbow (adduction/supination and flexion/extension), and two degrees of freedom of the wrist (adduction/abduction and flexion/extension), respectively.
Further, the determining the second rotation matrix on each arm joint according to the rotation angle value specifically includes:
Wherein, AndRepresenting a second rotation matrix on the shoulder arm joint, the elbow arm joint, and the wrist arm joint, respectively; And A rotation matrix representing an x-axis direction, a y-axis direction, and a z-axis direction;
the constructing a trunk end position model according to the second rotation matrix and the trunk length is specifically:
Wherein O represents a shoulder arm joint, A represents an elbow arm joint, B represents a wrist arm joint, and E represents an arm end; A trunk end position model taking N as a head end M as an end is represented; A homogeneous transformation matrix representing a large arm torso vector; a homogeneous transformation matrix representing a forearm torso vector; a homogeneous transformation matrix representing the palm torso vector.
As can be seen from the above description, since the arm muscle groups controlling each arm joint are different, an independent analysis and calculation are required for the position model of each arm joint, so that the subsequent independent analysis and determination of the stress state of each muscle group are facilitated, and the optimization of the motor vibration effect is ensured.
Further, the determining the virtual joint moment of the target arm gesture according to the trunk tail end position model and the arm tail end force data specifically includes:
Wherein, Representing the moment of the virtual joint,The jacobian matrix representing matrix P', T representing the homogeneous transformation matrix, and f representing the arm tip force data.
From the above description, since the arm end force data is commonly stressed by a plurality of different torsos, the current virtual joint moment is determined by performing joint calculation based on all the current torsos end positions and the arm end force data.
Further, the determining the nerve activation degree of the different arm muscles under the target arm posture according to the virtual joint moment comprises:
calculating the muscle activation degree of different arm muscles under the target arm gesture according to the virtual joint moment, wherein the muscle activation degree is specifically as follows:
Wherein, Represents the muscle activation, M represents a constant coefficient matrix set according to different muscle sensitivities,Representing a virtual joint moment;
determining the nerve activation degree of the arm muscle according to the muscle activation degree, specifically:
Wherein, The constant is represented by a value that is a function of,Indicating nerve activation activity.
From the above description, the activation of the arm muscle can be determined by calculating the virtual joint moment and the sensitivities of different arm muscles, so that the nerve activation corresponding to different muscle activation is determined based on the mapping relationship between the muscle activation and the nerve activation, and the feeling of the human nerve in different muscle activation states is determined, so that the same nerve feeling can be generated on the human nerve through motor vibration, the muscle activation state is simulated, and the illusion of action is generated.
Further, arm muscles provided with the vibration motor include the anterior and middle bundles of deltoid, the long and short head tendons of biceps brachii, the triceps brachii, the flexor muscle group of the forearm, the extensor carpi radialis and extensor muscle group;
The method for determining the vibration intensity of the vibration motor corresponding to the arm muscle according to the nerve activation activity comprises the following steps:
Wherein, AndThe vibration intensities of the vibration motors corresponding to the anterior and middle bundles of the deltoid muscle, the long and short head tendons of the biceps brachii, the flexor muscle group of the brachial triceps, the extensor carpi radialis and extensor muscle group of the forearm,Representing functions for neural networks, k 1、k2、k3、k4、k5、k6 and k 7 representing preset empirical parameters, u 1、u2、u3、u4、u5、u6 and u 7 representing neural activation of arm muscles corresponding to the seven axes, respectively.
From the above description, it is known that the movement of the human arm is achieved by the activation of the arm muscles, i.e. the muscles contract and then force is generated, thereby moving the joints and completing the movements, while each different muscle group controls the corresponding joint. The invention establishes a certain numerical relation between the nerve activation and the motor vibration intensity, thereby simulating the feeling of muscle group contraction through motor vibration, and inducing human action errors so as to generate good feeling of reality.
Referring to fig. 6 to 8, another embodiment of the present invention provides a force feedback device for a teleoperation system based on vibration induction, which includes an inertial sensor, a vibration motor and an information processing unit; the vibration motor is fixed at the arm muscle of the operator; the inertial sensor is fixed at the arm trunk of the operator;
The information processing unit is used for acquiring arm information of an operator through the inertial sensor and constructing an arm tail end model according to the arm information;
Controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture;
determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data;
determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment; and
And determining the vibration intensity of the vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity.
From the above description, the beneficial effects of the invention are as follows: and the slave end robot is controlled to execute the current arm gesture of the operator by acquiring the arm information of the operator to construct an arm end model, and arm end force data of the slave end robot are acquired under the arm gesture to determine the current arm stress condition of the slave end robot. Meanwhile, based on the current arm information of an operator and the force data of the tail end of the arm of the slave robot, the joint moment of the operator under the current arm gesture and when the tail end of the arm is stressed can be determined, and then the nerve activation degree of the arm muscle is determined through the joint moment, so that the vibration intensity of the vibration motor is calculated through the nerve activation degree to carry out force feedback. According to the invention, the vibration motor is arranged at the muscle of the human arm, and the corresponding vibration intensity is set according to the stress degree of the human arm, so that the subcutaneous susceptor is given a certain degree of mechanical vibration, the illusion of human actions is induced, good feeling of reality is realized, and the safety of the force feedback mode is high. In addition, the exoskeleton device is not required to be used for force feedback, so that the cost is effectively reduced, and the weight reduction of a force feedback mode is realized.
The force feedback method and device for the teleoperation system based on vibration induction provided by the embodiment of the invention can be applied to a main end robot of the teleoperation system to realize the force feedback with light weight, low cost and good in-situ effect, and the following is explained by a specific embodiment:
referring to fig. 1 to 5, a first embodiment of the present invention is as follows:
Referring to fig. 1 and 2, a vibration-induced force feedback method for a teleoperational system, comprising:
S1, acquiring arm information of an operator, and constructing an arm end model according to the arm information.
The arm information comprises the attitude angles of all the arm joints and the length of the trunk between the adjacent arm joints; the arm end model includes a standard position model and a standard posture model.
Specifically, step S1 includes:
S11, acquiring the attitude angles and the trunk lengths of the joints of the arms of the operator.
In some embodiments, the arm joints include shoulder arm joints, elbow arm joints, and wrist arm joints, then the torso between adjacent shoulder arm joints and elbow arm joints is the large arm, the torso between adjacent elbow arm joints and wrist arm joints is the small arm, and the torso between the wrist arm joints to the distal ends of the arms is the palm.
In some embodiments, the torso length may be set according to the length of the large arm, the small arm, and the palm as described in the chinese adult human body size (GB/T10000-2023) document.
S12, determining a first rotation matrix on each arm joint according to the attitude angle.
In some embodiments, step S13 is specifically:
(1)
Wherein, Representing the attitude angle in the x-axis direction acquired by the inertial sensor,Representing the attitude angle in the y-axis direction acquired by the inertial sensor,The attitude angle in the z-axis direction acquired by the inertial sensor is represented by i, i representing different arm joints, where i=1 represents shoulder arm joints, i=2 represents elbow arm joints, and i=3 represents wrist arm joints. In the above formula (1)AndThe calculation method of (2) and the following step 32AndThe calculation method of (2) is the same.
S13, constructing a standard position model and a standard posture model according to a first rotation matrix and the length of a trunk on each arm joint, wherein the standard position model and the standard posture model are specifically as follows:
Wherein P e represents a standard position model; r e represents a standard posture model; r 1 represents the first rotation matrix of the shoulder arm joint; r 2 represents a first rotation matrix of the elbow arm joint; r 3 represents a first rotation matrix of the wrist-arm joint; l 1 denotes a large arm torso vector, L 1=[0,0,-l1],l1 denotes a large arm torso length; l 2 denotes the forearm torso vector, L 2=[0,0,-l2],l2 denotes the torso length of the forearm; l 3 denotes a palm torso vector, and L 3=[0,0,-l3],l3 denotes a palm torso length.
It should be noted that, due to the rotation matrixThe rotation matrix is a3×3 matrix, so in order to characterize the direction of the trunk length, the standard position and standard posture of the arm end are determined by constructing a corresponding trunk vector and a first rotation matrix.
S2, controlling the slave end robot to execute the target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture.
In some embodiments, the slave end robot is a robotic arm. The device can remotely control the mechanical arm to execute the current target arm gesture of the operator, and at the moment, the mechanical arm operates related equipment in an actual scene, so that the tail end of the mechanical arm has real stress, but the tail end of the mechanical arm of the operator does not have real stress.
In some embodiments, the slave end robot is provided with a force sensor at the arm end, and arm end force data of the slave end robot in the target arm pose is acquired through the force sensor.
And S3, determining the virtual joint moment of the target arm gesture according to the arm information and the arm tail end force data.
Specifically, referring to fig. 3, step S3 includes:
S31, determining a rotation angle value of the target arm gesture in seven axial directions according to the arm information, wherein the rotation angle value is specifically:
Wherein, AndRespectively representing the attitude angles of the shoulder arm joint in the x-axis direction, the y-axis direction and the z-axis direction; And Representing the rotation angle values in seven axial directions; Element values representing the j-th column of the i-th row in matrix c; r 1 represents the first rotation matrix of the shoulder arm joint; r 2 represents a first rotation matrix of the elbow arm joint; r 3 represents a first rotation matrix of the wrist-arm joint; representing a unit vector in the z-axis direction.
Specifically, the left side in fig. 3 shows a schematic diagram of the coordinate system of different arm joints; the right side in fig. 3 shows seven axial schematic diagrams. The seven axes include three axes of shoulder arm joints (supination/supination, adduction/abduction, and flexion/extension), two axes of elbow arm joints (adduction/abduction and flexion/extension), two axes of wrist arm joints (adduction/abduction and flexion/extension). Wherein the adduction/abduction axis of the elbow joint coincides with the supination/supination axis of the wrist joint, so that in practice seven axes are represented by three axes of the shoulder joint, one axis of the elbow joint and three axes of the wrist joint, as shown on the right side in figure 3,Represents the adduction/abduction (x 1) of the shoulder arm joint,Representing the flexion/extension (y 1) of the shoulder arm joint,Indicating the supination/pronation (z 1) of the shoulder arm joint,Representing the flexion/extension (y 2) of the elbow arm joint,Represents adduction/abduction (x 3) of the wrist arm joint,Indicating the swing (y 3) of the wrist and arm joint,Indicating supination/supination (z 3) of the wrist and arm joint.
S32, determining a second rotation matrix on each arm joint according to the rotation angle value, wherein the second rotation matrix is specifically:
Wherein, AndRepresenting a second rotation matrix on the shoulder arm joint, the elbow arm joint, and the wrist arm joint, respectively; And A rotation matrix representing the x-axis direction, the y-axis direction, and the z-axis direction.
S33, constructing a trunk end position model according to the second rotation matrix and the trunk length, wherein the model specifically comprises the following steps:
Wherein O represents a shoulder arm joint, A represents an elbow arm joint, B represents a wrist arm joint, and E represents an arm end; A trunk end position model taking N as a head end M as an end is represented; A homogeneous transformation matrix representing a large arm torso vector; a homogeneous transformation matrix representing a forearm torso vector; a homogeneous transformation matrix representing the palm torso vector.
S34, determining a virtual joint moment of the target arm gesture according to the trunk tail end position model and the arm tail end force data, wherein the virtual joint moment is specifically as follows:
Wherein, Representing the moment of the virtual joint,The jacobian matrix representing matrix P', T representing the homogeneous transformation matrix, and f representing the arm tip force data.
And S4, determining nerve activation activities of different arm muscles under the gesture of the target arm according to the moment of the virtual joint.
Specifically, step S4 includes:
s41, calculating the muscle activation degrees of different arm muscles under the gesture of the target arm according to the moment of the virtual joint, wherein the muscle activation degrees are specifically as follows:
Wherein, Represents the muscle activation, M represents a constant coefficient matrix set according to different muscle sensitivities,Representing the virtual joint moment.
It should be noted that the number of the substrates,Wherein diag represents a diagonal matrix, m i of which correspond to the seven axes of the arm, respectively. In the practical application process, M can be finely adjusted according to the use sensing condition of the operator, and in general, M can be divided into three muscle sensitivities with different gradients, namely deltoid muscle group, brachial muscle group and forearm muscle group.
S42, determining the nerve activation degree of the arm muscle according to the muscle activation degree, wherein the nerve activation degree is specifically as follows:
Wherein, The activity of the muscle is indicated,The constant is represented by a value that is a function of,In the practical application process, the method comprises the following steps,Can be set to 0.021; Indicating nerve activation activity. Wherein the nerve activation activities u 1 to u 7 and the muscle activation degree To the point ofIn a one-to-one correspondence,To the point ofThe muscle activation of the arm muscles corresponding to the seven axes are respectively represented.
And S5, determining the vibration intensity of the vibration motor corresponding to the arm muscle according to the nerve activation degree, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity.
With reference to fig. 4, the arm muscles provided with the vibration motor include the anterior and middle bundles of the deltoid muscle, the long and short head tendons of the biceps brachii, the flexor muscle group of the triceps brachii and the forearm, the extensor carpi radialis and extensor muscle group of the extensor carpi radialis, the face a in fig. 4 represents the front of the human body, and the face B in fig. 4 represents the back of the human body. In some embodiments, the vibration motor may be fixed inside a flexible arm cuff, the flexible arm cuff is worn to the operator's arm, and the vibration motor is adjusted to correspond to the arm muscles. Or the vibration motors are fixed on the muscle of the arm one by means of binding bands or adhesion and the like.
The human body movement is realized by activating muscles, and the muscles shrink to generate force so as to move joints and complete actions. While the slave robot usually needs to perform work tasks against external environment force, so that corresponding vibration motors are required to be arranged on different muscles to simulate muscle feeling when a human body is actually stressed, thereby inducing motion illusion. Wherein, each arm joint has corresponding muscle which is responsible for the change control of the joint angle. The muscle group responsible for three degrees of freedom of the arm joints of the shoulder is deltoid, deltoid toe is used for controlling internal rotation (z1+) and flexion (y 1-) of the shoulder, deltoid midtoe is used for controlling abduction (x 1-) of the shoulder, and deltoid toe is used for controlling external rotation (z 1-) and extension (y1+). The muscle groups responsible for the two degrees of freedom of the elbow joints are the triceps brachii, which is the only muscle responsible for elbow joint extension, and biceps brachii, which is the only muscle responsible for elbow joint extension, which is used to control elbow flexion (y 2-). The muscle groups responsible for two degrees of freedom of the wrist and arm joint are the flexor muscle group for controlling wrist adduction (x 3+), the extensor muscle group for controlling wrist abduction (x 3-), the extensor muscle group for controlling wrist swing (y 3-).
Further, since the upper limb movements are the result of the cooperation of multiple muscles, each effort requires the cooperation of multiple muscles. The reason for the axial direction which is not involved in the control of the muscle group is that the frequency of the axial force is low, and meanwhile, based on the cooperative relationship of other muscle groups, the corresponding vibration can be arranged on the other muscle groups to assist in simulating the feeling of reality of the muscle group, so that a vibration motor is not arranged on part of the axial control muscle groups, thereby reducing the complexity of the system. For example, when the muscle group corresponding to the x1+ axis generates force, the muscle of the forearm cooperatively generates force, and the corresponding vibration can be set on the muscle of the forearm to assist in simulating the presence of the muscle group corresponding to the x1+ axis. When the y < 3+ > axially corresponding muscle group generates force, the triceps brachii muscle cooperatively generates force, and the corresponding vibration can be arranged on the triceps brachii muscle to assist in simulating the feeling of reality of the y < 3+ > axially corresponding muscle group.
In step S5: the vibration intensity of the vibration motor corresponding to the arm muscle is determined according to the nerve activation degree, specifically:
Wherein, AndThe vibration intensities of the vibration motors corresponding to the anterior and middle bundles of the deltoid muscle, the long and short head tendons of the biceps brachii, the flexor muscle group of the brachial triceps, the extensor carpi radialis and extensor muscle group of the forearm,Representing functions for neural networks, k 1、k2、k3、k4、k5、k6 and k 7 representing preset empirical parameters, u 1、u2、u3、u4、u5、u6 and u 7 representing neural activation of arm muscles corresponding to the seven axes, respectively.
In some embodiments, the vibration intensities of the vibration motors on the associated arm muscles are correlated, i.e., when the vibration intensity of the vibration motor on the primary arm muscle changes, the vibration intensity of the vibration motor on the secondary arm muscle also changes by a certain proportion. For example, there is a certain proportional relationship between the vibration intensities of the vibration motors located on the long-head tendon and the short-head tendon of the biceps brachii, and as shown in fig. 5, the vibration intensities of the vibration motors located on the long-head tendon of the biceps brachii are increased by the flexion of the elbow jointIncrease the vibration intensity of the vibration motor on the tendon of the biceps brachii short headThe force direction of the perceived muscle is simulated in this way.
Referring to fig. 6 to 8, a second embodiment of the present invention is as follows:
A force feedback device 200 for a teleoperation system based on vibration induction, comprising an inertial sensor 201, a vibration motor 202, and an information processing unit; the vibration motor 202 is fixed at an arm muscle of an operator; the inertial sensor 201 is fixed at the arm trunk of the operator; the information processing unit is configured to acquire arm information of an operator through the inertial sensor 201 and perform the steps in the vibration-induced force feedback method for a teleoperation system according to the first embodiment, so as to determine the vibration intensity of the vibration motor 202 at each arm muscle, thereby controlling the vibration motor 202 to perform force feedback on the arm of the operator according to the vibration intensity.
In some embodiments, since a human arm can be modeled as a seven-axis linkage structure, the upper arm and forearm can be considered as a rigid cylinder, with mass centered in the geometric center of the rigid cylinder, regardless of the cartilage tissue at the elbow joint. Therefore, in order to comprehensively and accurately track the seven axial position postures of the human arm, the inertial sensor of the device is arranged at the central positions of the big arm, the small arm and the palm. The device is provided with three inertial sensors 201, namely a first inertial sensor 2011 of a large arm, a second inertial sensor 2012 of a small arm and a third inertial sensor 2013 of a palm.
In some embodiments, the number of vibration motors 202 is 8, the 8 vibration motors being the first vibration motor 2021 of the deltoid toe and the second vibration motor 2022 of the deltoid mid-toe, the third vibration motor 2023 of the biceps longus and the fourth vibration motor 2024 of the biceps brachii, the fifth vibration motor 2025 of the triceps brachii, and the sixth vibration motor 2026 of the flexor muscle group, the seventh vibration motor 2027 of the extensor carpi radialis and the eighth vibration motor 2028 of the extensor muscle group, respectively. Among them, the third vibration motor 2023 provided to the long-head tendon of the biceps brachii is a main motor, and the fourth vibration motor 2024 provided to the short-head tendon of the biceps brachii is a sub-motor. In particular, to ensure that the motor vibration effects between different muscle groups can be differentiated, the spacing between the vibrating motors should be greater than 8cm.
In some embodiments, the apparatus further comprises a power supply unit electrically coupled to the inertial sensor, the vibration motor, and the information processing unit, respectively, to enable device power supply. And the information processing unit is electrically coupled with the inertial sensor and the vibration motor respectively, so that information transmission and acquisition among devices are realized.
In some embodiments, the device further comprises a flexible arm cuff, the inertial sensor, vibration motor, and information processing unit being secured to the flexible arm cuff. Further, the flexible arm sleeve is provided with a silica gel patch on the inner side wall close to the arm of the operator, and the vibration motor is embedded into the silica gel patch, so that the subcutaneous receptors of the arm muscles can be excited better in the mode, and meanwhile, the vibration motor and the skin can be guaranteed to be attached better, wherein the fixed positions of the inertia sensor and the vibration motor on the flexible arm sleeve are required to correspond to the positions of the arm trunk and the arm joints.
When an operator needs to control the slave robot, the operator only needs to wear the flexible arm sleeve on the arm, and adjust the positions of the inertial sensor and the vibration motor to be opposite to the arm trunk and the arm joints. In this way, the flexible arm sleeve can effectively track the movement posture of the human arm, and the action of the human arm is not limited, so that the problems that the force feedback device of the main-end teleoperation device worn by the current operator is inconvenient to move, high in cost and potential safety hazard exist are solved.
In summary, according to the force feedback method and device for a teleoperation system based on vibration induction provided by the invention, the inertial sensor is arranged at the trunk of the human arm, so that the degree of freedom of the human arm is comprehensively tracked by acquiring the arm posture information, the slave end robot is controlled to execute the corresponding target arm posture, and the arm tail end force data received by the slave end robot when the slave end robot interacts with the actual environment is further acquired, so that the stress condition of the slave end robot is determined. Meanwhile, the moment possibly born by the arm joints of the operator under the current stress condition is calculated based on the current arm posture and the arm tail end force data of the operator. Because the movement of the arms of the human body is realized through the activation of muscles, the muscles are contracted to generate force, so that the joints are moved to complete the action, and the activation of the muscles corresponds to a certain nerve activation, a certain numerical relation is established between the nerve activation and the vibration intensity of a motor, and the motor vibrates to give a certain degree of mechanical vibration to the subcutaneous susceptor, so that the feeling of muscle group contraction is simulated, and the action errors of the human are caused, so that good in-situ feeling is generated. In addition, the exoskeleton device is not required to be used for force feedback, so that the cost is effectively reduced, and the weight reduction of a force feedback mode is realized.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent changes made by the specification and drawings of the present invention, or direct or indirect application in the relevant art, are included in the scope of the present invention.

Claims (9)

1. A vibration-induced force feedback method for a teleoperational system, comprising:
Acquiring arm information of an operator, and constructing an arm end model according to the arm information;
Controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture;
determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data;
Determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment;
Determining the vibration intensity of a vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity;
the determining the nerve activation degree of different arm muscles under the target arm gesture according to the virtual joint moment comprises the following steps:
calculating the muscle activation degree of different arm muscles under the target arm gesture according to the virtual joint moment, wherein the muscle activation degree is specifically as follows:
Wherein, Represents the muscle activation, M represents a constant coefficient matrix set according to different muscle sensitivities,Representing a virtual joint moment;
determining the nerve activation degree of the arm muscle according to the muscle activation degree, specifically:
Wherein, The constant is represented by a value that is a function of,Indicating nerve activation activity.
2. The vibration-induced force feedback method for a teleoperational system according to claim 1, wherein the arm information includes a posture angle of each arm joint and a trunk length between adjacent arm joints; the arm tail end model comprises a standard position model and a standard posture model;
The obtaining arm information of the operator and constructing an arm end model according to the arm information comprises:
acquiring the attitude angles and the trunk lengths of the joints of the arms of an operator;
determining a first rotation matrix on each arm joint according to the attitude angle;
And constructing a standard position model and a standard posture model according to the first rotation matrix on each arm joint and the length of the trunk.
3. The vibration-induced force feedback method for a teleoperation system according to claim 2, wherein the constructing a standard position model and a standard posture model according to the rotation matrix on each arm joint and the trunk length is specifically:
Wherein P e represents a standard position model; r e represents a standard posture model; r 1 represents the first rotation matrix of the shoulder arm joint; r 2 represents a first rotation matrix of the elbow arm joint; r 3 represents a first rotation matrix of the wrist-arm joint; l 1 denotes a large arm torso vector, L 1=[0,0,-l1],l1 denotes a large arm torso length; l 2 denotes the forearm torso vector, L 2=[0,0,-l2],l2 denotes the torso length of the forearm; l 3 denotes a palm torso vector, and L 3=[0,0,-l3],l3 denotes a palm torso length.
4. The vibration-induced force feedback method for a teleoperational system of claim 2, wherein the determining the virtual joint moment of the target arm pose from the arm information and the arm tip force data comprises:
determining the rotation angle values of the target arm gesture in seven axial directions according to the arm information;
Determining a second rotation matrix on each arm joint according to the rotation angle value;
constructing a trunk end position model according to the second rotation matrix and the trunk length;
and determining the virtual joint moment of the target arm gesture according to the trunk tail end position model and the arm tail end force data.
5. The vibration-induced force feedback method for a teleoperation system according to claim 4, wherein the determining, according to the arm information, a rotation angle value of the target arm pose in seven axial directions is specifically:
Wherein, AndRespectively representing the attitude angles of the shoulder arm joint in the x-axis direction, the y-axis direction and the z-axis direction; And Representing the rotation angle values in seven axial directions; Element values representing the j-th column of the i-th row in matrix c; r 1 represents the first rotation matrix of the shoulder arm joint; r 2 represents a first rotation matrix of the elbow arm joint; r 3 represents a first rotation matrix of the wrist-arm joint; representing a unit vector in the z-axis direction.
6. The vibration-induced force feedback method for a teleoperation system according to claim 5, wherein the determining the second rotation matrix on each arm joint according to the rotation angle value is specifically:
Wherein, AndRepresenting a second rotation matrix on the shoulder arm joint, the elbow arm joint, and the wrist arm joint, respectively; And A rotation matrix representing an x-axis direction, a y-axis direction, and a z-axis direction;
the constructing a trunk end position model according to the second rotation matrix and the trunk length is specifically:
Wherein O represents a shoulder arm joint, A represents an elbow arm joint, B represents a wrist arm joint, and E represents an arm end; A trunk end position model taking N as a head end M as an end is represented; A homogeneous transformation matrix representing a large arm torso vector; a homogeneous transformation matrix representing a forearm torso vector; a homogeneous transformation matrix representing the palm torso vector.
7. The vibration-induced force feedback method for a teleoperation system according to claim 6, wherein the determining the virtual joint moment of the target arm pose from the torso-end position model and the arm-end force data is specifically:
Wherein, Representing the moment of the virtual joint,The jacobian matrix representing matrix P', T representing the homogeneous transformation matrix, and f representing the arm tip force data.
8. The vibration-induced force feedback method for teleoperation systems according to claim 4, wherein arm muscles provided with the vibration motor include anterior and medial bundles of deltoid muscles, longhead and shorthead tendons of biceps brachii, triceps brachii, and flexor, extensor and extensor muscle groups of the forearm;
The method for determining the vibration intensity of the vibration motor corresponding to the arm muscle according to the nerve activation activity comprises the following steps:
Wherein, AndThe vibration intensities of the vibration motors corresponding to the anterior and middle bundles of the deltoid muscle, the long and short head tendons of the biceps brachii, the flexor muscle group of the brachial triceps, the extensor carpi radialis and extensor muscle group of the forearm,Representing functions for neural networks, k 1、k2、k3、k4、k5、k6 and k 7 representing preset empirical parameters, u 1、u2、u3、u4、u5、u6 and u 7 representing neural activation of arm muscles corresponding to the seven axes, respectively.
9. The force feedback device for the teleoperation system based on vibration induction is characterized by comprising an inertial sensor, a vibration motor and an information processing unit; the vibration motor is fixed at the arm muscle of the operator; the inertial sensor is fixed at the arm trunk of the operator;
The information processing unit is used for acquiring arm information of an operator through the inertial sensor and constructing an arm tail end model according to the arm information;
Controlling a slave end robot to execute a target arm gesture according to the arm end model, and acquiring arm end force data of the slave end robot in the target arm gesture;
determining a virtual joint moment of the target arm posture according to the arm information and the arm tail end force data;
determining nerve activation activities of different arm muscles under the target arm gesture according to the virtual joint moment; and
Determining the vibration intensity of the vibration motor corresponding to the arm muscle according to the nerve activation activity, and controlling the vibration motor to perform force feedback on the arm of the operator according to the vibration intensity;
the determining the nerve activation degree of different arm muscles under the target arm gesture according to the virtual joint moment comprises the following steps:
calculating the muscle activation degree of different arm muscles under the target arm gesture according to the virtual joint moment, wherein the muscle activation degree is specifically as follows:
Wherein, Represents the muscle activation, M represents a constant coefficient matrix set according to different muscle sensitivities,Representing a virtual joint moment;
determining the nerve activation degree of the arm muscle according to the muscle activation degree, specifically:
Wherein, The constant is represented by a value that is a function of,Indicating nerve activation activity.
CN202410471429.2A 2024-04-19 Force feedback method and device for teleoperation system based on vibration induction Active CN118061200B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410471429.2A CN118061200B (en) 2024-04-19 Force feedback method and device for teleoperation system based on vibration induction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410471429.2A CN118061200B (en) 2024-04-19 Force feedback method and device for teleoperation system based on vibration induction

Publications (2)

Publication Number Publication Date
CN118061200A CN118061200A (en) 2024-05-24
CN118061200B true CN118061200B (en) 2024-07-12

Family

ID=

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101612043A (en) * 2009-08-04 2009-12-30 中国科学院合肥物质科学研究院 The wearable type human arm tremor detects and suppresses robot and method for suppressing tremor thereof
CN104932691A (en) * 2015-06-19 2015-09-23 中国航天员科研训练中心 Real-time gesture interaction system with tactile perception feedback

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101612043A (en) * 2009-08-04 2009-12-30 中国科学院合肥物质科学研究院 The wearable type human arm tremor detects and suppresses robot and method for suppressing tremor thereof
CN104932691A (en) * 2015-06-19 2015-09-23 中国航天员科研训练中心 Real-time gesture interaction system with tactile perception feedback

Similar Documents

Publication Publication Date Title
Ben-Tzvi et al. Sensing and force-feedback exoskeleton (SAFE) robotic glove
Hu et al. Development of sensory-motor fusion-based manipulation and grasping control for a robotic hand-eye system
Zhou et al. RML glove—An exoskeleton glove mechanism with haptics feedback
Burdea et al. Dextrous Telerobotics with force feedback–an overview. Part 1: Human factors
Mick et al. Reachy, a 3D-printed human-like robotic arm as a testbed for human-robot control strategies
Liu et al. A glove-based system for studying hand-object manipulation via joint pose and force sensing
Park et al. A dual-cable hand exoskeleton system for virtual reality
CN111645093B (en) Force sense feedback data glove for teleoperation
Fong et al. EMU: A transparent 3D robotic manipulandum for upper-limb rehabilitation
Naidu et al. A 7 DOF exoskeleton arm: Shoulder, elbow, wrist and hand mechanism for assistance to upper limb disabled individuals
Babu et al. Holding, grasping and sensing of prosthetic robot arm like a real human hand, a journey beyond limits: an extensive review
Odesanmi et al. Skill learning framework for human–robot interaction and manipulation tasks
JP2002182817A (en) Inner force representing device
WO2001018617A1 (en) Remote mechanical mirroring using controlled stiffness and actuators (memica)
Xie et al. Human-exoskeleton coupling dynamics of a multi-mode therapeutic exoskeleton for upper limb rehabilitation training
Zhu et al. Underactuated rehabilitation robotics for hand function
CN118061200B (en) Force feedback method and device for teleoperation system based on vibration induction
CN118061200A (en) Force feedback method and device for teleoperation system based on vibration induction
Pastor et al. A real-time motion tracking wireless system for upper limb exosuit based on inertial measurement units and flex sensors
Marcus et al. EXOS research on master controllers for robotic devices
Shanmugam et al. A Comprehensive Review of Haptic Gloves: Advances, Challenges, and Future Directions
Yan et al. Development of an assist upper limb exoskeleton for manual handling task
Esmatloo Fingertip position and force control for dexterous manipulation through accurate modeling of hand-exoskeleton-environment
Li et al. A novel EMG-based variable impedance control method for a tele-operation system under an unstructured environment
Mabrouk et al. Development of a Robotic Framework for Modeling and Simulation of an Upper Humanoid Robot

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant