CN110584790B - Arm stiffness-based teleoperation proportion control method for surgical robot - Google Patents
Arm stiffness-based teleoperation proportion control method for surgical robot Download PDFInfo
- Publication number
- CN110584790B CN110584790B CN201910743884.2A CN201910743884A CN110584790B CN 110584790 B CN110584790 B CN 110584790B CN 201910743884 A CN201910743884 A CN 201910743884A CN 110584790 B CN110584790 B CN 110584790B
- Authority
- CN
- China
- Prior art keywords
- arm
- joint
- stiffness
- operator
- rotation angular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B34/35—Surgical robots for telesurgery
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
- A61B34/74—Manipulators with manual electric input means
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Robotics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a teleoperation proportion control method of a surgical robot based on arm stiffness, which collects the same type of operation cases before the operation is performed, and further obtains the value range of the change level of the arm stiffness of the operation; when the operation is performed, the relation between the arm rigidity along with the joint rotation angular velocity and the change level of the joint rotation angular acceleration at the wrist and the joint rotation angular velocity and the joint rotation angular acceleration is calculated according to the relevant information of the rotation and the movement of the arm joint of the operator acquired in the operation process; and then obtaining the relation between the change level of the arm stiffness along with the curvature of the arm movement track and the change rate of the curvature, on the basis, establishing a model relation between the change level of the arm stiffness along with the angular velocity, the angular acceleration and the change rate of the movement track curvature, and normalizing the model relation by utilizing the value range of the change level of the arm stiffness to obtain the teleoperation proportion.
Description
Technical Field
The invention relates to the field of teleoperation of robots, in particular to a teleoperation proportion control method of a surgical robot based on arm rigidity.
Background
Teleoperation technology for surgical robots, in which an operator directly operates a main robot to perform a task from a robot side during teleoperation, is the basis for ensuring system stability and task performance.
When performing delicate or complex surgical teleoperations, the operator needs to maintain precise control and can automatically adjust the scale through the adaptive motion scaling frame to reach the remote target. Motion scaling is an important component of the master-slave paradigm. To accomplish a fine task, the hand movements of the master robot are reduced and replicated by the slave robots. Kinematic scaling enables the operator to perform precise operations while teleoperating. However, a small motion zoom prevents the operator from reaching distant targets without a tight grip, which is time consuming.
In an Adaptive Motion Scaling mechanism for seamless operation in the prior art (IEEE paper "a Self-Adaptive Motion Scaling frame for Surgical Robot Remote Control" in 2019), when performing fine or complicated operation, an operator can maintain precise Control while accelerating the Motion to a Remote target through Adaptive Motion Scaling; the frame is made up of three parts: 1) situational awareness, 2) skill level awareness, 3) task awareness. The scaling is adjusted according to the real-time monitoring of each part. The parameters of the motion scaling mechanism, fitting with a fuzzy bayesian network, the subject needs to perform the same task in the experiment, which is obviously impractical for clinical applications, and the framework does not fully consider the dynamic characteristics of the human arm.
In the prior art, adaptive motion scaling is realized based on an eyeball gaze-aided intention identification scheme (IEEE paper "imposed size-assisted adaptive motion scaling for high-definition instrument manipulation" in 2017, and Implicit gaze-aided adaptive motion scaling). By eye tracking and in conjunction with other sensors, the position of the target that the operator is attempting to reach is inferred, and the motion scale is changed accordingly. This information is then used to adjust the movement ratio so that the operator can reach the target more quickly. However, adaptive motion targeting for eye tracking is still difficult to fully integrate into a general surgical system, the effectiveness of eye gaze assisted intent recognition is not high when the operator wears glasses, and intraoperative doctors may not be able to view the entire surgical site condition. Therefore, to construct an adaptive framework, the kinematic data needs to be combined with the visual data, resulting in a more general solution. Whereas in existing studies these parameters can be automatically optimized in a user-specific manner using bayesian theorem, the subject is required to perform the same task in multiple experiments to evaluate the robot's behavior and select preferred parameters, which is not clinically realistic.
Disclosure of Invention
The invention provides a method for controlling the teleoperation proportion of a surgical robot based on arm stiffness, which is used for adjusting the zoom proportion of the teleoperation of the surgical robot in real time by monitoring the curvature of an expected track, the execution area of the track and the real-time change condition of hand muscles when an operator carries out teleoperation in real time.
In order to realize the task, the invention adopts the following technical scheme:
the teleoperation proportion control method of the surgical robot based on the arm rigidity comprises the following steps:
before the operation, the same type of operation cases are collected, and the relevant information of joint rotation and movement when the arm of an operator performs the operation is collected, so that the value range of the rigidity change level of the arm of the operation is obtained;
when the operation is performed, the relation between the arm rigidity along with the joint rotation angular velocity and the change level of the joint rotation angular acceleration at the wrist and the joint rotation angular velocity and the joint rotation angular acceleration is calculated according to the relevant information of the rotation and the movement of the arm joint of the operator acquired in the operation process; and then obtaining the relation between the change level of the arm stiffness along with the curvature of the arm movement track and the change rate of the curvature, on the basis, establishing a model relation between the change level of the arm stiffness along with the angular velocity, the angular acceleration and the change rate of the movement track curvature, and normalizing the model relation by utilizing the value range of the change level of the arm stiffness to obtain the teleoperation proportion.
Further, the method adopts equipment comprising:
the device 1: a serial robot having a plurality of directional degrees of freedom, the serial robot being an actuator for an operator who performs a surgical operation by controlling an end effector of the serial robot;
the device 2: a display provided in front of an operator for displaying an operation site of the operator;
the device 3: one or more cameras disposed above the display for capturing visual image information to obtain the chest position of the operator;
the device 4: one or more optical reflection markers which are attached to the right shoulder seam and the left shoulder crest of the operator and are used for obtaining the motion condition of the joint of the operator are matched with the equipment 3, and the motion condition of the joint of the operator is obtained by utilizing the position change of the reflection markers in the camera image;
the device 5: an electromagnetic tracker for measuring the position and movement of the arm;
the device 6 is as follows: one or more electromagnetic sensors for recording the movements of the wrist, applied to the operator's wrist, for calibrating the device 5;
the device 7: the bipolar electrode is used for acquiring an electromyographic signal of an arm of an operator.
Further, the device 7 employs three pairs of electrodes for obtaining electromyographic signals of the anterior, lateral and posterior deltoid bands, respectively, two pairs of electrodes for obtaining electrical signals of the long and lateral brachiocephalic muscles, respectively, one pair of electrodes for obtaining electromyographic signals of the biceps, and four pairs of electrodes for obtaining activation of the brachioradialis, flexor ulnar carpus, radial nerve and extensor tendon of the forearm, and a monopolar electrode for reference is connected to the arm operated by the surgeon.
Further, the collection operator arm joint rotation and motion's relevant information when carrying out the operation includes:
the information acquisition process comprises the following steps:
acquiring joint rotation angular velocities of joints of an operator by using the cooperation of the optical reflection mark and the camera;
and (3) acquiring the displacement and joint linear velocity of each joint of the operator by using an electromagnetic tracker, and calibrating by using an electromagnetic sensor.
Further, the collection operator arm joint rotation and motion relevant information when carrying out the operation still include:
the data preprocessing process comprises the following steps:
for the acquired joint rotation angular velocity, joint displacement and joint linear velocity, filtering data by adopting a twice second-order Butterworth filter, and reversing time in the second filtering to eliminate nonlinear phase shift;
and performing numerical differentiation on the joint linear velocity and the joint rotation angular velocity of the joint in different directions, and filtering to obtain the corresponding joint linear acceleration and joint rotation angular acceleration.
Further, the obtaining of the value range of the stiffness change level of the surgical arm includes:
creating a space weight matrix by using the rotation angular velocity, the displacement and the joint linear velocity of the joint, the joint linear acceleration and the joint rotation angular acceleration, and establishing a joint torque dynamic equation on the arm of the operator, thereby calculating a joint torque vector;
collecting electromyographic signals generated when the muscle of an arm of an operator moves in the process of performing an operation, obtaining electromyography, inputting the electromyography into a CEINMS modeling toolbox, and taking the joint torque vector as a calibration value to obtain an accurate estimation value of the joint torque vector;
obtaining a rigidity matrix of the joint by using an accurate estimation value of a joint torque vector and a joint displacement vector; calculating the Cartesian stiffness of the arm using the Jacobian matrix of the arm movements and the stiffness matrix of the joints; then, obtaining a left singular vector and a non-zero singular value through singular value decomposition of the Cartesian stiffness, and obtaining the arm stiffness;
the minimum value and the maximum value of the arm rigidity values calculated by a plurality of preoperative cases are respectively taken as a and b, and the change level value range [ a, b ] of the arm rigidity is obtained.
Further, the relationship between the variation level of the arm stiffness along with the joint rotation angular velocity and the joint rotation angular acceleration at the wrist and the joint rotation angular velocity and the joint rotation angular acceleration is calculated by the following formula:
wherein w (i) is a joint rotation angular velocity,
the angular acceleration of the joint rotation is taken as,
the level of arm rigidity variation with the joint rotation angular velocity and joint rotation angular acceleration at the wrist, alphaa,αbIs the set scaling.
Further, the relationship between the change level of the arm stiffness along with the curvature of the arm movement locus and the change rate of the curvature is obtained and specifically expressed as:
obtaining the joint linear velocity v (i) and the joint linear acceleration of the wrist joint of the operator at the ith moment
The curvature C (i) of the motion track of the arm, the linear velocity vector v (i) and the linear acceleration vector
Is obtained by the following formula:
then, carrying out numerical differentiation on the curvature C (i) on a time axis, and filtering by using a second-order filter to obtain the change rate Delta C (i) of the curvature; obtaining the variation level of the arm rigidity along with the curvature of the arm movement track through the following formula
Relationship to rate of change of curvature Δ c (i):
where β is the set scaling.
Further, the establishing of the model relationship of the change level of the arm stiffness along with the angular velocity, the angular acceleration and the change rate of the motion trajectory curvature is represented as:
further, the value range of the arm stiffness change level is used for normalizing the model relation to obtain a teleoperation proportion, and the formula is as follows:
the invention has the following technical characteristics:
1. the invention provides a new method for adaptively adjusting the teleoperation scale frame of the surgical robot, which can flexibly control the motion scaling of the surgical robot system.
2. The real-time change of the arm stiffness, which is a concern of the invention, can more accurately predict the arm grip condition and the intended arrival position of the operator from biology and anthropology (the arm grip condition of the operator can be predicted through the steps 3.1 and 3.2, and the distance of the intended arrival position of the author can be judged according to the arm grip of the operator), and the whole implementation process is simpler.
Drawings
FIG. 1 is a schematic configuration of an apparatus used in the method of the present invention;
FIG. 2 is a schematic view of a bipolar electrode configuration;
fig. 3 is a schematic view of an operator controlling an end effector of a tandem robot to perform a surgical operation.
Detailed Description
In human motion control, stability of the performance of tasks can be achieved by activating muscles, modulating the dynamics of the limb. By adjusting muscle contraction and changing the angle and position of the joints, the impedance of the arm and its components (viscosity, inertia and stiffness) can be varied to accommodate different tasks and required interactions. As the most important components, the stiffness of the arm also depends directly on the trajectory profile of the execution, the joint angular velocity, the angular acceleration, the reflection modulation and the expected disturbances present. The arm stiffness and the curvature of the motion trail form a trend of obvious positive correlation, and the arm stiffness and the joint angular velocity and the angular acceleration form a trend of negative correlation.
The electromagnetic tracking system and the optical tracking system can be used for collecting arm movement data of a user. The position and change data may be obtained by an optical tracking system including a reference mark portion, a position measurement portion, a tracking sensor portion, and a processing portion. The method mainly utilizes the position change of a light emitting source relative to a light receiving source in the moving process to generate a corresponding electric signal. Through the electromagnetic tracking system, the motion data and muscle activity state of the arm of the patient can be obtained, and the magnetic flux change generated by the muscle in motion is converted into an electric signal to be output.
The arm motion data and muscle activation conditions of the user are obtained, the rigidity of the joint can be estimated by combining a specific muscle skeleton model of the user, and the rigidity of the arm endpoint can be deduced through Jacobian matrix transformation of the arm.
According to the invention, through the existing research results, the stability of executing tasks can be realized by activating muscles and regulating the dynamic characteristics of limbs in the human motion control. By adjusting muscle contraction and changing the angle and position of the joints, the arm impedance and its components (viscosity, inertia and stiffness) can be changed to accommodate different tasks and required interactions, the most important of which is the arm stiffness. A new adaptive teleoperation scale framework (specifically, formula 8) is proposed. Different from the current self-adaptive motion scaling mechanism for seamless operation in the prior art, aiming at the current complexity and difficulty of realization of the frame, under the condition of only considering the dynamic characteristics of human arms, the distance of the real-time intention position of an operator can be obtained by obtaining the real-time rigidity change of the arms of the operator, so that the motion scaling is correspondingly changed, and the new self-adaptive teleoperation scaling frame has higher performability and convenience.
The invention relates to a teleoperation proportion control method of a surgical robot based on arm stiffness, and equipment used in the method is shown in figure 1 and comprises the following steps:
the device 1: a tandem robot having multiple degrees of freedom in directions, as shown in fig. 1; in this embodiment, the tandem robot is a Phantom Omni (3D robot system) having 6DOF (degrees of freedom), and the present embodiment is an operator performing a surgical operation by controlling an end effector (stylus) of the tandem robot, as shown in fig. 3.
The device 2: a 32-inch 2D display provided in front of the operator for displaying an operation site of the operator;
the device 3: one or more cameras arranged above the display for capturing visual image information and obtaining the position of the chest of the operator, cooperating with the device 4 to obtain information on the rotation speed, rotation angular speed, etc. of the various joints of the operator;
the device 4: one or more optical reflection marks which are attached to the right shoulder seam and the left shoulder crest of the operator and used for obtaining the motion condition of the joint of the operator, wherein the optical reflection marks are marked objects which are outstanding in color and can reflect light rays, and are matched with the equipment 3 for use, and the motion condition of the joint of the operator is obtained by utilizing the position change of the reflection marks in the camera image;
the device 5: an electromagnetic tracker for measuring the position and movement of the arm, suspended above the wrist of the operator, comprising a camera and a data processing system; the position and the motion condition of the arm acquired by the camera are processed by the data processing system and then converted into electric signals;
the device 6 is as follows: one or more 6-DOF electromagnetic sensors for recording wrist movements, affixed to an operator's wrist; for calibrating the device 5;
the device 7: bipolar electrodes for obtaining electromyographic signals, configured as shown in fig. 2; for example, electromyographic signals of anterior, lateral and posterior deltoid bands acquired using three pairs of electrodes, respectively (1-3 in fig. 2), electrical signals of long and lateral brachiocephalic muscles acquired using two pairs of electrodes, respectively (4-5 in fig. 2), electromyographic signals of biceps acquired using one pair of electrodes (6 in fig. 2), activation conditions of brachioradialis, flexor carpi ulnaris, radial nerve and extensor tendon acquired using four pairs of electrodes of the forearm (7-10 in fig. 2), and a monopolar electrode used as a reference is connected to the arm operated by the surgeon.
The device 8: one end effector, i.e. a slave manipulator robot, located near the site of operation of the patient, is the one that is in direct contact with the wound of the patient; the operation robot receives the remote control command of the serial robot, executes the relevant operation,
at the same time, a virtual reference coordinate system and a shoulder reference coordinate system on the display are established.
The configuration of these devices with the surgeon (operator) may be:
the tandem robot Phantom Omni may be located near the surgeon's wrist, in a position suitable for the operator to perform the surgical operation, depending on the area in which the patient is undergoing the surgical operation. The camera that obtains the chest position may be separate from the tandem robot Phantom Omni, positioned in its vicinity, allowing the camera to have a direct line of sight to the surgical field.
Meanwhile, for the acquisition of arm electromyography, fig. 2 shows the possible placement of human body surface electromyography electrodes to acquire the muscle movement of human body arms.
When the surgeon starts the surgical operation, the 2D display of the device 2 displays the operating site of the operator, and the surgeon starts the surgical operation by operating the end effector (stylus) of the tandem robot of the device 1, which sends a corresponding operation command to the end effector (device 8) to perform the specific surgical operation by the end effector. In the process, the optical reflection mark (equipment 4) changes along with the image of the movement of the shoulder of the surgeon in the camera (equipment 3), so that the movement condition of the shoulder of the surgeon is obtained, and then the movement condition (rotation speed, rotation angular speed and the like) of each joint and elbow of the surgeon can be obtained; the electromagnetic tracker (equipment 5) is used for acquiring the position and motion condition of an arm when a surgeon performs surgical operation, and meanwhile, the electromagnetic sensor (equipment 6) is used as calibration equipment of the equipment 5 to ensure that the conversion of a camera coordinate system is consistent with that of a wrist coordinate system; bipolar electrodes (device 7, configured as shown in fig. 2) acquire electromyographic signals to acquire muscle movements of the arm.
The method mainly comprises two parts:
the first part, determining the range of values of the variation level of the arm stiffness during the operation
The collection of case before carrying out the art, when the operator carries out the operation of the same type, after disposing equipment 1 to equipment 8 according to corresponding position (promptly according to the aforesaid mounting position of every equipment disposes), gather the relevant information of joint rotation and motion when operator's arm carries out the operation, and then obtain the value range [ a, b ] of this type of operation arm rigidity change level, concrete step includes:
step 1.1, information acquisition
Joint rotation angular velocities of respective joints (six-degree-of-freedom joints of an arm) of an operator are acquired by cooperation of the optical reflection mark (device 4) and the camera (device 3).
The displacement and joint linear velocity of each joint of the operator are acquired by an electromagnetic tracker (device 5) and calibrated by an electromagnetic sensor (device 6).
The data acquisition frequency of this step is 1kHz, i.e. data is acquired and calculated every 1 ms.
Step 1.2, data preprocessing procedure
Step 1.2.1, for the data obtained in step 1.1, because of circuit transmission interference and acquisition errors, a second-order Butterworth filter is adopted to filter the data, and time is reversed in the second filtering to eliminate nonlinear phase shift; the filter cut-off frequency is set to be lower frequency (4 Hz (-6dB) in the embodiment), and the filtered data is used as the real rotation angular velocity of each joint of the operator in the reference coordinate system of the shoulder of the operator and the displacement and the linear velocity of each joint of the operator.
And step 1.2.2, performing numerical differentiation on each coordinate axis (XYZ axis) of the shoulder reference coordinate system according to the joint linear velocity and the joint rotational angular velocity of each joint obtained in the step 1.1, filtering by using a second-order filter, wherein the frequency is lower (in the exemplary embodiment, 4Hz), and obtaining the corresponding joint linear acceleration and joint rotational angular acceleration.
Step 1.3, calculating the joint torque
Creating a Spatial Weight Matrix (SWM) by using a generation spatial weight matrix tool (which can adopt a spatial statistics tool kit of ArcGIS Pro company) for the rotational angular velocity, displacement and joint linear velocity of each joint processed in the step 1.2.1 and the joint linear acceleration and joint rotational angular acceleration obtained in the step 1.2.2, and obtaining a weight matrix M;
because the arm of the operator is not subjected to external force when performing the operation, a dynamic equation of the joint torque on the arm of the operator is established, as shown in a formula 1; where τ is the joint torque vector (6 x 1), C is the vector of each joint Coriolis force and centrifugal force, G is the gravity vector, and q,
are vectors, joint position, velocity and acceleration, respectively.
Using a muscle-nerve-skeleton (CEINMS) modeling tool kit (which is used as a standalone product and can be used in OpenSim GUI, the source code of the software can be found in gitubb), collecting an electromyographic signal generated when an operator moves the muscle of the arm during an operation by using the device 7, obtaining an electromyogram and inputting the electromyogram into the tool kit, and simultaneously using the value of the joint torque vector τ calculated by equation 1 as a calibration value, thereby obtaining an accurate estimation value of the joint torque vector τ.
For each joint (six-degree-of-freedom joint of the surgeon's manipulator), the stiffness matrix K of the joint is calculated using equation 2 belowj(i)：
Wherein q (i) and tau (i) are accurate estimated values of the joint displacement vector and the joint torque vector at the ith moment respectively.
Step 1.4, calculating arm stiffness
Using a Jacobian matrix J of arm movements (substituting the rotation speed and rotation angular velocity of the joint of the arm operated by the operator into the rotation speedThe Jacobian matrix formula of the series robot is obtained, J) is obtained and substituted into the formula 3 to obtain the Cartesian rigidity K of the arme(i)：
Ke(i)＝(JT)-1(i)Kj(i)J-1(i) Formula 3
In the above formula, j (i) represents the jacobian matrix at the i-th time; and then decomposing the singular value of the Cartesian stiffness to obtain a left singular vector and a non-zero singular value, thereby obtaining the arm stiffness Ke (i).
Through the steps 1.3 and 1.4, the arm gripping force of the operator can be predicted to judge the distance of the position which the author intends to reach.
Step 1.5, calculating the value range of the change level of the arm rigidity
And processing the preoperative cases according to the steps 1.1 to 1.4 to obtain the value ranges [ a, b ] of the arm stiffness change level of the operator in the current surgical operation.
For example, for a plurality of preoperative cases, of the calculated values of all the arm stiffness ke (i) (the data acquisition frequency is 1kHz, the value acquired in each case is a series of values), the minimum value and the maximum value are respectively taken as a and b, and thus, the value range [ a, b ] of the change level of the arm stiffness for a certain type of surgical operation is objectively obtained.
Second part, calculating the teleoperation scaling
Step 2.1, after the devices 1 to 8 are configured according to corresponding positions, when an operator performs operation, information acquisition and data preprocessing are performed according to the same method as that of the first part of the steps 1.1 and 1.2;
step 2.2, calculating the variation level of the arm rigidity along with the joint rotation angular velocity and the joint rotation angular acceleration of the wrist
Relationship with joint rotation angular velocity and joint rotation angular acceleration:
obtaining the joint rotation angular velocity w (i) and the joint rotation angular acceleration of the wrist joint of the operator at the i-th time
Then, since the arm stiffness and the joint rotation angular velocity and the joint rotation angular acceleration have negative correlation trends, the variation level of the arm stiffness along with the joint rotation angular velocity and the joint rotation angular acceleration of the wrist is obtained by equation 4
With joint rotation angular velocity w (i) and joint rotation angular acceleration
The relationship of (1):
in the above formula, i represents the i-th time, αa,αbFor the set scaling (0.5 for both embodiments).
Step 2.3, calculating the variation level of the arm rigidity along with the curvature of the arm movement track
Relationship to rate of change of curvature Δ c (i):
obtaining the joint linear velocity v (i) and the joint linear acceleration of the wrist joint of the operator at the ith moment
The curvature C (i) of the motion track of the arm, the linear velocity vector v (i) and the linear acceleration vector
The relationship of (a) is obtained by equation 5; then, carrying out numerical differentiation on the curvature C (i) on a time axis, filtering by using a second-order filter, wherein the frequency is lower (4 Hz in the exemplary embodiment), and obtaining the change rate Delta C (i) of the curvature;
because the rigidity of the arm and the curvature of the motion trail are in a trend of obvious positive correlation, the rigidity of the arm and the curvature of the motion trail are in a positive correlationObtaining the variation level of the arm stiffness along with the curvature of the arm motion track through the formula 6
Relationship to rate of change of curvature Δ c (i):
where i is the ith time and β is the set scaling (in the embodiment, β is 0.743).
Step 2.4, establishing the change level delta Ke (i) of the arm rigidity along with the angular velocity w (i) and the angular acceleration of the wrist joint
A model relationship of the curvature change rate Δ c (i) of the motion trajectory, the model being given by equation 7:
step 2.5, calculating the teleoperation scaling alpha
Normalizing delta Ke (i) by using the value range [ a, b ] of the change level of the arm stiffness obtained in the step 1.5, so as to obtain a teleoperation scaling alpha as shown in a formula 8, wherein alpha is more than 0 and less than or equal to 1; this equation 8 is the adaptive teleoperation scaling framework proposed in this scheme.
Before a doctor starts to perform a certain operation, a real preoperative case (for safety, the preoperative case can also be a corresponding operation area for simulating and operating a medical dummy) needs to be collected, the rotation and movement data of the joint of an operator are obtained in real time, and the value range [ a, b ] of the arm stiffness change level of the doctor during the operation is obtained through the first part. In the process of collecting real preoperative cases, a display (equipment 2) displays the operation part of a doctor, the doctor starts to operate a stylus end effector of a serial robot (equipment 1) to perform surgical operation, at the moment, the serial robot sends a corresponding operation instruction to equipment 8 directly, and the equipment 8 executes corresponding operation; and (3) collecting information by using the collecting step 1.1, and after the operation is finished, performing data statistics on the obtained data through the steps 1.2 to 1.5 to obtain the value ranges [ a, b ] of the stiffness change level of the arm.
According to the same equipment configuration, in the real operation process, the display (equipment 2) displays the operation part of a doctor, the doctor starts to operate a stylus end effector of the serial robot (equipment 1) to perform the operation of the surgery, information acquisition and preprocessing are performed according to the same method as in the steps 1.1 and 1.2, the serial robot sends corresponding operation instructions to a data buffer area between the equipment 1 and the equipment 8 to perform processing (a processing controller, the conversion of the control instructions is the same as a proportional controller), and the real-time calculated scaling ratio alpha is obtained through the steps 2.2 to 2.5 to serve as the scaling ratio of the control instructions, is multiplied and calculated to serve as a real control instruction and is transmitted to the end effector of the equipment 8, and the end effector executes the specific operation.
Claims (10)
1. The teleoperation proportion control method of the surgical robot based on the arm rigidity is characterized by comprising the following steps of:
collecting the same type of operation cases, and collecting the relevant information of joint rotation and movement when the arm of an operator performs operation, thereby obtaining the value range of the rigidity change level of the operation arm;
calculating the relationship between the arm stiffness along with the joint rotation angular velocity and the change level of the joint rotation angular acceleration at the wrist and the joint rotation angular velocity and the joint rotation angular acceleration according to the relevant information of the rotation and the movement of the arm joint of the operator acquired in the operation process; and then obtaining the relation between the change level of the arm stiffness along with the curvature of the arm movement track and the change rate of the curvature, on the basis, establishing a model relation between the change level of the arm stiffness along with the angular velocity, the angular acceleration and the change rate of the movement track curvature, and normalizing the model relation by utilizing the value range of the change level of the arm stiffness to obtain the teleoperation proportion.
2. The arm stiffness-based teleoperation proportional control method for a surgical robot as claimed in claim 1, wherein the method adopts equipment comprising:
the device 1: a serial robot having a plurality of directional degrees of freedom, the serial robot being an actuator for an operator who performs a surgical operation by controlling an end effector of the serial robot;
the device 2: a display provided in front of an operator for displaying an operation site of the operator;
the device 3: one or more cameras disposed above the display for capturing visual image information to obtain the chest position of the operator;
the device 4: one or more optical reflection markers which are attached to the right shoulder seam and the left shoulder crest of the operator and are used for obtaining the motion condition of the joint of the operator are matched with the equipment 3, and the motion condition of the joint of the operator is obtained by utilizing the position change of the reflection markers in the camera image;
the device 5: an electromagnetic tracker for measuring the position and movement of the arm;
the device 6: one or more electromagnetic sensors for recording the movements of the wrist, applied to the operator's wrist, for calibrating the device 5;
the device 7: the bipolar electrode is used for acquiring an electromyographic signal of an arm of an operator.
3. The arm stiffness based surgical robot teleoperational proportion control method as claimed in claim 2, wherein the device 7 employs three pairs of electrodes to respectively acquire electromyographic signals of anterior, lateral and posterior deltoid bands, two pairs of electrodes to respectively acquire electrical signals of long and lateral brachiocephalic triceps, one pair of electrodes to acquire electromyographic signals of biceps, four pairs of electrodes to acquire activation of brachioradialis, flexor ulnaris, radial nerve and extensor tendon of the forearm, and a single electrode for reference is connected to the arm operated by the surgeon.
4. The arm stiffness-based proportional control method for teleoperation of a surgical robot according to claim 1, wherein the collecting information related to joint rotation and movement of the arm of the operator during the surgical operation comprises:
the information acquisition process comprises the following steps:
acquiring joint rotation angular velocities of joints of an operator by using the cooperation of the optical reflection mark and the camera;
and acquiring the displacement and joint linear velocity of each joint of the operator by using an electromagnetic tracker, and calibrating by using an electromagnetic sensor.
5. The arm stiffness-based proportional control method for teleoperation of a surgical robot as claimed in claim 4, wherein the collecting information related to joint rotation and movement of the arm of the operator during the surgical operation further comprises:
a data preprocessing process:
for the acquired joint rotation angular velocity, joint displacement and joint linear velocity, filtering data by adopting a twice second-order Butterworth filter, and reversing time in the second filtering to eliminate nonlinear phase shift;
and performing numerical differentiation on the joint linear velocity and the joint rotation angular velocity of the joint in different directions, and filtering to obtain the corresponding joint linear acceleration and joint rotation angular acceleration.
6. The arm stiffness-based teleoperation ratio control method for a surgical robot according to claim 1, wherein the further obtaining of the value range of the stiffness variation level of the surgical arm comprises:
creating a space weight matrix by using the rotation angular velocity, the displacement and the joint linear velocity of the joint, the joint linear acceleration and the joint rotation angular acceleration, and establishing a joint torque dynamic equation on the arm of the operator, thereby calculating a joint torque vector;
collecting electromyographic signals generated when the muscle of an arm of an operator moves in the process of performing an operation, obtaining electromyography, inputting the electromyography into a CEINMS modeling toolbox, and taking the joint torque vector as a calibration value to obtain an accurate estimation value of the joint torque vector;
obtaining a rigidity matrix of the joint by using an accurate estimation value of a joint torque vector and a joint displacement vector; calculating the Cartesian stiffness of the arm using the Jacobian matrix of the arm movements and the stiffness matrix of the joints; then, decomposing singular values of the Cartesian stiffness to obtain a left singular vector and a non-zero singular value, and obtaining the arm stiffness;
the minimum value and the maximum value of the arm rigidity values calculated by a plurality of preoperative cases are respectively taken as a and b, and the change level value range [ a, b ] of the arm rigidity is obtained.
7. The arm stiffness-based surgical robot teleoperation proportional control method according to claim 1, wherein the arm stiffness is calculated according to the relationship between the variation level of the arm stiffness with the joint rotation angular velocity and the joint rotation angular acceleration at the wrist and the joint rotation angular velocity and the joint rotation angular acceleration by the following formula:
wherein w (i) is a joint rotation angular velocity,
the angular acceleration of the joint rotation is taken as,
the level of arm rigidity variation with the joint rotation angular velocity and joint rotation angular acceleration at the wrist, alphaa,αbIs the set scaling.
8. The arm stiffness-based proportional control method for teleoperation of a surgical robot according to claim 1, wherein the obtained relationship between the level of change of the arm stiffness along with the curvature of the motion trajectory of the arm and the rate of change of the curvature is specifically represented as:
obtaining the joint linear velocity v (i) and the joint linear acceleration of the wrist joint of the operator at the ith moment
The curvature C (i) of the motion track of the arm, the linear velocity v (i) and the linear acceleration
Is obtained by the following formula:
then, carrying out numerical differentiation on the curvature C (i) on a time axis, and filtering by using a second-order filter to obtain the change rate Delta C (i) of the curvature; obtaining the variation level of the arm rigidity along with the curvature of the arm movement track through the following formula
Relationship to the rate of change of curvature Δ c (i):
where β is the set scaling.
9. The arm stiffness-based proportional control method for teleoperation of a surgical robot according to claim 1, wherein the model relationship Δ ke (i) between the variation level of arm stiffness and the variation rate of angular velocity, angular acceleration and curvature of motion trajectory of the wrist joint is established as follows:
wherein the content of the first and second substances,
the change level of the arm rigidity along with the curvature of the arm movement track,
the level of variation of arm stiffness with the joint rotation angular velocity and joint rotation angular acceleration of the wrist, alphaa,αbW (i) is a joint rotation angular velocity of the wrist joint of the operator at the i-th time, Δ c (i) is a change rate of curvature, and β is a set scaling.
10. The arm stiffness-based teleoperation ratio control method for a surgical robot according to claim 1, wherein the teleoperation ratio is obtained by normalizing the model relationship by using the value range of the arm stiffness variation level, and the formula is as follows:
wherein, Δ ke (i) is a model relation of the change level of the arm stiffness along with the angular velocity, the angular acceleration and the change rate of the motion track curvature of the wrist joint, and the value range of the change level of the arm stiffness is [ a, b ].
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910743884.2A CN110584790B (en) | 2019-08-13 | 2019-08-13 | Arm stiffness-based teleoperation proportion control method for surgical robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910743884.2A CN110584790B (en) | 2019-08-13 | 2019-08-13 | Arm stiffness-based teleoperation proportion control method for surgical robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110584790A CN110584790A (en) | 2019-12-20 |
| CN110584790B true CN110584790B (en) | 2022-05-06 |
Family
ID=68854025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910743884.2A Active CN110584790B (en) | 2019-08-13 | 2019-08-13 | Arm stiffness-based teleoperation proportion control method for surgical robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110584790B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114378819B (en) * | 2022-01-18 | 2022-07-26 | 上海健康医学院 | Master-slave hand control method and device for digestive endoscopy minimally invasive surgery robot |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5623759B2 (en) * | 2010-02-26 | 2014-11-12 | 公益財団法人東京都医学総合研究所 | Identification and evaluation of parallel motor control function in brain based on electromyogram signal |
| CN106725856B (en) * | 2016-11-23 | 2020-05-05 | 深圳市罗伯医疗科技有限公司 | Control method and control device of surgical robot |
| CN106695794A (en) * | 2017-02-20 | 2017-05-24 | 苏州晨本智能科技有限公司 | Mobile machine arm system based on surface myoelectric signal and control method of mobile machine arm system |
| CN107662195A (en) * | 2017-09-22 | 2018-02-06 | 中国东方电气集团有限公司 | A kind of mechanical hand principal and subordinate isomery remote operating control system and control method with telepresenc |
| CN107618018B (en) * | 2017-10-26 | 2020-08-25 | 杭州电子科技大学 | Manipulator action speed proportional control method based on myoelectricity |
| CN108673505A (en) * | 2018-05-28 | 2018-10-19 | 南昌大学 | A kind of mechanical arm tail end precise motion control method |
-
2019
- 2019-08-13 CN CN201910743884.2A patent/CN110584790B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110584790A (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8515576B2 (en) | Surgical robot and robotic controller | |
| Dario et al. | Smart surgical tools and augmenting devices | |
| CN107049492A (en) | The display methods of surgical robot system and surgical instrument position | |
| EP3684280A2 (en) | Systems and methods for controlling a robotic manipulator or associated tool | |
| EP3503831A1 (en) | Motion execution of a robotic system | |
| CA2550468A1 (en) | Surgical robot and robotic controller | |
| CN208575867U (en) | A kind of long-range control robot arm system carrying medical detection sensor | |
| Cohn et al. | Surgical applications of milli‐robots | |
| Maimon-Mor et al. | Towards free 3D end-point control for robotic-assisted human reaching using binocular eye tracking | |
| Mouchoux et al. | Artificial perception and semiautonomous control in myoelectric hand prostheses increases performance and decreases effort | |
| TWI695765B (en) | Robotic arm | |
| JP6518932B2 (en) | Motion analysis device and motion analysis program | |
| JP2014004656A (en) | Manipulation system | |
| He et al. | Automatic light pipe actuating system for bimanual robot-assisted retinal surgery | |
| Dwivedi et al. | A shared control framework for robotic telemanipulation combining electromyography based motion estimation and compliance control | |
| Martin et al. | A novel approach of prosthetic arm control using computer vision, biosignals, and motion capture | |
| CN112891137A (en) | Upper limb rehabilitation robot system, robot control method and device | |
| Nisky et al. | A framework for analysis of surgeon arm posture variability in robot-assisted surgery | |
| CN110584790B (en) | Arm stiffness-based teleoperation proportion control method for surgical robot | |
| EP3823562B1 (en) | Methods for enabling movement of objects, and associated apparatus | |
| Úbeda et al. | Brain-machine interface based on EEG mapping to control an assistive robotic arm | |
| Gardner et al. | An unobtrusive vision system to reduce the cognitive burden of hand prosthesis control | |
| CN114098981A (en) | Head and neck auxiliary traction surgical robot with two cooperative arms and control method thereof | |
| Szczurek et al. | Enhanced Human-Robot Interface with Operator Physiological Parameters Monitoring and 3D Mixed Reality | |
| US20230028689A1 (en) | System and method for inter-arm registration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |