CN114366313B - Endoscope holding robot control method based on laparoscopic surgical instrument pose - Google Patents
Endoscope holding robot control method based on laparoscopic surgical instrument pose Download PDFInfo
- Publication number
- CN114366313B CN114366313B CN202210276612.8A CN202210276612A CN114366313B CN 114366313 B CN114366313 B CN 114366313B CN 202210276612 A CN202210276612 A CN 202210276612A CN 114366313 B CN114366313 B CN 114366313B
- Authority
- CN
- China
- Prior art keywords
- pose
- laparoscope
- endoscope
- surgical instrument
- instrument
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B2034/302—Surgical robots specifically adapted for manipulations within body cavities, e.g. within abdominal or thoracic cavities
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Robotics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Endoscopes (AREA)
Abstract
The invention discloses a method for controlling a endoscope holding robot based on the pose of a laparoscopic surgical instrument, which comprises the following steps: holding mirror robot includes: a plurality of joint units and position sensors provided in the joint units; the laparoscope holding robot clamps a laparoscope; arranging a laparoscopic surgical instrument pose sensor on a laparoscopic surgical instrument to acquire the spatial pose of the laparoscopic surgical instrument; calculating the space pose of the current laparoscope according to the positions of all joints acquired by the position sensors of all joint units; the method comprises the steps of collecting space pose information of a current laparoscopic surgical instrument pose sensor, and controlling a laparoscope to reach a target space pose through a laparoscope holding robot according to pose changes in the moving process of the laparoscopic surgical instrument. The invention uses the endoscopic surgical instrument as the active hand and the endoscope holding robot as the slave hand, thereby realizing the master-slave control and the follow-up control.
Description
Technical Field
The invention relates to the field of medical instrument control methods, in particular to a control method of a endoscope holding robot based on the pose of a laparoscopic surgical instrument.
Background
Laparoscopic surgery is a newly developed minimally invasive method and is a necessary trend for the development of future surgical methods. With the rapid advance of industrial manufacturing technology, the fusion of related disciplines lays a firm foundation for the development of new technology and new method, and the more and more skillful operation of doctors makes many past open operations replaced by intracavitary operations, thereby greatly increasing the operation selection opportunities.
Laparoscopic surgery is an operation performed using a laparoscope and laparoscopic surgical instruments: the laparoscope lens is inserted into the abdominal cavity by using a robot holding the endoscope, and the image shot by the laparoscope lens is conducted to a post-stage signal processing system through an optical fiber by using a digital camera technology and is displayed on a special monitor in real time. Then, the doctor analyzes and judges the state of the patient through images of different angles of the patient organ displayed on the monitor screen, and performs the operation by using the laparoscopic surgery instrument.
Minimally invasive surgery is a general trend and pursuit of surgical development. The golden standard for laparoscopic surgery is cholecystectomy, and, in general, most of the common surgical procedures, laparoscopic surgery, can be performed. Such as appendectomy, gastric and duodenal ulcer perforation repair, hernia repair, colectomy, splenectomy, adrenal gland resection, ovarian cyst removal, extrauterine pregnancy, hysterectomy and the like, almost all surgical operations can be adopted with the increasing perfection of laparoscopic techniques and the improvement of the operating level of laparoscopic surgeons.
Traditional laparoscopic surgery adopts voice control or footboard, handle etc. to control and holds mirror robot, and the doctor of main knife both hands need to leave surgical instruments to control, handle, or disperse energy and go through pronunciation or footboard control and hold mirror robot. Compared with the scheme that the marker points are arranged on the laparoscopic surgery mechanical arm, the pose of the laparoscopic surgery mechanical arm is acquired and calculated by arranging the camera capable of identifying the marker points, and the situation that the pose of the laparoscopic surgery mechanical arm cannot be tracked due to the fact that the camera is shielded exists.
Disclosure of Invention
The invention provides a method for controlling a mirror holding robot based on the pose of a laparoscopic surgical instrument.
A method for controlling a mirror holding robot based on the pose of a laparoscopic surgical instrument comprises the following steps:
1) holding mirror robot includes: a plurality of joint units and position sensors provided in the joint units;
the laparoscope holding robot is clamped with a laparoscope;
2) arranging a laparoscopic surgical instrument pose sensor on a laparoscopic surgical instrument to acquire the spatial pose of the laparoscopic surgical instrument;
3) calculating the space pose of the current laparoscope through the positions of all joints acquired by the position sensors of all joint units
；
4) Collecting space pose information of current laparoscopic surgery instrument pose sensor
And according to the pose change of the laparoscopic surgical instrument in the moving process, the laparoscope holding robot controls the pose of the laparoscope to the target space.
The invention uses the endoscopic surgical instrument as the active hand and the endoscope holding robot as the passive hand, thereby realizing the master-slave control and the follow-up control, and the laparoscope on the endoscope holding robot can follow the master-slave control mode and the follow-up control mode in the moving process of the laparoscopic surgical instrument.
In step 1), it is further preferable that the mirror holding robot includes:
a rotary joint unit on which a rotary joint position sensor is disposed;
the pitching joint unit is connected with the rotating joint unit and is provided with a pitching joint position sensor;
the lifting joint unit is connected with the pitching joint unit and is provided with a lifting joint position sensor;
the lifting joint unit is connected with the laparoscope;
the spatial pose of the laparoscope can be obtained through the joint positions obtained by the rotary joint position sensor, the pitching joint position sensor and the lifting joint position sensor.
In the step 2), the pose comprises a position part and a pose part, the installation position of the pose sensor is not limited to the handle, and the spatial pose of the surgical instrument can be obtained as long as the relative pose of the pose sensor and the instrument is fixed. Further preferably, a laparoscopic surgical instrument pose sensor is arranged on a handle of the laparoscopic surgical instrument to acquire a spatial pose of the handle.
The handle is provided with a mode change-over switch, such as a change-over switch of a master-slave control mode and a follow-up control mode, and the switch position can be arranged on the handle, so that the operation of a doctor is facilitated.
The handle is also provided with a clutch switch in a master-slave control mode, the clutch is pressed in the master-slave mode through the clutch switch, and the laparoscopic surgical instrument can be used as a control handle to control the space pose of the endoscope holding robot and the laparoscope on the endoscope holding robot; and when the clutch switch is released, the endoscope holding robot does not move along with the laparoscopic surgical instrument.
In the step 3), the space pose of the current laparoscope is calculated through the joint positions obtained by the rotary joint position sensor, the pitching joint position sensor and the lifting joint position sensor
。
In the step 4), the control can adopt a master-slave control mode or a follow-up control mode.
The master-slave control mode includes:
4.1.1) collecting the spatial pose information of the laparoscopic surgical instrument pose sensor of the next adjacent control period
；
4.1.2) calculation
And
at a position change Δ T therebetween T According to Δ T T To calculate the transformation quantity Delta T of the laparoscope space position information E ；
4.1.3) according to
And Δ T E To calculate the target space pose information of the next control period of the laparoscope
；
4.1.4) target space pose information according to laparoscope
And calculating the target position of each joint of the endoscope holding robot in a reverse-thrust manner, and adjusting each joint of the endoscope holding robot to the target position, so that the endoscope holding robot clamps the laparoscope to realize master-slave control in one control period.
And repeating the steps, so that the laparoscope holding robot holds the laparoscope to realize continuous master-slave control.
The clutch switch is used for establishing or disconnecting a master-slave control relationship, and the process takes effect when the clutch switch establishes the master-slave control relationship; when the clutch switch disconnects the master-slave control relationship, the process is not effective, and the mirror holding robot keeps the current spatial pose.
By adopting a master-slave control mode, in the operation process, a master surgeon needs to adjust the spatial pose of the laparoscope, presses a master-slave control mode button attached to the surgical instrument, controls the action of the laparoscope by moving the laparoscopic surgical instrument, and ensures that the action trends of the laparoscopic surgical instrument (instrument for short) and the laparoscope (endoscope for short) accord with the intuitive feeling of people, thereby realizing the control of the instrument action on the pose of the laparoscope. When the position of the instrument reaches the limit position in the adjusting process and cannot move continuously, so that the cavity cannot move continuously, the connection of the master-slave control is disconnected through a clutch button attached to the handle of the instrument, the clutch button is pressed down again after the instrument is adjusted to a proper pose, the master-slave control is reestablished, and the pose of the master hand can be adjusted continuously through the instrument.
The follow-up control mode includes:
4.2.1) according to
、
And calculating the position P of the end effector of the laparoscopic surgical instrument in the image plane S according to the laparoscopic field depth parameter and the laparoscopic field angle parameter T ；
4.2.2) setting the threshold t% in the form of a percentage when P T The area is t% of the center of the S (the area can be circled by geometric images such as circles, rectangles and the like), the angle of the endoscope observation instrument is considered to be reasonable, and the endoscope holding robot holds the endoscope to keep the current pose;
in step 4.2.2), the area of t% of the center is a designated area of t% of the image plane S occupied by the center of the image plane S outwards, the designated area can be defined by geometric images such as circles and rectangles, and t% is the percentage of the designated area and the image plane S;
4.2.3) when P T When the position is beyond the area of t% of the center of the S, the endoscope observation instrument is considered to be unreasonable in angle, and the endoscope holding robot adjusts the space pose of the clamped laparoscope according to the deviation direction to enable the P T And returning to the area t% of the center of S.
And repeating the process to realize a follow-up control mode.
In the follow-up control mode, the method for acquiring the pose of the instrument and the pose of the endoscope is the same as that in the master-slave mode.
The difference from the master-slave mode is that the master-slave mode laparoscope action completely depends on the action of the laparoscopic surgical instruments, the laparoscope is not moved when the laparoscopic surgical instruments are not moved, and in the slave mode, the laparoscopic action is related to the action of the laparoscopic surgical instruments but not completely depends on the action of the laparoscopic surgical instruments, so that the laparoscopic surgical instruments move in a small range and the laparoscope is not moved; when the laparoscopic surgical instrument moves outside the endoscope view field in a large range, the laparoscope starts to move along with the laparoscopic surgical instrument.
By adopting a follow-up control mode, in the operation process, a primary surgeon does not need to actively control and adjust the space pose of the laparoscope, only needs to pay attention to the operation, the laparoscope holding robot can automatically calculate the proper space pose of the laparoscope according to a laparoscope operation instrument pose sensor arranged on an operation instrument according to the follow-up control method provided by the invention, and the laparoscope is held by the laparoscope holding robot to move to the space pose, when the instrument is operated in a small range, the laparoscope does not move along, so that the instrument can be ensured to be in the visual field range of the laparoscope, the visual field is stable, and the instrument cannot move along with the high-frequency small-range movement of the laparoscope to shake; when the instrument is pulled, transferred and the like, the instrument moves in a large range and can move out of the scope visual field range, and at the moment, according to the algorithm, the scope automatically follows the instrument to act, so that the instrument in the scope visual field continuously exists.
The control of the invention can adopt a master-slave control mode or a follow-up control mode, and the necessity of the two modes is as follows: the follow-up mode may be used when performing conventional surgical procedures. However, when the operation is nearly finished, the abdominal cavity needs to be cleaned in a large range, and the pose of the endoscope needs to be adjusted in a large range for observation, so that a master-slave mode with a clutch switch is also indispensable.
Compared with the prior art, the invention has the following advantages:
the endoscope operation instrument is used as a master hand, the endoscope holding robot is used as a slave hand, master-slave control and follow-up control are realized, and compared with the traditional endoscope operation, the endoscope operation instrument saves labor and does not need manual assistants; compared with the endoscope holding robot controlled by sound control or pedals, handles and the like, the endoscope holding robot can realize the control of the endoscope observation angle and observation position only by moving the instruments without leaving the surgical instruments to control and handle by both hands of the doctor or dispersing energy to control the endoscope holding robot through voice or pedals. Compared with a control mode based on endoscope image identification, the method does not need a complex image algorithm, and can more reliably obtain the spatial poses of the endoscope, the instruments and the like. Compared with the scheme that the mark points are arranged on the instrument arm, and the pose of the instrument arm is acquired and calculated by arranging the camera capable of identifying the mark points, the technical scheme of the invention does not need to arrange the camera for identifying the pose of the instrument in an operating room, and the situation that the pose of the instrument cannot be tracked due to the fact that the camera is shielded does not exist.
Drawings
FIG. 1 is a schematic diagram of a control method of a laparoscope holding robot based on the pose of a laparoscopic surgical instrument.
FIG. 2 is a schematic diagram of the principle of master-slave control in the present invention.
Fig. 3 is a schematic diagram of the principle of the servo control in the present invention.
FIG. 4 is a schematic view of the mirror holding robot of the present invention mounted on a support table via a connecting rod.
FIG. 5 is a schematic structural diagram of a robot with a mirror held therein according to the present invention.
Detailed Description
A, system composition
The whole system can be divided into a laparoscope surgical instrument pose acquisition subsystem and a laparoscope holding robot subsystem, wherein the instrument pose acquisition subsystem can acquire position and pose data of an instrument hand-held part in a coordinate system which is at the RCM position of the instrument and is relatively static with a world coordinate system in an inertial navigation mode, a visual mode and the like, and the position and pose data can be used for guiding the laparoscope holding robot subsystem to clamp a laparoscope to complete following actions after being calculated by the method. The processing system for mapping operation can be listed separately or integrated into the robot system, and the whole system adopts a periodic control mode to finish data acquisition, operation, issuing and execution in one period.
As shown in fig. 4 and 5, the mirror holding robot is mounted on the support base 6 through the link 5, and the link 5 is mounted on the support base 6; the connecting rod 5 is a multi-section structure, the connecting rod 5 at least comprises three sections, the first section of connecting rod is connected with the second section of connecting rod through a rotating structure, the rotating direction of the rotating structure is the circumferential direction of the second section of connecting rod, the second section of connecting rod is connected with the third section of connecting rod through a rotating structure, and the rotating structure rotates in the radial direction of the third section of connecting rod, namely the rotating direction of the rotating structure is perpendicular to the axial direction of the third section of connecting rod. The robot for holding the mirror mainly comprises three parts: the rotary joint unit 1, the pitch joint unit 2, the lift joint unit 3 include: a rotary joint unit 1 mounted on the link 5; a pitch joint unit 2 connected to the rotary joint unit 1; and a lift joint unit 3 connected to the pitch joint unit 2; the elevation joint unit 3 is connected with a laparoscope 4, and the laparoscope 4 is mounted on the elevation joint unit 3. A rotary joint position sensor is arranged on the rotary joint unit 1, and a pitching joint position sensor is arranged on the pitching joint unit 2; the lifting joint unit 3 is provided with a lifting joint position sensor; the elevation joint unit 3 is connected with the laparoscope 4.
Second, control method
As shown in fig. 1, a method for controlling a laparoscope holding robot based on the pose of a laparoscopic surgical instrument includes the following steps:
1) arranging a laparoscopic surgical instrument pose sensor on a laparoscopic surgical instrument to acquire the spatial pose of the laparoscopic surgical instrument;
the handle is provided with a switch for switching between a master-slave control mode and a follow-up control mode. The handle is also provided with a clutch switch in a master-slave control mode;
2) calculating the space pose of the current laparoscope through the joint positions acquired by the rotary joint position sensor, the pitching joint position sensor and the lifting joint position sensor
；
3) Collecting space pose information of current laparoscopic surgery instrument pose sensor
And according to the pose change of the laparoscopic surgical instrument in the moving process, the laparoscope holding robot controls the pose of the laparoscope to the target space.
In the step 3), the control can adopt a master-slave control mode or a follow-up control mode; the control method comprises two control modes, wherein one control mode is a master-slave mode, namely the posture change of the surgical instrument is used for adjusting the posture of the endoscope, the process is similar to the process of using a master hand of a surgical robot to control a slave hand to drive the endoscope to move, and the endoscope action completely depends on the posture change of the surgical instrument, namely the mapping relation between the posture of the surgical instrument and the posture of the endoscope; the other mode is a follow-up mode, namely the endoscope action depends on the behavior of the surgical instrument rather than being directly mapped by the pose of the instrument. These will be described separately below.
1. Master-slave control: in a master-slave control mode, after the pose of the surgical instrument is acquired, the moving target of the endoscope is obtained through a mapping relation. Because the initial posture of the surgical instrument cannot be guaranteed to be the same as that of the endoscope, a relative mapping mode is adopted in the aspect of posture, namely the variation of the posture of the endoscope is the same as that of the posture of the surgical instrument; in the aspect of position, because the position of the instrument, particularly the depth of inserting the endoscope into the Trocar, is limited, position mapping is not carried out in master-slave control, namely the posture adjustment of the endoscope is controlled only through the posture of the instrument, and the depth of inserting the endoscope into the Trocar is not controlled. As shown in fig. 2, the specific method of master-slave control is as follows:
1) establishing a base coordinate system based on the geometrical characteristics of the endoscope and the instrument, respectively recording as an endo _ base coordinate system and a tool _ base coordinate system, wherein the postures of the endo _ base coordinate system and the tool _ base coordinate system are the same, and the endo _ base coordinate system and the tool _ base coordinate system are static relative to a world coordinate system in the motion process of the endoscope and the instrument;
2) establishing current coordinate systems of the endoscope and the instrument, which are respectively marked as an endo _ current coordinate system and a tool _ current coordinate system, wherein when the endoscope and the instrument are respectively in initial postures, the endo _ current coordinate system is superposed with the endo _ base, the tool _ current coordinate system is superposed with the tool _ base, and in the moving process of the endoscope and the instrument, the two coordinate systems are respectively fixedly connected on the endoscope and the instrument and move along with the endoscope and the instrument, but the posture transformation of the endo _ current coordinate system relative to the endo _ base coordinate system can be obtained in a robot subsystem for holding the endoscope, and the posture transformation of the tool _ current coordinate system relative to the tool _ base coordinate system can be obtained by an instrument posture acquisition subsystem;
3) establishing a next-time coordinate system of the endoscope and the instrument, which is respectively marked as an endo _ next (endoscope next time) coordinate system and a tool _ next (instrument next time) coordinate system, wherein the next time refers to the next data acquisition time (the current time and the next time are respectively in two control cycles which are adjacent front and back), for the instrument, the change of the tool _ next relative to the tool _ current reflects the posture change of the motion of the handheld instrument, and for the endoscope, the change of the tool _ next relative to the tool _ current reflects the motion result of the mapped instrument motion acting on the endoscope.
After the above coordinate system is established, the following details describe the process of master-slave control:
1) for the nth cycle in the control loop cycle, the tool _ current gesture is obtained, i.e., the tool_ base R tool_current In the (n + 1) th cycle, the tool _ next gesture is obtained, that is, the tool _ next gesture is obtained tool_base R tool_next ；
Where R represents a 3 x 3 rotation matrix, the left superscript of R represents the reference coordinate system, and the right subscript of R represents the current coordinate system. Each row in R is a unit row vector of 3 multiplied by 1, the first row is used for representing a direction vector of an X axis of a current coordinate system under a reference coordinate system, the second row is used for representing a direction vector of a Y axis of the current coordinate system under the reference coordinate system, and the third row is used for representing a direction vector of a Z axis of the current coordinate system under the reference coordinate system;
2) computing a transformation matrix from tool _ current to tool _ next tool_current R tool_next ， tool_ current R tool_next =( tool_base R tool_current ) -1tool_base R tool_next -1 represents the transpose of the matrix, the formula being a matrix multiplication of the two matrices;
3) mapping the transformation from tool _ current to tool _ next onto the mirror holder robot, namely: endo_ current R endo_next = tool_current R tool_next ；
4) calculating the target pose of endo _ current: endo_base R endo_next = endo_base R endo_current endo_ current R endo_next in the formula endo_base R endo_current The system can be obtained by combining the known design parameters of each joint unit according to the joint position measured by each joint position sensor of the robot holding the mirror, and can be solved by using a D-H method in robotics; in the formula endo_current R endo_next Obtained from step 3) above;
5) and taking the obtained endoscop-next coordinate system as an input target of the endoscope holding robot system, namely obtaining target angles of all joints of the endoscope holding robot through inverse operation of a D-H method in robotics, and guiding the endoscope holding robot to clamp the endoscope to complete following action.
In addition, because the motion posture range of the instrument is limited in the operation process, the clutch device can be connected to the instrument and used for disconnecting the master-slave mapping and reestablishing the master-slave mapping. Due to the adoption of the posture relative mapping mode, the movement of the instrument to the limit position can be realized, and the mapping is disconnected to adjust the position and is remapped again.
2. Follow-up control: besides the master-slave mapping control mode of adjusting the posture of the endoscope through the posture of the instrument, the system also has a follow-up control mode of the endoscope following the instrument in real time in the operation process. However, in the course of the operation, there are not only operations requiring a large-scale movement of the instrument such as transfer and pulling, but also operations requiring a small-scale repetitive movement of the instrument such as suturing, and if the follow-up control is always performed during the small-scale repetitive movement of the instrument, the visual field will be shaken, and the observation effect of the operator will be mapped. In addition, there are usually two instruments that cooperate to perform the procedure, and the particular instrument that the instrument follows is also a concern. The following control algorithm that comprehensively considers the above-described problems will be described in detail below.
As shown in fig. 3, the following steps and strategies are:
1) firstly, establishing a coordinate system for the tail end of an instrument, the Trocar of the instrument, the tail end of an endoscope and the Trocar of the endoscope, recording the coordinate system as a tool (tail end of the instrument) coordinate system, a tool _ Trocar (instrument stamp card) coordinate system, an endoscope (endoscope tail end coordinate system) coordinate system and an endo _ Trocar (endoscope stamp card) coordinate system, and recording a world coordinate system as a world coordinate system;
2) for each coordinate system established in 1), the poses of the tool _ trocar and the endo _ trocar in the world coordinate system world can be positioned and measured by adopting visual or other means before operation and are recorded as world T tool_trocar 、 world T endo_trocar ；
3) For the coordinate system established above, the pose of the tool in the tool _ trocar can be obtained by a sensor integrated on the instrument, and is recorded as tool_trocar T tool Similarly, the pose of the endoscope in the endo _ trocar can be obtained by numerical calculation of a joint sensor on the endoscope holding robot and is recorded as endo_trocar T endoscope ；
A homogeneous transformation matrix in which T is 4 × 4, the 3 × 3 area at the upper left corner of the matrix is the same as R (a 3 × 3 rotation matrix) explained above, the fourth column one to three rows in the matrix are column vectors representing positions, and the remaining fourth row is filled with a row vector (0, 0, 0, 1);
4) from 2 world T tool_trocar 、 world T endo_trocar And 3) of tool_trocar T tool 、 endo_trocar T endoscope To obtain world T tool 、 world T endoscope The formula is as follows: world T tool = world T tool_trocar tool_trocar T tool ， world T endoscope = world T endo_trocar endo_trocar T endoscope ；
5) by world T tool 、 world T endscope Calculating the pose of the tool in the endoscope coordinate system: endoscope T tool the formula is as follows: endoscope T tool =( world T endoscope ) -1world T tool ；
6) by endoscope T tool The vector representing the position of the tail end point of the instrument in the endoscope coordinate system can be obtained endoscope P tool ；
7) Root of herbaceous plantAccording to endoscope P tool And camera parameters, which can calculate the coordinates of the terminal point of the instrument in the endoscope image plane and are marked as P tool
8) Finding the central point of the plane of the endoscope image, and marking the coordinate as P center And a circular area is formed by taking the set value R as a radius, the area is marked as S, and the calculation method of R is as follows: r =
Wherein A is the area of the image plane, and t% is a threshold value set by a user and represents the percentage of the S area in the image plane;
9) when P is present tool And P center When the distance between the endoscope and the endoscope is smaller than the radius R, the instrument is considered to be in a proper area in the endoscope image, namely the endoscope pose is reasonable, and the endoscope holding robot only needs to keep the current pose;
10) when P is tool And P center When the distance between the two is larger than the radius R, the position of the instrument in the endoscope image is not reasonable, the pose of the endoscope is not reasonable, and the endoscope holding robot is supposed to follow the vector P center P tool The pose of the endoscope is adjusted in the pointed direction, so that the instrument returns to the central area of the visual field again to realize follow-up;
11) the central region S of the endoscope can also be represented by other shapes such as a rectangle, but the calculation P is calculated tool Whether the specific method in this area is different is not always an example.
Claims (1)
1. A method for controlling a endoscope holding robot based on the pose of a laparoscopic surgical instrument is characterized by comprising the following steps:
1) holding mirror robot includes: a plurality of joint units and position sensors provided in the joint units;
the laparoscope holding robot is clamped with a laparoscope;
the robot holding the mirror comprises:
a rotary joint unit on which a rotary joint position sensor is disposed;
the pitching joint unit is connected with the rotating joint unit and is provided with a pitching joint position sensor;
the lifting joint unit is connected with the pitching joint unit and is provided with a lifting joint position sensor;
the lifting joint unit is connected with the laparoscope;
2) arranging a laparoscopic surgical instrument pose sensor on a laparoscopic surgical instrument to acquire the spatial pose of the laparoscopic surgical instrument;
a switch for switching between a master-slave control mode and a follow-up control mode is arranged on the handle;
the handle is also provided with a clutch switch in a master-slave control mode;
3) calculating the space pose of the current laparoscope through the positions of all joints acquired by the position sensors of all joint units
；
The position sensor of each joint unit includes: a rotary joint position sensor, a pitching joint position sensor and a lifting joint position sensor;
4) collecting space pose information of current laparoscopic surgery instrument pose sensor
According to the pose change of the laparoscopic surgical instrument in the moving process, the laparoscope holding robot controls the pose of the laparoscope to the target space;
the control adopts a master-slave control mode and a follow-up control mode;
the master-slave control mode comprises the following steps:
4.1.1) collecting the spatial pose information of the laparoscopic surgical instrument pose sensor of the next adjacent control period
；
4.1.2) calculation
And
at a position change Δ T therebetween T According to Δ T T To calculate the transformation quantity Delta T of the laparoscope space position information E ；
4.1.3) according to
And Δ T E To calculate the target space pose information of the next control period of the laparoscope
；
4.1.4) target space pose information according to laparoscope
Calculating the target position of each joint of the endoscope-holding robot in a reverse-deducing manner, adjusting each joint of the endoscope-holding robot to the target position, and moving the clamped laparoscope to the target space pose by the endoscope-holding robot to complete master-slave control in a control period;
the follow-up control mode comprises the following steps:
4.2.1) according to
、
And calculating the position P of the end effector of the laparoscopic surgical instrument in the image plane S according to the laparoscopic field depth parameter and the laparoscopic field angle parameter T ；
4.2.2) setting a threshold t% in the form of a percentage when P T In the area t% of the center of S, the angle of the endoscope observation instrument is reasonable, and the endoscope holding robot holds the endoscope to keep the current pose;
the area of the central t% is a designated area of the image plane S, wherein the center of the image plane S outwards occupies t% of the image plane S;
4.2.3) when P T When the position is beyond the area of t% of the center of the S, the endoscope observation instrument is considered to be unreasonable in angle, and the endoscope holding robot adjusts the space pose of the clamped laparoscope according to the deviation direction to enable the P T Returning to the area t% of the center of S;
and repeating the process to realize a follow-up control mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210276612.8A CN114366313B (en) | 2022-03-21 | 2022-03-21 | Endoscope holding robot control method based on laparoscopic surgical instrument pose |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210276612.8A CN114366313B (en) | 2022-03-21 | 2022-03-21 | Endoscope holding robot control method based on laparoscopic surgical instrument pose |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114366313A CN114366313A (en) | 2022-04-19 |
| CN114366313B true CN114366313B (en) | 2022-08-02 |
Family
ID=81145760
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210276612.8A Active CN114366313B (en) | 2022-03-21 | 2022-03-21 | Endoscope holding robot control method based on laparoscopic surgical instrument pose |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114366313B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5820545A (en) * | 1995-08-14 | 1998-10-13 | Deutsche Forschungsanstalt Fur Luft-Und Raumfahrt E.V. | Method of tracking a surgical instrument with a mono or stereo laparoscope |
| WO2011108018A1 (en) * | 2010-03-02 | 2011-09-09 | Danieli, Giovanna | Instrument for controlling and moving laparoscopic instruments and to drive precision biopsies under ecographic or fluoroscopic guidance |
| KR20140112601A (en) * | 2013-03-11 | 2014-09-24 | 삼성전자주식회사 | Endoscopic surgical instrument |
| CN108814715A (en) * | 2018-04-17 | 2018-11-16 | 成都博恩思医学机器人有限公司 | A kind of the sound effect control method and system of the pedal of laparoscopic surgery robot |
| CN109758234A (en) * | 2019-02-15 | 2019-05-17 | 哈尔滨工业大学 | For the mobile laparoscope automatic control system of Minimally Invasive Surgery and its control method |
| CN113538522A (en) * | 2021-08-12 | 2021-10-22 | 广东工业大学 | Instrument vision tracking method for laparoscopic minimally invasive surgery |
| CN113633387A (en) * | 2021-06-21 | 2021-11-12 | 安徽理工大学 | Surgical field tracking supporting laparoscopic minimally invasive robot touch force interaction method and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105411681B (en) * | 2015-12-22 | 2018-07-03 | 哈尔滨工业大学 | The hand eye coordination control system and method for split type micro-wound operation robot |
| US10500000B2 (en) * | 2016-08-16 | 2019-12-10 | Ethicon Llc | Surgical tool with manual control of end effector jaws |
| CN209464087U (en) * | 2018-09-14 | 2019-10-08 | 乔艳晶 | A kind of laparoscope machinery holds mirror arm |
| CN109330685B (en) * | 2018-10-31 | 2024-02-02 | 南京航空航天大学 | Automatic navigation method for laparoscope of porous abdominal cavity surgical robot |
| CN112353361B (en) * | 2020-09-21 | 2023-07-25 | 常州市速瑞医疗科技有限公司 | 3D pleuroperitoneal cavity system based on master-slave integrated intelligent mirror supporting robot |
| CN113925613B (en) * | 2021-09-27 | 2023-07-07 | 中山大学 | Laparoscopic surgery holds mirror robot system |
-
2022
- 2022-03-21 CN CN202210276612.8A patent/CN114366313B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5820545A (en) * | 1995-08-14 | 1998-10-13 | Deutsche Forschungsanstalt Fur Luft-Und Raumfahrt E.V. | Method of tracking a surgical instrument with a mono or stereo laparoscope |
| WO2011108018A1 (en) * | 2010-03-02 | 2011-09-09 | Danieli, Giovanna | Instrument for controlling and moving laparoscopic instruments and to drive precision biopsies under ecographic or fluoroscopic guidance |
| KR20140112601A (en) * | 2013-03-11 | 2014-09-24 | 삼성전자주식회사 | Endoscopic surgical instrument |
| CN108814715A (en) * | 2018-04-17 | 2018-11-16 | 成都博恩思医学机器人有限公司 | A kind of the sound effect control method and system of the pedal of laparoscopic surgery robot |
| CN109758234A (en) * | 2019-02-15 | 2019-05-17 | 哈尔滨工业大学 | For the mobile laparoscope automatic control system of Minimally Invasive Surgery and its control method |
| CN113633387A (en) * | 2021-06-21 | 2021-11-12 | 安徽理工大学 | Surgical field tracking supporting laparoscopic minimally invasive robot touch force interaction method and system |
| CN113538522A (en) * | 2021-08-12 | 2021-10-22 | 广东工业大学 | Instrument vision tracking method for laparoscopic minimally invasive surgery |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114366313A (en) | 2022-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7275204B2 (en) | System and method for on-screen menus in telemedicine systems | |
| US8892224B2 (en) | Method for graphically providing continuous change of state directions to a user of a medical robotic system | |
| US9155592B2 (en) | Virtual measurement tool for minimally invasive surgery | |
| US8600551B2 (en) | Medical robotic system with operatively couplable simulator unit for surgeon training | |
| US6659939B2 (en) | Cooperative minimally invasive telesurgical system | |
| US9492240B2 (en) | Virtual measurement tool for minimally invasive surgery | |
| US7087049B2 (en) | Repositioning and reorientation of master/slave relationship in minimally invasive telesurgery | |
| KR20200078422A (en) | System and method for master/tool matching and control for intuitive movement | |
| WO2013048957A1 (en) | Robot-mounted surgical tables | |
| JP2018143560A (en) | System and method for manipulating endoscope | |
| WO2023083077A1 (en) | Method for maintaining rc point unchanged, and robotic arm, device, robot and medium | |
| WO2023083078A1 (en) | Mechanical arm, slave operation apparatus and surgical robot | |
| CN114366313B (en) | Endoscope holding robot control method based on laparoscopic surgical instrument pose | |
| US20220323157A1 (en) | System and method related to registration for a medical procedure | |
| Munoz et al. | Design and control of a robotic assistant for laparoscopic surgery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |