WO2022032964A1 - Procédé d'étalonnage de robot à deux bras, système, terminal et support de stockage - Google Patents

Procédé d'étalonnage de robot à deux bras, système, terminal et support de stockage Download PDF

Info

Publication number
WO2022032964A1
WO2022032964A1 PCT/CN2020/139303 CN2020139303W WO2022032964A1 WO 2022032964 A1 WO2022032964 A1 WO 2022032964A1 CN 2020139303 W CN2020139303 W CN 2020139303W WO 2022032964 A1 WO2022032964 A1 WO 2022032964A1
Authority
WO
WIPO (PCT)
Prior art keywords
needle tip
coordinate system
robot
dual
needle
Prior art date
Application number
PCT/CN2020/139303
Other languages
English (en)
Chinese (zh)
Inventor
熊璟
徐常福
夏泽洋
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2022032964A1 publication Critical patent/WO2022032964A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/34—Trocars; Puncturing needles
    • A61B17/3403—Needle locating or guiding means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/77—Manipulators with motion or force scaling
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/34—Trocars; Puncturing needles
    • A61B17/3403—Needle locating or guiding means
    • A61B2017/3413—Needle locating or guiding means guided by ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101—Computer-aided simulation of surgical operations
    • A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/107—Visualisation of planned trajectories or target regions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2063—Acoustic tracking systems, e.g. using ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B2090/364—Correlation of different images or relation of image positions in respect to the body
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image

Definitions

  • the present application belongs to the technical field of robot calibration, and particularly relates to a calibration method, system, terminal and storage medium for a dual-arm robot.
  • the dual-arm robotic system can not only control the ultrasonic probe and the puncture needle tube separately, but also flexibly control the needle insertion angle and posture in a small surgical space.
  • the ultrasonic probe calibration method using Locator and Phantom uses Locator and Phantom.
  • the markers corresponding to the locator such as the opto-magnetic locator
  • the equation expression between the ultrasonic image and the actual physical position of the corresponding model is established according to the geometric parameters of the model designed in advance.
  • the coordinate transformation matrix between the ultrasonic image coordinate system and the coordinate system fixed on the ultrasonic probe locator is obtained by iterative approximation solution or closed-form equation system.
  • Ultrasonic probe calibration method using robot operation and plane calibration device.
  • the method uses a robot to operate the ultrasonic probe to scan the plane calibration device, and then establishes a combination of equations for the calibration between the ultrasonic image and the actual physical position of the corresponding plane calibration device, and then uses the equation to solve the relationship between the ultrasonic image coordinate system and the robot manipulator coordinate system. Coordinate transformation matrix.
  • the existing robot calibration methods usually require the use of third-party trackers or mechanism models, and these devices have certain errors in their positioning and size, so it is difficult to ensure the puncture accuracy of the robot.
  • the existing methods are mainly used for calibration of freed-hand ultrasound-guided puncture and single-arm robot calibration of ultrasound-guided puncture, but have not been well applied in the calibration of dual-arm robots.
  • the present application provides a method, system, terminal and storage medium for calibrating a dual-arm robot, aiming to solve one of the above-mentioned technical problems in the prior art at least to a certain extent.
  • a dual-arm robot calibration method comprising:
  • the iterative closest point algorithm is used to solve the position registration model, and a coordinate transformation matrix from the ultrasound image coordinate system to the robot end flange coordinate system is obtained, and the ultrasound probe is calibrated according to the coordinate transformation matrix.
  • the needle tip position calibration includes:
  • the technical solution adopted in the embodiment of the present application further includes: the position calculation formula of the preset reference point in the robot base coordinate system is:
  • ER P Mtip is the coordinate from the top to the end flange
  • R T ER is the pose of the end flange in the robot base coordinate system
  • the coordinates of the needle tip of the puncture needle in the robot end flange coordinate system are:
  • inv() represents the inversion operation of the matrix.
  • the technical solution adopted in the embodiment of the present application further includes: the acquiring the needle tip ultrasound image of the puncture needle tube through the ultrasound probe includes:
  • the poses of the flanges at the ends of the left and right robotic arms of the dual-arm robot are recorded, and the position of the needle tip in the image coordinate system in the needle tip ultrasound image is marked.
  • the technical solution adopted in the embodiment of the present application further includes: the establishment of a position registration model of the needle tip of the puncture needle tube and the needle tip in the ultrasound image of the needle tip under the robot base coordinate system according to the position of the needle tip and the ultrasound image of the needle tip includes: :
  • the base coordinates of the needle tip of the puncture needle tube in the robot are established
  • the registration model whose position under the system is equal to the position of the needle tip in the robot base coordinate system in the ultrasound image of the needle tip:
  • R T EL is the pose of the end flange of the robotic arm on which the ultrasonic probe end effector is installed on the dual-arm robot
  • R P Ntip is the coordinate transformation matrix from the needle tip coordinate system to the robot base coordinate system
  • R P Ntip R T ER ⁇ ER P Ntip
  • IP Ntip is the position of the needle tip in the image coordinate system in the needle tip ultrasound image
  • EL T I is the coordinate transformation matrix from the ultrasound image coordinate system to the robot end flange coordinate system.
  • the technical solution adopted in the embodiment of the present application further includes: establishing a needle tip position calibration model, and after calibrating the needle tip position of the puncture needle tube according to the model, the method further includes:
  • a needle tip posture calibration model is established, and the model calibrates the needle tip posture of the puncture needle tube based on the needle tip position calibration result.
  • the calibration of the needle tip posture of the puncture needle tube based on the needle tip position calibration result includes:
  • the attitude matrix of the needle tip of the puncture needle tube is obtained.
  • the technical solution adopted in the embodiment of the present application further includes: the establishing a position registration model of the needle tip of the puncture needle tube and the needle tip in the ultrasound image of the needle tip under the robot base coordinate system further includes:
  • the position of the needle tip in the ultrasound image of the needle tip of the puncture needle tube and the needle tip in the ultrasound image of the needle tip are established.
  • EL T Ptip is the coordinate transformation matrix from any fixed needle tip on the ultrasound probe to the flange at the end of the robotic arm where the ultrasound probe is installed
  • Ptip T I is the coordinate transformation matrix from the ultrasound image coordinate system to the coordinate system of the robot end flange.
  • a dual-arm robot calibration system including a needle tip calibration module and an ultrasonic probe calibration module;
  • the needle tip calibration module is used to establish a needle tip position calibration model, and calibrate the needle tip position of the puncture needle tube according to the model;
  • the ultrasonic probe calibration module includes:
  • Ultrasonic image acquisition sub-module used for acquiring the needle tip ultrasonic image of the puncture needle tube through an ultrasonic probe; wherein, the ultrasonic probe and the puncture needle tube are respectively fixed on the end effectors of the left and right mechanical arms of the dual-arm robot;
  • Needle tip position registration sub-module used to establish a position registration model of the calibrated puncture needle tube needle tip and the needle tip in the needle tip ultrasound image under the robot base coordinate system according to the needle tip position calibration result and the needle tip ultrasound image ;
  • Coordinate calculation sub-module used to solve the position registration model by using the iterative closest point algorithm, obtain the coordinate transformation matrix from the ultrasound image coordinate system to the robot end flange coordinate system, and calibrate the ultrasound probe according to the coordinate transformation matrix
  • a terminal includes a processor and a memory coupled to the processor, wherein,
  • the memory stores program instructions for implementing the dual-arm robot calibration method
  • the processor is configured to execute the program instructions stored in the memory to control the calibration of the dual-arm robot.
  • a storage medium storing program instructions executable by a processor, where the program instructions are used to execute the dual-arm robot calibration method.
  • the beneficial effect of the embodiment of the present application is that the dual-arm robot calibration method, system, terminal and storage medium of the embodiment of the present application can demarcate the position of the needle tip of the puncture needle tube by establishing a point-to-point needle tip position calibration model, Then, based on the calibration result and the needle tip ultrasound image scanned by the ultrasound probe, a registration model between the calibrated needle tip position and the needle tip position in the needle tip ultrasound image is established. Finally, the iterative closest point registration algorithm is used to calculate the ultrasound image coordinate system to the robot end method.
  • the coordinate transformation matrix of the blue coordinate system realizes the high-precision calibration of the needle tip of the puncture needle and the ultrasonic probe of the dual-arm robot.
  • the present application does not require the use of third-party locators and models.
  • FIG. 1 is a flowchart of a dual-arm robot calibration method according to a first embodiment of the present application
  • FIG. 2 is a flowchart of a method for calibrating a needle tip position of a needle tube according to the first embodiment of the application;
  • FIG. 3 is a flowchart of a dual-arm robot calibration method according to a second embodiment of the present application.
  • FIG. 4 is a flowchart of a method for calibrating a needle tip attitude of a needle tube according to a second embodiment of the present application
  • Figure 5 shows the application of the embodiment of the present application to a dual-arm robot puncture system based on ultrasonic guidance, wherein (a) is to control the tip to reach a specified reference point, (b) is to calibrate the needle tip of the puncture needle in the robot end flange coordinate system (c) is the position coordinate of the calibrated ultrasonic probe tip in the robot end flange coordinate system, (d) is to control the ultrasonic probe to scan the puncture needle tip;
  • FIG. 6 is a schematic structural diagram of a dual-arm robot calibration system according to an embodiment of the application.
  • FIG. 7 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • FIG. 1 is a flowchart of a method for calibrating a dual-arm robot according to the first embodiment of the present application.
  • the dual-arm robot calibration method according to the first embodiment of the present application includes two parts: puncturing needle tip position calibration and ultrasonic probe calibration, and specifically includes the following steps:
  • Step 100 establishing a needle tip position calibration model, and calibrating the needle tip position of the puncture needle;
  • the embodiment of the present application utilizes the characteristics of the dual-arm robot itself that the dual-arm high-precision positioning and the dual-arm position coordinates are in the same robot base coordinate system to establish a needle tip position calibration model, and calibrate the needle tip position.
  • FIG. 2 is a flowchart of a method for calibrating a needle tip position of a needle tube according to an embodiment of the invention, which specifically includes the following steps:
  • Step 101 Design and process a high-precision top, and install it on the end flange of either arm of the dual-arm robot;
  • Step 102 control the top to reach the preset reference point through the dual-arm robot, and calculate the position of the reference point in the robot base coordinate system; the calculation formula is:
  • ER P Mtip is the coordinates from the top to the end flange, which can be obtained according to the preset top parameters;
  • R T ER is the pose of the end flange in the robot base coordinate system, which can be found in the robot operating system. read directly.
  • Step 103 Remove the tip, and install the end effector of the puncture needle tube on the end flange of any mechanical arm of the dual-arm robot;
  • Step 104 control the needle tip of the puncture needle to reach the preset reference point through the dual-arm robot, and record the pose matrix of the end flange in the robot coordinate system;
  • Step 105 establish an equation that the position of the needle tip of the puncture needle is equal to the position of the reference point;
  • Step 106 Solve the equation expression, and calibrate the coordinates ER P Ntip of the needle tip of the puncture needle in the robot end flange coordinate system:
  • inv( ) represents the inversion operation of the matrix.
  • Step 110 Fix the ultrasonic probe and the puncture needle tube on the end effectors of the left and right mechanical arms of the dual-arm robot respectively, and control the ultrasonic probe to scan the needle tip of the puncture needle tube through the dual-arm robot to obtain an ultrasonic image of the needle tip, and record the dual-arm robot respectively.
  • a tracker may also be fixed on the ultrasound probe or puncture needle tube instead of the end flange position.
  • Step 120 mark the position IP Ntip of the needle tip in the image coordinate system in the ultrasound image of the needle tip;
  • Step 130 Based on the calibration result of the needle tip position of the puncture needle tube, the needle tip position in the ultrasound image of the needle tip, and the positions of the flanges of the left and right arm end flanges of the dual-arm robot, establish the position of the needle tip of the puncture needle tube and the needle tip in the ultrasound image of the needle tip in the robot base coordinate system registration model;
  • the needle tip position in the needle tip ultrasound image and the positions of the flanges of the left and right arm ends of the dual-arm robot are combined to establish the needle tip of the puncture needle tube and the needle tip ultrasound in the robot base coordinate system.
  • the position registration model of the needle tip in the robot base coordinate system in the image, its equation expression is:
  • Step 140 using the iterative closest point algorithm to solve the position registration model, obtain the coordinate transformation matrix ELTI from the ultrasound image coordinate system to the robot end flange coordinate system, and calibrate the ultrasound probe according to the coordinate transformation matrix;
  • the calibration of the ultrasound probe is completed through steps 110 to 140 .
  • the dual-arm robot calibration method utilizes the characteristics of the dual-arm robot itself that the dual-arm high-precision positioning and the dual-arm position coordinates are in the same robot base coordinate system, and firstly establishes a point-to-point needle tip position calibration Model, calibrate the position of the needle tip of the puncture needle, and then establish the position registration model of the needle tip of the puncture needle tube and the needle tip in the ultrasound image of the needle tip in the robot base coordinate system based on the calibration result and the ultrasound image of the needle tip scanned by the ultrasonic probe.
  • the point registration algorithm calculates the coordinate transformation matrix from the ultrasonic image coordinate system to the robot end flange coordinate system, and realizes the high-precision calibration of the needle tip of the puncture needle and the ultrasonic probe of the dual-arm robot.
  • FIG. 3 is a flowchart of a method for calibrating a dual-arm robot according to the second embodiment of the present application.
  • the dual-arm robot calibration method according to the second embodiment of the present application includes two parts: puncturing needle tip position and attitude calibration and ultrasonic probe calibration, and specifically includes the following steps:
  • Step 200 establishing a needle tip position calibration model, and calibrating the needle tip position of the puncture needle tube;
  • the method for calibrating the position of the needle tip is the same as that in the first embodiment, which will not be repeated here.
  • Step 210 establishing a needle tip attitude calibration model, and calibrating the needle tip attitude of the puncture needle tube;
  • FIG. 4 is a flowchart of the method for calibrating the attitude of the needle tip of the needle tube according to the second embodiment of the present application, which specifically includes the following steps:
  • Step 211 control the needle tip of the puncture needle to reach the first arbitrary point, define this point as the origin (P o ) of the needle tip coordinate system, and calculate the position R of the point (P o ) in the robot base coordinate system according to the calibration result of the needle tip position P oTip , at the same time record the attitude matrix of the flange at the end of the robotic arm of the dual-arm robot in the robot coordinate system
  • Step 212 control the needle tip of the puncture needle to reach the second arbitrary point, define this point as the point (P y ) on the y-axis of the needle tip coordinate system, and calculate the point (P y ) under the robot base coordinate system according to the calibration result of the needle tip position The position of R P yTip ;
  • Step 213 Establish a vector equation system in the y-axis direction of the needle tip coordinate system, and solve the y-axis direction vector according to the matrix operation operation.
  • the vector equation system is:
  • ny is the y-axis direction vector
  • ⁇ y is the distance between R P oTip and R P yTip ;
  • Step 214 control the needle tip of the puncture needle to reach the third arbitrary point, define the point as a point on the z-axis of the needle-tip coordinate system, and obtain the z-axis direction vector n z ;
  • the calculation method of the z-axis direction vector is the same as that of the y-axis direction vector, which will not be repeated here.
  • Step 215 obtain the x-axis direction vector n x by cross-multiplying the y-axis direction vector and the z-axis direction vector, and then obtain a new z-axis direction vector through the cross-multiplication of the x-axis direction vector and the y-axis direction vector;
  • Step 216 Combine the x-axis direction vector, the y-axis direction vector and the new z-axis direction vector in turn to obtain the attitude matrix R R Tip of the needle tip of the puncture needle:
  • Step 220 Fix the ultrasonic probe and the puncture needle tube on the end effectors of the left and right mechanical arms of the dual-arm robot respectively, and control the ultrasonic probe to scan the needle tip of the puncture needle tube through the dual-arm robot to obtain an ultrasonic image of the needle tip, and record the dual-arm robot respectively.
  • Step 230 Mark the position IP Ntip of the needle tip in the image coordinate system in the needle tip ultrasound image
  • Step 240 Based on the calibration results of the needle tip position and posture of the puncture needle tube, the needle tip position in the needle tip ultrasound image, and the poses of the flanges at the left and right arm ends of the dual-arm robot, establish that the needle tip of the puncture needle tube and the needle tip in the ultrasound image of the needle tip are in the robot base coordinate system
  • the needle tip of the puncture needle tube is established in the robot base coordinate system by combining the needle tip position in the ultrasound image of the needle tip and the positions of the flanges of the left and right arm ends of the dual-arm robot.
  • the registration model whose position is equal to the position of the needle tip in the ultrasound image of the needle tip in the robot base coordinate system, its equation expression is:
  • EL T Ptip is the coordinate transformation matrix from any fixed needle tip on the ultrasound probe to the flange at the end of the robotic arm where the ultrasound probe is installed .
  • T I is the coordinate transformation matrix from the ultrasound image coordinate system to the robot end flange coordinate system.
  • Step 250 Solve the registration model by using the iterative closest point algorithm, obtain a coordinate transformation matrix Ptip T I from the ultrasound image coordinate system to the robot end flange coordinate system, and calibrate the ultrasound probe according to the coordinate transformation matrix;
  • the calibration of the ultrasound probe is completed through steps 220 to 250 .
  • the dual-arm robot calibration method of the second embodiment of the present application further establishes a needle tip attitude calibration model on the basis of the needle tip position calibration, and calibrates the needle tip attitude of the puncture needle tube, and based on the needle tip position and attitude calibration results of the puncture needle tube, Combined with the needle tip position in the needle tip ultrasound image and the pose of the left and right arm end flanges of the dual-arm robot, a registration model is established in which the position of the needle tip of the puncture needle in the robot base coordinate system is equal to the position of the needle tip in the needle tip ultrasound image under the robot base coordinate system. , so as to realize the calibration of the needle tip of the puncture needle and the ultrasonic probe of the dual-arm robot with high precision.
  • the embodiment of the present application does not need to use a third-party locator and model, and realizes the calibration of the needle tip of the puncture needle and the ultrasonic probe by integrating the mechanism of the dual-arm robot and the ultrasonic image, thereby improving the calibration accuracy of the dual-arm robot.
  • an experimental test is carried out by taking the application of the present application in a dual-arm robotic puncture system based on ultrasound guidance as an example. Specifically as shown in Figure 5, where (a) is to control the tip to reach the specified reference point, (b) is to calibrate the position coordinates of the needle tip of the puncture needle in the robot end flange coordinate system, (c) is to calibrate the ultrasonic probe at the end of the robot. The position coordinate calibration in the coordinate system of the flange at the end of the robot, (d) is to control the ultrasonic probe to scan the needle tip of the puncture needle.
  • the experimental results show that the calibration accuracy of the embodiment of the present application can reach 0.6467 ⁇ 0.3099 mm, which realizes the high-precision calibration of the needle tip of the puncture needle tube and the ultrasonic probe.
  • FIG. 6 is a schematic structural diagram of a dual-arm robot calibration system according to an embodiment of the present application.
  • the dual-arm robot calibration system of the embodiment of the present application includes a needle tip calibration module and an ultrasonic probe calibration module, wherein the needle tip calibration module includes:
  • Needle tip position calibration sub-module used to establish a needle tip position calibration model, and calibrate the needle tip position of the puncture needle tube; wherein, the embodiment of the present application utilizes the dual-arm robot itself and the dual-arm high-precision positioning and the dual-arm position coordinates are in the same robot base. The characteristics of the coordinate system are used to establish the needle tip position calibration model, and the needle tip position is calibrated. Specific calibration methods include:
  • ER P Mtip is the coordinates from the top to the end flange, which can be obtained according to the preset top parameters;
  • R T ER is the pose of the end flange in the robot base coordinate system, which can be found in the robot operating system. read directly.
  • inv( ) represents the inversion operation of the matrix.
  • Needle tip attitude calibration sub-module used to establish a needle tip attitude calibration model to calibrate the needle tip attitude of the puncture needle; wherein, the needle tip attitude calibration method includes:
  • the vector equation system is:
  • ny is the y-axis direction vector
  • ⁇ y is the distance between R P oTip and R P yTip ;
  • the calculation method of the z-axis direction vector is the same as that of the y-axis direction vector, which will not be repeated here.
  • the ultrasound probe calibration module includes:
  • Ultrasound image acquisition sub-module used to control the ultrasound probe to scan the needle tip of the puncture needle to obtain the needle tip ultrasound image, mark the position of the needle tip in the image coordinate system IP Ntip in the needle tip ultrasound image, and record the left and right arm end methods of the dual-arm robot respectively.
  • the poses R T EL and R T ER of Lan are respectively fixed on the end effectors of the left and right robotic arms of the dual-arm robot. It will be appreciated that a tracker may also be fixed on the ultrasound probe or puncture needle tube instead of the end flange position.
  • Needle tip position registration sub-module It is used to establish the base coordinates of the needle tip in the puncture needle tube and the needle tip in the ultrasound image of the needle tip on the robot based on the needle tip position calibration result, the needle tip position in the needle tip ultrasound image, and the poses of the left and right arm end flanges of the dual-arm robot The position registration model under the system;
  • the needle tip of the puncture needle tube and the needle tip in the needle tip ultrasound image are established in the robot base coordinate system
  • the registration model with equal lower positions, its equation expression is:
  • the needle tip position registration sub-module is also used to establish the needle tip of the puncture needle tube based on the position and attitude calibration results of the needle tip of the puncture needle tube, the needle tip position in the needle tip ultrasound image, and the positions of the flanges of the left and right arm ends of the dual-arm robot.
  • EL T Ptip is the coordinate transformation matrix from any fixed needle tip on the ultrasound probe to the flange at the end of the robotic arm where the ultrasound probe is installed .
  • T I is the coordinate transformation matrix from the ultrasound image coordinate system to the robot end flange coordinate system.
  • Coordinate calculation sub-module based on formula (3) or (7), the iterative closest point algorithm is used to solve the position registration model, and the coordinate transformation matrix EL T I from the ultrasonic image coordinate system to the robot end flange coordinate system is obtained. or Ptip T I , the ultrasound probe is calibrated according to the coordinate transformation matrix.
  • FIG. 7 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • the terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51 .
  • the memory 52 stores program instructions for implementing the above-mentioned dual-arm robot calibration method.
  • the processor 51 is configured to execute program instructions stored in the memory 52 to control the calibration of the dual-arm robot.
  • the processor 51 may also be referred to as a CPU (Central Processing Unit, central processing unit).
  • the processor 51 may be an integrated circuit chip with signal processing capability.
  • the processor 51 may also be a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component .
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • FIG. 8 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • the storage medium of this embodiment of the present application stores a program file 61 capable of implementing all the above methods, wherein the program file 61 may be stored in the above-mentioned storage medium in the form of a software product, and includes several instructions to make a computer device (which may It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the methods of the various embodiments of the present application.
  • a computer device which may It is a personal computer, a server, or a network device, etc.
  • processor processor
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes , or terminal devices such as computers, servers, mobile phones, and tablets.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Robotics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Pathology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

Procédé d'étalonnage de robot à deux bras, système, terminal (50) et support de stockage. Le procédé consiste : à établir un modèle d'étalonnage de position de pointe d'aiguille, et à étalonner la position de pointe d'aiguille d'un tube d'aiguille de ponction en fonction du modèle ; à acquérir une image ultrasonore de pointe d'aiguille du tube d'aiguille de ponction au moyen d'une sonde à ultrasons, la sonde à ultrasons et le tube d'aiguille de ponction étant respectivement fixés sur des effecteurs terminaux des bras mécaniques gauche et droit d'un robot à deux bras ; à établir, en fonction de la position de la pointe d'aiguille et de l'image ultrasonore de pointe d'aiguille, un modèle d'enregistrement de position de la pointe d'aiguille du tube d'aiguille de ponction et de la pointe d'aiguille dans la pointe d'aiguille de l'image ultrasonore dans un système de coordonnées basé sur un robot ; et à résoudre le modèle d'enregistrement de position au moyen d'un algorithme itératif de point le plus proche, à obtenir une matrice de transformation de coordonnées à partir d'un système de coordonnées d'image ultrasonore vers un système de coordonnées de bride d'extrémité de robot, et à étalonner la sonde à ultrasons en fonction de la matrice de transformation de coordonnées. Il n'est pas nécessaire d'utiliser un localisateur et un modèle tiers, ce qui améliore la précision d'étalonnage du robot à deux bras.
PCT/CN2020/139303 2020-08-12 2020-12-25 Procédé d'étalonnage de robot à deux bras, système, terminal et support de stockage WO2022032964A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010806742.9A CN111956329B (zh) 2020-08-12 2020-08-12 一种双臂机器人标定方法、系统、终端以及存储介质
CN202010806742.9 2020-08-12

Publications (1)

Publication Number Publication Date
WO2022032964A1 true WO2022032964A1 (fr) 2022-02-17

Family

ID=73365667

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/139303 WO2022032964A1 (fr) 2020-08-12 2020-12-25 Procédé d'étalonnage de robot à deux bras, système, terminal et support de stockage

Country Status (2)

Country Link
CN (1) CN111956329B (fr)
WO (1) WO2022032964A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114905548A (zh) * 2022-06-29 2022-08-16 武汉库柏特科技有限公司 一种双臂机器人基坐标系标定方法及其装置
CN115127452A (zh) * 2022-09-02 2022-09-30 苏州鼎纳自动化技术有限公司 一种笔记本电脑外壳尺寸检测方法、系统和存储介质
CN116898548A (zh) * 2023-09-13 2023-10-20 真健康（北京）医疗科技有限公司 穿刺手术机器人末端移动位置的确定方法及设备
CN117103286A (zh) * 2023-10-25 2023-11-24 杭州汇萃智能科技有限公司 一种机械手手眼标定方法、系统和可读存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111956329B (zh) * 2020-08-12 2022-04-26 中国科学院深圳先进技术研究院 一种双臂机器人标定方法、系统、终端以及存储介质
CN112828862B (zh) * 2020-12-30 2022-09-16 诺创智能医疗科技(杭州)有限公司 并联平台的主从映射方法、机械臂系统和存储介质
CN112754616B (zh) * 2020-12-30 2022-05-10 诺创智能医疗科技(杭州)有限公司 超声定位穿刺系统和存储介质
WO2022198615A1 (fr) * 2021-03-26 2022-09-29 中国科学院深圳先进技术研究院 Procédé et système d'étalonnage pour système de ponction de robot à deux bras
CN113133832B (zh) * 2021-03-26 2022-09-20 中国科学院深圳先进技术研究院 一种双臂机器人穿刺系统标定方法及系统
CN113160221B (zh) * 2021-05-14 2022-06-28 深圳市奥昇医疗科技有限责任公司 图像处理方法、装置、计算机设备和存储介质
CN113413216B (zh) * 2021-07-30 2022-06-07 武汉大学 一种基于超声影像导航的双臂穿刺机器人
CN115153782A (zh) * 2022-08-12 2022-10-11 哈尔滨理工大学 一种超声引导下的穿刺机器人空间注册方法
CN115363716A (zh) * 2022-09-01 2022-11-22 浙江深博医疗技术有限公司 一种穿刺手术辅助方法、系统、机器人和及存储介质
CN115488884B (zh) * 2022-09-07 2024-04-30 华南理工大学 远程机器人的超声探头的标定方法、系统、装置及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013140315A1 (fr) * 2012-03-23 2013-09-26 Koninklijke Philips N.V. Calibrage d'ultrason localisé d'intervention chirurgicale
CN103584885A (zh) * 2013-11-20 2014-02-19 哈尔滨工程大学 一种基于定位导航穿刺针的自由臂超声标定方法
CN104161546A (zh) * 2014-09-05 2014-11-26 深圳先进技术研究院 基于可定位穿刺针的超声探头标定系统及方法
CN104207801A (zh) * 2013-06-05 2014-12-17 上海工程技术大学 一种超声检测图像三维标定方法
CN110554095A (zh) * 2019-08-16 2019-12-10 上海工程技术大学 一种三维超声探头标定装置及标定三维超声探头的方法
CN111956329A (zh) * 2020-08-12 2020-11-20 中国科学院深圳先进技术研究院 一种双臂机器人标定方法、系统、终端以及存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1263433C (zh) * 2003-08-15 2006-07-12 北京航空航天大学 机器人导航骨科手术装置
CN105588525B (zh) * 2014-11-14 2019-09-20 北京配天技术有限公司 一种工具在机器人法兰坐标系上的标定方法及装置
CN105157725B (zh) * 2015-07-29 2018-06-29 华南理工大学 一种二维激光视觉传感器和机器人的手眼标定方法
EP4375934A2 (fr) * 2016-02-12 2024-05-29 Intuitive Surgical Operations, Inc. Systèmes et procédés d'estimation de pose et d'étalonnage de système d'imagerie en perspective dans une chirurgie guidée par image
CN110709024A (zh) * 2017-11-21 2020-01-17 直观外科手术操作公司 用于直观运动的主/工具配准和控制的系统和方法
CN110051436B (zh) * 2018-01-18 2020-04-17 上海舍成医疗器械有限公司 自动化协同工作组件及其在手术器械中的应用
KR102577448B1 (ko) * 2019-01-22 2023-09-12 삼성전자 주식회사 핸드 아이 캘리브레이션 방법 및 시스템
CN111055289B (zh) * 2020-01-21 2021-09-28 达闼科技（北京）有限公司 机器人的手眼标定方法、装置、机器人及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013140315A1 (fr) * 2012-03-23 2013-09-26 Koninklijke Philips N.V. Calibrage d'ultrason localisé d'intervention chirurgicale
CN104207801A (zh) * 2013-06-05 2014-12-17 上海工程技术大学 一种超声检测图像三维标定方法
CN103584885A (zh) * 2013-11-20 2014-02-19 哈尔滨工程大学 一种基于定位导航穿刺针的自由臂超声标定方法
CN104161546A (zh) * 2014-09-05 2014-11-26 深圳先进技术研究院 基于可定位穿刺针的超声探头标定系统及方法
CN110554095A (zh) * 2019-08-16 2019-12-10 上海工程技术大学 一种三维超声探头标定装置及标定三维超声探头的方法
CN111956329A (zh) * 2020-08-12 2020-11-20 中国科学院深圳先进技术研究院 一种双臂机器人标定方法、系统、终端以及存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114905548A (zh) * 2022-06-29 2022-08-16 武汉库柏特科技有限公司 一种双臂机器人基坐标系标定方法及其装置
CN115127452A (zh) * 2022-09-02 2022-09-30 苏州鼎纳自动化技术有限公司 一种笔记本电脑外壳尺寸检测方法、系统和存储介质
CN115127452B (zh) * 2022-09-02 2022-12-09 苏州鼎纳自动化技术有限公司 一种笔记本电脑外壳尺寸检测方法、系统和存储介质
CN116898548A (zh) * 2023-09-13 2023-10-20 真健康（北京）医疗科技有限公司 穿刺手术机器人末端移动位置的确定方法及设备
CN116898548B (zh) * 2023-09-13 2023-12-12 真健康（北京）医疗科技有限公司 穿刺手术机器人末端移动位置的确定方法及设备
CN117103286A (zh) * 2023-10-25 2023-11-24 杭州汇萃智能科技有限公司 一种机械手手眼标定方法、系统和可读存储介质
CN117103286B (zh) * 2023-10-25 2024-03-19 杭州汇萃智能科技有限公司 一种机械手手眼标定方法、系统和可读存储介质

Also Published As

Publication number Publication date
CN111956329A (zh) 2020-11-20
CN111956329B (zh) 2022-04-26

Similar Documents

Publication Publication Date Title
WO2022032964A1 (fr) Procédé d'étalonnage de robot à deux bras, système, terminal et support de stockage
TWI672206B (zh) 機械手臂非接觸式工具中心點校正裝置及其方法以及具有校正功能的機械手臂系統
WO2023082990A1 (fr) Procédé et appareil pour déterminer la pose de travail d'un bras robotique
CN110355754B (zh) 机器人手眼系统、控制方法、设备及存储介质
CN113133832B (zh) 一种双臂机器人穿刺系统标定方法及系统
JP6964989B2 (ja) 制御方法、ロボットシステム、物品の製造方法、プログラム、及び記録媒体
JP7035657B2 (ja) ロボットの制御装置、ロボット、ロボットシステム、並びに、カメラの校正方法
WO2020173240A1 (fr) Procédé et appareil d'étalonnage d'appareil d'acquisition d'images, dispositif informatique et support de stockage
WO2022141153A1 (fr) Système de ponction à positionnement par ultrasons et support de stockage
JPWO2018043525A1 (ja) ロボットシステム、ロボットシステム制御装置、およびロボットシステム制御方法
Pachtrachai et al. Hand-eye calibration with a remote centre of motion
CN112621743B (zh) 机器人及其相机固定于末端的手眼标定方法及存储介质
JP2018134695A (ja) ロボットシステム、ロボット制御装置、及びロボット制御方法
WO2018209592A1 (fr) Procédé de commande de mouvement pour robot, robot et dispositif de commande
WO2022199296A1 (fr) Procédé et appareil d'élimination d'erreur pour un robot de navigation chirurgicale et dispositif électronique
JPWO2018043524A1 (ja) ロボットシステム、ロボットシステム制御装置、およびロボットシステム制御方法
CN112754616A (zh) 超声定位穿刺系统和存储介质
TW201313415A (zh) 機械手臂的座標校正系統及方法
JP2009222568A (ja) ３次元形状データの生成方法および装置ならびにコンピュータプログラム
CN117064555A (zh) 手术机器人的摆位方法、装置、摆位系统和计算机设备
WO2022198615A1 (fr) Procédé et système d'étalonnage pour système de ponction de robot à deux bras
CN113787518A (zh) 一种机器人末端姿态控制方法、装置、设备及存储介质
CN116394254A (zh) 机器人的零点标定方法、装置、计算机存储介质
TWI799310B (zh) 機器人及機器人手眼校正方法
CN112743546B (zh) 机器人手眼标定位姿选择方法、装置、机器人系统及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20949455

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20949455

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 04.07.2023)