CN116392246A - Method and system for registering surgical robot coordinate system and CT machine coordinate system - Google Patents
Method and system for registering surgical robot coordinate system and CT machine coordinate system Download PDFInfo
- Publication number
- CN116392246A CN116392246A CN202310365385.0A CN202310365385A CN116392246A CN 116392246 A CN116392246 A CN 116392246A CN 202310365385 A CN202310365385 A CN 202310365385A CN 116392246 A CN116392246 A CN 116392246A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- mechanical arm
- machine
- calibration plate
- calibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000006243 chemical reaction Methods 0.000 claims description 65
- 239000011159 matrix material Substances 0.000 claims description 57
- 230000009466 transformation Effects 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 235000005811 Viola adunca Nutrition 0.000 claims description 6
- 240000009038 Viola odorata Species 0.000 claims description 6
- 235000013487 Viola odorata Nutrition 0.000 claims description 6
- 235000002254 Viola papilionacea Nutrition 0.000 claims description 6
- 230000035515 penetration Effects 0.000 claims 1
- 238000012549 training Methods 0.000 abstract description 4
- 238000003384 imaging method Methods 0.000 abstract 1
- 239000000758 substrate Substances 0.000 description 12
- 239000003550 marker Substances 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 5
- 239000000919 ceramic Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 4
- 229930040373 Paraformaldehyde Natural products 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- -1 polyoxymethylene Polymers 0.000 description 3
- 229920006324 polyoxymethylene Polymers 0.000 description 3
- 229920001169 thermoplastic Polymers 0.000 description 3
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000002861 polymer material Substances 0.000 description 2
- 239000004416 thermosoftening plastic Substances 0.000 description 2
- 102100024522 Bladder cancer-associated protein Human genes 0.000 description 1
- 101150110835 Blcap gene Proteins 0.000 description 1
- 229920001875 Ebonite Polymers 0.000 description 1
- 101100271175 Oryza sativa subsp. japonica AT10 gene Proteins 0.000 description 1
- 101100493740 Oryza sativa subsp. japonica BC10 gene Proteins 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 229920000642 polymer Polymers 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
- A61B17/34—Trocars; Puncturing needles
- A61B17/3403—Needle locating or guiding means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/107—Visualisation of planned trajectories or target regions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B2034/305—Details of wrist mechanisms at distal ends of robotic arms
Abstract
The invention discloses a method and a system for registering a surgical robot coordinate system and a CT machine coordinate system. The three-dimensional imaging device comprises a 3D structure light camera and a calibration plate, wherein the 3D structure light camera and the calibration plate are matched with a surgical robot console, a mechanical arm and a CT machine, a 3D vision system is used for registering a CT machine coordinate system with a mechanical arm coordinate system in the surgical robot, and the 3D structure light camera in the 3D vision system synchronously moves along with the mechanical arm and a puncture device fixedly arranged at the tail end of the mechanical arm. The coordinate registration process in the robot operation process is greatly simplified, the operation of the user is simple, and the learning and training time of medical staff is greatly shortened. And the 3D structure light camera and the puncture device are arranged at the tail end of the mechanical arm, so that the equipment volume of the operation robot is reduced, the equipment occupation space of a CT operating room is greatly reduced, and the operation robot can be developed in more hospitals.
Description
Technical Field
The invention relates to the field of puncture robots, in particular to a method and a system for registering a coordinate system of a surgical robot and a coordinate system of a CT machine.
Background
In the process of performing a puncture operation by using a robot, a doctor diagnoses a patient according to medical imaging equipment (for example, a CT machine) and makes an operation scheme, and then the operation robot performs the puncture operation according to the operation scheme given by the doctor. Only after the coordinate system of the surgical robot is registered with the coordinate system of the medical imaging device, the surgical robot can complete the surgical action according to the surgical scheme formulated by a doctor.
At present, most surgical robots adopt infrared binocular cameras to shoot a mark component fixed on a mechanical arm and a mark component fixed on a human body, and then coordinate registration of the mechanical arm and the CT machine is determined through a pattern recognition algorithm and a space geometric algorithm by combining CT images of the mark component fixed on the human body. The technology and the method need to fixedly install the marker component on the patient and the mechanical arm, and a large infrared binocular camera is arranged at a proper position outside the mechanical arm in a visual field, so that the whole system is complex in equipment, inconvenient to operate and can be operated only after long-time training of medical staff. Particularly, the mark component is fixed on the patient, and the mark component can be completed only by effective coordination of the patient, so that the wide popularization and application of the surgical robot are limited.
Disclosure of Invention
The invention aims to provide a method for registering a coordinate system of a surgical robot and a coordinate system of a CT machine, which belongs to a brand new registering method, and the method only needs to fixedly install a high-precision and small-size 3D structure light camera at the tail end of a mechanical arm, is matched with a calibration plate specially designed according to the 3D structure light camera, can finish registering the coordinate of the mechanical arm and the coordinate of the CT machine by placing the calibration plate on the CT machine, does not need to fix corresponding markers on the mechanical arm and a patient, does not need to be additionally provided with a large infrared binocular camera, so that surgical robot equipment is simplified, is simpler and more convenient to use and operate, can be widely popularized and applied, and is suitable for medical units of various levels.
The method for registering the coordinate system of the surgical robot and the coordinate system of the CT machine uses a 3D vision system to register the coordinate system of the CT machine and the coordinate system of a mechanical arm in the surgical robot, and a 3D structured light camera in the 3D vision system synchronously moves along with the mechanical arm and a puncture device fixedly arranged at the tail end of the mechanical arm.
Preferably, the 3D vision system comprises a 3D structured light camera mounted at the end of the mechanical arm and fixed with the puncture device into a whole, and a calibration plate capable of being directly placed on the CT machine and located in the shooting range of the 3D structured light camera.
Preferably, the method for registering the upper CT machine coordinate system and the mechanical arm coordinate system is characterized by comprising the following steps:
1) Moving a mechanical arm fixed with a 3D structure light camera into an operation area beside a CT machine;
2) Placing the calibration plate on the CT machine and being positioned in a range which can be shot by the 3D structured light camera;
3) Starting a registration program, wherein the mechanical arm carries the 3D structure light camera to sequentially move to a plurality of pose, after moving to each pose, starting the 3D structure camera to project a plurality of groups of structure light, collecting the pictures of the marked structures in the marking plates projected by the structure light, recording mechanical arm transformation parameters of each pose and the pictures of the marked structures in the marking plates through computer software, and obtaining the conversion relation between the coordinate system of the marking plates and the coordinate system of the mechanical arm by using a hand-eye calibration algorithm to obtain a third conversion matrix;
4) The mechanical arm is retracted to an initial position, the calibration plate is continuously placed on the CT machine and kept still, and the CT machine is started to scan the calibration plate; acquiring CT image data of four spherical calibration pieces on the calibration plate, and combining the spherical centers of the four spherical calibration pieces with a coordinate matrix of the calibration plate to obtain a conversion relation between a CT coordinate system and the coordinate system of the calibration plate, wherein the conversion relation is a fourth conversion matrix;
5) And obtaining the conversion relation between the CT machine coordinate system and the mechanical arm coordinate system as a fifth conversion matrix according to a third conversion matrix between the calibration plate coordinate system and the mechanical arm coordinate system and a fourth conversion matrix between the CT machine coordinate system and the calibration plate coordinate system, and completing coordinate registration of the CT machine coordinate system and the mechanical arm coordinate system.
Preferably, in the step 3), the following steps are performed:
3.1 A registration procedure is started, the mechanical arm carries the 3D structure light camera to sequentially move to n preset different poses, after moving to each pose, the conversion relation between the calibration plate coordinate system and the camera coordinate system under each pose is respectively obtained, and the conversion relation is a first conversion matrix W BC The method comprises the steps of carrying out a first treatment on the surface of the The conversion relation between the center point of the tail end of the mechanical arm and the coordinate system of the mechanical arm is a second conversion matrix W AT ；
3.2 Selecting a first transformation matrix W corresponding to two poses (i, j) BC And a second conversion matrix W AT The transformation matrix W of the camera coordinate system and the center point coordinate system of the tail end of the mechanical arm is obtained according to the following equation (1) CT (ij) ；
W AT (i) W CT (ij) W BC (i) ＝W AT (j) W CT (ij) W BC (j) (1)
3.3 Obtaining a transformation matrix W of the calibration plate coordinate system and the mechanical arm coordinate system for the two poses (i, j) according to the following equation (2) AB (ij) ；
W AB (ij) ＝W AT (i) W CT (ij) W BC (i) ＝W AT (j) W CT (ij) W BC (j) (2)
3.4 Repeating the steps 3.2) and 3.3), and selecting a first conversion matrix W of two different poses from the n poses BC And a second conversion matrix W AT N-1/2 of said conversion matrices W can be obtained after combination AB (ij) The method comprises the steps of carrying out a first treatment on the surface of the For n (n-1)/2 of said conversion matrices W AB (ij) Corresponding identical variable of (a) is flattenedAverage values of all the same variables are combined into a conversion relation between a calibration plate coordinate system and a mechanical arm coordinate system, and a third conversion matrix is called
Preferably, in the step 3.1), the following steps are performed after the mechanical arm moves to each pose:
3.1.1 The 3D structured light camera projects structured light to the calibration plate, a photo of a mark structure in the calibration plate, which is projected by the structured light, is collected, pose transformation parameters of the mechanical arm and the photo of the mark structure in the calibration plate, which is projected by the structured light, are recorded by computer software, a conversion relation between a coordinate system of the calibration plate for the pose and a coordinate system of a center point of the 3D structured light camera is obtained through a hand-eye calibration algorithm, and the conversion relation is a first conversion matrix W BC ；
3.1.2 According to the pose transformation parameters of the mechanical arm under each pose, acquiring the pose representation of the central point of the tail end of the mechanical arm under the mechanical arm coordinate system, namely, the conversion relation between the coordinate system of the central point of the tail end of the mechanical arm of the pose and the mechanical arm coordinate system, and obtaining a second conversion matrix W AT 。
Preferably, in the step 3), the number of pose positions of the 3D structured light camera moving along with the mechanical arm is 4-25.
Preferably, in step 3.1.1), the 3D structured light camera projects blue-violet LED light with a wavelength of 490nm, and the blue-violet LED light is in a stripe grating of 13-24 strips to the calibration plate.
Preferably, the calibration plate comprises a substrate, four spherical calibration pieces, and columnar connecting pieces, wherein the four spherical calibration pieces are used for fixedly connecting the substrate and the spherical calibration pieces, are used for locking the columnar connecting pieces and locking pieces of the substrate, the heights of the columnar connecting pieces are distributed in equal differential columns, all the columnar connecting pieces are fixedly arranged on the same side surface of the substrate, and a checkerboard is arranged on the side surface of the substrate, which is arranged on the columnar connecting pieces.
Preferably, the columnar connecting piece comprises a first connecting part connected with the spherical calibration piece and a second connecting part connected with the substrate, the first connecting part is provided with a cylindrical groove for the spherical calibration piece to sink into, the depth of the groove is larger than the radius of the spherical calibration piece, and the inner diameter of the groove is smaller than or equal to the diameter of the spherical calibration piece.
The system for registering the coordinate system of the surgical robot and the coordinate system of the CT machine is matched with a surgical robot console, a mechanical arm and the 3D structure light camera and a calibration plate which are used together, wherein the 3D structure light camera is fixedly arranged together with a puncture device arranged at the tail end of the mechanical arm and moves together with the puncture device along with the movement of the mechanical arm.
When the system is used for alignment, the operation robot and the mechanical arm with the 3D structure optical camera are pushed to the side of the CT machine, the calibration plate is placed in the range of the CT machine tool which can be shot by the 3D structure optical camera, and the coordinate alignment can be automatically completed after a program is started, so that the system is simple and convenient to operate and easy to learn and use. Thereby greatly simplifying the operation difficulty of the operation robot and greatly reducing the learning and training time of medical staff.
According to the invention, after the 3D structure light camera is applied to the surgical robot, the registration of the coordinate system of the surgical robot and the coordinate system of the CT machine can be rapidly and accurately realized, the registration precision can be within 1mm, the accurate positioning and identification are provided for the surgical robot to perform the puncture operation, and the operation precision is ensured.
In addition, the coordinate registration process in the robot operation process is greatly simplified through the use of the 3D structured light camera, the coordinate registration process is irrelevant to a patient, the operation of a user is simple, and the learning and training time of medical staff is greatly shortened. And the 3D structure light camera and the puncture device are arranged at the tail end of the mechanical arm, so that the equipment volume of the operation robot is reduced, the equipment occupation space of a CT operating room is greatly reduced, the conventional CT examination room can be fully used, and the operation of the robot can be carried out in more hospitals.
Drawings
Fig. 1 is a flow chart of a method for registering a surgical robot coordinate system with a CT machine coordinate system in the present invention.
Fig. 2 is a schematic diagram of the system for registering the surgical robot coordinate system with the CT machine coordinate system in the present invention.
Fig. 3 is a schematic structural view of the 3D structured light camera and the puncture device of the present invention after being assembled.
FIG. 4 is a schematic perspective view of the calibration plate according to the present invention.
FIG. 5 is a schematic diagram of the structure of a checkerboard in a calibration plate substrate according to the present invention.
Fig. 6 is a schematic cross-sectional view of four columnar connectors according to the present invention.
Fig. 7 is a schematic diagram of the working state structure of the system in the present invention.
Fig. 8 is a schematic diagram of the operation of registration in the present invention.
Fig. 9 is a second schematic diagram of the operation of registration in the present invention.
Description of the embodiments
The objects, technical solutions and advantages of the present invention will become more apparent by the following detailed description of the present invention with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention.
As shown in fig. 1, 2, 3, 8 and 9, the method for registering the coordinate system of the surgical robot and the coordinate system of the CT machine in the present invention is to register the coordinate system of the CT machine and the coordinate system of the mechanical arm in the surgical robot by using a 3D vision system, wherein a 3D structured light camera in the 3D vision system moves synchronously with the mechanical arm 4 and the puncture device 3 mounted at the end of the mechanical arm.
The 3D vision system for registering the surgical robot coordinate system with the CT machine coordinate system comprises a software part and a hardware part, wherein the software part is combined with the control system of the surgical robot, and the specific combination and software program can be realized as a person skilled in the art after having described the method of the invention in detail. The hardware part comprises a 3D structure light camera 1 arranged on a puncture device 3 at the tail end of a mechanical arm 4 of the surgical robot and a calibration plate 2 which can be stably placed on a CT machine. The 3D structured light camera 1 and the puncture device 3 are fixedly installed together through the mounting frame 32, and are both installed at the tail end of the mechanical arm 4, and synchronously move with the puncture device 3 along with the movement of the mechanical arm 4. The specific construction of the 3D structured light camera 1, the fixed mounting of the puncturing device 3 and the fixed mounting 32 is readily achievable by a person skilled in the art and will not be described in detail here.
As shown in fig. 3, the 3D structured light camera 1 in the present invention may be split into two parts, including a structured light source 30 and a DDP camera 31, which are respectively installed on two sides of the puncture device 3, so that the puncture needle 5 in the puncture device 3 is located between the structured light source 30 and the DDP camera 31, and after the structured light source 30 and the DDP camera 31 are fixedly installed, the light source coverage of the structured light source 30 and the shooting range of the camera 31 are located under the puncture device 3 in a manner of just overlapping, so as to better find the puncture position. As shown in fig. 7 and 9, the 3D structured light camera 1 of the present invention may be directly and fixedly mounted on the distal end of the robot arm 4 as a whole, and may be disposed parallel (or juxtaposed) to the puncture device 3.
As shown in fig. 4, the calibration plate 2 includes a base plate 20, four spherical calibration members 22, four column-shaped connectors 21 for fixedly connecting the base plate 20 and the spherical calibration members 22, and four locking members 23 for locking the column-shaped connectors 21 and the base plate 20.
Wherein the substrate 20 is a square ceramic substrate with a length of 160mm, a width of 140mm and a thickness of 5mm, each of four right-angle positions of the ceramic substrate is provided with a through hole 25, and marking structures 24 are uniformly distributed on one side surface of the ceramic substrate 20 and positioned in the middle of the four through holes 25. The logo structures 24 are geometric figures drawn or printed on the surface of the ceramic substrate 20, preferably in alternating black and white squares of the invention, as shown in fig. 5, to form a checkerboard pattern with sides of 8mm.
The columnar connectors 21 are preferably integrally formed of polyoxymethylene thermoplastic Polymer (POM) material, and a spherical calibration member 22 is respectively fixed above each columnar connector 21, wherein the spherical calibration member 22 is selected from materials which can be clearly imaged in a CT machine without generating artifacts, and preferably is AL with a diameter of 20mm 2 O 3 Ceramic balls.
The columnar connecting piece 21 and the spherical calibration piece 22 are made of materials, the density of the spherical calibration piece 22 is 1.5-4 times greater than that of the columnar connecting piece 21, and the spherical connecting piece can be accurately and clearly identified from a CT image without generating artifacts in the CT image.
As shown in fig. 6, the columnar connector 21 includes a first connecting portion 26 connected to the ball marker 22, a second connecting portion 27 connected to the base plate 20, and a connecting post 28 for connecting the first connecting portion 26 and the second connecting portion 27. The first connection portion 26, the second connection portion 27, and the connection post 28 are integrally formed of a polyoxymethylene thermoplastic crystalline polymer material. The heights of the four columnar connecting pieces 21 are different, the structures and the shapes of the first connecting part 26 and the second connecting part 27 of each columnar connecting piece 21 are the same, and the sizes are the same. The heights of the connecting posts 28 for connecting the first connecting portion 26 and the second connecting portion 27 are different, and the four connecting posts 28 are arranged in an equal-difference array, preferably 0mm, 20mm, 40mm, and 60mm.
The first connecting portion 26 is provided with a cylindrical recess 29 into which the ball marker 22 is sunk, the depth of the recess 29 is larger than the radius of the ball marker 22 but smaller than 2/3 of the diameter of the ball marker 22, the depth of the recess 29 is preferably 21mm, the diameter is preferably 19.9-20mm, the ball marker 22 is in a slightly clamped state when being sunk into the recess 29 by means of the characteristics of polyoxymethylene thermoplastic crystalline polymer material, and only a small amount of force is required to enter the recess 29, so that the center of the ball marker 22 can be completely located at the very center of the recess 29, and any movement such as rotation or the like can not occur after the ball marker 22 enters the recess 29, so that the center of the ball marker 22 is always located at the very center of the recess 29. In order to allow the ball marker 22 to enter the recess 29 smoothly, a vent 19 is provided in the inner bottom of the recess 29.
Preferably, the four locking pieces 23 are hard rubber supporting columns, so that the calibration plate 2 can be stably and non-slidably placed on the CT machine.
The dimensions of the checkerboard, the columnar connecting pieces 21 and the spherical calibration pieces 22 in the calibration plate 2 and the positions of the spherical calibration pieces 22 in the calibration plate 2 are precisely controlled, so that the spherical calibration pieces 22 can be clearly imaged in a CT machine without generating artifacts, and meanwhile, the spherical calibration pieces 22 can be stably fixed in the calibration plate, and the coordinates of the center point of the spherical calibration pieces 22 in the coordinate system of the calibration plate are precise.
As shown in fig. 1, 7, 8 and 9, the method for registering a surgical robot coordinate system and a CT machine coordinate system according to the present invention is used for registering an upper CT machine coordinate system and a robot arm coordinate system, and is performed according to the following steps:
1) The robot arm 4 to which the 3D structured light camera 1 is fixed is moved to a predetermined operation area beside the CT machine 6, and in this step, the robot arm 4 may be started in advance and the robot arm 4 may be moved to a predetermined position.
2) The calibration plate 2 is placed on a CT bed of the CT machine 6 and within the range that can be photographed by the 3D structured light camera 1.
3) Starting a registration program, synchronously moving the mechanical arm 4 carrying the 3D structured light camera 1 to a preset (preset in a surgical robot program) first pose, and executing the following steps:
3.1 Turning on a structural light source 30 in the 3D structural light camera 1, projecting blue-violet LED light with the wavelength of 490nm to the calibration plate 2, refracting the blue-violet LED light to the DDP camera 31 through 20 stripe gratings on the mark structure 24 of the calibration plate 2, converting gray codes of the stripe gratings into real-time data, performing a semi-precision matching algorithm through software, forming a point cloud image through a patch algorithm, calculating the content distance (distance from the corner point to the corner point of the chessboard) of the mark structure 24 of the calibration plate 2, and performing deviation correction calculation according to the true value of the mark structure 24 in the calibration plate 2 to obtain the conversion relation W between the coordinate system of the calibration plate and the coordinate system of the camera under the first pose BC1 ，
The conversion relation between the coordinate system of the calibration plate and the coordinate system of the camera is called as a first conversion matrix W BC 。
The specific algorithm involved in this step is a hand-eye calibration algorithm, which belongs to the known technology, and the specific principle and algorithm will not be described in detail.
In the step, the projection wavelength of the blue-violet LED light can be between 450nm and 550nm, and 13-24 stripe gratings can be arranged on the calibration plate 2 according to different wavelengths, and the embodiment is preferably with the wavelength of 490nm, and 20 stripe gratings are arranged on the calibration plate 2, so that the rapid calculation of computer software is facilitated.
3.2 According to the pose transformation parameters (coordinates x, y, z and angle theta) of the mechanical arm under each pose x 、θ y 、θ z ) Acquiring pose representation of the central point of the tail end of the first pose mechanical arm under the mechanical arm coordinate system, namely, the conversion relation between the mechanical arm central point coordinate system of the first pose mechanical arm and the mechanical arm coordinate system is a second conversion matrix W AT1 。
Specifically, the matrix is converted into a 4 multiplied by 4 homogeneous transformation matrix W through a rotation matrix and a translation matrix of the transformation matrix AT ：
4) Starting the mechanical arm 4 to move to the second pose, repeating the step 3), and respectively obtaining a first conversion matrix W of the calibration plate coordinate system and the camera coordinate system under the second pose BC2 Second transformation matrix W of center point of tail end of mechanical arm 4 and mechanical arm coordinate system AT2 ；
Repeating the steps, sequentially moving the mechanical arm 4 to 10 preset different poses to obtain a first transformation matrix W under each pose BCi Is W BC1 ，W BC2 ，W BC3 ，……W BC10 The method comprises the steps of carrying out a first treatment on the surface of the Second conversion matrix W ATi The method comprises the following steps: w (W) AT1 ，W AT2 ，W AT3 ，……W AT10 。
5) Obtaining a conversion matrix W between a camera coordinate system and a center point coordinate system at the tail end of the mechanical arm CT 。
The method specifically comprises the following steps: selecting a first transformation matrix W of any two poses (i, j) from the 10 poses of the step 4) BCi ，W BCj And a second conversion matrix W ATi ，W ATj Root of Chinese characterThe conversion relation between the camera coordinate system and the center point coordinate system of the tail end of the mechanical arm, namely the conversion matrix W, can be obtained according to the following equation (1) CT (ij) 。
W AT (i) W CT (ij) W BC (i) ＝W AT (j) W CT (ij) W BC (j) (1)
6) Obtaining a conversion matrix W between a calibration plate coordinate system and a mechanical arm coordinate system AB (ij) 。
Obtaining the conversion relation between the coordinate system of the calibration plate and the coordinate system of the mechanical arm for the two poses (i, j) according to the following equation (2), namely, the conversion matrix is W AB (ij) 。
W AB (ij) ＝W AT (i) W CT (ij) W BC (i) ＝W AT (j) W CT (ij) W BC (j) (2)
7) Repeating the steps 5) and 6), and selecting a first conversion matrix W of two different poses from 10 poses BC And a second conversion matrix W AT After two-by-two combination, 10 x 9/2=45 conversion matrices W can be obtained AB (ij) . For 45 conversion matrices W AB (ij) The corresponding identical variable in the calibration plate coordinate system is calculated to obtain an average value, the average values of all the identical variables are combined into a conversion relation between the calibration plate coordinate system and the mechanical arm coordinate system, and a third conversion matrix is called
It should be noted that 10 poses are preset in the embodiment, and the optimal number of poses capable of ensuring measurement accuracy and conversion matrix calculation accuracy is selected by combining the structural size of the calibration plate 2 and the characteristic parameters of the 3D structured light camera 1, and the number of poses can be between 4 and 20, which is determined according to the structural size of the calibration plate 2 and the characteristic parameters of the 3D structured light camera 1.
The same variable refers to a specific variable at each same position in the transformation matrix, e.g. u ABx ，u ABy ，u ABz Etc.
8) The mechanical arm 4 is controlled to retract, the calibration plate 2 is kept motionless on the CT machine 6, and the CT machine 6 is started to carry out close scanning on the calibration plate 2.
CT influence data of the four spherical calibration pieces 22 on the calibration plate 2 are obtained, the influence data comprise CT values and three-dimensional CT coordinates of all pixels in an image, the spherical center coordinates of the four spherical calibration pieces 22 under a CT coordinate system are calculated through an image recognition algorithm and a spherical center algorithm based on the CT values, the spherical center coordinates are represented by a coordinate matrix B,
coordinate matrix of sphere centers of four spherical calibration pieces 22 on the calibration plate 2>
The following equation can be obtained:
the coordinate matrix B is obtained, the above equation can be solved, and the conversion relation from the CT coordinate system to the coordinate system of the calibration plate can be obtained through calculation, namely, a fourth conversion matrix
9) Obtaining a third transformation matrix W of the calibration plate coordinate system and the mechanical arm coordinate system AB Fourth transformation matrix W from CT coordinate system to calibration plate coordinate system BD Then, the conversion relation between the CT machine coordinate system and the mechanical arm coordinate system can be obtained according to the following calculation formula (3), and the fifth conversion matrix W is called AD 。
Obtaining a fifth transformation matrix W of the CT coordinate system and the mechanical arm coordinate system AD After that, the CT machine coordinate system is completedCoordinate registration with the mechanical arm coordinate system is completed, so that coordinate registration of the CT machine coordinate system and the surgical robot coordinate system is completed.
In summary, the invention fixes a 3D structure optical camera at the tail end of the mechanical arm, adopts a calibration method of hands on eyes, does not need to adopt a binocular camera additionally arranged, can effectively reduce the influence of various background light sources in the CT chamber on measurement, and can achieve high measurement precision. The theoretical measurement precision can reach 0.001mm, and under the scene condition of the example, the precision of 0.1mm can be reached.
Claims (10)
1. A method for registering a coordinate system of a surgical robot with a coordinate system of a CT machine, characterized in that a 3D vision system is used to register the coordinate system of the CT machine with a coordinate system of a manipulator in the surgical robot, and a 3D structured light camera in the 3D vision system moves synchronously with the manipulator and a puncture device fixedly mounted at the end of the manipulator.
2. The method for registering a coordinate system of a surgical robot with a coordinate system of a CT machine according to claim 1, wherein the 3D vision system comprises a 3D structured light camera mounted at the end of the mechanical arm and integrally fixed with the puncture device, and a calibration plate directly placed on the CT machine and located within the photographing range of the 3D structured light camera.
3. The method for registering a surgical robot coordinate system with a CT machine coordinate system of claim 2, for registering a CT machine coordinate system with a robotic arm coordinate system, comprising the steps of:
1) Moving a mechanical arm fixed with a 3D structure light camera into an operation area beside a CT machine;
2) Placing the calibration plate on the CT machine and being positioned in a range which can be shot by the 3D structured light camera;
3) Starting a registration program, wherein the mechanical arm carries the 3D structure light camera to sequentially move to a plurality of pose, after moving to each pose, starting the 3D structure camera to project a plurality of groups of structure light, collecting the pictures of the marked structures in the marking plates projected by the structure light, recording mechanical arm transformation parameters of each pose and the pictures of the marked structures in the marking plates through computer software, and obtaining the conversion relation between the coordinate system of the marking plates and the coordinate system of the mechanical arm by using a hand-eye calibration algorithm to obtain a third conversion matrix;
4) The mechanical arm is retracted to an initial position, the calibration plate is continuously placed on the CT machine and kept still, and the CT machine is started to scan the calibration plate; acquiring CT image data of four spherical calibration pieces on the calibration plate, and combining the spherical centers of the four spherical calibration pieces with a coordinate matrix of the calibration plate to obtain a conversion relation between a CT coordinate system and the coordinate system of the calibration plate, wherein the conversion relation is a fourth conversion matrix;
5) And obtaining the conversion relation between the CT machine coordinate system and the mechanical arm coordinate system as a fifth conversion matrix according to a third conversion matrix between the calibration plate coordinate system and the mechanical arm coordinate system and a fourth conversion matrix between the CT machine coordinate system and the calibration plate coordinate system, and completing coordinate registration of the CT machine coordinate system and the mechanical arm coordinate system.
4. A method for registering a surgical robot coordinate system with a CT machine coordinate system according to claim 3, wherein in step 3), the steps of:
3.1 A registration procedure is started, the mechanical arm carries the 3D structure light camera to sequentially move to n preset different poses, after moving to each pose, the conversion relation between the calibration plate coordinate system and the camera coordinate system under each pose is respectively obtained, and the conversion relation is a first conversion matrix W BC The method comprises the steps of carrying out a first treatment on the surface of the The conversion relation between the center point of the tail end of the mechanical arm and the coordinate system of the mechanical arm is a second conversion matrix W AT ；
3.2 Selecting a first transformation matrix W corresponding to two poses (i, j) BC And a second conversion matrix W AT The transformation matrix W of the camera coordinate system and the center point coordinate system of the tail end of the mechanical arm is obtained according to the following equation (1) CT (ij) ；
W AT (i) W CT (ij) W BC (i) ＝W AT (j) W CT (ij) W BC (j) (1)
3.3 Obtaining two transformation matrices W of the calibration plate coordinate system and the mechanical arm coordinate system for the two poses (i, j) according to the following equation (2) AB (ij) ；
W AB (ij) ＝ W AT (i) W CT (ij) W BC (i) ＝W AT (j) W CT (ij) W BC (j) (2)
3.4 Repeating the steps 3.2) and 3.3), and selecting a first conversion matrix W of two different poses from 2n poses BC And a second conversion matrix W AT N-1/2 of said conversion matrices W can be obtained after combination AB (ij) The method comprises the steps of carrying out a first treatment on the surface of the For n (n-1)/2 of said conversion matrices W AB (ij) The corresponding identical variable in the calibration plate coordinate system is calculated to obtain an average value, the average values of all the identical variables are combined into a conversion relation between the calibration plate coordinate system and the mechanical arm coordinate system, and a third conversion matrix is called
5. The method for registering a coordinate system of a surgical robot with a coordinate system of a CT machine according to claim 4, wherein in the step 3.1), the following steps are performed after the robot arm is moved to each pose:
3.1.1 The 3D structured light camera projects structured light to the calibration plate, a photo of a mark structure in the calibration plate, which is projected by the structured light, is collected, pose transformation parameters of the mechanical arm and the photo of the mark structure in the calibration plate, which is projected by the structured light, are recorded by computer software, a conversion relation between a coordinate system of the calibration plate for the pose and a coordinate system of a center point of the 3D structured light camera is obtained through a hand-eye calibration algorithm, and the conversion relation is a first conversion matrix W BC ；
3.1.2 According to the pose transformation parameters of the mechanical arm under each pose, acquiring the pose representation of the central point of the tail end of the mechanical arm under the mechanical arm coordinate system, namely, the conversion relation between the coordinate system of the central point of the tail end of the mechanical arm of the pose and the mechanical arm coordinate system, and obtaining a second conversion matrix W AT 。
6. The method for registering a surgical robot coordinate system with a CT machine coordinate system of claim 4, wherein the number of poses of the 3D structured light camera moving with the robotic arm in step 3) is an even number between 4-20.
7. The method of claim 5, wherein the 3D structured light camera of step 3.1.1) projects blue-violet LED light with a wavelength of 490nm in 13-24 striped gratings to the calibration plate.
8. The method according to any one of claims 3 to 7, wherein the calibration plate comprises a base plate, four spherical calibration pieces, and columnar connectors for fixedly connecting the base plate and the spherical calibration pieces, locking pieces for locking the columnar connectors and the base plate, wherein the heights of the columnar connectors are distributed in equal-difference rows, all the columnar connectors are fixedly arranged on the same side surface of the base plate, and a checkerboard is arranged on the side surface of the base plate, where the columnar connectors are arranged.
9. The method for registering a coordinate system of a surgical robot with a coordinate system of a CT machine of claim 8, wherein the columnar connector comprises a first connector connected to the spherical calibration element and a second connector connected to the base plate, the first connector being provided with a cylindrical recess into which the spherical calibration element is sunk, the depth of the recess being greater than the radius of the spherical calibration element, and the inner diameter of the recess being less than or equal to the diameter of the spherical calibration element.
10. A system for a method for registering a surgical robot coordinate system with a CT machine coordinate system as claimed in any of the preceding claims 1-9, a 3D structured light camera and a calibration plate for use with a surgical robot console, a robotic arm, a CT machine, characterized in that the 3D structured light camera is fixedly mounted with a penetration device mounted at the end of the robotic arm, which is moved with the movement of the robotic arm.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310365385.0A CN116392246A (en) | 2023-04-07 | 2023-04-07 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
| PCT/CN2023/125927 WO2024027857A1 (en) | 2023-04-07 | 2023-10-23 | Method and system for registration of surgical robot coordinate system with ct scanner coordinate system |
| CN202380012400.1A CN117835933A (en) | 2023-04-07 | 2023-10-23 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310365385.0A CN116392246A (en) | 2023-04-07 | 2023-04-07 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116392246A true CN116392246A (en) | 2023-07-07 |
Family
ID=87008632
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310365385.0A Pending CN116392246A (en) | 2023-04-07 | 2023-04-07 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
| CN202380012400.1A Pending CN117835933A (en) | 2023-04-07 | 2023-10-23 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202380012400.1A Pending CN117835933A (en) | 2023-04-07 | 2023-10-23 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN116392246A (en) |
| WO (1) | WO2024027857A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563297A (en) * | 2023-07-12 | 2023-08-08 | 中国科学院自动化研究所 | Craniocerebral target positioning method, device and storage medium |
| WO2024027857A1 (en) * | 2023-04-07 | 2024-02-08 | 苏州派尼迩医疗科技有限公司 | Method and system for registration of surgical robot coordinate system with ct scanner coordinate system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6321137B1 (en) * | 1997-09-04 | 2001-11-20 | Dynalog, Inc. | Method for calibration of a robot inspection system |
| CN114643578B (en) * | 2020-12-18 | 2023-07-04 | 沈阳新松机器人自动化股份有限公司 | Calibration device and method for improving robot vision guiding precision |
| CN217162286U (en) * | 2021-12-13 | 2022-08-12 | 浙江德尚韵兴医疗科技有限公司 | Measuring tool for positioning precision of tail end of optical navigation robot based on medical image |
| CN115553888A (en) * | 2022-10-28 | 2023-01-03 | 苏州派尼迩医疗科技有限公司 | Puncture robot integrating 3D camera and needle inserting device and puncture method |
| CN116392246A (en) * | 2023-04-07 | 2023-07-07 | 苏州派尼迩医疗科技有限公司 | Method and system for registering surgical robot coordinate system and CT machine coordinate system |
-
2023
- 2023-04-07 CN CN202310365385.0A patent/CN116392246A/en active Pending
- 2023-10-23 WO PCT/CN2023/125927 patent/WO2024027857A1/en unknown
- 2023-10-23 CN CN202380012400.1A patent/CN117835933A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024027857A1 (en) * | 2023-04-07 | 2024-02-08 | 苏州派尼迩医疗科技有限公司 | Method and system for registration of surgical robot coordinate system with ct scanner coordinate system |
| CN116563297A (en) * | 2023-07-12 | 2023-08-08 | 中国科学院自动化研究所 | Craniocerebral target positioning method, device and storage medium |
| CN116563297B (en) * | 2023-07-12 | 2023-10-31 | 中国科学院自动化研究所 | Craniocerebral target positioning method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024027857A1 (en) | 2024-02-08 |
| CN117835933A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116392246A (en) | Method and system for registering surgical robot coordinate system and CT machine coordinate system | |
| KR101495011B1 (en) | Calibration and transformation of a camera system's coordinate system | |
| CN111012506B (en) | Robot-assisted puncture surgery end tool center calibration method based on stereoscopic vision | |
| CN101876532B (en) | Camera on-field calibration method in measuring system | |
| CN110666798B (en) | Robot vision calibration method based on perspective transformation model | |
| CN102692214B (en) | Narrow space binocular vision measuring and positioning device and method | |
| CN114041875A (en) | Integrated operation positioning navigation system | |
| CN113876426B (en) | Intraoperative positioning and tracking system and method combined with shadowless lamp | |
| CN109712232B (en) | Object surface contour three-dimensional imaging method based on light field | |
| CN112229323B (en) | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method | |
| CN116740187A (en) | Multi-camera combined calibration method without overlapping view fields | |
| CN104123726B (en) | Heavy forging measuring system scaling method based on vanishing point | |
| CN109785392B (en) | Calibration device and method for desktop-level virtual reality system | |
| CN1286438C (en) | Vision registering method for medical robot | |
| US20210174950A1 (en) | Stereoscopic marker device and three-dimensional real-time positioning method of orthopedic surgery | |
| CN116883471B (en) | Line structured light contact-point-free cloud registration method for chest and abdomen percutaneous puncture | |
| EP1657678B1 (en) | Patient registration assisted by video imaging | |
| WO2022237787A1 (en) | Robot positioning and pose adjustment method and system | |
| CN113855240B (en) | Medical image registration system and method based on magnetic navigation | |
| CN114404041A (en) | C-shaped arm imaging parameter calibration system and method | |
| Nižetić et al. | Calibration of a 5-Axis CNC Machine for Making Orthoses by Means of a Vision System | |
| CN113509264A (en) | Augmented reality system, method and computer-readable storage medium based on position correction of object in space | |
| CN112132903A (en) | Coordinate system calibration method and system for vision system and multi-axis motion system | |
| CN112581544A (en) | Camera calibration method without public view field based on parameter optimization | |
| CN217853001U (en) | Multi-eye stereo vision quick brain electrode positioning device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |