CN115227392B - Measuring system for skull micropore - Google Patents
Measuring system for skull micropore Download PDFInfo
- Publication number
- CN115227392B CN115227392B CN202210646459.3A CN202210646459A CN115227392B CN 115227392 B CN115227392 B CN 115227392B CN 202210646459 A CN202210646459 A CN 202210646459A CN 115227392 B CN115227392 B CN 115227392B
- Authority
- CN
- China
- Prior art keywords
- skull
- implantation
- camera
- micropore
- surface center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000003625 skull Anatomy 0.000 title claims abstract description 197
- 238000002513 implantation Methods 0.000 claims abstract description 164
- 239000013598 vector Substances 0.000 claims abstract description 83
- 238000005259 measurement Methods 0.000 claims abstract description 37
- 230000033001 locomotion Effects 0.000 claims description 55
- 238000005553 drilling Methods 0.000 claims description 36
- 230000009466 transformation Effects 0.000 claims description 33
- 239000011159 matrix material Substances 0.000 claims description 28
- 239000007943 implant Substances 0.000 claims description 22
- 230000003287 optical effect Effects 0.000 claims description 18
- 238000013519 translation Methods 0.000 claims description 13
- 238000000034 method Methods 0.000 claims description 6
- 210000000988 bone and bone Anatomy 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004080 punching Methods 0.000 description 4
- 238000001356 surgical procedure Methods 0.000 description 4
- 210000004556 brain Anatomy 0.000 description 3
- 210000005013 brain tissue Anatomy 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 239000011148 porous material Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B18/00—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body
- A61B18/18—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body by applying electromagnetic radiation, e.g. microwaves
- A61B18/20—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body by applying electromagnetic radiation, e.g. microwaves using laser
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/06—Measuring instruments not otherwise provided for
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B18/00—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body
- A61B2018/00315—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body for treatment of particular body parts
- A61B2018/00321—Head or parts thereof
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B18/00—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body
- A61B2018/00315—Surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body for treatment of particular body parts
- A61B2018/00565—Bone
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Life Sciences & Earth Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Engineering & Computer Science (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Physics & Mathematics (AREA)
- Radiology & Medical Imaging (AREA)
- Gynecology & Obstetrics (AREA)
- Robotics (AREA)
- Optics & Photonics (AREA)
- Electromagnetism (AREA)
- Otolaryngology (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The embodiment of the invention provides a measurement system of skull micropores, which relates to the technical field of surgical robots and comprises a binocular vision system and an implantation system; the binocular vision system is used for determining the center position of the reference surface of at least one implantation micropore in a coordinate system of a base camera and sending the center position of the reference surface of the at least one implantation micropore to the implantation system; the implantation system is used for acquiring a first surface center position and a skull canal direction vector of the at least one implantation micropore in a skull coordinate system, and determining an actual surface center position and an actual skull canal direction vector of the at least one implantation micropore in the reference camera coordinate system based on the reference surface center position of the at least one implantation micropore, the first surface center position and the skull canal direction vector of the at least one implantation micropore. The invention realizes the measurement of the skull micropore.
Description
Technical Field
The invention relates to the technical field of surgical robots, in particular to a measurement system for skull micropores.
Background
An invasive brain-computer interface requires the implantation of flexible electrodes into a target brain region of the brain to collect electrical signals from and generate electrical stimulation to brain neurons. Compared with the flexible electrode implantation operation based on the cranium window, the flexible electrode implantation operation based on the cranium micropore channel brings less trauma. The diameter of the skull micropore is in the order of hundred micrometers, a single micropore can be implanted with a flexible electrode, and if a plurality of flexible electrodes are required to be implanted, a plurality of micropores are required to be punched on the skull by utilizing a laser technology, so that a micropore array is formed.
Before the implantation of the flexible electrode, the surface center position of each micropore and the axial direction of the skull duct are required to be accurately measured; the implantation needle can then be implanted into brain tissue along the canal of the skull. The central position of the surface of the micropore can be observed from the outside, but because the skull duct is inside skull tissue and cannot be directly observed from the outside, the measurement of the axis direction of the skull duct is a difficult problem.
Therefore, there is a need for a measurement system for skull micro-holes to solve the above problems.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a skull micropore measurement system.
Specifically, the embodiment of the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides a measurement system for a microhole of a skull, including a binocular vision system and an implantation system;
the binocular vision system is used for determining the center position of the reference surface of at least one implantation micropore in a coordinate system of a base camera and sending the center position of the reference surface of the at least one implantation micropore to the implantation system;
the implantation system is used for acquiring a first surface center position and a skull canal direction vector of the at least one implantation micropore in a skull coordinate system, and determining an actual surface center position and an actual skull canal direction vector of the at least one implantation micropore in the reference camera coordinate system based on the reference surface center position of the at least one implantation micropore, the first surface center position and the skull canal direction vector of the at least one implantation micropore.
According to the measurement system of skull micro-holes provided by the invention, the implantation system is further used for controlling the needle-shaped tool to implant the flexible electrode into each implantation micro-hole based on the actual surface center position of the at least one implantation micro-hole in the reference camera coordinate system and the actual skull canal direction vector.
The skull micropore measurement system provided by the invention further comprises a laser drilling system and an implantation operation planning system;
the implantation operation planning system is used for generating space planning information of laser drilling on the target skull and sending the space planning information to the laser drilling system and the implantation system; the space planning information comprises a first surface center position of each implanted micropore in the skull coordinate system and a skull tunnel direction vector;
the laser drilling system is used for drilling holes in the target skull based on the central position of the first surface of each implanted micropore and the direction vector of the skull duct, so as to obtain a plurality of implanted micropores in the target skull.
According to the measurement system for the skull micropores provided by the invention, the laser drilling system is particularly used for controlling the drilling laser beam to pass through the first surface center position of each implanted micropore on the target skull based on the first surface center position of each implanted micropore and the skull pore canal direction vector, and controlling the direction of the drilling laser beam to be parallel to the skull pore canal direction of each implanted micropore so as to obtain a plurality of implanted micropores on the target skull.
According to the measuring system for the skull micropores, which is provided by the invention, the binocular vision system comprises a motion platform, and a first camera and a second camera which are arranged on the motion platform, wherein a microscope lens is arranged on each of the first camera and the second camera;
an included angle is formed between the optical axis of the first camera and the optical axis of the second camera, and the micro lens of the first camera is close to the micro lens of the second camera, so that the first camera and the second camera have an included angle of common vision.
According to the measurement system of the skull micro-holes, which is provided by the invention, the motion platform is used for adjusting the positions of the first camera and the second camera, so that the target implantation micro-holes enter the common field of view of the first camera and the second camera, and the current position of the motion platform is obtained;
the first camera is used for acquiring a first image of the target implantation micropore on the target skull and sending the first image to the motion platform;
the second camera is used for acquiring a second image of the target implantation micropore on the target skull and sending the second image to the motion platform;
the motion platform is further configured to determine a reference surface center position of the target implant microwell in the fiducial camera coordinate system based on a current position of the motion platform, the first image, and the second image.
According to the measurement system of the skull micro-holes provided by the invention, the motion platform is particularly used for determining the first surface center coordinate of the target implantation micro-holes in the first image based on the first image and determining the second surface center coordinate of the target implantation micro-holes in the second image based on the second image;
the motion platform is further specifically configured to determine a second surface center position of the target implantation micropore in the current camera coordinate system based on the first surface center coordinate, the second surface center coordinate, the parameter of the first camera, and the parameter of the second camera, and determine a reference surface center position of the target implantation micropore in the reference camera coordinate system based on the second surface center position, a first rotation transformation matrix between the motion platform and the first camera, and the current position of the motion platform; the reference camera coordinate system is a coordinate system taking the optical center of the first camera or the optical center of the second camera as an origin when the motion platform is located at a reference position, and the current camera coordinate system is a coordinate system corresponding to the first camera and the second camera after moving.
According to the measurement system of the skull micro-hole provided by the invention, the implantation system is specifically used for determining a second rotation transformation matrix and a translation vector from the skull coordinate system to the reference camera coordinate system based on the reference surface center position of the at least one implantation micro-hole and the first surface center position of the at least one implantation micro-hole;
the implantation system is further specifically configured to perform coordinate transformation on a first surface center position of the at least one implantation micropore in the skull coordinate system and a skull tunnel direction vector based on the second rotation transformation matrix and the translation vector, so as to obtain an actual surface center position of the at least one implantation micropore on the target skull in the reference camera coordinate system and an actual skull tunnel direction vector.
According to the measurement system of the skull micropore provided by the invention, the implantation system is also specifically used for acquiring the direction vector of the needle tip axis of the needle-shaped tool in the reference camera coordinate system;
the implantation system is further specifically configured to control the needle-shaped tool to rotate based on a third rotation transformation matrix, the direction vector and the actual skull canal direction vector, so that an axis of the needle-shaped tool is parallel to an actual skull canal direction of a corresponding implanted micropore, and control a terminal position of the needle-shaped tool so that the flexible electrode passes through an actual surface center position of the corresponding implanted micropore; the third rotational transformation matrix is a rotational transformation matrix of the reference camera coordinate system to a coordinate system of the implant system.
According to the measuring system of the skull micropores provided by the invention, the laser drilling system is also used for arranging a plurality of non-implanted micropores on the target skull; the non-implanted micropores do not penetrate the bone layer of the target skull.
According to the measuring system for providing the skull micropores, the binocular vision system sends the determined reference surface center position of at least one implantation micropore in the reference camera coordinate system to the implantation system, and the implantation system determines the actual surface center position and the actual skull tunnel direction vector of the at least one implantation micropore in the reference camera coordinate system based on the reference surface center position of the at least one implantation micropore in the reference camera coordinate system, the acquired first surface center position and the skull tunnel direction vector of the at least one implantation micropore. It can be seen that the present invention can determine the actual surface center position and the actual skull tunnel direction vector of the implanted micro-hole in the reference camera coordinate system based on the surface center position and the skull tunnel direction vector of the implanted micro-hole in the skull coordinate system and the reference surface center position of the implanted micro-hole in the reference camera coordinate system, thereby realizing the measurement of the skull micro-hole.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a measurement system for skull microholes according to the present invention;
FIG. 2 is a second schematic diagram of the measurement system for skull micro-holes provided by the invention;
FIG. 3 is a schematic diagram of a binocular vision system provided by the present invention;
fig. 4 is a schematic view of the structure of an implant system provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic structural diagram of a measurement system of a skull micropore provided by the invention, and as shown in fig. 1, the measurement system of the skull micropore comprises a binocular vision system 11 and an implantation system 12.
The binocular vision system 11 is used for determining the center position of the reference surface of at least one implantation micropore in a coordinate system of a base camera and transmitting the center position of the reference surface of the at least one implantation micropore to the implantation system 12;
the implantation system 12 is configured to obtain a first surface center position and a skull tunnel direction vector of the at least one implantation micropore in a skull coordinate system, and determine an actual surface center position and an actual skull tunnel direction vector of the at least one implantation micropore in the reference camera coordinate system based on the reference surface center position of the at least one implantation micropore, the first surface center position of the at least one implantation micropore, and the skull tunnel direction vector.
The reference camera coordinate system is a coordinate system with the optical center of the camera in the binocular vision system 11 as the origin, and the skull coordinate system is a coordinate system established with the position of the target skull.
Illustratively, the binocular vision system 11 may sequentially determine reference surface center positions Pi (i=1, 2, …, M) of M implanted micro-holes in the micro-hole array in the reference camera coordinate system, where Pi represents the reference surface center position of the ith implanted micro-hole in the reference camera coordinate system, m+.n, m+.3, N is the number of implanted micro-holes in the micro-hole array; in determining the reference surface center positions of the M implanted microwells in the reference camera coordinate system in the microwell array, the reference surface center positions Pi (i=1, 2, …, M) of the M implanted microwells in the reference camera coordinate system are transmitted to the implantation system 12.
The implantation system 12 is configured to receive the first surface center position H of the N implantation micropores in the skull coordinate system transmitted by the implantation surgery planning system i And skull tunnel direction vector D i And acquiring the first surface center positions H of corresponding M implantation micropores in the skull coordinate system i Then using the reference surface center positions Pi (i=1, 2, …, M) of the M implanted micro-holes, the first surface center positions H of the corresponding M implanted micro-holes i And skull tunnel direction vector D i The actual surface center positions and the actual skull canal direction vectors of the corresponding M implantation micropores in the reference camera coordinate system are determined, and the actual surface center positions and the actual skull canal direction vectors of the M implantation micropores are used for ensuring that an implantation tool (for example, an implantation needle) can smoothly pass through the implantation micropores.
According to the measuring system for the skull micropores, provided by the embodiment of the invention, the binocular vision system sends the determined reference surface center position of at least one implantation micropore in the reference camera coordinate system to the implantation system, and the implantation system determines the actual surface center position and the actual skull tunnel direction vector of at least one implantation micropore in the reference camera coordinate system based on the reference surface center position of at least one implantation micropore in the reference camera coordinate system, the acquired first surface center position and the skull tunnel direction vector of at least one implantation micropore. It can be seen that the present invention can determine the actual surface center position and the actual skull tunnel direction vector of the implanted micro-hole in the reference camera coordinate system based on the surface center position and the skull tunnel direction vector of the implanted micro-hole in the skull coordinate system and the reference surface center position of the implanted micro-hole in the reference camera coordinate system, thereby realizing the measurement of the skull micro-hole.
Optionally, the implantation system 12 is further configured to control the needle-like tool to implant a flexible electrode into each implantation microwell based on an actual surface center position of the at least one implantation microwell in the reference camera coordinate system and an actual skull canal direction vector.
Illustratively, when determining the actual surface center position and the actual skull canal direction vector of the M implant micro-holes in the reference camera coordinate system, the implantation system 12 may control the needle-shaped tool to implant the flexible electrode into each implant micro-hole based on the actual surface center position and the actual skull canal direction vector of the M implant micro-holes in the reference camera coordinate system, thereby achieving the implantation operation of the skull micro-holes.
Optionally, fig. 2 is a second schematic structural diagram of the measurement system of the micro-hole of the skull, and as shown in fig. 2, the measurement system of the micro-hole of the skull further comprises a laser drilling system 13 and an implantation surgery planning system 14;
the implantation surgery planning system 14 is configured to generate spatial planning information for laser drilling on the target skull, and send the spatial planning information to the laser drilling system 13 and the implantation system 12; the space planning information comprises a first surface center position of each implanted micropore in the skull coordinate system and a skull tunnel direction vector;
the laser drilling system 13 is configured to drill a target skull based on the first surface center position and the skull canal direction vector of each implanted micropore, so as to obtain a plurality of implanted micropores on the target skull.
Wherein the surface center position of the first implanted micropore designated on the target skull can be used as the origin of the skull coordinate system.
Illustratively, the spatial planning information includes first surface center positions H of N implant micro-holes in a skull coordinate system i (i=1, 2, …, N) and cranial canal direction vector D i (i=1, 2, …, N), N implanted microwells forming a microwell array, wherein N is greater than or equal to 3, the first surface center position H i Representing the three-dimensional position of the ith implanted micropore in the contour center of the target skull surface, skull canal direction vector D i Representing the three-dimensional direction of a skull canal formed by the ith micropore in the target skull bone layer, wherein the skull canal can be cylindrical; implant surgery planning system 14 may center position H of the first surface of N implant micro-holes in the skull coordinate system i (i=1, 2, …, N) and cranial canal direction vector D i (i=1, 2, …, N) is sent to the laser drilling system 13 and the implantation system 12 by means of transmission control protocol/internet protocol (Transmission Control Protocol/Internet Protocol, TCP/IP) or the like.
Specifically, the laser drilling system 13 is specifically configured to control a drilling laser beam to pass through the first surface center position of each implanted micropore on the target skull based on the first surface center position of each implanted micropore and the skull canal direction vector, and control the direction of the drilling laser beam to be parallel to the skull canal direction of each implanted micropore, so as to obtain a plurality of implanted micropores on the target skull.
For example, the laser drilling system 13 may determine the first surface center position H in the skull coordinate system according to the received N implant micro-holes i (i=1, 2, …, N) and cranial canal direction vector D i (i=1, 2, …, N), and punching is performed on the target skull. For the ith micropore, the 5-degree-of-freedom (three-directional degrees of freedom, pitch degree of freedom and yaw degree of freedom) pose of the punching laser beam in the skull coordinate system is defined by the first surface center position H i Vector D of direction of canal of skull i Determining, i.e. the position H at which the perforating laser beam should pass through the first surface i Position and direction of punching laser beam and skull duct direction vector D i Direction of indicationParallel. Finally, N.gtoreq.3 implantation micro-holes are formed in the target skull, which is then transferred into the implantation system 12. For example, a punching laser beam is used to punch 100-500 micrometers of implantation micropores on the target skull, and the shape of the surface hole of the target skull can be round or oval, which is beneficial to positioning the center of the surface hole of the target skull.
It should be noted that, the positions of the N implanted micropores on the target skull cannot be collinear, and a certain interval should be provided, so that the spatial distribution of the N implanted micropores is larger, which is beneficial to guaranteeing the final pose measurement accuracy.
According to the measuring system for the micropores of the skull, provided by the invention, the laser drilling system and the implantation system can read the space planning information of laser drilling on the target skull from the implantation operation planning system, the laser drilling system can drill holes on the target skull based on the space planning information of laser drilling on the target skull, and the implantation system can determine the actual surface center position and the actual skull duct direction vector of the implantation micropore in the reference camera coordinate system based on the space planning information of laser drilling on the target skull, so that the implantation operation planning system provides convenience for acquiring the space planning information for the laser drilling system and the implantation system.
Optionally, fig. 3 is a schematic structural diagram of a binocular vision system provided by the present invention, as shown in fig. 3, the binocular vision system includes a moving platform 31, and a first camera 32 and a second camera 33 disposed on the moving platform 31, where a microscope lens is installed on each of the first camera 32 and the second camera 33;
the optical axis of the first camera 32 and the optical axis of the second camera 33 form an included angle, and the micro lens of the first camera 32 is close to the micro lens of the second camera 33, so that the first camera 32 and the second camera 33 have an included angle of common field of view.
Preferably, the angle formed by the optical axis of the first camera 32 and the optical axis of the second camera 33 may be 45 degrees.
As illustrated in fig. 3, the first camera 32 and the second camera 33 are mounted on the same moving platform 31, the moving platform 31 is configured by three electric sliding tables, and the moving axes of each electric sliding table are perpendicular to each other, so that the moving axes of the three electric sliding tables form an XYZ orthogonal coordinate system, thereby enabling the three-dimensional translational movement of the moving platform 31 configured by the three electric sliding tables to change the three-dimensional positions of the first camera 32 and the second camera 33.
It should be noted that the electric slipway may be a precision electric slipway, and the movement resolution of each precision electric slipway may be 1 micrometer.
Before using the binocular vision system 11, it is also necessary to calibrate the internal parameters and the external parameters of the first camera 32 and the second camera 33 mounted on the motion platform 31, and calibrate the relative pose of the first camera 32 and the motion platform 31, and the relative pose of the second camera 33 and the motion platform 31. Wherein, the internal parameters comprise parameters such as magnification, focal length and the like, and the external parameters comprise parameters such as relative pose of the camera (the first camera 32 or the second camera 33) and the motion platform 31 and the like; the relative pose of the camera and the motion platform 31 comprises a rotation transformation matrix between the coordinate system of the motion platform 31 and the camera, a rotation transformation matrix between the two cameras, a translation vector and the like; when the motion stage 31 is at the set zero point position, the optical center of the first camera 32 or the optical center of the second camera 33 at this time is taken as the origin of the reference camera coordinate system.
Optionally, the motion platform 31 is configured to adjust the positions of the first camera 32 and the second camera 33, so that the target implantation micropore enters the common field of view of the first camera 32 and the second camera 33, and acquire the current position of the motion platform 31;
the first camera 32 is configured to acquire a first image of the target implantation micropore on the target skull, and send the first image to the motion platform 31;
the second camera 33 is configured to acquire a second image of the target implantation micropore on the target skull, and send the second image to the motion platform 31;
the motion stage 31 is further configured to determine a reference surface center position of the target implant microwell in the fiducial camera coordinate system based on a current position of the motion stage 31, the first image, and the second image.
Taking the target implantation microwell as the ith implantation microwell as an example, as shown in fig. 3, the first camera 32 and the second camera 33 are moved by the motion platform 31 such that the ith implantation microwell in the microwell array enters the common view of the first camera 32 and the second camera 33, a first image containing the ith implantation microwell is captured by the first camera 32, a second image containing the ith implantation microwell is captured by the second camera 33, and the current position P of the motion platform 31 is read Mi So that the motion platform 31 is based on the current position P of the motion platform 31 Mi The first image and the second image determine a reference surface center position of the target implant microwell in a fiducial camera coordinate system.
Specifically, the motion platform 31 is specifically configured to determine, based on the first image, a first surface center coordinate of the target implantation micropore in the first image, and determine, based on the second image, a second surface center coordinate of the target implantation micropore in the second image;
the motion platform 31 is further specifically configured to determine a second surface center position of the target implantation micropore in the current camera coordinate system based on the first surface center coordinate, the second surface center coordinate, the parameter of the first camera 32, and the parameter of the second camera 33, and determine a reference surface center position of the target implantation micropore in the reference camera coordinate system based on the second surface center position, the first rotation transformation matrix between the motion platform 31 and the first camera 32, and the current position of the motion platform 31; the reference camera coordinate system is a coordinate system with the optical center of the first camera 32 or the optical center of the second camera 33 as an origin when the moving platform 31 is located at the reference position, and the current camera coordinate system is a coordinate system corresponding to the first camera 32 and the second camera 33 after moving.
The movement platform 31 being located at the reference position means that the movement platform 31 is located at a preset reference position.
Illustratively, a motion platform 31When a first image and a second image are acquired, respectively performing image processing and feature extraction on the first image and the second image to obtain a first surface center coordinate p of an ith implanted micropore in the first image 1 And a second surface center coordinate p of an ith implanted micro-hole in the second image 2 Specifically, the first image and the second image can be respectively input into a segmentation model based on a convolutional neural network, so as to obtain a foreground image of an ith implanted micropore corresponding to the first image output by the segmentation model and a foreground image of an ith implanted micropore corresponding to the second image; then collecting effective pixels in the foreground image of the ith implanted micropore corresponding to the first image, and averaging the coordinates of all the effective pixels to obtain a first surface center coordinate p 1 Collecting effective pixels in the foreground image of the ith implanted micropore corresponding to the second image, and averaging the coordinates of all the effective pixels to obtain a second surface center coordinate p 2 。
Further, the first surface center coordinate p is obtained 1 And a second surface center coordinate p 2 In this case, the first surface center coordinates p may be based on 1 Center coordinates p of second surface 2 Parameters of the first camera 32 (inner and outer parameters) and parameters of the second camera 33 (inner and outer parameters), the second surface center position P of the ith implanted micro-hole in the current camera coordinate system is calculated using binocular vision three-dimensional position measurement algorithm Ci A second surface center position P Ci For the three-dimensional coordinate position, then calculate the reference surface center position P of the ith implanted micro-hole in the base camera coordinate system i ＝P Ci -R M ·P Mi Wherein R is M Representing a rotation transformation matrix between the motion platform coordinate system and the reference camera coordinate system; repeating the above steps until the measurement of M implanted micropores is completed, and finally obtaining the reference surface center positions { P } of M implanted micropores in the reference camera coordinate system i }(i＝1,2,…,M)。
It should be noted that, the calculation process of the binocular vision three-dimensional position measurement algorithm may refer to the prior art, and the present invention is not described herein.
It should be noted that, the segmentation model may be obtained by training based on an image sample including implanted micropores, and the specific training process may be to input the image sample into the initial segmentation model to obtain a micropore image output by the initial segmentation model, then construct a loss function based on the positions of micropores in the micropore image and the actual positions of labeled micropores, and optimize parameters of the initial segmentation model based on the loss function until convergence conditions are reached, so as to obtain the segmentation model.
According to the measuring system for the skull micropores, provided by the invention, the moving platform determines the reference surface center position of the corresponding implantation micropore in the reference camera coordinate system based on the first image captured by the first camera and the second image captured by the second camera, and the reference surface center position is the surface center position planned for the target skull based on the image captured by the camera, so that the implanting system can calculate the actual surface center position of the target skull by using the surface center position planned for the target skull.
Optionally, the implantation system 12 is specifically configured to determine a second rotational transformation matrix and a translation vector of the skull coordinate system to the fiducial camera coordinate system based on the reference surface center position of the at least one implantation microwell and the first surface center position of the at least one implantation microwell;
the implantation system 12 is further specifically configured to perform coordinate transformation on the first surface center position of the at least one implantation micropore in the skull coordinate system and the skull tunnel direction vector based on the second rotation transformation matrix and the translation vector, so as to obtain an actual surface center position of the at least one implantation micropore on the target skull in the reference camera coordinate system and an actual skull tunnel direction vector.
Illustratively, the implantation system 12 utilizes the reference surface center positions { P } of the M implanted micro-holes in the fiducial camera coordinate system obtained by the motion stage 31 i First surface center position H corresponding to (i=1, 2, …, M) and M implanted micro-holes i Solving a second rotation transformation matrix R from the skull coordinate system to the reference camera coordinate system S And translation vector t S . In particular, it can be based on a reference phaseReference surface center position { P ] of M implanted micro-holes in machine coordinate system i First surface center position H corresponding to (i=1, 2, …, M), M implant micro-holes i Second rotational transformation matrix R of skull coordinate system to reference camera coordinate system S And translation vector t S Constructing a target optimization function shown in the following formula (1), solving the target optimization function by using a least square method to minimize the error of the value of the target optimization function and obtain an optimal second rotation transformation matrix R S And translation vector t S 。
In the second rotation transformation matrix R S And translation vector t S Based on the second rotation transformation matrix R S And translation vector t S For the first surface center position H i Vector D of direction of canal of skull i Performing coordinate transformation to obtain the actual surface center positions X of M implanted micropores on the target skull in a reference camera coordinate system i ＝R S ·H i +t S And the actual skull canal direction vector V i ＝R S ·D i Actual surface center position X of M implanted micro-holes i And the actual skull canal direction vector V i Can be used for controlling the pose of the implantation system 12, and realizes that the implantation tool passes through the skull bone layer along the direction of the skull canal through the center of the implantation micropore so as to enter the brain tissue in the cranial cavity.
According to the measurement system of the skull micropores, provided by the invention, the implantation system can carry out coordinate transformation on the first surface center position of the implantation micropores in the skull coordinate system and the skull duct direction vector based on the second rotation transformation matrix and the translation vector from the skull coordinate system to the reference camera coordinate system, and finally the actual surface center position of the implantation micropores in the reference camera coordinate system and the actual skull duct direction vector are obtained. And the measurement of the implanted micropores is realized only through the first image and the second image captured after the movement of the first camera and the second camera by the motion platform in the binocular vision system without additional navigation positioning equipment, so that the structure of the measurement system is simplified.
Optionally, the implantation system 12 is further specifically configured to obtain a direction vector of the needle tip axis of the needle tool in the reference camera coordinate system;
the implantation system 12 is further specifically configured to control the needle-shaped tool to rotate based on the third rotation transformation matrix, the direction vector, and the actual skull canal direction vector, so that an axis of the needle-shaped tool is parallel to an actual skull canal direction of the corresponding implanted micropore, and control a terminal position of the needle-shaped tool so that the flexible electrode passes through an actual surface center position of the corresponding implanted micropore; the third rotational transformation matrix is a rotational transformation matrix of the reference camera coordinate system to the coordinate system of the implant system 12.
Illustratively, the implantation system 12 obtains the actual surface center position X of the M implantation micro-holes in the reference camera coordinate system i And the actual skull canal direction vector V i When the needle-shaped tool is used, the needle-shaped tool is used for carrying the flexible electrode to pass through the implanted micropore and enter the cranial cavity brain tissue. Before the flexible electrode is implanted into the ith implantation micropore, the binocular vision system 11 is used for measuring the direction vector V of the needle tip axis of the needle-shaped tool in the reference camera coordinate system N Further calculate V N And V is equal to i Included angle theta of (2) i And V N And V is equal to i Normal vector phi of the plane formed i . A third rotational transformation matrix R of the reference camera coordinate system to the coordinate system of the implant system 12 is known I In the coordinate system of the implantation system 12 in R I ·Φ i As the rotation axis by theta i The rotating angle is used for rotating movement, the gesture of the needle-shaped tool is adjusted, so that the axis of the needle-shaped tool is parallel to the direction of the actual skull canal of the ith implanted micropore, the tail end position of the needle-shaped tool is controlled, and the flexible electrode passes through the center position of the actual surface of the ith implanted micropore and further passes through the skull canal of the ith implanted micropore. FIG. 4 is a schematic view of the implantation system provided by the present invention, as shown in FIG. 4, with the needle-like tool aligned with the implantation microholes such that the needle-like tool implants the flexible electrode pairsQuasi-implantation into the microwells.
The skull micropore measurement system provided by the invention can be used for an implantation system 12 based on the actual surface center position X of M implantation micropores in a reference camera coordinate system i And the actual skull canal direction vector V i And the pose of the needle-shaped tool is controlled, so that the needle-shaped tool can implant the flexible electrode into each implantation micropore, and the accuracy of implanting the flexible electrode is improved.
Optionally, the laser drilling system 13 is further configured to provide a plurality of non-implanted micropores in the target skull; the non-implanted micropores do not penetrate the bone layer of the target skull.
The non-implanted micropores also form holes on the surface of the skull, but the pore canals of the non-implanted micropores do not penetrate through the skull bone layer, and the function of the non-implanted micropores is to increase the measuring points and improve the measuring precision.
According to the measuring system for the skull micropores, provided by the invention, the laser drilling and implantation processes of the skull micropores are cooperated, and the space planning information of the micropore array is obtained in the laser drilling stage; in the flexible electrode implantation stage, the actual tunnel directions corresponding to the actual skull tunnel direction vectors of the implanted micropores on the target skull can be obtained only by measuring the positions of the implanted micropores through a movable binocular vision system without additional navigation positioning equipment, and the directions are applied to ensure that an implantation tool can smoothly pass through the micropores.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. A measurement system of skull micropores, which is characterized by comprising a binocular vision system and an implantation system;
the binocular vision system is used for determining the center position of the reference surface of at least one implantation micropore in a coordinate system of a base camera and sending the center position of the reference surface of the at least one implantation micropore to the implantation system;
the implantation system is used for acquiring a first surface center position and a skull canal direction vector of the at least one implantation micropore in a skull coordinate system, and determining an actual surface center position and an actual skull canal direction vector of the at least one implantation micropore in the reference camera coordinate system based on the reference surface center position of the at least one implantation micropore, the first surface center position and the skull canal direction vector of the at least one implantation micropore;
the binocular vision system comprises a motion platform, and a first camera and a second camera which are arranged on the motion platform, wherein a microscope lens is arranged on each of the first camera and the second camera;
an included angle is formed between the optical axis of the first camera and the optical axis of the second camera, and the micro lens of the first camera is close to the micro lens of the second camera, so that the first camera and the second camera have an included angle of common vision;
the motion platform is used for adjusting the positions of the first camera and the second camera so that a target implantation micropore enters the common field of view of the first camera and the second camera, and acquiring the current position of the motion platform;
the first camera is used for acquiring a first image of the target implantation micropore on the target skull and sending the first image to the motion platform;
the second camera is used for acquiring a second image of the target implantation micropore on the target skull and sending the second image to the motion platform;
the motion platform is further used for determining a first surface center coordinate of the target implantation micropore in the first image based on the first image and determining a second surface center coordinate of the target implantation micropore in the second image based on the second image;
the motion platform is further specifically configured to determine a second surface center position of the target implantation micropore in the current camera coordinate system based on the first surface center coordinate, the second surface center coordinate, the parameter of the first camera, and the parameter of the second camera, and determine a reference surface center position of the target implantation micropore in the reference camera coordinate system based on the second surface center position, a first rotation transformation matrix between the motion platform and the first camera, and the current position of the motion platform; the reference camera coordinate system is a coordinate system taking the optical center of the first camera or the optical center of the second camera as an origin when the motion platform is located at a reference position, and the current camera coordinate system is a coordinate system corresponding to the first camera and the second camera after moving.
2. The measurement system of skull micro-holes according to claim 1, wherein,
the implantation system is further configured to control the needle-shaped tool to implant a flexible electrode into each implantation microwell based on an actual surface center position of the at least one implantation microwell in the reference camera coordinate system and an actual skull canal direction vector.
3. The measurement system of skull micro-holes of claim 1, further comprising a laser drilling system and an implantation procedure planning system;
the implantation operation planning system is used for generating space planning information of laser drilling on a target skull and sending the space planning information to the laser drilling system and the implantation system; the space planning information comprises a first surface center position of each implanted micropore in the skull coordinate system and a skull tunnel direction vector;
the laser drilling system is used for drilling holes in the target skull based on the central position of the first surface of each implanted micropore and the direction vector of the skull duct, so as to obtain a plurality of implanted micropores in the target skull.
4. A measurement system for a skull micropore according to claim 3,
the laser drilling system is specifically used for controlling a drilling laser beam to pass through the first surface center position of each implanted micropore on the target skull based on the first surface center position of each implanted micropore and the skull duct direction vector, and controlling the direction of the drilling laser beam to be parallel to the skull duct direction of each implanted micropore so as to obtain a plurality of implanted micropores on the target skull.
5. A measurement system for a skull micropore according to claim 2,
the implantation system is specifically configured to determine a second rotational transformation matrix and a translation vector of the skull coordinate system to the reference camera coordinate system based on the reference surface center position of the at least one implantation microwell and the first surface center position of the at least one implantation microwell;
the implantation system is further specifically configured to perform coordinate transformation on a first surface center position of the at least one implantation micropore in the skull coordinate system and a skull tunnel direction vector based on the second rotation transformation matrix and the translation vector, so as to obtain an actual surface center position of the at least one implantation micropore on the target skull in the reference camera coordinate system and an actual skull tunnel direction vector.
6. A measurement system for a skull micropore according to claim 5,
the implantation system is further specifically configured to obtain a direction vector of a needle tip axis of the needle tool in the reference camera coordinate system;
the implantation system is further specifically configured to control the needle-shaped tool to rotate based on a third rotation transformation matrix, the direction vector and the actual skull canal direction vector, so that an axis of the needle-shaped tool is parallel to an actual skull canal direction of a corresponding implanted micropore, and control a terminal position of the needle-shaped tool so that the flexible electrode passes through an actual surface center position of the corresponding implanted micropore; the third rotational transformation matrix is a rotational transformation matrix of the reference camera coordinate system to a coordinate system of the implant system.
7. A measurement system for a skull micropore according to claim 3,
the laser drilling system is also used for setting a plurality of non-implanted micropores on the target skull; the non-implanted micropores do not penetrate the bone layer of the target skull.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210646459.3A CN115227392B (en) | 2022-06-08 | 2022-06-08 | Measuring system for skull micropore |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210646459.3A CN115227392B (en) | 2022-06-08 | 2022-06-08 | Measuring system for skull micropore |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115227392A CN115227392A (en) | 2022-10-25 |
| CN115227392B true CN115227392B (en) | 2023-10-31 |
Family
ID=83669212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210646459.3A Active CN115227392B (en) | 2022-06-08 | 2022-06-08 | Measuring system for skull micropore |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115227392B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117982212B (en) * | 2024-04-03 | 2024-06-21 | 北京智冉医疗科技有限公司 | Electrode implantation apparatus and electrode implantation method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102262345A (en) * | 2011-06-21 | 2011-11-30 | 李�杰 | Method and device of twin-lens three-dimensional camera capable of changing visual angle, |
| CN103209656A (en) * | 2010-09-10 | 2013-07-17 | 约翰霍普金斯大学 | Visualization of registered subsurface anatomy reference to related applications |
| CN105848606A (en) * | 2015-08-24 | 2016-08-10 | 深圳市鑫君特智能医疗器械有限公司 | Intelligent orthopedic surgery system |
| CN109745119A (en) * | 2017-11-06 | 2019-05-14 | 艾博纽诺股份公司 | Device and method for calibrating and measuring the target point for cranial nerve navigator |
| CN111265301A (en) * | 2018-12-04 | 2020-06-12 | 格罗伯斯医疗有限公司 | Drill guide fixture, cranial insertion fixture and related methods and robotic systems |
| CN112155737A (en) * | 2020-10-16 | 2021-01-01 | 华志微创医疗科技（北京）有限公司 | System and method for implanting detection device into cranium |
| CN113100939A (en) * | 2021-04-06 | 2021-07-13 | 德智鸿(上海)机器人有限责任公司 | Orthopedic surgery navigation method, device, computer equipment, system and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2920567C (en) * | 2007-02-01 | 2019-03-05 | Ravish V. Patwardhan | Surgical navigation system for guiding an access member |
| EP2571465A2 (en) * | 2010-05-17 | 2013-03-27 | Advanced Bionics AG | Implantable hearing aid and system and method for implanting the same |
-
2022
- 2022-06-08 CN CN202210646459.3A patent/CN115227392B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103209656A (en) * | 2010-09-10 | 2013-07-17 | 约翰霍普金斯大学 | Visualization of registered subsurface anatomy reference to related applications |
| CN102262345A (en) * | 2011-06-21 | 2011-11-30 | 李�杰 | Method and device of twin-lens three-dimensional camera capable of changing visual angle, |
| CN105848606A (en) * | 2015-08-24 | 2016-08-10 | 深圳市鑫君特智能医疗器械有限公司 | Intelligent orthopedic surgery system |
| CN109745119A (en) * | 2017-11-06 | 2019-05-14 | 艾博纽诺股份公司 | Device and method for calibrating and measuring the target point for cranial nerve navigator |
| CN111265301A (en) * | 2018-12-04 | 2020-06-12 | 格罗伯斯医疗有限公司 | Drill guide fixture, cranial insertion fixture and related methods and robotic systems |
| CN112155737A (en) * | 2020-10-16 | 2021-01-01 | 华志微创医疗科技（北京）有限公司 | System and method for implanting detection device into cranium |
| CN113100939A (en) * | 2021-04-06 | 2021-07-13 | 德智鸿(上海)机器人有限责任公司 | Orthopedic surgery navigation method, device, computer equipment, system and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 用于引导机器人定位的汇聚式双目视觉算法实现;刘红帝等;机械工程学报;第58卷(第14期);161-169页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115227392A (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111012506B (en) | Robot-assisted puncture surgery end tool center calibration method based on stereoscopic vision | |
| CN107753106B (en) | Surgical robot for positioning operation and control method thereof | |
| CN115227392B (en) | Measuring system for skull micropore | |
| US9020613B2 (en) | Method and apparatus for robotically assisted cochlear implant surgery | |
| CN113400325B (en) | Navigation and positioning method for dental implant robot | |
| CN112006777B (en) | Nailing surgical robot system based on surface tracking and control method | |
| US20190298277A1 (en) | Surgical positioning apparatus, positioning system and positioning method | |
| CN110215284A (en) | A kind of visualization system and method | |
| CN111513850B (en) | Guide device, puncture needle adjustment method, storage medium, and electronic apparatus | |
| CN111759463A (en) | Method for improving positioning precision of surgical mechanical arm | |
| US11660043B2 (en) | Posture display method, device and system of guiding channel, and readable storage medium | |
| Qin et al. | Automated hooking of biomedical microelectrode guided by intelligent microscopic vision | |
| Eilers et al. | Navigated, robot assisted drilling of a minimally invasive cochlear access | |
| CN109091234A (en) | Minimally Invasive Surgery puncture needle robot system | |
| Lim et al. | Image-guided robotic mastoidectomy using human-robot collaboration control | |
| CN114516051B (en) | Front intersection method and system for three or more degrees of freedom robot vision measurement | |
| CN113580141B (en) | Pose solving method for 6-axis mechanical arm | |
| Xia et al. | Microscope-guided autonomous clear corneal incision | |
| CN113876433A (en) | Robot system and control method | |
| CN115542530B (en) | Method and device for aligning positions of tiny targets | |
| CN114310910A (en) | Control method, control equipment and auxiliary system suitable for puncture surgical mechanical arm | |
| DE10109310A1 (en) | Three-dimensional tracking of probe needles, biopsy needles or surgical instruments using a CT or MRT system with improved tracking provided by undertaking a calibration step using an infrared light source calibration frame | |
| Sharma et al. | Towards an Autonomous Minimally Invasive Spinal Fixation Surgery Using a Concentric Tube Steerable Drilling Robot | |
| CN215273284U (en) | Integrated surgical robot connecting base | |
| CN115641382B (en) | External parameter calibration method for orthogonal stereoscopic vision structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |