WO2015158756A1 - Procédé et dispositif permettant d'estimer un point de pivotement optimal - Google Patents

Procédé et dispositif permettant d'estimer un point de pivotement optimal Download PDF

Info

Publication number
WO2015158756A1
WO2015158756A1 PCT/EP2015/058137 EP2015058137W WO2015158756A1 WO 2015158756 A1 WO2015158756 A1 WO 2015158756A1 EP 2015058137 W EP2015058137 W EP 2015058137W WO 2015158756 A1 WO2015158756 A1 WO 2015158756A1
Authority
WO
WIPO (PCT)
Prior art keywords
instrument
pivot point
substantially parallel
lines
straight lines
Prior art date
Application number
PCT/EP2015/058137
Other languages
English (en)
Inventor
Benoit Rosa
Emmanuel VANDER POORTEN
Original Assignee
Katholieke Universiteit Leuven
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Katholieke Universiteit Leuven filed Critical Katholieke Universiteit Leuven
Publication of WO2015158756A1 publication Critical patent/WO2015158756A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • B25J9/1687—Assembly, peg and hole, palletising, straight line, weaving pattern movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/302—Surgical robots specifically adapted for manipulations within body cavities, e.g. within abdominal or thoracic cavities
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/371—Surgical systems with images on a monitor during operation with simultaneous use of two cameras
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39246—Control position and orientation of handled object
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39389—Laparoscopic surgery, camera on center of operated part, view around, scale

Definitions

  • the present invention provides methods and devices for estimating a pivot point, more specifically an optimal pivot point (OPP), for instruments in minimally invasive surgery (MIS).
  • OPP optimal pivot point
  • MIS minimally invasive surgery
  • an optimal pivot point will likely not coincide with the outer body surface.
  • a point can be defined as a point along the axis of the incision - or at least close to the axis - at a specific depth and pose relative to the incision for which interaction forces upon the body wall when instruments pivot around this point will be minimal with respect to some relevant metric, where the metric to be minimized could be for example the average force amplitude, the maximal force or stress on the tissue over a period of time, the root mean square of the forces or other relevant metrics.
  • the instrument arm remote center is indicated by the thick, center black band on the instrument cannula.
  • the thin black band located 1cm from the instrument cannula tip should be used as the insertion depth reference and should be located at the peritoneum.
  • This approach therefore uses a fixed distance between the thin black band placed at the peritoneum and the robotic arm's RCM. It has been shown to be a practical approach as no increased prevalence of trocar-site hernias has been reported for robot-assisted interventions. However it suffers from several drawbacks.
  • the RCM is computed from the vectors of the cylindrical instrument centreline at multiple poses.
  • the method relies on the assumption that instrument's edge lines can be extracted with high precision beforehand, such as by putting a white stripe on the instrument shaft and placing the instrument on a black background to ensure a high image intensity gradient at the instrument's edges, which in real-life surgical situations is far from trivial.
  • OPP pivot point
  • the invention provides methods for determining a pivot point, preferably an optimal pivot point, the method comprising:
  • determining comprises following steps: ⁇ extracting straight lines from the ex-vivo images, referred to as arbitrary straight lines, resulting in extracted arbitrary straight lines
  • filtering said extracted arbitrary straight lines, wherein said filtering comprises keeping at least two pairs of substantially parallel arbitrary straight lines and classifying said substantially parallel arbitrary straight lines in instrument or background lines; - estimating or determining the pivot point by fitting said instrument lines based on a statistical scheme.
  • At least three ex-vivo images (N) are provided.
  • the invention provides methods for determining an advantageously optimal pivot point, the method comprising:
  • filtering comprises: ⁇ obtaining at least one pair of substantially parallel straight lines from a first ex-vivo image, resulting in a first set of at least one pair of substantially parallel straight lines;
  • the straight lines are advantageously extracted from the ex-vivo images by image analysis without pre-knowledge of the instrument's position, orientation and dimension.
  • the straight lines are advantageously extracted through an edge detection algorithm.
  • the extracted straight lines can hence be referred to as arbitrary straight lines.
  • the at least two ex-vivo images hence advantageously refer to images of different poses of the instrument about the pivot point.
  • the pivot point is advantageously determined based on at least two pairs of substantially parallel instrument lines originating from different ex-vivo images, i.e. at least two pairs of the substantially parallel instrument lines refer to a different pose of the instrument (about the pivot point).
  • substantially parallel can refer to parallel with a tolerance of +/- 2 degrees or less, advantageously +/- 1 degree or less, advantageously +/- 0.5 degrees or less.
  • instrument lines are dynamic lines, such as lines which are not parallel between different ex-vivo images.
  • the method further may comprise the step of obtaining at least one pair of substantially parallel straight lines from one or more additional ex-vivo images, resulting in one or more additional sets of (at least one pair of) substantially parallel straight lines, and further comparing said additional sets of substantially parallel straight lines.
  • the step of comparing comprises classifying the substantially parallel straight lines of the one or more additional sets in pairs of the substantially parallel instrument lines and the background lines.
  • pairs of the substantially parallel straight lines having lines which are parallel in at least three images, such as at least three subsequent images, are considered background lines.
  • the pivot point is determined based on an adequate combination of two or more of the first, second and additional sets of pairs of substantially parallel straight lines.
  • filtering the extracted arbitrary straight lines is performed statistically.
  • extracting arbitrary straight lines is performed using a Hough transform.
  • Embodiments of the invention provide a method wherein the ex-vivo images (N) are provided by at least one camera, preferably at least two cameras, placed statically at least 30 cm from the pivot point.
  • at least two cameras it is advantageous to place the cameras such that each camera images the instrument under a different orientation.
  • the pivot point is determined from the ex-vivo images of each camera/viewing orientation independently, which results in a two dimensional location of the pivot point, advantageously in an image plane of the camera.
  • Methods may comprise a step of determining the location of the pivot point in three dimensional space by triangulation of the two dimensional locations of the pivot point obtained from different viewing orientations/different cameras.
  • the distance (d) is determined by the dimensions of said instrument, such as a distance between edges of the instrument.
  • the instrument is advantageously elongate and advantageously comprises straight edges extending along a longitudinal axis. It will be convenient to note that the instrument advantageously comprises edges which are parallel, and (substantially) straight over at least a portion along the edge.
  • the instrument has (right) cylindrical shape. Other shapes, such as (right) prismatic with polygonal base, are possible.
  • the instrument can be part of a larger, e.g. longer assembly comprising flexible parts.
  • the instrument hence can refer to an advantageously rigid end, such as a distal end, of a generally flexible assembly, such as a catheter.
  • determining the pivot point is performed by a RANSAC fitting.
  • moving the instrument about the pivot point is performed by making the instrument follow a conical trajectory.
  • the conical trajectory has an apex of the conical trajectory advantageously located in proximity of the pivot point. Letting the instrument perform a conical trajectory allows to view the instrument in extreme positions regardless of the viewing orientation from which the camera(s) are observing.
  • the instrument is a surgical instrument or a surgical instrument rigidly attached to a passive instrument holder or an instrument held by a surgical robot or a surgical robot.
  • Embodiments of the present invention further comprise monitoring the quality of movement of the instrument around a pivot point.
  • monitoring the quality of movement comprises measuring and/or evaluating a parameter related to the interaction between the instrument and the environment during a period of time.
  • the environment can be a material body forming an opening in or at which the pivot point is to be determined.
  • the material body advantageously supports the instrument during motion.
  • the material body can be a patient's body.
  • An optimality of the pivot point can be calculated by evaluating a metric during a span of time of suitable length, the metric being advantageously representative of the interaction between the instrument and the environment as described above.
  • a metric can be for example an average force amplitude, a maximum force or stress applied on tissue, an average tissue load, the displacement or rotation of the tissue in the region close by the incision or any other relevant metric.
  • a method according to the present invention further may comprise the step of determining a location of the remote center of motion of a RCM system, such as a robot or manipulator, in free space (not in the incision point) and aligning the remote center of motion to the determined pivot point.
  • the RCM system may hold an instrument for e.g. performing a surgical operation.
  • a method according to the invention may be applied twice or more, both to determine the location of the pivot point, and to determine the location of the remote center of motion of the mechanism/system. Applying this method twice may allow inferring the relative displacement that is to be undertaken by the mechanism to align the remote center of motion with the pivot point.
  • aligning the instrument with the determined optimal pivot point enables the instrument's axis to maximally pass through the determined optimal pivot point which in the case that the instrument is held by a mechanism with any kind of remote center of motion this alignment would be done by making that remote center of motion maximally coincide with the determined optimal pivot point.
  • the instrument is fixed to the RCM apparatus when determining the pivot point, the RCM should be unlocked and free to move (in any direction).
  • a three-step procedure is proposed for aligning a remote centre of motion of an RCM apparatus/system to an optimal pivot point.
  • a pivot point advantageously an optimal pivot point is determined as described in the above methods, in e.g. an incision in a patient with an instrument (which e.g. can be held by hand, or by a free-to-move manipulator, such as an RCM manipulator).
  • the remote centre of motion of the RCM apparatus/system is determined in free space, outside of the incision. This can be done using methods as described in the present invention, such as by attaching an instrument to the RCM apparatus/system and applying methods according to the present invention to the instrument. Alternatively, other methods could be used, such as by tracking markers.
  • the remote centre of motion of the RCM apparatus/system is aligned on the (optimal) pivot point.
  • a method is provided further comprising a calibrating step, wherein said calibrating step comprises calibrating the determined optimal pivot point at regular intervals in time.
  • Preferred embodiments of the present invention may comprise measuring means to confirm the validity of the identified optimal pivot point at regular intervals in time, whether such measuring means consists of a camera that is pointed at the region of the optimal pivot point and detects abnormal deformation of the tissue which could be explained by the estimated optimal pivot point no longer being valid, or whether such measuring means would measure the interaction force between the instrument being moved about the earlier optimal pivot point and where such forces would be higher than expected which could be explained by the estimated optimal pivot point no longer being valid or whether the state of the estimated optimal pivot point no longer being valid is measured by any other means.
  • the method according to embodiments of the invention provides signaling means, which is foreseen to inform when a level of optimality of an identified optimal pivot point is lower or higher than a certain threshold.
  • the signaling means advantageously informs the operator, assistant, colleague or system about the optimal or oppositely of the suboptimal nature of the currently estimated pivot point as to give advice that the pivot point identification procedure can be halted in case a sufficient quality is reached or that oppositely the identification procedure and the instrument motion should be continued or repeated in order to reach an estimated pivot point of sufficient optimal quality.
  • said signaling means is foreseen to inform when the validity of a pivot point is lower or higher than a certain threshold.
  • the signaling means may warn the operator, assistant, colleague or system that the quality of the current pivot point is to be verified and/or advises to conduct the procedure (again) to determine the pivot point.
  • the method further comprise providing feedback to a user.
  • providing feedback can be done visually and/or by using sounds and/or by using a display and/or by using a laser based system and/or by providing kinesthetic, tactile and/or vibro- tactile feedback.
  • providing feedback may comprise guiding a user to the determined (optimal) pivot point.
  • specific guidance be it visual, haptic (kinaesthetic, tactile or vibrotactile), auditory or any other is provided to a user or operator, assistant, colleague or system about the manner to perform the movement of the instrument to identify the optimal pivot point so as to improve the convergence of the pivot point estimation allowing faster reaching of a pivot point with pre-defined level of optimality.
  • means for providing feedback to a user may comprise a user interface that provides guidance information to the user in order to align a device with the optimal pivot point.
  • Such guidance could simplify motion in the preferred direction and prevent or even prohibit motion in any other direction.
  • the user interface could display this information directly on the device that is being aligned or could employ any other means that is found convenient.
  • this robot could be programmed to generate low resistive forces in the preferred direction compared to high resistive forces in any other direction so that the user could steer the robot intuitively relying on haptic feedback to perform the alignment.
  • tactile or vibro-tactile or other types of feedback could also be supplied to the user.
  • moving an instrument about a pivot point to determine or estimate the location of the pivot point may be performed by a surgical robot that performs this procedure in an automated, safe and gentle way and wherein the instrument motion is programmed so as to allow good and reliable estimation of the pivot point while at the same time ensuring minimal interaction with the tissue and wherein this ensuring of minimal interaction with the tissue is done for example by directly measuring the interaction forces while executing the instrument motion or by observing the motion of the surrounding tissue while moving the instrument or by any other means.
  • the method further provides means to estimate physiologically-induced motion patterns of the tissue in the area of the pivot point and in the absence or in the presence of an instrument, wherein such physiologically-induced motion patterns are for example induced by heartbeat or breathing and a method wherein this physiologically-induced motion is superimposed in an appropriate manner upon the location of the pivot point so that the location of the pivot point is dynamically altered in a manner that is synchronized with the physiological phenomenon, wherein an instrument holding device is programmed so as to follow the dynamically altered pivot point location.
  • ex-vivo image relates to images taken outside an organism, e.g. the patient's body, and thus for instance of the surgical scene, e.g. the movement of an instrument outside of the organism.
  • ex-vivo images exclude endoscopic images taken from within a patient's body.
  • at least one camera is used to provide the ex-vivo images.
  • any camera which can provide line (and/or possibly depth) information can be used.
  • a stereo-camera a time-of-flight camera or camera's using structured light information can be used.
  • a system comprising lasers, e.g. a laser pointing or laser line generating systems can be used to provide feedback to a user, instead of for instance systems based on displays or guidance by emitting sounds and commands.
  • laser pointing or laser line generating systems can be used in methods or assemblies according to aspects of the present invention as follows: - determine the pivot point according to methods of the invention;
  • the robot's RCM position This could be done by detailed identification of the robot's forward kinematic chain or by for instance using image information, for instance using the same method as for the OPP when the robot holds an instrument and moves around its RCM.
  • An alternative could be to use one or several specific markers that are placed at known positions on the robot itself as is implemented on the robot prototype that is used in the experiments reported in this text.
  • the vector can for example be conveniently selected to match the dimensions of a cannula, so that the offset introduced by the cannula is conveniently incorporated, but any other convenient offset could be envisioned.
  • an instrument is moved according to an arbitrary trajectory.
  • a pivot point more specifically an OPP, can be reliably extracted.
  • the angles of the instrument poses are preferably spanning over at least ⁇ /6 radians in image space.
  • a conic trajectory is suggested, however it is merely provided as an example in order to explain the algorithm according to embodiments of the present invention.
  • the instrument is moved about a pivot point, wherein the instrument extends through the pivot point.
  • the instrument is preferably a surgical instrument which can be coupled to a robotic arm.
  • Embodiments of the present invention provide an improved method for estimating the position of a pivot point, and more preferably an optimal pivot point, for handling instruments in minimally invasive surgery.
  • Such knowledge is of particular importance for robotic-assisted surgery where robots need to rotate precisely around a specific point in space in order to minimize trauma to the patient's body wall and to achieve good position control.
  • RCM Remote Center of Motion
  • RCM Remote Center of Motion
  • the RCM point is typically manually and/or visually aligned.
  • soft-RCM software approaches can be employed to program instrument motion about a specific point in space
  • this misalignment might lead to intolerably high forces on the patient's body wall, increasing risk for post-operative complications.
  • the misalignment might lead to high interaction forces between the instrument and the patient's body wall which might cause damage to the instruments, which simply deform the instrument and as such complicate precise position or force control of the instrument within the body cavity or which simply deform or displace the body wall which ultimately might also deform or displace the anatomical structure that forms the target of the instrument action and as such complicate precise control over this anatomical structure.
  • vitreoretinal microsurgery on the eye.
  • vitreoretinal procedures micro-surgeons typically introduce instruments into the eye through the pars plana to work at the opposite inner side of the eye-ball namely at the retina.
  • the eye behaves as a spherical joint that is loosely hinged by a plurality of eye muscles in the eye socket.
  • the target area i.e. the retina is directly connected to the incision point on the eye-wall. If instruments do not precisely move around the pivot inside the incision of the eye wall even low interaction forces upon this wall might cause the eye to rotate and as such complicate precision task on the back of the eye as all actions need to be accomplished upon a moving target in such case. Similar problems can appear in other surgical domains as well.
  • embodiments of the present invention provide the use of computer vision and a lightweight calibration procedure to estimate a pivot point, more specifically the optimal pivot point.
  • At least one, for instance one or two pre-calibrated cameras, for example stereo cameras, looking at the surgical scenes can be used hereto.
  • the surgeon is asked to make some short pivoting movements with an instrument of his choice passing through the insertion point while camera images are being recorded.
  • the physical properties of an instrument rotating around a pivot point are exploited in a RANSAC scheme in order to robustly estimate the position of the OPP in the image planes, said position which can then be used to provide guidance to align the robot's RCM.
  • triangulation is used to estimate its position in 3D.
  • embodiments of the present invention provide a novel method which is cost- effective and reliable. Moreover, it can be used within a visual servoing approach in order to automatically or semi-automatically place the RCM point. Alternatively results can be simply displayed on a screen to provide guidance to the surgeon. In other embodiments an image- guided alignment method can be provided as well.
  • the present invention provides a computer program product for, if implemented on a control unit, performing a method according to aspects of the present invention.
  • the present invention provides a data carrier storing a computer program product according to the second aspect of the present invention.
  • data carrier is equal to the terms “carrier medium” or "computer readable medium”, and refers to any medium that participates in providing instructions to a processor for execution.
  • Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media.
  • Non-volatile media include, for example, optical or magnetic disks, such as a storage device which is part of mass storage.
  • Volatile media include dynamic memory such as RAM.
  • Common forms of computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, or any other magnetic medium, a CD- ROM, any other optical medium, punch cards, paper tapes, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereafter, or any other medium from which a computer can read.
  • Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.
  • the instructions may initially be carried on a magnetic disk of a remote computer.
  • the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
  • a modem local to the computer system can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal.
  • An infrared detector coupled to a bus can receive the data carried in the infra-red signal and place the data on the bus.
  • the bus carries data to main memory, from which a processor retrieves and executes the instructions.
  • the instructions received by main memory may optionally be stored on a storage device either before or after execution by a processor.
  • the instructions can also be transmitted via a carrier wave in a network, such as a LAN, a WAN or the internet.
  • Transmission media can take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications. Transmission media include coaxial cables, copper wire and fibre optics, including the wires that form a bus within a computer.
  • the present invention provides in transmission of a computer program product according to the second aspect of the present invention over a network.
  • the present invention provides assemblies for determining a pivot point, such as an optimal pivot point, the device comprising:
  • processing means adapted to perform a method according to embodiments of the present invention
  • said instrument is provided in a previously provided incision in a patient's body.
  • the assemblyfurther may comprise monitoring means, said monitoring means adapted to determine the quality of movement of the instrument around a pivot point.
  • the assembly further may comprise means for providing feedback to a user.
  • Embodiments of the present invention provide assemblies which are used as a module for and/or part of a robot for minimal invasive surgery. Particular and preferred aspects of the invention are set out in the accompanying independent and dependent claims.
  • Methods of performing minimally invasive surgery are also described. Methods comprise making an incision in a patient's body and determining a pivot point at the incision according to any one of the methods described above. Methods of performing MIS can comprise moving an apparatus comprising a remote centre of motion to position the remote centre of motion at the pivot point. Methods can comprise attaching a surgical instrument to the apparatus such that it pivots on the remote centre of motion, and performing a surgical intervention with the surgical instrument.
  • the instrument is inserted in a previously provided incision in a patient's body.
  • Fig. 1 schematically illustrates a perspective projection camera model employed.
  • the 3D double cone C described by the moving instrument projects onto a set of lines L on the image plane 3.
  • the apex of the cone C coincides with the pivot point PP and projects onto a point PI (Pivot point in Image) on the image 3.
  • Examples of instrument centerlines 1 are shown and their projection 2 in the image 3 as well.
  • Reference 4 denotes the camera's focal point.
  • Figs. 2 A-D illustrate a RANSAC implementation according to aspects of the invention and applied on data samples and their representation in the image plane(s) of the camera(s).
  • Fig. 2 A illustrates the extraction of a tentative model from two sets 20, 20' of substantially parallel lines;
  • Fig. 1 schematically illustrates a perspective projection camera model employed.
  • the 3D double cone C described by the moving instrument projects onto a set of lines L on the image plane 3.
  • the apex of the cone C coincides with the pivot point PP and projects onto
  • FIG. 2 B illustrates a check for consensus with data, wherein the outlier lines 21 , 22 are provided as dashed lines and the inlier lines 20, 20' as solid lines;
  • Fig. 2 C illustrates the best model found after the k iterations with all sets 20, 20', 20", 20"' of substantially parallel inlier lines and finally
  • Fig. 2 D illustrates refining the Pivot point in Image (PI), wherein the tentative estimate of PI (i.e. Pl b ) is provided by a circle and the refined estimate of PI (i.e. ⁇ is provided by a cross, by using all the inliers and only them).
  • PI Pivot point in Image
  • Fig. 3 A shows the first experimental setup used as an example to validate a method according to embodiments of the invention.
  • Fig. 3 B illustrates mapping with the CAD model using the transform 77 and the fiducial markers 12 and the position of the three insertion points 1 1 in the CAD model.
  • Fig. 3 C illustrates a reprojection of PI in the image plane using the inverse transform 77 "1 .
  • Figs. 3 D-F represent images of the instrument 15 in different poses about the insertion point 1 1 .
  • Fig. 4 illustrates the results of the estimation of the PP according to embodiments of the invention, wherein said estimation is performed on a first experimental setup after removing failed detections using the epipolar constraint.
  • the solid dots 12 represent fiducial markers positions that allow computing the validation frame (wireframe plan with an inclination) and the location of the insertion point (represented by the circles 1 1 ).
  • the points 10, surrounded by the circles 1 1 are the estimated PP using the method according to aspects of the invention.
  • Fig. 5 illustrates an embodiment of the present invention of providing feedback to a user, e.g. a surgeon, wherein a laser pointing assembly is provided.
  • One or more lasers 53 or sets of lasers are mounted on a RCM robot 51 or any convenient location in the operating theatre.
  • the robot 51 is mounted on a base 52 and advantageously holds the instrument 55.
  • This laser 53 or lasers are set (for instance using motors to control the angle of the laser beam or controlling mirrors to deflect the orientation of the laser beam) to point towards the point 54 obtained by adding the RCM with a 3D vector u.
  • This 3D vector u can be computed by computing the pivot point PP according to methods of the invention, by determining a point P on the outer surface of an entry port (e.g. an incision in a body wall 50) in which the instrument 55 is to be inserted, wherein the 3D vector u is the vector pointing from PP to P. Then, the user preferably aligns the projected laser point 54 with P. When this is the case, the RCM of the robot is aligned with the optimal pivot point PP.
  • Figs. 6 A-C represent images of different steps of line extraction, pruning and PI estimation on overlayed images using a RANSAC algorithm on a 5 mm black instrument on obstructed background.
  • lines are extracted using a Hough algorithm.
  • Fig. 6B shows the result after line pruning, which retains sets of substantially parallel lines. Each light coloured line in Fig. 6B has an associated dark coloured line which is substantially parallel.
  • Fig. 6C shows the best estimate 16 after application of a RANSAC algorithm on the lines of Fig. 6B.
  • the only relevant components of the device are A and B.
  • the terms first, second, third and the like in the description and in the claims are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.
  • the terms top, bottom, over, under and the like in the description and the claims are used for descriptive purposes and not necessarily for describing relative positions. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other orientations than described or illustrated herein.
  • Embodiments of a method according to the present invention comprise extracting straight lines from camera images and preferably uses a RANSAC scheme to differentiate the inliers -instrument lines providing information about the location of a pivot point— from the outliers — other straight lines detected in the field of view and that are not related to the location of a pivot point, preferably an optimal pivot point.
  • a pivot point is advantageously static. For a certain workable and practical time period, advantageously no major adjustments are made to the surgical scene and the location of the (optimal) pivot point is assumed to be static. This assumption is in line with the principle of a free-joint system described by Ortmaier et al cited above who estimate the PP-location for optimal control of his passive RCM-robot. Note that Ortmaier et al relies on dedicated instruments and sensors embedded into the robot to estimate a pivot point, which the present invention does not rely on. However if the pivot point is not static, aspects of the present invention provide methods to calibrate said PP in time and thus provide an optimal pivot point at all times.
  • moving an instrument about a pivot point comprises that an operator, for instance a surgeon moves said instrument, advantageously freely and gently in an incision with or without cannula, without a specific surgical task in mind, which will move the instrument about the optimal pivot point.
  • the instrument is advantageously moved about the incision (or pivot point) applying minimal torque to the instrument.
  • moving the instrument is advantageously performed such that the instrument follows a conical trajectory, i.e. the instrument is moved about a pivot point PP such that the envelope of the positions of the instrument's centerlines 1 is conical.
  • the instrument preferably describes a double cone C in 3D space. The apex of the cones preferably coincides with the PP and instrument centerlines will intersect with this point.
  • a perspective projection camera model is used.
  • the latter comprises that images captured by an ex-vivo camera can be modelled by a perspective projection model.
  • the camera can be calibrated beforehand or manufactured specifically in order to avoid image distortion.
  • C will project into the image plane 3 forming a set of lines L that intersect on the projection of the PP into the image plane 3.
  • This point will be referred to as a Pivot point in Image (PI) in the following (see for instance Fig.1 ).
  • an algorithm according to aspects of the invention to extract a PI from ex-vivo camera images is provided in the following. Contrarily to existing methods known in the art, the proposed invention advantageously does not rely on explicit tracking of the instrument throughout the images. Rather, a statistical scheme is preferably adopted. Since the surgeon moves the instrument freely, an adequately sized set of N ex-vivo images (for instance 100 to 200 images) with different positions of the instrument is acquired. These ex-vivo images, can advantageously go through the following processing pipeline: Line extraction step
  • Detecting straight lines for instance by a Canny edge detection step, wherein straight lines are extracted from within the ex-vivo images by looking at their representations in Hough Space. While being computationally costly, this operation can be performed online during image acquisition at a rate of for instance 30 fps.
  • Alternative to a Canny edge detection other image-based methods known in the art can be applied here to detect straight lines, for instance based on hue-characteristics of the image.
  • the following approach is advantageously used to tailor the method and extract relevant lines (for instance the instrument's edges) from the ex-vivo image. Since the instrument can be considered to be a long straight cylinder, it is likely that its edges will be among the longest lines that can be found in the images. The longer the lines, the larger the number of intersections or votes in Hough space that can be associated to such a line. Unfortunately, since there is no guarantee that the instrument edge lines are in reality the longest straight lines in the image (longer straight lines could come from other instruments, background lines from the operating table, lines from the ceiling and so on), rather than searching for the line with maximal amount of intersections, an appropriate threshold value a is employed and all lines with more votes than a will be withheld. Starting from a predetermined value, a is adjusted until the average number of detected lines M in the first, for instance 20, images falls in a predetermined range, for instance between 20 and 100.
  • the angle precision for parallel line extraction algorithms is preferably half a degree or less.
  • the angle precision can be chosen such that one balances the desirable features which one wants to achieve. For instance when considering a smaller angle precision, one should take a longer computational time in account and possible confusion (i.e. if for instance we have a 0.01 degree precision, we could detect a plurality of lines that are separated by 0.01 degrees that come from an edge of the instrument).
  • the angle precision is preferably set by taking into account the type of line that is to be detecting (width of the edge), the image noise and the distance between the camera and the instrument. Line pruning step, whereby substantially parallel lines are retained
  • instrument edge lines and other background lines that should be eliminated.
  • instrument lines are considered substantially parallel within preferably half a degree precision as indicated above.
  • this assumption would not be applicable when endoscopic images, thus in-vivo images, would be used, as these in-vivo images are taken at a closer distance ( ⁇ 30cm) from the instruments.
  • lines from the instrument don't project on the at least one camera into substantially parallel lines, because of the projective nature of the camera.
  • the tolerance is therefore dependent on the parameters of the camera, the distance to the scene and the instrument size.
  • the pivot point in Image is estimated, advantageously based on iteratively checking an instantaneous estimate of the PI for consensus with data.
  • RANSAC a statistical method for fitting data to a model in the presence of a large number of outliers as illustrated by Fischler et al in "Random sample consensus: a paradigm for model fitting with application to image analysis and automated” (1981 ), is preferably adopted to estimate the position of the PI, but methods such as GroupSac (by Ni et al in Computer Vision, 2009 IEEE 12th International Conference on (pp. 2193-2200), betaSac (by Meier et al in British Machine Vision Conference 2010) or any other statistical methods for fitting data known in the art could be applied as well, such as M-estimators and Least Median of Squares (LmedS).
  • RANSAC relies on the random selection of data for making a tentative model.
  • the method iteratively checks tentative models for consensus with the data in order to determine the best model, i.e. the model that has the greatest number of inliers within the data. Since this is a statistical scheme there is no guarantee that the best model which is found is actually accurate.
  • the process advantageously ends when the likelihood of finding a better model is less than a selected threshold ⁇ .
  • PI estimation mehtods such as RANSAC
  • Figs. 2 A-D A possible implementation of PI estimation mehtods, such as RANSAC, according to embodiments of the invention is depicted in Figs. 2 A-D. Each iteration preferably comprises following three steps:
  • PI 9 is made from two randomly selected pairs 20, 20' of parallel lines (Fig. 2A). PI 9 can be computed as the point in image-space that minimizes the distance to the four lines. Because an identical instrument is being observed, the pair of lines must have a same distance d between them.
  • a last computational step can also be applied to minimize the sum of squared distances to the set of inliers in order to refine the estimated position of PI, PI : where L s is a line belonging to the set of inliers and dist is the Euclidean distance operator.
  • Algorithms according to aspects of the invention such as the one described above advantageously provide the location of the projection of the pivot point in the image plane of at least one camera.
  • knowledge of the PP location in 3D (three- dimensional) space is more convenient.
  • the above method can be expanded towards 3D and metric distance values as follows.
  • a stereo-camera is used to estimate the PP position in 3D.
  • a stereo-camera can be simply made by for instance fixing together two planar image cameras, such as webcams.
  • intrinsic (i.e. perspective projection model and distortion parameters) and extrinsic (i.e. rotation and translation between the cameras) parameters are preferably calibrated.
  • this calibration is preferably done only once: the cameras are fixed relative to each other and focus and zoom are not changed.
  • the extrinsic parameters are then expressed in the reference frame of the stereo- camera and metric positions are estimated in this frame.
  • This section describes the implementation details and experimental validation of a method according to embodiments of the invention by providing examples that were conducted on an artificial abdominal wall.
  • the first experimental setup used to check the accuracy of the image processing algorithm in detecting a pivot point is depicted in Figs 3A-3C.
  • the setup is designed to provide a fixed and precisely known pivot point in space.
  • a rigid metallic frame 13 is clamped to a table.
  • a validation plate, here a wooden plate 14, is laser cut so that a 5 mm, a 8 mm and a 10 mm hole 1 1 are located at well-known positions with respect to four fiducial markers 12.
  • the different holes 1 1 are used as insertion point.
  • This design forces the inserted instrument 15 to rotate about a pivoting point 1 1 , similar to a real surgical practice, but without the variability induced by the compliant body wall, as shown in Figs. 3D-F.
  • the positions of the fiducial markers 12 are extracted from the images. The markers are used to compute the position of the insertion point 1 1 .
  • the perspective transformation 77 between the quadrangle formed by the fiducial markers in the image plane and the square they form in the CAD model of the wooden plate is used to this end.
  • the 3D position of the fiducial markers can then be computed by simple triangulation.
  • the second setup (not shown) is designed to investigate the precision of the pivot detection algorithm.
  • the experimental setup is constructed to simulate more closely the behaviour of a real body wall.
  • the same mockup base is used.
  • the wooden plate is now replaced by a flexible foam material simulating the body wall (referred to as pelvi trainer).
  • the stiffness of the used material is comparable to the one of the abdominal wall, but it is here homogeneous (i.e. does not reproduce the layered structure of the body wall).
  • a trocar is inserted through an incision and instruments are inserted in it. In this case, the actual position of the 77 in the images is not known. Given the limited set of experiments that are being conducted of the assumption of a static OPP, is considered justifiable.
  • the setup then gives insight in the robustness (precision) of the method.
  • the setup can be used to test the validity of the assumption that if a surgeon or any other staff moves an instrument freely and gently in the trocar point, without a specific surgical task in mind, he/she will move the instrument about the optimal pivot point. If this assumption is true and the users limit forces exerted on the flexible body wall, the inter-users variability should be low.
  • At least one, for instance one or two Logitech C920 USB webcams are used depending on the context (mono- or stereo-camera). Images are acquired using OpenCV, from the Open Source Computer Vision Library http://opencv.org, with a resolution of 640x480 pixels. Camera calibration is preferably done using the calibrateCamera and stereoCalibrate functions of OpenCV. A 18 x13 checkerboard pattern with squares of 12.0mm may be used. For better accuracy, the intrinsic parameters of each camera are first calibrated independently. Afterwards the extrinsic parameters for the stereo-camera are calibrated. Calibration is considered successful when the average reprojection error falls below 1 pixel. For each experiment, the protocol was as follows:
  • the mockup is prepared, either with the rigid frame or with the pelvi trainer;
  • the camera or stereo-camera is positioned at a distance of at least 30cm to 1 m from the surgical scene, providing ex-vivo images;
  • the user picks an instrument 15 of his choice; (4) the user inserts the instrument 15 in the according keyhole 1 1 , which refers to an incision, and moves it around while ex-vivo images are recorded by the camera;
  • Ni is preferably between 20 and 100.
  • the absolute average maximum number of substantial parallel lines N in the ex-vivo images would then be equal to Using Eq. (1 ) and (3) the number k of iterations in the worst-case scenario is then 184212.
  • most parasite lines are filtered in the second step of the algorithm and the number of iterations is much smaller, between preferably 100 and 3000 on average.
  • the RANSAC part of the algorithm is then computed in less than one second using a single thread on a 3 Ghz CPU.
  • Fig. 6A-C shows an example of PI estimation using the first experimental setup. Comparing Fig. 6A with Fig. 6B, shows the effect of a filtering stage based on determining sets of parallel lines, which removes most of the background lines. Fig. 6C shows application of the RANSAC algorithm to find a suitable estimate 16 of the PI .
  • Validation of the PI estimation using the first example setup illustrated that the RANSAC scheme, used in embodiments of the invention, allows one to find a good estimate of the PI.
  • the error between ⁇ and the ground truth information was found to be encouragingly low: the average error was 1 .13 pixels, with a maximum error of 2.51 pixels and a standard deviation of less than 1 pixel.
  • the success rate of our statistical recognition scheme was 98.9%.
  • Fig.4 presents the set of estimated PP 10 (after removing failed detections using the epipolar constraint) together with the wooden plate fiducial points 12 and the incisions 1 1 .
  • the exact location of the PI was performed for the second example, comprising the second experimental setup within a deformable body wall.
  • the focus here was to derive the precision of the entire method including errors induced by the flexible wall and the user manipulation of the instrument.
  • Six subjects were instructed to take an instrument of their choice, insert it into the 10 mm trocar and move it around for a few seconds while images were being recorded. The only instructions were to make large movements in order to describe a cone similar to the one of Fig. 1 and to grasp the instrument gently. No particular instruction about the speed of the gesture was given.
  • the six subjects did the experiment 15 times each, making a total of 90 experiments. For each run the 3D position of the OPP was estimated. Among those experiments, three experiments failed.
  • the failures were successfully detected using the epipolar constraint.
  • the size of the obtained cluster of points was studied. This parameter corresponds to the precision - rather than the accuracy - of the OPP detection method.
  • the centroid of the cluster of points was computed as the average of the X, Y and Z positions of each valid point in the cluster.
  • the centroid was located 19 mm under the wall outer surface, which is coherent with the fact that the wall of the mockup was 40 mm thick and consisted out of homogeneous material.
  • the distance D, to the centroid was computed for each point / ' of the cluster of points.
  • the average distance error to the centroid was found to be 3.54 mm (standard deviation 2.81 mm, 90th percentile 8.00 mm).
  • the discrepancy of the data around the centroid is significantly larger for this experiment than for the wooden frame experiment (unequal variances t-test, p ⁇ 0.01 ). This difference most certainly comes from the flexible nature of the simulated body wall and the manipulation of the user. In fact, the user is required to apply a null torque in order to be able to estimate the true OPP position. This is never the case in practice, hence a possible error in the determination of the OPP. Nevertheless, one can see that the discrepancy in the results remains low with a 90th percentile of the error at 8.00 mm.
  • Methods according to the invention further may comprise a calibration step which incorporates the case where the pivot point is dynamic, e.g. due to the fact that the patient is breathing and/or the incision point is not static.
  • the tolerances ⁇ and Z2 will need to be tuned accordingly.
  • Embodiments of the present invention provide a statistical method which can estimate the position of the optimal pivot point in minimally invasive surgery in order to align the RCM of a surgical robot or program a redundant surgical robot.
  • the proposed methods advantageously take only 5 to 20 seconds of time for the surgeon, most of the computations being done online.
  • Methods according to aspects of the present invention furthermore can be carried out with only a single external stereo camera placed statically at a convenient location approximately 30 cm, preferably at least 30 cm, preferably up to 1 m or more, preferably at least 30 cm to 100 cm from the instrument, in particular the entry port or pivot point.
  • the present invention uses images from outside the body (i.e.
  • the cameras' zoom and focus are fixed when performing methods of the invention.
  • the intrinsic and extrinsic camera parameters are advantageously pre-calibrated in order to avoid problems of radial distortion in images, a phenomenon that frequently complicates the use of laparoscopic images.
  • methods according to aspects of the invention can be used for surgical robots with an RCM using a visual servoing scheme that helps automatically determine and align the RCM with the optimal pivot point. It will be convenient to note that methods according to aspects of the invention can also be used for passive devices, such as manipulators, that implement RCMs to guide the surgeon in its positioning.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Robotics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Mechanical Engineering (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Manipulator (AREA)

Abstract

La présente invention concerne des procédés et des dispositifs permettant de déterminer la position d'un point de pivotement optimal pour des instruments de manipulation en chirurgie mini-invasive, comprenant les étapes consistant à fournir au moins deux images ex-vivo d'un mouvement de l'instrument dans le temps et à extraire des câbles d'instrument sensiblement parallèles desdites images séparées par une même distance pour déterminer le point de pivotement.
PCT/EP2015/058137 2014-04-16 2015-04-15 Procédé et dispositif permettant d'estimer un point de pivotement optimal WO2015158756A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GBGB1406821.7A GB201406821D0 (en) 2014-04-16 2014-04-16 Method and device for estimating an optimal pivot point
GB1406821.7 2014-04-16

Publications (1)

Publication Number Publication Date
WO2015158756A1 true WO2015158756A1 (fr) 2015-10-22

Family

ID=50845079

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2015/058137 WO2015158756A1 (fr) 2014-04-16 2015-04-15 Procédé et dispositif permettant d'estimer un point de pivotement optimal

Country Status (2)

Country Link
GB (1) GB201406821D0 (fr)
WO (1) WO2015158756A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018109851A1 (fr) * 2016-12-14 2018-06-21 オリンパス株式会社 Système manipulateur médical
EP3473202A4 (fr) * 2016-06-23 2020-02-19 Universidad De Malaga Procédé de manipulation d'un système robotique pour chirurgie mini-invasive
CN112932669A (zh) * 2021-01-18 2021-06-11 中山大学 一种执行视网膜层防渗漏隧道的机械臂控制方法
US11510552B2 (en) * 2017-06-23 2022-11-29 Olympus Corporation Medical system and operation method therefor

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008071014A1 (fr) * 2006-12-15 2008-06-19 Ao Technology Ag Procédé et dispositif pour verrouillage distal assisté par ordinateur de clous intramédullaires
EP2051647A1 (fr) * 2006-08-15 2009-04-29 AO Technology AG Procédé et dispositif pour fermeture distale assistée par ordinateur de clous intramédulaires
US20100331855A1 (en) * 2005-05-16 2010-12-30 Intuitive Surgical, Inc. Efficient Vision and Kinematic Data Fusion For Robotic Surgical Instruments and Other Applications

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100331855A1 (en) * 2005-05-16 2010-12-30 Intuitive Surgical, Inc. Efficient Vision and Kinematic Data Fusion For Robotic Surgical Instruments and Other Applications
EP2051647A1 (fr) * 2006-08-15 2009-04-29 AO Technology AG Procédé et dispositif pour fermeture distale assistée par ordinateur de clous intramédulaires
WO2008071014A1 (fr) * 2006-12-15 2008-06-19 Ao Technology Ag Procédé et dispositif pour verrouillage distal assisté par ordinateur de clous intramédullaires

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JASON T WILSON ET AL: "Evaluating remote centers of motion for minimally invasive surgical robots by computer vision", ADVANCED INTELLIGENT MECHATRONICS (AIM), 2010 IEEE/ASME INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 6 July 2010 (2010-07-06), pages 1413 - 1418, XP031855890, ISBN: 978-1-4244-8031-9 *
MICHAEL GSCHWANDTNER ET AL: "Track Detection for Autonomous Trains", 29 November 2010, ADVANCES IN VISUAL COMPUTING, SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, PAGE(S) 19 - 28, ISBN: 978-3-642-17276-2, XP019157644 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3473202A4 (fr) * 2016-06-23 2020-02-19 Universidad De Malaga Procédé de manipulation d'un système robotique pour chirurgie mini-invasive
WO2018109851A1 (fr) * 2016-12-14 2018-06-21 オリンパス株式会社 Système manipulateur médical
US11540699B2 (en) 2016-12-14 2023-01-03 Olympus Corporation Medical manipulator system
US11510552B2 (en) * 2017-06-23 2022-11-29 Olympus Corporation Medical system and operation method therefor
CN112932669A (zh) * 2021-01-18 2021-06-11 中山大学 一种执行视网膜层防渗漏隧道的机械臂控制方法
CN112932669B (zh) * 2021-01-18 2024-03-15 广州市微眸医疗器械有限公司 一种执行视网膜层防渗漏隧道的机械臂控制方法

Also Published As

Publication number Publication date
GB201406821D0 (en) 2014-05-28

Similar Documents

Publication Publication Date Title
US11135020B2 (en) Image processing device and method, surgical system, and surgical member
US11357581B2 (en) Method for using a physical object to manipulate a corresponding virtual object in a virtual environment, and associated apparatus and computer program product
US10675098B2 (en) Configuration marker design and detection for instrument tracking
KR101650161B1 (ko) 이미지 내에서 수술기기를 찾기 위한 기준 마커 디자인 및 탐지
EP2967297B1 (fr) Système de validation dynamique et de correction d'enregistrement pour une navigation chirurgicale
Iyer et al. A single arm, single camera system for automated suturing
US11896441B2 (en) Systems and methods for measuring a distance using a stereoscopic endoscope
WO2015149041A1 (fr) Visualisation tridimensionnelle quantitative d'instruments dans un champ de vision
Rosa et al. Estimation of optimal pivot point for remote center of motion alignment in surgery
Dong et al. Robust trocar detection and localization during robot-assisted endoscopic surgery
US11443501B2 (en) Robotic surgical safety via video processing
US10154882B2 (en) Global laparoscopy positioning systems and methods
WO2015158756A1 (fr) Procédé et dispositif permettant d'estimer un point de pivotement optimal
US20240081921A1 (en) System and method for verification of conversion of locations between coordinate systems
US11832883B2 (en) Using real-time images for augmented-reality visualization of an ophthalmology surgical tool
Lee et al. Simultaneous optimization of patient–image registration and hand–eye calibration for accurate augmented reality in surgery
EP4203762A1 (fr) Tampon de localisation entourant au moins une partie de l'oeil d'un patient et ayant des éléments de suivi optique
US20200205911A1 (en) Determining Relative Robot Base Positions Using Computer Vision
US20230139402A1 (en) Systems and methods for registration feature integrity checking
US20220108475A1 (en) Camera calibration using fiducial markers on surgical tools
US20230252681A1 (en) Method of medical calibration
Yang et al. An algorithm for accurate needle orientation
Dehghani et al. Colibri5: Real-Time Monocular 5-DoF Trocar Pose Tracking for Robot-Assisted Vitreoretinal Surgery
Navarro et al. Enhancing perception in minimally invasive robotic surgery through self-calibration of surgical instruments
KR20190079870A (ko) 최소 침습 수술을 위한 수술로봇 시스템 및 이를 이용한 충돌방지 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15719984

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15719984

Country of ref document: EP

Kind code of ref document: A1