JP2021087665A - Surgery support system and method for acquiring surface information thereof - Google Patents

Abstract

To provide a surgical instrument for controlling and detecting a contact angle of a positioning probe on an anatomical surface.SOLUTION: A surgery support system includes: an instrument having a tool and a manipulator connected to the tool; a space sensor system for detecting space information on the tool; and a computer system for operating a motion state of the manipulator. A method for acquiring surface information by the surgery support system includes: a process for defining a target area and a plurality of reference points on a virtual anatomical model of an object; a process for urging a user to generate sampling information by sampling a plurality of sampling points on the object corresponding to the reference points by using the instrument; and a process for designating the sampling information as surface information on the sampling points. Each piece of the sampling information includes coordinates of the sampling points, a contact angle of the tool in the sampling points, and parameters related to the contact.SELECTED DRAWING: Figure 1

Description

The present invention relates to a surgical support system. More specifically, it relates to a method of alignment that improves the accuracy and accuracy of computer-assisted surgery performed by a surgical assistance system.

Many surgeries require high manual accuracy on the part of the surgeon. For example, orthopedic surgery involves the surgeon placing the bone of interest at the correct location and angle in order for the surgeon to fit a given implant into the bone or to shape the bone to create the desired geometric profile. Requires milling, drilling, or sawing. Such surgery is usually performed freehand by the surgeon holding a particular surgical instrument and following an anatomical locus. Therefore, the accuracy of surgery depends on the skill of the surgeon in following a given surgical plan with handheld surgical instruments.

Taking advantage of information technology and robotics, computer-assisted surgery has provided a reliable option in improving the accuracy and accuracy of surgery. Computer-assisted surgery utilizes computer technology to visualize the surgical field in a preoperative virtual environment and represent a surgical concept that enables more accurate preoperative diagnosis and clear surgical planning. In computer-assisted surgery, patient alignment is a virtual three-dimensional (3D) image collected by computer medical imaging such as computed tomography (CT) or magnetic resonance imaging (MRI). It is an important preoperative procedure that correlates the location of the dataset with the location of the patient. Patient alignment eliminates the need to keep the patient in the same exact position during preoperative scans and surgery, ensuring the geometric accuracy of surgery.

Traditional alignment methods primarily include reference alignment and surface matching alignment. Reference alignment registers a particular anatomical region with a computer-assisted navigation system by detecting multiple reference markers attached to anatomical landmarks and defining the anatomical region. Surface matching alignment is performed by using a mechanical or ultrasonic probe to identify the coordinates of a set of points on the anatomical surface structure of the surgical field. Therefore, it provides higher accuracy in spatial perception and reduces surgical invasiveness over reference alignment.

In conventional surface matching alignment, the probe is connected to a sensor to detect mechanical contact between the probe and the anatomical surface. For example, Shen et al. Disclose in US Pat. No. 8,615,286 a device and method for locating a patient's bone surface. This device and method utilizes a force sensor installed at the base of a thin probe to detect the resistance of the material encountered by the probe and engage the probe with bone or soft tissue, or have a different hardness than the probe. Identify the type of tissue engagement.

However, in actual surgery, the probe tends to slide off the intended contact surface due to lack of control over the precise contact angle of conventional thin probes, or changes in hardness between different layers of tissue. Therefore, the accuracy of alignment is significantly reduced. None of the existing techniques provide a means by which the contact angle of the probe on the anatomical surface can be controlled or the effectiveness of mechanical contact between the probe and the surface can be determined. ..

An object of the present invention is to provide a surgical instrument that controls and detects the contact angle of an alignment probe on an anatomical surface.

Another object of the present invention is to provide a alignment method for surgical instruments that confirms the mechanical contact between the alignment probe and the anatomical surface.

Embodiments of the present invention provide a surgical support system. This system is electrically connected to the tool and an instrument with a manipulator connected to the tool, a spatial sensor system that detects the spatial information of the tool, and the instrument, the spatial sensor system, and the user interface, and is detected by the spatial sensor system. Includes a computer system that manipulates the manipulator's motion state according to the spatial information of the tool.

In a preferred embodiment, the instrument of the surgical assistance system further comprises a force sensor that detects the force and / or torque maintained by the tool.

In a preferred embodiment, the force sensor is located between the tool and the manipulator and / or within the tool.

In a preferred embodiment, the contact-related parameters are the force maintained by the tool, the torque maintained by the tool, the output power of multiple actuators of the manipulator, and the stable contact between the tool and one sampling point. Includes duration.

Another embodiment of the present invention provides a method of acquiring surface information for alignment by a surgical support system. The method is (S1) a step of defining a target region on a virtual anatomical model of interest and a plurality of reference points of the target region by a computer system; (S2) reference using an instrument via a user interface. A process that prompts the user to generate sampling information by sampling a plurality of target sampling points corresponding to the points, and each sampling information is detected by the coordinates of one sampling point and a spatial sensor system. A step of including the contact angle of the tool at one sampling point and the parameters related to the contact; (S3) a step of designating the sampling information as the surface information of the sampling point by the computer system;

In a preferred embodiment, sampling points are sampled by allowing a computer system to manipulate the manipulator's motion state to control sampling points one at a time so that the tips of the probes come into contact. ..

In a preferred embodiment, prior to step S1, the following steps are further included: the step of defining a plurality of schematic reference points of the target region on the virtual anatomical model by a computer system; the instrument via the user interface. The process of encouraging a user to generate schematic spatial data by using it to sample multiple schematic sampling points of interest corresponding to a reference point; the process of assigning the schematic spatial data to a virtual anatomical model by a computer system.

In a preferred embodiment, the approximate sampling points are configured by allowing the computer system to manipulate the manipulator's motion state to control the approximate sampling points one at a time so that the tips of the probes come into contact with each other. It will be sampled.

In a preferred embodiment, after step S2, the following steps are further included: (S21) a step of verifying the current sampling information; (S22) a step of checking the sampling state; (S23) a step of filtering the sampling information.

In a preferred embodiment, step (S21) further comprises: (S211) determining whether the parameters contained in the current sampling information meet at least one sampling criterion; (S212) parameter is sampling criterion. If the condition is satisfied, the current sampling information is indicated by the first mark, and if the parameters do not meet the sampling criteria, the current sampling information is indicated by the second mark; (S213) The current sampling information is indicated by the first mark. When indicated by, the process proceeds to step S22, and when the current sampling information is indicated by the second mark, the process proceeds to step S2.

In a preferred embodiment, the sampling criteria are that the force maintained by the tool is greater than or equal to the force threshold, that the lateral force maintained by the tool is less than or equal to the lateral force threshold, and that the tool is maintained. The torque is less than or equal to the torque threshold, the output power of multiple actuators of the manipulator is greater than or equal to the power threshold, and / or the duration of stable contact between the tool and one sampling point is greater than or equal to the time threshold. including.

In a preferred embodiment, step (S22) includes: (S221) determining whether sampling information for all sampling points is indicated by the first mark; (S222) for all sampling points. When the sampling information is sampled, the process proceeds to step S23, and when at least one sampling information of all sampling points is not sampled, the process proceeds to step S2.

In a preferred embodiment, step S23 is performed by screening the sampling information indicated by the second mark from the sampling information indicated by the first mark.

In a preferred embodiment, after step S3, the following steps are further included: (S4) the step of assigning surface information to the virtual anatomical model and aligning the virtual anatomical model with the position of the coordinate system; (S5) surface. The step of refining the virtual anatomical model according to the information; and (S6) the step of updating the operation plan according to the surface information and the refined virtual anatomical model.

According to various embodiments of the invention, the surgical assistance system provides an accurate and efficient method for alignment. The method defines a target area on the surface of the subject and a set of reference points distributed within the target area that covers multiple surface features of the surface and is a machine between the probe and the surface of the surgical instrument. Verify target contact and effectively improve the accuracy and accuracy of computer-assisted surgery.

The accompanying drawings show one or more embodiments of the invention and, together with the present specification, illustrate the principles of the invention. Wherever possible, the same reference numerals are used throughout the drawing to indicate the same or similar elements of the embodiment.

According to common practice, the various described features are not drawn to a certain scale and are drawn to emphasize the features related to this disclosure. Similar reference numerals indicate similar elements throughout the drawings and text.

It is a block diagram of the operation support system which is an embodiment of this invention. It is the schematic of the operation support system which is an embodiment of this invention. It is a perspective view of the handheld instrument of the operation support system which is an embodiment of this invention. It is a side view of the handheld instrument of the operation support system which is an embodiment of this invention. It is a side view of the manipulator of the handheld instrument of the operation support system which is an embodiment of this invention. It is a side view of another manipulator of the handheld instrument of the operation support system which is an embodiment of this invention. It is the schematic of the operation state of the operation support system which is an embodiment of this invention. It is the schematic of the operation state of the space sensor system of the operation support system which is an embodiment of this invention. It is a flowchart of the operation method of the operation support system which is an embodiment of this invention. It is the schematic of the snap of the alignment process of the operation support system which is an embodiment of this invention. It is a flowchart of the process of acquiring the surface information for the alignment of the operation support system which is an embodiment of this invention. It is the schematic of the target area and the reference point generated by the operation support system which is an embodiment of this invention. It is the schematic of the snap of the alignment process of the operation support system which is an embodiment of this invention. It is a flowchart of the process of acquiring the surface information for the alignment of the operation support system which is an embodiment of this invention. It is a flowchart of the process of verifying the sampling information for the alignment of the operation support system which is an embodiment of this invention. This is an example of filtering sampling information and acquiring surface information for positioning the surgical support system according to the embodiment of the present invention.

Hereinafter, the present invention will be described more fully with reference to the accompanying drawings showing various exemplary embodiments of the invention. However, the present invention can be implemented in many different embodiments and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided so that the present disclosure is thorough and complete and the scope of the present disclosure is fully communicated to those skilled in the art. Throughout, similar reference numbers indicate similar elements.

The terms used herein are for the purposes of describing only certain embodiments and are not intended to limit the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly indicates that they are not. The terms "comprises" and / or "comprising", or "includes" and / or "including", or "has" and / or "having" Is used herein to identify the presence of the features, regions, integers, processes, behaviors, elements, and / or components described, but one or more other features, regions, integers, processes, behaviors. It will be further understood that it does not preclude the existence or addition of, elements, components, and / or groups thereof.

It will be understood that the terms "and / or" and "at least one" include any combination and all combinations of one or more of the related listed items. Terms such as first, second, and third may be used herein to describe various elements, components, areas, parts, and / or parts, but these elements, components, areas, parts, etc. It will also be understood that and / or parts should not be limited by these terms. These terms are used only to distinguish one element, component, area, part or part from another element, component, area, layer or part. Thus, the first element, component, region, part or part described below may be referred to as a second element, component, area, layer, or part without departing from the teachings of the present disclosure. ..

Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. In addition, terms such as those defined in commonly used dictionaries should be construed as having meaning consistent with their meaning in the context of the relevant technology and the present disclosure and are expressly herein. Unless defined, it will be understood that it is not interpreted in an ideal or overly formal sense.

See FIGS. 1 and 2. In embodiments of the invention, the pre-surgery and surgical support system 1000 in surgery includes surgical hardware linked to electronic modules and processor executable instructions. The surgical support system 1000 includes an instrument system 1100, a space sensor system 1500, a user interface 1800, and a computer system 1700 electrically connected to the instrument system 1100, the space sensor system 1500, and the user interface 1800. In one embodiment, the surgical support system 1000 allows a user (eg, a surgeon) to perform surgery on a subject (eg, a patient) with the instrument system 1100 with reference to the user interface 1800. At least one medical imager 1910 communicates with the surgical support system 1000. The medical imager is configured to acquire a medical image of the subject and transmit the image to the surgical support system 1000. Spatial sensor system 1500 is configured to generate spatial information of objects and environments. The computer system 1700 generates a virtual anatomical model according to the medical image, generates a surgical plan according to the virtual anatomical model, tracks the surgical environment according to the spatial information received from the spatial sensor system 1500, and exercises the manipulator 1210. It is configured to change the state of motion of the control or manipulator. The user interface 1800 visualizes the anatomical model and allows the user to navigate the surgical field according to the surgical plan.

As shown in FIG. 2, the instrument system 1100 of the surgery support system 1000 includes a handheld instrument 1200 that performs surgery on a subject. In one embodiment, the instrument system 1100 may further include a support arm 1450 connected to the instrument 1200 to reduce the weight load on the user's hands and optionally provide more operational stability during surgery. In another embodiment, the support arm 1450 coupled to the instrument 1200 comprises at least one actuator. According to the control signal generated by the computer system 1700, the actuator may change the motion state of the support arm 1450 and move the instrument 1200 to a designated position.

See FIGS. 3 and 4. According to one embodiment, the handheld appliance 1200 includes a tool 1250, a tool stand 1260, a manipulator 1210, and an appliance housing 1280. Tool 1250 is configured to contact or modify the anatomical surface of a body part of interest. One end of the tool 1250 and the manipulator 1210 are connected to the tool installation table 1260, and the tool 1250 is stably connected to the manipulator 1210. The manipulator 1210 is a mechanism controlled by a computer system 1700 that manipulates the position and orientation of the tool 1250. The instrument housing 1280 is coupled to the manipulator 1210 to accommodate at least a portion of the manipulator 1210 and provide one or more handles 1284 that allow the user to hold and operate the instrument 1200 during the operation of the surgical assistance system. ..

In one embodiment, the tool 1250 may be a probe or indicator that contacts or evaluates the anatomical site of interest and detects the structure or condition of the anatomical site. The Tool 1250 is a drill bit, bar, curette, saw, screwdriver, or other commonly used in surgery to modify or remove part of the tissue at an anatomical site by drilling, milling, cutting, or scrapping. It may be a tool. In some embodiments, the tool 1250 is a mechanical, optical, or ultrasonic probe that performs surface matching alignment. The tool may be, but is not limited to, a rigid probe, a pressure sensor, a piezoelectric sensor, an elastomer sensor, an optical camera, a laser scanner, or an ultrasonic scanner.

In one embodiment, the tool pedestal 1260 is connected to the tool 1250 and a first aspect of the robot control platform 1230 of the manipulator 1210. The tool installation base 1260 includes a tool adapter 1265 and a motor 1270 connected to the tool adapter 1265. The tool adapter 1265 may be a clamp or other fixed structure that holds one end of the tool 1250 and avoids dislocation of the tool during operation. The motor 1270 may be a direct current (DC) motor or an AC (AC) motor that converts electrical energy into mechanical energy to generate linear force or rotational force to drive the tool 1250. In an alternative embodiment, the motor may be placed at the rear end of the appliance to reduce the load on the manipulator 1210 during operation of the appliance and redistribute the weight of the appliance 1200 to improve user argonomics. .. Further, as shown in FIGS. 5 and 6, the tool pedestal 1260 is connected to the first aspect of the platform 1230 and is a force sensor 1235 that detects the force and / or torque maintained by the tool 1250 during surgery. May further be included. In other embodiments, the force sensor 1235 may be located on the probe or tool of the instrument. Alternatively, the instrument 1200 may further include another force sensor (not shown) located on the probe or tool. The force sensor may be, but is not limited to, a strain gauge, a force sensitive resistor, a pressure converter, a piezoelectric sensor, an electroactive polymer, or an optical fiber bending sensor.

In one embodiment, the manipulator 1210 is connected to a platform 1220, a platform 1230 connected to a tool installation platform 1260, a second side of the platform 1230 away from the tool 1250, and a first aspect of the platform 1220 facing the platform 1230. Includes a plurality of joints 1245a, 1245b attached to the platform 1230 and a plurality of actuators 1240 connected to a pedestal 1220 on a second side surface of the pedestal 1220 away from the platform 1230. As shown in FIG. 4, the pedestal 1220 may be fixed on or housed in the appliance housing 1280. The manipulator 1210 may be a parallel manipulator such as a Stewart manipulator with 6 degrees of freedom (DOF) for high spatial efficiency and high maneuverability. In addition, preferably the manipulator is made of stainless steel or carbon fiber, allowing the manipulator 1210 to have sufficient durability against the forces and / or torque generated from the tool 1250 in contact with the subject during surgery. The manipulator is placed in a specific mechanical structure.

In one embodiment, the joint of the manipulator 1210 may be, but is not limited to, a rotary joint, a prism joint, a spherical joint, a universal joint, a cylinder joint, or any combination thereof that allows for the desired DOF. .. As illustrated in FIGS. 5 and 6, the manipulator 1210 with a common Stewart platform with six DOFs has a universal joint 1246 and a spherical shape to allow a wide range of motion and various motion states of the manipulator 1210. It may include a joint 1247. The manipulator 1210 may further include a plurality of connectors. Each connector connects with one joint 1245a and one joint 1245b, allowing a wider range of movement of the tool 1250.

In one embodiment, the actuator 1240 of the manipulator 1210 connected to the base 1220 on the opposite side of the joint drives the joint and, if there is a connector, is configured to move according to a control signal transmitted from the computer system 1700. .. In an alternative embodiment, the actuator 1240 and the joint may be located on the same side as the base 1220. As illustrated in FIG. 6, the actuator 1240 is arranged between the base 1220 and the platform 1230, and each of the actuators 1240 is joined by a universal joint 1246 and a spherical joint 1247. For high accuracy and strong sustainability, the plurality of actuators 1240 may be linear actuators.

See again FIGS. 3 and 4. In one embodiment, in addition to accommodating the manipulator 1210 and providing a handle, the instrument housing 1280 allows the user to guide, stop or adjust the movement of the tool 1250, or perform other functions of the instrument 1200. The control module 1282 for enabling the above may be further included.

In one embodiment, the handheld instrument 1200 may be used with a calibrator 1300 configured to calibrate the motion state of the manipulator 1210 with respect to the instrument housing 1280 to ensure the geometric accuracy of the instrument 1200.

In one embodiment, the instrument 1200 may include at least one inertial measuring unit that detects the acceleration, velocity, displacement, angular velocity and / or angular acceleration of the instrument 1200.

See FIG. 7. The instrument 1200 may be used in conjunction with the trocar 1400, especially in minimally invasive surgery, to provide an entrance to the body for the instrument 1200's tool 1250 to reach the anatomical site of interest. In an alternative embodiment, the trocar 1400 may be detachably coupled to the platform 1230 of the manipulator 1210 to allow the trocar 1400 and the tool 1250 to enter the anatomical site simultaneously.

Here, reference is made to FIG. According to embodiments of the present invention, the spatial sensor system 1500 of the surgical support system 1000 is configured to be capable of detecting and tracking spatial information (eg, position and orientation) of at least one target object. To. The spatial sensor system includes at least one spatial marker frame 1550 detachably attached to the target object and a spatial sensor 1510 having at least one camera receiving a signal transmitted from the spatial marker frame 1550.

As illustrated in FIGS. 7 and 8, the target object can be an instrument 1200, a trocar 1400, or a selected anatomical site. In one embodiment, the spatial marker frame 1550 comprises a plurality of markers 1555 that emit electromagnetic signals, sound waves, heat or other perceptible signals, and an adapter 1560 that is detachably attached to a target object that holds the markers 1555. including. The adapter allows the target object to be tracked by the spatial sensor 1510. In another embodiment, the spatial sensor system 1500 may further include a signal generator (not shown) located on or in a predetermined position on the spatial sensor 1510. As a result, the signal transmission by the marker 1555 may be active or passive. In other words, the signal emitted by the marker 1555 may be generated by the marker sphere, or the marker 1555 may be covered with a reflective material and the signal generated by the signal generator may be reflected by the marker 1555 on the spatial sensor 1510. ..

In one embodiment, the signal received by the spatial sensor 1510 is transmitted to the computer system 1700 and converted into the detected spatial coordinate system and spatial information of the target object by triangulation or other transformation algorithms. Further, as illustrated in FIG. 8, the markers 1555 of the spatial marker frame 1550 may be arranged on the adapter 1560 in a particular pattern. Therefore, the computer system 1770 can generate the direction information of the target object accordingly. The computer system 1700 may generate control signals according to spatial and directional information to control the movement of the manipulator 1210 of the instrument 1200 or change the motion state of the manipulator. Alternatively, the computer system may generate the instructions shown on the user interface 1800 to prompt the user to move the appliance 1200 to a specified position or direction.

According to an embodiment of the present invention, the computer system 1700 of the surgical support system 1000 includes a processor and a storage unit. The processor may be a general purpose processor, a special purpose instruction set processor, or a special purpose integrated circuit that performs operations on a data source such as a storage unit or other data stream. For example, the processor is an ARM-based processor or an 8086x processor. In some embodiments, the processor may further include multiple digital or analog inputs / outputs and may be a real-time operating system (RTOS) processor. The storage unit may store digital data allocated by the processor for immediate use in the computer system. Storage units are volatile or dynamic random access memory (DRAM) and static random access memory such as flash memory, read-only memory (ROM), programmable read-only memory (PROM), and erasable programmable read-only memory (EPROM). It may be non-volatile such as (SRAM).

According to one embodiment, the user interface 1800 includes at least one output device that presents information to the user and at least one input device. The information presented by the user interface 1800 is a surgical plan, a two-dimensional (2D) or 3D reconstructed image, a 2D or 3D drilling state (eg, tool position, angle, depth, or bend), tool compensation. It may include, but is not limited to, range, user guidance, warning areas, notification of tool bias from the surgical plan, and notification of the tool's power sustainability limit. The output device may be a display, a light indicator, or other visual means. Alternatively, the output device may be a speech synthesizer or other audio means, or may further include a speech synthesizer or other audio means. The input device can convert the command input by the user into an electric signal. The input device may be a pedal, keyboard, mouse, touch panel, voice recognition interface, or gesture recognition interface.

See FIG. According to an embodiment of the present invention, a method of performing computer-assisted surgery by the surgery support system 1000 includes a step of receiving a plurality of medical images from (S110) medical imager 1910 and (S120) a three-dimensional virtual according to the medical image. A step of generating an anatomical model, (S130) a step of urging the user to indicate a position of interest on the virtual anatomical model, and (S141) a virtual anatomical model obtained from a medical image, instructions. A step of generating a surgical plan according to position and physiological and / or pathological information, a step of encouraging the user to start the operation according to the surgical plan (S160), and a step of assisting the user during the operation (S170). And, including.

In step S110, the medical imager 1910 is a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, or any other commonly used image capable of acquiring a continuous cross-sectional image of the scanned object. It may be a medical imaging device. In a preferred embodiment, as shown in FIG. 10, the marker patch 1600 is attached to the subject near the intended surgical site when the medical image is taken, facilitating the positioning of the surgical environment. Specifically, the marker patch 1600 comprises at least one marker 1555 detectable by the spatial sensor 1510 of the spatial sensor system, and a plurality of reference markers 1610 that cause marking on the image captured by the medical imager 1910. May include. The reference marker 1610 may be made of lead, iron, calcium, or other radiation opaque metal. Thus, as illustrated in FIG. 10, if the marker patch 1600 is attached to the subject when taking a medical image, the resulting virtual anatomical model 2110 will have multiple radiation opaque spots corresponding to the reference marker 1610. Will be included.

In another embodiment, the markers 1555 on the marker patch 1600 may be placed concentrically with the reference marker 1610 to avoid signal discrepancies caused by changes in the surface contour of the subject. Alternatively, the marker patch 1600 is placed with a material that is optically readable by the spatial sensor system 1510 and is radiation opaque to the medical imager device 1910 to ensure high consistency between the acquired signals. You may.

See FIG. 9 again. In step S130, the user is prompted to indicate one or more sites of interest on the virtual anatomical model 2110 via user interface 1800. The site of interest may include the intended surgical site or specific anatomical landmarks or surface features. The user can also label or define specific landmarks or surface features on the virtual anatomical model. In step S141, the surgical plan generated by the method may include surgical details such as the approach position and angle of the tool 1250, the depth and path of planned drilling, the proposed type of tool, and the proposed type of screw. ..

In step 160, after the surgery plan is generated, the computer system 1700 prompts the user to start the surgery according to the surgery plan. The user can adjust or edit the surgery plan before the surgery begins. In step S170, the surgical support system 1000 assists the user during the planned surgery by adjusting the motion state of the manipulator 1210 according to the spatial information of the tool detected by the spatial sensor system 1500, via the user interface 1800. Notify the user. In addition, in some embodiments, medical images are taken during surgery to monitor the location, angle, and depth of the perforated path to ensure compliance with the surgical plan and to ensure compliance with the new surgical plan. Helps determine the need to redefine or recalibrate the instrument.

In step S130, after the user has selected a position of interest in the virtual anatomical model, the method of one embodiment is (S151) a step of acquiring surface information of a plurality of sampling points on the anatomical site of interest. (S152) A step of assigning surface information to the virtual anatomical model and matching the virtual anatomical model with the position of the coordinate system established by referring to the spatial information acquired by the spatial sensor system 1500. It may also be included.

Specifically, as shown in FIG. 11, the step S151 of acquiring the surface information of the sampling points according to one embodiment generates (S251) a step of generating a sampling pattern and (S252) a step of generating sampling information of the sampling points. This includes a step of encouraging the user and a step of designating sampling information as surface information of the sampling point (S2521).

In one embodiment, step S251 of generating a sampling pattern includes step (S2510) of defining a target region and a plurality of reference points on a virtual anatomical model. As shown in FIG. 12, after the user has indicated a site of interest on the virtual anatomical model 2110 via user interface 1800, the computer system 1700 has a target region 2120 and a plurality of reference points 2130 within the target region. , May be defined. Preferably, the target region 2120 is a region with characteristic or representative surface features (eg, the facet or spinous process of the spine), or a trocar accessible region. In at least one embodiment, the reference points 2130 are distributed within the target region 2120 so that the plurality of surface features are covered by the reference points 2130. Preferably, the amount of reference point 2130 may be 50 or more. Reference points 2130 may be arranged orthogonally, concentrically, spirally, or preferably in other patterns covering a significant number of surface features. In some embodiments, the target accuracy of the reference point 2130 (ie, the distance between two adjacent reference points 2130) may be set to 3-4 times the resolution of the spatial sensor 1510. For example, if the spatial sensor 1510 has a resolution of 0.25 mm, the distance between two adjacent reference points may default to 1 mm. In a preferred embodiment, the target accuracy is set to about 0.3 mm. In addition, reference points 2130 may be prioritized according to local physiological characteristics such as bone density calculated from medical images to define sampling pathways. The user may also manually define or adjust the target area 2120 and the amount and position of the reference points via the user interface 1800.

Preferably, a rough matching process may be performed prior to step S2510 to facilitate acquisition of surface information in subsequent steps. According to an embodiment of the present invention, the step S251 of generating a sampling pattern corresponds to (S2511) a step of defining a plurality of approximate reference points in a target region on a virtual anatomical model and (S2512) a reference point. It further includes a step of encouraging the user to generate the schematic spatial data by sampling a plurality of schematic sampling points on the object to be performed, and (S2513) a step of allocating the schematic spatial data to the virtual anatomical model.

In step S2511 of one embodiment, the approximate reference points may be defined by the computer system 1700 according to, for example, an image processing algorithm. Alternatively, the approximate reference point may be manually defined by the user via user interface 1800. As illustrated in FIG. 10, the approximate reference points 2215a-f are preferably located near the radiation opaque spot corresponding to the reference marker 1610. The amount of approximate reference points may vary, preferably 1-10, or more preferably 4-6. After defining the schematic reference points, the user can use a probe placed on the instrument 1200 to perform multiple schematics on or around the marker patch 1600 attached to the object corresponding to the schematic reference points 2215a-f. You will be prompted to sample the sampling point. The computer system 1700 manipulates the motion state of the manipulator 1210 so that the tip of the probe contacts the approximate sampling point, or the tip of the probe along a predetermined approximate sampling path formed by the approximate sampling point. Approximate sampling points may be sampled one at a time in succession by allowing them to slide. In one embodiment, the spatial marker frame 1550 is attached to the instrument 1200 to allow the spatial sensor system 1500 to track the coordinates of the probe during sampling and acquire the schematic spatial data 2255 corresponding to the schematic reference points 2215a-f. .. Then, in step S2512, the approximate spatial data is assigned to the virtual anatomical model, and the virtual anatomical model is pre-matched to the coordinate system recorded in the computer system 1700.

Alternatively, the rough matching process may be automatically performed by the surgical support system 1000. For example, in an embodiment of the invention in which a reference marker 1610, which is readable by the spatial sensor system 1510 and is radiation opaque to the medical imager 1910, is placed on the marker patch 1600 of interest, the sampling pattern. The generating step S251 further includes the step of assigning the schematic spatial data of the reference marker 1610 on the object to the virtual anatomical model 2110.

13 is referred to with FIG. After the sampling pattern is generated in step S251, a plurality of sampling points 2140 on the object corresponding to the reference point 2130 are sampled using the alignment probe installed in the instrument 1200 as in step S252. Prompts the user to generate sampling information 2310. The computer system 1700 manipulates the motion state of the manipulator 1210 so that the tip of the probe contacts the approximate sampling point, or the tip of the probe along a predetermined approximate sampling path formed by the approximate sampling point. Approximate sampling points may be sampled one at a time in succession by allowing them to slide. In one embodiment, the sampling information includes the coordinates of the sampling point, the contact angle α of the probe at the sampling point detected by the spatial sensor 1510, and the force maintained by the probe, the output power of the actuator, and the duration of stable contact. It may include parameters related to contact, such as.

In one embodiment, the forces maintained by the probe, including vertical forces (ie, forces parallel to the direction of the probe) and lateral forces (ie, forces perpendicular to the direction of the probe), are on instrument 1200. It can be detected by the force sensor 1235. The contact angle α may be defined as the angle of the probe with respect to the horizontal plane. For the duration of stable contact, the position of the spatial marker frame 1550 on the platform 1230 of the instrument 1200 remains substantially unchanged, or the acceleration, velocity, displacement, angular velocity detected by the inertial measuring unit of the instrument 1200. , And / or may be defined as the duration of time for which the angular acceleration remains substantially zero. In addition, Trocar 1400 may be utilized to guide the probe during surface sampling.

In step S2521, after all sampling points 2140 on the object corresponding to reference point 2130 have been sampled, the computer system 1700 designates or defines sampling information as surface information for the sampling points, similar to step S2521. See FIG. 9 again. After acquiring the surface information in step S151, the surface information is assigned to the virtual anatomical model generated in step S152, and the virtual anatomical model is aligned with the position of the coordinate system established by the spatial sensor system 1500. In one embodiment, an algorithm is used to identify the surface area in the virtual anatomical model that best fits the obtained surface information. The algorithm may be an iterative closet point (ICP), coherent point drift (CPD), or any algorithm that minimizes the difference between two point clouds and matches the two datasets.

In a more efficient embodiment, in step S252, after a predetermined amount of sampling points 2140 have been sampled by the user, the computer system 1700 sampled sampling points while the user continues to sample the next sampling point. 2140 may be assigned to a virtual anatomical model. The allocation results are displayed on the user interface 1800, and the sampled sampling points are already sufficient for the user to align the virtual anatomical model to the position of the coordinate system and stop sampling the remaining sampling points accordingly. You may decide whether or not there is.

In some embodiments of the invention, the acquired surface information is utilized, for example, by updating existing surface features in the virtual anatomical model, or by adding new surface features to the virtual anatomical model. Thereby, the resolution of the virtual anatomical model generated from the medical image in step S110 may be refined or improved. The surface information may also be used to update the surgical plan generated in step S141. Alternatively, surface information and a refined virtual anatomical model may be used together to ensure high accuracy of the surgical plan.

See FIG. According to another embodiment of the present invention, the step S151 of acquiring the surface information of the sampling points prompts the user to generate the (S251) sampling pattern and the (S252) sampling point sampling information. A step, a step of verifying the current sampling information (S253), a step of checking the sampling state (S254), a step of filtering the sampling information (S255), and (S2521) sampling information as surface information of the sampling point. Includes a designated process.

Specifically, as shown in FIG. 15, after the sampling information is generated in step S252, the computer system 1700 inspects the parameters according to at least one predetermined sampling criterion, as in step S253, to obtain the sampling information. To verify. A given sampling criterion is maintained by a probe that is above the vertical force threshold, a lateral force that is maintained by a probe that is below the lateral force threshold, and a probe that is below the torque threshold. It may include, but is not limited to, the torque to be applied, the output of the actuator that is greater than or equal to the power threshold, and the duration of stable contact between the probe and the contact surface that is greater than or equal to the time threshold. In some embodiments, the vertical force threshold may be 5 Newtons (N), the lateral force threshold may be 0.5N, or the torque threshold may be 0.05 (millinewton). It may be Newton × meter) mNm. The power threshold may be 0.3 amps and the time threshold may be 1 second.

See FIG. According to an embodiment of the present invention, in step S253 for verifying the current sampling information, (S2531) the contact-related parameters contained in the current sampling information are at least one of the predefined sampling criteria. The step of determining whether or not the parameters are satisfied, the step of indicating the current sampling information by the first mark when the (S2532) parameter meets the sampling criteria, and the current sampling information when the (S2533) parameter does not meet the sampling criteria. When the step of indicating the second mark and (S252) the current sampling information is indicated by the second mark, the step of generating the sampling information again, or (S254) when the current sampling information is indicated by the first mark. It includes a step of checking the sampling state.

In the exemplary embodiment shown in FIG. 15, in step S252, one sampling information is obtained by the user contacting the sampling point 2140 on the object corresponding to the reference point 2130 in the virtual anatomical model 2110 from a certain angle. Is generated, in step S2531, the computer system 1700 determines whether the parameters contained in one sampling information satisfy at least one of the sampling criteria. If the parameters do not meet the sampling criteria, the computer system 1700 indicates the sampling information as "invalid" (S2532) and prompts the user to re-contact the same sampling point 2140 from different angles via the user interface 1800 (S2532). S252). In some embodiments, when the computer system 1700 skips a sampling point and sampling of that sampling point fails (eg, the parameter does not meet the sampling criteria), the next sampling point is continuous over a predetermined time. You may urge the user to contact you.

If the parameter meets the sampling criteria, the computer system 1700 indicates the sampling information with a first mark (eg, "valid" or "correct") (S2533) and proceeds to check the current sampling state (S254). If the sampling information of all the sampling points corresponding to the reference point 2130 defined in step S2513 has not been sampled, the computer system 1700 passes through the user interface 1800 and follows the next sampling on the subject according to the predefined sampling route. Encourage the user to touch the point. Computer system 1700 repeats the verification process until all sampling points are sampled correctly. After all the sampling information has been sampled, the computer system 1700 proceeds to step S255 and filters the acquired sampling information by screening from valid to invalid. Then, as illustrated in FIG. 16, step S2521 is executed, the filtered sampling information is designated as surface information, and a list of surface information of sampling points is acquired.

In a more efficient embodiment, step S254 further determines whether to prompt the user to sample the next sampling point on a given sampling path according to the important surface features associated with the sampled sampling point. Including that. Important surface features may include, but are not limited to, peaks and valleys on the surface, as determined according to the direction and intensity of force detected by the probe during sampling. If the computer system 1700 determines that it does not proceed as originally planned, the computer system 1700 determines, for example, a direction that potentially contains one or more important surface features according to the parameters in the latest sampling information. A new sampling point may be identified by defining a sampling point in that direction as the next sampling point. Alternatively, the computer system 1700 may redefine the reference point or sampling route and start sampling according to the newly defined sampling route.

In summary, according to various embodiments of the invention, the surgical assistance system provides an accurate and efficient method for alignment. The method defines a target area on the surface of the subject and a set of reference points distributed within the target area that covers multiple surface features of the surface and is a machine between the probe and the surface of the surgical instrument. Verify target contact and effectively improve the accuracy and accuracy of computer-assisted surgery.

The above description is merely an embodiment of the present invention and is not intended to limit the scope of the present invention. Many modifications and modifications according to the claims and specification of the present invention are still within the scope of the present invention. Moreover, each of the embodiments and claims need not achieve all of the disclosed advantages or properties. Moreover, the abstract and the title of the invention serve only to facilitate the search of patent documents and are by no means intended to limit the scope of the claims.

Claims (27)

It is a method of acquiring surface information by a surgical support system.
The surgery support system
An instrument with a tool and a manipulator connected to the tool;
Spatial sensor system that detects spatial information of the tool;
A computer system that is electrically connected to the instrument, the spatial sensor system, and a user interface and manipulates the motion state of the manipulator according to the spatial information of the tool detected by the spatial sensor system;
The method is
(S1) A step of defining a target region on a virtual anatomical model of a target and a plurality of reference points of the target region by the computer system;
(S2) A step of urging a user to generate sampling information by sampling a plurality of sampling points of the target corresponding to the reference point using the instrument via the user interface.
Each of the sampling information includes the coordinates of one sampling point, the contact angle of the tool at the one sampling point detected by the spatial sensor system, and the parameters associated with the contact. (S3) A method comprising the step of designating the sampling information as surface information of the sampling point by the computer system. The sampling points are sampled by allowing the computer system to manipulate the motion state of the manipulator to control the sampling points so that the tips of the probes come into contact one at a time. The method according to claim 1. The method of claim 1, further comprising the following steps prior to step S1:
A step of assigning schematic spatial data of a plurality of reference markers on the object to a virtual anatomical model by the computer system.
A step in which the reference marker is readable by the spatial sensor system and is radiopaque to the medical imager associated with the surgical assistance system. The method of claim 1, further comprising the following steps prior to step S1:
The step of defining a plurality of schematic reference points of the target region on the virtual anatomical model by the computer system;
Through the user interface, the step of encouraging the user to generate schematic spatial data by sampling a plurality of schematic sampling points of interest corresponding to the reference point using the instrument; and by the computer system. The step of assigning the schematic spatial data to the virtual anatomical model. The approximate sampling points are sampled by allowing the computer system to manipulate the motion state of the manipulator to control the approximate sampling points one at a time so that the tips of the probes come into contact with each other. The method according to claim 4. The method according to claim 1, wherein a plurality of surface features are covered by the reference points by distributing the reference points in the target region. The method according to claim 1, further comprising the following steps after the step S2:
(S21) Step of verifying current sampling information;
(S22) A step of checking the sampling state; and (S23) A step of filtering the sampling information. The method of claim 7, wherein step S21 further comprises the following steps:
(S211) A step of determining whether the parameters included in the current sampling information satisfy at least one sampling criterion;
(S212) A step of indicating the current sampling information with a first mark when the parameter satisfies the sampling standard, and indicating the current sampling information with a second mark when the parameter does not satisfy the sampling standard; (S213) A step of proceeding to the step S22 when the current sampling information is indicated by the first mark, and proceeding to the step S2 when the current sampling information is indicated by the second mark. The method of claim 8, wherein the sampling criterion comprises a vertical force maintained by the tool greater than or equal to a vertical force threshold. 8. The method of claim 8, wherein the sampling criteria include that the lateral force maintained by the tool is less than or equal to the lateral force threshold. 8. The method of claim 8, wherein the sampling criteria include that the torque maintained by the tool is less than or equal to the torque threshold. The method according to claim 8, wherein the sampling reference includes that the output powers of the plurality of actuators of the manipulator are equal to or higher than the power threshold value. The method of claim 8, wherein the sampling criterion comprises a duration of stable contact between the tool and one sampling point greater than or equal to a time threshold. The method of claim 7, wherein step S22 comprises the following steps:
(S221) A step of determining whether the sampling information of all the sampling points is indicated by the first mark; and (S222) When the sampling information of all the sampling points is sampled, the process proceeds to the step S23. The step of proceeding to the step S2 when at least one sampling information of all the sampling points has not been sampled. The method according to claim 7, wherein the step S23 is performed by screening the sampling information indicated by the second mark from the sampling information indicated by the first mark. The method according to claim 1, further comprising the following steps after the step S3:
(S4) A step of assigning the surface information to the virtual anatomical model and aligning the virtual anatomical model with a position in a coordinate system. 16. The method of claim 16, further comprising the following steps after step S4:
(S5) A step of refining the virtual anatomical model according to the surface information. 16. The method of claim 16, further comprising the following steps after step S4:
(S6) A step of updating the operation plan according to the surface information. Surgical support system
An instrument with a tool and a manipulator connected to the tool;
Spatial sensor system that detects spatial information of the tool;
A computer system that is electrically connected to the instrument, the spatial sensor system, and a user interface and manipulates the motion state of the manipulator according to the spatial information of the tool detected by the spatial sensor system;
The computer system includes a non-temporary computer-readable medium that stores a program for acquiring surface information.
The program can be executed by at least one processor of the computer system and
The program, when executed by the processor, comprises instructions that cause the surgical assistance system to perform an operation that includes the following steps:
(S1) A step of defining a target region on a virtual anatomical model of a target and a plurality of reference points of the target region by the computer system;
(S2) A step of urging a user to generate sampling information by sampling a plurality of target sampling points corresponding to the reference point using the instrument via the user interface.
Each of the sampling information includes the coordinates of one sampling point, the contact angle of the tool at the one sampling point detected by the spatial sensor system, and the parameters associated with the contact. S3) A step of designating the sampling information as surface information of the sampling point by the computer system. 19. The system of claim 19, wherein the device comprises at least one handle that allows the user to hold and operate the device. 19. The system of claim 19, wherein the instrument further comprises a force sensor that detects at least one of the forces and torques maintained by the tool. 21. The system of claim 21, wherein the force sensor is coupled between the tool and the manipulator. 21. The system of claim 21, wherein the force sensor is located within the tool. 19. The system of claim 19, wherein the parameters associated with the contact include forces maintained by the tool. 19. The system of claim 19, wherein the parameters associated with the contact include torque maintained by the tool. 19. The system of claim 19, wherein the parameters associated with the contact include the output power of a plurality of actuators of the manipulator. 19. The system of claim 19, wherein the parameters associated with the contact include a duration of stable contact between the tool and one sampling point.

Images