JP2019122768A - Surgical robotic systems with target trajectory deviation monitoring and related methods - Google Patents

Abstract

To provide a method of operating a surgical robotic system including a robotic arm configured to position a surgical end-effector with respect to an anatomical location of a patient in accordance with, e.g., movement of the patient due to breathing.SOLUTION: Position information may be received where the position information is generated using a sensor system 200 remote from a robotic arm 104 and remote from a patient 210. The position information may include position information relating to a tracking device 116 affixed to the patient, and position information relating to a surgical end-effector 112. The robotic arm may be controlled to move the surgical end-effector to a target trajectory relative to the anatomical location of the patient on the basis of the position information generated using the sensor system.SELECTED DRAWING: Figure 1

Description

  This application is a continuation-in-part of US patent application Ser. No. 15 / 609,334, filed May 31, 2017, which is a continuation-in-part of US patent application Ser. No. 15 / 157,444, which is a continuation-in-part of US patent application Ser. No. 15 / 095,883, filed Apr. 11, 2016; It is a continuation-in-part of US patent application Ser. No. 14 / 062,707 filed on May 24, which is a part of US patent application Ser. No. 13 / 924,505 filed on Jun. 21, 2013. Divisional continuation application, which claims priority to Provisional Application No. 61 / 662,702, filed on June 21, 2012, and Provisional Application No. 61, filed on March 15, 2013 Against / 800,527 Claims above rights, all of which is incorporated in its entirety herein by reference for all purposes.

  The present disclosure relates to medical devices, and more particularly, to a robotic system for surgery and associated methods and devices.

  Position recognition systems for robot assisted surgery are used to determine and track the position of a particular object in three dimensions (3D). In robot-assisted surgery, for example, certain objects, such as surgical instruments, need to be tracked with a high degree of accuracy when the instruments are positioned and moved by a robot or doctor.

  Position recognition systems based on infrared signals can use active and / or passive sensors or markers to track objects. For passive sensors or markers, the tracked object can include a passive sensor, such as a reflective spherical ball, and the passive sensor is located at strategic locations of the tracked object. An infrared transmitter transmits a signal, and a reflective spherical ball reflects the signal to help determine the three-dimensional position of the object. In active sensors or markers, the tracked object comprises an active infrared transmitter, such as a light emitting diode (LED), and thus generates its own infrared signal for 3D detection.

  Using either an active or passive tracking sensor, the system can measure the time taken to receive one or more infrared cameras, digital signals, known locations of active and / or passive sensors, distances, response signals, Geometrically resolve the three-dimensional position of the active and / or passive sensors based on information from or to other known variables, or combinations thereof.

  Thus, these surgical systems can utilize position feedback to accurately guide the motion of the robotic arm and tool relative to the patient's surgical site. However, patient movement (e.g., by respiration) can affect positioning accuracy.

  In some embodiments of the inventive concept, a method may be provided for operating a surgical robotic system that includes a robotic arm configured to position a surgical end effector relative to an anatomical location of a patient. Position information may be received, which is generated using a sensor system remote from the robotic arm and remote from the patient. The position information may include position information regarding a tracking device (e.g., reference based or dynamic reference based) fixed to the patient and position information regarding the surgical end effector. The robotic arm may be controlled to move the surgical end effector to a target trajectory relative to the patient's anatomical location based on position information generated using the sensor system. After controlling the robotic arm to move to the target trajectory relative to the anatomical position of the patient, the robotic arm is controlled to lock the position of the surgical end effector. While the position of the surgical end effector is locked, the deviation between the actual trajectory of the surgical end effector with respect to the anatomical position and the target trajectory of the surgical end effector with respect to the anatomical position is determined. obtain. Furthermore, the offset may be determined based on positioning information generated using the sensor system after locking the position of the surgical end effector. Additionally, a user output indicating the offset may be generated in response to determining the offset.

  In some other embodiments of the inventive concept, a method may be provided for operating a surgical robotic system that includes a robotic arm configured to position a surgical end effector relative to an anatomical location of a patient. If the motion model of the anatomical position with respect to the tracking device for the multiple phases of the respiratory cycle provides multiple offsets of the anatomical position with respect to the tracking device, then each one of the plurality of offsets is of the respiratory cycle Access to the model may be provided to be associated with each one of the plurality of stages. Position information may be generated using a sensor system remote from the robotic arm and remote from the patient. The position information may include information on the position of the tracking device fixed to the patient and the position of the surgical end effector when the tracking device moves by breathing the patient. Multiple phases of the breathing cycle may be detected as the tracking device moves due to the patient's breathing. The robotic arm may be controlled to maintain the surgical end effector in a target trajectory relative to the patient's anatomical position as the patient's breathing moves the tracking device. Controlling receiving positional information, detecting multiple steps, and using multiple offsets to determine the location of the anatomical location when the tracking device is moved by the patient's breathing , Based on.

  In yet another embodiment of the inventive concept, a method may be provided for operating a surgical robotic system that includes a robotic arm configured to position a surgical end effector with respect to an anatomical location of a patient. The motion model of the anatomical position with respect to the tracking device (e.g. reference based or dynamic reference based), wherein the first offset of the anatomical position relative to the tracking device determines the target trajectory for the first phase of the breathing cycle The second offset of the anatomical position with respect to the tracking device may be provided for multiple phases of the breathing cycle, as used for determining the target trajectory for the second phase of the breathing cycle. First position information may be received, the first position information being generated using a sensor system remote from the robotic arm and remote from the patient. The first position information may include information on a first position of the tracking device fixed to the patient and a first position of the surgical end effector. The first phase of the patient's breathing cycle may be detected, and the robotic arm is anatomically based on the first position information and from the first position of the tracking device in response to detecting the first phase of the respiratory cycle. Based on the use of the first offset to determine the first position, the surgical end effector can be controlled to move to a target trajectory relative to the patient's anatomical position. Second position information generated using the sensor system may be received, the second position information including information on a second position of the tracking device fixed to the patient and a second position of the surgical end effector. The second phase of the patient's respiratory cycle may be detected, and the robotic arm is anatomically based on the second position information and from the second position of the tracking device in response to detecting the second phase of the respiratory cycle. The surgical end effector may be controlled to move to a target trajectory relative to the anatomical position of the patient based on using a second offset to determine a second position of the position.

  Other methods and associated surgical systems, and corresponding methods and computer program products according to embodiments of the present subject matter will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It will be. It is intended that all such surgical systems and corresponding methods and computer program products be included within the description herein, be within the scope of the inventive subject matter, and be protected by the accompanying claims. Be done. Furthermore, all the embodiments disclosed herein may be implemented separately or may be implemented in combination in any manner and / or combination.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to provide a further understanding of the present disclosure and form part of the present application, illustrate certain non-limiting embodiments of the inventive concept. In the drawing:
FIG. 1 is an overhead view of a possible placement of the robotic system, patient, surgeon and other healthcare personnel during a surgical procedure. FIG. 2 illustrates a robotic system that includes positioning of a surgical robot and camera with respect to a patient, according to one embodiment. FIG. 3 shows a surgical robotic system according to an exemplary embodiment. FIG. 4 shows a portion of a surgical robot in accordance with an illustrative embodiment. FIG. 5 shows a block diagram of a surgical robot in accordance with an illustrative embodiment. FIG. 6 shows a surgical robot according to an exemplary embodiment. 7A-7C show an end effector according to an exemplary embodiment. FIG. 8 shows the surgical instrument and the end effector before and after inserting the surgical instrument into the guide tube of the end effector according to one embodiment. 9A-9C illustrate portions of an end effector and a robotic arm in accordance with an illustrative embodiment. FIG. 10 illustrates a dynamic reference array, an imaging array, and other components in accordance with an illustrative embodiment. FIG. 11 shows a method of registration according to an exemplary embodiment. 12A-12B illustrate an embodiment of an imaging device according to an exemplary embodiment. FIG. 13A illustrates a portion of a robot including a robotic arm and an end effector, according to an illustrative embodiment. FIG. 13B is an enlarged view of the end effector with multiple tracking markers fixedly secured thereon, as shown in FIG. 13A. FIG. 13C is a tool or instrument having a plurality of tracking markers rigidly affixed thereon, according to one embodiment. FIG. 14A is an alternative version of an end effector having moveable tracking markers of a first configuration. FIG. 14B is the end effector shown in FIG. 14A with the movable tracking marker in a second configuration. FIG. 14C shows a tracking marker template in the first configuration of FIG. 14A. FIG. 14D shows a template of tracking markers in the second configuration of FIG. 14B. FIG. 15A shows an alternative version of the end effector with only a single tracking marker fixed on it. FIG. 15B shows the end effector of FIG. 15A with the instrument disposed through the guide tube. FIG. 15C shows the end effector of FIG. 15A with the instrument in two different positions, and shows the resulting logic for determining whether the instrument is placed in the guide tube or outside the guide tube. FIG. 15D shows the end effector of FIG. 15A, but the instrument is in the guide tube at two different frames and at a relative distance to a single tracking marker on the guide tube. FIG. 15E shows the end effector of FIG. 15A relative to a coordinate system. FIG. 16 is a block diagram of a method of navigating and moving an end effector of a robot to a desired target trajectory. 17A and 17B also show an instrument for inserting an expandable implant having fixed and movable tracking markers in the contracted and expanded positions, respectively. Also, FIGS. 18A and 18B show an instrument for inserting an articulating implant with fixed and movable tracking markers in the insertion position and in the inclined position, respectively. FIG. 19A illustrates one embodiment of a robot including a replaceable or alternative end effector. FIG. 19B shows an embodiment of a robot with an instrument style end effector coupled thereto. 20A, 20B, and 20C are schematic illustrations of meter configurations, according to some embodiments of the inventive concept. 21A and 21B are schematic representations of trajectories at various stages of breathing, according to some embodiments of the inventive concept. FIG. 22 is a block diagram illustrating a robot controller according to some embodiments of the inventive concept. Figures 23 and 24 are flowcharts illustrating the operation of a surgical robotic system according to some embodiments of the inventive concept. Same as above.

  It is to be understood that this disclosure is not limited in its application to the details of construction and the arrangements of components set forth in the description of this specification or illustrated in the drawings. The teachings of the present disclosure may be used, practiced, implemented in various ways, or implemented in other embodiments. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting. The terms "including", "comprising" or "having" and variations thereof as used herein are meant to include the items listed thereafter and their equivalents and additional items. Do. Unless otherwise specified or limited, the terms "mounted," "connected," "supported," and "coupled" and their Variations are widely used and encompass both direct and indirect attachment, connection, support and bonding. Furthermore, "connected" and "coupled" are not limited to physical or mechanical connections or couplings.

  The following discussion is presented to enable a person of ordinary skill in the art to make and use embodiments of the present disclosure. Various modifications to the illustrated embodiments will be readily apparent to those skilled in the art, and the principles herein may be applied to other embodiments and applications without departing from the embodiments of the present disclosure. it can. Thus, embodiments are not intended to be limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein. The following detailed description should be read with reference to the drawings, where like elements in different drawings have like reference numerals. The drawings, which are not necessarily to scale, depict selected embodiments and are not intended to limit the scope of the embodiments. One skilled in the art will recognize that the examples provided herein have many useful alternatives and fall within the scope of the embodiments.

  Referring now to the drawings, FIGS. 1 and 2 illustrate a surgical robotic system 100 according to an exemplary embodiment. The surgical robot system 100 includes, for example, a surgical robot 102, one or more robot arms 104, a base 106, a display 110, an end effector 112 including a guide tube 114, and one or more tracking units. And a marker 118. The robotic robotic system 100 can include a patient tracking device 116 that also includes one or more tracking markers 118 adapted to be secured directly to the patient 210 (eg, to the bone of the patient 210). The surgical robot system 100 can also use, for example, the camera 200 positioned on the camera stand 202. The camera stand 202 can have any suitable configuration for moving, orienting and supporting the camera 200 to a desired position. Camera 200 is within a given measurement volume that can be viewed from the perspective of camera 200, for example, active and passive tracking markers 118 (shown as part of patient tracking device 116 of FIG. A suitable camera, such as one or more infrared cameras (e.g., a bifocal camera or a stereo photo camera), can be identified that can be identified by the 13B enlargement. The camera 200 can scan a given measurement volume and detect light coming from the marker 118 to identify and determine the three-dimensional position of the marker 118. For example, the active marker 118 can include an infrared light emitting marker (eg, an infrared light emitting diode (LED)) activated by an electrical signal and / or the passive marker 118 can be, for example, an illuminator on the camera 200 or the like. The infrared-reflecting retroreflective markers (eg, they reflect incident IR radiation in the direction of incident light) may be emitted by any suitable device.

  FIGS. 1 and 2 also show possible configurations of the arrangement of the surgical robot system 100 in an operating room environment. For example, the robot 102 can be positioned near or next to the patient 210. Although depicted close to the head of the patient 210, it will be appreciated that the robot 102 can be positioned at any suitable location near the patient 210, depending on the area of the patient 210 undergoing surgery. Camera 200 may be separate from robotic system 100 and placed on the foot of patient 210. This position allows the camera 200 to have a direct visual line of sight relative to the surgical area 208. Again, it is contemplated that the camera 200 can be positioned at any suitable location with a line of sight relative to the surgical field 208. In the configuration shown, the surgeon 120 is positioned opposite the robot 102 but can still manipulate the end effector 112 and the display 110. The surgeon assistant 126 can also be positioned opposite the surgeon 120 so that it can access both the end effector 112 and the display 110. If desired, the positions of the surgeon 120 and the assistant 126 may be reversed. Conventional areas for the anesthesiologist 122 and a nurse or operating room technician 124 may remain unhindered by the position of the robot 102 and the camera 200.

  With respect to the other components of the robot 102, the display 110 can be attached to the surgical robot 102, and in other exemplary embodiments, the display 110 is separated from the surgical robot 102 and together with the surgical robot 102. In the operating room or in a remote place. The end effector 112 may be coupled to the robotic arm 104 and controlled by at least one motor. In the exemplary embodiment, the end effector 112 is a guide tube 114 capable of receiving and orienting a surgical instrument 608 (described further herein) used to perform a surgical procedure on the patient 210. Can be included. The term "end effector" as used herein is used interchangeably with the terms "end effector" and "effectuator element". Although generally indicated by the guide tube 114, it will be appreciated that the end effector 112 can be replaced with any suitable instrument suitable for use in surgery. In some embodiments, end effector 112 can include any known structure for performing the movement of surgical instrument 608 in a desired manner.

  The surgical robot 102 can control the translation and orientation of the end effector 112. The robot 102 can, for example, move the end effector 112 along x, y, z axes. The end effector 112 is of the x-, y-, and z-axes (so that one or more of the Euler angles (eg, roll, pitch, and / or yaw) associated with the end effector 112 can be selectively controlled). One or more may be configured to selectively rotate about the z-frame axis. In some exemplary embodiments, selective control of the translation and orientation of the end effector 112 is substantially improved as compared to, for example, a conventional robot that utilizes a six degree of freedom robot arm that includes only a rotational axis. Enable the execution of medical procedures with the stated accuracy. For example, the robotic system 100 is used to operate on the patient 210, the robotic arm 104 is positioned above the body of the patient 210, and the end effector 112 is selected towards the body of the patient 210 with respect to the z-axis Can be angled.

  In some exemplary embodiments, the position of the surgical instrument 608 may be dynamically updated so that the surgical robot 102 can always know the position of the surgical instrument 608 during surgery. it can. As a result, in some exemplary embodiments, the surgical robot 102 can quickly move the surgical instrument 608 to the desired position (without the physician's request) without further assistance from the physician. In further embodiments, the surgical robot 102 can be configured to modify the path of the surgical instrument 608 if the surgical instrument 608 deviates from the selected pre-planned trajectory. In some exemplary embodiments, the surgical robot 102 may be configured to allow for the stop, change, and / or manual control of movement of the end effector 112 and / or the surgical instrument 608. Thus, in use, in an exemplary embodiment, a physician or other user can operate the system 100 to stop, modify, or manually control the autonomous movement of the end effector 112 and / or the surgical instrument 608 Have the option to Further details of surgical robot system 100 including control and movement of surgical instrument 608 by surgical robot 102 can be found in co-pending US patent application Ser. No. 13 / 924,505, which is incorporated by reference in its entirety. It is incorporated in the specification.

  Robotic surgical system 100 may include one or more tracking markers 118 configured to track movement of robotic arm 104, end effector 112, patient 210, and / or surgical instrument 608 in three dimensions. it can. In the exemplary embodiment, a plurality of tracking markers 118 are attached (or otherwise fixed) to the outer surface of the robot 102, such as on the base 106 of the robot 102, on the robot arm 104, and / or on the end effector 112. ) But not limited thereto. In an exemplary embodiment, at least one tracking marker 118 of the plurality of tracking markers 118 may be attached or otherwise secured to the end effector 112. One or more tracking markers 118 may be further attached (or otherwise fixed) to the patient 210. In the exemplary embodiment, multiple tracking markers 118 are placed on the patient 210 away from the surgical area 208 to reduce the possibility of being obscured by the surgeon, surgical tools, or other portions of the robot 102. It can be arranged. Additionally, one or more tracking markers 118 can be further attached (or otherwise secured) to the surgical tool 608 (eg, a screwdriver, a dilator, an implant inserter, etc.). Thus, tracking marker 118 enables tracking of each of the marked objects (eg, end effector 112, patient 210, and surgical tool 608) by robot 102. In the exemplary embodiment, system 100 is positioned, for example, within end effector 112, surgical instrument 608 (eg, tube 114 of end effector 112) using tracking information collected from each of the marked objects. And the relative position of the patient 210 can be calculated.

  Markers 118 can include radiopaque or optical markers. The markers 118 can be of any suitable shape including spheres, spheroids, cylinders, cubes, cuboids and the like. In an exemplary embodiment, one or more markers 118 may be optical markers. In some embodiments, positioning of the one or more tracking markers 118 on the end effector 112 may maximize accuracy of position measurement by functioning to confirm or confirm the position of the end effector 112 it can. Further details of surgical robot system 100, including control, movement and tracking of surgical robot 102 and surgical instrument 608 can be found in US Patent Application No. 2016/0242849, which is incorporated herein by reference in its entirety. Be incorporated.

  An exemplary embodiment includes one or more markers 118 coupled to a surgical instrument 608. In an exemplary embodiment, for example, these markers 118 coupled to the patient 210 and the surgical instrument 608 and the markers 118 coupled to the end effector 112 of the robot 102 may be conventional infrared light emitting diodes (LEDs), or A trackable Optotrak <(R)> diode can be included using a commercially available infrared optical tracking system such as Optotrak <(R)>. Optotrak (R) is a Northern Digital Inc., Waterloo, Ontario, Canada. Is a registered trademark of In other embodiments, the marker 118 can include a conventional reflective sphere that can be tracked using a commercially available optical tracking system such as a Polaris Spectra. Polaris Spectra is a product of Northern Digital, Inc. Is a registered trademark of In the exemplary embodiment, marker 118 coupled to end effector 112 is an active marker including an infrared light emitting diode that can be turned on and off, and marker 118 coupled to patient 210 and surgical instrument 608 is Includes passive reflecting spheres.

  In an exemplary embodiment, light emitted from and / or reflected by the marker 118 may be detected by the camera 200 and used to monitor the position and movement of the marked object. In alternative embodiments, the marker 118 can include a radio frequency and / or electromagnetic reflector or receiver, and the camera 200 can include or replace a radio frequency and / or electromagnetic receiver it can.

  Similar to the surgical robot system 100, FIG. 3 shows the surgical robot system 300 and the camera stand 302 in a docked configuration according to an exemplary embodiment of the present disclosure. The robot system for operation 300 controls the display 304, the upper arm 306, the lower arm 308, the end effector 310, the vertical post 312, the caster 314, the cabinet 316, the tablet drawer 318, the connector panel 320, Included is a robot 301 that includes a panel 322 and a ring 324 of information. Camera stand 302 can include a camera 326. These components are described on a larger scale with reference to FIG. FIG. 3 shows the robotic system 300 in a docking configuration with the camera stand 302 nested with the robot 301, eg, when not in use. Those skilled in the art will appreciate that the camera 326 and the robot 301 may be separated from one another and may be placed in any suitable position during a surgical procedure, for example, as shown in FIGS.

  FIG. 4 shows a base 400 consistent with an exemplary embodiment of the present disclosure. The base 400 may be part of a surgical robotic system 300 and may include a cabinet 316. Cabinet 316 can house certain components of surgical robotic system 300 including, but not limited to, battery 402, power distribution module 404, platform interface board module 406, computer 408, handle 412, and tablet drawer 414. . The connections and relationships between these components are described in more detail with reference to FIG.

  FIG. 5 shows a block diagram of certain components of an exemplary embodiment of a surgical robotic system 300. The robotic system 300 can include a platform subsystem 502, a computer subsystem 504, a motion control subsystem 506, and a tracking subsystem 532. Platform subsystem 502 may further include battery 402, power distribution module 404, platform interface board module 406, and tablet charging station 534. Computer subsystem 504 can further include a computer 408, a display 304, and a speaker 536. Motion control subsystem 506 may further include drive circuit 508, motors 510, 512, 514, 516, 518, ballasts 520, 522, 524, 526, end effector 310, and controller 538. Tracking subsystem 532 may further include position sensor 540 and camera converter 542. System 300 can also include a foot pedal 544 and a tablet 546.

  Input power is provided to system 300 via a power supply 548 that may be provided to power distribution module 404. Power distribution module 404 is configured to receive input power and to generate different power supply voltages provided to other modules, components, and subsystems of system 300. The power distribution module 404 can be configured to supply different voltages to the platform interface module 406, but the voltages can be from the computer 408, the display 304, the speakers 536 such as the power motors 512, 514, 516, 518 and the end effector Providing to 310 other components such as driver 508, motor 510, ring 324, camera converter 542, and other components for system 300 (e.g., a fan for cooling the electrical components in cabinet 316) Can.

  Power distribution module 404 may also provide power to other components, such as tablet charging station 534, which may be disposed within tablet drawer 318. The tablet charging station 534 may be in wireless or wired communication with the tablet 546 to charge the table 546. Tablet 546 may be used by the surgeons described herein, consistent with the present disclosure.

  Power distribution module 404 may be connected to a battery 402 that acts as a temporary power source when power distribution module 404 does not receive power from input power 548. At other times, the power distribution module 404 can serve to charge the battery 402 as needed.

  Other components of platform subsystem 502 may also include connector panel 320, control panel 322, and ring 324. Connector panel 320 may function to connect different devices and components to system 300 and / or related components and modules. Connector panel 320 can include one or more ports that receive lines or connections from different components. For example, connector panel 320 may be associated with a ground terminal port for grounding system 300 to other equipment, a port for connecting foot pedal 544 to system 300, tracking subsystem 532 (position sensor 540, camera converter 542, camera stand 302) And the camera 326 can be included. Connector panel 320 may also include other ports that allow USB, Ethernet, HDMI communication to other components, such as computer 408.

  Control panel 322 can provide various buttons or indicators that control the operation of system 300 and / or provide information regarding system 300. For example, the control panel 322 may engage a button to turn the system 300 on or off, a button to raise or lower the vertical post 312, and a caster 314 to prevent physically moving the system 300. Stabilizers 520-526 designed to mate can include buttons for raising or lowering. In the event of an emergency, other buttons may shut down the system 300, but this will remove all motor power, apply mechanical braking and stop all operation. The control panel 322 can also have line power indicators of the battery 402 or indicators that notify the user of particular system status, such as the charge status.

  Ring 324 can be a visual indicator that informs the user of system 300 of the different modes in which system 300 is operating and certain alerts for the user.

  Computer subsystem 504 includes a computer 408, a display 304, and a speaker 536. Computer 504 includes an operating system for operating system 300 and software. Computer 504 can receive and process information from other components (eg, tracking subsystem 532, platform subsystem 502, and / or motion control subsystem 506) to display the information to the user. Additionally, computer subsystem 504 may also include speakers 536 for providing audio to the user.

  Tracking subsystem 532 may include position sensor 504 and transducer 542. Tracking subsystem 532 may correspond to camera stand 302 including camera 326 described with respect to FIG. Position sensor 504 may be camera 326. The tracking subsystem may track the position of particular markers located on different components and / or instruments of the system 300 used by the user during a surgical procedure. This tracking may be performed in a manner consistent with the present disclosure, including the use of infrared technology to track the position of active or passive elements such as LEDs or reflective markers, respectively. The position, orientation, and placement of structures having these types of markers may be provided to a computer 408 shown to a user on display 304. For example, surgical instruments 608 that have these types of markers and can be tracked in this manner (which may be referred to as navigation space) are displayed to the user in relation to a three-dimensional image of the patient's anatomy. It may be done.

  The motion control subsystem 506 may be configured to physically move the vertical post 312, upper arm 306, lower arm 308, or rotate the rotating end effector 310. Physical movement can occur through the use of one or more motors 510-518. For example, motor 510 can be configured to lift or lower column 312 vertically. The motor 512 can be configured to move the upper arm 308 laterally around the point of engagement with the vertical post 312, as shown in FIG. The motor 514 may be configured to move the lower arm 308 laterally around the point of engagement with the upper arm 308, as shown in FIG. The motors 516 and 518 can be configured to control the roll, one to control the tilt, to move the end effector 310, thereby allowing multiple angles to move the end effector 310. Can be provided. A controller for controlling these movements via the load cell, which is arranged on the end effector 310 and triggered by the user engaging these load cells in order to move the system 300 in the desired way It can be achieved by 538.

  In addition, system 300 indicates the position of the surgical instrument or component on the three-dimensional image of the patient's anatomy on display 304 on display 304 (which may be a touch screen input device). Can provide automatic movement of the vertical column 312, the upper arm 306, and the lower arm 308. The user can initiate this automatic movement by stepping on the foot pedal 544 or some other input means.

  FIG. 6 shows a surgical robotic system 600 according to an exemplary embodiment. The surgical robotic system 600 can include an end effector 602, a robotic arm 604, a guide tube 606, an instrument 608, and a robotic base 610. The instrument tool 608 can be attached to a tracking array 612 that includes one or more tracking markers (such as markers 118) and can have an associated trajectory 614. The track 614 represents a movement path (e.g., a path in which the tool tool 608 is inserted into the patient, etc.) that the tool tool 608 is configured to move when positioned or secured through the guide tube 606 Can. In an exemplary operation, robot base 610 is configured to be in electronic communication with robot arm 604 and end effector 602, and surgical robotic system 600 operates to operate a user (eg, a surgeon) on patient 210. Support. The surgical robotic system 600 may be consistent with the surgical robotic systems 100 and 300 described above.

  A tracking array 612 may be mounted to the instrument 608 to monitor the position and orientation of the instrument tool 608. Tracking array 612 may be attached to instrument 608 and may include tracking markers 804. As best shown in FIG. 8, tracking marker 804 may be, for example, a light emitting diode and / or other type of reflective marker (eg, marker 118 described elsewhere herein). The tracking device may be one or more eye gaze devices associated with the surgical robotic system. By way of example, the tracking device may be one or more cameras 200, 326 associated with the robotic system 100, 300 for the robotic arm 604, the robotic base 610, the end effector 602, and / or the patient 210. The tracking array 612 may be tracked for a defined area or relative orientation of the instrument 608 in a related manner. The tracking device may be consistent with the structure described in connection with the camera stand 302 and the tracking subsystem 532.

  7A, 7B, and 7C also show top, front, and side views, respectively, of the end effector 602 according to an exemplary embodiment. End effector 602 can include one or more tracking markers 702. The tracking markers 702 may be light emitting diodes or other types of active and passive markers, such as the tracking markers 118 described above. In the exemplary embodiment, tracking marker 702 is an active infrared emitting marker activated by an electrical signal (eg, an infrared light emitting diode (LED)). Thus, the tracking marker 702 may be activated such that the infrared marker 702 is visible to the camera 200, 326 or may be deactivated such that the infrared marker 702 is not visible to the camera 200, 326. Thus, when the marker 702 is active, the end effector 602 is controlled by the system 100, 300, 600, and when the marker 702 is deactivated, the end effector 602 is locked in place, the system 100, 300, It can not move by 600.

  The markers 702 are positioned on or in the end effector 602 so that the markers 702 are viewed by one or more cameras 200, 326 or other tracking devices associated with the robotic system 100, 300, 600 It may be done. The camera 200, 326 or other tracking device can track the end effector 602 as it moves to different positions and viewing angles following the movement of the tracking marker 702. The position of the marker 702 and / or the end effector 602 may also be shown on the displays 110, 304 associated with the robotic system 100, 300, 600, eg, the display 110 shown in FIG. 2 and / or the display 304 shown in FIG. Good. The displays 110, 304 allow the user to ensure that the end effector 602 is at a desired position with respect to the robotic arm 604, the robotic base 610, the patient 210, and / or the user.

  For example, as shown in FIG. 7A, the tracking device located away from the surgical area 208 and facing the robot 102, 301 and the cameras 200, 326 has a general orientation of the end effector 602 relative to the tracking device. Markers 702 may be disposed around the surface of end effector 602 so that at least three markers 702 can be viewed through the area. For example, distributing the markers 702 in this manner allows the tracking device to monitor the end effector 602 as the end effector 602 is moved and rotated within the surgical area 208.

  Further, in the exemplary embodiment, end effector 602 can include an infrared (IR) receiver that can detect when external camera 200, 326 is ready to read marker 702. During this detection, the end effector 602 can illuminate the marker 702. Due to the IR receiver detecting that the external camera 200, 326 is ready to read the marker 702, the duty cycle of the marker 702, which may be a light emitting diode, needs to be synchronized to the external camera 200, 326 You can tell that. This can also allow for lower power consumption by the robotic system as a whole, so that the markers 702 are only illuminated at appropriate times instead of being illuminated continuously. Moreover, in the exemplary embodiment, the marker 702 can be powered off to prevent interference with other navigation tools, such as different types of surgical instruments 608.

  FIG. 8 shows one type of surgical instrument 608 that includes a tracking array 612 and a tracking marker 804. Tracking marker 804 may be of any type described herein, including, but not limited to, a light emitting diode or a reflective sphere. The marker 804 is monitored by a tracking device associated with the robotic system 100, 300, 600 and may be one or more eye cameras 200, 326. The cameras 200, 326 can track the position of the instrument 608 based on the position and orientation of the tracking array 612 and the markers 804. A user, such as a surgeon 120, is fully aware of the tracking array 612 and the marker 804 by the tracking device or camera 200, 326 and displays the instrument 608 and the marker 804 on, for example, the display 110 of an exemplary surgical robotic system The appliance 608 can be directed to do so.

  The manner in which the surgeon 120 places the instrument 608 into the guide tube 606 of the end effector 602 and adjusts the instrument 608 is apparent in FIG. The hollow tubes or guide tubes 114, 606 of the end effector 112, 310, 602 are sized and configured to receive at least a portion of the surgical instrument 608. Guide tubes 114, 606 are configured to be oriented by robotic arm 104 so that the insertion and trajectory of surgical instrument 608 can reach the desired anatomical target within or on the patient 210. Surgical instrument 608 may include at least a portion of a generally cylindrical instrument. While a screw driver is illustrated as a surgical tool 608, it will be appreciated that any suitable surgical tool 608 may be positioned by the end effector 602. As one example, the surgical instrument 608 can include one or more guidewires, cannulas, retractors, drills, reamers, screwdrivers, insertion tools, removal tools, and the like. Although the hollow tubes 114, 606 are shown as having a generally cylindrical shape, one of ordinary skill in the art would appreciate that the guide tubes 114, 606 may be any suitable suitable for receiving the surgical instrument 608 and accessing the surgical site. Can have various sizes and configurations.

  9A-9C also illustrate portions of end effector 602 and robotic arm 604 in accordance with an illustrative embodiment. End effector 602 can further include a body 1202 and a clamp 1204. The clamp 1204 can include a handle 1206, a ball 1208, a spring 1210, and a lip 1212. The robotic arm 604 can further include a recess 1214, a mounting plate 1216, a lip 1218, and a magnet 1220.

  The end effector 602 can mechanically interface and / or engage with the surgical robotic system and the robotic arm 604 via one or more couplings. For example, end effector 602 can engage with robotic arm 604 via positioning and / or reinforcing couplings. These connections allow the end effector 602 to be secured to the robotic arm 604 outside of the flexible and sterile barrier. In an exemplary embodiment, the positioning coupling may be a magnetic kinematic mount, and the reinforcing coupling may be a five bar overcenter clamping linkage.

  For positioning couplings, the robotic arm 604 can include a nonmagnetic material, one or more recesses 1214, a lip 1218, and a mounting plate 1216, which can be a magnet 1220. Magnets 1220 are mounted below each of the recesses 1214. The portion of clamp 1204 comprises a magnetic material and is attracted by one or more magnets 1220. The magnetic attraction of the clamps 1204 and the robotic arm 604 causes the balls 1208 to be seated within their respective recesses 1214. For example, the ball 1208 shown in FIG. 9B seats in a recess 1214 as shown in FIG. 9A. This seat can be thought of as a magnetically assisted kinematic coupling. The magnet 1220 can be configured to have sufficient strength to support the full weight of the end effector 602 regardless of the orientation of the end effector 602. Positioning coupling is any style of kinematic mount that uniquely limits six degrees of freedom.

  With respect to the reinforcement coupling, portions of clamp 1204 can be configured to be a fixed ground link, and such clamp 1204 can function as a five bar linkage. The closing clamp handle 1206 may secure the end effector 602 to the robotic arm 604 when the lip 1212 and the lip 1218 engage the clamp 1204 to secure the end effector 602 and the robotic arm 604. When the clamp handle 1206 is closed, the spring 1210 may be stretched or stressed while the clamp 1204 is in the locked position. The lock position may be a position that provides an off center link mechanism. If there is no force applied to the clamp handle 1206 to release the clamp 1204, the linkage does not open. Thus, in the locked position, the end effector 602 can be securely fixed to the robotic arm 604.

  The spring 1210 may be a curved beam of tension. The spring 1210 can be constructed of a material that exhibits high stiffness and high yield strain, such as virgin PEEK (polyetheretherketone). The linkage between the end effector 602 and the robotic arm 604 can provide a sterile barrier between the end effector 602 and the robotic arm 604 without interfering with the fastening of the two couplings.

  The reinforcing coupling may be a link mechanism with a plurality of spring members. The reinforcement coupling can be latched with a cam or friction based mechanism. The reinforcement coupling may be a sufficiently strong electromagnet that assists in securing the end effector 102 to the robotic arm 604. The reinforcement coupling slides the interface between the end effector 602 and the robot arm 604, and completely separates the screw mechanism, the overcenter link mechanism, or the end effector 602 fixed by the cam mechanism from the robot arm 604 and / or the robot arm 604 It may be a color of

  Referring to FIGS. 10 and 11, a specific registration procedure is performed to track the object and target anatomical structure of the patient 210, both in the navigation space and in the image space, before or during a surgical procedure. It is also good. As shown in FIG. 10, a registration system 1400 can be used to perform such registration.

  To track the position of the patient 210, the patient tracking device 116 can include a patient anchor 1402 that will be secured to the rigid anatomical structure of the patient 210, and the dynamic reference base (DRB) 1404 can be The fixture 1402 may be fixedly attached. For example, a patient securement device 1402 may be inserted into the opening 1406 of the dynamic reference base 1404. The dynamic reference base 1404 can include a marker 1408 that appears to be a tracking device such as the tracking subsystem 532. These markers 1408 may be reflective spheres, such as optical markers or tracking markers 118, as previously described herein.

  Patient fixator 1402 may be attached to the rigid anatomical structure of patient 210 and remain attached during a surgical procedure. In the exemplary embodiment, the patient fixation device 1402 is attached to a rigid area of the patient 210, for example, a bone located away from the target anatomy targeted for surgical procedures. In order to track the target anatomical structure, the dynamic reference base 1404 is temporarily placed on or near the target anatomical structure to register the dynamic reference base 1404 at the location of the target anatomical structure. It is associated with the target anatomical structure through the use of registration fixtures placed.

  Registration fixture 1410 is attached to patient fixture 1402 using pivot arm 1412. The pivoting arm 1412 is attached to the patient lock 1402 by inserting the patient lock 1402 into the opening 1414 of the registration lock 1410. The pivoting arm 1412 is attached to the registration fixture 1410, for example, by inserting the knob 1416 into the opening 1418 of the pivoting arm 1412.

  The pivoting arm 1412 can be used to place the registration fixture 1410 on the target anatomical structure, the location of which using the tracking marker 1420 and / or the fiducial 1422 on the registration fixture 1410 It can be determined in image space and navigation space. The registration fixture 1410 can include a collection of markers 1420 visible in the navigation space (eg, the markers 1420 can be detectable by the tracking subsystem 532). Tracking marker 1420 may be an optical marker visible in infrared light, as previously described herein. The registration fixture 1410 can also include a collection of reference points 1422, such as bearing balls, visible in the imaging space (eg, a three-dimensional CT image). As described in more detail with reference to FIG. 11, the registration anchor 1410 can be used to associate the target anatomical structure with the dynamic reference base 1404, thereby providing an image of the anatomical structure. Descriptions of objects in the navigation space can be superimposed. A dynamic fiducial base 1404 located at a distance from the target anatomical structure provides a fiducial point by which the registration fixture 1410 and / or the pivoting arm 1412 can be removed from the surgical area.

  FIG. 11 provides an exemplary method 1500 for registration, in accordance with the present disclosure. The method 1500 may begin at step 1502 and import a graphical representation (or image) of the target anatomical structure into the system 100, 300, 600, eg, the computer 408. This graphical representation may be a three-dimensional CT or fluoroscopic scan of a target anatomical structure of patient 210, including registration fixture 1410 and imaging pattern 1420 of a detectable reference.

  In step 1504, the imaging pattern of the reference point 1420 is detected, registered in the imaging space, and stored in the computer 408. Optionally, at this point in step 1506, a graphical representation of registration fixture 1410 can be superimposed on the image of the target anatomical structure.

  At step 1508, by recognizing the marker 1420, the navigation pattern of the registration fixture 1410 is detected and registered. Marker 1420 may be an optical marker that is recognized in the navigation space by infrared light by tracking subsystem 532 via position sensor 540. Thus, the location, orientation, and other information of the target anatomical structure is registered in the navigation space. Thus, registration fixtures 1410 can be recognized both in the image space by the use of reference point 1422 and in the navigation space by the use of marker 1420. At step 1510, the registration of the registration fixture 1410 in the image space is transferred to the navigation space. This transfer is performed, for example, by using the relative position of the imaging pattern of the reference 1422 compared to the position of the navigation pattern of the marker 1420.

  At step 1512, the registration of the navigation space of registration fixture 1410 (registered in the image space) is further transferred to the navigation space of dynamic registration array 1404 attached to patient fixture 1402. Thus, as the navigation space is associated with the image space, removing the registration fixture 1410 and using the dynamic reference base 1404 to track the target anatomical structure in both the navigation space and the image space it can.

  In steps 1514 and 1516, the navigation space may be overlaid on the image space and the object using markers that can be recognized in the navigation space (eg, surgical instruments 608 using optical markers 804). Objects can be tracked through the graphical representation of the surgical instrument 608 on the image of the target anatomical structure.

  12A-12B also illustrate an imaging device 1304 that may be used with the robotic system 100, 300, 600 to obtain pre-operative, intra-operative, post-operative, and / or real-time image data of the patient 210. Any suitable subject can be imaged for any suitable procedure using imaging system 1304. Imaging system 1304 may be any imaging device, such as imaging device 1306 and / or a C-arm 1308 device. It may be desirable to acquire x-rays of the patient 210 from several different locations without the need for frequent manual repositioning of the patient 210 as required by the x-ray system. As shown in FIG. 12A, the imaging system 1304 may be in the form of a C-arm 1308 that includes an elongated C-shaped member that terminates in a “C” shaped opposite distal end 1312. C-shaped member 1130 can further include an x-ray source 1314 and an image receptor 1316. Space in the C-arm 1308 of the arms can provide room for a physician to diagnose a patient substantially free of interference from the x-ray support structure 1318. As shown in FIG. 12B, the imaging system includes a gantry housing 1324 mounted to a support structure imaging device support structure 1328, such as a wheeled mobile cart 1330 that includes a wheel 1332 that can surround an image capture portion not shown. An imaging device 1306 can be included. The image capture units are disposed at approximately 180 degrees relative to one another and an x-ray source and / or emitter attached to a rotator (not shown) for tracking of the image capture units, x-ray receiving and / or image receiving And can be included. The image capture unit may be operable to rotate 360 degrees during image acquisition. The image capture unit can be rotated about a center point and / or axis to allow image data of the patient 210 to be acquired from multiple directions or in multiple planes. While a particular imaging system 1304 is illustrated herein, it will be appreciated that any suitable imaging system may be selected by one of ordinary skill in the art.

  Referring now to FIGS. 13A-13C, surgical robotic system 100, 300, 600 may be configured to receive end effector 112, 602, surgical instrument 608, and / or patient 210 (eg, patient tracking device 116) to the desired surgical area. Depends on the exact positioning of In the embodiment shown in FIGS. 13A-13C, with reference to the tracking markers 118, 804 are fixedly attached to the instrument 608 and / or a portion of the end effector 112.

  FIG. 13A shows a portion of a surgical robotic system 100 that includes a robot 102 that includes a base 106, a robotic arm 104, and an end effector 112. Other elements not shown, such as a display, a camera, etc. may also be present as described herein. FIG. 13B shows an enlarged view of an end effector 112 having a guide tube 114 and a plurality of tracking markers 118 rigidly secured to the end effector 112. In this embodiment, a plurality of tracking markers 118 are attached to the guide tube 112. FIG. 13C shows an instrument 608 (in this case, a probe 608A) having a plurality of tracking markers 804 secured to the instrument 608. As described elsewhere herein, instrument 608 may include any suitable surgical instrument such as a guide wire, cannula, retractor, drill, reamer, screwdriver, insertion tool, removal tool, etc. It is not limited to them.

  When tracking the instrument 608, the end effector 112, or other objects tracked in 3D, the array of tracking markers 118, 804 may be fixedly attached to the tool 608 or a portion of the end effector 112. The tracking markers 118, 804 are preferably mounted such that the markers 118, 804 do not block (eg, do not interfere with surgery, visibility, etc.). The markers 118, 804 may be fixed to the instrument 608, the end effector 112, or other object to be tracked, for example using the array 612. Typically, three or four markers 118, 804 are used with the array 612. Array 612 may include straight portions, cross pieces, and may be asymmetric such that markers 118, 804 are in different relative positions and positions from one another. For example, as shown in FIG. 13C, a probe 608A having a 4-marker tracking array 612 is shown, and FIG. 13B shows an end effector 112 having a different 4-marker tracking array 612.

  In FIG. 13C, tracking array 612 functions as a handle 620 for probe 608A. Thus, the four markers 804 are attached to the handle 620 of the probe 608A and away from the shaft 622 and the tip 624. Stereographic tracking of these four markers 804 allows the instrument 608 to be tracked as a rigid body, and the tracking system 100, 300, 600 can be tipped while the probe 608A is moved forward of the tracking camera 200, 326. The position of 624 and the orientation of the shaft 622 can be accurately determined.

  Markers 118, such as each tool 608, end effector 112, to enable automatic tracking of one or more tools 608, end effector 112, or other objects (eg, multiple rigid bodies) tracked in 3D 804 are asymmetrically arranged at known inter-marker spacing. The reason for the asymmetric alignment is to make it clear which markers 118, 804 correspond to particular locations on the rigid body, and whether the markers 118, 804 are viewed from the front or back, ie, they are mirrored. is there. For example, if markers 118, 804 were placed in a square on tool 608 or end effector 112, it would be unclear to system 100, 300, 600 which markers 118, 804 correspond to which corners of the square . For example, in the case of the probe 608A, it is unknown which marker 804 is closest to the shaft 622. Thus, it is not known how the shaft 622 extended from the array 612. Thus, each array 612, and thus each tool 608, end effector 112, or other object being tracked must have a unique marker pattern so that it can be distinguished from other tools 608 or other objects being tracked. You must. Asymmetry and unique marker patterns allow the system 100, 300, 600 to detect individual markers 118, 804 and check marker spacing against stored templates, which tools 608, end effector 112 Allows you to determine if it is representing another object. The detected markers 118, 804 can then be sorted automatically and assigned to each tracked object in the correct order. Without this information, as important, for example, alignment of tool tip 624 and shaft 622, unless the user manually specifies which detected marker 118, 804 corresponds to which position on each rigid body It would not be possible to perform rigid body calculations to extract certain geometric information. These concepts are generally known to those skilled in the art of 3D optical tracking methods.

  Referring now to FIGS. 14A-14D, an alternative version of end effector 912 having moveable tracking markers 918A-918D is shown. In FIG. 14A, an array having moveable tracking markers 918A-918D is shown in a first configuration, and in FIG. 14B, moveable tracking markers 918A-918D are shown in a second configuration inclined relative to the first configuration. . FIG. 14C shows a template of tracking markers 918A-918D, for example as viewed by the cameras 200, 326 in the first configuration of FIG. 14A. FIG. 14D shows a template of tracking markers 918A-918D, as seen, for example, by cameras 200, 326 in the second configuration of FIG. 14B.

  In this embodiment, four marker array tracking is contemplated, but not all markers 918A-918D are at a fixed position relative to the rigid body, instead one or more array markers 918A-918D may for example be tested The latest information on the tracked rigid body can be provided without interrupting the process for automatic detection and sorting of the markers 918A-918D that are adjusted and tracked in.

  When tracking any tool, such as a guide tube 914 connected to the end effector 912 of the robotic system 100, 300, 600, the main purpose of the tracking array is to update the position of the end effector 912 in the camera coordinate system It is. When using a fixation system, for example, as shown in FIG. 13B, the array 612 of reflective markers 118 fixedly extends from the guide tube 114. Since the tracking marker 118 is fixedly connected, the information of the marker position in the camera coordinate system also provides the exact position of the center line, tip and back end of the guide tube 114 in the camera coordinate system. Typically, information regarding the position of the end effector 112 from such an array 612, and information regarding the position of the target trajectory from another tracked source, moves the guide tube 114 to align with the trajectory, and orbits the tip It is used to calculate the required movement that must be input for each axis of the robot 102 to move to a specific position along the vector.

  In some cases, the desired trajectory is at an awkward or unreachable position, but can be reached if the guide tube 114 can be pivoted. For example, if the guide tube 114 is pivotable upwards beyond the limits of the pitch (wrist angle) axis, a very steep trajectory pointing away from the base 106 of the robot 102 may be reachable, although If the guide tube 114 is mounted parallel to the plate connecting it to the end of the wrist it may not be reachable. To reach such a trajectory, the base 106 of the robot 102 can be moved or different end effectors 112 with different guide tube attachments can be replaced with working end effectors. Both of these solutions can be time consuming and cumbersome.

  As best seen in FIGS. 14A and 14B, if array 908 does not have one or more of markers 918A-918D in a fixed position, instead one or more of markers 918A-918D adjust, pivot, If able to pivot or move, the robot 102 can provide updated information about the object being tracked without interrupting the detection and tracking process. For example, one of the markers 918A-918D may be fixed in place and the other markers 918A-918D may be movable. Two of the markers 918A-918D may be fixed in place and the other markers 918A-918D may be movable. Three of the markers 918A-918D may be fixed in place and the other markers 918A-918D may be movable. Alternatively, all of the markers 918A to 918D may be movable.

  In the embodiment shown in FIGS. 14A and 14B, markers 918A, 918B are fixedly connected directly to the base 906 of end effector 912 and markers 918C, 918D are fixedly connected to tube 914. Similar to array 612, array 908 can be provided to attach markers 918A-918D to end effector 912, instrument 608, or other object to be tracked. However, in this case, the array 908 is comprised of a plurality of separate components. For example, markers 918A, 918B may be connected to base 906 by a first array 908A and markers 918C, 918D may be connected to a guide tube 914 by a second array 908B. Markers 918A may be attached to the first end of the first array 908A, and markers 918B may be separated by a linear distance and secured to the second end of the first array 908A. The first array 908 is substantially linear, but the second array 908B has a bent or V-shaped configuration, with the root end of each connected to the guide tube 914 from which it is V-shaped It branches at the distal end and has a marker 918C at one distal end and a marker 918D at the other distal end. Although specific configurations are illustrated herein, it will be appreciated that other asymmetric designs are contemplated, including different numbers and types of arrays 908A, 908B and different arrangements, numbers, and types of markers 918A-918D. .

  Guide tube 914 may be movable, pivotable, or pivotable relative to base 906, for example, across hinges 920 or other connectors to base 906. Thus, the markers 918C, 918D are movable to pivot, pivot or move the markers 918C, 918D as the guide tube 914 pivots, pivots or moves. As best shown in FIG. 14A, guide tube 914 has longitudinal axis 916 aligned in a substantially normal or vertical orientation such that markers 918A-918D have a first configuration. Referring now to FIG. 14B, the guide tube 914 pivots such that the longitudinal axis 916 is at an angle to the vertical, such that the markers 918A-918D have a second configuration different from the first configuration. , Swivel or move.

  In contrast to the embodiment described in FIGS. 14A-14D, with all four markers 918A-918D fixedly attached to the guide tube 914, the guide tube 914 and the arm 104 (e.g. a wrist attachment) And when the swivel is adjusted by the user, the robotic system 100, 300, 600 can not automatically detect that the orientation of the guide tube 914 has changed. The robot system 100, 300, 600 tracks the position of the marker array 908 and assumes incorrect robot axis motion assuming that the guide tube 914 is attached to the wrist (robot arm 104) in the previous orientation. calculate. One or more markers 918A-918D (eg, two markers 918C, 918D) are fixedly held on the tube 914 and one or more markers 918A-918D (eg, two markers 918A, 918B) Holding the swivel across the swivel allows for automatic detection of the new position and the correct robot movement is calculated based on the detection of the new tool or end effector 112, 912 on the end of the robot arm 104.

  One or more markers 918A-918D are configured to be moved, pivoted, pivoted, etc. according to any suitable means. For example, markers 918A-918D may be moved by hinges 920 such as clamps, springs, levers, slides, toggles, or to move markers 918A-918D individually or in combination, or array 908A, 908B individually or Any for moving in combination or for moving any part of the end effector 912 relative to another, or for moving any part of the tool 608 relative to another Other suitable mechanisms may be used.

  As shown in FIGS. 14A and 14B, the array 908 and the guide tube 914 simply loosen the clamps or hinges 920 and move a portion of the arrays 908A, 908B relative to the other portions 908A, 908B to move the hinges 920. It may be possible to configure the guide tube 914 to be directed to different positions by readjustment. For example, two markers 918C, 918D are fixedly interconnected with the tube 914, and two markers 918A, 918B are fixedly interconnected with the base 906 of the end effector 912 attached to the robot arm 104 via the hinge 920. obtain. The hinge 920 is in the form of a clamp, such as a wing nut that can be loosened and tightened to allow the user to quickly switch between the first configuration (FIG. 14A) and the second configuration (FIG. 14B) It may be.

  Cameras 200, 326 detect markers 918A-918D, for example, in one of the templates identified in FIGS. 14C and 14D. If the array 908 is in the first configuration (FIG. 14A) and the tracking cameras 200, 326 detect the markers 918A-918D, the tracked markers match the array template 1 as shown at 14C. If the array 908 is in the second configuration (FIG. 14B) and the tracking cameras 200, 326 detect the same markers 918A-918D, the tracked markers will coincide with the array template 2 as shown in FIG. 14D. The array template 1 and the array template 2 are recognized by the system 100, 300, 600 as two different tools having a uniquely defined spatial relationship between the guide tube 914, the markers 918A-918D, and the robot attachment. Thus, the user can adjust the position of the end effector 912 between the first and second configurations without notifying the system 100, 300, 600 of the change, the system 100, 300, 600 being a robot Adjust the motion of 102 appropriately to be in orbit.

  In this embodiment, there are two assembly locations that match the marker array with a unique template that allows the system 100, 300, 600 to recognize the assembly as two different tools or two different end effectors. Between any of these two positions or at any position on the outer swivel (ie, array template 1 and array template 2 shown in FIGS. 14C and 14D, respectively) markers 918A-918D do not match any of the templates and the system 100, 300, 600 do not detect the array present despite the individual markers 918A-918D being detected by the camera 200, 326, and the markers 918A-918D are temporarily out of view of the camera 200, 326 It has the same effect as if it were blocked. It will be appreciated that other array templates may be present, for example, for other configurations that identify different instruments 608 or other end effectors 112, 912, etc.

  In the described embodiment, two separate assembly positions are shown in FIGS. 14A and 14B. However, it will be appreciated that there may be multiple discrete locations on the swivel joint, straight joint, combination swivel joint and straight joint, peg board, or other assembly. Here, unique marker templates can be created by aligning the positions of one or more markers 918A-918D in the array with one another, each discrete position being matched to a specific template and known differently Define a unique tool 608 or end effector 112, 912 with the following attributes: Further, although illustrated for end effector 912, it will be appreciated that movable fixation markers 918A-918D may be used with any suitable instrument 608 or other object to be tracked.

  When tracking a complete rigid array of three or more markers attached to the end effector 112 of a robot using an external 3D tracking system 100, 300, 600 (eg, as shown in FIGS. 13A and 13B), the camera The 3D position of each section of the robot 102 in the 200, 326 coordinate system can be directly tracked or calculated. The geometric orientation of the joint with respect to the tracker is known by design, and the linear or angular position of the joint is known from the encoders of each motor of the robot 102 and of all the movable parts from the end effector 112 to the base 116 Fully define the 3D position. Similarly, if the tracker is attached to the base 106 (not shown) of the robot 102, the robot 106 from the base 106 to the end effector 112 based on known joint geometry and joint positions from the encoders of each motor. It is also possible to track or calculate the 3D position of each section.

  In some situations, it may be desirable to track the position of all segments of the robot 102 from less than three markers 118 fixedly attached to the end effector 112. Specifically, when tool 608 is introduced into guide tube 114, it may be desirable to track only one additional marker 118 to track the full rigid motion of robot 902.

  Referring now to FIGS. 15A-15E, an alternative version of an end effector 1012 having only a single tracking marker 1018 is shown. The end effector 1012 may be similar to the other end effectors described herein and may include a guide tube 1014 extending along the longitudinal axis 1016. A single tracking marker 1018 similar to the other tracking markers described herein can be secured to the guide tube 1014. This single marker 1018 serves the purpose of adding the missing degrees of freedom to enable full rigid body tracking and / or as a surveillance marker to ensure that assumptions regarding robot and camera positioning are valid It can serve for the purpose of functioning.

  A single tracking marker 1018 may be attached to the robotic end effector 1012 as a rigid extension to the end effector 1012 that protrudes in any convenient direction and does not block the surgeon's view. The tracking marker 1018 can be fixed at the guide tube 1014 or any other suitable position on the end effector 1012. When attached to the guide tube 1014, the tracking marker 1018 may be located at a position between the first end and the second end of the guide tube 1014. For example, in FIG. 15A, a single tracking marker 1018 extends forward from the guide tube 1014 and is located on the longitudinal midpoint of the guide tube 1014 and below the entrance of the guide tube 1014. It is shown as a reflective ball attached to the end. This position not only allows the marker 1018 to be viewed entirely by the camera 200, 326, but also does not block the view of the surgeon 120 and does not collide with other tools or objects in the vicinity of the surgery. Further, the guide tube 1014 with the markers 1018 in this position allows the marker array on any tool 608 introduced into the guide tube 1014 to be viewed simultaneously with the single marker 1018 on the guide tube 1014 Designed.

  As shown in FIG. 15B, when a snug fit tool or instrument 608 is placed in the guide tube 1014, the instrument 608 is mechanically constrained in four of six degrees of freedom. That is, the instrument 608 can not be rotated in any direction except for the longitudinal axis 1016 of the guide tube 1014 and the instrument 608 translates in other directions along the longitudinal axis 1016 of the guide tube 1014 I can not do it. In other words, instrument 608 can only translate and rotate along the centerline of guide tube 1014. If two more parameters are known, such as (1) the rotation angle around the longitudinal axis 1016 of the guide tube 1014, and (2) the position along the guide tube 1014, the position of the end effector 1012 in the camera coordinate system is complete Defined in

  Next, as shown in FIG. 15C, the system 100, 300, 600 has the tool 608 actually located inside the guide tube 1014 and instead instead of outside the guide tube 1014, somewhere in the field of view of the camera 200, 326 You should be able to know when you are. Tool 608 has a longitudinal axis or center line 616 and an array 612 having a plurality of tracked markers 804. Using rigid body calculations, based on the tracking position of the array 612 on the tool 608, it can be determined where the centerline 616 of the tool 608 is located within the camera coordinate system.

A fixed normal (vertical) distance D _F from the single marker 1018 to the centerline or longitudinal axis 1016 of the guide tube 1014 is fixed and geometrically known, tracking the position of the single marker 1018 be able to. Thus, if the detection distance D _D from the tool centerline 616 to the single marker 1018 matches the known fixed distance D _F from the guide tube centerline 1016 to the single marker 1018, then the tool 608 is in the guide tube 1014 Determined that there is a point (the center line 616, 1016 of the tool 608 and the guide tube 1014 coincide) or that this distance D _D is at a point in the locus of positions that may correspond to the fixed distance D _F it can. For example, as shown in FIG. 15C, the detection distance D _D in the normal direction from the tool center line 616 to the single marker 1018, at the two positions, the both data frames represented by a transparent tool 608 (tracking (The marker coordinates) correspond to the fixed distance D _F from the guide tube center line 1016 to the single marker 1018. Thus, additional considerations may be required to determine when the tool 608 is placed in the guide tube 1014.

Referring now to FIG. 15D, using the programmed logic, the tool 608 can move against a single sphere 1018 in space to satisfy the condition that the tool 608 is moving within the guide tube 1014. It is possible to find a frame of tracking data where the detection distance D _D from the tool centerline 616 to the single marker 1018 remains fixed at the correct length, even though it has moved by some minimum distance. For example, the first frame F1 may be detected by the tool 608 at the first position, and the second frame F2 may be detected by the tool 608 at the second position (ie linear with respect to the first position) Moved to The markers 804 on the tool array 612 can move from the first frame F1 to the second frame F2 by more than a given amount (e.g., more than 5 mm total). Even with this movement, the detection distance D _D from the tool center line vector C ′ to the single marker 1018 is substantially the same in both the first frame F1 and the second frame F2.

  Logically, the surgeon 120 or user can place the tool 608 in the guide tube 1014 and slightly rotate or slide it into the guide tube 1014, the system 100, 300, 600 From the five markers (four markers 804 on tool 608 plus a single marker 1018 on guide tube 1014), it can be detected that it is in guide tube 1014. Knowing that the tool 608 is in the guide tube 1014, it is possible to calculate all six degrees of freedom that define the position and orientation of the robotic end effector 1012 in space. Without the single marker 1018, even though the tool 608 is known to be reliably within the guide tube 1014, where the guide tube 1014 is located along the tool center line vector C 'and the guide tube 1014 It is unknown how to rotate with respect to the center line vector C '.

  As highlighted in FIG. 15E, a single marker 1018 is tracked in conjunction with the four markers 804 on tool 608 so that centerline vector C 'and single marker 1018 and centerline vector C of guide tube 1014 and tool 608 are tracked. The 'normal vector can be constructed. This normal vector is at a known orientation relative to the robot's forearm (distantly parallel to that segment in this example) distal to the wrist and intersects the centerline vector C 'at a particular fixed location It has an orientation. For convenience, as shown in FIG. 15E, three mutually orthogonal vectors k ', j', i 'can be configured, and rigid body positions and orientations of the guide tube 1014 can be defined. One of three mutually orthogonal vectors, k ', is composed of a center line vector C', and the second vector j 'is composed of a normal vector passing through a single marker 1018, and a third The vector i 'is the vector product of the first and second vectors k', j '. The robot's joint positions for these vectors k ', j', i 'are known when all joints are zero, and are fixed, so using rigid body calculations, when the robot is at home position, The position of any part of the robot relative to the vectors k ', j', i 'can be determined. During movement of the robot, the position of the tool marker 804 (while the tool 608 is in the guide tube 1014) and the position of the single marker 1018 are detected from the tracking system and the angle / linear position of each joint is known from the encoder For example, the position and orientation of any part of the robot can be determined.

  In some embodiments, it may be useful to fix the orientation of the tool 608 relative to the guide tube 1014. For example, the end effector guide tube 1014 may be oriented at a particular position about its axis 1016 to allow for machining or positioning of the implant. The orientation of any objects attached to the tool 608 inserted into the guide tube 1014 can be seen from the tracked markers 804 on the tool 608 but additional tracking markers 1018 on the guide tube 1014 (or in other embodiments, Without multiple tracking markers, the direction of rotation of the guide tube 1014 itself in the camera coordinate system is unknown. This marker 1018 provides a “clock position” of −180 ° to + 180 ° essentially based on the orientation of the marker 1018 with respect to the centerline vector C ′. Thus, a single marker 1018 can provide additional degrees of freedom to allow full rigid body tracking and / or act as a surveillance marker to ensure that assumptions regarding robot and camera positioning are valid. be able to.

  FIG. 16 is a block diagram of a method 1100 for navigating and moving an end effector 1012 (or any other end effector described herein) of the robot 102 to a desired target trajectory. Another use of a single marker 1018 on the robotic end effector 1012 or guide tube 1014 is part of a method 1100 that enables automatic safe movement of the robot 102 without a full tracking array attached to the robot 102. In this method 1100, the tracking camera 200, 326 is not moved with respect to the robot 102 (ie, they are in a fixed position), the coordinate system of the tracking system and the coordinate system of the robot are registered together, the position of the guide tube 1014 And function when the robot 102 is calibrated so that it can be accurately determined in the robot's Cartesian coordinate system based solely on the encoded position of each robot axis.

  In this method 1100, the tracker and robot coordinate systems must be registered together, which means that a coordinate transformation from the tracking system Cartesian coordinate system to the robot Cartesian coordinate system is required. For convenience, this coordinate transformation may be a 4x4 translation and rotation matrix as is well known in the robotics field. This transformation is called Tcr and refers to "transformation-camera to robot". Once this transformation is known, a new frame of tracking data, received as x, y, z coordinates in vector form for each marker tracked, can be multiplied by the 4x4 matrix, and the resulting x, y, z coordinates are robot In the coordinate system of. In order to obtain Tcr, the robot's full tracking array is tracked while fixedly attached to the robot at a known position in the robot's coordinate system, and then the known rigid body method calculates the transformation of the coordinates. Used for It is clear that any tool 608 inserted into the guide tube 1014 of the robot 102 can provide the same rigid body information as the fixedly mounted array when the additional markers 1018 are also read. That is, the tool 608 may be inserted into any position in the guide tube 1014 at any rotation in the guide tube 1014. There is no need to insert in a fixed position and orientation. Thus, Tcr can be determined by inserting an optional tool 608, including a tracking array 612, into the guide tube 1014 and reading the array of tools 612 and the single markers 1018 of the guide tube 1014. At the same time, the encoder on each axis can determine the current position of the guide tube 1014 in the robot's coordinate system.

  The logic for navigating and moving the robot 102 to the target trajectory is provided in the method 1100 of FIG. Before entering loop 1102, assume that the transformation Tcr was previously stored. Thus, before entering the loop 1102, after the robot base 106 is fixed in step 1104, one or more frames of tracking data of the tool inserted in the guide tube while the robot is stationary is stored Be done. At step 1106, from this static data and previous calibration data, a transformation of the robot guide tube position from camera coordinates to robot coordinates Tcr is calculated. Tcr should be valid as long as the cameras 200, 326 do not move relative to the robot 102. If the cameras 200, 326 move relative to the robot 102 and need to reacquire Tcr, the system 100, 300, 600 prompts the user to insert the tool 608 into the guide tube 1014 and then necessary. Calculations can be performed automatically.

  In the flowchart of method 1100, each frame of data collected is a snapshot of the tracked position of DRB 1404 on patient 210, the tracked position of single marker 1018 on end effector 1014, and the position of each robot axis. It consists of From the position of the robot's axis, the position of a single marker 1018 on the end effector 1012 is calculated. This calculated position is compared to the actual position of the marker 1018 recorded from the tracking system. If the values match, it can be guaranteed that the robot 102 is at a known position. The transformation Tcr is applied to the tracked position of the DRB 1404 so that the robot's 102 target is provided with respect to the robot's coordinate system. The robot 102 may then be instructed to move to reach the target.

  After steps 1104, 1106, loop 1102 receives rigid body information of DRB 1404 from the tracking system 1108, converts target tip and trajectory from image coordinates to tracking system coordinates 1110, and camera tip coordinates and target coordinates And convert the robot coordinates (apply Tcr) 1112. The loop 1102 further comprises the steps 1114 of receiving a single stray marker position for the robot from the tracking system and converting 1116 the single stray marker from tracking system coordinates to robot coordinates (apply storing Tcr) 1116 including. Loop 1102 also includes determining 1118 the current position of a single robot marker 1018 in the robot coordinate system from forward kinematics. The information from steps 1116 and 1118 is used to determine if stray marker coordinates from the transformed tracked position match the calculated coordinates less than a predetermined tolerance (step 1120) . If yes, proceed to step 1122 to calculate and apply robot motion to the targets x, y, z and trajectory. If no, proceed to step 1124 and stop and request complete array insertion into the guide tube 1014 before continuing with the process and recalculate Tcr after the array is inserted at step 1126. Thereafter, the steps 1108, 1114 and 1118 are repeated.

  This method 1100 has advantages over methods in which continuous monitoring of a single marker 1018 is omitted to confirm position. Without the single marker 1018, it is possible to use Tcr to determine the position of the end effector 1012 and transmit the end effector 1012 to the target position, but that the robot 102 is actually at the expected location Can not confirm. For example, if the cameras 200, 326 hit and Tcr is no longer valid, the robot 102 will move to the wrong position. Thus, a single marker 1018 provides safety value.

  For a given fixed position of the robot 102, theoretically, the tracking camera 200, 326 remains unmoved as the single tracked marker 1018 is a single point rather than an array It is possible to move to a new position. In such a case, the system 100, 300, 600 does not detect any errors, as the calculated position of the single marker 1018 matches the tracked position. However, if the robot's axis moves the guide tube 1012 to a new position, the calculated position and the tracked position will be inconsistent and the safety check will be valid.

  The term "monitoring marker" can be used, for example, with reference to a single marker in a fixed position relative to DRB 1404. In this example, if the DRB 1404 is bumped or otherwise removed, the relative position of the monitoring markers may change, alerting the surgeon 120 that there may be navigation problems. . Similarly, in the embodiments described herein, using a single marker 1018 on the robot's guide tube 1014, the system 100, 300, 600 determines whether the camera 200, 326 has moved relative to the robot 102. Can be checked continuously. The system 100, 300, 600 is not a user if the tracking system's coordinate system loses registration with the robot's coordinate system due to the camera 200, 326 hitting or malfunctioning or the robot malfunctioning. Can warn you and make corrections. Thus, this single marker 1018 can also be considered as a surveillance marker for the robot 102.

  In the case of a full array permanently attached to the robot 102 (eg, multiple tracking markers 702 on the end effector 602 shown in FIGS. 7A-7C), such functionality of a single marker 1018 such as a robot surveillance marker is Cameras 200, 326 are not required to be at a fixed position relative to robot 102, and Tcr is not necessary because it is updated at each frame based on the tracked position of robot 102. The reason for using a single marker 1018 instead of a full array is that the full array is more bulky, which prevents the surgeon's view and access to the surgical area 208 more than a single marker 1018 Because the line of sight to the full array is more easily blocked than the line of sight to the single marker 1018.

  Referring now to FIGS. 17A-17B and 18A-18B, an instrument 608 such as an implant holder 608B, 608C that includes both fixed and movable tracking markers 804, 806 is shown. The implant holder 608B, 608C may have a handle 620 and an outer shaft 622 extending from the handle 620. The shaft 622 may be disposed substantially perpendicular to the handle 620, as shown, or in any other suitable orientation. The inner shaft 626 can extend through an outer shaft 622 having a knob 628 at one end. The implants 10, 12 are connected at the other end to the shaft 622 at the tip 624 of the implant holder 608B, 608C using a typical connection mechanism known to those skilled in the art. The knob 628 can be rotated, for example, to expand or articulate the implants 10,12. U.S. Patent Nos. 8,709,086 and 8,491,659, which are incorporated herein by reference, describe expandable fusion devices and attachment methods.

  When tracking tools 608, such as implant holders 608B, 608C, tracking array 612 comprises array 612 or otherwise fixed to implant holders 608B, 608C, with fixed markers 804 and one or more movable A combination of markers 806 can be included. The navigation array 612 can include at least one or more (e.g., at least two) fixed position markers 804 located at known positions relative to the implant holder devices 608B, 608C. These fixed markers 804 can not move in any direction relative to the geometry of the instrument and are useful to define where the instrument 608 is in space. In addition, there is at least one marker 806, which can be attached (eg, slide, rotate, etc.) to the fixed marker 804 and attached to the array 612 or the instrument itself. The system 100, 300, 600 (eg, software) relates the position of the moveable marker 806 to a particular position, orientation, or other attribute of the implant 10 (eg, between the expandable vertebral bodies shown in FIGS. 17A-17B. Spacer height, angle of articulating interbody spacer shown in FIGS. 18A-18B). Thus, the system and / or user can determine the height or angle of the implants 10, 12 based on the position of the moveable marker 806.

  In the embodiment shown in FIGS. 17A-17B, four fixation markers 804 are used to define the implant holder 608B, and the fifth moveable marker 806 is slid within a predetermined path to achieve an implant height (eg, Feedback can be provided at the retracted or expanded position). FIG. 17A shows the expandable spacer 10 at its initial height, and FIG. 17B shows the spacer 10 in the expanded state with the moveable markers 806 translated to different positions. In this case, the moveable marker 806 approaches the fixation marker 804 when the implant 10 is expanded, but it is contemplated that this movement may be reversed or otherwise different. The amount of linear translation of the marker 806 corresponds to the height of the implant 10. Although only two positions are shown, it is possible to have this as a continuous function, so that any predetermined expansion height can be correlated to a particular position of the moveable marker 806.

  Referring now to FIGS. 18A-18B, four fixation markers 804 are used to define the implant holder 608C, and a fifth moveable marker 806 is a predetermined path to provide feedback on implant joint angles. It is configured to slide within. FIG. 18A shows the articulated spacer 12 in its initial linear state, and FIG. 18B shows the spacer 12 in an articulated state at an offset angle where the moveable marker 806 has been moved to a different position. The linear translational amount of the marker 806 corresponds to the joint angle of the implant 12. Although only two positions are shown, it is possible to have this as a continuous function, whereby any given joint angle can be correlated to a particular position of the moveable marker 806.

  In these embodiments, the moveable marker 806 slides continuously to provide feedback regarding the attributes of the implant 10, 12 based on position. It is also contemplated that there may be close positions where the moveable marker 806 must be able to provide further information regarding implant attributes. In this case, the careful configuration of all markers 804, 806 correlates with the particular geometry of the implant holders 608B, 608C and the implants 10, 12 at a particular orientation or a particular height. In addition, any movement of the moveable marker 806 can be used for other variable attributes of any other type of navigated implant.

  Although shown and described with respect to linear movement of the moveable marker 806, the moveable marker 806 should not be limited to just sliding. There may be applications where rotation or other movement of the marker 806 may be useful to provide information regarding the implant 10, 12. The relative change in position between the set of fixed markers 804 and the moveable markers 806 can be relevant information regarding the implants 10, 12 or other devices. Further, although expandable and articulated implants 10, 12 are illustrated, the instrument 608 may include other medical devices and spacers, cages, plates, fasteners, nails, screws, rods, pins, wire structures, sutures, anchors Materials such as clips, staples, stents, bone grafts, biologics, cements, etc. can also function.

  Referring now to FIG. 19A, it is envisioned that the robotic end effector 112 is interchangeable with other types of end effectors 112. Further, it is contemplated that each end effector 112 may perform one or more functions based on the desired surgical procedure. For example, end effector 112 having guide tube 114 may be used to guide instrument 608 as described herein. Further, end effector 112 can be replaced, for example, by a different or alternative end effector 112 that controls a surgical device, instrument, or implant.

  The alternative end effector 112 may include one or more devices or instruments coupled to the robot and controllable by the robot. As a non-limiting example, the end effector 112 shown in FIG. 19A may be a retractor (eg, one or more retractors disclosed in US Pat. Nos. 8,992,425 and 8,968,363) One or more mechanisms for inserting or placing a surgical device may be included, such as: It may be a stand-alone intervertebral fusion device (such as the expandable implants illustrated in US Pat. Nos. 8,845,734, 9,510,954 and 9,456,903) Fusion devices (such as implants as exemplified in US Patent Nos. 9,364,343 and 9,480,579), expandable bone excision apparatus (US Patent Nos. 9,393,128 and 9, Bone correction implants such as 173, 747), articulating spacers (such as implants illustrated in US 9,259, 327), faceted prostheses (US 9, 539, 031) Devices, etc.), laminoplasty devices (such as the devices illustrated in US Pat. No. 9,486,253), spinous process spacers (US Pat. No. 9, Implants such as 92,082), inflatable products, fasteners including multi-axial screws, single plane screws, pedicle screws, post screws etc., bone fixation plates, rod constructions and modification devices (US Pat. Devices such as 882, 803), artificial and natural disks, motion preservation devices, implants, and spinal cord stimulators (such as devices exemplified in US Pat. No. 9,440, 076), and other surgical applications It is a device. The end effector 112 may be one or more directly or indirectly coupled to a robot to provide bone cement, bone grafts, live cells, pharmaceuticals, or other deliverables to a surgical target. An instrument can be included. End effector 112 may also include one or more instruments designed to perform discectomy, scoliosis, vertebral suture, dilation, or other surgical procedures.

  The end effector itself and / or the implant, device or instrument can include one or more markers 118 such that the location and position of the markers 118 can be identified in three dimensions. It is contemplated that the markers 118 can include the active or passive markers 118 described herein that are directly or indirectly visible to the camera 200. Thus, for example, one or more markers 118 disposed on the implant 10 can provide tracking of the implant 10 before, during and after the implant.

  As shown in FIG. 19B, the end effector 112 may include an instrument 608 or a portion thereof coupled to the robotic arm 104 (eg, the instrument 608 may be configured with the robotic arm 104 by the coupling mechanism shown in FIGS. 9A-9C). Can be controlled by the robot system 100). Thus, in the embodiment shown in FIG. 19B, the robotic system 100 can insert the implant 10 into a patient and cause the expandable implant 10 to expand or contract. Thus, the robotic system 100 can be configured to assist the surgeon or to operate partially or completely independently. Thus, it is envisioned that robotic system 100 can control each alternative end effector 112 for its particular function or surgical procedure.

  Although the robots and associated systems described herein are generally described with reference to spinal applications, the robotic system is configured to be used in other surgical applications. Other surgeries include trauma or other orthopedic surgeries (eg, placement of intramedullary nails, plates, etc.), skull, nerves, cardiothoracic, blood vessels, colorectal, oncology, dental and other surgeries And treatments, but is not limited thereto.

  Then, during robotic spine (or other) treatment, a dynamic reference base (DRB) can be fixed to the patient (eg, to the patient's bone) and used to track the patient's anatomy. Because the patient is breathing, the position of the DRB (attached to the patient's body) may vibrate. The position of the fixed guide tube of the end effector can be controlled automatically by robot control to keep it continuously aligned with the target trajectory during such vibrations. However, once the surgical tool has been introduced into the guide tube, the automatic position control can be stopped for safety reasons and the robot will remain in a static pose and in a fixed state. After this, while the end effector (e.g., a surgical tool) remains locked in place relative to the operating room, patient movement (e.g., by respiration) can cause a deviation from the target trajectory. Thus, this offset / shift (unrecognized, unintended) may reduce the accuracy of the system and / or surgery.

  In some embodiments of the inventive concept, detection of patient motion (eg, by respiration) may be improved and / or positioning may be improved. For example, information from the remote sensor system can be used to generate an indication of the effects of breathing on the positioning of the robotic end effector. Such offsets are the offsets (differences) between the actual end effector trajectory and the target (ie, planned) end effect trajectory used for placement of, for example, a spinal screw (or other medical device / implant / procedure) Can be monitored on the basis of If patient breathing is significant, the resulting offset may change the distance of the end effector's actual trajectory from the target (plan) trajectory. In some embodiments of the inventive concept, this offset may be provided on the display 110. In some embodiments, the graphic meter may be shown on the display 110 in three separate sections. These sections may be colored to indicate the degree of shift in green, yellow and red. In procedures where respiration is considered to be excessive, the user (eg, a surgeon) may limit the amount of respiration or facilitate overall patient breathing over a short period of time to help position the end effector more accurately. Anesthesiologist can be required to stop.

  20A, 20B, and 20C illustrate three embodiments of a breathing meterer structure that may be provided as a graphic meter on display 110. FIG. In FIG. 20A, a meter using a rounded dial configuration may be illustrated having a "needle" 2001a indicating the degree of deviation. In FIG. 20B, a meter using a horizontal bar configuration can be illustrated with a "needle" 2001b indicating the degree of deviation. In FIG. 20C, a meter using a vertical bar configuration can be illustrated with a “needle” 2001 c indicating the degree of deviation. In any of the embodiments of FIGS. 20A-C, the “needle” may be omitted if the illumination is used to indicate the degree of deviation. For example, the green area (or a portion thereof) can be illuminated (without illuminating the yellow and red areas) to indicate low deviations, the green area and the yellow area (or a portion of the yellow area) being moderate The green area, the yellow area, and the red area (or a portion of the red area) may be illuminated to indicate the degree of misalignment (the red area may not be illuminated). It can be done. In addition, the meter can be used to dynamically display changing deviations (eg, due to respiration) in real time.

  Additionally or alternatively, when the robotic system should update the position of the end effector to reduce steady-state errors generated by deviations from the planned position using a meter or other visual or auditory output Can be displayed. For example, because the DRB is fixed to the patient, the meter may reflect a steady deviation from the target trajectory that results from the patient's motion on the surgical bed. By providing this information to the user (e.g., a surgeon or other member of the surgical team) when the user selects when to activate the robotic arm (i.e., close the feedback loop) to reduce steady-state misalignment. May be possible.

  The magnitude of the deviation between the actual trajectory and the target trajectory can vary widely among different patients, as not everyone will breathe with the same intensity in the same way. Furthermore, the desired deviation between the end effector's target trajectory and the actual trajectory (the optimum deviation is considered to be zero) can be displayed at the green end of the meter, so that the end effector's target trajectory and the actual trajectory Extreme misalignment between can be displayed at the red end of the meter. Furthermore, the difference between the desired and the extreme deviation can result from the movement of only one to two millimeters of the DRB.

  Thus, the meter displays the real-time deviation between the end effector's actual trajectory and the target trajectory (e.g., caused by a single cycle of a breathing movement and / or a single movement of the patient's body). Can be used to Such offsets displayed are robotically controlled to move the end effector to the target trajectory and allow the user (e.g., a recognition (e.g., a user) to be able to settle to the correct target (plan) position defined by the end effector's target trajectory). , Or other surgical team members). By tracking the motion using the DRB, if the use of any instrument causes a significant shift in the patient's position, the meter will display a change, and thus the user is again required to robotically control the end effector Informing it to move to the target trajectory, thereby reducing the offset between the actual trajectory and the target trajectory until any such offset is within tolerance.

  In some embodiments, the surgical robotic system may use feedback from a remote sensor (eg, tracking camera 200) to determine the position of the DRB and the robotic arm end effector, and to adjust the DRB relative to the anatomical position. Where the position is substantially fixed, a fixed offset may be used to determine a particular anatomical position of the patient relative to the DRB. For example, if the DRB is fixed to the spine and the anatomical position is a position on the spine due to the placement of the screw, then the position of the DRB relative to the anatomical position is fixed at the DRB during all stages of respiration. Can be substantially fixed (even if the spine moves for breathing) so that it can be used to determine an anatomic position based on the position of the

  In some other embodiments, the offset between the DRB and the anatomical location may be variable. For example, if the DRB is fixed to the spine and the anatomical location is in soft tissue (e.g., an organ such as the lung) remote from the DRB, the offset between the DRB and the anatomical location may vary between respiratory cycles. Can change over various stages. In such cases, the position of the DRB can not only be used to track the targeted anatomical structure. The robotic robot system may use modeling to provide a variable offset that is used to determine the position of the target anatomical position based on the position of the DRB during various stages of the breathing cycle.

  In some embodiments of the inventive concept, the determination of the deviation between the target trajectory and the actual trajectory of the meter of Figures 20A, 20B, and / or 20C and / or the end effector is a biopsy of the lung or organ It may be used to provide improved robotic guidance during procedures that may be affected by respiration, such as other soft tissue treatments. As shown in FIGS. 21A and 21B, if the motion of the lesion to be biopsied is not fixed relative to the movement of the DRB during respiration (ie, the offset between the DRB and the lesion is variable throughout the breathing cycle), In addition, mathematical or experimental predictions of the offset of the target needle trajectory relative to the attachment position of the DRB at various stages of respiration may be helpful. For example, the offset between the DRB and the lesion when the lungs contract as shown in FIG. 21A may be different than the offset between the DRB and the lesion when the lungs as shown in FIG. 21B. Such predictions may be based on modeling of the tissue and / or estimations that model when different parts of the lung move during respiration. That is, by examining how the lungs normally function during each phase of respiration, a computational model can be created that provides the location of any position of the lungs throughout the respiratory cycle. The model may then be applied to a particular patient having a lesion at a particular location. Alternatively, the path of lesion movement may be obtained experimentally for the patient by taking x-rays or other types of images while simultaneously recording the stages of breathing. Data frames containing images with respiratory stages may be compiled, and a lesion offset vs. stage look-up table or suitable formula is created and referenced during the surgical procedure to guide the robotic arm positioning the end effector. This information can be used to determine the offset of the DRB relative to the lesion at various stages of respiration, such that knowledge of the location of the DRB can be used to determine the corresponding position of the lesion over different stages of respiration. It can be used.

  FIG. 21A (lung contracted) and FIG. 21B (lung expanded) are schematic views of the torso from an axial perspective showing the effect of respiration at the spatial location of the lesion to be DRB. In this example, tracking the lesion based on a fixed offset from the DRB may not be sufficient to accurately track the location of the lesion so that the size and direction of motion of the bone to which the DRB is attached is that of the lung The size and direction of movement of the lesion may be different. Although the location of lung lesions can not be tracked directly during the various stages of respiration, the location of lung lesions relative to the anchor point of the DRB can be modeled or otherwise determined. The robotic guide tube (or other end effector) position may be adjusted so that its trajectory intersects the actual lesion position during multiple / all stages of lung contraction / inflation.

  Functionally, the robotic system tracks the DRB directly as base coordinates, and also for soft tissue biopsy based on variable offset look-up tables or mathematical models (ie positional offsets) and respiratory staging. The position of the guide tube can be adjusted. In one embodiment, the robot system can adjust its position continuously and automatically based on the tracked respiration so that the surgeon can deploy a biopsy needle at any time to accurately target the lesion. In another embodiment, the robot is maintained at a position corresponding to a particular stage, and when the biopsy needle is in the proper position for manual deployment by the surgeon, a meter (eg, FIGS. 20A, 20B, And / or can be displayed to the user through a 20C meter). The surgeon will check the meter and will deploy a biopsy needle as appropriate. In another embodiment, the robotic system can be kept current at a position corresponding to a particular stage, and automatically deploy a biopsy needle at the appropriate time without manual intervention by the surgeon.

  Thus, embodiments of the inventive concept, in determining how to limit the patient's breathing level to be corrected, or how to reduce / eliminate the shift in the DRB and / or trajectory of the end effector on the robotic arm, It can be used to help the surgeon. In addition, a graphic meter can be used to indicate the deviation between the actual trajectory and the target trajectory by using an instrument in the end effector guide tube that causes a shift in the DRB. Furthermore, modeling of variable offsets more accurately locates the anatomical location relative to the DRB in situations where the positional offset between the DRB and the anatomical location (e.g., the lesion) changes over various stages of respiration. It can be used to make decisions. For example, the first offset may be used to determine the position of the anatomical location based on the position of the DRB during the first phase of respiration (e.g., contraction), and the second offset may be used to determine the second position of respiration. It can be used to determine the position of the anatomical position based on the position of the DRB during the phase (eg, dilation). Thus, this information automatically and continuously positions / repositions the end effector over the respiratory cycle to maintain the end effector on the target trajectory relative to the moving anatomical location (eg, a lesion) and / or It can be used by the surgical robotic system to automatically deploy the surgical tool from the end effector when properly aligned on the target trajectory relative to the anatomical location (e.g., a lesion).

  FIG. 22 is a block diagram illustrating elements of a robotic system controller (eg, implemented within computer 408). As shown, the controller includes an input interface circuit 2201 (also referred to as an input interface), an output interface circuit 2203 (also referred to as an output interface), a control interface circuit 2205 (also referred to as a control interface), and a memory. It may include processor circuitry 2207 (also referred to as a processor) coupled to circuitry 2209 (also referred to as memory). Memory circuit 2209 may include computer readable program code that, when executed by processor circuit 2207, causes the processor circuit to perform operations in the embodiments disclosed herein. In other embodiments, processor circuitry 2207 may be defined to include memory such that no additional memory circuitry is required.

  As described herein, the operation of the wireless terminal UE may be performed by the processor 2207, the input interface 2201, the output interface 2203, and / or the control interface 2205. For example, processor 2207 may receive user input through input interface 2201, such user input may include user input received through foot pedal 544, tablet 546, etc. Processor 2207 may also receive position sensor input received from tracking system 532 and / or camera 200 through input interface 2201. The processor 2207 may provide an output through the output interface 2203 and such out is information displaying the graphic / visual information on the display 304 and / or the audio output to be provided through the speaker 536 May be included. Processor 2207 may provide robot control information to motion control subsystem 506 through control interface 2205, which may be robot arm 104 (also referred to as Robotic arm) and / or motion of end effector 112. Can be used to control

  Operation of the robotic robot system (including a robotic arm configured to position the surgical end effector relative to the patient's anatomical position) may, in some embodiments of the inventive concept, with reference to the flowchart of FIG. Is described here. For example, the modules may be stored in memory 2209 of FIG. 22 and these modules may cause the processor 2207 to perform the operations of the flowchart of FIG. 23 when the instructions of the module are executed by the processor 2207. Can be provided.

  At block 2301, the processor 2207 receives user input (eg, input from a surgeon or other member of the surgical team) through the input interface 2201 and moves the surgical end effector to the target trajectory relative to the patient's anatomical position. It can be moved. The target trajectory may be the position and / or alignment of the end effector relative to the anatomic position used to perform the surgical procedure. Furthermore, to enable motion of the robotic arm 104 used to position the end effector 112, an input device such as a "normally off" foot pedal 544 is required to request an active input from the user. User input may be provided. In the foot pedal example, for example, the user may be asked to actively depress the foot pedal to allow motion of the robotic arm and / or end effector, and the positions of the robotic arm and end effector may It can be locked when not actively pressed.

  At block 2303, the processor 2207 may receive position information generated using a sensor system (eg, camera system 200) remote from the robotic arm 104 and remote from the patient. Location information may be received through input interface 2201. The position information may include position information associated with the tracking device (eg, a reference base or a dynamic reference base (DRB) fixed to the patient) and position information associated with the surgical end effector 112.

  At block 2305, the processor 2207 controls the robotic arm 104 (eg, via signaling transmission / reception through the control interface 2205) to generate the patient's information based on the position information generated using the sensor system. The surgical end effector 112 may be moved to a target trajectory for anatomical position. Further, the processor 2207 may control the robotic arm to move the surgical end effector to the target trajectory in response to receiving user input to move the surgical end effector as described. . Thus, as long as the user input permitting motion is maintained, the operations of blocks 2303 and 2305 may continue through block 2307 to the surgical end effector until the end effector is positioned in the target trajectory. When the user input enabling motion is stopped before reaching the target trajectory (eg, the user's foot disengages from the pedal 544), the robotic arm may be locked at blocks 2307 and 2309 before reaching the target trajectory .

  Once the surgical end effector is positioned in the target trajectory, the processor 2207 may receive user input through the input interface 2201 to lock the position of the surgical end effector at block 2307. Such input may be responsive to the user stopping the input enabling motion at block 2307 (eg, by removing foot from foot pedal 544). In response to such input, processor 2207 may control the robotic arm to lock the position of the surgical end effector at block 2309 (eg, via control to signal transmission / reception through control interface 2205).

  While the position of the surgical end effector is locked, the processor 2207 receives position information generated using a sensor system (eg, camera system 200) remote from the robotic arm 104 and remote from the patient You can keep doing. Location information may be received through input interface 2201. The position information may include position information regarding the tracking device (e.g., DRB) fixed to the patient and position information regarding the surgical end effector 112. In block 2313 where the position of the end effector is locked, processor 2207 determines the deviation between the actual trajectory for the anatomic position of the surgical end effector and the target trajectory for the anatomic position of the surgical end effector. The offset may be determined based on positioning information generated using the sensor system after locking the position of the surgical end effector.

  At block 2315, the processor 2207 may generate a user output indicative of the offset, the user output being generated in response to determining the offset. The user output may be displayed as a graphic meter on display 304 using, for example, the display configuration described above with respect to FIGS. 20A, 20B, and / or 20C. In addition, user output (eg, a graphic meter) may be used while the position of the surgical end effector is locked at blocks 2309, 2311, 2313, and 2315 (eg, additional input to move the surgical end effector is at block 2321). Can be dynamically updated to reflect changing deviations).

  In some embodiments, determining the deviation at block 2313 dynamically determines the deviation based on a model of movement of the anatomic position relative to the tracking device for multiple phases of the breathing cycle. May be included. The model is based on a first offset (ie, positional offset) of the anatomic position relative to the tracking device used to determine the target trajectory for the first phase of the respiratory cycle and a second phase of the respiratory cycle A second offset (ie, positional offset) of the anatomical position relative to the tracking device used to determine the target trajectory of the Thus, generating the user output may include dynamically generating a user output that displays the deviation based on the offsets for the multiple phases of the real-time breathing cycle. For example, the bellows belt may provide input through input interface 2201 that enables processor 2207 to determine the patient's breathing phase (eg, lung inflation or lung contraction). The use of bellows belts is described, for example, in US Pat. No. 9,782,229, the entire disclosure of which is incorporated herein by reference.

  Based on this respiratory stage information, processor 2207 may use the first offset to determine the location of the anatomical location at a first point in time during the first respiratory phase (eg, inflation of the lungs), processor 2207 The second offset may be used to determine the position of the anatomical location at a second point in time during a second breathing phase (eg, contraction of the lungs). Thus, processor 2207 may generate a first offset at a first point in time based on the target trajectory determined at block 2313 using the first offset, and processor 2207 may generate a second offset at block 2313. The second offset at the second time may be generated based on the target trajectory determined using, and the processor 2207 may generate corresponding user output corresponding to the first and second offsets in real time.

  While the end effector is locked in place, the processor 2207 may determine at block 2317 whether the deviation exceeds a threshold. In response to the deviation exceeding the threshold at block 2317, the processor 2207 may generate a user output providing notification of the excess deviation while the position of the surgical end effector is locked. Such notification may be provided through output interface 2203 as an audible output / alert using speaker 536 or as a visual output / alert using display 304. Such deviations above the threshold may occur, for example, if the patient moves or is moved on the operating table. In response to such an alert, or for other reasons, the user can use the block 2321 to provide input for moving the surgical end effector (eg, by depressing the foot pedal 544) It may be decided to reposition the end effector. In response to receiving such user input through input interface 2201 in block 2321 and in response to receiving location information through the input interface in block 2303, processor 2207 uses the sensor system Based on the generated second position information and the second position of the tracking device, the robotic arm may be controlled at block 2305 to move the surgical end effector to a target trajectory relative to the patient's anatomical position.

  In some embodiments of FIG. 23, the surgical end effector may include a guide configured to guide the placement of the surgical instrument manually inserted through the guide tube. Once the surgical end effector is locked and the user is satisfied with the placement, the user can insert a surgical instrument through the guide and perform a medical procedure. The user may, for example, use the graphic meter to determine that the end effector is properly positioned prior to performing the procedure.

  In some other embodiments, the processor 2207 may use a pattern of deviations between the actual trajectory of the surgical end effector and the target trajectory of the surgical end effector based on positioning information generated using the sensor system. Can be determined. Such a pattern of misalignment may occur, for example, by breathing the anatomical position and tracking device while the surgical end effector is locked in place. Further, the end effector may be a surgical instrument (eg, a biopsy needle) and the processor 2207 controls the end effector to surgically operate based on the offset pattern (while the end effector is locked in position). The device can be deployed automatically to make physical contact with the patient's anatomical location. In other words, the processor 2207 may select the timing of deployment to coincide with the movement of the anatomical position, placing the anatomical position to align with the surgical instrument of the locked end effector.

  Operation of the robotic robotic system (including a robotic arm configured to position the surgical end effector relative to the patient's anatomical position) may, in some embodiments of the inventive concept, refer to the flowchart of FIG. Is described here. As mentioned above, the modules may be stored in memory 2209 of FIG. 22 and these modules are instructions such that when the instructions of the module are executed by processor 2207, processor 2207 performs the operations of the flowchart of FIG. Can be provided.

  At block 2401, the processor 2207 may provide access to a model of movement of an anatomical position relative to a tracking device for multiple stages of a patient's respiratory cycle. The model may provide multiple offsets of anatomical position relative to the tracking device such that each one of the plurality of offsets is associated with each one of the multiple phases of the breathing cycle. For example, the model may provide a first offset of anatomical position relative to the tracking device, which first offset determines the target trajectory of the first phase of the breathing cycle (e.g., lungs contracted as shown in FIG. 21A). The model may include a second offset of the anatomical position relative to the tracking device, the second offset being a second phase of the breathing cycle (e.g., lungs dilated as shown in FIG. 21B). Used to determine the target trajectory. In addition, each offset is provided for any number of stages of the breathing cycle, including, for example, fully inflated, fully contracted, partially inflated / contracted, etc. It is also good.

  The model may be provided in controller memory 2209 or may be accessed from a memory and / or a database external to the controller. The model may be provided using a look-up table of respiratory stages and respective offsets, or the model may be provided as a mathematical relationship between respiratory stages and respective offsets. The model may be created by taking anatomical medical images at various stages of the respiratory cycle while using a bellows belt to detect the respiratory stage prior to treatment. The medical image may then be used to determine various offsets of each breathing stage.

  At blocks 2405, 2407, 2409, and 2411, processor 2207 receives position information, detects a respiration phase, and performs actions to control the robotic arm to maintain a target trajectory until treatment is complete at block 2411. It can.

  At block 2405, the processor 2207 may receive position information generated using the sensor system away from the robotic arm and away from the patient, and the location information may be used as the tracking device moves with the patient's breath. And information regarding the position of the tracking device fixed to the patient and the position of the surgical end effector.

  At block 2407, the processor 2207 may detect multiple stages of the breathing cycle as the tracking device moves with the patient's breath. Processor 2207 may detect a stage of breathing based on, for example, information received from a bellows belt.

  At block 2409, the processor 2209 may control the robotic arm to maintain the surgical end effector in a target trajectory relative to the patient's anatomical location as the tracking device moves with patient respiration. Controlling receiving positional information, detecting multiple steps, and using multiple offsets to determine the location of the anatomical location when the tracking device is moved by the patient's breathing , Based on.

  As an example, first position information may be received at block 2405 and a first respiration phase may be detected at block 2407. In response to detecting the first position information and the first breathing stage, the processor 2207 detects the first stage of the breathing cycle based on the first position information generated using the sensor system Controlling the robotic arm at block 2409 based on using the first offset to determine the first position of the anatomical position from the first position of the tracking device in response to the patient's anatomy The surgical end effector can be maintained in the target trajectory relative to the target position.

  If treatment is being continued at block 2411, second position information may be received at block 2405 and a second breathing phase may be detected at block 2407. In response to detecting the second position information and the second breathing phase, the processor 2207 detects the second phase of the breathing cycle based on the second position information generated using the sensor system Controlling the robotic arm at block 2409 based on using the second offset to determine the second position of the anatomical position from the second position of the tracking device in response to the patient's anatomy The surgical end effector can be maintained in the target trajectory relative to the target position.

  At blocks 2405, 2407, 2409, and 2411, the medical procedure may be completed manually or automatically while maintaining the end effector in the target trajectory. According to some embodiments, the end effector continuously and automatically maintains the guide in the target trajectory to facilitate more accurate placement of the medical device over the period required to complete the procedure. In the meantime, the user (e.g., a surgeon) may be a guide so that the medical instrument can be manually inserted through the guide. In other embodiments, the end effector may include a medical instrument (eg, a biopsy needle) that may be automatically deployed by the robotic system while the end effector is maintained in the target trajectory.

  In the above description of various embodiments of the inventive concept, the terms used herein are for the purpose of describing the specific embodiments only, and are not intended to limit the inventive concept. I want you to understand. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this inventive concept belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having meanings consistent with the meanings in the specification and the context of the relevant art, and are clearly defined in the present specification. It will also be appreciated that unless it is to be interpreted as ideal or overly formal.

  When one element is referred to as "connected", "coupled", "responsive" to another element, or as a variant thereof, It may be directly connected, coupled or responsive, or there may be intervening elements. In contrast, an element is “directly connected”, “directly coupled”, “directly responsive”, or a variation of “directly connected” to another element. When referred to as, there are no intervening elements. The same numbers refer to the same elements throughout. Furthermore, as used herein, “coupled,” “connected,” “responsive,” or a variant thereof is wirelessly coupled and connected. Or include responding. As used herein, the singular forms "a, an" and "the" are intended to include the plural as well, unless the context clearly indicates otherwise. Well-known functions or configurations may not be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

  Although the terms first, second, third, etc. may be used herein to describe various elements / actions, it is understood that these elements / actions should not be limited to these terms You see. These terms are only used to distinguish one element / action from another. Thus, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments without departing from the teachings of the inventive concept. The same reference numbers or the same reference designators indicate the same or similar elements throughout the specification.

  As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes" ), “Have”, “has”, “having” or variants thereof are open ended and one or more of the stated features, integers, elements, steps, It does not exclude the presence or addition of components, functions, but one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the generic abbreviation "eg" derived from the Latin phrase "exemplary Gracia" is intended to introduce or identify a generic example of the aforementioned item. May not be used to limit such items. Specific items can be specified from the more general recitation, using the general abbreviation "i.e." derived from the Latin phrase "id est".

  The exemplary embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or apparatus) and / or computer program products. It will be understood that the blocks of the block diagrams and / or flowchart illustrations, and combinations of blocks of the block diagrams and / or flowchart illustrations can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions are provided to processor circuitry of general purpose computer circuitry, special purpose computer circuitry, and / or other programmable data processing circuitry, for execution through a computer and / or other programmable data processing device processor These instructions translate the transistor, the value stored in the memory location, and other hardware components in the circuit to realize the function / operation specified in the block diagram and / or the block and / or block block. It is possible to create means (functions) and / or structures for implementing the functions / operations specified in the block diagram and / or the flowchart block or block, thus controlling and / or controlling.

  These computer program instructions may also be stored on a tangible computer readable medium instructing the computer or other programmable data processing device to function in a particular manner, and those instructions stored on the computer readable medium May produce an article of manufacture containing instructions that implement the functions / acts specified in the block diagrams and / or flowchart blocks or blocks. Thus, embodiments of the inventive concept may be collectively referred to as “circuits”, “modules” or variants thereof hardware and / or software (firmware, resident software, etc.) running on a processor such as a digital signal processor (Including microcode etc.).

  It should also be noted that in some alternative implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the function / operation involved. Further, the functionality of a given block of the flowchart and / or block diagram may be separated into multiple blocks, and / or the functionality of two or more blocks of the flowchart and / or block diagram is at least partially May be integrated into the Finally, other blocks may be added / inserted between the illustrated blocks and / or blocks / operations may be omitted without departing from the scope of the inventive concept. Furthermore, although some of the figures include arrows on the communication path to indicate the main direction of communication, it should be understood that communication may occur in the opposite direction to the illustrated arrows.

  While several embodiments of the inventive concept are disclosed in the foregoing specification, it is envisioned that many variations of the inventive concept and other embodiments pertain to the inventive concept, as described above. It benefits from the teachings presented in the description and the associated drawings. Thus, it is understood that the inventive concepts are not limited to the specific embodiments disclosed above, but that many modifications and other embodiments are intended to be included within the scope of the appended claims. Ru. It is further envisioned that features of one embodiment may be combined or used with features of the different embodiments described herein. Furthermore, although specific terms are employed herein and in the following claims, they are used in a generic and descriptive sense only and the inventive concepts described or the following claims. It is not used for the purpose of limiting. The entire disclosure of each patent and patent publication cited herein is hereby incorporated by reference in its entirety as if such each patent or patent publication was individually incorporated herein by reference. Incorporated into the book. Various features and / or potential advantages of the inventive concept are set out in the accompanying claims.

Claims (20)

A method of operating a surgical robotic system including a robotic arm configured to position a surgical end effector with respect to an anatomical position of a patient, comprising:
Receiving position information generated using a sensor system remote from the robot arm and from the patient, the position information being relative to a tracking device fixed to the patient Receiving, including positional information regarding the surgical end effector;
Controlling the robotic arm to move the surgical end effector to a target trajectory relative to the anatomical position of the patient based on the position information generated using the sensor system;
Controlling the robotic arm to lock the position of the surgical end effector after controlling the robotic arm to move to the target trajectory relative to the anatomical position of the patient;
While the position of the surgical end effector is locked, the actual trajectory of the surgical end effector with respect to the anatomical position and the target trajectory of the surgical end effector with respect to the anatomical position Determining an offset between them, wherein the offset is determined based on the positioning information generated using the sensor system after locking the position of the surgical end effector And
Generating a user output indicative of the deviation, wherein the user output is generated in response to determining the deviation. Receiving the user input to move the surgical end effector to the target trajectory relative to the anatomical location of the patient;
Controlling the robotic arm to move the surgical end effector to the target trajectory is responsive to receiving the user input to move the surgical end effector to the target The method of claim 1, comprising controlling the robotic arm to move to a trajectory. The method wherein the user input comprises a first user input,
Receiving a second user input to lock the position of the surgical end effector after controlling the robotic arm to move to the target trajectory;
Controlling the robotic arm to lock the position of the surgical end effector in response to receiving the second user input locking the position of the surgical end effector The method according to claim 2, further comprising: receiving, controlling the robotic arm to lock the position of an effector. Controlling the robotic arm to move the surgical end effector to the target trajectory based on first position information generated using the sensor system and a first position of the tracking device Controlling the robotic arm to move the fore end effector to the target trajectory, the method comprising
Receiving a third user input to move the surgical end effector to the target trajectory after generating the user output indicating the offset;
The surgical end effector is configured to receive the third user information based on second position information generated using the sensor system and a second position of the tracking device in response to receiving the third user input. 4. The method of claim 3, further comprising: controlling the robotic arm to move to the target trajectory relative to an anatomical position.   The method of claim 1, wherein generating a user output comprises displaying the user output for presentation on a display.   6. The method of claim 5, wherein the user output is displayed as a graphic meter on the display.   The method according to claim 6, wherein the graphic meter is dynamically updated while the position of the surgical end effector is locked.   The method of claim 1, further comprising generating a user output providing a notification of the excess deviation in response to the deviation exceeding a threshold while the position of the surgical end effector is locked. .   The method according to claim 1, wherein the surgical end effector comprises the guide configured to guide the placement of a surgical instrument manually inserted through a guide tube. The determining between the deviation may be based on the positioning information generated using the sensor system, between the actual trajectory of the surgical end effector and the target trajectory of the surgical end effector. Determining the pattern of misalignment, the end effector comprising a surgical instrument, the method comprising
The method according to claim 1, further comprising: automatically deploying the surgical instrument to make physical contact with the patient's anatomical location based on the pattern of misalignment.   Determining the offset is used such that a first offset of the anatomical position relative to the tracking device determines the target trajectory for a first phase of a respiratory cycle, the anatomical position relative to the tracking device Based on a model of movement of said anatomical position relative to said tracking device for multiple phases of the respiratory cycle, such that a second offset of is used to determine said target trajectory for the second phase of the respiratory cycle The method of claim 1, further comprising: dynamically determining the offset, wherein generating the user output comprises dynamically generating the user output that displays the offset based on the model. The method described in. A robotic arm configured to position the surgical end effector relative to the patient's anatomical position;
A controller coupled to the robot arm, wherein the controller is configured to:
Position information generated using a sensor system remote from the robotic arm and remote from the patient, wherein the position information is relative to a tracking device fixed to the patient and the surgical end effector Containing location information about
Controlling the robotic arm to move the surgical end effector to a target trajectory relative to the anatomical position of the patient based on the position information generated using the sensor system;
Controlling the robotic arm to move it to the target trajectory relative to the anatomical position of the patient, and then control the robotic arm to lock the position of the surgical end effector;
While the position of the surgical end effector is locked, between the actual trajectory of the surgical end effector with respect to the anatomical position and the target trajectory of the surgical end effector with respect to the anatomical position And determining the position of the surgical end effector, and then determining the position based on the positioning information generated using the sensor system;
A surgical robotic system configured to generate a user output indicative of the offset, and wherein the user output is generated in response to determining the offset. The controller
Receiving user input to move the surgical end effector to the target trajectory relative to the anatomical position of the patient;
Controlling the robotic arm to move the surgical end effector to the target trajectory is responsive to receiving the user input to move the surgical end effector to the target The surgical robotic system according to claim 12, further comprising: controlling the robotic arm to move to a trajectory. The user input comprises a first user input and the controller
Further configured to receive a second user input that locks the position of the surgical end effector after controlling the robotic arm to move to the target trajectory;
Controlling the robotic arm to lock the position of the surgical end effector in response to receiving the second user input locking the position of the surgical end effector The surgical robotic system according to claim 13, comprising controlling the robotic arm to lock the position of an effector. Controlling the robotic arm to move the surgical end effector to the target trajectory based on first position information generated using the sensor system and a first position of the tracking device. Controlling the robotic arm to move the fore end effector to the target trajectory, the controller including:
Receiving a third user input to move the surgical end effector to the target trajectory after generating the user output indicating the offset;
The surgical end effector is configured to receive the third user information based on second position information generated using the sensor system and the second position of the tracking device in response to receiving the third user input. The surgical robotic system according to claim 14, further configured to control the robotic arm to move the target trajectory relative to an anatomical position. The determining between the deviation may be based on the positioning information generated using the sensor system, between the actual trajectory of the surgical end effector and the target trajectory of the surgical end effector. Determining the pattern of misalignment, wherein the end effector comprises the surgical instrument and the controller is configured to:
13. The surgical device of claim 12, further configured to automatically deploy the surgical instrument to make physical contact with the patient's anatomical location based on the pattern of misalignment. Robot system. A method of operating a surgical robotic system including a robotic arm configured to position a surgical end effector with respect to an anatomical position of a patient, comprising:
Providing access to a model of movement of said anatomical location to a tracking device for multiple phases of a respiratory cycle, wherein each of said plurality of offsets comprises a plurality of said plurality of respiratory cycles. Providing the plurality of offsets of the anatomical position relative to the tracking device to be associated with each one of the stages of
Receiving position information generated using a sensor system away from the robot arm and away from the patient, wherein the position information is when the tracking device is moved by the patient's breathing Receiving, including information regarding the position of the tracking device fixed to the patient and the position of the surgical end effector;
Detecting the plurality of stages of the breathing cycle as the tracking device moves with the patient's breath;
Controlling the robotic arm to maintain the surgical end effector in a target trajectory relative to the patient's anatomical location as the tracking device moves by the patient's breathing Receiving the position information, detecting the plurality of steps, and using the plurality of offsets to determine the position of the anatomical position as the tracking device moves with the patient's breath. And controlling based on. The model is used to determine the target trajectory for a first phase of a respiratory cycle, a first offset of the anatomical position relative to the tracking device, and a second phase of the respiratory cycle Providing a second offset of the anatomical position with respect to the tracking device used to determine the target trajectory, and controlling the robotic arm;
A first position of the anatomic position from a first position of the tracking device based on first position information generated using a sensor system and in response to detecting the first phase of the breathing cycle Controlling the robotic arm to maintain the surgical end effector in the target trajectory relative to the anatomic position of the patient based on using the first offset to determine a position;
A second position of the anatomic position from a second position of the tracking device based on second position information generated using a sensor system and in response to detecting the second phase of the respiratory cycle Controlling the robotic arm to maintain the surgical end effector in the target trajectory relative to the anatomic position of the patient based on using the second offset to determine two positions; 21. The method of claim 17 comprising. A robotic arm configured to position the surgical end effector relative to the patient's anatomical position;
A controller coupled to the robot arm, wherein the controller is configured to:
Providing access to a model of the movement of said anatomical position to a tracking device for multiple phases of a respiratory cycle, wherein said model comprises one each of a plurality of offsets, each of said plurality of phases of said respiratory cycle Providing the plurality of offsets of the anatomical position relative to the tracking device to be associated with one;
Receiving location information generated using a sensor system away from the robot arm and away from the patient, the location information being transferred to the patient as the tracking device moves by the patient's breathing Information on the position of the fixed tracking device and the position of the surgical end effector;
Detecting the plurality of stages of the breathing cycle as the tracking device moves with the patient's breath;
Controlling the robotic arm to maintain the surgical end effector in a target trajectory relative to the anatomical location of the patient as the tracking device moves by breathing the patient, the controlling being performed Receiving the plurality of stages, and using the plurality of offsets to determine the location of the anatomical location as the tracking device moves with the patient's breath. A surgical robotic system configured to be based on: The model is used to determine the target trajectory for a first phase of a respiratory cycle, a first offset of the anatomical position relative to the tracking device, and a second phase of the respiratory cycle Providing a second offset of the anatomical position with respect to the tracking device, used to determine the target trajectory, and controlling the robotic arm is generated using a sensor system The first offset is determined to determine a first position of the anatomical position from a first position of the tracking device based on position information, and in response to detecting the first phase of the respiratory cycle. Controlling the robotic arm to maintain the surgical end effector in the target trajectory relative to the anatomical position of the patient based on use; A second position of the anatomic position from a second position of the tracking device based on second position information generated using a sensor system and in response to detecting the second phase of the respiratory cycle Controlling the robotic arm to maintain the surgical end effector in the target trajectory relative to the patient's anatomical position based on using the second offset to determine a two position; 20. The surgical robotic system of claim 19, comprising: