JP2010525838A - Method, apparatus, and system for non-mechanically limiting and / or programming movement along one axis of a manipulator tool - Google Patents

Abstract

  Rather than mechanically restricting the movement of a medical robot to generate a single axis movement, the movement of the medical robot tool along one axis is achieved by electronically limiting the medical robot movement. A method, an apparatus (such as a computer readable medium), and a system (such as a computer system) for performing. Such movement of the tool is along one axis even when the user moves the input device connected to the medical robot to another axis during the one-axis movement. Further disclosed is a technique for automating uniaxial movement, whereby it can be programmed to stop at a target location and start at or near a second (eg, starting) location This is useful for procedures such as brain biopsy, chest biopsy, or transplantation, and does not require the user to manipulate the input device to generate a real-time response movement of the medical robot. A user can execute a command to instruct the medical robot to execute.

Description

CROSS-REFERENCE This application of the relationship application, by reference are incorporated herein, which claims the priority of US Provisional Patent Application Serial No. 60/912146, filed April 16, 2007. The co-pending international application PCT / US08 / 60541 is also incorporated herein by reference.

  The method, apparatus, and system of the present invention relate generally to the field of surgical robots, and more specifically, manipulators (eg, robot arms with multiple degrees of freedom) can be moved by a manipulator along one axis. Relating to non-mechanical restrictions.

One example of a procedure that can be performed by the methods, devices, and systems of the present invention is an automated biopsy. An example of a surgical robot that can be used for procedures involving the methods, devices and systems of the present invention is disclosed in US Pat. No. 7,155,316 (the '316 patent), which is incorporated herein.
In order to perform stereotactic procedures (eg, take a needle or small tool and hit a target in 3D space), it is limited to the extent that the tool can deviate from its planned trajectory. It is advantageous. Therefore, in order to perform a stereotaxic procedure using a master / slave interface using a robot, all inputs to the master controller in the X and Y coordinates are disabled and tool tip movement is restricted to the Z axis. It may be desirable to do so.

  Current procedures that use frame-based or frameless stereotactic tools generate a Z-lock condition due to mechanical limitations. The most common method for stereotaxic procedures (frame-based) involves securing a rigid head frame to the patient's head. This frame serves as a mechanical means to guide the localization tool through the planned path by mechanically limiting X and Y axis movement. Other frameless stereotactic tools that use mechanical arms or tool attachments fix the patient's head in space, place the mechanical arm in a pre-planned path position, and machine the degree of freedom associated with the arm. The stereotaxic procedure is performed by locking automatically. The result is a mechanical Z-lock along a pre-planned path.

  In both frame-based and frameless stereotactic procedures, a pre-planned path is derived from images taken hours before the procedure. However, the brain is not fixed inside the cranial cavity and can be mislocated as a result of injury, tumor, hydration, and postural changes. These relatively small brain misalignments can be a problem in accuracy and may raise safety concerns. As a result, post-surgical images and other tools are used to ensure an accurate and safe procedure using existing tools. Furthermore, frame-based stereotactic procedures also require that a head frame be attached to the patient's head, which is uncomfortable and time consuming.

  Planning before surgery and imaging after surgery take a lot of time. In addition, a frameless stereotaxy navigation system needs to be able to line of sight between the patient's head and the surgeon's tools. This can be a problem for a surgeon who needs to be positioned by the patient's head to guide the stereotactic tool to the target.

Aspects of the method and system of the present invention allow a user, such as a surgeon, to perform a robot (including a tool coupled to a robot arm and a tool integrated with the robot arm) along one axis, such as the longitudinal axis of the tool. Allows automatic movement of one tool in the arm to be set and executed. Such movement is particularly advantageous when performed as an automated biopsy of tissue such as brain tissue or breast tissue. Automatic movement is provided during a stereotaxic procedure or when one or both robot arms are prepared when some or all of the robot arms are located within the open or closed magnet lumen of the magnetic resonance imaging machine. And may be programmed to take place during a microsurgical procedure during which such automatic movement is performed. A robot that can be manipulated by the techniques of the present invention can also be characterized as a computer aided device.

  In some aspects, the system of the present invention takes the form of a computer system that is useful in simulating, planning, and / or performing automated surgical procedures. The computer system receives at least the following functions: data specifying a target location for a tool held by a medical robot; during automatic movement, the tool moves from it toward the target location In response to a user command to initiate automatic movement, so that the tool moves along an axis defined by the second location and the target location, The medical robot is moved. The data specifying the target location can be the coordinates of the tool tip (eg Cartesian coordinates), or the coordinates of the location spaced from the tool along the tool's vertical axis in any coordinate system, or Data sufficient to enable the determination of such coordinates (based on known parameters such as the robot arm link length, the robot arm's ability to find a coordinate system solution using forward kinematics) Joint values, etc.).

  In some aspects, the apparatus of the present invention takes the form of a computer readable medium that includes machine readable instructions for performing the following: held in a robotic arm configured to be used in surgery, Or receiving a command to limit the movement of the instrument integrated therewith along one axis; receiving the position and orientation of the input device, provided that the input device is a master / slave relationship in which the input device is a master And the difference between the position and orientation of the input device and the previous position and orientation of the input device is consistent with the desired movement of the device; and Send a signal (s) to cause movement of the device in response, provided that this movement is along one axis and along any axis different from the one axis. Movement does not contain a. The signal (s) can be any suitable form of data that contains sufficient information to properly move the robot arm. For example, the signal (s) may represent a set of joint displacements and / or joint velocities that are output to a local controller for the robot arm or directly to individual joint actuators.

  In some aspects, a user may prepare a procedure by providing input to a computer system via an input device, such as a hand controller coupled to a robotic arm as a master in a master-slave relationship. The user may also use one or two suitable input devices such as touch screen controls (eg, buttons, slider bar, drop-down menus, tabs, etc.), mouse, or others. Input may be given via the graphic user interface (GUI). Some aspects of the system of the present invention are computer systems that allow a user to select a simulation mode (eg, by touching) for setting up automatic movement or for training. A GUI that allows selection (via a control such as a button, mouse, etc.) may be configured to be displayed on the display screen. The computer system can also be used to select the type of surgery (eg, via contact, mouse, etc.), such as microsurgery, stereotactic surgery with one robot arm, stereotactic surgery with the other robot arm. One or more controllers (which can be selected) may be configured to be displayed on the GUI.

  The computer system also displays one or more controls (eg, selectable via touch, mouse, etc.) on the GUI to activate power to: May be configured: a robot arm (eg, via a separate button); a base motor for adjusting the height of the base on which the robot arm is placed during a microsurgical procedure A digitizing arm useful during a physical positioning process to position a structure (eg, a radio frequency coil assembly) in a fixed relationship with a portion of a subject relative to one or both robotic arms; arm); a field camera useful for imaging the surgical field during microsurgery; and a magnet bore in the magnetic resonance imaging system A Boakamera (s) to be. The computer system also includes one or more controllers (e.g., selectable via contact, mouse, or others) to activate a single axis lock (e.g., Z axis lock), and any Another button or buttons for controlling whether the robot arm is associated with a single axis lock may be configured to be displayed on the GUI.

  The computer system also displays two-dimensional images of a portion of the subject (one at a time) on one or more additional display screens and a three-dimensional representation of the portion of the subject (eg, One or more additional GUIs for displaying (a set of 2D images forming a 3D data set of images representing the volume) may be configured to be displayed. If only one such GUI is provided on one additional display screen, the computer system will select whether the user will display either a 2D image or a 3D image (one at a time). One or more controls (eg, buttons, tabs, etc. that can be selected via touch, mouse, or others) may be configured to be displayed.

  The computer system may also be configured to display zoom buttons, slider bars, or others (eg, which can be selected / operated via contact, mouse, or others), thereby enabling 2D display. The selected user can zoom in on a given 2D image, in which case the 2D image remains centered when its dimensions are enlarged or reduced. The computer system also provides a cross that represents where the user either (a) the working tip (eg, the distal end) of the tool of the selected robot arm, or (b) the end of a line extending from the tool tip. Display a controller (e.g., selectable via contact, mouse, or others) that can activate the tracking function for one of the two robot arms, displayed as crosshairs It may also be configured as a selection via a controller (eg, contact, mouse, or others) that allows the user to activate the extension line display and control the length of the extension line Button, slider bar, or other) that can be configured.

  The user operates an input device connected to the selected robot arm, and the user's movement is held in or integrated with the robot arm (in a simulation mode in which the robot arm does not actually move). Changing the position of the tool will result in the crosshairs moving so that the displayed 2D image (in 2D display mode) matches the apparent depth of the tool (or extension line) to the subject To change. Alternatively, a given 2D image may include an oblique slice oriented perpendicular to the tool axis. Such a slice interpolates pixels between 2D slices to obtain an off-axis image. Similarly, as the 2D slices that make up the 3D image are removed or added depending on the depth of the tool / extension line inside the subject, the 3D image also changes in response.

  The computer system may also be configured to display the following when either 2D or 3D display is selected: ie, automatic movement along one axis (eg, automatic biopsy) Can be used to set the target location (corresponding to the location of the crosshair when the target location button is selected), for example, selecting via contact, mouse, or others Section that corresponds to the plan for automated biopsy, including controller display; a second point that defines the path of the tool movement along with the target location (arbitrary movement may not start exactly from the starting point) Can be characterized as a starting point, and this second point corresponds to the location of the crosshair when the second button is selected); the robot arm to which the user is concerned , Can be used if you want to place it in a fixed position for automated movement procedures (eg within a preset distance in the range from zero to some relatively small distance (eg 2 cm)) A tool alignment function that moves the arm so that the tool tip is at or near the starting point; and the user holds the input device (eg, with the user's finger to hold the button on the input device) Execution function that the user can press to initiate automatic movement if enabled (by pressing).

  The computer system also includes an indicator (eg, a colored circle) for the target location selected by the user; and a line extending from this indicator to the tool tip or extension line tip (whichever is used), The computer system may be configured to display a line that is designed to show the user a route through the subject if the line is followed, and the computer system is configured to display a line when the second point is selected. Is configured to change the appearance (eg, change the color or shape of the line).

  Thus, in some aspects, the computer system may be configured to perform at least the following functions: a command to identify a target location for a tool used in automatic movement by a robotic arm ( Receive (for example, by touching the screen displaying the relevant GUI); receive a command identifying the starting location for the tool; on a path (eg, a line) defined at least in part by the starting location and the target location A function that receives commands to perform automatic movement along; and performs automatic movement such that a tool having a vertical axis travels along a path (eg, along one axis). This path may also be aligned with the longitudinal axis of the tool.

  In some aspects, the computer system may also be configured to receive a command that selects which robotic arm to use for automatic movement (eg, prior to a command to identify a target location). In some aspects, the computer system may be configured to receive a command indicating a simulation and / or setup mode (e.g., prior to a command identifying the target location), the mode holding the tool Or disconnecting the input device connected to the robot arm integrated therewith in a master / slave relationship so that the movement of the input device does not cause the movement of the robot arm in the simulation mode.

  In some aspects, the computer system also indicates activation of the input device by the user (e.g., prior to a command identifying the target location) (such as by a user touching a button on the input device with the user's hand). The activation may be configured to receive a command, and when the user determines where to place a tracking indicator for selecting a target location and a starting location, the user indicates the location of the apparent tool tip. Allows to change the position of the tracking index. In some aspects, the computer system may also be configured to receive a command indicating a new (eg, second) target location (eg, after a command identifying the starting location).

  In some aspects, the computer system may also be configured to receive a command indicating a new (eg, second) start location (eg, after a command identifying the start location). In some aspects, the computer system may also be configured to receive a command instructing the end of the simulation and / or setup mode (eg, after a command identifying the start location). In some aspects, the computer system can be used to select a mode in which the input device works in a master-slave relationship with the robot arm (eg, selecting via contact, mouse, or others). The controller may be displayed on the GUI.

  In some aspects, when the computer system is in master-slave mode, the command to activate the input device (eg, by holding the input device and pressing a button on the input device with a user's finger) May be configured to receive. In some aspects, the computer system may be configured to receive an instruction to perform an automatic movement along a path defined at least in part by the start location and the target location, the computer system also ( Only after receiving a command indicating that the input device has been activated (so that the user must hold the input device in order for the automatic movement to proceed), the robotic arm is within one axis along the path. It may be configured to move the tool.

  When the computer system receives a command to stop automatic movement (for example, by holding the input device with a user's finger and pressing a button on the input device), the robot arm completes the automatic movement. And a button (e.g., selectable via touch, mouse, or others) to continue automatic movement, reverse direction, or stop Messages including others may be configured to be displayed on the GUI, and the input device is activated (eg, by holding the input device with a user's finger and pressing a button on the input device). If you first receive a command indicating that it is active, you will receive a command to continue, reverse or stop, depending on which button is activated or otherwise Sea urchin may be configured.

  Any aspect of any of the methods, apparatus, and systems of the present invention comprises, rather than comprises, includes / includes / contains / haves the steps and / or features described above, or It may be consisted essentially of them. That is, in any claim, the term “consist of” or “consist essentially of” is used in order to change the scope of a given claim from that which would otherwise be done using an open-ended linking verb. The phrase may be substituted by any of the above open-ended linking verbs.

  The following drawings are presented by way of example and not limitation. The same reference numerals do not necessarily indicate the same structure, system, or indication. Rather, the same reference numerals may be used to indicate similar features or features with similar functions. For clarity of illustration, not all features of that aspect are always shown in every figure in which each aspect appears. The controller devices, manipulators and tools shown in the figures are drawn to scale, meaning that the dimensions of the described elements are relatively accurate relative to each other.

FIG. 1A is a perspective view of one embodiment of two input devices (hand controllers) that can be used consistently with the techniques of the present invention. FIG. 1B is an enlarged view of a left-handed input device. FIG. 1C is a diagram of different fields of view showing the tool held by the robot arm in the first position of the stereotaxic procedure. FIG. 1D is a view of different fields of view showing the tool held by the robot arm in the first position of the stereotaxic procedure. 1E is a diagram of different fields of view showing the tool held by the robot arm in the first position of the stereotaxic procedure. FIG. 2A is a diagram of different fields of view showing the tool of FIGS. 1C-1E in a second position of the stereotaxic procedure. FIG. 2B is a diagram of different fields of view showing the tools of FIGS. 1C-1E in a second position of the stereotaxic procedure. FIG. 2C is a diagram of different fields of view showing the tool of FIGS. 1C-1E in a second position of the stereotaxic procedure. FIG. 3A is a diagram of different fields of view showing the tool of FIGS. 1C-1E in a third position of the stereotaxic procedure. FIG. 3B is a diagram of different fields of view showing the tool of FIGS. 1C-1E in a third position of the stereotaxic procedure. FIG. 3C is a diagram of different fields of view showing the tool of FIGS. 1C-1E in a third position of the stereotaxic procedure. FIG. 4 is a perspective view of a workstation for use in planning and controlling the tool movement shown in FIGS. 1C-3C. FIG. 5 is a graphical user interface that can be used to prepare and control the tool movement shown in FIGS. 1C-3C. FIG. 6 is a diagram illustrating the GUI of FIG. 5 in training / simulation mode including both robot arms.

FIG. 7 shows the GUI of FIG. 5 in training / simulation mode with the left arm selected for stereotactic surgery. FIG. 8 is a diagram showing the GUI of FIG. 5 in the training / simulation mode in which the right arm is selected for stereotactic surgery and the Z-axis lock function for the tool of that arm is activated. FIG. 9 is a diagram illustrating the GUI of FIG. 5 in training / simulation mode with microsurgical application selected, both arms enabled, and both tools selected. FIG. 10 is a view in training / simulation mode in which the left arm is selected for stereotaxic surgery (as in FIG. 7) and the Z Axis Lock function for the tool in that arm is activated. It is a figure which shows GUI of. FIG. 11 shows another GUI that can be used to prepare for the tool movement shown in 1C-3C. An indicator in the form of a crosshair corresponding to the location of the tool tip is shown on the 2D image and represents the location of the tool tip for the portion of the subject in the 2D image. FIG. 12 shows the GUI of FIG. 11 in 3D mode with a representation of the tool selected for use in training / simulation displayed at the relevant depth inside the 3D image of the portion of the subject. This GUI reflects that the user has selected the “Plane Cut” option, resulting in the oblique slice being cut to the relevant tool tip depth on the head. FIG. 13 is an illustration of an alarm box that may appear on a GUI similar to that in FIG. 5 if a problem occurs during an automated procedure.

  The term "comprise" (and all forms of riserise such as "comprises" and "comprising"), "have" (and all forms of have such as "has" and "having"), "include" (and " All forms of include such as "includes" and "including") and "contain" (and all forms of contain such as "contains" and "containing") are open-ended connective verbs. As a result, a method, apparatus, or system comprising one or more described steps or elements (“comprises”, “has”, “contains”, or “includes”) However, it is not limited to having only those steps or elements, and has elements or steps not described (ie, included in the scope). Similarly, a method, apparatus, or system that includes one or more of the described features (“comprises”, “has”, “contains”, or “includes”) includes those features, but only It is not limited to having, and you may have the characteristic which is not described.

  Similarly, a computer-readable medium “comprising” (or “coded”) with machine-readable instructions for performing certain steps is a computer-readable medium having machine-readable instructions for implementing at least the described steps. However, the scope also includes media having machine-readable instructions for implementing additional undescribed steps. Further, a computer system configured to perform at least some function is not limited to performing only the described function, as long as the system is configured to perform the described function, It may be formed by an unspecified method (s).

The terms “a” and “an” are defined herein as one or more, unless expressly stated otherwise. The term “another” is defined as at least a second or more. The term “substantially” is at least close to (including) a given value or condition, more preferably within its 1% range, most preferably within its 0.1% range. Defined.
In some aspects, the present invention is a software enabled uniaxial lock for tool movement along a single axis by a robot arm having multiple degrees of freedom. This software solution allows a robot arm with an unlimited number of degrees of freedom to operate in the same way as a robot or device that is mechanically restricted to movement along one axis of the tool. Prior to treatment, a command is sent to the software and given using any suitable input device, such as a button on a touch screen on the GUI or a button on an input device (eg, a hand controller). The tool (means a tool held by the robot arm or a tool integrated with the robot arm; the tool of the present invention is more particularly characterized as a medical tool or a surgical tool. Can be locked to one axis.

  Devices to which the techniques of the present invention can be applied include, in some aspects, slave robots that receive commands from a master input device such as a hand controller. An example of a pair of input devices that can be used to control two different robot arms of a medical or surgical robot system, respectively, is shown in FIG. 1A. When the input devices 20 that are mirror images of each other are held in the direction of the stylus 25 that can be held like a long pen, the tool is integrated with or held by the slave robot arm. Lever 27 that can be actuated (eg, grasping lever 27 can cause forceps to close) and activation that can be touched or held for a short period of time to activate the input device An enable / disable button 29 is included. One way to hold the input device 20 is to hold the stylus 25 so that the lever 27 can be gripped with the index finger and the button 29 can be touched with the thumb. FIG. 1B shows an enlarged view of the left-handed input device 20.

  Generating open and closed form forward and reverse kinematics solutions, one skilled in the art can use the joint values that characterize the position of each joint in the robot (taking into account the allowed movement axes). ) Joint angle that must be achieved to find the solution of the commanded tool tip position, then capture the commanded tool tip position and move the surgical tool to the commanded tool tip position along one axis You may be made to ask for the solution of.

One method of generating a non-mechanical single-axis tool movement lock (eg, after receiving a command to generate it) requires the following:
a) An input device (eg, hand controller) command is acquired in a tool tip space (eg, orthogonal coordinates X, Y, Z, roll, pitch, yaw). This acquisition involves receiving hand controller signal (s) (command (s), or data) that represents the position and orientation of the hand controller; uniaxial (eg, tool tip movement is limited thereto) Determining (eg, calculating) the delta value of the movement of the hand controller (in the axis related by the conversion to one axis); and using the hand controller delta to determine the corresponding delta value at the tool tip including. Once conversion from delta values in hand controller space to delta values in tool tip space is sought, all tool tip delta values may be ignored, except for deltas along one axis of interest. This can also be achieved efficiently by simple zeroing of non-uniaxial parameters obtained from the hand controller or by calculating all delta values for each axis and using only the deltas in the uniaxial direction.

b) Take the current position of the manipulator (this is a term that can describe the robot arm) and execute the forward kinematics solution to obtain the tool tip X, Y, Z, roll, pitch, yaw .
c) adding the uniaxial delta determined in step a) to the current manipulator tip position determined in step b).
d) Use this new tip position to perform inverse kinematics to find the required joint angle solution.
e) Command the manipulator to a new joint value.
f) Repeat from step a).

In some aspects, the speed of execution of the above loop can be arbitrarily reduced to produce a linear motion at the tool tip. The longer it takes or the larger steps it takes, the more non-linearity can be generated when the movement between each Cartesian coordinate position is in the joint space, and the joint will move as desired Interpreted linearly over range. Smaller movements on the order of 10 milliseconds result in an undetectable non-linearity between each Cartesian coordinate tip command and valid linear motion.
Further, as discussed in more detail below, in some aspects, the movement of the tool along one axis may be preprogrammed to be automated.

  Referring now to FIGS. 1C-3C, detailed views of the manipulator 100 and surgical tool 150 show that when the manipulator 100 causes movement of the tool along one axis in a stereotactic procedure (such movement is a microsurgical procedure. It can be achieved in any other procedure) and is shown in various positions. A manipulator 100 which is an example of a multi-degree-of-freedom robot arm (specifically, the manipulator 100 can be characterized as a six-degree-of-freedom slave manipulator, and is disclosed in US Pat. No. 7,155,316 (patent 316)). And an assembly 200 comprising a radio frequency coil device (coupled to the head clamp and further coupled to the operating room table by a flexible articulated arm), and a camera 190 (similar in operation) Only one of them is visible and the other is on the opposite side of the extension board). The extension board 260 may be coupled to any suitable table or other structure having a patient support surface (not shown). In the illustrated view, a schematic view of a patient head 300 held in a head clamp of assembly 200 is shown.

  1C and 1D show the manipulator 100 in a first position, in which the surgical tool 150 is located outside the head 300 and near the opening 350 in the head 300. Can be a burr hole or any other suitable surgical opening. However, the tip 160 of the tool 150 is outside the opening 350. FIG. 1E is a side view of the position shown in FIGS. 1C and 1D and does not include the assembly 200 for clarity. 2A and 2B show the manipulator 100 moved to a second position in which the distal end 160 of the surgical tool 150 is moved along the axis 110 by the manipulator 100, but on the other axis. As a result, the tip 160 penetrates the boundary of the opening 350. FIG. 2C is a side view of the position shown in FIGS. 2A and 2B and does not include the assembly 200 for clarity.

  3A and 3B show the manipulator 100 moved to a third position, in which the tip 160 has moved along the axis 110 into a location inside the head 300, the tip of which is Any number of functions can be performed, such as being further manipulated by a user / operator (eg, a surgeon) to perform a tissue biopsy. FIG. 3C is a side view of the position shown in FIGS. 3A and 3B and does not include the assembly 200 for clarity. The axis 110 is substantially aligned with the longitudinal axis of the tool 150 (not separately shown) (fully aligned in the illustrated embodiment). For other tools with bends and angles, the tool and tool tip still move along one axis, but the axis may not coincide with the tool's own longitudinal axis. is there.

  FIG. 4 controls a manipulator 100 (or two such manipulators) and a surgical tool 150 (or two such surgical tools respectively held by two such manipulators). FIG. 3 shows a perspective view of a workstation 400 that can be used for In some aspects, the workstation 400 includes the input device 20 shown in FIGS. 1A and 1B that controls the movement of the manipulator 100. The workstation 400 includes a series of display screens, including display screens 401 and 402, in addition to a table to which the input device is secured, each of which is used in preparation for a procedure using the manipulator 100. A graphical user interface (GUI) can be provided.

  In a preferred embodiment, the GUI shown on the display screen 410 may be used to select two points, which in an automated uniaxial movement, move the associated tool tip along it. An axis (or path or trajectory) will be defined (such a screen is referred to herein as a command status display (CSD)), and the GUI shown on the display screen 402 is May be used to display one or more images from a 3D image data set of a subject taken using a 3D imaging device, such as a magnetic resonance imaging device, It may be assumed that the decision on which point to select on screen 402 has been made by the operator (such a screen Is referred to herein as a magnetic resonance imaging display (MRID)). The other display screen shown in FIG. 4 may be used to show other images or displays associated with any procedure using one or both manipulators 100.

  FIGS. 5-11 illustrate various screen displays that can be used as a GUI for displays 401 and 402. (Buttons, slider bars, radio buttons, checkboxes, drop-down menus for touching the screen, operating an input device such as a mouse, or receiving user input by any other suitable means, as shown in the figure A plurality of controllers (such as, etc.) are provided on each screen. Only the controllers related to the features presented in this disclosure are considered.

  In some aspects, the computer system may be configured such that upon starting a primary application supported by the computer system, the user is presented with a start screen as illustrated in FIG. One of ordinary skill in the art having the benefit of this disclosure will be able to achieve the features (including graphical user interface) described below and illustrated without undue experimentation to achieve the features (software, hardware, or Machine readable instructions) that can be implemented in any combination of two or more of these. FIG. 5 illustrates a desired mode for one or both manipulators (such as “Z Axis Lock”, which represents the ability of the manipulator to move the tool along only one axis), or a given manipulator 100. FIG. 6 illustrates a CSD 401 that can be used to prepare a desired procedure, such as automatic movement (eg, along one axis) of a surgical tool according to FIG.

  In addition to the treatment type (microsurgery, stereotactic surgery left arm, stereotaxic surgery right arm), this display shows the height of the right arm, left arm, and arm (manipulator 100) supported on a movable and lockable base. Includes a base motor for adjustment, a field camera for imaging during microsurgery, and a power selection option for the digitizing arm used for the physical part of subject image versus manipulator positioning. The power button is shown in the “Mode Controls” tab as well as “Surgery Type”. The manipulator 100 is shown without highlights on the GUI shown in FIG. 5, and any manipulator can be used for either training / simulation or treatment using the buttons at the bottom left of the screen. It also means that it is not selected.

  A suitable technique for positioning one or more two-dimensional images of a portion of a subject with one or both manipulators 100 is disclosed in co-pending international application PCT / US08 / 60538, incorporated by reference. Has been. Once suitable positioning has been achieved, including both an MRI positioning aspect that positions the imaged subject in a physical structure and a physical positioning aspect that positions a given manipulator in that physical structure, preparation can begin.

  In one exemplary aspect, the user can simulate by selecting the “Training Simulation Mode” button under the “User Settings” tab shown in CSD 401 of FIG. A mode can be selected. By selecting the simulation mode, the user can view the simulated movement of manipulator 100 and surgical tool 150 in response to the movement of the input device without causing actual movement of manipulator 100. . The word “simulation” also appears near the bottom of the display, as shown for example in FIGS. In the simulation mode, the user can view the possible movement paths of the surgical tool 150 that can be used in the surgical procedure. This allows the user to evaluate multiple possible paths for the manipulator 100 and surgical tool 150 before defining the paths that are actually used in the procedure as follows.

  CSD 401 in FIG. 7 describes the system in a simulated stereotactic mode with the left arm enabled. Here, this version of the CSD 410 shows only one manipulator as a result of selecting the left arm and is shown in a highlighted state. Also shown is the top of the RF coil device (from assembly 200) placed over a graphical representation of the subject's head (eg, head 300). It is also possible that the user has a left arm and a camera such as “Bore Camera” (or camera 190 shown in FIGS. 1C-3C, which affects the magnetic field generated in the MRI environment. Shows that the power supply for the digitizing arm and the power supply to the digitizing arm has been activated (not selected "Right Arm" button and "Base Motor" button) Is not selected and is grayed out).

  FIG. 8 shows a version of CSD 401 showing that the user has put the right arm into stereotactic mode and Z-axis lock mode, in which case the tool selected for use by the right manipulator is , Shown in the lower right part of the screen (this is the same biopsy tool shown in FIGS. 1C-3C). The mode of the displayed manipulator shown in FIG. 8 is the stereotactic right arm (as shown in the button in FIG. 5), the master / slave button shown in FIG. Master / slave mode by selection, and right arm by selecting “Right Arm” in the “Arm Enable” box of the “Mode Controls” tab shown in FIG. Activation was achieved by the user's selection. Next, the user activates the input device associated with the right manipulator by pressing button 29 on right hand controller 20. Once the input device is activated, the user has not put the system into training / simulation mode, so the user operates the activated input device to allow the user to move the tool along one axis. Manipulators can be deployed at the desired position and orientation.

  When the manipulator is in place, the user can override the manipulator control by pressing button 29 again, otherwise the user is shown in FIG. 8 (in the illustrated embodiment). The Z axis for the tool held by that manipulator by selecting “Right Tool” in the “Z Axis Lock” box of the “Mode Control” tab You can proceed to enable the lock. In this mode, the tool held by the manipulator is centered on the axis defined by the top of the tool held there by the tool holder portion coupled to the end effector of the manipulator (in this embodiment, it is centered over the entire length of the tool). Such movement is performed in the forward or backward direction depending on the movement of the user's input device. When the user no longer wants to lock the tool movement to such an axis, the user can press the same “Right Tool” button to disable the mode.

FIG. 9 shows a version of CSD 401 showing that the user has selected the microsurgical mode and the simulation mode, enabled both manipulators (both are highlighted), and selected the tools for them. It is. From this version of CSD 401, the user can activate the Z axis lock function for the tools of both arms. The tool selected for each manipulator is shown next to the manipulator (bipolar forceps on the left and biopsy tools on the right).
FIG. 10 shows one version of CSD 401, in this case a simulated stereotactic surgery (eg, by selecting the “Stereotaxy Left Arm” button shown in FIG. 5). The left arm mode is selected, the left arm is enabled, and the Z Axis Lock function is selected for the left tool.

  Referring now to FIG. 11, MRID 402 illustrates the GUI, which allows the user to view the 2D tabs “2D Tools” and “2D View” at the bottom left of the screen, and “2D View”, and 3D imaging mode (such as MRI device) of the subject's part (head etc.) as reflected in the 3D tabs “3D Tool” and “3D View” at the bottom right of the screen It is possible to switch between the 2D view and the 3D view taken in step 1. In FIG. 11, an indicator (in this example, a crosshair) is inside an image or part of a subject displayed in a data set of images (which can also be a 3D image consisting of multiple slices of a 2D image), The associated tool tip (e.g., surgical tool 150, or in other embodiments, the slider bar shown below the "Tool Tip Extension:" box below the tool being tracked) It is displayed at the location of the extension line extending from the tool tip by the distance selected using.

  In the FIG. 11 version of MRID 402, the 2D Tools tab has been selected and a two-dimensional image is shown with a crosshair index indicating the location of the tip of the right tool inside the subject. These crosshairs appear in response to the user selecting the “Track” button below the section for the relevant tool (s). By selecting the Track option on MRID 402, when operating the relevant input device, the user looks at the MRID and, as the tool tip advances past the subject, the tool tip relative to the subject. The location (or the tool tip extension line endpoint) can be followed (or tracked) regardless of whether the user is in simulation / training mode.

  In the version of MRID 402 shown in FIG. 12, the 3D Tools tab is selected and the location of the tool tip relative to the subject's head is shown in 3D, in this case near the right of the screen. As a result of selecting the “Plane Cut” button inside the “Right Tool” box, the 3D image in this aspect is on a plane perpendicular to the axis along which the tool tip advances. An incision is shown. The user manipulates the orientation of the 3D image with any suitable input device (eg, spaceball) to overlay the displayed image so as to obtain the desired view of the tissue affected by the proposed tool position and path. Can be moved with This superposition function becomes available following receipt of the physical positioning and MRI positioning processes and tool selection.

  Selecting the “Simple” button will replace the tool image with, for example, a thin red line of the same length as the tool so as not to obstruct the view of the small structure. When the “Wedge Cut” button is selected, the display 3D image at the location of the tool tip / extension line end is incised by incising the wedge, and three orthogonal planes (eg, sagital, Axial, coronal) are exposed, where the tip / extension line end of the tool is at the junction of the three surfaces. Such an incision option allows the user to evaluate the internal structure of the 3D MR image to determine the optimal path of the tool involved during the procedure.

  Following the positioning procedure described above, an exemplary aspect of a series of steps that can be used to set up and perform a procedure (eg, an automated biopsy) is presented below. The user may first select a mode on the CSD, such as Stereotaxy Left Arm Mode, then enable the left arm and turn on the bore camera (s). The user then selects a simulation mode on the CSD and the left manipulator (which is the left version of the manipulator 100, eg, from FIGS. 1C-3C, or US Pat. No. 7,155,316 (patent 316)). May be decoupled from the movement of the associated input device (such as input device 20). On the MRID, the user then selects the 2D Tools tab and Track mode / function in the “Left Tools” box and the crosshairs when 2D mode is selected May appear superimposed on the 2D image of the subject.

  The user may select a non-zero “Tool Tip Extension” value using the slider bar if a tool tip extension line is desired. If the Tool Tip Extension function is set beyond 0.0 mm, the crosshair will track the location of the end of the extension line. If this parameter is set to zero, the tracking function will illustrate a crosshair on the 2D slice image at the location of the tool tip (distal end). As the tool or extension line passes through the subject (eg, brain), a subsequent 2D image (eg, 2D slice) is shown. Similarly, if a tool or extension line is pulled from the subject, the previous 2D slice is shown.

  In this exemplary aspect, the user is virtual or simulated by grasping the left input device and activating an enabling button (eg, button 29) on the input device (eg, using the thumb). Tool movement may be activated. The user then intends the virtual manipulator and manipulator surgical tool shown on the CSD based on visual cues obtained by picking up the input device and switching between 2D and 3D MRID views as needed. Can be moved to the target area. In one aspect, the CSD and 2D and 3D MRID images can be updated in real time to indicate the location of the virtual manipulator and its tools (when in simulation mode).

  Once the user has determined the desired target location, the user may disable the input device so that movement of the input device does not lead to further movement of the virtual manipulator and its tools. The user then presses “Target” on either the 2D or 3D version of the MRID screen under “Automated Biopsy” (see, eg, FIGS. 11 and 12) The target location can be selected and stored in X, Y, Z Cartesian coordinates of the tool tip in image space (eg, magnetic resonance image space), which is then converted to robot space. . If the extension line value is equal to zero, these coordinates are registered as the tool tip, and if the value is greater than zero, it is registered as the end of the extension line, resulting in the target location index being a 2D field of view. Will appear at the crosshair location (eg, the red circle) and the extended line end location in 3D view to represent the intended target.

  The user then activates the input device if it is disabled (in the same way that the input device was disabled) and intends the tool tip or extension line end to the subject. Move towards the insertion point. A path indicator (eg, a green line) that connects the tip / extension line end to the selected target is then visible in 3D view, resulting in the user penetrating through the trajectory and the proposed tool tip path. You can see any organization that is played or disturbed. The user may then move the input device to the desired entry point (which may be, for example, on the surface of the brain or head or at a short distance outside the head). If cranial perforations are already provided, the user may ensure that the path passes through the cranial perforations without touching the head. When the entry point and the trajectory are acceptable, the user may then disable the input device, although this is not essential.

  The user may then press the button labeled “Start Point” on either the 2D version or 3D version of the MRID, and the indicator (eg, a green circle) will appear in the 2D view. Indicates the position of the crosshairs and the intended starting point that appears at the end of the extended line in 3D view, and this starting point is the X, Y, and Z orthogonal coordinates of the tool tip in image space (eg, magnetic resonance image space) This image space is then converted to robot space. If the extension line value is equal to zero, these coordinates are registered as the tool tip, and if the value is greater than zero, they are registered as the end of the extension line. In this aspect, the indicator changes in some way (eg, the green line turns red) to indicate that the line (or path) has been set, the start point, end point, and trajectory (or path) Appears on the CSD.

  If the user wants to change the location of the Start Point or Target, the user uses the input device to move the simulated tool tip / extension line to a new location and 2D MRID In either the version or the 3D version, the “Start Point” or “Target” button can be pressed again. In one aspect, the system requests the user to press the button associated with it again to confirm replacing the old point with the new point. After choosing an acceptable trajectory, the user can exit simulation mode by re-specifying the button on the CSD (eg, touching it again on the screen).

  After selecting / determining the desired trajectory for the selected tool, the user selects the master / slave mode on the CSD and activates the input device (eg, by pressing button 29) (shown in FIG. 4). Moving the input device (while viewing the MRID and / or bore camera or field camera image shown on the display screen 403 being moved) to move the manipulator closer to the starting point selected for the movement; Automatic movement can be performed by setting the azimuth as close as possible to the selected trajectory. The user may then disable the input device.

  On the MRID, under “automatic biopsy” in 2D or 3D view, the user can press the “tool alignment” (see, eg, FIGS. 11 and 12) button, and the manipulator Move to align and place the tool tip at or near the selection start point (eg, about 2 cm radially outward from the start point along the programmed trajectory). The user may then press the “Execute” button (under “Automated Biopsy”), and the user activates the input device and automatically moves (eg, automatic biopsy). You are prompted to start.

  The user then grasps the input device, activates the system, and activates the automatic biopsy by activating the input device in the same way as previously activated (eg, by pressing button 29). Start. By taking this step, the user will hold the input device to perform the procedure. As a result of enabling the input device, the tool moves forward at a predetermined speed towards the target location (which can be set in the initialization file), and at the point of the target location, the surgical tool is biopsy Pre-programmed functions can be performed, such as material removal. In one embodiment, the surgical tool is a biopsy tool with two small sharp spatules that are open perpendicular to the longitudinal axis of the tool and away from each other about the axis. Open, rotate 90 degrees clockwise, close again, resulting in tissue capture. The surgical tool can then reverse the direction straight out along the insertion trajectory.

  If a problem occurs during execution of automatic movement, the user can deactivate the input device (eg, by pressing button 29) and stop the movement. The CSD presents a selection box to the user, such as that shown in FIG. 13, which includes options to stop, continue, and reverse direction. Once a selection is made, the tool moves again when the user activates the input device.

  Thus, in addition to providing a uniaxial lock for movement of a given surgical tool during any procedure, aspects of the method, apparatus, and system of the present invention allow a user (eg, a surgeon) to perform an actual surgical procedure. You can also select the desired path by simulating multiple paths to the surgical tool, evaluating those paths against the tissue they affect, and selecting the target and starting points before To. The apparatus and system of the present invention is configured to (electronically) limit the tool to a linear path, so that only a starting point and a target point are required to determine the tool path. Aspects of the apparatus and system of the present invention can also include a plurality of safety features that allow the user to maintain control of the tool.

  The method aspects of the present invention may be coded as software stored on any suitable computer-readable medium (eg, tangible computer readable media), including: Any suitable form including, but not limited to, hard drive media, optical media, RAM, SRAM, DRAM, SDRAM, ROM, EPROM, EEPROM, tape media, cartridge media, flash memory, memory stick, and / or others There is a memory or data storage device. A tangible computer readable medium includes any physical medium that can store or transfer information. Such an aspect is characterized as a tangible computer-readable medium having (or having instructions coded on) computer-executable (eg, machine-readable) instructions for performing certain step (s). It is done.

The term “substantial computer-readable medium” does not include wireless transmission media such as carrier waves. However, the term “computer-readable medium” includes wireless transmission media in its scope, and some aspects of the methods of the invention may include wireless transmission media that carry the above-described computer-readable instructions. The software can be written by any technique known in the art. For example, the software may be any one or more computer languages (eg, ASSEMBLY, Pascal, FORTRAN, BASIC, C, C ++, C #, JAVA (registered). (Trademark), Perl, Python) or using scientific packages such as, but not limited to, Matlab (R), R, S-plus (R), and SAS (R) it can. The code may be for compiling it on a common platform (e.g., Microsoft (R), Linux (R), Apple Macintosh (R) OS X, Unix (R)).

  Well-established cross-platform libraries such as OpenGL® may be utilized to implement the method, apparatus and system of the present invention. Multithreading may be used wherever applicable to reduce computation time on modern single processor and multiprocessor based hardware platforms. As discussed above and shown in the figure, the software may include a GUI that can provide a more intuitive sense to the user when executing the software. Different fields may be accessed by screen touch, mouse and / or keyboard. Alarms, cues, etc. may occur via pop-up windows, audible alerts, or any other technique known in the art.

  Using a computer with a processor (eg, one or more integrated circuits) programmed with firmware and / or execution software for some (including all) of the steps described in the above section May be implemented. Some (including all) of the steps described in the sections above may be performed using a distributed computing environment, which is an example of a computer system (eg, a control system). In a distributed computing environment, any suitable number of connection media (eg, a local area network (LAN), a wide area network (WAN), or, but not limited to, Ethernet, an in-house computer network, an intranet) A plurality of computers, such as those connected by the Internet and other computer networks including the Internet), and the connection between the computers can be wired or wireless.

  Servers and user terminals can be part of any computer system. Furthermore, suitable computer system aspects may be implemented on an application specific integrated circuit (ASIC) or large scale integrated (VLSI) circuit, and additionally (or alternatively), resource virtualization, virtual computing And / or cloud computing may be used to achieve a specified function. Indeed, those skilled in the art may utilize any number of suitable structures capable of performing logical operations to accomplish the functions described above in a computer system consistent with this disclosure.

  Descriptions of well-known processing techniques, components and equipment have been omitted in unnecessary detail so as not to unnecessarily obscure the methods, apparatus and systems of the present invention. The descriptions of the methods, apparatuses and systems of the present invention are illustrative and not limiting. Certain substitutions, modifications, additions and / or rearrangements that are within the scope of the claims but not explicitly described herein will be apparent to those skilled in the art based on this disclosure.

For example, one MRID is disclosed that allows the user to switch between displaying 2D and 3D images, whereas in an alternative embodiment, two separate display screens are provided for 2D and 3D images, respectively. May be used for As another example, as examples of suitable movements that can be pre-programmed with the techniques described above, automatic movements for biopsy of brain tissue have been described above, but others that can be automated using the techniques of the present invention, etc. There are a number of surgical and / or diagnostic movements, including chest biopsy, drug implantation, electrode implantation (eg, for epilepsy), stem cell implantation, and from the spine without view Perforation of bone spurs, etc.

In addition, in the development of production aspects, a number of implementation-specific decisions must be made, such as compliance with system-related or business-related constraints that vary from implementation to implementation, to achieve a developer's specific objectives. It will be understood that this must be done. Such development efforts are complex and time consuming, but will still be routine for those skilled in the art having the benefit of this disclosure.
The appended claims use the terms “means for” and / or “step for” in a given claim to limit means-plus-function. Should not be construed as including such limitation unless expressly stated otherwise.

Claims (21)

Receiving a command to restrict movement of an instrument held or integrated with a robotic arm configured for use in surgery along one axis;
Receiving the position and orientation of the input device, provided that the input device is connected to the robot arm in a master-slave relationship in which the input device is a master, the position and orientation of the input device, and the input device The difference between the previous position and orientation of the device is consistent with the desired movement of the device; and sending signal (s) that cause movement of the device in response to the desired movement, provided This movement is along one axis and does not include movement along any axis different from the one axis.
A computer readable medium comprising machine readable instructions for performing   The computer readable medium of claim 1, wherein the robot arm has six degrees of freedom. Receiving the position and orientation of the input device, wherein the input device is connected to a robot arm in a master-slave relationship, wherein the input device is a master, and the position and orientation of the input device and the input device The difference between the previous position and orientation of the device is consistent with the desired movement of the device held by or integrated with the robot arm; and the movement of the device in response to the desired movement , But this movement is along one axis, and because of the difference between the position and orientation of the input device and the previous position and orientation of the input device, Does not include any other such movement along any axis different from the one axis, even if other movements are commanded,
A computer readable medium comprising machine readable instructions for performing   The computer readable medium of claim 3, wherein the robot arm has six degrees of freedom. Receiving a command to limit movement of a surgical tool held by or integrated with a robotic arm configured for use in surgery along a single axis of movement;
Receiving sufficient data to allow determination of a position of a portion of the surgical tool and an orientation of the surgical tool, provided that the surgical tool is based on the orientation and the single axis of movement. Having a vertical axis defining
Receiving a command to move the surgical tool; and sending a signal (s) to move the surgical tool along the single axis of movement;
A computer readable medium comprising machine readable instructions for performing At least the following functions:
Receiving a command to limit movement of an instrument held or integrated with a robotic arm configured for use in surgery along one axis;
Receiving the position and orientation of the input device, provided that the input device is connected to a robot arm in a master-slave relationship, and the difference between the position and orientation of the input device and the previous position and orientation of the input device is Consistent with the desired movement of the device; and sending signal (s) that cause movement of the device in response to the desired movement, provided that this movement is along one axis, Does not include any movement along any axis different from
A computer system configured to run.   The computer system of claim 6, wherein the robot arm has six degrees of freedom. At least the following functions:
Receiving the position and orientation of the input device, provided that the input device is connected to a robot arm in a master-slave relationship, and the difference between the position and orientation of the input device and the previous position and orientation of the input device is Consistent with a desired movement of the surgical tool held by the robotic arm; and sending a signal or signals that cause movement of the surgical tool in response to the desired movement, This movement is along one axis, and due to the difference between the position and orientation of the input device and the previous position and orientation of the input device, even if another movement is commanded, either Does not include any other such movement along the axis of
A computer system configured to run.   The computer system of claim 8, wherein the robot arm has six degrees of freedom. At least the following:
Receiving a command to limit movement of a surgical tool held by or integrated with a robotic arm configured for use in surgery along a single axis of movement;
Receiving sufficient data to allow determination of a position of a portion of the surgical tool and an orientation of the surgical tool, provided that the surgical tool is based on the orientation and the single axis of movement. Having a vertical axis defining
Receiving a command to move the surgical tool; and sending a signal (s) to move the surgical tool along the single axis of movement;
A computer system configured to run. A computer system useful for simulating, planning and / or executing automated surgical procedures,
At least the following functions:
Receive data specifying the target location for the tool held by the medical robot;
Receiving data specifying a second location of the tool from which the tool moves toward the target location during automatic movement; and in response to a user command to initiate automatic movement, the second Moving the medical robot so that the tool moves along an axis defined by the location of the target and the target location;
The computer system configured to implement   The computer system of claim 11, wherein the movement is performed only when data is received indicating that the user has activated a hand controller coupled to the medical robot in a master-slave relationship. At least the following additional features:
Let the tool operate at the target location,
13. A computer system according to any of claims 11 and 12, further configured to execute: At least the following additional features:
Before receiving data specifying the second location, a trajectory plan line extending from the target location to the tip of the tool is displayed superimposed on a three-dimensional representation representing a portion of the subject, and a display version by simulation of the tool In response to a user input from a hand controller connected to, the trajectory plan line is moved.
14. A computer system according to any of claims 11 to 13 configured to execute: At least the following functions:
Receiving a command specifying a target location for a surgical tool held by or integrated with a medical robotic arm within a portion of the patient;
Receiving a command to designate a second location for the surgical tool, wherein the second location is a location from which the surgical tool moves toward the target location;
Receiving a command from a user to initiate automatic movement of the surgical tool along an axis defined by the second location and the target location;
Sending sufficient instruction (s) to cause the medical robotic arm to achieve the automatic movement, thereby moving the surgical tool from at least the second location to the target location; Automatic movement is performed at a predetermined speed, rather than a real-time response to operation of the input device by a user, and receives an instruction to stop the automatic movement before the automatic movement is completed.
A computer system configured to run. The following additional features:
Display a field of view from a simulation of the surgical tool, including an indicator of the path from the second location to the target location;
The computer system of claim 15, further configured to perform: A computer system useful for simulating, planning and / or executing automated surgical procedures,
At least the following functions:
In response to user input, displaying an image of one or both arms of a medical robot system having two arms;
Receive commands to operate in simulation mode;
In response to user input, displaying a two-dimensional image of a portion of the subject;
In response to user input, display a target location index superimposed on the 2D image;
Move the target location indicator in response to user input;
In response to user input, display a different 2D image of a portion of the subject;
In response to user input, an image representing a tool superimposed on a three-dimensional representation of a portion of the subject is displayed, wherein the tool is shown at the same relative location as the target location indicator Having a tip or a line extending from the tool,
Moving the tool tip or the line in response to user input, and changing the 3D representation in response to user input;
A computer system configured to run. At least the following additional features:
Receive a directive specifying the target location for the tool;
Receiving a command specifying a second location for the tool from which the tool moves toward a target location during automatic movement;
Moving the medical robot in response to a user command to initiate the automatic movement so that the tool moves along one axis defined by the second location and the target location;
The computer system of claim 17, wherein the computer system is configured to execute.   The tool moving along one axis is held by one of the arms of the medical robot system, and the movement indicates that the user has activated a hand controller coupled to that arm in a master / slave relationship. The computer system of claim 18, wherein the computer system is performed only when data is received. At least the following additional features:
Let the tool operate at the target location,
20. A computer system according to any of claims 18 and 19, configured to execute   A computer readable medium comprising machine readable instructions for performing at least the functions of any of claims 11-20.