JP5444209B2 - Frame mapping and force feedback method, apparatus and system - Google Patents

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to US Provisional Application Serial No. 60/912144, filed April 16, 2007, which is incorporated herein by reference.
An example of a surgical robot that the method, apparatus and system of the present invention can be used in the procedures associated therewith is disclosed in US Pat. No. 7,155,316, incorporated herein by reference.

The method, apparatus and system of the present invention relate generally to the field of surgical robotics, and more specifically, the display position of the slave manipulator is relative to the operator regardless of the actual position and orientation of the slave manipulator. As can be seen intuitively, it relates to techniques for controlling slave manipulator movement by the operator controlling the master manipulator in a way that the movement of the slave manipulator can be presented to the operator via the display. . The method, apparatus and system of the present invention also relate to a force feedback system.

  In a broad sense, the technique of the present invention is a multi-degree of freedom such as a slave manipulator (e.g., a robot arm disclosed in US Pat. Is related to controlling the movement of the robot arm having

  If the operator commands the slave manipulator to move by manipulating the master manipulator while viewing the actual movement of the slave manipulator through the display, the slave manipulator's actual position and orientation of the slave manipulator The slave manipulator moves and the slave manipulator position and orientation are independent of the master manipulator position and orientation at any given time, so that the display of its movement to the operator is intuitive, regardless of the actual position and orientation. The display is in a location that does not correspond to the actual location of the displayed information (for example, the operator may have a stereo camera system installed in another room next to the slave manipulator (in another room)). Use was taken via the stereo display of the slave manipulator, viewing the slave manipulator).

  Controlling the movement of the slave manipulator in this way gives an intuitive feel for the movement of the slave manipulator, regardless of the actual position and orientation of the slave manipulator relative to the operator and / or the master manipulator that the operator is operating. To give. For example, a slave manipulator holding an instrument (eg, a surgical tool) (or multiple slave manipulators if there are multiple master manipulators) of the actual microscope taking the displayed image (s) The operator sees it at the same relative position and orientation as if the operator was holding the tool below.

  That is, some aspects of the techniques of the present invention are embodied in a control system configured for use with at least one medical robotic arm, the control system configured to perform at least the following: Is done. That is, the calculated motion of the tool of the medical robot arm is determined. In this case, the determined motion is based on the input value received from the control device that provides the input from the user, and the calculated motion is determined by the tool. It is expressed in an observer coordinate system based on the azimuth and the azimuth of the observer reference structure; and outputs an operation signal for moving the medical robot arm in accordance with the calculated movement. The operation signal may be any suitable type of data including information sufficient to properly move the medical robot arm. For example, the operational signal may represent a set of joint displacements and / or joint velocities that are output by the control system to the local controller of the medical robotic arm or directly to individual joint actuators.

  The techniques of the present invention also relate to providing force feedback to the operator in an intuitive manner. Force feedback can be programmed from the actual force measured at the slave manipulator (such as the force at the tool tip), or by the operator, or at the slave manipulator joints (including speed limits and joint angle limits). It may be derived from the resulting virtual force or from a combination of real and virtual forces. The actual force may be measured by a slave manipulator configured to measure the tool tip force by coupling the surgical tool to the end effector via a force sensor and a flexible coupler.

  The flexible coupler can facilitate operation of the surgical tool without providing the tool tip force with a load path that bypasses the force sensor. For example, a flexible coupler is compliant in all directions except for rotation about its central axis, thereby allowing any component of the force at the tool tip (eg, the end effector of a robot arm). (Via the robot arm). As a result, all of a given tool tip force can be balanced by the force acting through the force sensor, and calculation of that tool tip force can be achieved from the value measured by the force sensor. .

  Any aspect of the method, apparatus (eg, computer readable medium), and system (eg, computer system such as a control system) of the present invention comprises the aforementioned functions, steps, and / or features (comprise / include / contain). may consist of them or consist essentially of them (instead of / have). That is, in any claim, the phrase “consist of” or “consist essentially of” is intended to change the scope of a given claim from the scope that would otherwise not use an open-ended linking verb. Alternatively, any of the above open-ended linking verbs may be substituted.

  The following drawings are presented by way of example and not limitation. The same reference numerals do not necessarily indicate the same structure, system, or indication. Rather, identical reference numerals are used to indicate similar features or features with similar functions. For clarity of illustration, not all features of each aspect are always shown in every figure in which that aspect appears. The controller devices, manipulators and tools shown in the figures are drawn to scale, meaning that the dimensions of the described elements are relatively accurate relative to each other.

FIG. 1 is a perspective view of a medical robot system suitable for use in aspects of the present invention. FIG. 2 is a schematic diagram of the robot system of FIG. Several coordinate systems related to aspects of the present invention. FIG. 3 is a perspective view of a workstation that can be used by a remote operator to control the robotic system of FIG. A controller device coordinate system is also shown. FIG. 4 is a perspective view of a controller device (shown as a hand controller) and associated coordinate system that can be used to control the robotic system of FIG. FIG. 5 is a schematic diagram illustrating the location relative to the observer reference structure. FIG. 6 is a schematic diagram illustrating several coordinate systems related to aspects of the present invention. FIG. 7 is a perspective view of a medical robot arm suitable for use with aspects of the present invention illustrating the alignment of the observer coordinate system.

FIG. 8 is a top view of a two-armed medical robot system suitable for use in aspects of the present invention showing the alignment of the corresponding observer coordinate system. FIG. 9 is a top view of a medical robot arm suitable for use with aspects of the present invention showing the alignment of the observer coordinate system. FIG. 10 is a top view of a medical robot arm suitable for use with aspects of the present invention showing the alignment of the observer coordinate system. FIG. 11 is a side view, top view, and end view of an end effector of a medical robotic arm suitable for use with aspects of the present invention. The end effector holds the tool. 12 is a side view of the end effector of FIG. 11 showing the force at the tool tip as measured by a sensor coupled to the end effector. FIG. 13 is an exploded view of the end effector of FIG. FIG. 14 is a diagram illustrating an exemplary arrangement of an overhead camera and medical robotic arm suitable for use in the method, apparatus and system of the present invention.

  The term "comprise" (and all forms of riserise such as "comprises" and "comprising"), "have" (and all forms of have such as "has" and "having"), "include" (and " All forms of include such as "includes" and "including") and "contain" (and all forms of contain such as "contains" and "containing") are open-ended connective verbs. That is, a method including a step includes at least the described step, but is not limited to having only the described step.

Similarly, a computer readable medium comprising machine readable instructions for performing certain steps has machine readable instructions for performing at least the described steps, but for performing additional undescribed steps. A computer-readable medium that also includes a medium having machine-readable instructions. Further, a computer system configured to perform at least certain functions is not limited to performing only the described functions, as long as the system is configured to perform the described functions. It may be configured in a manner that is not done or in a plurality of ways.

The terms “a” and “an” are defined herein as one or more, unless expressly stated otherwise. The term “another” is defined as at least a second or more.
In some aspects, a surgical robot, such as that disclosed in US Pat. No. 6,057,086, can be controlled by an operator using a virtual microscope display that shows an image of the surgical scene viewed from the microscope. The operator controls the manipulator (robot arm of the robot) using two hand controllers located in alignment with the virtual microscope view in the scene. The effect is that the manipulator holding the surgical tool in the virtual microscope view is visible to the operator at the same relative position and orientation as if the surgeon is holding the surgical tool under the actual microscope.

  In such aspects, to achieve intuitive control of a given manipulator by a surgeon, the manipulator responds to an operator input in a direction that mimics the movement that the surgeon (operator) performs with his hand. I have to do it. If the surgeon pushes his hand away from his body, the surgeon would expect the manipulator in the microscope view to move away in the same visual direction. Since the visual perspective does not map directly to any physical reference frame on the manipulator, a virtual observer reference frame is created to map what the operator expects into what the manipulator performs. This reference frame mapping allows the hand controller stylus and surgical tool to be independently oriented while maintaining tool movements intuitive to the operator.

  The goals for creating an intuitive mapping include: (a) A platform (robot base) to which the robot arm is coupled to the user while performing a procedure, from a stylus movement that approaches or moves away from the user. Generating a tool movement along the tool axis from a stylus movement in a direction along the stylus axis, and (b) is there. The determination of the transformation scheme that can be used to achieve the frame mapping of the present invention is optional, including by the positioning method disclosed in co-pending international application PCT / US08 / 60538, which is incorporated by reference. This can be done by any suitable method.

  In such an aspect, the virtual observer frame has a positive X-axis that always points in the direction away from the robot base and appears to point in the virtual microscope frame to the direction away from the operator. The positive direction Z-axis of the hand controller reference frame is in a direction along the hand controller stylus axis from the rear end to the front end of the stylus. The positive Z-axis of the tool is along the tool axis from the tool holder toward the tip, so movement only along the Z-axis of the stylus will produce movement along only the Z-axis of the tool.

  In such an aspect, the virtual observer frame reference axis “X” is defined to always point in the direction away from the operator, and this axis is positioned as the direction from the bottom to the top of the microscopic image shown to the operator. It is done. In practice, the observer frame is a virtual frame that is aligned with the tool axis on the manipulator end effector and has the following characteristics: “Z” is mapped to the stylus axis on the hand controller and is the length of the tool Defined along the vertical direction; “X” is mapped as described above so that at the tool tip, the projection of that axis when viewed from above is always aligned with the X axis of the microscope view. The “Y” axis is determined from the right-hand rule of the coordinate frame.

  Therefore, this virtual observer frame is the tool tip frame, but is rotated around the tool Z axis so that the projection of the tool X axis onto the robot base XY plane is aligned with the robot base X axis. ing. As the manipulator moves, this virtual observer frame is updated frequently, if not always, to keep the X axis aligned with the microscope camera X axis. This frame is independent of the orientation of the tool and manipulator or the orientation of the tool frame. As a result, the X and Y hand controller commands in the camera observer frame always result in the intended X and Y movement when viewed in the microscope camera image, and the stylus Z movement is always along the tool axis. Will be.

1.0 Observer Frame Mapping The following description relates to an example of a method for implementing aspects of the frame mapping technique of the present invention suitable for use with surgical robots of the type disclosed in US Pat. .
FIG. 1 illustrates a robotic system suitable for use with the frame mapping technique of the present invention. The medical robot arm 100 and the second medical robot arm 101 are attached to a system base 10 that is a common base. Each robotic arm may have a tool (which may also be characterized as an instrument) suitable for medical (eg, surgical) procedures.

  FIG. 2 illustrates several coordinate systems related to aspects of the present invention. The observer reference structure 200 can be any structure that remains fixed in the orientation of the image presented to the remote operator (eg, for the duration of the medical procedure). That is, any structure that is fixed relative to the overhead camera that provides the operator with images used during the medical procedure can serve as the observer reference structure 200. In the embodiment shown in FIGS. 1 and 2, the system base 10 remains fixed with respect to the orientation of the camera 21 and is therefore suitable as the observer reference structure 200. FIG. 14 shows an overhead camera 21 installed above a patient undergoing a procedure performed by the medical robot arm 100, which may be a stereo camera. Other suitable observer reference structures 200 include an overhead camera 21 and a structure attached to the overhead camera 21.

  The orientation of the reference structure coordinate system 210 is fixed to the orientation of the observer reference structure 200, and the coordinate system is generally horizontal, with the X axis pointing in the direction pointing to the patient's area where the surgical procedure is performed. Selected to have. This area of interest remains generally fixed in place relative to the observer reference structure 200 during the surgical procedure. Such orientation provides a correlation between the X axis of the reference structure coordinate system 210 and the image presented to the operator by the overhead camera. This orientation selection is such that the reference structure coordinate system 210 corresponds to the vertical axis of the image and the + X direction points to the top of the image. The Z axis is chosen to point upwards and the resulting Y axis direction follows the convention of the right hand coordinate system.

  The orientation of the observer coordinate system 250 is determined from the orientation of the tool 400 and the orientation of the observer reference structure 200. In some aspects, the Z axis of the observer coordinate system 250 is aligned with the tool axis 460 of the tool 400, and the X axis of the observer coordinate system 250 is its projection in the XY plane of the reference structure coordinate system 210. Are aligned to be parallel to the X axis of the reference structure coordinate system 210 and to point in the same direction. FIG. 6 shows an observer coordinate system 250, which is (1) parallel to the Z axis aligned with the tool axis 460 and (2) the X axis of the reference structure coordinate system 210. The orientation is determined to have an X axis with a projection 269 in the XY plane of the reference structure coordinate system 210 pointing in the same direction. The resulting Y-axis direction of the observer coordinate system 250 follows the convention of the right-handed coordinate system. 7-11 further illustrate the alignment of the observer coordinate system 250. FIG. 11 shows a side view, top view, and end view of a coordinate system 250 having an end effector 110, a tool 400, a tool axis 460, and an axis 260 aligned with the tool axis 460 of the medical robot arm.

  In a system that includes multiple robot arms that operate simultaneously, such as a robot arm suitable for use in a microscopic surgical procedure, each robot arm has a unique observer coordinate system, and the orientation of each observer coordinate system is , Depending on the orientation of the observer reference structure and the orientation of the tool of the respective robot arm. In some aspects, the orientations of the plurality of observer coordinate systems may change, all of which are represented by their projections in the XY plane of the reference structure coordinate system and the X axis of the reference structure coordinate system. It will have an X axis of each observer coordinate system that is parallel and aligned to point in the same direction.

  FIG. 8 shows a two-arm configuration (medical robot arm 100 and second medical robot arm 101), which has the following: (1) an observer aligned with the tool axis 460 of the tool 400 The coordinate system 250, the X axis of the observer coordinate system 250 is such that the projection of the X axis on the XY plane of the reference structure coordinate system 210 is parallel to the X axis of the reference structure coordinate system 210, And (2) an observer coordinate system 251 aligned with the tool axis 461 of the tool 401, the observer coordinate system whose X axis is the reference structure coordinate system The projection of its X axis in the XY plane of 210 is aligned to be parallel to the X axis of the reference structure coordinate system 210 and pointing in the same direction.

  FIG. 3 shows a workstation suitable for a remote operator to control some aspects of the medical robot arm 100 and the second medical robot arm 101. Multiple video monitors are available to the operator to view information from video cameras, magnetic resonance imaging systems, or other data presentation systems. A viewing instrument 22 may provide high resolution images from an overhead stereo camera. In some aspects, the viewing device 22 is a binocular microscope. The controller device 300 and the second controller device 301 give user inputs of desired behavior to the medical robot arm 100 and the second medical robot arm 101, respectively. Each controller device corresponds to a controller device coordinate system (350, 351).

  FIG. 4 shows another view of one aspect of the controller device 300 and the second controller device 301. The Z axis of the controller device coordinate system 350 is aligned with the vertical axis of the controller device 300 in the same manner that the Z axis of the observer coordinate system 250 is aligned with the tool axis 460 of the tool 400. The + X axis is aligned to point away from the operator, and the Y axis orientation follows the convention of the right-handed coordinate system. With the alignment of the controller device coordinate system 350, the X axis, Y axis, and Z axis of the controller device coordinate system 350 viewed from the operator's viewpoint on the workstation correspond to the X of the observer coordinate system 250 viewed from the observer reference structure. It becomes the same azimuth | direction with respect to an axis | shaft, a Y-axis, and a Z-axis (refer FIG. 6). Since the observer coordinate system 250 is fixed to the orientation of the overhead camera that provides the image of the target area displayed on the visual field device 22, the X, Y, and Z axes of the controller device coordinate system 350 are This is intuitively related to the direction seen by the operator when viewing the target area.

  Thus, the input signal to the controller device 300 represented in the controller device coordinate system 350 can be directly mapped to the motion of the tool represented in the observer coordinate system 250, and the generated motion is This is intuitive to the operator. For example, the +2 movement (in arbitrary units) in the X axis of the controller device coordinate system 350 generated by the operator moving the controller device 300 away from the operator's body is in the X direction of the observer coordinate system 250. It can be mapped directly to the calculated motion of the tool 400 +2 (in the same unit).

  After solving the inverse kinematics problem, an appropriate control signal can be output and the robot arm 100 can be actuated to obtain movement of the tool 400 in a direction away from the observer reference structure 200. This movement appears as a movement of the tool 400 toward the top of the screen on the visual field device 22 and closely matches the movement of the controller device operator. In some aspects, the absolute value of the calculated motion may be scaled while maintaining the same direction (eg, in the above example, +1 instead of +2 in the X direction of the observer coordinate system 250, Mapping to the motion calculated by the tool 400).

  FIG. 5 illustrates the intuitive relationship obtained through the use of the observer coordinate system and the controller device coordinate system of aspects of the technique of the present invention. In some aspects, the observer reference structure is placed near the patient as if it were the primary surgeon. The placement of the auxiliary surgeon can be similar to that performed when a human primary surgeon is present. An operator who controls the robotic system from a remote workstation is presented with the same view as a human surgeon who is manipulating the tool by hand and viewing the area of interest through the binoculars, which was imaged by the microscope head Present the left and right views of the image to the surgeon. The robot system operator manipulates the robot's tool through the controller device as if it were a tool in front of the operator, corresponding to the field of view imaged by the human surgeon's microscope. The target area is viewed through a binocular microscope that presents left and right fields of view imaged by the camera.

2.0 Observer Frame Mapping Used for Force Feedback Determination of the coordinate system described above is also useful in providing force feedback to the operator in an intuitive manner. In some aspects, the measured force, typically measured in the sensor coordinate system associated with the sensor's mounting orientation, is converted to an observer coordinate system equation that is then transferred to the controller device coordinate system. Mapping may be performed. Measurement forces may be combined with virtual forces, which provide additional information to the operator, such as virtual collisions, joint limits, and motion divergence sources. May be based on

  The control system may be configured to include a three-dimensional model of robot arms, tools, and other structures related to operator force feedback. In some aspects, collisions between tools are allowed, but other collisions may be prohibited. When the intended movement causes a collision between models, the movement of the manipulator is temporarily stopped, and the feedback force is fed back to the controller device in the direction opposite to the movement by the command. Collision detection algorithms are well known in the art and those skilled in the art will understand to use these algorithms to determine collision avoidance limits. The feedback force applied to the controller device may be proportional to the distance commanded by the collision avoidance limit. Additional exclusion limits that are not related to the physical structure may be modeled and input by the operator.

  In some aspects, joint limitations are handled in a manner similar to collisions. When the joint enters a range in the vicinity designated for the joint restriction, the movement of the manipulator is temporarily stopped, and the force is fed back to the controller device in the direction opposite to the commanded movement. This feedback force applied to the controller device may be proportional to the angle over which the commanded joint angle exceeds the limit.

Motion divergence occurs when the actual manipulator position deviates or lags from the commanded position. This can occur if the commanded motion is too fast for the manipulator actuator to handle, or if something is pressed against the manipulator. In some aspects, when a deviation occurs between the commanded trajectory and the actual trajectory, a force is fed back to the controller device in a direction opposite to the commanded motion. The feedback force applied to the controller device may be proportional to the deviation error.
In some aspects, force feedback applied to the controller device is based on virtual collisions, joint limits, and motion deviation sources. Each virtual force feedback source may be scaled and then summed or the summed force scaled. The control system may be configured to monitor and sum only non-zero values.

  FIG. 12 is a diagram showing a part of a medical robot arm capable of providing a tool tip force by measurement. Tool tip force 901 acts on tool 400, resulting in a force that can be measured by first sensor 120 and second sensor 122. The tool 400 is held by a first tool holder 150 and a second tool holder 152, which are coupled to the first sensor 120 and the second sensor 122, respectively. The first sensor 120 and the second sensor 122 are attached to the end effector 110. In addition, the flexible coupler 130 is coupled to the end effector 110, and the tool 400 can be operated via the end effector tool operating gear 140. However, the flexible coupler 130 is designed to accommodate all directions except for rotation about its central axis, and therefore does not transmit any component of the tool tip force 901 to the end effector 110. . Accordingly, the entire tool tip force 901 is balanced by the force acting through the first sensor 120 and the second sensor 122, and the calculation of the tool tip force 901 is calculated by the first sensor 120 and the second sensor 122. This can be done from the value measured by

  FIG. 13 represents the components shown in FIG. 2 shown in an exploded view. In one aspect, the first sensor 120 and the second sensor 122 are 6-axis sensors, each using a plurality of silicon strain gauges inside a cylindrical sensor. The first sensor 120 and the second sensor 122 may be commercially available force sensors, but due to consistency with the magnetic resonance imaging (MRI) environment, the available candidates are limited or require modification. There are things to do. In some aspects of a medical robotic arm that incorporates force feedback from measurements, the first sensor 120 and the second sensor 122 are MR-enabled by replacing the ferromagnetic material with titanium, ATI industrial automation. A modified version of the Nano17 force / torque sensor from (ATI Industrial Automation).

The measurement and processing of force by measurement in some embodiments is as follows.
Each force sensor outputs an electrical signal that is passed through a calibration matrix that is unique to each sensor, and the final output is the 6-axis force and torque for each sensor. The coordinate orientations of the two force sensor outputs are then aligned so that the X / Y / Z axis forces represent the same direction for each sensor. If the controller device does not feed back torque to the user, the torque is ignored. The forces from the two sensors are then summed to calculate the total actual force, which is a combination of bias force, tool and holder gravity, tool actuation force, and tool tip force. These forces are handled as follows.

  Due to drift (over time) in the sensor, a bias force may be present and may be erased after scanning a new tool. When the stylus is enabled (after tool change), the force from the force sensor is read and set to zero. If the tool does not change the orientation, this will also remove the force due to gravity, but with the change in orientation, the force component (X / Y / Z) due to gravity applied to the tool / holder will change. Will cause harmful force feedback. To avoid this, gravity compensation is used. In the initial position, the force due to gravity is known from the mass of the tool (eg from the initialization file) and the orientation of the tool (from robot kinematics). As the orientation changes, the force due to gravity is eliminated by simply adding a change in the calculated gravity vector to the detected force.

  Knowing the mass (from the initialization file) and the orientation of the tool and fixture (from the robot kinematics), their gravity force is mathematically calculated and erased so that the user can weight the tool / holder , Thereby minimizing fatigue and maximizing the perception of the tool tip force.

  The method aspects of the present invention may be coded as software stored on any suitable computer-readable medium (eg, tangible computer readable media), including: Any suitable form including, but not limited to, hard drive media, optical media, RAM, SRAM, DRAM, SDRAM, ROM, EPROM, EEPROM, tape media, cartridge media, flash memory, memory stick, and / or others There is a memory or data storage device. A tangible computer readable medium includes any physical medium that can store or transfer information. Such an aspect is characterized as a tangible computer-readable medium having (or having instructions coded on) computer-executable (eg, machine-readable) instructions for performing certain step (s). It is done.

The term “substantial computer-readable medium” does not include wireless transmission media such as carrier waves. However, the term “computer-readable medium” includes wireless transmission media in its scope, and some aspects of the methods of the invention may include wireless transmission media that carry the above-described computer-readable instructions. The software can be written by any technique known in the art. For example, the software may be any one or more computer languages (eg, ASSEMBLY, Pascal, FORTRAN, BASIC, C, C ++, C #, JAVA (registration). (Trademark), Perl, Python) or using scientific packages such as, but not limited to, Matlab (R), R, S-plus (R), and SAS (R) it can. The code may be for compiling it on a common platform (e.g., Microsoft (R), Linux (R), Apple Macintosh (R) OS X, Unix (R)).

  Well-established cross-platform libraries such as OpenGL® may be utilized to implement the method, apparatus and system of the present invention. Multithreading may be used whenever applicable to reduce computation time on modern single processor and multiprocessor based hardware platforms. As discussed above and shown in the figure, the software may include a GUI that can provide a more intuitive sense to the user when executing the software. Different fields may be accessed by screen touch, mouse and / or keyboard. Alarms, cues, etc. may occur via pop-up windows, audible alerts, or any other technique known in the art.

  Using a computer with a processor (eg, one or more integrated circuits) programmed with firmware and / or execution software for some (including all) of the steps described in the above section May be implemented. Some (including all) of the steps described in the sections above may be performed using a distributed computing environment, which is an example of a computer system (eg, a control system). In a distributed computing environment, any suitable number of connection media (eg, a local area network (LAN), a wide area network (WAN), or, but not limited to, Ethernet, an in-house computer network, an intranet) A plurality of computers, such as those connected by the Internet and other computer networks including the Internet), and the connection between the computers can be wired or wireless.

  Servers and user terminals can be part of any computer system. Furthermore, suitable computer system aspects may be implemented on an application specific integrated circuit (ASIC) or large scale integrated (VLSI) circuit, and additionally (or alternatively), resource virtualization, virtual computing And / or cloud computing may be used to achieve a specified function. Indeed, those skilled in the art may utilize any number of suitable structures capable of performing logical operations to accomplish the functions described above in a computer system consistent with this disclosure.

  Descriptions of well-known processing techniques, components and equipment have been omitted in unnecessary detail so as not to unnecessarily obscure the methods, apparatus and systems of the present invention. The descriptions of the methods, apparatuses and systems of the present invention are illustrative and not limiting. Certain substitutions, modifications, additions and / or rearrangements that are within the scope of the claims but not explicitly described herein will be apparent to those skilled in the art based on this disclosure. In addition, in the development of production aspects, a number of implementation-specific decisions must be made, such as compliance with system-related or business-related constraints that vary from implementation to implementation, to achieve a developer's specific objectives. It will be understood that this must be done. Such development efforts are complex and time consuming, but will still be routine for those skilled in the art having the benefit of this disclosure.

  The appended claims use the terms “means for” and / or “step for” in a given claim to limit means-plus-function. Should not be construed as including such limitation unless expressly stated otherwise.

Claims (19)

A method for controlling movement of a robot arm (100) of a medical robot system, the robot arm being operatively associated with a tool (400), wherein the medical robot system includes:
An operator-controlled medical robot having at least one robot arm;
A camera (21) configured to provide an image of the tool; and
A reference structure (200) having a fixed orientation relative to the camera;
The method is:
a. Receiving an input value from the controller device (300) representing the desired movement of the tool, wherein the input value is represented in the controller device coordinate system (350);
b. Determining a calculated motion of the tool based on the input value, wherein the calculated motion is a first axis (260) oriented in a fixed relationship with the axis (460) of the tool (400), and Represented in the observer coordinate system, including a second axis oriented perpendicular to the first axis, and determined in the X and Y axes of a rectangular coordinate system (210) determined in relation to the reference structure The projection of the second axis onto a plane parallel to the X axis and aligned to point in the same direction;
c. Determining a robot arm commanded motion based on the calculated motion and the position of the robot arm; and
d. Outputting an operation signal for moving the robot arm according to the movement by the command;
including. An input value can be represented in the controller device coordinate system as an input vector having an input absolute value and an input direction;
The computed motion can be represented in the observer coordinate system as a computed motion vector having an absolute value of the computed motion and a direction of the computed motion; and
The method of claim 1, wherein the calculated direction of motion is equal to the input direction. Determining a calculated observer force value expressed in an observer coordinate system and based on a virtual force value; and
Output a feedback signal to the controller device according to the observer power value by the calculation.
Further including
The method according to claim 1 or 2. Determining a calculated observer force value, expressed in the observer coordinate system, based on the force value measured from the robot arm; and
Output a feedback signal to the controller device according to the observer power value by the calculation.
The method according to claim 1, further comprising: The force value obtained by the measurement represents a force measured at a location on the robot arm, and is represented by a sensor coordinate system. The determination of the observer force value by calculation is performed by determining the force value by the measurement and the robot arm. The method according to claim 4, based on the position of The method according to claim 4 or 5, wherein the calculated observer force value is also based on the virtual force value. The calculation of the virtual force value comprises using a virtual collision by one or more calculations, using one or more joint limits, motion deviations, or a combination thereof. The method described. The medical robot includes two robot arms (100, 101), each operatively associated with a tool (400, 401), the method comprising:
Receiving a second input value from a second controller device (301) representing a second desired movement of the second tool, wherein the second input value is a second controller device coordinate system (351); );
Determining movement of the second tool (401) by the second calculation based on the second input value, provided that the movement by the second calculation is determined by the second observer coordinate system (251); The second observer coordinate system is represented by a first axis (260) oriented in a fixed relationship with the axis (460) of the second tool (401), and perpendicular to the first axis. Said second axis to a plane represented by the X and Y axes of a Cartesian coordinate system (210) represented in an observer coordinate system and determined in relation to said reference structure, comprising an oriented second axis Aligned so that their projections are parallel to the X axis and point in the same direction; Determining a second commanded motion of the second robot arm based on the second calculated motion and the position of the second robot arm; and
Outputting a second operation signal for moving the second robot arm according to the movement of the second command;
The method according to any one of claims 1 to 7, further comprising: Determining an observer force value from a second calculation represented in a second observer coordinate system and based on the second virtual force value; and
Outputting a second feedback signal to the controller device in accordance with the observer power value obtained by the second calculation.
The method of claim 8, further comprising: Determining an observer force value from a second calculation, expressed in a second observer coordinate system, based on a force value from a second measurement from a second medical robot arm; and
Outputting a second feedback signal to the controller device in accordance with the observer power value obtained by the second calculation.
The method of claim 8, further comprising: The force value by the second measurement represents a force measured at a location on the second robot arm, and is represented by a sensor coordinate system. The determination of the observer force value by the second calculation is as follows: The method of claim 10, based on a force value from the second measurement and a position of the second robot arm. The method according to claim 10 or 11, wherein the observer force value according to the second calculation is also based on the second virtual force value. The calculation of the second virtual force value comprises using a virtual collision by one or more calculations, one or more joint limits, motion deviation, or a combination thereof. The method according to 9 or 12. The method of claim 1, wherein the observer coordinate system further comprises a third axis perpendicular to both the first axis and the second axis. 15. The method of claim 14, wherein the controller device coordinate system includes three orthogonal axes, and input values received from the controller device are directly mapped to tool calculated motion. The controller device coordinate system includes three orthogonal axes, and each axis of the controller device coordinate system (350) viewed from the observer's viewpoint is the observer coordinate system (250) viewed from the viewpoint of the reference structure (200). 15. A method according to claim 14, wherein the orientation is identical to the corresponding axis of the. A computer readable medium comprising machine readable instructions for performing the method of any one of claims 1-16. A control system configured to control movement of at least one robot arm (100, 101) of a medical robot system, the robot arm being operatively associated with a tool (400, 401); The medical robot system is:
An operator-controlled medical robot having at least one robot arm;
A camera (21) configured to provide an image of the tool; and
A reference structure (200) having a fixed orientation relative to the camera;
The control system is:
Computer equipment operatively associated with the medical robot, the computer equipment being configured to implement the method of any one of claims 1-16, and
At least one controller device (300, 301) operatively associated with the computer equipment and configured to receive an input of a desired movement of a robot arm;
including. 19. A control system according to claim 18, further comprising a viewing device (22) configured to display an image of the tool.