WO2021173044A1 - Procédé de commande d'une caméra dans un complexe de chirurgie robotisée - Google Patents

Procédé de commande d'une caméra dans un complexe de chirurgie robotisée Download PDF

Info

Publication number
WO2021173044A1
WO2021173044A1 PCT/RU2021/050047 RU2021050047W WO2021173044A1 WO 2021173044 A1 WO2021173044 A1 WO 2021173044A1 RU 2021050047 W RU2021050047 W RU 2021050047W WO 2021173044 A1 WO2021173044 A1 WO 2021173044A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
manipulator
movement
control
surgeon
Prior art date
Application number
PCT/RU2021/050047
Other languages
English (en)
Russian (ru)
Inventor
Дмитрий Юрьевич ПУШКАРЬ
Рахим Суфьянович НАХУШЕВ
Original Assignee
Ассистирующие Хирургические Технологии (Act), Лтд
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ассистирующие Хирургические Технологии (Act), Лтд filed Critical Ассистирующие Хирургические Технологии (Act), Лтд
Publication of WO2021173044A1 publication Critical patent/WO2021173044A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/37—Master-slave robots
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/04—Viewing devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1602—Programme controls characterised by the control system, structure, architecture
    • B25J9/1605—Simulation of manipulator lay-out, design, modelling of manipulator

Definitions

  • the invention relates to assisting surgical systems for minimally invasive surgical operations. More specifically, the invention relates to a method for controlling a camera in an operator controller compensation system for controlling a robotic surgical system.
  • the invention can be used in various fields where it is necessary to visually interact with the working field located in a place that is difficult for the operator to access: medical robotics, industrial manipulators, interaction with objects in environments that are dangerous for the operator's life, and more.
  • the robotic surgical system is a complex consisting of three main units, shown in figure 1.
  • the complex consists of a master device 100, a control system 200 and an actuator 300.
  • Controllers 100 act as a master device, with which the surgeon directly interacts - mechanisms, acting with his hands on which, the surgeon generates at least three translational and three rotational movements sufficient for carrying out a robotic surgery.
  • Actuators 300 are manipulators 310, 320, on which surgical instruments 400 and an endoscopic camera 500 are attached, which makes it possible to transmit an image of the operating field.
  • a three-dimensional imaging system is used, which transmits the image from a stereoscopic camera to a 3D screen located in front of the surgeon, with the effect of real depth.
  • the interaction of all devices of the complex is carried out using the control system 200, which is the central node of the system.
  • Endoscopic cameras as applied to robotic surgery provide stereoscopic visualization and are one of the important units of the robot-assisting complex, since they provide the transmission of an image of the operating field to the operating surgeon.
  • special attention is paid to the convenience of camera control, in particular, the intuitive interaction of the surgeon with the actuator.
  • Robotic surgery compares favorably with laparoscopy in that the camera is controlled directly by the operating surgeon (in laparoscopy, an assistant is often involved for this function).
  • the controller can, at the command of the surgeon, control either a manipulator with an instrument or a manipulator with a camera. In this case, the surgeon only needs to switch the controller from instrument control to camera control using the foot pedal.
  • the main problem in camera control is that it operates in a spherical coordinate system, and the controllers that control it operate in a Cartesian coordinate system. If during the operation it is necessary to maintain a constant size and maintain the selected scale of the operating field, then when the camera is rotated, it is necessary to perform its additional compensating movement along the axis of the direction of vision: approaching or moving away, depending on which direction the camera is turned. Traditionally, this procedure has to be performed by a surgeon.
  • the selected methods and methods of controlling the camera transmitting the stereo image to the operator are important and sometimes decisive for the operation of the robot-assitant complex as a whole. This is primarily due to the need to implement a compensation system for the operator's controller, which will programmatically implement the method of an intuitive interface for the interaction of the surgeon with the manipulator on which the endoscopic camera is attached.
  • the interaction of the surgeon with the camera is different from the control of surgical instruments. This is due to the number of degrees of freedom: to control the camera, it is sufficient to control three translational degrees of freedom, and to control the instrument, three rotational degrees of the jaws (tip) of the instrument are added to them.
  • the technical problem to be solved by the present invention is the intuitive control of the camera movement for observing the operating field without the need for mastering additional compensation skills during control, improving the quality and efficiency of robotic surgery.
  • the technical result of the present invention is to create a method of interaction of a control controller with a mechanical manipulator, which is part of a robotic surgical complex, with a camera attached to it, which makes it possible to reduce or completely remove the number of compensatory movements performed by the surgeon during control camera, thereby reducing the duration of the operation, reducing the risk of surgeon errors, as well as reducing the surgeon's fatigue based on the most intuitive camera control procedure.
  • a method for compensating for camera movements includes: providing compensation for the plane of movement of the camera manipulator; ensuring scaling of movements of the control controller; ensuring the leveling of the discrepancy between the center line of the frame and the horizon line.
  • the proposed method for controlling the movement of a manipulator with a camera attached to it must meet the following requirements:
  • the compensated manipulator movement should accurately reproduce the position of the camera in the operating area in accordance with the control commands of the surgeon's control controller.
  • the proposed method for compensating the camera movement will allow changing the position of the end of the camera in one plane depending only on the depth of immersion of the camera relative to the "zero point".
  • the introduction of additional methods to compensate for the discrepancy between the center line of the frame and the horizon line will reduce the drawbacks that arise when the field of view of the camera changes and the surgeon adapts to the result.
  • the method described below for controlling the movement of the camera manipulator by the control controller is configured to facilitate the movement of the stereo camera that visualizes the surgical area in the operating field due to the compensatory scaling of the controller movements.
  • the method involves controlling the camera in a coordinate system that is similar to the Cartesian coordinate system used to move tools.
  • the fidelity of the reproduction of the control commands is guaranteed by a mechanical manipulator to which the camera is attached, the design features of which are not considered in this application, but which guarantees at least three translational degrees of freedom for the camera.
  • the accuracy of determining the change in position, as well as the generation of control commands, is guaranteed by the control controller, which ensures the transmission of the change in position in at least three translational degrees of freedom.
  • the technical result is achieved by creating a method for controlling the movement of the camera fixed on the manipulator of the robotic surgical complex, which includes two controllers for controlling the robotic surgical complex, each of which is designed to digitize the movements of the surgeon's hands and provides the transmission of motion vectors along three translational and three rotational degrees of freedom, representing the difference between the coordinates of the control controller in the initial position and the coordinates of the controller when the position of the surgeon's hand changes, while each control controller is configured to switch to the camera control mode using the control pedal, at least one manipulator with a camera attached to it for viewing the operating field, providing its movement along three translational degrees of freedom, an automatic control system, which is connected with both controllers controlled by the surgeon, and at least one manipulator, while cn is characterized by the following steps: transfer of motion vector data along three translational and three rotational degrees
  • the scaling of the displacement vector is carried out taking into account the distance of the camera from the zero point of the camera in the local coordinate system of the camera, relative to which the position vector of the camera changes with the beginning at the point where the camera enters the hole in the patient's body and with the end that coincides with the actual position of the end of the camera.
  • Figure 1 shows a block diagram of a robot-assisting complex.
  • Figure 2 depicts a structural model of a robotic surgical system that is used for the present invention.
  • Figure 3 is a perspective view of a surgeon control controller used in the present invention.
  • Figure 4 depicts a general schematic block diagram of the operation of the surgeon control controller.
  • Figure 5 depicts the location of the local spherical coordinate system of the manipulator, originating at the "zero point”.
  • Figure 6 depicts the location of the Cartesian orthogonal coordinate system of the manipulator with the origin at the "zero point”.
  • Figure 7 depicts a simulation of the change in the position of the end of the camera when changing each coordinate separately. Changing the length of the camera relative to the "zero point”.
  • Figure 8 depicts a simulation of the change in the position of the end of the camera when changing each coordinate separately. Changing the tilt of the camera relative to the longitudinal axis.
  • Figure 9 depicts a simulation of the change in the position of the end of the camera when changing each coordinate separately. Changing the tilt of the camera relative to the transverse axis.
  • Figure 10 shows the trajectory of movement of the camera manipulator when changing the angle Q along the longitudinal axis at constant f and R.
  • the movement is carried out in a circle with a radius R.
  • Figure 11 schematically shows the relationship between movements of a surgeon control controller and movement of a camera arm in a Cartesian coordinate system.
  • Figure 11A schematically shows the movements of the surgeon's hand in the Cartesian coordinate system control controller.
  • the movement of the camera in the Cartesian coordinate system of the camera manipulator is shown in figure 11B.
  • Figure 12 shows the trajectories of the camera tip in the Cartesian coordinate system of the camera and the compensated view of this trajectory after applying the proposed camera motion compensation algorithm.
  • Figures 12A, 12B and 12B show the trajectories of the camera tip during its various movements.
  • Figure 13 depicts the movement made by the surgeon's control controller and the deviation of the frame centerline from the horizon.
  • Figure 13A shows an in-frame image at zero tilt angle of the camera arm.
  • Figure 13B shows an in-frame image at a positive tilt angle of the camera arm.
  • Figure 13B depicts the required hand movement of the operator to reconstruct the horizon line in the controller's Cartesian coordinate system.
  • Figure 14 is a schematic representation of a camera manipulator zooming algorithm.
  • Figure 15 is a schematic representation of a detailed functional diagram of a camera control.
  • Figure 16 depicts a block diagram of the operation of the automatic control system of the camera manipulator.
  • Figure 19 shows the sequence of movement of the camera manipulator without using the proposed method of compensating for camera movement (without the applied plane compensation algorithm).
  • Figure 21 shows the sequence of movement of the camera manipulator using the proposed method of compensating for camera movement (using the applied plane compensation algorithm).
  • Figure 22 shows a sequence of images along a projected trajectory without horizon compensation.
  • Figure 23 shows a sequence of images along a projected horizon compensated trajectory.
  • Figure 24 depicts an object of the operating field with applied control points (left), the image of the first control point in the frame (right).
  • Figure 25 shows the results of a series of experiments after statistical processing of data reflecting the average time for positioning the camera with and without the proposed method of controlling camera movement.
  • Figure 26 shows the results of a series of experiments after statistical processing of data reflecting the average distance of movement of the surgeon's arms during positioning. manipulator of the camera with the use of the proposed method of controlling the movement of the camera and without it.
  • robot technological complex means complex systems or complexes in surgery using a robot assistant during an operation.
  • Robot assistive systems or “robotic assisted surgical systems” are robotic systems designed to perform medical operations. These are not autonomous devices, robotic assistive systems are controlled by surgeons during the operation.
  • chatronic complex or “mechatronic system” means a complex or system with computer control of motion, which is based on knowledge in the field of mechanics, electronics and microprocessor technology, computer science and computer control of the movement of machines and assemblies.
  • surgeon refers to the performing the operation of the surgeon.
  • surgeon refers to the performing the operation of the surgeon.
  • surgeon refers to the performing the operation of the surgeon.
  • surgeon refers to the performing the operation of the surgeon.
  • surgeon and “surgeon” in the present description of the invention are synonymous.
  • connection means functionally connected, and any number or combination of intermediate elements between the connected components (including the absence of intermediate elements) can be used.
  • the description of exemplary embodiments of the present invention below is provided by way of example only and is intended for illustrative purposes and is not intended to limit the scope of the disclosed invention.
  • the present solution relates to a method for controlling the movement of a camera fixed on a mechanical manipulator of a robotic surgical complex, which allows obtaining an image of the operating field during a surgical operation.
  • the robotic surgical complex consists of three interconnected main units: a patient trolley (actuator) 300, an automatic control system 200 and a control console (driver) 100, which receives commands from a surgeon 101 to further transform them into movement of surgical instruments fixed in the manipulator or to ensure the generation of control commands from the surgeon for other units of the robotic surgical complex.
  • the stereo imaging system which includes a camera 500 for obtaining an image of the surgical field 330 and a monitor of the imaging system 130 for displaying a three-dimensional image of the surgical field obtained from the camera.
  • imaging system includes a camera 500 for obtaining an image of the surgical field 330 and a monitor of the imaging system 130 for displaying a three-dimensional image of the surgical field obtained from the camera.
  • the efficiency of the operation depends on the quality of the image obtained by the surgeon.
  • the modern development of mechatronic systems and the constant improvement of robotic surgery imposes new requirements on the developed devices of the imaging system.
  • the control console 100 is located outside the sterile zone of the surgical unit and is configured to control: manipulators 301, 302, 303 with surgical instruments attached to them; a manipulator 320 with a camera 500 attached to it; directly by the surgical instruments themselves.
  • the control involves a controller 120 operated by the surgeon's hands and pedals operated by the surgeon's feet.
  • manipulators with surgical instruments and a camera are mounted on a patient trolley, which is designed to support and position them relative to the patient.
  • the robotic surgical system can have any number of manipulators, such as one or more manipulators.
  • Figure 2 shows three manipulators 301, 302, 303, made with the ability to move in three planes and rotate in three planes, as well as the manipulator 320 of the camera 500. All manipulators indicated in the structural diagram and being part of the patient cart have general mechanical characteristics and design features.
  • Each manipulator has a casing and a manipulator connection unit, to which a surgical instrument or camera can be detachably attached, the movement and position of which the surgeon can change by manipulating / controlling using a control controller that digitizes the movement of the surgeon's hand.
  • the surgeon controller 120 allows control of surgical instruments and a camera located within a patient during surgery.
  • the control controller converts the mechanical movements of the surgeon's hand over the entire natural range of motion in six degrees of freedom to generate control commands for robotic surgical complex.
  • the surgeon's controller generates a command to move the surgical instrument. Additionally, the controller controls the turning and opening / closing of the jaw on the surgical instrument.
  • the surgeon has the ability to generate at least three translational and three rotational degrees of freedom and additionally at least one degree of freedom when closing / opening jaws, which is sufficient to control a surgical instrument during a surgical operation.
  • Pedal 110 in this application is understood as a contact switching device (mechanical or electronic) capable of switching on / off the flow of current in a circuit.
  • a pedal, button, switch, switch and the like can act as such an apparatus.
  • the pedal 110 is a footswitch that closes an electrical circuit when the surgeon depresses the pedal with his foot.
  • the pedals are designed to change the operating modes of the control controller or to switch additional functionality, for example, coagulation, laser, and the like. Footswitches allow the surgeon to control the camera, instruments, electrosurgical instruments.
  • the automatic control system 200 based on the data received from the controllers 120 and the signal from the pedals 110, generates control commands that can be directed both to the manipulators 301, 302, 303 with surgical instruments and to the camera manipulator 320.
  • Pressing and holding footswitch 110 disables controllers 120 from operating surgical instruments (from operating manipulators 301, 302, 303 with surgical instruments) and enables and allows movement of manipulator 320 with camera 500. In this mode, both controllers 120, right and left work at the same time. When the pedal 110 is released, the controllers 120 will again operate the surgical instruments.
  • the automatic control system 200 receives data on three translational and three rotational degrees of freedom from the control controller 120 and generates on their basis three translational and three rotational movements of the manipulator with a surgical instrument.
  • the camera pedal 110 When the camera pedal 110 is depressed, it is sufficient to generate three translational degrees of freedom.
  • the rotational degrees of freedom of the controllers 120 are locked.
  • the image of the area to be operated, broadcast by the stereoscopic camera 500, is available to the surgeon on the monitor 130 of the imaging system.
  • a control controller it is proposed to use a mechanical structure 600 with a parallel structure, the view of which is shown in figure 3.
  • This structure 600 allows you to provide at least three translational degrees of freedom by reciprocating movement of the mechanism along three mutually orthogonal axes indicated in figure 3, which are sufficient to control the camera arm in the composition of the robotic surgical complex.
  • the depicted controller 600 consists of at least two platforms: a fixed support 610 and a movable 620, and a parallel structure, preferably made on the basis of a delta-type mechanism (delta robot or deltapod), as well as a drive mechanism that drives into the movement of the delta robot, while ensuring minimal backlash.
  • delta-type mechanism delta robot or deltapod
  • the delta robot consists of three arms 630, located at an angle of 120 ° relative to each other and attached to the support platform 610.
  • the advantage of the delta robot design is the use of parallelograms containing rods 640 of constant length, arranged in parallel in pairs and connected to each other by means of cardan joints ...
  • the parallelograms are connected at one end by the corresponding levers 630, and at the other end are connected to the movable platform 620.
  • This design allows you to maintain the spatial orientation of the robot mechanisms.
  • the movable platform 620 is always parallel to the support platform 610.
  • the connection of the arms 630 to the support platform 610 is made through the upper bearing assemblies 650 to provide the necessary angles for the initial state of the delta robot.
  • the upper bearing assemblies 650 are fixed to a support platform 610. Mounted on the upper bearing assemblies 650, the levers 630 form an equilateral triangle at the centers of the connection, the angles of which affect the size of the useful working area of the delta robot. Increasing the length of the arm increases the Z travel. The dimensions for X and Y travel are given by parallelograms.
  • a handle is attached to the movable platform 620 of the controller 600 using any of the known methods.
  • the design of such a handle is beyond the scope of this application.
  • the use of a parallel structure provides high positioning accuracy, reduces stress on the operator's arms, increases the range of movement of the arms, and minimizes restrictions on the surgeon's ability to manipulate surgical instruments, including the camera.
  • the reproduction of commands from the control controller to the direct movement of the camera is carried out by a mechanical manipulator, which is capable of moving in at least three translational degrees of freedom and providing rotation around the longitudinal axis of the device on which the camera (endoscope) is attached.
  • a block diagram 700 of the control controller operation algorithm is shown in FIG. 4.
  • a third step of operation 730 the pedal control signal P is compared with zero to determine the operating mode of the control controller.
  • each the surgeon's control controller at a constant rate transmits the three translational and three rotational degrees of freedom data to the automatic control system (step 750).
  • Data can be represented as two different vectors:
  • packets with six coordinates are sent from the controllers to the automatic control system.
  • step 730 When you press the pedal responsible for switching to the camera manipulator control mode, if at step 730 the control signal from the pedal is different from zero, then the rotational degrees of freedom on the right and left controller are blocked (step 740), leaving the possibility only for translational movements ...
  • step 750 the pedal signal and the three translational degrees of freedom data from the controllers are passed to the automatic control system (step 750), thereby notifying the system that the camera manipulator has begun to operate.
  • a camera is broadly defined in this application as any device structurally configured to generate a stereo image when inserted into a patient's body.
  • An image of the operating field can be obtained optically using fiber optics, objectives and miniaturized imaging systems, for example, using a video endoscope (hereinafter referred to as an endoscope).
  • the camera is inserted into the body through an opening in the patient's body.
  • the entry point / plane is defined as the "zero point” since all movements of the instrument or camera in the patient's body are provided by two rotational degrees of freedom about this point and one linear degree of freedom.
  • a trocar is located at the “zero point” for additional fixation of instruments and a camera.
  • the camera has a field of view that depends on the characteristics of the camera lens. The camera transmits an image of only that part of the surgical field that fell into the field of view.
  • the camera's field of view can have an area configured to be visible to the camera at any given time.
  • the field of view moves with the camera, allowing you to visually inspect the operating area.
  • the change in the field of view also depends on the peculiarities of camera movement. In this case, the surgeon must be guaranteed unambiguity in understanding the movement of the camera and the associated movement / change of the field of view within the operating field.
  • manipulator the work of which should be guided by some features that arise due to the specifics of robotic surgery. Since surgical instruments, as well as a stereo camera are introduced into the patient's body through the holes, the trocar is inserted, the position changes are made relative to the "zero point".
  • the local coordinate system of the manipulator is described by a spherical coordinate system, in which movements are carried out by angles f, q relative to the "zero point” around the longitudinal and transverse axes, as well as the radius of the camera extension R relative to the "zero point” ... Coordinates (cp.O.R) form the manipulator's local spherical coordinate system.
  • the local coordinate system of a mechanical manipulator can be considered as a change in the camera length R, as well as tilts f, q in both directions relative to the "zero point" around the longitudinal and transverse axes. This is a sufficient set of coordinates with which you can reproduce the control commands generated for the camera manipulator.
  • the location of the local spherical coordinate system with the origin at the "zero point” is shown in figure 5.
  • Figure 5 shows the manipulator assembly to which the camera is attached by means of fastening with the possibility of changing its orientation and position.
  • the movement of the manipulator can be described in the Cartesian orthogonal coordinate system with the origin at the "zero point" (figure 6).
  • the CU plane in this case will be parallel to the plane of the lower platform of the manipulator (not shown in the drawing), and the OZ axis is perpendicular to this plane and is positively directed from the "zero point" towards the end of the stereoscopic camera.
  • Figures 7-9 show changes in the position of the end of the camera when changing each coordinate separately.
  • Figure 7 depicts the change in the length of the chamber relative to the "zero point". This coordinate in the local coordinate system is responsible for approaching or moving away from the operated work area. It is important to note that within the framework of modeling a change in position, the mechanical features of the manipulator are not considered, but attention is paid to how the position of the camera changes when a certain coordinate is changed.
  • Figure 8 reflects the change in the tilt of the camera relative to the longitudinal axis, which allows the surgeon to change the position in the operating field along the longitudinal axis in the field of view. Changing the tilt of the camera with respect to the transverse axis is shown in figure 9. This change allows the surgeon to change the position in the operating field along the transverse axis in the field of view. Thus, any position of the camera in the operating field can be unambiguously determined using these three local coordinates for the mechanical manipulator on which the camera is attached.
  • Figure 10 shows as an example the trajectory of movement of the camera manipulator (trajectory of the end of the camera) only when the angle of inclination along the longitudinal axis Q is changed, at constant values of the angle of inclination along the transverse axis and the coordinates of the camera length f and R.
  • the camera will move along the inner the surface of a sphere with a radius equal to R.
  • the controller moves in its own orthogonal Cartesian coordinate system.
  • the difference in local coordinate systems between the controller and the manipulator leads to the fact that the surgeon has to make additional movements with the controller, which, during a long operation, can lead to premature fatigue and errors when holding a given operating field. Reducing the number of movements performed by the surgeon is one of the criteria for improving the quality of the operation.
  • Figure 11A schematically shows the movements of the surgeon's hand in the Cartesian coordinate system of the control controller, which provide the necessary movement of the chamber manipulator.
  • the surgeon To move the field of view of the camera relative to the object under study, the surgeon must first move the controller in the same direction.
  • the camera manipulator moves along the inner surface of the sphere with a radius equal to the length of the stereoscopic camera from the "zero point”.
  • the movement of the camera in the Cartesian coordinate system of the camera manipulator is shown in figure 11B.
  • the surgeon must manually compensate for the distance by which the position of the camera has changed relative to the initial distance from the object under study.
  • the surgeon brings the camera to the desired position in two blocks of movements. For example, to move the camera manipulator along one axis OX along the radius, the surgeon had to perform two movements - along the OX axis in the same direction as the manipulator, and along the OZ axis, towards the object under study in order to compensate for the deviation caused by movement along the radius , and thus provide linear movement.
  • the surgeon's perception of images can also introduce distortions that arise due to the difference in the coordinate system of the control controller and the camera.
  • there is no intuitive control of the camera since the rectilinear movement of the control controller is forced to oblique movement of the actuator (manipulator with the camera), which leads to discrepancies in the perception of the surgeon.
  • the surgeon needs additional training and getting used to.
  • the method of compensation of camera movement will allow changing the position of the end of the camera in one plane, equidistant from the operated area and depending only on the depth of immersion of the camera relative to the "zero point".
  • This compensated movement will facilitate camera control in the operating field by controlling and reproducing camera movement in similar Cartesian coordinate systems.
  • this leads to a leveling of differences in the local coordinate systems of the master and actuator, which increases the intuitiveness of camera control for the surgeon, eliminates unnecessary operator actions and reduces the number of movements of the surgeon control controller, which leads to an increase in the quality and efficiency of operations.
  • Figures 12A, 12B and 12C show the trajectories of the camera tip in the Cartesian coordinate system (movement along the inner surface of a sphere with a radius equal to R) and the compensated view of this trajectory (straight-line movement) after applying the proposed method of compensating for camera movement when changing the angle of inclination along the longitudinal axes at fixed coordinates of the camera length and tilt angle along the transverse axis on the control controller, when changing the tilt angle along the transverse axis at fixed coordinates of the camera length and tilt angle along the longitudinal axis on the control controller and when changing the camera tilt angles along the longitudinal and transverse axes with fixed camera length coordinate.
  • Figure 13A shows the image in the frame at zero tilt angle a of the camera manipulator.
  • Figure 13B shows the image in the frame with a positive tilt angle a of the camera manipulator a - the angle between the middle line of the frame and the horizon line.
  • Figure 13B depicts the required hand movement of the operator to reconstruct the horizon line in the controller's Cartesian coordinate system.
  • the operator presses the pedal, which transmits a signal to the automatic control system and controllers.
  • Rotation is carried out by simultaneously moving both controllers along a circle clockwise or counterclockwise in a given plane.
  • the diameter of the circle is the distance between the controllers.
  • the process of aligning the horizon line in the frame is a periodic procedure for the surgeon, which leads to additional hand movements.
  • Automation of the processes of leveling the horizon line and moving the field of view the camera relative to the investigated object will eliminate unnecessary actions on the part of the operator and reduce the number of movements by the controller.
  • FIG. 14A shows the increment of the coordinate along the X axis for the surgeon's control controller in the Cartesian coordinate system of the control controller.
  • Figure 14B shows the change in the position of the camera when the control controller moves in the Cartesian coordinate system of the camera manipulator.
  • the proposed method for compensating the plane of camera movement implies the introduction of additional algorithms into the automatic control system. Automating the horizontal alignment and zooming of camera movements will increase intuitive control and reduce surgeon fatigue.
  • the described method for controlling the movement of the camera manipulator included in the robotic surgical system according to the present invention is adapted to reduce the number of movements of the stereo camera that visualizes the surgical area in the operating field.
  • the diagram shows the main stages of the implementation of the claimed method, namely, data transfer from controllers, analysis and processing of the data obtained by applying sequential algorithms for transforming the coordinates of the camera manipulator from the moment the automatic control system receives the absolute coordinates of the position of the surgeon's hand and the state of the pedal from the controllers.
  • the automatic control system performs four sequential algorithms, the first two of which are designed to perform the main function - manipulator control, and two additional ones allow to compensate for the above disadvantages.
  • the vectors R! T , D r are transmitted from the right controller of the surgeon to the automatic control system. Vectors are transmitted from the left controller of the surgeon to the automatic control system
  • the camera manipulator control pedal When the camera manipulator control pedal is pressed, the camera control mode is activated, the signal from the pedal is transmitted not only to the automatic control system, but also to the controllers that block the rotational degrees of freedom, and the R R and R L vectors are reset to zero:
  • the vectors obtained by the automatic control system are processed using the algorithm for differentiating and combining motion. Control is carried out using the relative movement of the controller from the moment the camera starts control, initialized by pressing the pedal. Thus, the flexibility of the control system is increased.
  • the camera is controlled by both controllers at the same time.
  • the resulting AD vector can be scaled, inverted, rotated around orthogonal axes, and composited with turns. Any of the listed mathematical operations can be performed using a cross product.
  • the scaling operation takes place in the scaling and integration unit.
  • the scaled increment of the surgeon's controllers obtained at the previous stage is added to the current position of the chamber manipulator in its local Cartesian coordinate system.
  • the in-plane motion compensation unit receives the calculated position of the camera manipulator in the local Cartesian coordinate system.
  • the obtained data is simultaneously transmitted to the manipulator movement control unit with the camera attached to it for its movement inside the patient in order to display the surgical field.
  • the manipulator with the camera carries out both translational movements to compensate for the movement of the camera and controllers, and rotation around its longitudinal axis to level the horizon line and the center line of the frame of the image obtained from the camera.
  • step (810) movement data of the surgeon's right and left controllers is obtained.
  • step (820) it is determined whether the signal is from the pedal. In the event that a signal is sent to the automatic control system from the pedal that switches the controllers to the camera control mode, a signal is initiated to start the operation of the automatic control system of the camera manipulator.
  • step (830) the actual data received from the controller is sequentially sent to the calculating unit of derivatives for the right and left controllers, the increment of which is the total movement.
  • this movement is necessarily scaled taking into account the distance of the endoscope from the zero point.
  • the resulting increment is integrated with the position of the manipulator in its Cartesian coordinate system.
  • This compensated plane of motion is translated into the spherical coordinates of the manipulator f, q, I ⁇ , on the basis of which, at step (860), the compensation value of the horizon line R z ° mp is calculated.
  • the resulting data set (f, q, R, R z ° mp ) is transmitted to the camera manipulator (step 870). Then the steps are repeated.
  • the proposed method for controlling camera movement adds an intuitive interaction with the robotic surgical complex and increases the efficiency of the robotic surgery.
  • the principle of operation of the compensation of camera movements is based on the transition from one coordinate system to another, which will allow the surgeon to work in similar coordinate systems for both the master and the actuator. In turn, such work will allow the surgeon to control the camera almost immediately, since the control becomes intuitive. There is no need for the surgeon to acquire additional camera control skills, since the compensated camera movement accurately reproduces the position of the camera in accordance with the operator's control commands.
  • the applied algorithms for the compensation of the horizon line and the scaling algorithms make it possible to reduce the average time of positioning the manipulator with the camera in the patient's body to observe the operating field and to reduce the number of movements of the surgeon's hands.
  • the operability of the plane compensation algorithm was tested by performing linear movements of the camera manipulator along the Cartesian X and Y axes in the coordinate system of the camera manipulator using the surgeon's controller. At the same time, the projection on the Z-axis of the distance from an arbitrary point of the object under study to the zero point of the manipulator was observed.
  • Figure 19 shows frames of the sequential movement of the camera manipulator when the plane compensation algorithm is forcibly disabled. On all frames, the zero point and the trajectory of the end of the camera are indicated.
  • Figure 21 shows frames of the sequential movement of the camera manipulator when the plane compensation algorithm is enabled. On all frames, the zero point and the trajectory of the end of the camera are indicated.
  • FIG. 23 shows a sequence of frames plotted on the trajectory of the camera manipulator.
  • the study of efficiency was carried out by measuring the time, the number of movements with multiple changes in the field of view in the camera positioning mode.
  • the study involved surgeons with experience working on a robotic-assisted complex.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Automation & Control Theory (AREA)
  • Manipulator (AREA)

Abstract

L'invention concerne un procédé de commande du mouvement d'une caméra fixée sur un manipulateur d'un complexe de chirurgie robotisée. Ce complexe comprend deux contrôleurs de commande qui peuvent chacun numériser les mouvements des mains du chirurgien et assurer la transmission des vecteurs de mouvement. Chaque contrôleur de commande peut être commuté en mode de commande de la caméra. Le complexe comprend un manipulateur sur lequel est fixée la caméra et qui assure son déplacement, et un système de commande automatique. On effectue une transmission des données des vecteurs de mouvement des contrôleurs de commande droit et gauche vers le système de commande automatique. On effectue une commutation des contrôleurs de commande en mode de commande de caméra. On effectue une commande du mouvement du manipulateur par un déplacement simultané des contrôleurs de commande et en exécutant les étapes suivantes: sauvegarder les vecteurs de mouvement obtenus des contrôleurs de commande dans le système de commande automatique; traiter et réunir les vecteurs de mouvement des contrôleurs de commande droit et gauche afin de définir un déplacement global; mettre à l'échelle le vecteur de déplacement obtenu; compenser le mouvement; compenser l'écart de la ligne d'horizon par rapport à la ligne médiane de la trame; transmettre simultanément les données vers des mécanismes exécuteurs du manipulateur. Il est ainsi possible de réduire le nombre de mouvements effectués par un chirurgien lors de la commande de la caméra.
PCT/RU2021/050047 2020-02-25 2021-02-25 Procédé de commande d'une caméra dans un complexe de chirurgie robotisée WO2021173044A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
RU2020108119A RU2721461C1 (ru) 2020-02-25 2020-02-25 Способ управления камерой в роботохирургическом комплексе
RU2020108119 2020-02-25

Publications (1)

Publication Number Publication Date
WO2021173044A1 true WO2021173044A1 (fr) 2021-09-02

Family

ID=70735372

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/RU2021/050047 WO2021173044A1 (fr) 2020-02-25 2021-02-25 Procédé de commande d'une caméra dans un complexe de chirurgie robotisée

Country Status (2)

Country Link
RU (1) RU2721461C1 (fr)
WO (1) WO2021173044A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2757991C2 (ru) * 2020-07-06 2021-10-25 Общество с ограниченной ответственностью "Толикети" Способ автоматизированного управления роботизированным операционным экзоскопом

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090326556A1 (en) * 2008-06-27 2009-12-31 Intuitive Surgical, Inc. Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the positioning and orienting of its tip
US9188973B2 (en) * 2011-07-08 2015-11-17 Restoration Robotics, Inc. Calibration and transformation of a camera system's coordinate system
US20170189127A1 (en) * 2016-01-06 2017-07-06 Ethicon Endo-Surgery, Llc Methods, Systems, And Devices For Controlling Movement Of A Robotic Surgical System
US9907458B2 (en) * 2006-01-25 2018-03-06 Intuitive Surgical Operations, Inc. Center robotic arm with five-bar spherical linkage for endoscopic camera

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU185415U1 (ru) * 2018-07-05 2018-12-04 Федеральное государственное бюджетное образовательное учреждение высшего образования "Поволжский государственный технологический университет" Устройство для лапароскопических операций

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9907458B2 (en) * 2006-01-25 2018-03-06 Intuitive Surgical Operations, Inc. Center robotic arm with five-bar spherical linkage for endoscopic camera
US20090326556A1 (en) * 2008-06-27 2009-12-31 Intuitive Surgical, Inc. Medical robotic system providing computer generated auxiliary views of a camera instrument for controlling the positioning and orienting of its tip
US9188973B2 (en) * 2011-07-08 2015-11-17 Restoration Robotics, Inc. Calibration and transformation of a camera system's coordinate system
US20170189127A1 (en) * 2016-01-06 2017-07-06 Ethicon Endo-Surgery, Llc Methods, Systems, And Devices For Controlling Movement Of A Robotic Surgical System

Also Published As

Publication number Publication date
RU2721461C1 (ru) 2020-05-19

Similar Documents

Publication Publication Date Title
US20220175475A1 (en) Estimation of a position and orientation of a frame used in controlling movement of a tool
CN107049492B (zh) 手术机器人系统及手术器械位置的显示方法
US8892224B2 (en) Method for graphically providing continuous change of state directions to a user of a medical robotic system
US8123675B2 (en) System and method for augmentation of endoscopic surgery
JP6262216B2 (ja) 零空間を使用して操作アーム間の衝突を回避するためのシステム及び方法
JP2023101524A (ja) 遠隔操作医療システムにおけるオンスクリーンメニューのためのシステム及び方法
CN110996825B (zh) 用于在多个器械臂之间切换控制权的系统和方法
EP3713508A1 (fr) Systèmes et procédés d'enregistrement et de commande maître/outil pour un mouvement intuitif
CN111278350A (zh) 定位外科手术机器人系统的摄像头以捕获患者体腔内的图像
EP3892227A1 (fr) Robot chirurgical et procédé d'affichage d'image d'un patient placé sur une table chirurgicale
US20240189049A1 (en) Systems and methods for point of interaction displays in a teleoperational assembly
KR20230058115A (ko) 수술 로봇에 의한 내시경의 제어
RU122281U1 (ru) Роботизированное устройство для проведения малоинвазивных эндоскопических операций
WO2021173044A1 (fr) Procédé de commande d'une caméra dans un complexe de chirurgie robotisée
Draelos et al. Teleoperating robots from arbitrary viewpoints in surgical contexts
US20220211270A1 (en) Systems and methods for generating workspace volumes and identifying reachable workspaces of surgical instruments
Lobontiu et al. Robotic surgery and tele-surgery: Basic principles and description of a novel concept
US12023117B2 (en) Projection of user interface pose command to reduced degree of freedom space for a surgical robot
US20230011725A1 (en) Projection of user interface pose command to reduced degree of freedom space for a surgical robot
US20230270510A1 (en) Secondary instrument control in a computer-assisted teleoperated system
US20220323157A1 (en) System and method related to registration for a medical procedure
Tang et al. Implementation of an intuitive writing interface and a laparoscopic robot for gynaecological laser assisted surgery
JP2023018313A (ja) 手術支援システムおよびピボット位置設定方法
CN116848569A (zh) 生成虚拟现实引导的系统和方法
CN112437643A (zh) 用于操纵组织的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21759917

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21759917

Country of ref document: EP

Kind code of ref document: A1