RU2757991C2 - Method for automated control of a robotic operational exoscope - Google Patents

Abstract

FIELD: medical equipment.

SUBSTANCE: invention relates to medicine, namely to a method for automated control of a robotic operating exoscope, based on mechanical movement of the video camera of the exoscope, using a closed system of combining tracking drives forming an internal circuit and performing controlled movement of the moving parts of the manipulator according to the degrees of freedom thereof. The angular coordinates of the moving parts of the manipulator are measured therein. The input control impacts on said closed system are formed by means of technological actions of the surgeon. The focal length of the optical system of the video camera of the exoscope is altered. An image of the surgical operation area is formed on the monitor screen based on the signals from the video camera of the exoscope. The input control impacts on the internal circuit are formed in the external circuit of a two-circuit system for automated control of the observation processes closed through the surgeon using the robotic operating exoscope. For this purpose, the coordinates of the spatial position of the head of the surgeon observing the image of the surgical operation area on the monitor screen are measured in the initial state and in the current time of performing the surgical operation. Signals completing the measurements of the position of the head of the surgeon in the initial state to the signals corresponding to the current spatial position of the head of the surgeon are formed. The completing signals are converted into signals proportional to the coordinates of the required spatial position of the video camera of the exoscope. The input control impacts are formed for the local movement tracking drives of the manipulator links in accordance with the converted signals for completing to the signals corresponding to the required spatial position of the video camera of the exoscope.

EFFECT: increase in the efficiency of using the operation time, reduction in the labour intensity of the work of the surgeon due to the reduction in the volume of auxiliary supporting technological operations and the improvement in the quality of performed surgical operations are achieved.

3 cl, 8 dwg

Description

The technical field to which the invention relates

The invention relates to the field of medical technology, providing information support for surgical operations (assistance), and can be used for remote control of the spatial parameters of the field of view of operating exoscopes.

State of the art

For many years, binocular operating microscopes have been the standard intraoperative equipment for performing microsurgical operations requiring high magnification imaging of the surgical field.

A feature of the functioning of the optical scheme of a surgical microscope is the fact that the greater the magnification and optical resolution that the surgeon uses, the shallower the depth of field. As a result, when working at high magnification, the surgeon has to constantly change the focal length of the optical system of the microscope. And when performing minimally invasive and low-traumatic surgical approaches, a decrease in the size of the trepanation window leads to an increase in the number of movements of the microscope, and, accordingly, to an increase in the time to adjust the focus and the total duration of the surgical intervention.

The known method of controlling the spatial orientation and focal length of the optical system of the operating microscope, implemented in the device "Marika" [1]. The method allows the surgeon, without interrupting the operation, to control the position, orientation and focal length of the microscope using the lips and chin.

The disadvantage of this method is that working with a microscope requires the surgeon to have a permanent forced, connected with the microscope, head position and directing his gaze strictly into the binoculars of the microscope, which leads to a low efficiency of using the operation time, increased fatigue of the surgeon and an increase in the labor intensity of his work.

With the latest advances in optics and digital video signal processing, surgical imaging systems such as exoscopes are becoming more common in surgery. In contrast to the operating microscope, the exoscope is much smaller and is fixed at a distance of 25-75 cm above the operating field, which provides the surgeon with a large space for work, and the enlarged image obtained by the exoscope video system is displayed on the monitor. Also, there is no requirement for a forced position of the head, tied to the observation angle and to the distance from the head to the surgical field. However, the known operating exoscopes have also inherited the disadvantages of operating microscopes associated with low efficiency in the use of operating time and increased labor intensity of the surgeon.

The known method of visualization of the operating field, implemented in an exoscopic surgical system [2]. The device contains a lens system for observing the field of a surgical object, an object illumination system and a system of shafts and brackets for manual control of the exoscope.

The exoscope is controlled by the surgeon manually, that is, the surgeon is forced to interrupt the operation in order to transfer the hand or hands of both hands to the handle or control handles of the exoscope. In accordance with this method, the surgeon, or the surgeon's assistant, stops the operation, removes his hands with the instrument from the working area of the operation, frees his hands from the instruments, manually removes the visualizer body, i.e. a video camera (VC), which forms the observation field of view, from mechanical clamps, observes the image from the VC on the monitor screen, manually moves the VC body in turn relative to the degrees of freedom of the mechanical suspension (tripod), based on the results of visual analysis of the image, selects the VC position that suits it in the entire working space the observation area, mechanically fixes all the degrees of freedom of the mechanical suspension, takes the instrument in his hands and re-introduces them into the working area of the operation. Thus, a significant number of stages of the operation when using this device is performed manually.

The disadvantages of this method are the low efficiency of using the operation time and the increased complexity of the surgeon's work.

The known method of remote control of the endoscope camera [3], according to which commands to change the spatial position of the head of the digital endoscope camera and the lighting module are generated using foot pedals. The method of remote control using a foot pedal includes the following technological operations:

- moving the camera body and camera head to the left and right;

- moving the camera body up and down;

- change of focal length;

- regulation of light intensity.

The disadvantage of the described control method is the increased complexity of the operating surgeon's work due to the need to distract him to carry out control operations using the legs.

There is a known method of remote control of a medical exoscope, implemented by a device worn on a finger or a hand, disclosed in a remote control system [4].

The remote control system consists of navigation equipment, a robotic arm, a visualization system with a camera and its control unit.

Exoscope video camera, fixed on the distal link of the manipulator, all links of which rotate with the help of drives controlled in a closed loop according to the mismatch signals between the reference influences and the signals of the spatial position of the exoscope from the navigation equipment. Specifying influences are formed with the help of a device worn on the surgeon's finger or on the hand of his hand.

Such a construction of this remote control system requires the surgeon to change the position of the fingers in order to turn on the corresponding switches during the execution of the surgical intervention when forming the control actions on the closed loop. However, the fingers of the operating surgeon are always occupied with surgical instruments. As a result, the use of this system when the fingers are occupied with additional control movements reduces the accuracy of surgical manipulations and lowers the quality of surgical and especially microsurgical techniques, where even low-amplitude incorrect movements are unacceptable.

The disadvantages of this method are the increased labor intensity of the surgeon and a decrease in the quality of the surgical technique.

The known method of assistance in performing a surgical operation using a medical navigation system, consisting of a locating meter of the spatial coordinates of the access port and the spatial coordinates of the imaging device, computing device, pedal or joystick that form the setting action, and a multi-link manipulator with drives according to the number of degrees of freedom [5 ] - prototype.

In accordance with this method, the setting action is generated by manually moving the access port or by mechanically pressing the pedal or mechanically moving the joystick slider, or by voice commands; tracking of reference actions due to angular displacements of the manipulator links during the formation of a single feedback on the measurements of the spatial coordinates of the image forming device (including a video camera - VC); generating a signal at the start of tracking when the tracking error exceeds the tracking error threshold within a certain predetermined period of time; forming an image of the area of surgical interest on the screen using a video signal generated by the imaging device; generating information about the spatial coordinates of the image forming device according to the local coordinates of the position of the links of the manipulator relative to each other; automatic change of the focal length of the optical circuit of the imaging device after its spatial movement, changing the distance between it and the area of surgical interest.

The disadvantages of this method are the low efficiency of the use of the operation time and the increased complexity of the surgeon's work.

Disclosure of invention

The purpose of the invention is to improve the efficiency of using the operation time, reduce the labor intensity of the surgeon by reducing the volume of auxiliary supporting technological operations and improving the quality of surgical operations.

This goal is achieved by the fact that the input control actions on the closed system for combining servo drives (internal circuit) are formed in the external circuit, closed through the surgeon, a two-circuit system for automated control of observation processes using a robotic operating exoscope, for which the coordinates of the spatial position of the head of the surgeon observing by depicting the surgical operation zone on the monitor screen, both once in the initial state and at the current time of the surgical operation, signals are generated that supplement the measurements of the position of the surgeon's head in the initial state to signals corresponding to the current spatial position of the surgeon's head, and the complement signals are converted into signals, proportional to the coordinates of the required spatial position of the exoscope video camera, form the input control actions for the local servo drives for the movement of the manipulator links in accordance with the pre taken signals of complement to the signals corresponding to the required spatial position of the exoscope video camera.

Reducing the complexity of the surgeon's work and increasing the efficiency of using the operation time is achieved by the fact that the spatial position and orientation of the VC is controlled by the movements of the surgeon's head, who examines the image of the surgical operation zone on the monitor, and selects the necessary and convenient spatial and optical image parameters for the operation as follows. as if the surgeon was looking directly at the surgical site optically. At the same time, such a "virtual look" can be at any point of the hemisphere above the real area of the surgical operation, while the surgeon's head mainly looks perpendicularly at the monitor screen, making small angular and translational movements limited by the size of the operation area itself. For this, technical solutions are used that implement a virtual optical similarity between the field of view (FOV) of the surgeon's gaze (in relation to the monitor screen) and FV VK (in relation to the real area of the surgical operation).

The principle of increasing the efficiency of automation of the processes of controlled observation by the surgeon over the area of the surgical operation with minimal distraction of the attention of the surgeon and parts of his body (except for the head) is as follows.

In the process of performing the operation, the surgeon's hands are occupied with instruments placed in the surgical operation zone (SZO), and therefore they must be relieved of all supporting functions that are not directly related to the process of surgical intervention. Therefore, it is necessary to create a closed tracking loop that would take over the functions of controlling the spatial position and orientation (APO) of the VC exoscope without the participation of body parts (except for the head) of the surgeon. This automated control principle is illustrated in FIG. 1.

For a comprehensive solution to the problem, it would be necessary to relate the parameters of the VK PPO with the angular position of the surgeon's pupils, as is implemented, for example, in the helmet-mounted target designation and pilot display system. But the overall dimensions of such helmets at the current level of development of this technique unacceptably complicate the surgeon's operation.

Therefore, in this technical solution, the pupils of the eyes are replaced with a section of the surgeon's head, in which the sensors for measuring the APR of this section of the head are located, perhaps in the maximum proximity to the eye sockets. Naturally, the real movements of the surgeon's pupils could introduce certain distortions, but they are minimized due to the relatively large distances from the surgeon's eyes to the monitor screen.

With such a technical solution, it is necessary to take into account not only the PPO of the eyes and VC, but also the peculiarities of the formation of the optical parameters of two different observation systems: "the surgeon's eyes - the monitor screen" and "VC - real CCO". Such a complete accounting leads to a significant complication of the device that implements this method, therefore, as a particular implementation of the method, we will consider an automated control device for the application software with simplified (reduced) blocks for performing the operations of this method.

Let us explain the ideology of automation of observation processes using the block diagram shown in Fig. 2

и

) и ее поступательного перемещения (

и

). Размещение датчиков ориентации и датчиков поступательного перемещения показано на упрощенной схеме движения головы на фиг. 3.Surgeon I, examining the image 105 on the monitor 3, formed by VK 13, by rotational movements of the head, as well as its translational movements in the orthogonal directions of the horizontal plane, achieves the desired "angle" of the image 100, i.e., in fact, it implements the functions of the discriminator 1 closed automated loop (external). The choice of the desired angle leads to a specific PPO of the surgeon's head 110 with some lag 115 from his decision to the movement of his head (taken into account by block 2 of the dynamics of head movement). At the same time, together with the head, spatial movements are performed by sensors of 4 head rotation angles rigidly attached to it (

and

) and its translational movement (

and

). The location of the orientation sensors and translational motion sensors is shown in the simplified head movement diagram in FIG. 3.

(собственно датчик ориентации, положение которого условно показано на этой схеме, не совпадающей с точкой размещения глазных впадин) в процессе трех поворотов переходит из положения

в положение

затем в результате двух поступательных движений - в положение

Такое смещения радиус-вектора

показано на фиг. 4.Simplification of a specific biomechanical model 5 consists in the hypothesis adopted in this device that implements the proposed method: the head rotates around a point O _g stationary with respect to the head, which can only perform strictly horizontal translational movements along the axes O _g X _g and O _g Z _g . Therefore, the vertex of the radius vector

(the orientation sensor itself, the position of which is conventionally shown in this diagram, which does not coincide with the point of placement of the eye sockets) in the process of three rotations moves from the position

into position

then, as a result of two forward movements - to the position

Such a displacement of the radius vector

shown in FIG. 4.

естественно, не равняется расстоянию R_вк от ВК до ЗО, поэтому в самом общем случае переход от перемещения головы хирурга к необходимому перемещению ВК должен производиться в виде нелинейного преобразования. Но, для рассматриваемого частного случая примем упрощающую гипотезу, что ВК в пространстве полностью повторит поступательный путь (с учетом поправочного коэффициента

), пройденный в том же пространстве глазом хирурга, т.е. радиус-вектор

переместится из положения

через положение окончания вращательного движения головы

в конечное положение

It should be noted that the distance from the surgeon's eyes to the monitor screen is indicated by

naturally, it is not equal to the distance R _vk from the VC to the ZO, therefore, in the most general case, the transition from the movement of the surgeon's head to the necessary movement of the VC should be made in the form of a nonlinear transformation. But, for the particular case under consideration, we accept the simplifying hypothesis that the VC in space will completely repeat the forward path (taking into account the correction factor

), traversed in the same space by the surgeon's eye, i.e. radius vector

will move out of position

through the position of the end of the rotational movement of the head

to the final position

In this case, converting the measured signal of the "virtual eye" PSO 120 (the location on the head of the orientation angle sensor) in the block 5 for simulating the movement of the surgeon's head, we obtain a signal 125 of the spatial position of the "virtual eye". This signal, in turn, is converted in block 6 for matching the PZ of the "virtual eye" of the surgeon and PZ VK, which makes it possible to determine the signal 130 of the required PPO VK.

If we consider the diagram of the working observation area (RZN), which is shown in Fig. 5, then it can be concluded that if it is necessary to move the VC to sufficiently far apart RZN sub-regions (for example, from the D _n sub-region to the D _{k sub-region} ), the surgeon will have to perform rotational movements of his head with a large amplitude (span). This situation leads to the fact that the monitor screen will leave the field of view of the surgeon.

To overcome such a disadvantage, this method provides a second control mode for the VK PPO. In this mode, the surgeon performs not joint, but alternate rotational movements of the head around three axes, each of which is associated with one of the three parameters of the VK PPO. When the amplitude 135 of this head movement exceeds a predetermined threshold, a signal 140 is generated in block 7, which leads to a change in the controllable in this case parameter PPO VK with a constant speed (movement in space), while the surgeon is able to return the head to its original position for a calm monitoring the change in the image of the surgical operation zone. If the VK PPO suits the surgeon, he makes a rotational movement of his head around the same axis, but in the other direction, and when the amplitude 135 of this head movement exceeds a predetermined threshold (with the opposite sign), a signal 140 is generated in block 7, stopping the VC in the PPO at the moment the time of this excess.

The choice of the required mode is carried out by the surgeon by changing the state of the switching element 8, the output signal of which 150 can be equal to either signal 130, or signal 140, or signal 145.

The VC takes the starting position for performing the first mode of functioning of the robotic exoscope - tracking the movements of the surgeon's head.

Depending on the specific kinematic scheme of the manipulator (mechanical suspension VK), block 9 generates signals 155 proportional to the required values of the angles of rotation for each of the degrees of freedom of this suspension.

Signals 155 is a control action on the internal closed loop III, the mismatch signal 165 of which is formed by subtracting from signal 155 signal 160 PPO VK, due to its measurement by a system of sensors 12 located on the links of the manipulator using links 180. Block 10 forms the structure of interconnection local drives 11 of the multi-link manipulator, as a result of which corresponding control signals 170 are supplied to each of the local drives. These drives, forming the control mechanical moments 175, move the VC 13 of the exoscope to the desired position. With the help of the video signal 185, the ZHO image is transmitted to the monitor 3.

The control device for the field of view of the VC exoscope during the operation is, from the point of view of control processes, a servo system (SS), the generalized block diagram of which is shown in Fig. 6.

Closing with a single negative feedback in this SS (outer contour) is performed by the surgeon himself by comparing the real position of the VC according to the image observed on the monitor and according to the desired image formed by the surgeon, based on his professional competence.

In this device, which implements the inventive method, the object of control (OU) is a multi-link manipulator with its own specific kinematic scheme inherent only to it, in this case a manipulator with 5 degrees of freedom

The executive body (IO) is a set of five electromechanical drives based on stepper motors.

The regulator forms the hardware composition of the control device, connections between the elements (structure) of the device, modes and scenarios for the implementation of these modes.

In the proposed technical solution, the discriminator, the regulator, the IO and the OA are combined into an integral system, with the qualities inherent in just such a composition of elements and connections, which is commonly called a combination of servo drives (OSP).

The implementation of the installation of the payload (VK) of the multi-link manipulator at a given point of the RZN with the required PPO VK according to this method is carried out by combining four servo drives (Drive I, Drive II, Drive III and Drive IV), which, from the point of view of the operation of the proposed method, technically represent is a unified complex technical control system - OSB, the structural diagram of which is shown in Fig. 7.

The V drive implements the following function.

Since the object of observation is located in the horizontal plane, when rotating around the vertical axis, the image rotates, which is observed on the monitor screen. With the help of the V drive, the VC is rotated around its optical axis by an angle equal to the angle of rotation of the manipulator around the vertical axis.

FIG. 7 details the inner contour of the automated control system for a robotic operating exoscope (block diagram of an automated device that implements this method).

FIG. 7 shows the local closed contours that are part of the OSB:

I-1, II-1, III-1, IV-1 and V-1 - regulators of five local servo drives (SP);

I-2, II-2, III-2, IV-2 and V-2 - executive motors of five local joint ventures;

II-3, III-3 and IV-3 - tilt sensors of three local SP;

I-3 and V-3 - sensors of the angle of rotation of two local joint ventures;

I-4, II-4, III-4, IV-4 and V-4 - digital blocks for processing measurement signals;

The rest of the designations correspond to those shown in FIG. 2.

Blocks 3, 4, 5 and 6 are an external closed loop controller.

Negative feedback of the inner loop, i.e. SIR is formed by a sensor IV-3, the digital processing unit 4 IV-measurement signals forming the signal measurement of the angular position φ VC _VC. The basic Drive IV is fed with the control input action ϕ of the _ctrl.vk , as a result, the signal ε _{int.contour of the} mismatch of the inner contour of the automated control system of the robotic operational exoscope is generated.

FIG. 8 shows the general scheme of the operation by the surgeon in accordance with the proposed method.

Blueprints

FIG. 1 shows the principle of automated control of an exoscope.

FIG. 2 shows the ideology of automation of observation processes using a block diagram.

FIG. 3 shows the placement of orientation sensors and translational movement sensors on a simplified head movement diagram.

FIG. 4 shows the offsets of the radius vector

FIG. 5 shows a diagram of the working area of observation.

FIG. 6 shows a generalized block diagram of the servo control system for the field of view of the VC exoscope.

FIG. 7 shows the local closed loops that are part of the servo drive combination.

FIG. 8 shows the general scheme of the operation by the surgeon in accordance with the proposed method.

Sources of information

1. Patent RU 2331383, publ. 08/20/2008.

2. Patent EP 2514357 A1, publ. 24.10.2012.

3. Patent US 5,762,603, publ. 9.06.1998.

4. Patent US 2017/0239010 A1, publ. 13.01.2017

5. Patent US 10588699 B2, publ. 03/17/2020.

Claims (3)

1. A method for automated control of a robotic operating exoscope, based on the mechanical movement of the exoscope video camera using a closed system for combining servo drives (inner contour), carrying out controlled movement of the movable links of the manipulator along its degrees of freedom, measuring the angular coordinates of the movable links of the manipulator, the formation of input control actions on this closed system by technological actions of the surgeon, changing the focal length of the optical system of the exoscope video camera, forming an image of the surgical operation zone on the monitor screen by signals from the exoscope video camera, operations, the input control actions on the closed system for combining the servo drives (internal circuit) are formed in the external circuit, closed through the surgeon, a double circuit system of automated control of observation processes using a robotic operating exoscope, for which the coordinates of the spatial position of the surgeon's head observing the image of the surgical operation zone on the monitor screen are measured, both once in the initial state and at the current time of the surgical operation, signals are generated that complement measuring the position of the surgeon's head in the initial state to signals corresponding to the current spatial position of the surgeon's head, converting the complement signals into signals proportional to the coordinates of the required spatial position of the exoscope video camera, generating input control actions for the local servo drives for moving the manipulator links in accordance with the converted complement signals to signals corresponding to the required spatial position of the exoscope video camera. 2. The method according to claim 1, characterized in that when the values of the complement signals corresponding to the current spatial position of the surgeon's head exceed a constant predetermined threshold, these complement signals form a sign of the direction of angular movements of the surgeon's head around three degrees of freedom of its movements, depending on the sign of the direction of the surgeon's head movements, control signals are generated to change the spatial position of the exoscope video camera at a constant speed, the signals of the required spatial position of the exoscope video camera are recorded in the form of memorizing the input control action of the inner contour when the direction of the angular movements of the surgeon's head changes. 3. The method according to claim 1, characterized in that to generate signals corresponding to the required spatial position of the exoscope video camera, the complement signals corresponding to the current spatial position of the surgeon's head are converted into signals corresponding to the spatial coordinates of the center of the field of view of the surgeon's observation relative to the image of the surgical operation zone on the monitor screen, signals proportional to the coordinates of the required spatial position of the exoscope video camera are generated by adding signals to the initial values of such signals to complement the values of the spatial coordinates of the center of the field of view of the surgeon's observation relative to the image of the surgical operation zone on the monitor screen.

Images