WO2024018011A1

WO2024018011A1 - Control device and system, and system having a medical operation instrument, a data capture device and a data processing apparatus

Info

Publication number: WO2024018011A1
Application number: PCT/EP2023/070175
Authority: WO
Inventors: Florian Huber; Chunman Fan; Mirko KUNZE; Kirsten Klein; Lena Felber; Thorsten Ahrens; Hans-Georg Mathé
Original assignee: Karl Storz Se & Co. Kg
Priority date: 2022-07-21
Filing date: 2023-07-20
Publication date: 2024-01-25

Abstract

The invention relates to a control device (10) having a first, optical sensor (12) and a second sensor (14) with an acceleration sensor and/or a gyroscope, wherein the optical sensor (12) is designed to capture a first piece of information about a first translational and rotational component (18) of a displacement of a first manually displacable instrument (16) relative to the optical sensor (12), and the second sensor (14) is designed to capture a second piece of information about a second component (20) of the displacement of the first instrument (16), wherein the control device (10) is designed to transmit the first and the second piece of information for positioning a second instrument (24) such that the second instrument (24) is displaced in correspondence with the first instrument (16). The invention also relates to a system (60) having a control device (10), a first instrument (16), a second instrument (24) and a motor-driven control unit (22) for positioning the second instrument (24), and to a system having a medical operation instrument.

Description

Control device and system, as well as system with a medical surgical instrument, a data acquisition device and a data processing device

The present invention relates to a control device and a system with a control device, a first instrument, a second instrument and a motor control unit for positioning the second instrument.

There are several established methods for controlling a robotic system in medical technology. This includes, for example, a joystick that is attached to a device with a screw. This type of control is primarily used to control a robotic arm, which in turn positions an endoscope. However, reaching the joystick can be uncomfortable and controlling the endoscope feels unnatural.

Other robotic systems use voice control as a control mechanism. This technology is very popular in certain markets (e.g. USA), but is being used in other regions This method is used less frequently. Voice control has a certain inaccuracy and delay. In addition, only one degree of freedom can be controlled at any given time. In addition, communication in the operating room must be subordinate to the voice-controlled system when the robot arm is repositioned.

Another option is gesture tracking. Hands-free control of the robot arm is recorded, for example, with a headband that has an infrared receiver. When the surgeon nods his head, the receiver detects the movement and commands the arm to reposition the endoscope accordingly.

It is also possible to use imaging algorithms that identify the tip of an instrument. The robot moves the instrument, particularly an endoscope, so that this tip is centered in the image. However, controlling the endoscope based on video sequence analysis is a challenge because automatic object recognition in the image is a very difficult topic, and deterministic behavior of the endoscope control cannot always be guaranteed in complex situations.

Finally, there is a new method called "Eye Tracking" that uses smart glasses (e.g. augmented reality glasses) to track the movements of the surgeon's eye, locate the intended visual target, and use the tracking information as commands to reposition the Robot arm or motorized endoscope holder.

It is an object of the present invention to provide an improved control device and system that provide intuitive control to the user.

According to one aspect, the object is achieved by a control device with a first, optical sensor and a second sensor with an acceleration sensor and / or a gyroscope, the optical sensor being designed to provide first information about a first, translational and rotational component of a displacement a first, manually displaceable instrument relative to the optical sensor, and the second sensor is designed to detect a second piece of information about a second component of the displacement of the first instrument, the second component being a roll, pitch and yaw of the first instrument , wherein the control device is designed to send the first and second information for positioning a second instrument, so that the second instrument is displaced corresponding to the first instrument. Such a control device enables the user with an instrument, the first instrument that he is already operating, to move the instrument as desired. The reaction of the second instrument takes place immediately and in accordance with the displacement of the first instrument, so that the user achieves an intuitive and direct displacement of the second instrument by displacing the first instrument. In this case, the first and second information is sent, in particular directly or indirectly, to a motor control unit which is designed to position the second instrument.

In an advantageous embodiment, the control device has a continuous passage which is designed to accommodate a shaft of the first instrument.

This configuration enables a defined interaction between the control device and the first instrument.

In a further advantageous embodiment, the optical sensor is arranged such that light reflected from the first instrument reaches the optical sensor.

This configuration makes it possible to reliably detect the translational and rotational components of the displacement of the first instrument. In particular, the passage is in the field of vision of the optical sensor.

In a further advantageous embodiment, the control device has a light source which is designed to emit light to illuminate a shaft of the first instrument.

This configuration makes it possible to reliably detect the translational and rotational components of the displacement of the first instrument. The light is emitted in particular in the direction of the passage and is at least partially reflected there by a shaft of the first instrument in the direction of the optical sensor.

In a further advantageous embodiment, the light is emitted with a structured pattern.

This configuration makes it possible to reliably detect the translational and rotational components of the displacement of the first instrument.

In a further advantageous embodiment, the optical sensor is arranged in a hermetically sealed housing with a translucent pane. This configuration makes it possible for the optical sensor to be protected from external influences, but still be able to detect or observe the translational and rotational components of the displacement of the first instrument. This disk is preferably either arranged at a certain angle to the surface of the optical sensor or provided with an anti-reflective coating of 800 to 900 nm in order to prevent reflections from the disk from disturbing the optical sensor.

In a further advantageous embodiment, the control device has an actuating device, the control device being designed to cause a displacement of the second instrument when the first instrument is displaced when the actuating device is in an activated state and to cause no displacement of the second instrument when the first instrument is displaced in a deactivated state to cause the second instrument to be relocated.

This embodiment allows the user to easily switch to a first operating mode in order to relocate the second instrument using the first instrument, or to switch to a second operating mode in order to work with the first instrument without displacing the second instrument. For example, the user can first position an endoscope with a camera using the first instrument in the first operating mode, then work with the first instrument in the second operating mode, then reposition the endoscope again in the first operating mode, and so on.

In a further advantageous embodiment, the control device has a configuration device which is designed to apply a factor to the first and/or the second information, so that a displacement of the first instrument with respect to one or more degrees of freedom results in a displacement of the second instrument a factor is applied to this or these degrees of freedom.

This embodiment makes it possible to configure a translation of the first displacement into the second displacement. For example, if insertion of the first instrument should only lead to a smaller insertion of the second instrument, a positive factor smaller than 1 can be set for this degree of freedom. If a shift in the first instrument is generally intended to lead to smaller shifts in the second instrument, a factor smaller than 1 is chosen for all degrees of freedom. If a shift in the first instrument is generally intended to lead to larger shifts in the second instrument, a factor greater than 1 is chosen for all degrees of freedom. Using a negative factor, a mirror image translation can be chosen for one or more degrees of freedom if desired by the user. In a further advantageous embodiment, the control device further has an instrument trocar to which the control device is fixed.

In a further advantageous embodiment, a first reference coordinate system of the first instrument is formed from three axes perpendicular to one another in pairs, with a z-axis of the three axes being directed from a distal end of the first instrument towards a proximal end of the first instrument, an x-axis of the three axes follows the earth's gravitational field and one y-axis of the three axes is perpendicular to the x-axis and the z-axis.

This configuration enables a practical and computationally easy to implement definition of the first reference coordinate system of the first instrument.

In a further advantageous embodiment, a second reference coordinate system of the second instrument is formed from three axes perpendicular to one another in pairs, with a z-axis of the three axes being directed from a distal end of the second instrument towards a proximal end of the second instrument, an x-axis of the three axes follows the earth's gravitational field and one y-axis of the three axes is perpendicular to the x-axis and the z-axis.

This configuration enables a practical and computationally easy to implement definition of the second reference coordinate system of the second instrument.

Embodiments of a preferred specific embodiment are described below, and individual elements of this embodiment can also be selectively combined with the previously generally described control device.

In this specific embodiment, a sensor unit is positioned in the vicinity of an instrument trocar and is kinetically coupled to it, the instrument trocar being designed to accommodate the first instrument. The sensor unit has an optical sensor, in particular a tracking sensor, and a 3-axis accelerometer and/or a 3-axis gyroscope. The optical tracking sensor is arranged to provide a direct optical access path to the instrument shaft when the first instrument is inserted into the instrument trocar while the sensor is enclosed in a hermetic housing. The accelerometer and/or the gyroscope record the absolute roll and pitch movement as well as the relative yaw movement of the sensor unit and thus of the instrument trocar. The optical tracking sensor records the translation and rotation of the first instrument. This sensor has a laser-based light source that projects a pattern onto the surface of the instrument shaft of the first instrument. The hermetic housing of the optical tracking sensor has a glass window that provides the optical access path to the instrument shaft. This window is preferably either arranged at a certain angle to the surface of the sensor or provided with an anti-reflective coating of 800 to 900 nm to prevent reflections from the glass from interfering with the sensor.

Additionally, the control device or system has an activation method that signals when the user starts the endoscope control mode. This activation method can be, for example, a foot switch, a button on the instruments or voice control. In addition, there is a connection between the second instrument, here a robot arm with an endoscope, and the first instrument, which precisely defines how the endoscope and the camera are positioned relative to the endoscope positioner, here the robot arm.

This enables direct and intuitive assignment of certain degrees of freedom. The inclination of the first instrument can be easily translated into the inclination of the second instrument or the endoscope, and the translation of the instrument shaft can be mapped to a translation of the second instrument or the endoscope.

However, this sensor arrangement has no information about the absolute orientation of the first instrument in the horizontal plane or about how the first instrument is positioned in relation to the second instrument or the endoscope or how the user or surgeon relates to the first instrument or the second instrument, here the endoscope. Therefore, a direct association between the movement of the first instrument and the second instrument is not possible. Therefore, the first information and second information mentioned are recorded and serve as the basis for a calculation as to which movement of the first instrument leads to an endoscope movement to the right or to the left.

The proposed mapping method, which is applicable to all embodiments described in this disclosure, does not require user calibration or additional sensors. Instead, the gravity vector and instrument axis are used to calculate the left-right direction according to the right-hand rule. The axis of gravity can be represented using the thumb and the direction of insertion of the instrument using the index finger. Then it will Direction in which the middle finger points is considered to be right. When the user moves the instrument tip to the right, the camera moves to the right side of the image. Accordingly, the direction of rotation can also be determined using the right-hand rule. The thumb can point in the direction of insertion. Then the curved fingers point in the direction that causes the image on the screen to rotate clockwise.

Such observations can be carried out in both a Cartesian and a spherical coordinate system. In the latter, the sensor unit forms a spherical coordinate system, in the center of which the sensor unit is located (position or horizontal orientation of the sensor unit are irrelevant). The tip of the first instrument can be viewed as a point in the coordinate system, the instrument shaft as its vector. Moving the tip in the positive direction of the azimuth unit vector causes the image to move leftward, moving in the direction of the positive polar angle unit vector causes the image to move downward. Moving the tip in the positive direction of the radius unit vector causes a zoom movement.

This mapping method was not chosen arbitrarily, but was identified as preferred because it is based on the surgeon's daily experience. If the first instrument were actually an endoscope with a camera, it would move the image on the screen in the same way that this mapping method does when the surgeon uses the first instrument to control the image rather than the endoscope directly. This will simplify the learning process for trained surgeons. In addition, the mapping method is based on the instrument itself. Every person in the operating room can understand which instrument movement on the first instrument causes which endoscope movement, i.e. movement of the second instrument. This allows everyone in the operating room to easily control the system, regardless of their positioning in relation to the monitor, the first instrument and second instrument or the endoscope or the camera.

In this context, reference is made to the explanations at https://www.scratchapixel.com/lessons/mathematics-physics-for-computer-graphics/lookat-function.

According to a further aspect, the object is achieved by a system with a previously described control device, a first instrument, a second instrument and a motor control unit for positioning the second instrument.

In an advantageous embodiment, the first instrument is designed to be guided in an instrument trocar. In a further advantageous embodiment, the second instrument is an endoscope, the endoscope in particular having a camera.

The present invention also relates to a system with a medical surgical instrument, a data acquisition device and a data processing device.

Capturing data during surgery has become more important. The documentation of the operation has been a long-standing requirement. However, it is still a challenge today to document an operation in such a way that details of the operation can be understood even after the operation has been completed.

Over time, however, the information technology support of an operation has also become more important. For example, by activating a specific instrument, a specific phase of an operation can be recognized. This information can then be processed immediately, i.e. during the operation, e.g. to make certain settings on one or more instruments or devices.

However, in practice it turns out again and again that information technology support does not always provide a complete overall picture of an operation. It is therefore an object of the present invention to provide an improved system that at least supports the execution or documentation of the operation and optionally opens up further possibilities for obtaining information.

As a solution, a system is proposed with a medical surgical instrument which is designed to be hand-held by the surgeon at least temporarily during an operation, a data acquisition device and a data processing device, the data acquisition device having a pose detector which is designed to repeatedly provide pose information for the surgical instrument to capture, wherein the pose information has at least one position or orientation information, and to send it to the data processing device, and wherein the data processing device is designed to process the pose information and support the implementation or documentation of the operation.

A special feature of this approach is that the surgical instrument is no longer viewed as just an actuator, whether guided by hand or robotically, but is used as an input medium to collect other important information. The pose detector can be designed as a separate device that can be detachably fixed to the surgical instrument. The pose detector can also be an integrated part of a motor-operated device. This device can already have one or more detectors that determine and, if necessary, monitor a pose of the device and thus of the surgical instrument in space.

However, the pose detector can also be formed inherently by the motor-operated device. The fact is that such a device, in particular a motorized robot arm, can assume specific positions in space through specific positioning positions of at least one motor. If the position of the device is now changed manually, the position of the at least one motor also changes, which in turn allows conclusions to be drawn about the current pose of the surgical instrument.

In this case, the known causality that a desired, specific position is specified, the positioning position of the at least one motor is controlled and the specific position is thus established in the space of the motor-operated device is reversed. The position in space now becomes a selectable parameter that specifies the position of the at least one motor, from which the position in space can be determined.

One aspect of the present invention addresses a topic that the inventors have identified in connection with the present invention. This topic will be referred to below as a data gap, especially as a data gap for manual operation steps.

Depending on the procedure, the methodology and the surgeon, surgical robots are only used for certain sections of the operation. While the start and conclusion of the operation are always carried out at the operating table, preparatory steps such as preparation before resection or intermediate steps in complex procedures can also be carried out manually at the operating table. During this manual phase, significantly less procedure-related data can be collected than when using the robot.

By way of example, this concerns one or more elements from the group comprising the type of instrument used, the instrument position, the change in the instrument position, the instrument condition, the instrument use, the useful life of an instrument, the camera position, the change in the camera position, the relative position between camera(s) and Instrument(s), the relative movement between camera(s) and instruments^), the relative acceleration between camera(s) and instrument(s), collisions between camera(s) and/or instruments(s). However, procedure-related data collected on the robot can be used to generate medically and economically relevant added value. The invention can advantageously reduce or close these data gaps during the manual phases at the operating table.

It is therefore proposed to carry out continuous data collection during a surgical procedure that is partially telemanipulated or carried out with a telemanipulator. For this purpose, in particular, a sensor unit is attached to the trocar and kinetically coupled to record the instrument position in the operating room. This can be done with any number of trocars.

By combining information about the instrument type, including, for example, jaw part geometry and/or kinematics, etc., and the instrument position, a digital image of reality can be created, as when used on a surgical robot, and captured and saved for further processing.

Possible applications include the following aspects: automated documentation, recognition of phases during the operation, detection of emergency situations, for example through an instrument movement, monitoring of registered pivot points, evaluations for training purposes, integration of expert systems and retrieval or control of exact positions, e.g. when changing from robotic guidance to manual guidance or vice versa.

In addition, a security feature can be implemented, which allows, for example, restricted areas to be defined. A partially robotically manipulated instrument cannot be moved into these impermissible areas, even when operated manually, or can only be moved against increased resistance. This is achieved by appropriately controlling a motorized robot arm, which is part of a surgical robot or telemanipulator.

Another aspect of the present invention addresses another topic that the inventors have recognized in connection with the present invention. This topic will be referred to below as an input unit, in particular as an input unit for controlling an endoscope on the operating table.

With surgical robots, the focus is on the main surgeon working at the console. However, it is necessary to carry out work steps manually on the operating table. This is done either by the main surgeon before working at the console or by the assistant during the robotic procedure. Stand for these manual phases The supporting functions of the robot are not available. This includes, for example, tracking the camera.

Therefore, for the manual phases at the operating table and for parallel work by the main surgeon at the console and assistant at the operating table, a table-based input system for camera control should temporarily replace the input on the console, so that the full range of functions of the robot for camera control, including automatic, can still be used at the operating table Tracking is available.

A device and a method are therefore proposed in which, via the console of the telemanipulator as well as the evaluation of the input on the laparoscopic instrument, a robotically guided endoscope can be controlled in parallel by the console of the telemanipulator as well as the evaluation of the input on the laparoscopic instrument e.g. controlling the endoscope on the robot using a manual instrument.

Another aspect of the present invention addresses another topic that the inventors have recognized in connection with the present invention. This topic will be referred to below as a security system, in particular as a redundant security system.

A requirement for the approval of a medical device is an adequate safety concept. For critical functions, this is usually based on a redundant design. Arm and instrument positions are currently primarily recorded and calculated via the sensors in the robot arm joints.

The integration of a sensor system, in particular a sensor system that is arranged near a trocar or near an puncture site, enables an independent additional measurement modality, so that the sensors can be used to monitor a safety system of a surgical robot. For example, shear forces can be recorded in the registered trocar point, which can occur when the position or registration of the pivot point or the puncture site changes. Such a situation can occur, for example, due to a movement of the patient.

It is therefore proposed to detect an instrument position by means of a sensor system, in particular a sensor system that is arranged near a trocar or near an puncture site, in order to compare the actual and target values of the kinematics of the surgical robot, in particular of at least one arm joint position of the surgical robot, to be able to carry out. With regard to the aspects mentioned above, it should be noted that this further data collection does not take place in the instrument itself. Here the detection is essentially limited to the tip of the instrument, so that, for example, an absolute position cannot be easily detected - however, only immediate variables such as forces, temperature, etc. can be detected. Instead, the data collection here is carried out in particular at a pivot point or through a passive holder.

One or more of the following values are preferably determined: absolute position, relative position in combination with the current image from the camera, movement curves, kinematics, dynamics and forces. The latter can be calculated in particular from a combination of image and absolute position, e.g. how much an instrument bends under the weight of the liver.

In one embodiment, the data processing device is designed to determine the operator's movement patterns.

The surgeon, i.e. the person who operates the instrument that is monitored with regard to the pose, is observed or accompanied as he works with the surgical instrument. In particular, conclusions can be drawn about the current condition of the surgeon, the so-called “surgeon mood”. The surgical instrument can be freely hand-held or coupled to a motor device in which the surgeon can guide the surgical instrument essentially without resistance.

In a further embodiment, the data processing device is designed to detect a tremor in the guidance of the instrument.

Intentional rhythmic movements can be identified, e.g. based on their frequency or deflection, and can be differentiated from any tremor caused by the surgeon. The identification of a tremor can, for example, indicate a strenuous position of the instrument or tension on the part of the surgeon and can be signaled to the surgeon or another person.

In a further embodiment, the data processing device is designed to detect signs of stress and/or hecticness in the surgeon.

Both absolute limit values, e.g. too high a movement speed, excessive deflection, driving into impermissible areas or rooms, etc., as well as relative changes to an empirically determined movement pattern of the operator or a group of operators can be determined. For example, this can increased speed, excessive movements or deviation from usual routes or spaces, etc., can be a sign of stress. This can be signaled to the surgeon or another person.

In a further embodiment, the data processing device is designed to detect signs of tiredness in the surgeon.

Movement dynamics that are too slow or reaction times that are too long can indicate that the surgeon has become tired. Here too, absolute limit values, e.g. a movement speed that is too low, remaining in a position for too long without doing any activity, etc., as well as relative changes to an empirically determined movement pattern of the surgeon or a group of surgeons can be determined.

In a further embodiment, the data processing device is designed to determine a three-dimensional camera position even if only a two-dimensional image is available.

The additional information recorded means that measurements, e.g. distance, distance or area measurements, can be made even in a two-dimensional image. In addition, it can also be warned that critical structures may be affected or, if the movement continues, will be affected. In this way, possible collisions or ruptures can be warned. As already explained, resistance to a potentially dangerous movement can also be built up or the movement can be blocked by appropriately controlling a robot arm.

In a further embodiment, the data processing device is designed to control the image tracking of a camera.

This can bring advantages, particularly in terms of data quality, but can also represent a security feature. The image tracking can be carried out in particular by a combination of image recognition and the pose information, in particular position data, so that the surgeon has the areas relevant to him in view. It can also be ensured that all instruments are displayed in the visible image section. Alternatively, the position of instruments that are not visible can be shown in an expanded display.

In a further embodiment, the data processing device is designed to determine a usage model. The determined usage model or operator model can include a technical load measurement of the instruments, which allows predictions to be made about the service life or remaining service life of one or more instruments. Typical movement data of the surgeon can also be determined and/or a cleaning intensity can be determined.

In a further embodiment, the data processing device is designed to create a learning curve analysis from movement data.

From the repeated recording of the pose information in conjunction with a time recording, times, paths and possible contact and/or proximity to critical structures are recorded. Based on this data, it can be subsequently determined where the surgeon can improve. In addition, any unsuccessful steps in the operation can be reproduced in a simulation, in particular by virtually re-enacting the situation, and improved in the sense of learning.

In a further embodiment, the data processing device is designed to support a non-surgeon.

The term non-surgeon should be understood as a person who does not lead or significantly design the medical procedure. This can in particular be an assisting person, e.g. assistant at the table. The data processing device can thus control image processing in which the position of his instrument is displayed to the assistant, especially if the instrument cannot be seen in the image representation currently desired by the surgeon. A hidden direction that cannot be seen in the image can also be displayed. Finally, the assistant can be supported in capturing or picking up an instrument with the camera at the trocar exit and accompanying the instrument in returning it to the target structure.

In a further embodiment, the data processing device is designed to automatically create reports and/or carry out a workflow analysis

The movement data is used to automatically divide the intervention into different phases. On the one hand, this can occur after the completion of an operation (post-op). In particular, an automated operation report can be created and/or it can be documented by stitching several images that the entire site has been inspected. On the other hand, workflows can also be optimized during the operation (intra-operation), such as determining the time for ordering the next patient. There may be delays in the operation Comparison to the average values of the hospital can be determined and, based on such a determination, a possible automated adjustment of the operation plan can be made.

In a further embodiment, the data processing device is designed to control a desired pose of the surgical instrument.

This makes it easier to change or coordinate at the console and/or at the operating table. In particular, an instrument position can be controlled again when changing from a hand-held guidance to a robotic guidance. A camera position can also be assumed again in a similar way.

It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or alone, without departing from the scope of the present invention.

It is particularly pointed out that the aspects of the various systems described can be combined, both with the control device described and with one or more of the other systems described.

The drawings, description and claims contain numerous features in combination. It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or alone, without departing from the scope of the present invention. An exemplary embodiment of the invention is shown in the drawing and is explained in more detail in the following description. It shows:

Fig. 1 shows an embodiment of a control device as part of an embodiment of a system.

1 shows an embodiment of a control device 10, which is part of a system 60, which also has a first instrument 16, a second instrument 24 and a motor control unit 22 for positioning the second instrument 24. The first instrument 16 is designed to be guided in an instrument trocar 40. The second instrument 24 here is an endoscope 54, the endoscope 54 in particular having a camera 56. The control device 10 is detachably fixed to the instrument trocar 40.

The control device 10 has a first, optical sensor 12 and a second sensor 14 with an acceleration sensor and / or a gyroscope, the optical sensor 12 is designed to record first information about a first, translational and rotational component 18 of a displacement of a first, manually displaceable instrument 16 relative to the optical sensor 12.

The second sensor 14 is designed to detect second information about a second component 20 of the displacement of the first instrument 16, the second component 20 being a roll, pitch and yaw of the first instrument 16.

The control device 10 is designed to send the first and second information to the motor control unit 22 for positioning a second instrument 24, so that the second instrument 24 is displaced corresponding to the first instrument 16. The second instrument 24 is detachably fixed to the motor control unit 22.

The control device 10 has a continuous passage 26, which is designed to accommodate a shaft 28 of the first instrument 16, in particular in a form-fitting manner with respect to the circumference of the shaft 28. The optical sensor 12 is arranged such that light reflected by the first instrument 16 58 reaches the optical sensor 12. The optical sensor 12 is arranged in a hermetically sealed housing 32 with a translucent disk 34. The light 56 comes in particular from a light source 30, which is designed to emit light 56 for illuminating a shaft 28 of the first instrument 16. The light 56 is emitted in particular with a structured pattern.

The control device 10 has an actuating device 36, wherein the control device 10 is designed to cause a displacement of the second instrument 24 when the actuating device 36 is in an activated state when the first instrument 16 is displaced and in a deactivated state when the first instrument is displaced 16 does not cause any displacement of the second instrument 24.

The control device 10 also has a configuration device 38, which is designed to apply a factor to the first and/or the second information, so that a displacement of the first instrument 16 with respect to one or more degrees of freedom results in a displacement of the second instrument 24 regarding this or these degrees of freedom, which is subjected to a factor.

A first reference coordinate system 42 of the first instrument 16 is formed from three axes perpendicular to one another in pairs, a z-axis of the three axes being directed from a distal end 44 of the first instrument 16 towards a proximal end 46 of the first instrument 16, an x-axis Axis of the three axes follows the earth's gravitational field G and a y-axis of the three axes is perpendicular to the x-axis and the z-axis. A second reference coordinate system 48 of the second instrument is formed from three axes perpendicular to one another in pairs, with a z-axis of the three axes directed from a distal end 50 of the second instrument 24 towards a proximal end 52 of the second instrument 24, an x-axis of the three axes follows the earth's gravitational field G and one y-axis of the three axes is perpendicular to the x-axis and the z-axis.

The motor control unit 22 has a processing unit 62, here a microprocessor, and a memory 64. The memory 64 has instructions with which, when these instructions are executed by the processing unit 62, control commands for the motor control unit 22 are determined from the first and second information.

A preferred programmatic implementation of the control by the control device is described below. It is assumed here that a camera of an endoscope, which represents the second instrument, is controlled by means of the first instrument, here in the function of a joystick. When this type of control is active, the system is in a camera control mode (KSM). Variables that are assigned to the camera have the identifier “cam”. Variables that are assigned to the joystick have the identifier “joy”.

When the camera control mode is switched on, the following start values are saved in variables. This is necessary because all control commands are calculated from the difference between these start values and the new measured values, see Table 1:

Table 1 If the camera control mode has been activated and initialized as explained above, the first reference coordinate system of the first instrument, i.e. the joystick, is first determined.

As already stated, the z-axis is defined by the instrument and points from the tip of the instrument to the handle. In principle, this axis can be mathematically assigned a negative sign, which results in a reversal of direction. However, the orientation suggested here is standard in camera technology; see in particular the so-called LookAt function.

As already described, two further axes are defined. In order to clearly define this, the gravitational axis is used. This is always known by the inertial sensors and can therefore be used. This axis, which follows the gravitational field, will be referred to here as the x-axis for the sake of clarity. The y-axis ultimately results as a cross product between the x-axis and z-axis. This is shown in Table 2 below:

Table 2

The calculation of the second reference coordinate system of the second instrument, here the camera or the endoscope, is carried out in accordance with the first instrument, whereby the endoscope data is now used here, see Table 3:

Table 3

Now that the first and second reference coordinate systems have been determined, the following steps are continuously carried out, see Table 4, from which important parameters for controlling the second instrument are then calculated, preferably deflection, deflection direction, roll value and insertion depth:

Table 4

The drawings, description and claims contain numerous features in combination. It is understood that the above-mentioned features can be used not only in the combination specified in each case, but also in other combinations or alone, without departing from the scope of the present invention. Control device (10) with a first, optical sensor (12) and a second sensor (14) with an acceleration sensor and / or a gyroscope, the optical sensor (12) being designed to provide first information about a first, translational and rotational Component (18) to detect a displacement of a first, manually displaceable instrument (16) relative to the optical sensor (12), and the second sensor (14) is designed to provide second information about a second component (20) of the displacement of the first Instrument (16), wherein the control device (10) is designed to send the first and second information for positioning a second instrument (24), so that the second instrument (24) is displaced corresponding to the first instrument (16). becomes. Furthermore, a system (60) with a control device (10), a first instrument (16), a second instrument (24) and a motor control unit (22) for positioning the second instrument (24), as well as a system with a medical surgical instrument disclosed.

Claims

Claims Control device (10) with a first, optical sensor (12) and a second sensor (14) with an acceleration sensor and / or a gyroscope, the optical sensor (12) being designed to provide first information about a first, translational and rotational component (18) of a displacement of a first, manually displaceable instrument (16) relative to the optical sensor (12), and the second sensor (14) is designed to provide second information about a second component (20) of the displacement of the first instrument (16), the second component (20) being a roll, pitch and yaw of the first instrument (16), the control device (10) being designed to provide the first and second information for positioning a second instrument (24) so that the second instrument (24) is displaced corresponding to the first instrument (16). The control device (10) of claim 1, wherein the control device (10) includes a through passage (26) adapted to receive a shaft (28) of the first instrument (16). Control device (10) according to claim 1 or 2, wherein the optical sensor (12) is arranged such that light (58) reflected from the first instrument (16) reaches the optical sensor (12). Control device (10) according to one of the preceding claims, wherein the control device (10) has a light source (30) which is designed to emit light (56) for illuminating a shaft (28) of the first instrument (16). The control device (10) of claim 4, wherein the light (56) is emitted in a structured pattern. Control device (10) according to one of the preceding claims, wherein the optical sensor (12) is arranged in a hermetically closed housing (32) with a light-transmitting disk (34). Control device (10) according to one of the preceding claims, wherein the control device (10) has an actuating device (36), the control device (10) being designed to be in an activated state of the actuating device (36) when the first instrument (16 ) to cause a displacement of the second instrument (24) and, in a deactivated state, not to cause a displacement of the second instrument (24) when the first instrument (16) is displaced. Control device (10) according to one of the preceding claims, wherein the control device (10) has a configuration device (38) which is designed to apply a factor to the first and/or the second information so that a displacement of the first instrument ( 16) with respect to one or more degrees of freedom leads to a displacement of the second instrument (24) with respect to this or these degrees of freedom, which is subject to a factor. Control device (10) according to one of the preceding claims, further comprising an instrument trocar (40) to which the control device (10) is fixed. Control device (10) according to one of the preceding claims, wherein a first reference coordinate system (42) of the first instrument (16) is formed from three axes perpendicular to one another in pairs, a z-axis of the three axes coming from a distal end (44) of the first instrument (16) is directed towards a proximal end (46) of the first instrument (16), an x-axis of the three axes follows the gravitational field (G) of the earth and a y-axis of the three axes perpendicular to the x-axis and to z-axis stands. Control device (10) according to one of the preceding claims, wherein a second reference coordinate system (48) of the second instrument is formed from three axes perpendicular to one another in pairs, a z-axis of the three axes being derived from a distal end (50) of the second instrument (24). is directed towards a proximal end (52) of the second instrument (24), an x-axis of the three axes follows the gravitational field (G) of the earth and a y-axis of the three axes perpendicular to the x-axis and the z-axis stands. System (60) with a control device (10) according to one of the preceding claims, a first instrument (16), a second instrument (24) and a motor control unit (22) for positioning the second instrument (24). System (60) according to claim 12, wherein the first instrument (16) is designed to be guided in an instrument trocar (40). System (60) according to claim 12 or 13, wherein the second instrument (24) is an endoscope (54), the endoscope (54) in particular having a camera (56). System with a medical surgical instrument which is designed to be hand-held by the surgeon at least temporarily during an operation, a data acquisition device and a data processing device, the data acquisition device having a pose detector which is designed to repeatedly capture pose information of the surgical instrument, wherein the Pose information has at least one position or orientation information, and to send it to the data processing device, and wherein the data processing device is designed to process the pose information and support the implementation or documentation of the operation.