WO2023062231A1 - Surgical system comprising haptics - Google Patents

Abstract

The present invention relates to a computer-assisted method comprising the steps of: - reading in geometric data of an object; - assigning density values or hardnesses to volumetric portions of the object; - visualizing the object by means of the computing unit (101) on the basis of the read-in data in a virtual three-dimensional space, thus obtaining a virtual object (110); - tracking the movement of a simulation instrument (187); - providing a simulation device (180) comprising haptics (185) for the user (U), optionally comprising a simulation instrument (187); - guiding the simulation device (180) or the simulation instrument (187) in the real space; - visualizing a virtual surgical instrument (187') in the virtual space; - identifying a collision between the virtual surgical instrument (187') and the virtual object (110) and/or volumetric portions thereof; - determining a force feedback for the duration of the collision taking into consideration the density values or hardnesses of the volumetric portions; - transferring the force feedback determined to the simulation device (180) or the haptics (185) thereof.

Description

Description

Operating system with haptics

The present invention relates to a computer-supported or computer-implemented method according to claim 1, a computer system according to claim 18 and a system according to claim 19 or according to the preambles or generic terms of these claims.

Fractures, such as those of the bones of the hand, can lead to mechanical impairment and limited mobility of the hand. That is why surgeons often have to surgically treat and stabilize fractures of the hand. To fix the bone fragments, specially developed, thin, sharp K-wires (also known as Kirschner wires) of different thicknesses are drilled through a small incision in the skin in a minimally invasive manner into the broken bone. In these complex, surgical procedures with limited visibility, it is extremely important to avoid injury to risk structures such as nerves or vessels. Many years of experience as well as extensive theoretical knowledge and practical training are required for surgeons to carry out a minimally invasive hand operation correctly and safely. It is currently possible to train bores using synthetic and/or thawed cadaver bone. However, animal carcass bones offer limited realism due to poorly modeled human anatomy, and human cadaver parts are very expensive and limited in availability. Additional training traditionally takes place in the operating room, which is very time-consuming (based on on the compensable time in the operating room and the time of the surgical instructor ) . Therefore, in clinical training, in addition to the exchange of experience between inexperienced doctors and more experienced colleagues, training devices on computers are also used , on or . with which even difficult interventions can be practiced without risk.

An object of the present invention is to specify a further computer-aided method. A computer system and a system are also to be specified.

The object according to the invention is achieved by a computer-aided or computer-implemented method having the features of claim 1 and by a computer system having the features of claim 18 . In addition, it is solved by a system with the features of claim 19 .

According to the invention, a computer-aided method is thus proposed which includes the following steps:

Reading geometric data of an object into a computing unit; optional : assigning density values or hardnesses to volume sections of the object ;

Visualizing the obj ects using the computing unit based on the data read, z. B. in a virtual, three-dimensional (3D) space, on a display or monitor, e.g. B. a 3D monitor (in this case the visualization can be a stereoscopic display on the monitor); optional: providing a real 3D object, optionally based on the geometric data; providing a simulation device with - z . B. manual or bi-manual - haptic, optionally with a simulation instrument for the user, wherein the simulation instrument can be in particular a volume-removing instrument, which in turn can be, for example, a drill or milling cutter; optional : providing a real-time optical tracking camera ;

Tracking the movement of the simulation device or the simulation instrument in real space, e.g. B. by acquiring position values of a haptic arm, robot arm or parts thereof or the simulation instrument, by means of the optional tracking camera or in another way;

guiding the simulation device or the simulation instrument in the real space by the user; Visualizing a virtual surgical instrument, for example on the display or monitor; determining a collision between the virtual surgical instrument on the one hand and the virtual object and/or volume sections of the virtual object on the other hand; determining a force feedback for the duration of the collision, taking into account the density values or hardnesses of the volume sections; and transferring the determined force feedback to the simulation device or the haptics of the same, preferably continuously or permanently.

The steps of the invention described above

Methods can optionally be carried out in the above-mentioned order, with a time overlap or simultaneously. In particular, in some embodiments, the last four of the steps listed above may be performed multiple times and/or in a loop.

According to the invention, a computer system is also proposed which comprises at least one data memory as mentioned herein and which, as disclosed herein, is stored with geometric data, a large number of objects, in particular a common object class and/or the like, the stored geometric data being obtained using a method according to the invention became.

Object classes can have a wide variety of classifications, they can classify coarser and finer structures.

Obj ect classes can, for example, reflect anatomical structures in a grossly simplified manner, such as body parts, e.g. B. hand, arm, joint, etc., but also be subdivided more finely, for example by tissue structure, such. B. certain bones, etc. or even finer, such as cartilage, periosteum, cortical bone, spongiosa, etc.

Object classes may include or be directed to bones with fracture surfaces, optionally subdivided into classified fractures such as Tossy I through III or other classifications particularly common in bone surgery. Further distinctions (injury of the young patient versus the older patient, with or without signs of osteoporosis, etc.) can also lead to object classes or subclasses.

According to the invention, a system is also proposed which has a computer system according to the invention, a surgical robot and a control unit for controlling the surgical robot. For its part, the surgical robot has a holder for a surgical instrument, which can be arranged, for example, in or on a robot arm of the surgical robot. The control unit controls the surgical robot based on data stored in the data memory of the computer system, which was stored therein using the method according to the invention.

Embodiments according to the invention can have some, some or all of the following features in any combination, unless this is technically impossible for a person skilled in the art. Advantageous developments of the present invention are also the subject matter of the dependent claims and embodiments.

In all of the following statements, the use of the expression "may be" or "may have" etc. synonymous with "is preferably" or "has preferably" etc. to understand and is intended to explain embodiments of the invention.

Whenever numerical words are mentioned herein, the person skilled in the art understands them as indicating a numerical lower limit. Provided that this does not lead to a discernible contradiction for the person skilled in the art, the person skilled in the art always reads "at least one" or "at least one" when specifying "a" or "an". This understanding is also included in the present invention, as is the interpretation that a numerical word such as "a" can alternatively be meant as "exactly one", wherever this is technically possible for a person skilled in the art. Both are covered by the present invention and apply to all numerals used herein. Whenever spatial information, such as e.g. B. from "above",

"Bottom", "left" or "right", the person skilled in the art understands the arrangement in the figures attached here and/or in the state of use. "Bottom" is closer to the center of the earth or the lower edge of the figure than "above" .

Advantageous further developments of the present invention are the subject matter of dependent claims and embodiments.

If an embodiment is mentioned here, it represents an exemplary embodiment according to the invention.

If it is disclosed herein that the object according to the invention has one or more features in a specific embodiment, it is also disclosed herein that the object according to the invention expressly does not have this or these features in other embodiments that are also according to the invention, e.g. B. in the sense of a disclaimer. For each embodiment mentioned here, it is therefore the case that the opposite embodiment, for example formulated as a negation, is also disclosed.

If method steps are mentioned herein, the device according to the invention is configured in some embodiments to carry out one, several or all of these method steps in any combination, particularly if these are steps that can be carried out automatically, or to carry out corresponding devices which preferably by name to the designation of the respective

Based on the method step (e.g. "determine" as a method step and "device for determining" for the device, etc.) and which can also be part of the device(s) according to the invention or can be connected to it in a signal connection, to be controlled accordingly.

When reference is made herein to programmed or configured, these terms may be interchangeable in some embodiments.

If a signal or communication connection between two components is discussed here, this can be understood to mean a connection that exists during use. This can also be understood to mean that there is a preparation for such a signal connection (wired, wireless or implemented in some other way), for example by coupling the two components, for example by means of pairing, etc.

Pairing is a process that occurs in the context of computer networks to establish an initial association between computing devices for the purpose of communication. The most well-known example of this is the establishment of a Bluetooth connection, by means of which various devices (e.g. smartphone, headphones) are connected to one another. Pairing is also sometimes referred to as bonding.

The computer system has a control or regulating device. It may cause all or substantially all of the method steps to be performed, or any combination of method steps disclosed herein, in particular if these are carried out mechanically or automatically. The method according to the invention can be carried out essentially or completely by the open-loop or closed-loop control device. It can be partially carried out by the control or regulation device.

If a “display” is mentioned here, this can be a monitor, for example an autostereoscopic (AS), single-user 3D monitor (such as that sold by SeeFront GmbH, Hamburg, Germany, www.seefront.com), a display , a surgical microscope, a head-up display (HUD), for example in a visual aid or glasses, VR (Virtual Real! ty) glasses or the like.

If "geometric data" is mentioned here, it can be or include real or virtual data. Recorded data of a real 3D object (hereinafter: real object), e.g. in the form of results of imaging methods, can also This includes virtual data of a calculated virtual object. It is irrelevant whether the calculated data is based on recorded data from a real 3D object or not. "Geometric data" can be or include medical data.

If a “target course of the operation” is mentioned here, then in some embodiments this course is a course of the operation, in particular a pre-determined or inputted one, e.g Surgical history that has proven itself or is believed to work well in real-world operations, or "target surgical history" includes such history. Here, a "target operation history" have been recorded in real operations and/or in at least one simulation in which the surgeon has guided the simulation instrument or the virtual surgical instrument. In other embodiments, a "target operation history" is formed from available data from a collective of users of the present invention, who do not have to be experienced operators, but among whom there may well be such, extracted. The extracted data were preferably already during or evaluated, in particular positively, after the same has been saved The evaluation and/or the extraction can optionally have taken place using artificial intelligence (AI) or machine learning or deep learning.

If we are talking about a "sequence" here, it can consist of a sequence of motion states or, in individual cases, just one motion state. A sequence can be a data record which contains values for one or more parameters, which relating to the movement or guidance of the simulation instrument or the simulation device, includes, e.g. the position of the simulation device or its sections in space, the angle or angles of the robot arm segments of the simulation device to one another, which ultimately determines the position of the virtual surgical instrument when entering the virtual Obj ect accounts for the transition from one segment of the virtual obj ect to the next, the force that is required to arrive against the induced force feedback of the haptic or to overcome it, etc. In addition or as an alternative, forces are directed towards the are set in relation to path sections, as well as motion states, which can also be described in "sequence". The terms "virtual" and "visualized" may be interchangeable herein in some embodiments. They generally denote the computerized representation or simulated imitation of a real object or instrument on a display (see above).

In some embodiments, the haptic is not a hand-held device for manipulating/controlling a virtual robotic arm, particularly not during surgery on a patient (and therefore, e.g., not a sterile robotic arm, even in sections, not a robotic arm in a hospital operating room etc . ) .

In some embodiments, a simulation instrument, if used, is a real instrument manually operated in real space for the user's training, directly or indirectly. Its movement in real space serves as a template for guiding the virtual surgical instrument on the virtual object, i.e. on the display, e.g. B. the monitor .

In some embodiments, a real surgical instrument, in contrast, denotes an actual operation, e.g. B. a patient, real instrument used. It can later be guided by an operator at the operating table or by the operating robot.

The virtual surgical instrument is preferably a simulation of a real surgical instrument, as in z. B. inventively trained, surgical interventions will be used. The virtual surgical instrument is preferably represented by polygonal data well known in the art of computer graphics. For the haptic rendering, the virtual surgical instrument is defined by a large number of points on its surface. In some embodiments, however, only an active area of the virtual surgical instrument is correspondingly provided with points in order to save computational effort. The active area is the area of the virtual surgical instrument that comes into contact with the virtual object during the method according to the invention, ie z. B. the tip or cutting edge rather than a shaft or handle of the virtual surgical instrument.

An example of a haptic rendering was given by Zilles, C.; Salisbury, JK (1995) publishes: "A constraint-based god-object method for haptic display" (in: Proc. 1995 of IEEE/RSJ International Conference on Intelligent Robots and Systems, Human Robot Interaction and Cooperative Robots) , also by McNeely et al (McNeely WA, Puterbaugh KD, Troy JJ; "Six Degree-of-Freedom Haptic Rendering Using Voxel Sampling" in Proc, of the 26th ^annual conference on Computer graphics and interactive techniques (ACM SIGGRAPH '99) , pages 401 -408, 1999).

The haptic referred to herein may be a force feedback device. It can be structured as known from US 2015/0325147 A1. Their disclosure in this regard is hereby also made the subject matter of the present disclosure by reference.

In order to simulate the haptic force feedback of each state using a finite state machine (FSM, engl .: Finite State Machine), for a stable and real-time capable simulation of the entire drilling process can be divided into individual sub-processes. In an FSM, any complex process can be divided into a finite number of simple, logical states, which together are responsible for the behavior of the system. So-called "virtual fixtures" can then be defined for the individual states and the transitions from one state to another. Virtual fixtures are abstract sensory information that is superimposed on a remote environment (process or robot control from a safe distance) as sensory feedback The concept of virtual fixtures was first proposed by Rosenberg as an orientation method for the controlled guidance of robot controllers in a remote environment. These guidance aids are in a virtual surface as sensory information about the real work area. The user of a robot controller takes through the superimposed virtual information haptic or visual feedback Virtual Fixtures interact only with the user and not the workspace, have no mass , no physical or mechanical limitations and can be easily created or modified Performance when guiding a robot can be significantly improved. Virtual Fixtures may also describe herein virtual control mechanisms capable of constraining a user's movement along selected degrees of freedom. With the help of these restrictions, a complex task can be broken down into elementary subtasks so that it is safer and easier for the user to carry out. In this way, for example, a straight bore can be represented as a 3D displacement vector of the instrument position in the axial direction. Virtual fixtures are implemented in some embodiments using proportional differential (PD) controllers. A PD is a very fast and controllable control engineering algorithm that improves the stability of a computer controlled system. The positional deviation and the angular orientation error between a reference setpoint (proxy) and the control point (CP) of the haptic arm are calculated as well as their derivatives and with the proportional (P) and derivative (D) gains factors (gains) are multiplied. This generates the damping forces and torques that push the haptic arm to a desired setpoint until its position (the control point of the haptic arm) corresponds to the desired setpoint position. Since the force vectors act against the direction of movement of the haptic arm control point, the user cannot leave the target position of the proxy with the haptic arm. Such an implementation is provided for in some embodiments.

In some embodiments of the computer-assisted method according to the invention, the geometric data are or include medical data.

In some embodiments, the force feedback changes as the virtual surgical instrument penetrates into the virtual object.

The change can, for example, take place continuously, repeatedly, in stages, under parameter control, as a function of the associated density values or hardnesses, or in some other way. In some embodiments, data relating to sequences of movement of the simulation instrument/the simulation device and/or the real object and/or the virtual surgical instrument and/or the virtual object are stored in the respective space, for example in a space provided and/or provided for this purpose and/or suitable data storage, recorded and/or stored.

In certain forms of execution, data relative to movement sequences, ie for example the simulation instrument relative to the real object or of the virtual surgical instrument relative to the virtual obj ect, recorded and/or stored.

In particular, data relating to the angle and effort required when the virtual surgical instrument enters the virtual object, distances covered, resistance to the force feedback overcome during a movement and the like can be of interest and stored.

In some embodiments, a target course of surgery or sections for this is created and stored from a large number of recorded data relating to the sequences which simulate a virtual course of surgery on the virtual object and/or sections of a course of surgery.

In these embodiments, the creation can include or require a selection and/or a qualitative evaluation. For example, a virtual surgical history can be given the "target surgical history" attribute if it is assigned a minimum score received (e.g. in the form of scores, school grades, etc.) . For a target operation history z. B. after a

Series of training simulations, all of which may have been saved or temporarily saved, e.g. B. the best rated course of the operation can be collected, which can optionally be done automatically.

It may suffice that the target course of the operation is not an entire operation, as is the case with the instrumental operation of a patient, e.g. B. when treating a fracture, is common or necessary. Rather, the course of the target operation can optionally only include specific operation sequences or instrument deployments, particularly relevant instrument guidance, moments during the operation, etc. depict . It can be made up of a series of sequences that are each felt to be particularly worthy.

Thus, in some forms of execution is provided to create a target operation history based on a sequence or succession of partial target operation history, the z. B. were carried out or obtained on different days, in different training sessions, by different surgeons using the device according to the invention or the method according to the invention.

An evaluation as stated above can be carried out by a person in certain embodiments, e.g. B. the user, his academic teacher or a third party, alternatively or additionally, by comparing the simulated guidance of the real or virtual instrument or of the object by the user with a target operation history already stored. In order to avoid repetition, the above definition of a target Operation history referenced . By using artificial intelligence (AI) or machine learning or Deep learning can be such an assessment basis for the system according to the invention, z. B. Create or optimize target surgical procedures from stored sequences.

For the purpose of evaluation, especially as part of training, it can be provided that a surgical task be subdivided into individual operation steps. Each operation step can be evaluated after completion and e.g. B. are scored, for example as known from "Serious Games" or "Game Based Learning". The maximum number of points and the experience points for a surgical step are specified, for example, by an expert in hand surgery using an evaluated questionnaire. It can be provided that a new surgical step can only be started after the previous surgical step has ended. After carrying out all the requested operational steps, e.g. B. a final score with a maximum number of points can be calculated. Points can e.g. B. be awarded or deducted if a risk structure is violated, for arranging the virtual object (by moving the real object) into the optimal starting position for the pending intervention, for correctly setting z. B. K-wires, as examples of virtual surgical instruments, the number of attempts required for this, etc. Points can be awarded in comparison with a "gold standard", which includes e.g. the provision of a haptic corridor (see below), the orientation and the drilling depth of the K-wire, the damage to bones associated with the present Fracture not related, breaking of the K-wire due to excessive pressure from the simulation device and/or the like.

In some embodiments, a drill guide is designed as an optical extension of the virtual surgical instrument, e.g. B. a K-wire, provided, it shows the chosen and / or the optimal or future drilling channel.

In some embodiments, risk structures and/or bones can be visualized via the transparency of the skin.

In some embodiments it can be provided that a corticalis detector is provided. He can e.g. B. turn the representation of a hand bone green on the display when the K-wire tip makes contact with the cortex of a bone . In this way, the user can see which bone is currently being drilled, whether the K-wire tip is within the spongiosa, the cortical bone or the opposite cortical bone, etc.

Optional haptic corridors can also visually provide the "gold standard" for the position and location of the K-wires. The "haptic corridor" can be a simple cylinder in Cartesian space, fully defined by its radius , length , directional vector , and entry point . A haptic corridor represents a special type of "virtual fixture" and can be visualized on the display, e .g . by a cylinder . The corridor is generated independently of the drilling simulation . It can also generate additional haptic feedback . It can be anywhere on of the ad are positioned and preferably does not interact with other world objects. An exception to this may be provided: if the user moves the K-wire close to the haptic corridor, which represents the ideal drilling channel for the K-wire as determined by an expert, then the K-wire is slowly inserted via the haptic arm into this Ideal position and rotation drawn . For example, in some embodiments it can be provided that the color of the corridor changes, for example from orange to yellow, as soon as the K-wire tip dips into the haptic corridor.

In addition, it can be provided that the corridor continues to change color, e.g. B. turns green if the entire K-wire axis is also within the corridor, as the ideal position has thus been reached.

The simulated attractive force can optionally be calculated using a potential force model. Such a continuous potential force model has a first phase in which the force increases and a second phase in which the force decreases. So if the K-wire tip approaches the target point, the force of attraction increases steadily and leads the user in the direction of the ideal position until it is reached. In order to avoid vibrations around the target point, the force preferably decreases to zero with decreasing distance from the target point. Simplified and transferred to a one-dimensional (ID) problem, the attractive potential force can be calculated, for example, from a simplified Gaussian function.

In some forms of execution, a representation of the management of the virtual surgical instrument from the target operation course or the section thereof, in particular relative to a virtual obj ect on a display. A display provided and/or suitable for this purpose can be or include, for example, a monitor, an operating microscope, a head-up display, for example in a visual aid, virtual reality glasses and/or another type of projection . The presentation of the guide can be in the form of a movie, a sequence of images, an augmented or mixed reality display, etc. take place .

In some embodiments, the system is designed to visually superimpose a target operation course over a real operation scenario in real space, for example to show the operator an ideal path for the instrument.

In some embodiments, for example, levels with different levels of difficulty can be implemented here, which can or should be achieved during training, similar to e.g. B. "Serious Games" or "Game Based Learning" . For this purpose, the display can, for example, be adapted more and more to a real operation scenario. This means, for example, that suitable drilling channels are still displayed at a lower level, but perhaps not at a higher level.

In some embodiments, it can be provided that the surrounding tissue, for example skin, appears transparent, e.g. B. to reveal a bone layer and/or endangered tissue, e.g. B. Nerve pathways, display . In some embodiments, the geometric data is or includes real data, e.g. B. data on body geometry or anatomy (also briefly: patient data) of at least one real patient, which z. B. were obtained by computed tomography (CT) and / or magnetic resonance imaging (MRI). For example, the patient data may be or include values pertaining to a hand or a joint.

In some embodiments, the geometric data are or preferably include data and/or values beyond human medicine, such as veterinary medicine, material processing (in particular if chip-removing).

In some embodiments, the computer-assisted method according to the invention also has the step of acquiring the geometric data to be read in on or from a real object, e.g. B. by means of imaging methods and/or as disclosed herein. The real object can be a patient or a part of a patient's body, for example.

In certain embodiments, the step of calculating a 3D model from 2D data sets that have been read in is also included in the present invention.

An object to be surgically treated, which in the field of medicine can consist of several bones, is determined by the surfaces of its components and the spaces they enclose, which is why volume models with a large number of voxels are suitable for its mathematical description. In a volume model based on voxels, direct access to the three-dimensional voxel grid using object marks connected to the voxels can be used to determine whether an object or free space is present at any position within the volume. Density values can also indicate the hardness of the respective voxel and thus quantify the resistance that a specific voxel will offer to its manipulation with a virtual surgical instrument.

In some embodiments, the method also has the step of segmenting the geometric data, with segmented volume data being generated. This preferably takes place automatically (Automated Component Segmentation, AOS), in particular by assigning density values (or hardnesses) to the generated volume segments. The segmentation preferably also allows individual components to be edited (rotating, moving, etc.). The result of the segmentation in the case that the geometric data have or describe bony structures is preferably such that the bony structures which are present without bony contact with one another are present or separate from one another. were segmented accordingly.

The segmentation can result in groups of voxels, or in each case delimit those that differ from voxels of other or neighboring groups of voxels by the density value that the voxels of the groups under consideration have. Thus, voxels which—e.g. B. all or most of them - have a density value of a first range of density values, are combined in a common first group. voxels which - e.g. B. all or in the majority - a density value of one Having a second range of density values can be combined in a common, second group.

In some embodiments, the geometric data is or includes segmented, in particular automatically segmented, volume data.

In some embodiments, the geometric data is or includes medical volume data in DICOM (Digital Imaging and Communications in Medicine) format.

DICOM is generally recognized as an open standard for storing and exchanging information in medical image data management. This information can be, for example, digital images, additional information such as segmentation, surface definitions or image registrations. When this format is used in the system according to the invention, calculation steps for changing or adapting data formats can advantageously be omitted. The system according to the invention can thus be used with geometric data from all over the world without any adjustment effort.

Medical volume data in DICOM format represents, for example, the starting point for the model of the virtual hand used with the system according to the invention as an example of a virtual object. In this case, DICOM data are recorded in a computed tomography (CT) and/or a magnetic resonance tomography (MRT) and as a result contain a 3D data set consisting of slice images of the human hand. In some embodiments, the computer-aided method according to the invention further has the step of augmenting the read-in geometric data or read-in data sets and/or data sets based at least partially on read-in geometric data. Optionally, storing the augmented data in a data memory of the computer system according to the invention is included in the present invention.

For augmentation as a form of data expansion or enlargement of the data pool, in which the amount of data is increased, for example by adding (slightly) modified data copies of existing data or newly created synthetic data from existing data (morphology) , from the Approaches known from the literature can be used.

In some embodiments, the geometric data is or includes augmented data.

In some embodiments, the computer-assisted method according to the invention also has the step of storing a large number of target operation histories (see definition above) or sections thereof in a data memory of the computer system according to the invention. In this case, the target course of the operation or the sections thereof can be or have been obtained from a large number of recorded data relating to sequences which show the virtual course of the operation or sections.

In particular, all parameters that were recorded, such. B. angle , force etc . get saved . In addition or as an alternative, film sequences, videos or the like can be recorded, which in particular show the virtual guidance or movement of the virtual surgical tool, especially relative to the virtual object. They can serve as a support, reminder or guide for the future surgeon.

In some embodiments, the method further comprises the step of providing data that was extracted from the data memory from the target surgical procedures or sections thereof and additionally the step of transmitting this data, e.g. B. in a memory device of a surgical robot, the selection of the data preferably fulfilling at least one predetermined criterion.

The extraction can be done using artificial intelligence (AI) or machine learning or Deep learning happen or have happened.

A predetermined criterion for the selection of the data can, for example, be a previous selection from a data pool, for example by setting a "marker", for example by the user, a second person, for example a supervisor or an experienced operator or surgeon, an algorithm, etc. and /or be or include a qualitative assessment of the course of the target operation or sections thereof.

This data can be, for example, data on position, movement,

entrance angles, on applied force,

Rotational speeds or the like of a Bohrinstruments and / or the like for or during an operation. This data can change multiple times throughout the operation, e.g. B. the force can change depending on the penetration depth and / or bone density. Therefore, this data can be composite data, e.g. B. sequentially include the relevant data and parameters.

Data processing steps, for example in order to bring the data into a suitable format for use in controlling a surgical robot, for example machine data, are also covered by the present invention.

To treat a patient using the system according to the invention, after image data has been created, for example by x-raying the relevant body part such as the wrist in two planes, special recording techniques such as the Stecher recording and/or computed tomography (CT) or another imaging method can be used to determine of volume data, the image data, preferably DICOM data, are transformed into a virtual 3D model. In it, using the example of an existing fracture, the bone fragments can be automatically repositioned there with the aid of the present invention. The result can be corrected manually if necessary.

The 3D model can, e.g. B. by means of Automated Component Segmentation, ACS, z. B. as disclosed herein, all necessary spatial data (x, y, z coordinates and vector data for the volume rendering) as well as the density data (voxel density) of the bones of interest, e.g. B. every single bone of the hand and preferably the (lower) arm included. Likewise, the location of the fracture and the Fracture elements can be found visually in the 3D model in the form of x, y, z coordinates and vector data. Based on this, the repositioned spatial data (x, y, z coordinates and vector data) are calculated and optionally displayed.

In some embodiments, it is the system that, e.g. B. by means of a gripper arm, the storage or displacement of the body part to be operated on, such as the patient's arm here, on a base, preferably in a - carries out pre-calculated position inside - optimally exactly. The system can specify, e.g. B. arm and hand must be stored. The data used for this purpose is z. B. determined using the image data, preferably DICOM data, and the calculated 3D model.

In some embodiments, x, y, z coordinates of selected points (also known as markers) of the extremity or another part of the body, here the arm and the hand, are used to reposition the fracture pieces. There is preferably a constant comparison with also determined position values of the markers (likewise x, y, z coordinates) up to the precalculated, optimal position for the pending operation.

In some embodiments, the coordinates required for this purpose are recorded using suitable devices, e.g. B. by means of one or more tracking cameras of the system, which record the position of the arm and the hand, the control device, for example, as DICOM data accessible in order to arrive at the optimal, pre-calculated position. The robot of the system is preferably guided through the comparison of the coordinates and knows its position exactly. The verification is done by the system or the AI, which can be monitored by the doctor.

In some embodiments, a robotic arm determines and/or marks the location for a skin incision. It can be calculated using the previously collected data. The skin incision can be made automatically using the robotic arm.

The path line of each instrument, e.g. B. a scalpel, can be planned using a coordinate system (also using x, y and z coordinates) and corresponding vector data and optionally tracked or recorded using tracking cameras. be corrected via a target-actual comparison. The force with which the skin has to be incised depends on the condition of the skin and can be individually corrected or increased in some versions. be customizable .

In some versions, the robotic arm changes the set of instruments independently.

In some embodiments, the robotic arm thus actuates a drill with a K wire that is drilled into the bone or bones to be repaired at a precalculated point and angle of entry and with a force that is also precalculated.

In some embodiments, the robot arm has a position or Position determination sensors that determine the vector of the robot arm, the axis of the robot arm, for example, coincides with the K-wire vector. The The entry point of the drill is defined using the x, y, z coordinates of the K-wire tip and the entry point here as an example by hand.

The drilling force (in the unit [mN]) is in some

Execution forms variable and is preferably regulated in comparison with the 3D model in order to take into account the bone density (cortical bone, spongiosa).

In some embodiments, the x, y, z coordinates of the K-wire tip and the end point are continuously compared with the pre-calculated end point position in order to achieve an optimal penetration depth. The K-wire is then pre-drilled into the bone and fixed there.

A cannulated drill can be used as an additional surgical instrument and drills the hole for the screw over the K-wire, e.g. B. to the subchondral lamella le of the bone of interest .

Preferably, the vector of the robotic arm's cannulated drill matches the vector of the K-wire to ensure the correct entry angle of the drill. The calculation of the vector can be taken over by the system in advance. The correct positioning of the robot arm can also be determined using the built-in position or Position determination sensors and the tracking camera (s) are ensured.

In some embodiments, the entry point of the cannulated drill is based on the current position of the K-wire in real space. The drilling force (in the unit [mN]) is optionally set using the

Resistance measurement regulated in the robot arm. A constant comparison can be made with the data of the created 3D model, for example to take into account the existing bone density (cortical bone, spongiosa).

The penetration depth of the cannulated drill is preferably also defined via the x, y and z coordinates and the vector data of the K-wire or oriented to them.

In some embodiments, a comparison with a preferably current x-ray image can be carried out for this purpose in this step.

After another change of instrument to the cannulated screw, or when another or another robot arm is used, in some embodiments this is secured with a z. B. screwed into the bone using the 3D model or this underlying data preferably already precalculated torque.

In some embodiments, the entry angle and the entry point of the screw are determined, for example, analogously to the determination of the entry angle and entry point of the drill.

A preferably variable torque (for example in the unit [Nm]) can take place by means of a torque measurement in the robot arm. A constant comparison with the data of the created 3D model can also be carried out in order to take account of the existing bone density (cortical bone, spongiosa). The penetration depth the end position of the screw end point is in some forms of execution z. B. via its x, y and z coordinates in comparison with the data of the vector of the K-wire. When screwed in, the tip of the screw preferably reaches exactly the area of the subchondral lamella of the relevant bone. At this point, a comparison with the pre-calculated end point position can also take place.

In some embodiments, a comparison with a preferably current X-ray image can also be carried out here, and the K-wire can be removed by the robot.

In some embodiments, the method further comprises the step of recording data relating to sequences of movement of the simulation instrument/device and/or the real object in real space and/or the virtual surgical instrument and/or the virtual object in virtual Space, or sections thereof, which were carried out in the context of the method according to the invention. In these embodiments, a comparison of one or more properties of the recorded data relating to the sequence of the movement of the simulation instrument with at least one stored in the data memory provided can also be included in the target operation course or in sections thereof by the method according to the invention be .

In some embodiments, the method also includes the step of evaluating the quality of the result of the comparison using predetermined criteria and outputting a statement about the quality for the user to take note of. In some embodiments of the system according to the invention, this additionally includes a user interface for the user to input control commands for controlling the surgical robot. Such a user interface can be a joystick, a keyboard, a computer mouse, etc. or be or comprise a combination thereof.

In some embodiments of the method according to the invention, postoperative data, for example from surveys of treating physicians and/or patients, can be or have been collected and are included or have been included in the provision of target operation courses.

In some embodiments of the method according to the invention, fractures can be or have been classified, with the data of this classification being able to be included in the selection of a target course of the operation. Such a classification can be, for example, the AO classification (working group for osteosynthesis issues), which represents a system for describing the localization and nature of broken bones (fractures). The aim of this classification is a preferably worldwide unambiguous assignment of fractures, which can enable standardized use of the system according to the invention and standardized treatment.

In order to transfer the conditions of an operating room to the training system in a realistic manner, an X-ray image simulation is provided in some embodiments. For this purpose, a virtual X-ray field display is shown in the visualization. The X-ray field can optionally, for example using a 3D mouse, touch widgets or the like, enlarged, reduced, rotated or shifted.

In some embodiments, an X-ray foot switch can be provided which, when actuated, requests an X-ray image. In response to this request, the current rotation data of the virtual object (rotation between the X-ray field level and the object visualization) and the delta positions between the X-ray field and the virtual object (e.g. a virtual hand) are optionally used to convert the DICOM data set of the virtual object ects to rotate and crop to the area currently in the x-ray field. The orientation and delta position between the X-ray cross and the K-wire tip (as a virtual surgical instrument) can also be applied to the DICOM data set of the K-wire. In the next step, all pixel values of the DICOM data sets (grey values between zero and 255 of the 8-bit slice images) along the X-ray beam path (perpendicular to the X-ray field level) are summed separately for the virtual object and the K-wire. In this way, the 3D data set is reduced to a 2D image plane. These reduced 2D images can be post-processed using filters such as DiscreteGaussianlmageFilter , ResamplelmageFilter , RescalelntensityImageFilter ( linear scaling of pixel values ) and LaplacianSharpeninglmageFilter ( image sharpening by emphasizing edges ) from ITK and then summed up pixel by pixel .

In some embodiments, several K-wires can be present in the X-ray field. The K-wire data set is then used several times, but in different orientations and/or rotations. In some embodiments, the haptic arm, which is used for training, and the robotic arm, which is used for automated operations, are one and the same, or at least type-identical, "arms". Possibly of an optional simulation tool (drill, milling cutter, K-wire, etc.) dependent extent or measure free - moving as a simulation device as well as its control when he wears the surgical tool, namely according to pre-calculated trajectories or target operation courses.

The robot arm of the surgical robot, which can also be the haptic arm mentioned here, has a sufficiently sensitive, tactile sensor system. It allows e.g. B. Detect, measure and render external forces and/or resistances due to contacts.

In some embodiments, the robotic arm has one or more end effects for receiving relevant surgical operating instruments (such as drills and milling cutters). Optionally, other relevant sensors (such as force sensors) are/are also integrated into the end effector(s).

The automatic component segmentation can e.g. B. using a machine learning model (herein also called KI model or model for short) are executed.

A machine learning model is a calibratable method to produce an output from input values. In a training phase, the model is provided with a number of sample data sets and a function that evaluates the "quality" of the outputs. The training serves to calibrate the model in such a way that the outputs that the model outputs for the training data are as optimal as possible. A supervised learning approach can be used here. A data record of manually segmented data can optionally be made available. Correct segmentations are operated manually. The system then has the opportunity to learn correct segmentation from this. The same can be done for classification and regression tasks.

The model optimized in this way, or even optimal, produces outputs that are sufficiently close to the correct assignments.

However , unsupervised learning or reinforcement learning are also well - known approaches for arriving at a suitable model .

To ensure the required quality of the model ls, a test or validation phase is preferably added after the training has taken place. Here, the trained model can be applied to new, previously unknown data and the quality of the outputs checked.

In the present case, a sufficient number of data sets are provided as a data collection, e.g. B. in the form of patient images showing the most well-known damage z. B. of the hand, but also show intact hands. A few dozen patient cases can suffice here; be augmented . Assignments are made manually in the individual images. If the computer-aided method according to the invention herein for damage z. B. applied to the hand, this is purely exemplary and not to be understood as limiting. The application of the computer-assisted method according to the invention on other parts of a patient's body, e.g. B. hip , head , knee , foot etc . , is also encompassed by the present invention .

In these training images, a region of interest (ROI) can be manually defined, which may can encompass the entire picture. The model to be trained can also define an ROI, but this is not absolutely necessary. If a ROI is defined, it can be defined automatically, e.g. B. the ROI is only expanded to such an extent that the average density value of the ROI does not fall below a minimum value. In this way z. B. a black border are disregarded.

Overlays of bones and spatial relationships of adjacent bones or bones or bone fragments lying one on top of the other in the projection direction can be recognized in these training images as in later patient images by contrast differences and/or different white values. By comparing bone fragments, bone parts and/or bone fragments, some of which can be strongly displaced in the area of existing fractures, for example with an intact bone situation, the model can then be based on the structure of the fracture surfaces, the still given spatial proximity of fragments (also after fracture) and in view of some intact neighboring bones, assign associated fragments or fracture surfaces to each other, comparable to a puzzle. The assignment of individual fragments to each other can be modeled with knowledge of average, maximum or minimum thicknesses and

Lengths of the bones involved in the structure of the hand, if necessary . in relation to each other, take place. In some embodiments, this assignment can also be made in connection with another body measurement, such as the length of a forearm bone. Identical thicknesses of the cortical bone in the respective marginal area are also suitable for making an assignment of bone fragments or fracture surfaces that belong to one another or for checking an assumed assignment. This is particularly the case when there are recordings, e.g. B. the hand under different beam paths ( two planes are usually sufficient here ) clearly possible .

If one assumes that undamaged bones are marked by continuous borders in the image, then the individual, undamaged bone can be identified in the image. Neighboring bones are separated from one another by the continuous omission of a continuous cortical bone (or border or frontal delimitation of the bone in the picture) .

Density differences between the cortical bone and spongiosa can also be distinguished using the existing white values in the image. To recognize such density differences, even with superimposed bone structures in the image, can have been communicated to the model by means of the training data. Taking them into account can further serve to identify the peripheral contour of each bone and/or bone fragment. With the open source software ImageJ, for example, the bone structures or bone segments identified in this way can be extracted into individual image stacks in order to be able to process them separately.

A division of the respective bone into the structures cortical and spongiosa can take place, this is purely optional.

The segmentation of the spongiosa can be removed. The resulting hollow interior can be understood as the inner boundary of the cortex, which can be smoothed out if desired.

using e.g. B. the ITK Filter BinaryFillholelmageFil ter and BinaryErodelmageFilter automatically postprocesses the segmentation of the cortical bone.

Using Meshlab, the surfaces extracted from the raw data can then be optionally post-processed (topological remeshing, exact geometry, smoothing). You can then e.g. B. be saved as STL models .

After aligning the surfaces according to the training images, the voxel values are optionally calculated by resampling into a dataset that can be used to extract the density values (for the haptic feedback).

An automatic density assignment for individual voxels based on the completed segmentation or highlighting of the cortical bone can then follow. The data sets, which are used as training data for the AI models presented here, can additionally map an expected data variability as well as possible. This means that the selection of the data records follows the patient statistics (e.g. age, gender, location of the fracture). For this purpose, annotation protocols are developed and necessary adjustments are implemented in the software. These primarily serve to improve cooperation with the medical experts and the necessary, specialized visualization and interaction for the segmented fracture elements.

Some or all of the embodiments according to the invention can have one, several or all of the advantages mentioned above and/or below.

According to the invention, it may be possible to prepare surgeons for performing a real operation by being able to improve their technical skills and their anatomical knowledge in relation to the specific case or the specific disease or accident pattern or in general by training on the system according to the invention. Individual surgical steps can be internalized and subsequently transferred to the upcoming surgical situation(s).

Surgical robots can always offer the same precision and quality in surgery, also because, unlike humans, they do not tire. A loss of concentration on the part of the (mechanical) surgeon does not have to be expected, even in the case of longer operations. The quality of the surgical intervention can therefore be kept consistently high over the period of use of the surgical robot become . An advantage of the present invention can therefore consist in providing surgical robots with surgical sequences from a large number of surgical interventions, simulations by experienced operators and/or surgeons and/or training sequences rated as very good, in which a real surgical instrument guided by the surgical robot is used for surgical care To guide a patient is to provide an optimal sequence of movements. This advantageously contributes to increasing the quality of the operation and patient safety.

A further advantage of the present invention can be that the present inventive system can fall back on hundreds, thousands and more data sets of the most varied surgical interventions and can learn from them, while a surgeon is limited to his own experience, which is limited in scope, if necessary. supplemented by the experience of academic teachers such as experienced surgeons, course leaders, a senior physician, etc. , must leave . The system according to the invention can thus include significantly more information in the work process, which can lead to significantly more knowledge and experience in a short time than the specific surgeon could possibly ever gather. This can advantageously lead to more precise surgical results and thus significantly increase patient safety.

Surgical robots can advantageously be trained by means of the present invention in order to be used worldwide, in particular in countries with poor medical equipment or infrastructure, and to be able to take over a wide variety of processes. Advantageously, the surgical robots can also needs to be controlled by experts over large distances.

In particular, this can help to save costs, for example for highly specialized experts on site

A further advantage of the present invention can consist in the fact that operation times and times for their preparation on the part of the human surgeon are significantly reduced. This can also indirectly contribute to patient safety.

By means of the present invention, utilization of the operating theater can advantageously be significantly optimized due to the better ability to plan operations. This can indirectly increase the efficiency of the clinic or practice in which the invention is used and help to save costs.

Another advantage of the present invention can be that operations can be carried out around the clock. Due to the omission of the "human factor" in some embodiments, the use of the present invention is not subject to any restrictions that can occur in humans, such as tiredness and declining ability to concentrate.

According to the invention, fractures that are actually present can be “trained” before they are surgically treated, discussed in advance with the surgical team and, as a result of ideal preparation (repeated practice on the exact image of the surgical case), the real operation can be carried out with a significantly lower probability of complications. Improved surgical preparation leads to lower risk for the patient and can additionally provide the required

Reduce operations time, which is also a cost issue.

All the advantages that can be achieved with the method according to the invention can also be achieved undiminished in certain embodiments according to the invention with the devices according to the invention, and vice versa.

The present invention is described below purely by way of example with reference to the attached figures. In them, the same reference symbols designate the same or similar components. The following applies:

Fig. 1 shows a schematically simplified sequence of the computer-aided method according to the invention in an exemplary embodiment;

Fig. 2 shows, in a schematically simplified manner, devices for carrying out the acquisition step of geometric data by means of a computer tomograph during an exemplary embodiment of the method according to the invention;

Fig. 3 shows schematically simplified devices for carrying out the method according to the invention in a first exemplary arrangement;

Fig. 3a shows schematically simplified devices for carrying out the method according to the invention in a further exemplary arrangement; 3b shows schematically simplified devices for carrying out the method according to the invention in a third exemplary arrangement;

4 shows a schematically simplified exemplary embodiment of a system according to the invention with a computer system according to the invention, a surgical robot, a control unit and a user interface; and

FIG. 5 shows an exemplary scheme for the course of a surgical intervention using the method according to the invention or using the system according to the invention.

1 shows a schematically simplified sequence of a computer-aided method according to the invention in an exemplary embodiment.

In Fig. 1 method steps are shown. Units and devices which can be used to carry out the method according to the invention are described in more detail in FIGS. 2 to 4 . Reference is made here to their reference numerals used therein.

In Fig. 1, method step S1 represents reading geometric data of an object into a computing unit 101.

Here, the term "object" can, for example,

Patient P or a single part of his body, such as. B. arm, hand, calf, foot, fracture, ligaments, etc., be or include these(s). The reading in of data from a medical or non-medical real object, for example a dummy, is also covered by the present invention. The geometric data can be or include medical data.

In an optional step Ssrf, the geometric data of the object to be read in can be recorded or have been recorded on a real object, for example on a patient P, for example by means of a computer tomograph (see FIG. 2) and are therefore already available.

In addition, the recorded data or data sets can be segmented, in particular automatically, in one or more segmentation step(s) Ss _eg in order to obtain segmented volume data, in particular in preparation for step S2 (see there).

A segmentation z. B. the fracture to be treated acutely, but also of risk structures, can be done using machine learning or deep learning. For this purpose, MRT and/or CT data can be segmented semi-automatically to automatically and loaded into the training system.

In addition, the recorded data or data sets can be or have been augmented in one or more augmentation steps SAU ₉ in order to increase the amount of data or the number of data sets.

Steps Sseg and SAU ₉ can be carried out either before reading in step S1 with the acquired data or after reading in step S1 with the read data. This is shown in Fig. 1 by arrows. The means that in some embodiments the data that has been read in can already be available in advance in a segmented and/or augmented form.

In some embodiments, the geometric data of the object to be read in can already be present as medical volume data in DICOM (Digital Imaging and Communication in Medicine) format or can include data that is present in this format. In other embodiments, the geometric data to be read in of the object z. B. two-dimensional data, e.g. B. in the form of image data (MRI, CT, etc.). These can be used herein to arrive at 3D models, which can be done automatically using software.

Creating the 3D model can include virtual repositioning of fracture segments, dislocated bones or parts, etc. be or include, which may be done automatically, such as described herein for automatic segmentation.

Provision can be made for recognizing or determining individual bones or fracture segments as separate objects, which can in particular take place automatically, for example as described here for automatic segmentation.

The bones are automatically segmented as a 3D or voxel model, preferably after the region of interest (ROI, can also be the entire image) has been roughly defined. The aim is to separate individual bones or

Recognizing fracture fragments in order to then be able to move them separately in space, similar to a jigsaw puzzle. The main challenges are that bone parts and fragments in fractures vary greatly in shape and size and are also partially displaced can . Furthermore, in the case of fragments, the cortical bone is naturally broken up and the contrast between spongiosa and the surrounding tissue is reduced. It is important to recognize individual bones (whose segmentation mask is directly adjacent to the segmentation mask of other bone parts) as separate objects, ie to separate adjacent bones and/or bone fragments. In order to obtain a good data basis for the segmentation of the individual structures in the images, these are segmented manually by physicians prior to the automation. Annotation software can be used for this. Training the AI tools will enable them to automatically segment the bones and other structures of interest, e.g. B. can be realized through the use of neural networks. The neural segmentation networks can be adapted according to the existing images and problems, especially in a training stage for the AI.

In the system according to the invention, an AI trained in this way can then support physicians as an AI assistance system in planning the reduction of fractures using forecasts based on the images of the fracture, which is provided in some embodiments. The system can automatically make reduction suggestions based on bone or fracture bodies manually segmented by physicians.

The reading in or creation of geometric data as explained herein can thus allow the use of the system according to the invention as a surgical training and planning system which can be used for selected pathologies, since in some embodiments it is based on real or actual Fracture images of a specific patient who may not yet have been surgically treated can generate a simulation corresponding to his fracture images. This simulation can then be used to test the real, pending intervention in advance by means of a simulation. Thus, an operation planning is possible through the combination of haptic and visual feedback on the basis of the individual concrete fracture. The patient can be "pre-operated" in a simulation environment, which in the case of complex interventions makes a significant contribution to the safety of the surgeon and the patient. In addition, the simulated case can be made available to other users after the simulation or subsequent operation on the patient be made available as a training simulation Step S2 represents an assignment of density values or hardnesses to volume sections or to the segmented volume data of the object.

Density values or hardnesses include, for example, values for the density of skin, subcutaneous tissue, muscle tissue, etc. or the hardness of bones, bone components (cortical, cancellous, etc.) etc. These can vary from case to case (individually). For example, an osteoporotic bone has different hardness values than a healthy bone, at least in sections, and the cortical and spongiosa also differ from one another in this respect. In the non-medical as well as in the medical field, these values can be density values or hardening of materials, for example implants (ceramic, metal, etc.) or include such values.

For bones, the density values z . B. from the white values in the X-ray images, i.e. the CT data. je "Whiter" , the denser . The position of soft tissue is obtained, for example, via data from the MRT images .

In method step S3, the object is visualized as a virtual object 110 by means of the computing unit 101 using the data read in in a virtual, three-dimensional (3D) space, for example on a monitor or another display 105 (as described above).

The optional step S4 represents providing a real 3D object 190 . This real 3D object 190 can optionally be based on the previously read geometric data, but it does not have to be. It is preferably provided and/or suitable to simulate the real object on which the geometric data is based in the virtual 3D space that is visualized on the display 105 . Gross simplifications, such as simple geometric bodies such as a cylinder, a tube, etc. can in some

Embodiments can also optionally be used as a real object.

A simulation device 180 with a haptic 185 is provided for a user U in method step S5. Alternatively, this is already provided. The simulation device 180 can include a simulation tool 187 for the user, but this is purely optional.

The simulation device 180 can be or include a robotic arm or a haptic arm with a haptic for a haptic force feedback. Examples of such haptic arms are z. B. as "Virtuoso 6D Desktop" from the company .

Haption GmbH of Aachen, Germany. The haptics 185 can preferably be operated intuitively for movements of the simulation device 180 or the simulation instrument 187, e.g. B. manual or bi-manual . In particular, it can be or comprise a device for haptic force feedback.

The haptics 185 are preferably provided and/or suitable, based on the density values or hardnesses assigned to volume sections of the virtual object 110 in step S2, to simulate resistances and/or counteracting forces that arise when operating a real instrument under real conditions, by the user during the would be haptically perceptible if the operation of the patient P, which is simulated here, would have to be observed and/or if necessary. would have to be overcome (see also the following steps S 10 and S i l).

The optional step S6 represents providing a tracking camera, in particular an optical real-time tracking camera 195 .

Tracking cameras, such as the MicronTracker from ClaroNav, Ontario, Canada, www. claronav . com, are known to those skilled in the art.

In step S7, the movements of the simulation device 180 or the simulation instrument 187 are tracked by means of sensors, such as position sensors, position sensors, optical sensors and the like, and by means of the computing unit 101. For this purpose, the computing unit 101 can use the data from the tracking camera 195 supplied data operate (see Fig. 3b.), On data from sensors, which in the hand section 183 or in certain sections of the haptic arm, for example in the arm segments 181 and/or in the joints 182, are arranged (see FIG. 3, FIG. 3a), fall back, or the like.

In step S8 of the method according to the invention, the hand section 183 and thus the simulation device 180 or the optional simulation tool 187 guided by a user in real space.

This guidance can be used for training purposes and/or for the preparation and/or planning of an upcoming operation to be performed on the patient P using a real instrument corresponding to the simulation instrument 187 .

In step S9, a virtual surgical instrument 187' is visualized in virtual, three-dimensional (3D) space, for example on a monitor or another display (105).

A virtual surgical instrument 187' is preferably provided and/or suitable for simulating and/or visualizing the optional simulation instrument 187, and in particular its movements in the virtual 3D space visualized on the display 105, regardless of whether such a real simulation instrument 187 is provided for the purpose of realistic training situation or z. B. is not intended to avoid the risk of injury or is replaced by a dummy.

Step S10 represents a determination of a collision between the virtual surgical instrument 187' in the virtual 3D space on the one hand and the virtual object 110 and/or volume sections of the virtual object 110 in the virtual space on the other hand. In step Sil, a force feedback, e.g. B. determined for the duration of the collision taking into account the density values or hardnesses of the volume sections. The determined force feedback is then transmitted in step S 12 , preferably continuously or permanently, to the simulation instrument 187 or the haptics 185 of the simulation device 180 .

The haptic 185 is preferably controlled to urge the hand portion 183 to a calculated position by applying an appropriate restoring force to the force feedback device held by the user.

The force feedback felt by the user U when guiding the simulation instrument 187 can change with the virtual penetration of the virtual surgical instrument 187 ′, which simulates the simulation instrument 187 in virtual space, into the virtual object 110 . The change can, for example, take place continuously, repeatedly, in stages, under parameter control, as a function of the associated density values or hardness levels and/or in some other way.

The method steps S 1 to S 12 of the method according to the invention that are carried out here can optionally be carried out in the above-mentioned order, with a time overlap or simultaneously.

In particular, steps S 10 to S 12 can be performed several times and/or in loops in some embodiments, which can be seen in FIG. 1 is represented by a recursion arrow. In some embodiments, it can be provided that sequences of the movement of the virtual surgical instrument 187' and/or the virtual object 110, for example in a data memory 103 provided and/or provided and/or suitable for this purpose (see Fig. 3a and Fig. 3b ) to record . In particular, data relating to the angle and effort required when the virtual surgical instrument 187 ′ of the virtual 3D space enters the virtual object 110 , distances covered, resistances of the force feedback overcome during a movement of the virtual surgical instrument 187 ′ and the like can be of interest and get saved .

In method step S13, a target course of surgery or sections for this is created from a large number of recorded sequences, which simulate a virtual course of surgery on virtual object 110 or sections of the course of surgery thereon, or defined and stored.

Thus, in step S14, a representation of the guidance of the virtual surgical instrument 187' from the target course of the operation or the section thereof, in particular relative to an operation object, can be displayed on a monitor or a display 105. This can be done in advance, in parallel/overlapping and/or after a currently running simulation. The representation can be or include a film or a sequence of (individual) images; it can be implemented using augmented reality or something comparable. In method step S15, a multiplicity of target operation courses or sections thereof, which are from a multiplicity of recorded sequences which show the virtual operation course or sections thereof, are stored in the data memory 103. A jump to step S 18 is possible here, as explained below.

In step S16 of the method according to the invention, sequences of the movement in relation to space and time of the virtual surgical instrument 187' and/or the virtual object 110 in virtual space (or also the movement of these relative to one another), the simulation instrument 187 and/or or a real object 190 in real space (or also the relative movement of these to each other) or in each case of sections for this. In addition, one or more properties of the recorded movement sequences of the simulation instrument 187 and/or the virtual surgical instrument 187' are compared with at least one target surgical procedure or sections thereof stored in the data memory 103 provided. To avoid repetition, reference is made to the above definition of a target operation history.

In method step S17, the quality of the comparison is evaluated using predetermined criteria. This can be done, for example, by means of a point score, a comparison with the performance of other users, the awarding of school grades, a level increase or the like.

Optionally, a statement about the quality

acknowledgment to the user or third parties. Such a statement can be made optically, acoustically and/or mechanically or haptically. It can optionally take place on an external device, for example a smartphone. Optionally, the statements can be stored over several comparisons over time, for example in the data memory 103 , in particular in order to recognize and/or store improvements in the quality of the guidance of the simulation instrument 187 .

In step S18, data can advantageously be extracted from the large number of target operation courses or sections thereof stored in step S15, for example by means of artificial intelligence or according to human criteria. This extracted data can be provided in a suitable data format and/or as machine data and transmitted to a storage device 203 of a surgical robot 200 in order to be read out by the surgical robot 200 and/or used to control it.

The selection of the data for extraction and provision fulfills at least one predetermined criterion. In particular, this is data that corresponds to a target course of the operation and therefore in particular ensures a good quality of the data for an upcoming (real) course of the operation using the operation robot 200 . It can be irrelevant whether this target course of the operation was entered by an experienced surgeon, for example as part of an operation planning, or was extracted from training data.

The above exemplary embodiment of the method according to the invention comprises a series of steps which are of an optional nature and/or are already Further developments of the method according to the invention, as defined in claim 1, relate.

2 shows schematically simplified the devices for carrying out the acquisition step Ssrf (see FIG. 1) of geometric data, here data of a patient P and for example by means of a computer tomograph 50, during an exemplary embodiment of the method according to the invention. This corresponds to the acquisition of geometric data on a real object, which in other embodiments can refer to the reading in of data that has already been collected.

The devices for carrying out steps S1 to S3 are also shown in a schematically simplified manner. The data recorded here, for example, by means of the computer tomograph 50 are read into a computing unit 101 (compare step SI in _FIG . SAU ₉ and step S2 in Fig. 1). The data, their segmentation and the allocation of density values or hardnesses can be stored in the data memory 103.

Finally, a virtual object 110, which is based on the data of the real object (in Fig. 2 this is the patient P as an example), is displayed in a virtual, three-dimensional (3D) space on a monitor 105 by means of the computing unit 101 using the data that has been read in represented or visualized. Additional image processing steps required for this , such as e.g. B. Smoothing, filtering or the like are also included in the method according to the invention.

Fig. 3 shows schematically simplified devices for carrying out the method according to the invention in an exemplary arrangement. Fig. 3 shows a computer system 100 according to the invention in an exemplary embodiment with a simulation instrument 187 (compare step S5 of FIG. 1).

Reference is made to the description of FIG. 1 and fig. 2 referenced.

The simulation instrument 187 is part of a simulation device 180 and is preferably simulated by a virtual surgical instrument 187' in the virtual space that is visualized on the display 105.

In the example of FIG. 3, the simulation device 180 further includes a haptic arm, which in turn can be or include a haptic 185, for example a device for haptic force feedback (compare step S 8 in FIG. 1).

Furthermore, in the example of FIG. 3 the simulation device 180 has a hand section 183 for manual guidance of the simulation instrument 187 by the user U, whose hand is indicated schematically.

Gyroscopic devices, for example in the form of position sensors, can be provided within the hand section 183 and/or in the haptic arm in order to

Coordinates (x, y, z ) of a location of the tip of the To be able to detect the simulation instrument 187 in space as precisely as possible and to be able to use these to calculate virtual coordinates (x', y', z') for the tip of the virtual surgical instrument 187' in virtual space. The movement of the real object 190 in space is followed (“tracked”) and simulated in virtual space by tracking the virtual surgical instrument 187′ (compare step S7 in FIG. 1).

3a shows schematically simplified devices for carrying out the method according to the invention in a further exemplary arrangement.

Reference is made to the description of FIGS. 1 and 3 .

Fig. 3a shows the computer system 100 according to the invention of Fig. 3, again with the simulation device 180 with the simulation instrument 187.

In the example in FIG. 3a, a real object 190 can also be seen, which has the shape of a cylinder purely by way of example and not as a limitation (see S4 of FIG. 1). The real object 190 can serve as a means for the user to indirectly move the virtual object, such as a hand, a foot, and so on. If the user moves the real object 190, he can correspondingly move the virtual object 190 representing a hand, a foot, etc. in the monitor view.

The real object 190 can optionally be based on the previously read geometric data. It can for example be a dummy z. B using 3D printing based on patient data was created, which were recorded by means of the computer tomograph 50 (see FIG. 2).

The real object 190 is preferably provided and/or suitable for simulating the real object on which the geometric data is based in the virtual space that is visualized on the display 105, with geometric similarity or even identity between the real object 190 and the associated virtual object 110 does not arrive in virtual space. In order to track the movements of the real object 190 with the virtual object 110, the real object can have sensors, in particular gyroscopic devices, and/or optically detectable structures, for example markers. The real world object 190 may also optionally be tracked as described herein or in other ways. Its movements in real space can be simulated using the representation and movement of the virtual object 110 .

3b shows schematically simplified devices for carrying out the method according to the invention in a third exemplary arrangement.

Reference is made to the description of FIG. 3a 3 .

Fig. 3b shows the arrangement of Fig. 3a supplemented by a

Tracking camera 195 (compare step S6 in FIG. 1).

The purely optional tracking camera follows ("tracks") the movements of the simulation instrument 187, in particular its tip, by means of optical sensors and the computing unit 101. For this purpose, the computing unit 101 uses the data from the tracking camera to calculate the coordinates (x, y, z) one To be able to detect the position of the tip of the simulation instrument 187 in space as precisely as possible and to be able to use this to calculate virtual coordinates (x′, y′, z′) for the tip of the virtual surgical instrument 187′ in virtual space. The movement of the real object 190 in space is optionally also tracked and simulated in virtual space (compare step S7 in FIG. 1).

With the arrangements of FIG. 3 to fig. 3b, in particular, sequences of the movement of the simulation instrument 187 and/or the virtual 3D object 110 in space or sections thereof can be recorded. In addition, in some embodiments, one or more properties of the recorded movement of the simulation instrument 187 can be compared with at least one target operation history (or sections thereof) stored in the provided data memory 103 (compare step S 16 of FIG. 1). The tracking, in particular of the simulation instrument 187 and/or the virtual 3D object 110, can take place in an alternative way, for example by recording the position of the simulation device 185 or its sections in space, the angle(s) of the robot arm segments of the simulation device 180 relative to one another, as already stated herein.

Optionally, the real object 190 is also not tracked with a tracking camera, but is also provided with an arm whose position in space (angle, attitude, position) etc. is recorded and evaluated accordingly.

As already explained above, the quality of the comparison can be evaluated and, in particular, via feedback be used by the user for training purposes by trying to bring the management of the simulation instrument 187 in line with a stored target operation history.

Alternatively or additionally, an upcoming operation could be planned in advance with the best possible steps and the corresponding data extracted, made available and transmitted to a storage device 203 of a surgical robot 200, which includes a suitable data format and/or machine data to be read out by the surgical robot and/or to be used for its control (see FIG. 4).

As already explained elsewhere, the above all for FIGS. The haptic arm described in FIGS. 3, 3a and 3b already correspond to that robot arm which, in some embodiments as a robot arm of the surgical robot 200, will optionally perform the operation on the patient. Instead of speaking of a haptic arm on the one hand and a robot arm on the other hand, in some embodiments it is therefore possible to speak of a force feedback/tool control arm in a simplified manner.

Fig. 4 shows a schematically simplified exemplary embodiment of a system 1 according to the invention with a computer system 100 according to the invention, a surgical robot 200, a control unit 150 and an optional user interface 170.

The computer system 100 according to the invention has at least one computing unit 101 and at least one data memory 103 as described herein. Data store 103 is , as herein revealed , stored , i . H . it stores geometric data, a large number of objects, in particular a common object class as described herein, at least one target operation course, durations of previous, comparable operations, evaluation modalities or the like, which were obtained by means of an embodiment of the method according to the invention .

The surgical robot 200 has a robot arm with robot arm segments 201 and joints 202 . Furthermore, the surgical robot 200 has a receptacle 205 for a real surgical instrument 207 , preferably at a free end of the robot arm. The real operating instrument 207 can be a drill or a milling cutter, for example, or can include such a tool.

The system 1 according to the invention also includes a control unit 150 which is configured or programmed to control the surgical robot 200 based on data stored in the data memory 103 or based on data stored in the data memory 203 of the surgical robot 200 .

In the example of FIG. 4, the system 1 optionally also includes a user interface 170, which is provided and/or suitable for a surgeon to enter control commands for controlling the surgical robot 200.

The surgical robot 200 can thus, supported for example by the use of artificial intelligence (AI) or machine learning or Deep learning, as described herein, based on semi-autonomous or autonomous operations perform on the data that has been transferred to its storage device 203 . See also Fig. 5 .

Although there is talk here, and especially with reference to the figures, of a haptic arm on the one hand and a robot arm on the other hand, this should not be understood as limiting. As already explained above, in some embodiments only one robot arm is used for all the applications mentioned here.

Fig. 5 shows an exemplary scheme for the course of an operative intervention using the method or method according to the invention. using the system 1 according to the invention.

Reference is made to the reference numbers of the preceding figures.

An operative scaphoid fracture treatment, as will be possible in the future using the present invention, serves as an exemplary starting scenario. The scenario is divided into individual steps 01 to 015.

In this case, 01 represents the presentation of a patient, for example in the emergency room of a clinic, who is complaining of severe pain in, for example, the right hand. The diagnosis of the emergency room physician after the creation of image data, for example by means of an X-ray of the wrist in two planes, a Stecher image and/or a computed tomography (CT) or another imaging method for determining volume data, results in the present example as a scaphoid fracture (fracture of the os scaphoideum , also : scaphoid) . 02 stands for the transformation of the image data available from 01, preferably DICOM data, using the present invention into a virtual 3D model in which the bone fragments are automatically repositioned using the present invention. A responsible surgeon optionally checks the repositioning carried out in seconds in the 3D model and, if necessary, corrects it manually using the present invention.

The 3D model, e.g. B. by means of Automated Component Segmentation, ACS, z. B. as disclosed herein, all necessary spatial data (x, y, z coordinates and vector data for the volume rendering) as well as the density data (voxel density) of the bones of interest, e.g. B. every single bone of the hand and preferably the (lower) arm included. Likewise, the position of the fracture and the fracture elements can be found visually in the form of x, y, z coordinates and vector data in the 3D model. Based on this, the relocated spatial data (x, y, z coordinates and vector data) are calculated and optionally displayed on a monitor/display 105 .

After the diagnosis, which in turn can be made by the system or manually, the patient is prepared for the subsequent operation by a surgical team, and his right arm is placed on an arm table or an arm bench of the 0P table. After anesthesia, the surgical team pre-positions the arm and positions the robot and the X-ray device, for example a C-arm, with the option of digital volume tomography (DVT). This is shown in Fig. 5 by 03. In the following, the further operation process can run completely autonomously. However, the responsible surgeon preferably monitors every step and can therefore intervene to correct it at any time. However, not every step that can run automatically has to be taken over by the system. The invention also provides for any such steps to be carried out manually, despite the fact that they can be automated. In such cases, the invention can be limited to the automated steps. In practice, they can follow one another directly, alternatively manual intermediate steps can be provided or the direct sequence of automatic steps can be interrupted.

04 represents the displacement of the patient's arm by means of an autonomous gripping farm on a corresponding base, here with the dorsal side down, based on data that were previously obtained in step 02, preferably in an - optimally exactly - pre-calculated position. After evaluating the data that is now available, the system or its AI preferably determines how the arm and hand must be positioned. For example, the wrist is hyperextended over a pad that serves as a hypermochlion, so that the two fracture elements of the scaphoid are actually reduced by approaching them. The data used for this was determined in step 02 using the image data, preferably DICOM data, and the calculated 3D model. X, y, z coordinates of selected points (also known as markers) of the extremity or another part of the body, here the arm and the hand, can be used to reposition step 04, for example those features that are always the same for each arm and each hand, which are individually unique (e.g., fingertips, thumb, etc.). It is preferably done a constant comparison with the position values of the markers determined in step 02 (also x, y, z coordinates) up to the pre-calculated, optimal position for the operation.

The coordinates required for this purpose are recorded in this and the following steps using suitable devices, e.g. B. by means of one or more tracking cameras, which record the position of the arm and the hand, make it accessible to the control device 150, for example as DICOM data, in order to find the optimal position according to step 02.

In this way, the positions of the robot, arm and hand of the patient, the C-arm and/or the like are also checked, which is shown in FIG. 5 is represented by 05 . The robot is preferably guided by the comparison of the coordinates and knows its position exactly. The verification is done by the system or the AI, which can be monitored by the doctor.

06 represents the optional creation of an X-ray image in two planes (or a DVT) as well as the transmission of the captured image data to the control unit 150, which is programmed to control and/or regulate the entire process. Based on this data, the control unit 150 checks the correct position of the hand and the bones or bones before the actual drilling. their markers.

Determined in 07 and possibly . marks an optional - and in addition to the autonomous gripping arm second - autonomous robotic arm 200 the position for a skin incision. It can be calculated using the data from step O2. The The surgeon has the option of checking the correctness of the determination or the marking or checking for plausibility, for example using a screen or a console. Then, optionally after confirmation by the surgeon, the skin incision is made automatically by means of the robotic arm 200 .

The path line of the scalpel can be planned using a coordinate system (also using x, y and z coordinates) and corresponding vector data and tracked or recorded using tracking cameras 195 . be corrected via a target-actual comparison. The force with which the skin has to be incised depends on the condition of the skin and can be individually corrected or increased in some versions. be customizable . It moves in the millinewton [mN] range and is applied by means of a resistance force measurement in the robot arm 200 .

The set of instruments is changed in 08 on the autonomous robotic arm 200, the robotic arm 200 now simulates a drill with a K wire as a real operating instrument 207. The K-wire is placed in a z . B. drilled into the scaphoid at step O2 , predicted point and entry angle , and with a force also predicted at step O2 .

In this case, using the robot arm 200 installed position or. Position determination sensors determine the vector of the robot arm 200, with the axis of the robot arm 200 matching, for example, with the K-wire vector. The drilling entry point is defined via the x, y, z coordinates of the K-wire tip and the entry point on the hand. To determine the data required for this, you can also refer to 02 be resorted to . With the help of tracking cameras that track the position of the hand, this process may be corrected e.g. B if the hand was moved.

The drilling force (in the unit [mN]) is optionally variable and is preferably regulated in comparison with the 3D model created in step O2 in order to take account of the bone density (cortical bone, spongiosa).

The x, y, z coordinates of the K-wire tip and the end point are continuously compared with the pre-calculated end point position in order to achieve an optimal penetration depth. For this purpose, in addition to the tracking cameras 195 and the position or Position sensors also include detection by means of imaging, e.g. B. by means of fluoroscopy or by means of the X-ray machine.

09 represents a radiological control and confirmation by the surgeon. The K-wire is then pre-drilled up to the radius and fixed there.

The same robotic arm 200 changes autonomously in O10 to a cannulated drill as a further surgical instrument 207 and drills the hole for the screw using the K-wire, exactly down to the subchondral lamella of the scaphoid bone. Alternatively, a robot arm other than that designated 200 can be used for this purpose.

At this point, the vector of the cannulated drill of the robotic arm 200 preferably matches the vector of the K-wire to ensure the correct entry angle of the drill. The vector can be calculated in advance by the system, e.g. B. in O2 . The Correct positioning of the robot arm 200 can also be done by means of the position or Position determination sensors and the tracking camera (s) are ensured.

The entry point of the cannulated drill is based on the current position of the K-wire in real space, for example. Again, the tracking camera(s) 195 can be used for this.

The drilling force (in the unit [mN]) is optionally regulated by means of the resistance force measurement in the robot arm 200. A constant comparison with the data of the 3D model created in step O2 can be carried out in order to take account of the existing bone density (cortical bone, spongiosa), for example.

The penetration depth of the cannulated drill is preferably also defined via the x, y and z coordinates and the vector data of the K-wire, with the drill not being advanced to the tip of the K-wire, but beforehand, preferably at the subchondral lamella of the os scaphoideum, is stopped in its longitudinal movement. At this point, a comparison with the pre-calculated end point position can also take place. As explained, this can preferably lie as precisely as possible on the subchondral lamella of the scaphoid bone. For this purpose, the tracking camera(s) 195 can be used, alternatively or additionally, the position or position determination sensors . In some embodiments, a comparison with a preferably current X-ray image can be carried out for this purpose in this step.

Oll represents the pick-up of the cannulated screw by the robotic arm 200, the length of which z. B. was already calculated in step 02 (see above) or was obtained in a later step, but preferably from the data obtained in step O2.

After a renewed change of instruments to the cannulated screw, or the use of another or another robot arm, this is screwed into the os scaphoideum in step 012 with a torque that has preferably already been precalculated in step O2.

The determination of the entry angle and the entry point of the screw takes place, for example, analogously to the determination of the entry angle and entry point of the drill.

A preferably variable torque (for example in the unit [Nm]) can take place by means of a torque measurement in the robot arm 200 . A constant comparison with the data of the 3D model created in step O2 can also be carried out in order to take account of the existing bone density (cortical bone, spongiosa), for example.

The penetration depth the end position of the screw end point is e.g. B. via its x, y and z coordinates in comparison with the data of the vector of the K-wire. When screwed in, the tip of the screw preferably reaches exactly the area of the subchondral lamella of the os scaphoid . At this point, a comparison with the pre-calculated end point position can also take place. For this purpose, the tracking camera(s) 195 can be used, alternatively or additionally, the position or position sensor installed in the robot arm 200. position determination sensors .

In some embodiments, a comparison with a preferably current X-ray image can also be carried out here.

The gripping farm 200 removes the K-wire in step 013 .

This ends the autonomous part of the operation of this embodiment, and humans intervene again in what is happening.

014 represents the surgeon treating the surgical wound and the anesthetist initiating the recovery process.

At the end (015) is the discharge of the patient.

Reference List

1 system

50 computer tomograph

100 computer system

101 unit of account

103 data storage

105 monitor ; Advertisement

110 virtual obj ect

150 control unit

170 user interface

180 simulation device with haptics

181 arm segment of the haptic arm

182 joint of the haptic arm

183 hand section

185 haptic ( generator )

187 simulation tool

187 ' virtual ( or visualized )

Opera at ions instrument

190 real obj ect

195 tracking camera

200 surgical robots with robotic arm

201 robotic arm segment

202 joint

203 storage device

205 recording

207 real surgical instrument

01 to 015 Steps of an example operation

P patient S1 to S18 process steps

SAug augmentation step(s)

Ssrf capture step

S Seg segmentation step(s) U user

Claims

Expectations

1. Computer-aided method, with the steps:

Reading geometric data of an object into a computing unit (101);

assigning density values or hardnesses to volume sections of the object; wherein the geometric data is or comprises automatically segmented volume data; Visualizing the object using the computing unit (101) based on the data read in in a virtual, three-dimensional (3D) space on a display or a monitor (105) to achieve a virtual object (110); optional: providing a real 3D object, optionally based on the geometric data; Providing a simulation device (180) with haptics (185), optionally with a simulation instrument (187), for the user (U); optional: providing an optical real-time tracking camera (195);

Tracking the movement of the simulation device (180) or the simulation instrument (187) in real space, optionally using the tracking camera (195);

guiding the simulation device (180) or the simulation instrument (187) in space by a user;

Visualizing a virtual surgical instrument (187') in virtual, three-dimensional (3D) space on a display or monitor (105); determining a collision between the virtual surgical instrument (187') on the one hand and the virtual object (110) and/or volume sections of the virtual object (110) in the virtual space on the other hand;

determining a force feedback for the duration of the collision, taking into account the density values or hardnesses of the volume sections;

Transmission of the specific force feedback on the simulation device (180) or the haptics (185) of the same. Computer-aided method according to claim 1, wherein the geometric data are or include medical data. Computer-aided method according to claim 1 or 2, wherein the force feedback changes with virtual penetration of the virtual surgical instrument (187') into the virtual object (110). Computer-assisted method according to one of the preceding claims, wherein data relating to sequences of movement of the simulation instrument (187) and/or the real object (190) and/or the virtual surgical instrument (187') and/or the virtual object (110) in the respective corresponding Space in a provided data storage (103) are recorded and / or stored. Computer-assisted method according to claim 4, wherein from a large number of recorded data relating to the sequences which am a virtual course of the operation virtual object (110), or sections of the course of the operation, adjust, a target course of the operation or sections for this purpose are created and stored. Computer-assisted method according to claim 5, wherein a representation of the management of a virtual surgical instrument (187 ') from the target operation history or the section thereof, in particular relative to a virtual object (110), on a monitor or a display (105) is displayed. Computer-assisted method according to any one of the preceding claims, wherein the geometric data is or comprises real data, e.g. B. Data on body geometry or anatomy. Computer-aided method according to one of the preceding claims, further comprising the step :

Acquisition of the geometric data to be read in on a real object (190) or on a patient (P). Computer-aided method according to one of the preceding claims, further comprising the step :

Segmenting the geometric data to generate segmented volume data, in particular 75 automatically, in particular under assignment of

Density values for the generated volume segments. Computer-aided method according to one of the preceding claims, wherein the geometric data is or includes segmented, in particular automatically segmented, volume data. Computer-aided method according to one of the preceding claims, wherein the geometric data are or include medical volume data in DICOM format. Computer-aided method according to one of the preceding claims, further comprising the step:

Augmenting the geometric data that has been read in and storing it in the data memory ( 103 ) . Computer-assisted method according to one of the preceding claims, wherein the geometric data is or includes augmented data. Computer-aided method according to one of the preceding claims, further comprising the step:

Store a variety of destination

Operation courses or sections thereof, obtained from a variety of data relating to recorded sequences, which the virtual 76

Show surgical history or sections, im

Data storage (103) . A computerized method as claimed in any one of claims 5 to 14, further comprising the step

Providing data extracted from the target operation courses or sections thereof from the data memory (103) and transmitting them to a storage device (203) of a surgical robot (200), the selection of the data fulfilling at least one predetermined criterion. A computerized method according to any one of claims 5 to 15, further comprising the step of:

Recording data relating to sequences of movement of the simulation instrument (187) and/or the real object (190) and/or the virtual surgical instrument (187') and/or the virtual object (110), each in the corresponding space, or sections thereof and comparing one or more properties of the recorded data with respect to the sequence of movement of the simulation instrument (187) and/or the real object (190) and/or the virtual surgical instrument (187') and/or the virtual object (110). at least one target operation history or sections thereof stored in the data memory (103) provided. 77 The computerized method of claim 16, further comprising the step of:

Assessing a quality of the comparison based on predetermined criteria and outputting a statement about the quality for the user's attention. Computer system (100), comprising: a data memory (103), as mentioned in the preceding claims, with stored geometric data, a multiplicity of objects, in particular a common object class, obtained by means of a method according to one of claims 1 to 17. System (1 ) , with a computer system (100) according to claim 18; a surgical robot (200) with a receptacle (205) for a real surgical instrument (207); a control unit (150) for controlling the surgical robot (200) based on data stored in the data memory (103). The system (1) of claim 19, further comprising: a user interface (170) for entering control commands for controlling the

Surgical robot (200) by the user.