DE102020126029A1 - Surgical assistance system and display method - Google Patents

Abstract

The invention relates to a surgical assistance system (1) for use in a surgical intervention, comprising: at least one medical imaging device (2; 16; 17) with a recording head (4) and an, in particular spaced, actuating section (3; 18; 19), which is intended and adapted to create an intracorporeal recording (IA) in a patient (P), a detection device (5), preferably with at least one camera (6), which is intended and adapted to capture a surgical intervention area of the patient ( P) together with the actuating section (3; 18; 19) of the imaging device (2; 16; 17) at least optically and to create an extracorporeal recording (EA), a display device, in particular an OP monitor (7), for Representation of a visual content, wherein the surgical assistance system (1) also has a control unit (8), which is adapted for the extracorporeal recording (EA) of the acquisition g device (5) and the intracorporeal recording (IA) of the imaging device (2; 16; 17) to determine a position of the recording head (4) by the detected actuating section (3; 18; 19) in combination with a defined relation to the recording head (4), a superimposition representation (U) of the intracorporeal recording (IA) in or to generate on the extracorporeal recording (EA), in which, based on the position of the recording head (4), the intracorporeal recording (IA) is displayed in the correct position and orientation and thus in the correct position in the extracorporeal recording (EA), and the overlay display (U) output visually through the display device. In addition, the invention relates to a recording display method, an electronically readable storage medium and a sterile room (100) with an assistance system (1) according to the independent claims.

Description

technical field

The present invention relates to a surgical assistance system for use in a surgical intervention with at least one medical, in particular surgical, imaging device with an imaging recording head and an, in particular spaced, actuating section. The actuating section can in particular be a handling section/handpiece/gripping section for manual handling as well as a section guided by a robot. The imaging device is intended and adapted to generate an intracorporeal recording in a recording direction in a patient with the aid of the recording head and to make it available digitally (data processing). The surgical assistance system also has a detection device, preferably with or consisting of at least one camera, which is provided and adapted for at least optically detecting a defined area, here a surgical intervention/treatment area of the patient, together with the operating section of the medical imaging device and to create an extracorporeal recording and also make it available digitally. To display visual content, the surgical assistance system has a display device/display device, in particular an OP monitor/OP display. In addition, the invention relates to a recording display method, an electronic storage medium and a medical sterile room according to the preambles of the independent claims.

State of the art

Surgical assistance systems are known from the prior art which allow an intracorporeal recording, ie a recording inside the body, such as a video endoscope recording, to be displayed on a monitor in an operating room. During his work, a person involved in an operation (OP participant), in particular a surgeon, must keep an eye on the monitor with the inner tissue that is not visible to him, on the one hand, and on the other hand monitor and adjust the patient and the endoscope, in particular to determine a position and alignment of the endoscope head arranged intracorporeally track in the patient. In this case, the surgeon is required to perform a great deal of mental connection between the two modalities, which is tiring and increases the susceptibility to errors. Although there is currently the technical possibility of displaying an extracorporeal recording of the operating environment or the surgical intervention area and the anatomy of the patient on a separate, second monitor, and thus at least reducing a tiring head movement of the surgeon, the mental challenge and the work to be done nevertheless remain Effort to combine the two recordings with mental merging of the information, in particular the position and orientation of the intracorporeal recording in the patient's body relative to this. Current visualization systems and methods in an operating theater thus separate between visualization within the patient (intracorporeal) using an endoscope or a surgical microscope, for example, and visualization outside of the patient (extracorporeal). This poses a great challenge, especially for those involved in the operation, who have insufficient spatial imagination for this purpose and makes it difficult to link information.

Summary of the Invention

It is therefore the object of the invention to avoid or at least reduce the disadvantages of the prior art and in particular to provide a surgical assistance system as well as a (recording) display method and a sterile room that the OP participants, in particular the (leading ) surgeons, at a glance, provides an intuitive and supportive fusion of extracorporeal and intracorporeal imaging information that improves hand-eye coordination and ensures safe access to the surgical environment, specifically to detect tissue damage from careless and unintentional actions as well as prolong a to avoid surgical intervention. In particular, as far as possible, all those involved in the operation should be provided with a surgical assistance function that has not been achieved so far.

In a generic surgical assistance system, this task and the objectives of the invention are achieved according to the invention by the features of claim 1, in a generic presentation method, according to the invention by the features of claim 11, in the case of a generic storage medium, according to the invention by the features of claim 12, and in a medical sterile room, according to the invention the features of claim 13 achieved.

In principle, the invention thus provides for a surgical assistance system to record an intracorporeal region (not visible from the outside) and to record an extracorporeal region (visible from the outside). The recording of the intracorporeal area is then, due to the special configuration of the control unit, which depends on the position of the recording head relative to the recording of the extracorporeal area integrates the intracorporeal exposure into the extracorporeal exposure, displayed in the correct position in the exposure of the extracorporeal area. For this purpose, the position of an actuating section of the imaging device that is visible (from the outside) is detected (in space) and the associated position (i.e. position and orientation) of the imaging head (in particular not visible from the outside) is determined via a determinable or known relationship between the actuating section and the imaging head. Based on this information, the intracorporeal recording is displayed in the correct position in the extracorporeal recording, as would be seen by the detection device or, for example, the camera of the detection device. In other words, the intracorporeal recording, in particular an intracorporeal image, is displayed in the extracorporeal recording at its actual, current position in 3-dimensional space, as seen by the detection device, in particular the camera.

According to the invention, a control unit (ECU)/calculation unit/computer unit is specially adapted or configured to digitally record and process the extracorporeal recording of the recording device and the intracorporeal recording of the imaging device. The intracorporeal and/or extracorporeal recording can be digitally provided, recorded and processed as a current (live) video recording. The control unit is also adapted to use the position-detected actuation section in combination with a defined relation to the recording head to determine a position of the recording head and thus its field of view, and ultimately the position of the intracorporeal recording via this. On the basis of the position of the recording head determined by the control unit and the two extracorporeal and intracorporeal recordings, the control unit generates an overlay display of the intracorporeal recording on or in the extracorporeal recording, in which the intracorporeal recording is in the correct position / in the correct position and/or in the correct orientation / in the correct orientation and therefore in the correct position / is shown in the correct position in the extracorporeal recording. This overlay display that is generated is finally output visually by the display device, for example the surgical monitor.

Because the detection device detects the position of the visible part of the imaging device, namely the actuation section, which inevitably lies outside the patient's body, the control unit is adapted to determine a position of the actuation section relative to the detection device or a reference coordinate system of the detection device. The control unit uses a known or determinable relation of the actuating section to the recording head to calculate or determine the position of the recording head, which is in particular not visible and is arranged inside the body, from the detected position of the operating section. A position of the recording head relative to the reference coordinate system (reference KOS) is thus also determined. Based on this location with a position in 3D space and an (intracorporeal) recording direction (indicating a field of view) together with the extracorporeal recording, the control unit then generates the overlay representation.

By means of the assistance system, the surgeon can, for example, rotate an actuating section and, via this, the recording head, about a longitudinal axis, with the correct orientation still being displayed in the overlay display. The fact that the recording head was rotated with a camera, for example, and thus necessarily also the intracorporeal image recorded by the camera, but the control unit "rotates back" the image again by recognizing the actuating section and, as a result, determining the position of the recording head the intracorporeal image is always displayed in the correct position in the extracorporeal image. This helps the surgeon to concentrate only on the display device and to extract the information directly from there without having to perform any mental twisting and positioning.

It was recognized that the surgical assistance system according to the invention offers the possibility of displaying the information in the correct position on a single display device, such as an operating room monitor, which means that everyone involved in the operation, regardless of their spatial imagination, has an overview of the operational events obtained. In the central overlay display, the relevant information from the intracorporeal as well as the extracorporeal recording is combined with one another in a meaningful way, so that the recordings can be visually recorded in the correct relation to one another at a glance.

In other words, the two recordings, the extracorporeal and intracorporeal recordings, are recorded in relation to one another, in particular by recording the 3D geometry of the operating section and a known transformation from the operating section to the optical field of view of the medical imaging device. The control device is adapted to superimpose these two recordings recorded in relation to one another and to display the intracorporeal recording in the correct position in the extracorporeal recording.

In this way, a surgical assistance system is made available to those involved in the operation, which offers an intuitive, supportive merging of information from a visible (patient) body image and an intracorporeal image that is not visible to the eye. This assistance system improves a surgeon's hand-eye coordination and ensures safe access to the surgical environment. Due to the rapid provision of information, an intervention time can be further reduced and a treatment error rate can be further reduced.

The surgical assistance system can also significantly improve a learning curve for surgical interventions and also increase acceptance for minimally invasive surgical interventions.

The term positionally correct defines that the intracorporeal recording is reproduced at the actual, correct position in the extracorporeal recording in the overlay display. In the simplest example, this would be the case if a camera of the detection device is aimed vertically from above at a lying patient and provides the body anatomy as an extracorporeal recording and a (video) recording head of a video-capable endoscope also at a point on the patient's body vertically from above is directed into the patient that exactly at the position of the recording head the video image recorded by the recording head is superimposed as an intracorporeal recording and is thus superimposed. If the recording head is now moved relative to the body surface while maintaining the recording direction, the intracorporeal recording moves with it in the correct position. If the recording head is rotated around the axis of the recording direction, the image (intracorporeal recording) rotates accordingly if the display is only correct in terms of position.

The term orientation-correct defines that the intracorporeal recording is reproduced in the actual, correct orientation in the extracorporeal recording in the overlay display. In the example given above with parallel recording directions, the ultrasound probe can be physically rotated about its recording axis and thus also the recorded image for an orientation-correct display, but the orientation of the recording head changes, so that the recorded image is rotated back against the rotation for an orientation-correct display will. In this way, the intracorporeal recording always remains in the correct orientation relative to the extracorporeal recording.

The term position-correct defines the combination of position-correct and orientation-correct, in which the intracorporeal recording is displayed in the correct position as well as in the correct orientation in the extracorporeal recording. With a parallel alignment of the recording directions of the extracorporeal recording and the intracorporeal recording, the superimposed intracorporeal recording is displayed with the same aspect ratios as in the actual intracorporeal recording. If, in this example, the recording direction of the intracorporeal recording is angled in relation to the recording direction of the extracorporeal recording, an extension or a dimension of the recording of the associated direction is compressed according to the angle, until at a 90° angle between the two recording directions there is only one one-dimensional line is created in the overlay plot.

This position-correct display can be compared to a type of digital magnifying glass function that penetrates the body or looks into it, but in which the intracorporeal output image is provided by the imaging medical device (live). Depending on the orientation of the magnifying glass, the corresponding overlay display is then displayed, with the magnifying glass being controlled or moved by the actuating section in the present case. This has the advantage that the person involved in the operation sees the recordings of the inside of the body directly in the recording of the outside of the body, from which it is immediately apparent where the recorded organs or tissue (in the position shown) are located in the body.

Advantageous designs are claimed in the dependent claims and are explained below.

In a preferred embodiment, the control unit can be adapted to generate the position-correct representation of the intracorporeal exposure in the extracorporeal exposure using augmented reality (AR) methods.

According to a further preferred embodiment, the control unit can be adapted to generate the correctly positioned representation of the intracorporeal exposure in the extracorporeal exposure by: adapting the detection device, the position and orientation of the operating section and thus a local coordinate system of the operating section relative to a reference coordinate system of the to detect the detection device and to provide it to the control unit in the form of a coordinate transformation; a determinable or predetermined/known coordinate transformation from the coordinate provide the system of the actuating section to the coordinate system of the recording head of the control unit or store it in a memory unit of the control unit; and the control unit is adapted, starting from a reference coordinate system of the detection device, to determine a position and an orientation of the recording head relative to the reference coordinate system via the recorded coordinate transformation to the actuating section and via the determinable or stored coordinate transformation to the recording head; the detection device is also adapted to detect the position and orientation of the patient and thus the extracorporeal exposure relative to the reference coordinate system and thus determine a relative position of the recording head and thus the intracorporeal exposure to the extracorporeal exposure; and the control unit is adapted to generate an overlay display based on the determined relative position of the recording head and thus the intracorporeal recording to the position of the extracorporeal recording, in which the intracorporeal recording is displayed in the correct position in the extracorporeal recording.

In particular, at the specific position of the recording head, the intracorporeal recording can be projected with the correct orientation, vertically upwards onto the body surface and superimposed in the extracorporeal recording.

In particular, the at least one imaging medical device can be a receptive endoscope, in particular a rigid video endoscope. A rigid video endoscope has the advantage that a geometric relationship between the handpiece as the operating section and the recording head on the front always remains the same and is known. With a previously known geometry of the handpiece, which in particular is not axisymmetric in order to also detect a rotation about a recording direction, a stored (static) coordinate transformation can always be used to deduce or calculate the coordinate system of the recording head and thus its position from the local coordinate system of the actuating section . Based on the position of the recording head with the associated field of view, the position of the intracorporeal recording can then be inferred. In particular, in the case of a rigid video endoscope, a distance from the recording head to the recording can be regarded as small, so that this is negligible. Alternatively, a distance from the intracorporeal recording can also be determined via the recording head and correspondingly taken into account in a transformation from recording head to recording. It is particularly important that the orientation of the recording head corresponds to the orientation of the recording, ie the intracorporeal recording also rotates when the recording head is rotated. In one embodiment, the video endoscope can have a stereo camera system in order to create an intracorporeal 3D image. In particular, the video endoscope provides a round intracorporeal recording, which is displayed in the correct position in the overlay display, for example as a circle or as an ellipse. Alternatively or additionally, the imaging device can be a microscope and/or an ultrasound device. All medical devices have in common that they can produce (at least two-dimensional) images from inside the body that are not visible to the surgeon's eye. In particular, these medical imaging devices provide real digital recordings that can be easily manipulated and processed with computer support.

In a further preferred embodiment, the detection device can have an operating room light with a centrally built-in camera and/or have a camera attached separately to a stand or arm, the camera forming the reference coordinate system of the detection device. A surgical light is present in sterile rooms, in particular in operating rooms/operating theaters, and is always aimed at the surgical intervention area in order to illuminate it. If a camera, such as a 2D camera or a 3D stereo camera, is now used centrally, the recording direction is also directed towards the surgical intervention area with a good field of view. In addition, the surgical light can be easily moved and rearranged together with the camera to provide a new field of view or another, new extracorporeal image from different perspectives. Integration into existing OR systems is particularly easy and inexpensive. Alternatively or additionally, a camera can be attached to an extension arm or to a stand and, in particular, a recording direction can be adjustable.

Advantageously, a geometry of the actuating section can be stored in the memory unit of the control unit, so that using a standard (2D) camera of the detection device with a CMOS sensor and an evaluation method of machine vision (machine vision), for example, via the actuating section detected by the camera with the stored geometry of the actuating section, the position of the actuating section can be detected by the detection device. In this way, the operating section can be reconstructed from 2D to 3D in order to ultimately determine its position in space relative to the reference coordinate system. Alternatively or additionally, the detection device can also be a 3D camera, for example a stereo camera, for the spatial detection of the operating portion, which are processed and interpreted by the control unit. Alternatively or additionally, the detection device can have an infrared camera (with infrared transmitter and infrared receiver) in order to carry out or at least support a three-dimensional detection of the actuating section.

In a preferred embodiment, the control unit can also be adapted to generate and display in the overlay display predefined (standardized) intersection points or intersection lines, in particular those stored in the memory unit, for a given surgical intervention. Thus, in addition to the two different real recordings, the surgeon is given virtual assistance, for example in order to define entry points for endoscopes or to carry out incisions.

Particularly preferably, the surgical assistance system can be combined in a single, preferably portable, assistance module, so that it can be positioned at different locations and in the vicinity of different operating tables as required. The assistance system preferably has its own energy source in order to act autonomously.

In particular, the surgical assistance system can have an interface in order to communicate with at least one other medical device. Anatomical data of the patient can preferably be stored in the memory unit or anatomical data, for example from a previous X-ray exposure, can be provided via the interface and included in the overlay display. In this way, the two recordings can also be linked to the patient's anatomical data.

According to one aspect of the invention, the control unit can be adapted to display an intracorporeal part / section of the imaging device that is not visible to the surgeon or the surgical participant, in particular an internal, rigid endoscope shaft of the video endoscope, in the overlay display in order to show the surgeon the Specify tip of the endoscope in the display device. In the case of the rigid video endoscope, for example, the geometry of the operating section up to the recording head is known. This geometry can be stored in the storage unit so that, starting from the visible operating section, the invisible but known endoscope shaft with recording head is digitally superimposed.

The detection device can preferably have a plurality of cameras in order to provide views from different positions and perspectives. Depending on his requirements, a surgeon can then select the best view in each case, which is then output by the display device as a superimposition display with the positionally correct intracorporeal recording(s) calculated by the control unit.

Advantageously, the surgical assistance system can have at least two (a large number of) imaging devices, in particular at least two endoscopes, in order to generate at least two intracorporeal recordings in the superimposition display and display them accordingly. In particular, the intracorporeal recordings can even overlap and/or superimpose one another, so that a coherent intracorporeal section can be displayed.

According to a further aspect of the invention, the actuating section can have optical markers/landmarks in order to further improve the detection or detection of its position in space and to integrate it into the assistance system. Optionally, anatomical landmarks can also be arranged on the patient in order to better detect the position and location of the patient. In addition, an instrument can also be provided with landmarks in order to further improve calculation of the overlay representation.

According to one embodiment, the surgical assistance system can be adapted to detect an orientation of the operating section in order to indirectly detect an orientation of a perspective of the surgeon.

In particular, the patient can be three-dimensionally recorded in space by means of the recording device, and the intracorporeal recording can preferably be mathematically projected perpendicularly onto the body surface for the correct position.

In particular, the medical imaging device is adapted to create a 3-dimensional intracorporeal image, so that both the intracorporeal image and the extracorporeal image are available as a 3-dimensional image. The control unit is adapted to generate the position-correct display of the intracorporeal image in the extracorporeal image on the basis of these 3-dimensional images together with the determined position of the image-taking head. In this way, an overlay display becomes even more precise.

In other words, an operating theater monitor that is mounted on a wall of the operating room or on a stand can be used to display the extracorporeal recording with the patient's anatomy. The intracorporeal recording, such as a (live) image from the inside of the body, is then placed or superimposed on or over this extracorporeal recording, in particular using augmented reality methods. The intracorporeal exposure is thus displayed in its 3D position relative to the extracorporeal exposure, as detected by the detection device.

With regard to a generic recording display method for the central display of two different recordings, the objects and objectives of the invention are achieved by the steps: creating an intracorporeal recording by at least one medical imaging device, detecting a surgical intervention area and an operating section of the imaging device and creating an extracorporeal recording a detection device, determining the position of the recording head with the aid of the detected actuating section in combination with a defined (geometric) relation to the recording head, generating a superimposed representation of the intracorporeal recording on the extracorporeal recording, the intracorporeal recording being correctly positioned on the basis of the position of the recording head in the extracorporeal recording is shown, and outputting the overlay display by the display device. With this display method, a merged central recording can be created and output from two recordings, which offer those involved in the surgery an unprecedented overview.

At this point it is pointed out that features of the method according to the invention can be transferred both to the surgical assistance system according to the invention and vice versa.

With regard to a generic electronic storage medium, the object of the invention is achieved in that the steps of the display method according to the invention are stored on the storage medium.

With regard to a generic sterile room, the object of the invention is achieved in that the medical sterile room has a surgical assistance system according to the invention.

character list

• 1 eine schematische Ansicht eines erfindungsgemäßen chirurgischen Assistenzsystems einer ersten bevorzugten Ausführungsform,
• 2 in einer schematischen perspektivischen Ansicht das Videoendoskop aus 1,
• 3 eine detaillierte Ansicht der Ausgabe über einen OP-Monitor der 1,
• 4 eine perspektivische Ansicht eines erfindungsgemäßen chirurgischen Assistenzsystems einer weiteren, zweiten bevorzugten Ausführungsform, und
• 5 ein Flussdiagramm eines erfindungsgemäßen Darstellungsverfahrens einer bevorzugten Ausführungsform.

The invention is explained below on the basis of preferred embodiments with the aid of the accompanying figures. Show it:

• 1 a schematic view of a surgical assistance system according to the invention of a first preferred embodiment,
• 2 shows the video endoscope in a schematic perspective view 1 ,
• 3 a detailed view of the output via a surgical monitor 1 ,
• 4 a perspective view of a surgical assistance system according to the invention of a further, second preferred embodiment, and
• 5 a flowchart of a presentation method according to the invention of a preferred embodiment.

The figures are schematic in nature and are only intended to help understand the invention. Identical elements are provided with the same reference symbols. The features of the various embodiments are interchangeable.

Detailed description of preferred embodiments

1 FIG. 12 shows a schematic view of a surgical assistance system 1 (referred to simply as assistance system below) of a first preferred embodiment. The assistance system 1 is used in a medical sterile room in the form of an operating room 100 in a preferred embodiment in order to support those involved in the operation with suitable visualization. A surgical intervention, in this case a minimally invasive intervention, is performed on a patient P (shown only schematically here) in a central, sterile surgical intervention area.

A rigid video endoscope 2 (hereinafter simply called endoscope) with a handpiece/grip 3 and a spaced, front recording head/video head 4 protrudes into the interior of the patient P's body on the endoscope shaft side. For visualization purposes, the recording head 4 has an endoscope video camera (not shown here), which uses a CMOS sensor to take a two-dimensional intracorporeal recording IA of the body interior of the patient P in the direction of a longitudinal axis of the endoscope 2 as an up-to-date (live video) Recording created and made available digitally. The handpiece 3 for manual grasping and handling by a surgeon (in 1 is only the hand of the surgeon is shown), however, protrudes outside the body of the patient P and is visible from the outside, also for those involved in the operation.

The assistance system 1 also has a recording device 5 for data and visual recording of the surgical procedure outside the body of the patient P of the patient P as well as the handpiece 3 of the endoscope 2 is optically recorded by means of a CMOS sensor and also created as a two-dimensional extracorporeal recording EA in the form of a current (live video) recording (as an optical digital image) and makes it available digitally. In addition to the extracorporeal recording EA, the geometric position of the handpiece 3 relative to the camera 6 is also recorded by the camera 6 using an evaluation method of machine vision.

The assistance system 1 has a special electronic control unit 8 (ECU) for recording, processing, calculating on the basis of these recorded images and the recorded handpiece 3 and for controlling, which is specially configured compared to the prior art. Specifically, the control unit 8 is adapted to receive the provided (digital) data of the intracorporeal recording IA and the extracorporeal recording EA, i.e. the two “live video transmissions” as well as the position of the actuating section 3 digitally detected by the detection device 5, to include data. A defined geometric relationship between the handle 3 and the recording head 4 in the form of a (fixed) coordinate transformation is also stored in a memory unit 9 of the control unit 8, since the geometric arrangement of the two sections relative to one another does not change. Based on this data, the control unit 8 is able, as explained below, to carry out an intelligent merging of the intracorporeal exposure and the extracorporeal exposure and to generate an overlay display U in which the intracorporeal exposure IA of the endoscope is displayed in the correct position in the extracorporeal exposure EA.

For this purpose, the (geometric) position of the intracorporeal exposure IA is first determined by the control unit 8 . The handpiece 3 of the endoscope 2 has a (previously) defined local handpiece coordinate system 11 (hereinafter referred to as the H coordinate system), with a local recording head coordinate system 12 (hereinafter referred to as the AK coordinate system called) can be determined. In other words, the endoscope 2 has a local H coordinate system 11 on the handpiece 3 and an AK coordinate system 12 on the tip of the endoscope 2 on the recording head, with a relation to one another being uniquely specified via a coordinate transformation. The control unit 8 can determine the position of the recording head 4 via the detected position of the handpiece 3 together with the coordinate transformation. Since the intracorporeal recording IA is created in the area of the tip and frontal of the endoscope 2 in the direction of the longitudinal axis of the endoscope 2 and a distance between the recording head 4 and the intracorporeal recording IA is negligible, it can be assumed or approximated that the position of the AK coordinate system 12 corresponds to the position of the intracorporeal recording IA. The control unit 8 can use this transformation chain to determine the location, ie the position and the orientation of the intracorporeal recording IA relative to a reference coordinate system 13 of the detection device 5 , here lying in the camera 6 .

Based on the (information on) the position of the recording head 4 and thus the intracorporeal recording IA, the control unit 8 then generates the overlay display U, in which the intracorporeal recording IA of the endoscope is displayed in the correct position in the extracorporeal recording EA. This overlay display is finally displayed on an operating room monitor 7, which the surgeon can easily see.

The present surgical assistance system 1 shows the surgeon the image from the inside of the patient in the correct position in the image of the external anatomy of the patient in a single image or a single image, so that the surgeon can grasp it easily and intuitively where and with what orientation the displayed inner tissue is in the body. The assistance system 1 provides a type of viewing function or digital magnifying glass function through a surface of the body, the actual intracorporeal recording IA being used as the image of the magnifying glass. This provides all those involved in the surgery with a hitherto unparalleled overview of what is happening in the surgery. In particular, navigation of the endoscope 2 is simplified and demands on the surgeon for additional intellectual effort are minimized, since the surgeon can navigate imaging devices intuitively.

• Referenzkoordinatensystem 13 - Handstück-Koordinatensystem 11 - Aufnahmekopf-Koordinatensystem 12 - Intrakorporales Aufnahme-Koordinatensystem 14,
• wobei in der vorliegenden Ausführungsform das IA-Koordinatensystem 14 mit dem AK-Koordinatensystem 12 gleichgesetzt wird. Alternativ kann natürlich auch ein Abstand zu einem Gewebe erfasst und entsprechend in die Berechnung mit eingebunden werden. Weiter alternativ kann auch ein fixer Abstandswert festgesetzt und als Koordinatentransformation berücksichtigt werden. Insbesondere kann auch die 3D-Struktur der intrakorporalen Aufnahme erfasst und bei einer Erzeugung der Überlagerungsdarstellung verarbeitet und berücksichtigt werden.

The determination of the position-correct display of the intracorporeal exposure IA in or on the extracorporeal exposure EA is described below with reference to FIG 2 and 3 explained. 2 shows the rigid video endoscope 2 in a detailed view 1 . The video endoscope 2 takes the intracorporeal recording IA, which in 2 is shown only schematically as a rectangular recording plane. This recording IA has an intracorporeal recording coordinate system 14 (IA coordinate system). The local AK coordinate system 12 is also shown at the tip of the endoscope 2, ie at the recording head 4. A coordinate transformation can be used to draw conclusions from the intracorporeal recording coordinate system 14 to the AK coordinate system 12 and vice versa. In particular, the approximation is assumed that the distance between the two coordinate systems in the endoscope 2 is so small that it can be assumed that the intracorporeal recording coordinate system 14 lies on the AK coordinate system 12 . Starting from the AK coordinate system 12, the coordinate system of the handpiece 11, which is visible, can be determined by means of a coordinate transformation from the tip, which is located in the patient's body when the endoscope 2 is used and is therefore not visible from the outside, and vice versa. In this way, a position of the intracorporeal exposure IA is determined via three coordinate transformations starting from the reference coordinate system 13:

• Reference coordinate system 13 - handpiece coordinate system 11 - recording head coordinate system 12 - intracorporeal recording coordinate system 14,
• in the present embodiment, the IA coordinate system 14 is equated with the AK coordinate system 12. Alternatively, of course, a distance to a tissue can also be recorded and included in the calculation accordingly. As a further alternative, a fixed distance value can also be established and taken into account as a coordinate transformation. In particular, the 3D structure of the intracorporeal recording can also be recorded and processed and taken into account when generating the overlay display.

Since the detection device 5 detects the position of the handpiece 3 from the outside, the position of the intracorporeal receptacle IA can be clearly determined in this way. In this way, a position of the intracorporeal exposure IA relative to the extracorporeal exposure EA in space can be determined. The position-correct representation of the intracorporeal exposure in the overlay representation U is then generated via suitable image transformations from 3D to 2D (with the associated loss of a spatial dimension) using the information on the relative position of the intracorporeal exposure IA to the extracorporeal exposure EA. Suitable mathematical (algebraic) or information technology projection methods for generating the overlay representation U can be used for this purpose. In particular, the body of the patient is also recorded 3-dimensionally by the recording device 5 and is correspondingly taken into account in processing by the control unit when the superimposition representation U is generated. In particular, both the intracorporeal image IA and the extracorporeal image EA can be present as a 3-dimensional image and the control unit 8 can generate the position-correct representation of the intracorporeal image IA in the extracorporeal image on the basis of these images. In this way, an overlay representation U becomes even more precise.

3 shows the overlay display U output by the surgical monitor in an enlarged view 1 , which is the to 1 and 2 explained detected position of the handpiece 3 and thus the position of the intracorporeal recording IA operated. The intracorporeal recording IA generated by the endoscope 2 is initially circular when the control unit 8 is provided (black edges in the case of a rectangular recording are not processed). The rigid endoscope 2 is not vertical or horizontal, but is introduced into the patient P through an entry point at an angle of approximately 45° to the horizontal and thus to the abdominal surface of the patient's body. Consequently, a recording direction of the endoscope 2 is also inclined at an angle of 45°. Since the extracorporeal recording is essentially shown perpendicular to the patient, the inclination must be taken into account accordingly in order to obtain a position-correct display. The decisive factor is the orientation of the recording direction of the intracorporeal recording IA to the recording direction of the extracorporeal recording EA and their angles to one another.

In order to take the inclination into account, the control unit 8 is adapted to run an internal algorithm. The control unit 8 initially positions the intracorporeal exposure IA in the correct position at the tip of the endoscope 2 in the extracorporeal exposure EA. A correct “rotation” of the intracorporeal recording IA is also carried out by the control unit 8 on the basis of the specific position of the recording head 4 and reproduced in the overlay representation U. However, in order to compensate for a "distortion" of the intracorporeal image IA compared to the extracorporeal image EA, which would arise with a circular display equal to the actual image, one dimension must also be compressed, in the overlay display the dimension in the recording direction of the endoscope 2. In 3 a first dimension of the intracorporeal image and a second dimension perpendicular thereto are shown. So are the up If the directions of taking of the intracorporeal exposure IA and the extracorporeal exposure EA are parallel, the dimensions of the intracorporeal exposure IA remain unchanged. If the two recording directions are at an angle of 90° to one another, then one dimension is compressed completely to 0 (as a mathematical projection) and no intracorporeal recording IA is shown in the overlay representation U. At this point it should be noted that there are image methods to also distort and change an image plane, so that a "displayed image direction" can also be taken into account instead of a recording direction.

Due to the rotation and compression, the intracorporeal exposure IA is displayed with the correct orientation. Consequently, the intracorporeal exposure IA is shown in the correct position and orientation and thus in the correct position in the extracorporeal exposure EA. In addition, the non-visible endoscope shaft is superimposed virtually in the overlay representation U, so that the surgeon is provided with an even better overview.

4 shows a further, second preferred embodiment of a surgical assistance system 1 in which, in addition to a rigid endoscope 2, an ultrasound device 16 and a surgical microscope 17 are also used as medical imaging devices. The single detection device 5 has a 3D stereo camera system with two cameras 6 for the spatial detection of the respective operating sections in the form of the handpiece 3 of the rigid endoscope 2, a gripping piece 18 of the ultrasound device 16 and a handle 19 of the surgical microscope 17. In addition, the detection device 5 can create an extracorporeal recording EA of the surgical intervention area via the stereo camera system.

The principle of the surgical assistance system 1 is the same as above 1 until 3 explained. In the second embodiment in 4 However, instead of just one endoscope 2, three imaging devices 2, 16, 17 are now used. All three imaging devices 2 , 16 , 17 each generate an intracorporeal recording IA based on their intended special mode of operation and make these available to the control unit 8 . Since the control unit 8 is adapted, analogously to the above explanation, to determine a position of the recording head 4 and thus the intracorporeal recording IA by means of the detection device 5, it can carry this out (separately) for all three imaging devices 2, 16, 17 in the present case . In this way, three intracorporeal exposures IA are integrated in a single extracorporeal exposure. Here, all three intracorporeal recordings IA are displayed in the correct position in the overlay representation U in order to give the surgeon an even better and more comprehensive overview of what is happening in the operation. Since the three intracorporeal recordings IA form their own overlay representations, so to speak, they can be stacked on top of one another like planes. The assistance system 1 is therefore not subject to any limits with regard to the number of imaging devices 2 . In particular, the control unit 8 can be adapted to also connect the three intracorporeal images IA to one another and to overlay information. An opacity can also be reduced or a transparency of the intracorporeal recording can be increased in order not to lose any information.

In the second embodiment, the assistance system is designed as a mobile assistance system 1 so that it can be positioned at different locations and in the vicinity of different operating tables as required.

5 shows a display method according to the invention of a preferred embodiment. In step S1 an intracorporeal recording IA is created by at least one medical imaging device 2 . This is followed by step S2, in which the position of a surgical intervention area and an actuating section 3 of the imaging device 2 is detected by a detection device 5 and an extracorporeal recording EA is created. In method step S3, the position of the recording head 4 of the medical imaging device 2 is determined with the aid of the detected operating section 3 and thus the position of the operating section 3 in combination with a defined relation to the recording head 4 . This is followed by step S4 of generating an overlay display U of the intracorporeal image IA on the extracorporeal image EA, the intracorporeal image IA being displayed in the correct position in the extracorporeal image EA on the basis of the determined or determined position of the image head 4 . Finally, the overlay display U is output by the display device 7 in step S5.

Reference List

1

surgical assistance system

2

rigid video endoscope

3

handpiece

4

recording head

5

detection device

6

camera

7

surgical monitor

8th

control unit

9

storage unit

10

surgical light

11

Handpiece coordinate system

12

Recording head coordinate system

13

reference coordinate system

14

Intracorporeal recording coordinate system

15

Extracorporeal recording coordinate system

16

ultrasonic device

17

Surgical Microscope

18

Gripper part ultrasonic probe

19

(Microscope) handle

100

operating room

S1

Step Create intracorporeal exposure

S2

Step Capture and create extracorporeal recording

S3

Step Determining the position of the recording head

S4

Create Overlay Plot step

S5

Step Output Overlay Plot

P

patient

i.a

Intracorporeal recording

EA

Extracorporeal recording

u

overlay display

Claims (13)

Surgical assistance system (1) for use in a surgical intervention, comprising: at least one medical imaging device (2; 16; 17) with a recording head (4) and an, in particular spaced, actuating section (3; 18; 19), which is provided for this and is adapted to create an intracorporeal recording (IA) in a patient (P), a detection device (5), preferably with or consisting of at least one camera (6), which is provided and adapted to capture a surgical intervention area of the patient (P ) together with the actuating section (3; 18; 19) of the imaging device (2; 16; 17) at least optically and to create an extracorporeal recording (EA), a display device, in particular an OP monitor (7), for display of a visual content, characterized by a control unit (8) adapted for the extracorporeal recording (EA) of the detection device (5) and the intracorporal le recording (IA) of the imaging device (2; 16; 17) to determine a position of the recording head (4) by the position-detected actuating section (3; 18; 19) in combination with a defined relation to the recording head (4), a superimposition representation (U) of the intracorporeal recording (IA) in or to generate on the extracorporeal recording (EA), in which, based on the position of the recording head (4), the intracorporeal recording (IA) is displayed in the correct position and orientation and thus in the correct position in the extracorporeal recording (EA), and the overlay display (U) output visually through the display device. Surgical assistance system (1) according to claim 1 , characterized in that the control unit (8) is adapted to generate the position-correct representation of the intracorporeal exposure (IA) in the extracorporeal exposure (EA) in the overlay representation (U) by methods of augmented reality. Surgical assistance system (1) according to one of the preceding claims, characterized in that the control unit (8) is adapted to generate the positionally correct representation of the intracorporeal recording (IA) in the extracorporeal recording (EA) by the detection device (5) for this is adapted to detect the position and orientation of the operating section (3; 18; 19) and thus a coordinate system (11) of the operating section (3; 18; 19) relative to a reference coordinate system (13) of the detection device (5) and the control unit ( 8) in the form of a coordinate transformation, a determinable or predetermined coordinate transformation from the coordinate system (11) of the actuating section (3; 18; 19) to the coordinate system (12) of the recording head (4) of the control unit (8) is provided or in a memory unit (9) is stored in the control unit (8), and the control unit (8) is adapted for this, starting from a Reference coordinate system (13) of the detection device (5) via the detected coordinate transformation to the operating section (3; 18; 19) and to determine a position and an orientation of the recording head (4) via the determinable or predetermined coordinate transformation to the recording head (4), the detection device (5) is in particular also adapted to determine the position and orientation of the patient (P) and thus the extracorporeal recording (EA) and the control unit (8) and the control unit (8) is adapted, based on the determined position of the recording head (4) and thus the relative position of the intracorporeal recording (IA) to the position of the extracorporeal recording (EA) to an overlay display (U). generate in which the intracorporeal exposure (IA) is displayed in the correct position in the extracorporeal exposure (EA). Surgical assistance system (1) according to one of the preceding claims, characterized in that the at least one imaging medical device (2; 16; 17) is a receptive endoscope, in particular a rigid video endoscope (2), and/or a surgical microscope (17) and / or an ultrasonic device (16). Surgical assistance system (1) according to one of the preceding claims, characterized in that the detection device (5) has a camera (6) installed centrally in an operating room light (10) and/or a camera provided separately on a stand or arm which the camera (6) forms the reference coordinate system (13). Surgical assistance system (1) according to one of the preceding claims, characterized in that a geometry of the actuating section (3; 18; 19) is stored in the memory unit (9), so that by means of a 2D camera (6) the detection device (5) and an evaluation method using a machine vision method, the position of the actuating section (3; 18; 19) can be detected by the detection device (5). Surgical assistance system (1) according to one of the preceding claims, characterized in that the control unit (8) is adapted to display a non-visible, intracorporeal part of the imaging device (2; 16; 17), in particular the video endoscope, in the overlay display (U ) additionally display in order to display the recording head (4), in particular the tip of the video endoscope, to the operator via the display device (7). Surgical assistance system (1) according to one of the preceding claims, characterized in that the detection device (5) has a large number of cameras (6) in order to provide views from different positions and perspectives. Surgical assistance system (1) according to one of the preceding claims, characterized in that the surgical assistance system (1) has at least two imaging devices (2; 16; 17), in particular has at least two endoscopes, in order to show at least two intracorporeal To generate recordings (IA) and display them in the overlay display (U). Surgical assistance system (1) according to one of the preceding claims, characterized in that the actuating section (3; 18; 19) has at least one optical marker in order to improve detection of its position in space. Image display method for the central display of two different images , characterized by the steps: (S1) creating an intracorporeal image (IA) by at least one medical imaging device (2; 16; 17), (S2) detecting a surgical intervention area and an operating section (3; 18; 19) of the imaging device (2; 16; 17) and creating an extracorporeal recording (EA) by a detection device (5), (S3) determining the position of a recording head (4) of the imaging device (2; 16; 17) with the aid of the detected actuation section (3; 18; 19) in combination with a defined relation of this to the recording head (4), (S4) generating an overlay representation (U) of the intracorporeal recording (IA) on the extracorporeal recording (EA), based on the position of the recording head (4), the intracorporeal recording (IA) is shown in the correct position in the extracorporeal recording (EA), and (S5) outputting the Überla ger representation (U) by the display device (7). Electronically readable storage medium, characterized in that on the storage medium the method steps of the recording presentation method claim 11 are saved. Medical sterile room, such as an operating room (100), characterized in that the medical sterile room has a surgical assistance system (1) according to one of Claims 1 until 10 having.

Images