DE102019114817B4 - Imaging system and method of observation - Google Patents

Abstract

Stereo imaging system, in particular a medical imaging system, which has: - an observation instrument (10, 50, 110) with an image acquisition unit (30, 70, 130) for acquiring first image data and second image data which can be combined for stereo observation, - at least a position sensor (144, 146) for detecting an orientation, in particular a rotational orientation, of the instrument (10, 50, 110) in relation to a position reference (166), and a control device (150) which, depending on the orientation of the instrument (10, 50, 110) can be operated in a first playback mode and a second playback mode, the control device (150) being designed to erect images output with the image signal in the second playback mode depending on the orientation, characterized in that the control device (150 ) is designed to output an image signal which, in the first playback mode, is a stereo signal based on the first image aten and the second image data, and in the second playback mode comprises a mono signal based on the first image data or the second image data, and the control device (150) in a first angle of rotation range of the instrument (10, 50, 110) in the first playback mode and in a second angle of rotation range of the instrument (10, 50, 110) can be operated in the second playback mode.

Description

The present disclosure relates to a stereo imaging system with an observation instrument with an image acquisition unit for acquiring first image data and second image data which can be combined for stereo observation.

In particular, the present disclosure relates to a medical imaging system. Such imaging systems are regularly used to observe the human and / or animal body.

Imaging systems suitable for stereo observation usually include observation instruments with stereoscopic observation optics and stereo image sensors, that is to say, for example, two image sensors offset from one another. So there are two channels of observation. Image sensors typically include CCD sensors, CMOS sensors, and the like. When using two offset image sensors, one of the two image sensors is assigned to the right eye (first observation channel) and the other of the two image sensors is assigned to the left eye of the observer (second observation channel).

Imaging systems suitable for stereo observation are sometimes also referred to as 3-D imaging systems. It should be understood that this does not necessarily capture 3D data. However, the 3D image generated in this way allows the observer to perceive depth.

Medical imaging systems include systems for endoscopic observation, for example. For the purposes of the present disclosure, systems for endoscopic image processing include designs in which a probe-like instrument can be inserted into natural or artificially created body openings in order to capture images of the interior of the body. This also includes so-called video endoscopy systems.

From the US 9 848 758 B2 an instrument in the form of a stereo endoscope is known in which image data for the right channel and the left channel are corrected, and in which identification information is obtained on the basis of a test during assembly of the endoscope, which is intended to ensure that the correction data is actually also obtained for the right and left channels respectively.

From the WO 2010/105946 A1 An endoscope is known, with an image sensor and an inertial sensor at the distal end, the inertial sensor being designed to detect a tilt of the image sensor relative to the gravitational field, the endoscope also having a control device, the image sensor comprising an array of image pixels in order to Generate image data, wherein the inertial sensor is designed to generate a tilt signal representing the tilt, and wherein the control device is designed to combine the tilt signal with the image data while replacing part of the image data with the tilt signal to form a combination signal.

From the DE 10 2017 219 621 A1 a medical visualization system is known, comprising an endoscope and a microscope, which are aligned in different orientations and whose axes of vision can be inclined to one another. The endoscope is provided with a movement sensor, an angle of the viewing axis of the endoscope relative to the viewing axis of the microscope being determined. A display device is provided which displays a first image based on the microscope data and a second image based on the endoscope data, the orientation of the second image being able to be aligned as a function of the angle of the viewing axis of the endoscope.

From the US 2018/0042453 A1 a stereoscopic endoscopy device is known which has two image recorders and consequently two channels. A control device is provided which is designed to detect adhering foreign objects so that, if necessary, only one of the two channels is used when the other channel is impaired due to the foreign objects.

From the WO 2016/012248 A1 a stereo endoscope with distally arranged image recorders is known. Embodiments with a straight viewing direction and embodiments with an oblique viewing direction are described.

From the US 7,108,657 B2 an endoscopic visualization device with two image recorders is known, which are offset from one another, it being proposed that the two image recorders each be designed to be rotatable about an axis perpendicular to their sensor plane. In this way the captured image can be rotated. An image erecting function is thus given.

From the US 7,037,258 B2 a video endoscope with means for image erection is known. The endoscope is designed as a monoscopic instrument. It is therefore an instrument with an observation channel. The image is rotated or erected electronically.

From the US 7,134,992 B2 Another video endoscope with means for image erection is known. It is suggested to save the captured image in To erect depending on a determined orientation of the instrument. The endoscope has an image sensor arranged at the distal end. It is a monoscopic observation instrument with an observation channel. Acceleration sensors are used to record the orientation.

The present disclosure also relates to medical imaging systems that are designed to observe the body from outside the body. In other words, these systems do not provide for the instrument to be at least partially inserted into a body opening. Such systems, in particular their observation instruments, are referred to as so-called exoscopes, for example. Such an exoscope with stereoscopic optics is from the US 10,122,897 B2 known. It is proposed to make the optical unit rotatable in order to erect the image of the instrument as required. The orientation of the stereo base of the stereoscopic optics can thus be adapted so that stereoscopic observation is made possible even if the orientation or position of the instrument is changed.

The present disclosure relates, at least in exemplary configurations, to imaging systems and instruments for white light observation. This includes the detection of visible light, in particular parts of the electromagnetic spectrum that are visible to the human eye. This applies, for example, to a range from 380 nm to 750 nm (nanometers). However, this is not to be understood as restrictive.

In particular, this does not rule out that an imaging system according to the disclosure can also acquire image information for observation in the infrared or near-infrared range or in the ultraviolet range. This is conceivable, for example, for instruments for PDD (photodynamic diagnostics) and / or PDT (photodynamic therapy) as well as for instruments for fluorescence observation.

At least in essential embodiments, however, this should not include any radiological imaging methods (for example radiography, computed tomography, magnetic resonance tomography, ultrasound observation). Accordingly, the focus of the present disclosure is not on imaging systems for cross-sectional viewing. In contrast, essential aspects of the present disclosure relate to the observation of surfaces or objects close to the surface. It goes without saying that combined imaging systems are nevertheless conceivable.

When using observation instruments of the type described above, it is often necessary to rotate them or move them in some other way, for example around their longitudinal axis (axis of the instrument shaft or the observation optics) in order to change the image section being observed. This is the case in particular with hand-held or hand-held instruments. However, even with instruments that are rigidly received, degrees of freedom of movement can be provided in order to position the instrument and its observation optics in the desired manner.

In the case of observation instruments with only one image sensor, the alignment of the image sensor with respect to a reference horizon (defined for example by a playback device, for example a monitor) can be adjusted by rotating the image mechanically or electronically. Image erection can be carried out in this way. In the context of the present disclosure, the image erection feature is to be understood as a function in which the orientation (for example the directions above, below, right, left) of the displayed image is largely or completely retained, although the image sensor or the observation optics is rotated ( in the sense of a rolling movement). This definition can also be applied analogously to observation instruments with an inclined line of sight, for example to instruments with lenses whose “normal” is not parallel to the axis of rotation. In the case of a lens, the “normal” is roughly the optical axis. For example, in the case of an image sensor that is aligned concentrically to the lens, the “normal” is approximately perpendicular to the sensor plane. The optical axis of the objective is normally essential for defining the viewing direction.

However, it is also conceivable not to straighten the image so that the displayed image rotates with a rotation or rolling movement of the instrument. Corresponding playback modes can optionally be selected.

In the case of an instrument designed for stereo observation, the erection of the image poses various challenges. The main reason for this is the so-called stereo basis, i.e. an offset (distance or angle) between the two observation channels or image sensors that are combined for stereo observation and stereo reproduction. The stereo base is ideally aligned with the reference horizon of the playback device, ideally parallel or almost parallel to it. The reference horizon of the playback device usually corresponds to a horizontal line. An observer with a straight head aligns his eyes (mental line through the two eyes) parallel to this.

A general design goal for imaging systems in the medical or industrial environment is to make the respective probe part small. This can also include miniaturization of the image capture unit. This applies, for example, to a diameter of the probe or the objective assembly. This design goal is all the more important for endoscopic instruments that are used in neurosurgery (brain surgery, spinal surgery, etc.). However, this makes mechanical solutions for image erection difficult.

In general, it is desirable to design shaft assemblies of endoscopic instruments with the smallest possible shaft diameter. This can minimize trauma to the patient in creating the access opening. Therefore, this design goal applies generally to instruments for minimally invasive surgery.

At the same time, there is also a need for stereoscopic image acquisition or observation in the case of instruments that are as small as possible (with regard to the shaft diameter, for example). One approach for a stereoscopic instrument, for example an endoscope, can be to arrange two image sensors at or near the distal end of the instrument. The two image sensors are regularly arranged next to one another and therefore have an effect on the overall diameter of the instrument in this area. It would be very costly to provide additional provisions for mechanical image erection in this distal area.

At the same time, such instruments are also often rotated about the longitudinal axis of the shaft during use. This applies in particular to instruments with an oblique line of sight. As already indicated above, however, this entails challenges in stereoscopic perception, at least if the image is to remain upright even during a rolling movement. Conversely, if you do not stand upright, from the observer's point of view, orientation “in the picture” is made more difficult.

Instruments with only one image sensor, i.e. with only one observation channel (monoscopic instruments) are less problematic in this regard. The captured image can be erected mechanically (actually by hand or via an actuator) as well as electronically (digital erection). The mechanical erection includes a rotation of the sensor in relation to the shaft or a bearing piece of the instrument. The electronic image erection includes image processing measures so that, if necessary, it can be ensured that the image is reproduced upright.

Against this background, the present disclosure is based on the object of specifying a stereo imaging system with an observation instrument and a method for stereo observation which allow a practically perceptible reproduction even when the instrument or the image acquisition unit of the instrument is rotated, the reproduced image can be erected if necessary. Nevertheless, a stereoscopic display (or 3D display) should be made possible in the largest possible rotation range. Intuitive operation is preferably possible. In a state in which no stereoscopic acquisition and reproduction is sensible, the imaging system or the method preferably still allows a basic functionality, so that at least one observation channel can be used.

Furthermore, within the scope of the present disclosure, a corresponding computer program for controlling a stereo imaging system is to be specified.

• - ein Beobachtungsinstrument mit einer Bilderfassungseinheit zur Erfassung erster Bilddaten und zweiter Bilddaten, welche zur Stereo-Beobachtung kombinierbar sind,
• - zumindest einen Lagesensor zur Erfassung einer Orientierung, insbesondere einer Drehorientierung, des Instruments in Bezug auf eine Lagereferenz,
• - eine Steuereinrichtung, die in Abhängigkeit von der Orientierung des Instruments in einem ersten Wiedergabemodus und einem zweiten Wiedergabemodus betreibbar ist, wobei die Steuereinrichtung dazu ausgebildet ist, mit dem Bildsignal ausgegebene Bilder im zweiten Wiedergabemodus in Abhängigkeit von der Orientierung aufzurichten, wobei die Steuereinrichtung zur Ausgabe eines Bildsignals ausgebildet ist, das im ersten Wiedergabemodus ein Stereo-Signal auf Basis der ersten Bilddaten und der zweiten Bilddaten umfasst, und im zweiten Wiedergabemodus ein Mono-Signal auf Basis der ersten Bilddaten oder der zweiten Bilddaten umfasst, und wobei die Steuereinrichtung in einem ersten Drehwinkelbereich des Instruments im ersten Wiedergabemodus und in einem zweiten Drehwinkelbereich des Instruments im zweiten Wiedergabemodus betreibbar ist.

According to a first aspect, this object is achieved by a stereo imaging system, in particular a medical imaging system, which has the following:

• - an observation instrument with an image acquisition unit for acquiring first image data and second image data, which can be combined for stereo observation,
• - At least one position sensor for detecting an orientation, in particular a rotational orientation, of the instrument in relation to a position reference,
• A control device which can be operated in a first playback mode and a second playback mode as a function of the orientation of the instrument, the control device being designed to erect images output with the image signal in the second playback mode as a function of the orientation, the control device for outputting an image signal is formed, which in the first reproduction mode comprises a stereo signal based on the first image data and the second image data, and in the second reproduction mode comprises a mono signal based on the first image data or the second image data, and wherein the control device in a first Rotation angle range of the instrument can be operated in the first playback mode and in a second rotation angle range of the instrument in the second playback mode.

The object of the invention is completely achieved in this way.

According to the invention, on the one hand, the ability for stereo observation is used. This applies in any case in such positions (especially rotational positions) of the image acquisition unit in which the orientation of the stereo base does not deviate from the position reference (e.g. from the horizon or artificial horizon) or only within reasonable limits.

However, if the signals provided by the position sensor regarding the orientation (rotational position or roll position) show that the stereo functionality can no longer be used due to the rotation of the image acquisition unit, then the control device can output a mono signal (use of only one observation channel) . This can be the case when the operator can no longer orient himself well in the image, for example when controlling surgical instruments that are visible there.

The above measure has the advantage that image erection, in particular electronic image erection, is made possible, with stereoscopic reproduction based on two observation channels or monoscopic reproduction based on one observation channel being possible depending on the current angle of rotation.

This means that in certain orientations of the observation instrument or of its image capture unit, only a two-dimensional reproduction without depth impression is used. At the same time, this type of reproduction can also be used for observation.

If, however, it is determined via the position sensor that the current rotational orientation (roll position) of the image acquisition unit enables stereoscopic reproduction, i.e. reproduction with depth impression (3D reproduction), then a switch can be made to stereoscopic reproduction. With regard to the image erection, it is conceivable, when the image acquisition unit is rotated by about 180 °, to interchange the two observation channels for the reproduction and to rotate their image information by 180 ° in each case. In this way, too, an erection of the image can be realized at least for a small angle of rotation range (around 180 ') even with stereoscopic reproduction.

According to this approach, at least in exemplary configurations, an instrument can be implemented in which, on the one hand, stereo observation and, on the other hand, electronic image erection is possible. This relates to exemplary embodiments and is not to be understood as restrictive.

At the same time, this approach is particularly suitable for instruments with a small shaft diameter, that is to say for instruments for minimally invasive surgery, in particular for neurosurgery (brain surgery, spinal surgery). In exemplary embodiments, the observation instrument is a medical observation instrument. The smaller the shaft diameter, the more difficult it becomes to implement mechanical solutions for image erection.

The observation instrument is usually designed as a video endoscope or video exoscope. A video endoscope is designed to observe the inside of the body and can be inserted into the inside of the body through a natural or artificial opening. A video exoscope is designed to observe the body from outside the body. The instruments can be designed for human medicine and, if necessary, for veterinary medicine. At the same time, industrial applications for video endoscopes and video exoscopes are also conceivable.

The position reference to which the position signal determined by the position sensor relates can, for example, be a horizon (possibly an artificial horizon). The position reference results, for example, from the interpupillary distance (or the orientation of the associated vector) of the observer when viewing the reproduced image.

By way of example, the image acquisition unit comprises two image sensors that are spaced apart from one another. These can be parallel to one another but also slightly inclined relative to one another. One of two observation channels is assigned to each of the two image sensors (right and left). The two image sensors can be combined with one another for stereo observation of an object in an object plane. The two image sensors are spaced from one another and are accordingly adapted to the disparity of the eyes of an observer.

This refinement is not to be understood as restrictive. In general, the stereo basis results from distal apertures of the objective unit or their distance from one another. This is also decisive for the position of the image sensors and their distance.

In the case of mono playback or 2D observation, only one of the two channels, i.e. only one of the two image sensors, is used. This does not necessarily mean that only one of the two image sensors actually delivers a signal. However, only the signal from one of the two channels is used at least when the image is reproduced.

The orientation of the instrument is detected via the position sensor in relation to a reference, for example in relation to a horizon. The reference can be a general (global) reference or a reference that is defined on a case-by-case basis. Usually, a horizon is set up that reflects the position and orientation of the two eyes of an observer.

In the first playback mode, the first image data and the second image data are combined in such a way that stereo playback (3D playback) is possible. In the second playback mode, either the first image data or the second image data are used, so that mono playback (2D playback) is possible.

At least according to exemplary configurations, the observation instrument is a hand-held or hand-guided instrument. However, this is not to be understood as restrictive. According to an alternative embodiment, the observation instrument is held on a tripod or the like. The orientation of the instrument, in particular its roll orientation, can be changed directly via the operator and / or via a motor-driven mechanism. The method according to the disclosure can be designed accordingly.

• - Bereitstellung eines Beobachtungsinstruments mit einer Bilderfassungseinheit zur Erfassung erster Bilddaten und zweiter Bilddaten, welche zur Stereo-Beobachtung kombinierbar sind,
• - Erfassung einer Orientierung, insbesondere einer Drehorientierung, des Instruments in Bezug auf eine Lagereferenz, und
• - Betreiben des Bildgebungssystems in Abhängigkeit von der Orientierung des Instruments in einem ersten Wiedergabemodus oder einem zweiten Wiedergabemodus, umfassend:
  • im ersten Wiedergabemodus Ausgabe eines Bildsignals, das ein Stereo-Signal auf Basis der ersten Bilddaten und der zweiten Bilddaten umfasst,
  • im zweiten Wiedergabemodus Ausgabe eines Bildsignals, das ein Mono-Signal umfasst, und
  • bedarfsweises Aufrichten der auszugebenen Bilder in Abhängigkeit von der erfassten Orientierung, zumindest im zweiten Wiedergabemodus,
  • wobei das Betreiben des Bildgebungssystems in einem ersten Drehwinkelbereich des Instruments im ersten Wiedergabemodus und in einem zweiten Drehwinkelbereich des Instruments im zweiten Wiedergabemodus erfolgt.

According to a further aspect of the present disclosure, the object of the invention is achieved by a method for stereo observation, the method comprising the following steps:

• - Provision of an observation instrument with an image acquisition unit for acquiring first image data and second image data, which can be combined for stereo observation,
• - Detection of an orientation, in particular a rotational orientation, of the instrument in relation to a position reference, and
• - Operating the imaging system as a function of the orientation of the instrument in a first playback mode or a second playback mode, comprising:
  • in the first playback mode output of an image signal which comprises a stereo signal based on the first image data and the second image data,
  • in the second playback mode, outputting a picture signal which comprises a mono signal, and
  • if necessary, erecting the images to be output depending on the captured orientation, at least in the second playback mode,
  • wherein the imaging system is operated in a first angle of rotation range of the instrument in the first playback mode and in a second angle of rotation range of the instrument in the second playback mode.

In this way, too, the object of the invention is completely achieved.

The method can also be referred to as an imaging method for stereo observation. It goes without saying that the method can be configured in accordance with the exemplary configurations of the imaging system described in the context of this disclosure, and vice versa.

In an exemplary embodiment, the method comprises the provision of a control device for controlling the imaging system.

In a further exemplary embodiment of the method, the method comprises the provision of a stereo imaging system with an observation instrument designed for stereo observation, with at least one position sensor for detecting an orientation, in particular a rotational orientation, of the instrument, and with a control device which, as a function of the orientation of the instrument can be operated at least in a first playback mode or in a second playback mode, the control device being designed to output an image signal which, in the first playback mode, comprises a stereo signal based on the first image data and the second image data, and in the second playback mode comprises a mono signal on the basis of the first image data or the second image data, and wherein the control device is designed to erect images output with the image signal in the second reproduction mode as a function of the orientation.

In the first playback mode, the output images, at least in exemplary configurations, are output not erected, that is to say are output without being erected. Accordingly, the displayed images would rotate along with the angle of rotation (roll angle) of the instrument with respect to the positional reference.

According to an exemplary embodiment of the imaging system or the method, the control device is designed to align output images in the second playback mode so that the orientation of the displayed image with respect to a display horizon does not change or only changes within defined limits. The result of the alignment is then the reproduction of erected images.

Image erection can be realized in this way. In the context of the present disclosure, the term image erection describes a function in which the displayed image maintains its orientation (top - bottom - right - left), even when the instrument is rotated. In particular, what is known as a rolling movement, that is to say a rotation of the image acquisition unit or one coupled to it, is used Lens assembly turned off. The rotation defines an axis for the rolling motion. For example, the rotation of an endoscope can take place around the longitudinal axis of an instrument shaft. However, flexible or partially deflectable observation instruments are also known in which no rigid longitudinal axis is installed. Furthermore, observation instruments with a shaft and an observation head received thereon are also known, in which the rotation does not take place around the longitudinal axis of the shaft but around an axis through the observation head. This axis is, for example, perpendicular to the longitudinal axis of the shaft.

According to a further exemplary embodiment of the imaging system or of the method, the control device is designed to output images to be output in the first playback mode without being erected, so that changes in the orientation of the instrument are accompanied by changes in the orientation of the images to be output.

The term “not erect” is understood in the context of the present disclosure in such a way that the images to be output are output without being erected, that is to say not erect. Accordingly, for example, an object in the image to be output would also be rotated by 5 ° when the instrument was rotated by 5 ° about its longitudinal axis during playback.

It has been shown that in the first playback mode, under certain circumstances, slight angles of rotation are also acceptable, at which the observer can perceive the stereoscopic effect (3D effect with impression of depth) and at the same time orientation in the image is easy. It is therefore proposed in the context of this embodiment to maintain the stereoscopic display in the first display mode within limited angles of rotation (also referred to as roll angles). This makes the imaging system more tolerant of slight rotation.

The control device can be operated in a first angle of rotation range of the instrument in the first playback mode, wherein the control device can be operated in a second angle of rotation range of the instrument in the second playback mode. For example, the first angle of rotation range includes at least one neutral position of the instrument in relation to the position reference. A rotation angle range is at least a defined subsection of the available rotation angle.

The neutral position includes, for example, a parallel alignment of a connecting line between the two image sensors or the apertures of the two channels with respect to the horizontal (or an artificial horizon). It goes without saying that the neutral position or starting position can also be defined on a case-by-case basis, using suitable operating elements.

By way of example, the neutral position includes an erect state (perpendicular to the stereo base) with respect to the position reference (for example the horizon). The angle of rotation can be recorded based on the neutral position. A roll angle is usually an angle that describes a rotation about a longitudinal axis (of a shaft and / or of the image acquisition unit).

The horizon that defines the neutral position can be adapted during operation at least according to an exemplary embodiment. In this way, it is possible, for example, to react to global movements while taking along the observation instrument. This relates, for example, to repositioning the patient while taking the observation instrument with him. The adaptation of the artificial horizon can be used to largely maintain the position and orientation of further instruments in the reproduced image if the patient and the observation instrument are rotated, but these further instruments are not.

Conceivable angles of rotation can be defined on the one hand in degrees (in relation to a 360 ° full circle) or in hours and minutes in relation to a 12 o'clock clock face. A neutral position accordingly corresponds to an alignment of 0 ° or 0 o'clock. In contrast, a position rotated by 180 ° corresponds to an alignment of 180 ° or 6 o'clock. Intermediate positions for the angle of rotation result accordingly. In an exemplary embodiment, the first angle of rotation range and the second angle of rotation range add up to 360 °.

For example, the first angle of rotation range comprises a range between 11 o'clock and 1 o'clock or 330 ° and 30 °. Accordingly, the second angle of rotation range can include a range between 1 o'clock and 11 o'clock or between 30 ° and 330 °. Accordingly, the instrument would be completely rotatable about the longitudinal axis.

However, it is also conceivable not to make the instrument completely rotatable about its longitudinal axis (or to make the image erect only for such a partial area). As an example, a partial range between 9 o'clock and 3 o'clock or 270 ° and 90 ° is taken into account. Accordingly, the first angle of rotation range can furthermore include, for example, a range between 11 o'clock and 1 o'clock or 330 ° and 30 °. However, the second rotation angle range then has (partial) sections between 9 o'clock and 11 o'clock and between 1 o'clock and 3 o'clock (corresponding to a first section between 270 ° and 330 ° and a second section between 30 ° and 90 °).

According to a further exemplary embodiment of the imaging system or the method, the first angle of rotation range has two sections offset from one another by 180 °. If the instrument, starting from the neutral position, is rotated by 180 °, the image acquisition unit or the two image sensors for the first and second image data are again aligned parallel to the position reference (horizon), rotated. The image sensors are reversed. Such a rolling movement of about 180 ° is generally referred to as a flip. Stereo playback is also possible in the section offset by 180 °. This can also include a (static) erection of the image. To erect the image, the image information (first and second image data) for the first observation channel and the second observation channel would have to be swapped (swap right with left, and vice versa), and the partial images (right and left) would each have to be rotated by 180 °. Nevertheless, in addition to this “static” image erection, there is no continuous erection with small rotations starting from 180 °.

Accordingly, the first angle of rotation range comprises a range with two sections. A first section is given, for example, between 11 a.m. and 1 a.m. or 330 ° and 30 °. A second section is given, for example, between 5 a.m. and 7 a.m. or between 150 ° and 210 °. Accordingly, the second angle of rotation range comprises a first section and a second section. The first section covers a range between 1 o'clock and 5 o'clock or between 30 ° and 150 °. The second section covers a range between 7 a.m. and 11 a.m. or between 210 ° and 330 °. In the area between 11 a.m. and 1 a.m. (or 330 ° and 30 °) and between 5 a.m. and 7 a.m. (or 150 ° and 210 °), stereoscopic observation and image reproduction is then possible. It goes without saying that the above information for the first angle of rotation range and the second angle of rotation range are to be understood as exemplary and not restrictive. The first angle of rotation range can be aligned symmetrically to the perpendicular (0 ° position). Furthermore, however, angular ranges that are not symmetrical with respect to the perpendicular are also conceivable.

According to a further exemplary embodiment of the imaging system or the method, the control device is designed to swap the first image signal and the second image signal and to rotate the first image signal and the second image signal by approximately 180 °.

This is preferably done in the section of the first angle of rotation range which is offset from the neutral position by 180 °. In this way, static image erection with a 3D function can also be provided for a flip.

Accordingly, the first reproduction mode according to this embodiment comprises two modes. The first mode of operation relates to an initial alignment in accordance with the position reference (which defines the neutral position, for example). The second operating mode concerns the flip mode (rotation by 180 ° for this). Accordingly, the first section can also be referred to as a reference section and the second section as a flip section.

According to a further exemplary embodiment of the imaging system or of the method, the second angle of rotation range includes at least one position of the instrument rotated by 90 ° with respect to the position reference. This also applies, for example, to a position rotated by 270 °. In any case, in such an area (roll angle of 90 ° or 270 °), the stereo functionality cannot be used without further ado if an erect image is desired. It is therefore justifiable to use only one observation channel in this area in order to output a 2D image. This image can then, at least in exemplary embodiments, be electronically rotated in order to straighten the image if necessary.

According to a further exemplary embodiment of the imaging system or of the method, the first angle of rotation range, based on an angle scale, at which 0 ° (degrees) describes an ideal alignment of the instrument with respect to the position reference, comprises a first section, which is a range with a first boundary between 310 ° and 350 ° and the second limit between 10 ° and 50 °, and wherein the first angle of rotation range preferably has a second section offset by 180 ° with respect to the first section.

By way of example, the first angle of rotation range is arranged symmetrically with respect to a vertical (at 0 °). Accordingly, the first angle of rotation range includes, for example, a range between +/- 10 ° (plus / minus 10 °) and +/- 50 ° with respect to the vertical, i.e. covers 0 ° +/- 10 ° for a small area and 0 ° + for a large area / - 50 °. For example, the first angle of rotation range covers a range of 0 ° +/- 45 °. In a further example, the first angle of rotation range covers a range of 0 ° +/- 30 °. Accordingly, the second section can cover a range from 180 ° +/- 10 ° up to 180 ° +/- 50 °, for example 180 ° +/- 30 ° or 180 ° +/- 45 °.

Viewed in absolute terms, in one exemplary embodiment the first section of the first angle of rotation range extends between 330 ° and 30 °, the second section of the first angle of rotation range extending between 150 ° and 210 °. The second angle of rotation range in which the instrument can be operated in the second playback mode can extend complementarily to the first angle of rotation range, that is to say, for example, with a first section between 30 ° and 150 ° and a second section between 210 ° and 330 °.

In a further exemplary embodiment, the first section of the first angle of rotation range extends between 315 ° and 45 °, the second section of the first angle of rotation range extending between 135 ° and 225 °. Accordingly, the second angle of rotation range in this exemplary embodiment comprises a first section between 45 ° and 135 ° and a second section between 225 ° and 315 °.

In a third exemplary embodiment, the first section of the first angle of rotation range extends between 345 ° and 15 °, the second section of the first angle of rotation range extending between 165 ° and 195 °. Accordingly, the second angle of rotation range in this exemplary embodiment comprises a first section between 15 ° and 165 ° and a second section between 195 ° and 345 °.

It goes without saying that other exemplary embodiments for the first angle of rotation range, that is to say for the first playback mode, and the second angle of rotation range, that is to say the second playback mode, are also conceivable.

The first angle of rotation range, that is to say the range with stereo reproduction but without comprehensive image erection, is selected, for example, in such a way that the operator can still orientate himself well in the image, which is often the case with a slight inclination. There is the advantage of stereo playback. Depending on the specific use of the imaging system or depending on the objective and subjective preferences of different operators, the first angle of rotation range or the second angle of rotation range can be defined.

It is conceivable to fix the first angle of rotation range. It is also conceivable to specify a small number of variants for the first angle of rotation range (about +/- 30 ° or +/- 45 °, each related to the vertical - at 0 ° and possibly at 180 °), between which the operator can choose can. It is also conceivable to give the operator the option of freely defining the first angle of rotation range and thus also the second angle of rotation range within wide limits.

In general, the sections can include areas of at least 355 ° to 5 ° (that is to say at least +/- 5 °) and of at least 175 ° to 185 °. This includes exemplary embodiments in which the first angular range comprises values between +/- 30 ° and +/- 45 °, in each case based on a 0 ° position and possibly a 180 ° position. However, this is not to be understood as restrictive. Other angular ranges are conceivable. If one of the two rotation angle ranges is defined, the corresponding other rotation angle range can be derived. In this exemplary embodiment, the two rotation angle ranges add up to 360 °. Each of the two rotation angle ranges can be provided twice on the full circle, offset by 180 °.

According to a further exemplary embodiment, the first angle of rotation range comprises sections between 315 ° and 45 ° and between 135 ° and 225 °. The second angle of rotation range thus includes sections between 45 ° and 135 ° and between 215 ° and 315 °. According to a further exemplary embodiment, the first angle of rotation range comprises sections between 330 ° and 30 ° and between 150 ° and 210 °. The second angle of rotation range thus includes sections between 30 ° and 150 ° and between 210 ° and 330 °. However, this is not to be understood as restrictive.

According to a further exemplary embodiment of the imaging system or the method, the control device can be operated to enable an adapted transition when changing between the first playback mode and the second playback mode, the transition preferably being an adaptation between the orientation of the mono signal in the second playback mode and the orientation of the stereo signal in the first playback mode at a switching angle between the first playback mode and the second playback mode.

The transition (also referred to as transition mode) makes the change between the stereo display and the 2D display more harmonious for the operator. Image signals that are output in the first playback mode (stereo) are regularly processed separately for the right and left eye. This can include, for example, fields that show the observation object from two slightly different viewing angles. When changing between the first playback mode and the second playback mode (and vice versa) there is accordingly a change between a display with two different fields and a display with uniform images for the right and left eyes. For example, during the transition, one of the two fields is deliberately faded out, whereupon the other field becomes decisive for both eyes (and vice versa in the other display mode).

According to a further exemplary embodiment of the imaging system or the method, the control device is designed to align images output in the second playback mode dependent on the angle of rotation, so that a transition between the first playback mode and the second playback mode is preferably made with few or no jumps.

In this way, a steady or almost steady transition is made possible without a “jumping” image. There are therefore preferably no sudden changes in the rotational orientation of the image when the instrument is rotated and there is a change between the first and the second playback mode. In other words, at least in exemplary embodiments, a transition between the first reproduction mode and the second reproduction mode with little or no rotation angle jump is made possible. It goes without saying that smaller jumps can also be perfectly acceptable and possible. However, these jumps should not have any adverse effects on the visual orientation of the observer in the displayed image.

This configuration takes into account the fact that, at least in exemplary embodiments, the first playback mode comprises tolerant angular ranges (about 330 ° to 30 ° or even larger) in which the image is stereoscopically captured and displayed in the object plane, in which case (small ) Area, a strict straightening of the image is dispensed with. Stereo playback only makes sense if the displayed image rotates with the rotation of the instrument.

However, if one then leaves the first angular range in which the first display mode is possible, an abrupt transition to the second display mode would possibly result in an immediate straightening of the image with respect to the positional reference. However, this has the consequence that the image would “jump forward” or “jump back” from the observer's point of view. It is therefore envisaged, at least in exemplary configurations, to provide a transfer mode in order to make this transition less abrupt.

According to a further exemplary embodiment of the imaging system or of the method, the control device interpolates the images to be output between the non-erect state and the erected state in the second reproduction mode.

In this way, the transition can be designed uniformly and ideally without jerks or with little jerks by means of suitable interpolation steps. This improves perception and avoids excessive visual stress on the user during perception.

The orientation of the image to be output changes as a function of the angle of rotation of the instrument, and an upright position nevertheless takes place. The interpolation of the images (or the interpolation of the orientation of the images) helps avoid or at least reduce jumps in the transition between the first playback mode and the second playback mode. The interpolation takes place, for example, between an actual rotation during the transfer between the first reproduction mode and the second reproduction mode and a target orientation which corresponds to the (ideally or almost ideally) erected image.

This means that there is no “jump” of the image or no abrupt rotation of the image immediately during the transition between the first playback mode and the second playback mode. Instead, the image is continuously moved or rotated. Thus, on the one hand, the observer perceives the change between the first playback mode and the second playback mode (which also includes a change between 3D and 2D). Furthermore, the first playback mode with the stereoscopic playback can be designed to be sufficiently tolerant, so that the stereo effect is not only possible with an absolutely precise alignment of the image acquisition unit with respect to the position reference.

According to a further exemplary embodiment of the imaging system or the method, the control device is designed to display images to be output in the second playback mode between a switching angle that is assigned to the transition between the first playback mode and the second playback mode, and in which the image is in particular an orientation corresponding to the switching angle has, and to erect a critical angle or critical angle range of the instrument in the second playback mode.

By way of example, the critical angle describes an area in which the reproduced image is completely erect, i.e. in which the orientation of the reproduced image corresponds to the target orientation defined by the position reference or a selected artificial horizon. By way of example, the critical angle is assigned to a rotational orientation of the instrument of 90 ° and 270 °. So there can be two critical angles. It is also conceivable to offer the complete erection of the image in a critical angle range, that is, not only at a specific angular position. Such a critical angle range can, for example, cover a range of 90 ° +/- 15 °. A complementary, second critical angle range can, for example, cover a range of 270 ° +/- 15 °. This means that when the instrument is rotated in this area, the displayed image does not change its upright position or only changes it insignificantly.

According to a further exemplary embodiment of the imaging system or the method, the switching angle, based on an angle scale at which 0 ° describes an ideal alignment of the instrument with respect to the position reference, is 25 ° to 50 °, preferably 30 ° to 45 °. If the first angular range is symmetrical, there is accordingly a further switching angle symmetrical to the vertical, for example in the range between 310 ° and 335 °, preferably at 315 ° to 330 °. In an exemplary embodiment, the switching angles correspond to the limits of the first rotation angle range in the first playback mode.

According to a further exemplary embodiment of the imaging system or the method, the control device is designed to rotate images to be output in the second playback mode between the critical angle or critical angle range and a further switching angle in such a way that the orientation of the displayed image is adapted to the further switching angle when the instrument is rotated towards the wider switching angle.

The further switching angle describes the transition between the second playback mode and the first playback mode in an overhead orientation of the instrument, in view of the at least approximately 180 ° rotated position of the instrument for stereoscopic playback, the first image signal and the second image signal are swapped and each be rotated by 180 °, at least in one exemplary embodiment. The picture is basically upright, with certain angles of rotation being tolerated for robust stereo reproduction.

In an exemplary embodiment, the reproduced image is consequently not shown in a state that is rotated beyond the (first) switching angle. In a first exemplary embodiment, this means that the reproduced image is only shown in a state rotated within +/- 45 ° both in the first reproduction mode and in the second reproduction mode. In a further exemplary embodiment, the reproduced image is only shown in a state rotated within +/- 30 ° both in the first reproduction mode and in the second reproduction mode.

According to a further exemplary embodiment of the imaging system or the method, the further switching angle, based on an angle scale at which 0 ° describes an ideal alignment of the instrument with respect to the position reference, is 130 ° to 155 °, preferably 135 ° to 150 ° , the critical angle being around 90 °. With a symmetrical alignment of the first angular range, there is accordingly a further switching angle symmetrical to the vertical, for example in the range between 205 ° and 230 °, preferably 210 ° to 225 °.

According to a further exemplary embodiment of the instrument, the control device allows a change in the position reference, in particular a change from an ideal position reference that is predetermined by the orientation of the image acquisition unit. The position reference usually results from the arrangement of the two apertures of the image acquisition unit. For example, the instrument is in an ideal alignment in which an observer can stereoscopically perceive an upright image if the two apertures are arranged offset from one another in a horizontal plane (horizontal alignment of the stereo base).

Accordingly, this alignment would also be the reference for the image erection in the second display mode. However, there are also known applications in which an adaptation of the position reference is desired. This can be the case, for example, if the observation object (for example the patient) is repositioned, but if at the same time the association between the observation object and the (erected) image output by the observation instrument should be maintained. An artificial horizon can then be defined, which results from a selected offset (offset angle) relative to the original horizon (for example in the ideal alignment).

In such an operating mode with an adapted artificial horizon, the newly set artificial horizon can now represent the new position reference for the alignment of the instrument. In the second playback mode, the displayed images can be erected taking into account the new position reference. It goes without saying that, in the first playback mode, when the artificial horizon is changed, the physical alignment of the image acquisition unit is not changed. When changing from the second playback mode to the first playback mode, there is a transition between the erected 2D image and the non-erect stereoscopic image. The picture would jump “forwards” or “backwards”, whereby the transition can be designed fluently as described above. Whether the image jumps "forwards" or "backwards" depends on the direction of rotation. Something similar can result if the first playback mode includes, in addition to the first section (neutral position), a second section (flip position) offset from this by 180 °.

For detecting the rotational position / orientation of the image capturing unit, at least one is exemplary Position sensor provided. The position sensor detects, for example, a rolling position (rotation about the longitudinal axis) of the observation instrument and / or the image recording unit. In the case of a stereo endoscope, the position sensor detects a rolling position of the instrument shaft, for example.

Various configurations of the position sensor are conceivable. This can include one or more acceleration sensors. Acceleration sensors can detect relative positions. For example, gyroscopes or similar sensors can be used to record an absolute position / orientation. It goes without saying that the position sensor can comprise at least one discrete sensor. In exemplary embodiments, the position sensor is arranged in the instrument. The position sensor is arranged, for example, in the shaft of the instrument, in the handle or in the observation head.

The position sensor can be arranged at the proximal end or at the distal end of the instrument. If several sensors are installed, they can be spatially distributed. A spatial distribution allows the position or the orientation to be recorded, taking into account the relative orientation between the position sensors.

However, it is also conceivable to design the position sensor at least partially in terms of software. The position sensor does not necessarily have to be a discrete sensor. The position sensor can also be a tracker located on or in the instrument, which is recorded optically by an external navigation system, via magnetic field changes or direct electromagnetic signal transmissions, and which enables the position of the instrument to be calculated. A position of the recorded and reproduced image can also be recorded via image processing operations, in particular when a movement is being tracked. This can include, for example, pattern recognition, so that a position is determined in the image and this position is continuously maintained during the movement of the instrument or the image acquisition unit, at least in the second playback mode.

According to a further exemplary embodiment of the imaging system or the method, the observation instrument is designed as an instrument with an inclined viewing direction. In the context of the present disclosure, an inclined viewing direction is to be understood as meaning, for example, an inclination of an optical axis of the objective with respect to a longitudinal axis (axis of rotation) of the shaft.

Instruments with a viewing direction inclined by 10 °, 15 °, 20 °, 30 ° or 45 °, for example, are known. Such angles of inclination are also conceivable within the scope of the present disclosure. However, this is not to be understood as restrictive. At least in exemplary embodiments, the term inclined viewing direction does not include an absolutely lateral viewing direction (tilted 90 ° with respect to the longitudinal axis). Such instruments are referred to as side-viewing instruments.

Instruments with a variable viewing direction are also known, in which an angle of inclination can be adjusted with respect to the longitudinal axis. An instrument with an inclined viewing direction can also be referred to as an angled viewing instrument. The angle of inclination of the viewing direction includes, for example, ranges between 10 ° and 60 ° with respect to the longitudinal axis.

Instruments with an inclined line of sight pose particular challenges when rolling around the shaft axis when stereoscopic observation is important. Instruments with an inclined line of sight are often deliberately rotated around the shaft axis in order to vary the current field of view or to change the representation of objects in the field of view. In this way, different areas or observation objects can be observed depending on the respective roll angle. However, at least in exemplary embodiments, this means that certain sections of the rotary areas can be recorded stereoscopically, whereas other sections can only be recorded and reproduced monoscopically, at least when an image erection is desired.

If, in stereoscopic observation, the individual images are erected individually, then the orientation of the stereo base would also have to be adjusted depending on the angle of rotation when the instrument is rotated (rolling) so that stereoscopic observation with the right and left eye is possible. Accordingly, an additional combined, internal pivoting movement of the image sensors or the lenses of the image acquisition unit would be required so that the stereo base is adapted to the desired image orientation during playback and the given rotational orientation of the image acquisition unit. However, this is hardly feasible mechanically. In addition, the inclined viewing direction makes it difficult to compose an erect stereo image with any rotational orientation of the image capture unit.

According to a further exemplary embodiment of the imaging system or the method, the observation instrument carries the image acquisition unit, the image acquisition unit having a stereo image sensor or two individual sensors offset from one another.

The sensors of the image acquisition unit are arranged, for example, on or near the distal end of the instrument or on its shaft. According to an alternative embodiment, the sensors of the image acquisition unit are arranged on or near the proximal end of the instrument or on its shaft.

According to a further exemplary embodiment, the imaging system also has a display unit with at least one display, in particular a 3D screen and / or 3D glasses. For example, 3D glasses can be used in combination with a 3D screen. However, 3D glasses are also known which have their own displays. Such display units are referred to, for example, as HMDs (Head Mounted Displays). The method according to the disclosure can use such equipment.

The display is preferably suitable, on the one hand, for reproduction in stereo mode (3D reproduction) using the image data of both observation channels and, on the other hand, also for reproduction in mono mode (2D reproduction) using the image data of only one observation channel.

The object of the present disclosure is also achieved by a computer program that has program code that is designed to cause an imaging system according to one of the embodiments described herein to carry out the steps of the method according to one of the embodiments described herein when the computer program is based on the Control device of the imaging system is executed.

It goes without saying that the features of the invention mentioned above and those yet to be explained below can be used not only in the respectively specified combination, but also in other combinations or on their own, without departing from the scope of the present invention.

• 1: eine gebrochene perspektivische rückwärtige Ansicht eines Beobachtungsinstruments in Form eines Endoskopes;
• 2: eine gebrochene seitliche Teilansicht des Instruments gemäß 1;
• 3: perspektivische vordere Ansicht eines Beobachtungsinstruments in Form eines Exoskopes;
• 4: eine gebrochene seitliche Teilansicht des Instruments gemäß 3;
• 5: eine vereinfachte, schematische Ansicht eines Bildgebungssystems mit einem Beobachtungsinstrument in Form eines Endoskopes;
• 6: eine vereinfachte, schematische Ansicht eines Bildgebungssystems mit Stereo-F u nktionalität;
• 7: eine frontale Ansicht auf eine Bilderfassungseinheit zur Stereo-Beobachtung;
• 8: eine weitere Ansicht der Bilderfassungseinheit gemäß 7 in einer gegenüber 7 rotierten Darstellung;
• 9 bis 14:
  • verschiedene vereinfachte schematische Darstellungen zur Veranschaulichung des Zusammenhangs zwischen einem Darstellungsmodus und einer aktuellen Drehorientierung eines Beobachtungsinstruments;
• 15: eine vereinfachte schematische Darstellung zur Veranschaulichung der Bildkomposition bei der Stereo-Wiedergabe;
• 16: eine weitere Darstellung der Anordnung gemäß 15 zur Veranschaulichung der Bildkomposition bei um 180° verdrehter Lage;
• 17: eine weitere vereinfachte schematische Darstellung zur Veranschaulichung des Wechsels zwischen einem ersten Wiedergabemodus und einem zweiten Wiedergabemodus;
• 18: ein schematisch vereinfachtes Blockdiagramm zur Veranschaulichung einer Ausführungsform eines Verfahrens zur Stereo-Beobachtung mit einem Beobachtungsinstrument; und
• 19: ein weiteres schematisch vereinfachtes Blockdiagramm zur Veranschaulichung einer weiteren Ausführungsform eines Verfahrens zur Stereo-Beobachtung mit einem Beobachtungsinstrument.

Further features and advantages of the invention emerge from the following description of several preferred exemplary embodiments with reference to the drawings. Show it:

• 1 : a broken perspective rear view of an observation instrument in the form of an endoscope;
• 2 : a broken side partial view of the instrument according to FIG 1 ;
• 3 : perspective front view of an observation instrument in the form of an exoscope;
• 4th : a broken side partial view of the instrument according to FIG 3 ;
• 5 : a simplified, schematic view of an imaging system with an observation instrument in the form of an endoscope;
• 6th : a simplified, schematic view of an imaging system with stereo functionality;
• 7th : a front view of an image acquisition unit for stereo observation;
• 8th : a further view of the image capture unit according to FIG 7th in an opposite 7th rotated representation;
• 9 until 14th :
  • various simplified schematic representations to illustrate the relationship between a representation mode and a current rotational orientation of an observation instrument;
• 15th : a simplified schematic representation to illustrate the image composition in stereo playback;
• 16 : a further representation of the arrangement according to 15th to illustrate the composition of the image when the position is rotated by 180 °;
• 17th : a further simplified schematic representation to illustrate the change between a first playback mode and a second playback mode;
• 18th : a schematically simplified block diagram to illustrate an embodiment of a method for stereo observation with an observation instrument; and
• 19th : Another schematically simplified block diagram to illustrate a further embodiment of a method for stereo observation with an observation instrument.

1 illustrates an exemplary embodiment of an observation instrument on the basis of a perspective representation 10 in the form of an endoscope 12th . 2 shows a corresponding partial side view. The observation instrument 10 is exemplary as a stereo endoscope 12th designed. The endoscope 12th is trained to observe the inside of the body.

General are medical and non-medical (technical) applications for endoscopes 12th and similar observation instruments 10 known. The endoscope 12th has a shaft 14th on, of a longitudinal axis 16 Are defined. The shaft exemplarily has 14th a distal end and a proximal End on. In the context of the present disclosure, a distal end is an end which faces away from the observer (operator of the instrument) and which faces the object to be observed. Furthermore, within the meaning of the present disclosure, a proximal end is an end which faces away from the object to be observed and which faces the observer (operator of the instrument).

At the proximal end of the shaft 14th points the endoscope 12th a housing 20th on. The operator can use the endoscope 12th in the area of the housing 20th grab and lead. When used as a medical instrument 10 the shaft can be used to observe the inside of the body 14th of the instrument 10 at least partially inserted into a natural or artificial body opening.

The case 20th shows exemplary controls 22nd and connections 24 , 26th on. The connections 24 , 26th relate, for example, to an electrical supply line, a signal line, a lighting connection and connections for liquids or gases that are required during the medical procedure.

2 that illustrates the instrument 10 for example as an angled view endoscope 12th is designed. At the distal end of the shaft 14th is an image capture unit 30th with a cone of vision / field of vision 32 indicated. The field of view 32 or its center (axis 34 ) is opposite the longitudinal axis 16 inclined. Angle of inclination can correspond, for example, to 30 °, 45 °, 60 ° and the like. Larger or smaller angles of inclination are also conceivable. There are also oblique viewing endoscopes 12th known with adjustable viewing direction (adjustable angle of inclination). The axis 34 is an example of the normal to the sensor surface of the image acquisition unit 30th designed.

Also illustrated in 2 a double arrow denoted by 36 indicates a rotational movement of the instrument 10 around the longitudinal axis 16 of the shaft 14th . Such a movement can be called a rolling movement. The longitudinal axis is accordingly 16 in this embodiment a roll axis. When moving according to the arrow 36 is also the opposite of the longitudinal axis 16 inclined image capture unit 30th rotates. Accordingly, a significantly larger area of an object field can be observed overall through such a rolling movement, even when the instrument 10 otherwise it is not moved.

the 3 and 4th illustrate another observation tool 50 , which in the present case is a so-called exoscope 52 is designed. The instrument is exemplary 50 a stereo exoscope 52 . The exoscope 52 has a shaft 54 with a longitudinal axis 56 on. At the proximal end of the shaft 54 points the exoscope 52 a housing 60 with controls 62 on. Also on the housing 60 , especially at its proximal end, connections 64 , 66 educated. Also the exoscope 52 can be done by the operator in the area of the housing 60 held and guided and positioned in this way. In principle, it is also conceivable that an exoscope 52 on a tripod or the like. It can be a passive tripod (without the possibility of motorized adjustment) or an active manipulator (comparable to a robot, for example). However, it can also be used as a hand-held / hand-held instrument 50 conceivable.

At the distal end of the shaft 54 instructs the instrument 50 an observation head 68 with an image capture unit 70 on. The image capture unit 70 has in the exemplary embodiment a field of view or a cone of vision 72 on, the axis of which is denoted by 74. In the exemplary embodiment according to FIGS 3 and 4th is the axis 74 approximately perpendicular to the longitudinal axis 56 through the shaft 54 aligned. The axis 74 is, for example, a normal to a sensor surface of the image acquisition unit 70 . However, this is not to be understood as restrictive. The axis 74 can also be an optical axis of the optical arrangement of the image acquisition unit 70 regardless of how the image sensor or sensors are aligned for this purpose.

Furthermore, in 4th indicated with 78 a working distance. It goes without saying that the working distance 78 in relation to the other dimensions of the instrument 50 does not necessarily have to be shown authoritatively. General is the exoscope 52 according to the 3 and 4th in contrast to the endoscope 12th according to the 1 and 2 trained to observe an object (patient, technical object) from the outside (outside the body). In this respect, exoscopes are similar to other observation instruments such as microscopes.

In 4th is also at 76 a rotary movement / rolling movement of the image capture unit 70 indicated. It is conceivable, the exoscope 52 to design in such a way that the image capture unit 70 around the axis 74 is rotatable. A different axis of rotation can be given in the case of an oblique view instrument. In the design according to 3 and 4th can the image capture unit 70 in the observation head 68 be rotated. Accordingly, the exoscope will not 52 rotates in its entirety. For the rotation 76 On the one hand, manual actuation and, on the other hand, motorized actuation can be provided.

The working distance 78 of exoscopes is regularly significantly larger than that of endoscopes. The working distance 78 can for example Include ranges from 100 mm (millimeters) up to 500 mm. However, this is not to be understood as restrictive. Usually, exoscopes also have a sufficiently large depth of field, which is significantly greater than the depth of field of endoscopes. For example, the depth of field can include ranges of at least 5 mm, preferably of at least 10 mm, more preferably of at least 20 mm. However, this is also not to be understood as restrictive.

Both when observing with the endoscope 12th as well as when observing with the exoscope 52 it is advantageous if a stereoscopic display is possible. A stereoscopic representation allows a depth impression (spatial impression) and thereby facilitates, for example, navigation with other instruments in the area being observed.

Accordingly, both endoscopes 12th as well as exoscopes 52 with 3D function (in the sense of a stereoscopic observation) known. Such instruments then usually have two image sensors that are offset from one another or one stereo sensor with corresponding sensor areas that are offset from one another. The stereo observation is made possible by two observation channels with two spaced apart apertures, which are adapted to the stereoscopic vision with the right eye and the left eye.

So-called 3D monitors are known for stereo playback, for example, which make a stereoscopic effect usable by using special aids (3D glasses). Furthermore, so-called HMDs (Head Mounted Displays) are known, that is to say video glasses with a stereo effect worn by the observer himself.

However, the stereoscopic observation and reproduction reaches its limits when the instrument rolls, during which the stereo base (imaginary line by which the two sensors / sensor surfaces are offset from one another) changes with respect to a reference horizon. This is particularly the case when an image erection is desired. The reference horizon is usually defined by the arrangement of a display unit and finally by the arrangement of the human eyes (interpupillary distance, disparity).

If the stereo basis of the image acquisition unit no longer corresponds at least approximately to the reference horizon, then the stereoscopic reproduction can no longer be guaranteed with a sufficiently high quality while maintaining the image erection. This applies at least when trying to "artificially" maintain the alignment of the output image. Stereo observation without erecting the image is possible, but orientation in the non-erect image becomes more difficult as the angle of rotation increases. With instruments for stereo observation it is therefore not just possible to implement measures for (digital) image erection. The 3D image is regularly formed on the basis of two observation channels. An image erection (digital / electronic or via a corresponding actuator for rotating the respective sensor) could rotate the respective individual image in the first channel and in the second channel in order to erect it. However, there would then no longer be sufficient correspondence between the reference horizon and the position of the stereo base in large angular ranges. The present disclosure addresses this issue.

With reference to 5 and 6th an exemplary configuration of an imaging system is shown on the basis of two schematically greatly simplified block diagrams 100 with an observation instrument 110 in the form of a stereo endoscope 112 illustrated. The endoscope 112 has a shaft 114 on, of a longitudinal axis 116 Are defined. At the distal end of the shaft 114 sits an image capture unit 130 . In the exemplary embodiment according to 5 is the image capture unit 130 so obliquely with respect to the shaft 114 oriented that the axis 134 opposite the longitudinal axis 116 is inclined. The endoscope 112 also has at the proximal end of the shaft 114 a housing 120 on. The image capture unit 130 is on an observation object 140 aligned. The curved double arrow 136 in 5 illustrates a rolling motion of the instrument 112 around the longitudinal axis 116 .

In the exemplary embodiment is in the housing 120 furthermore a sensor unit 142 added, the example of a first sensor 144 having. With the first sensor 144 it is, for example, one or more acceleration sensors. In this way, changes in position / changes in position of the instrument 110 are recorded. In addition, the sensor unit 142 for example a second sensor 146 provided, such as a gyroscope. The sensors 144 , 146 can be referred to as position sensors. Accordingly, the sensor unit 142 absolute measuring position sensors 146 and relative measuring position sensors 144 exhibit. It goes without saying that the sensor unit 142 also in another position near the instrument 112 can be provided, for example close to the image capture unit 130 . Other configurations of the sensor unit 142 with the at least one position sensor 144 , 146 are easily conceivable.

It is also conceivable the sensor unit 142 or at least one of the position sensors 144 , 146 outside the observation instrument 110 to to implement. Monitoring of the position / orientation of the instrument can be an example 110 can also be implemented via external sensors (optical or electromagnetic tracking of markers, or the like). In principle, a digital implementation of the at least one position sensor is also conceivable. This can include, for example, image processing operations (pattern recognition, motion tracking, etc.). Combined designs for recording the orientation, in particular the current roll position of the instrument or the image recording unit, are easily conceivable.

The instrument 110 is connected to a control device via a signal line (wired or wireless) 150 with at least one control unit 152 coupled. The control device 150 can also be at least partially in the instrument 110 be integrated. The control device 150 however, it can also be designed as a separate control device. The control device 150 can generally be designed as a centralized or decentralized / distributed control device. The control device 150 is designed to be used by the instrument 110 Receive data. These data are based on image information / image signals generated with the image acquisition unit 130 were recorded. The control device 150 is designed for data processing, in particular for processing and preparing image data.

An exemplary embodiment of a control device 150 is sold by the applicant under the name "IMAGE1 S" as a so-called camera platform. In addition to the instrument (endoscope or exoscope), a stereo observation system usually includes such a camera platform and at least one display unit (such as a 3D monitor).

In the design according to 5 is the control device 150 in turn (wireless or wired) via a signal line with a playback unit 160 coupled for stereo playback. The playback unit 160 is designed, for example, as a monitor, in particular as a 3D monitor. It goes without saying that additional equipment, such as suitable glasses, may be required to view a 3D image (stereo image). The playback unit 160 allows the observed object to be reproduced 140 , compare the reference number 164 to name the displayed image. In 5 a positional reference is also indicated at 166. The location reference 166 comprises at least one horizon or a coordinate system on which the reproduced image is located 162 oriented. With suitable alignment between the image capture unit 130 , the observed object 140 and the displayed image 164 on the display 162 a stereoscopic reproduction is made possible.

It goes without saying that the in 5 The arrangement shown also around a light source or other for use with the instrument 110 suitable units can be added.

6th shows in addition to 5 further design features of the exemplary embodiment of the imaging system 100 . The image capture unit 130 is at the distal end of the shaft 114 educated. The image capture unit 130 comprises, for example, a first image sensor 180 and a second image sensor 182 . In the exemplary embodiment shown, these are also at the distal end of the shaft 114 arranged. However, this is not to be understood as restrictive. Designs with image sensors arranged proximally are also conceivable.

The image sensors 180 , 182 is a lens assembly 184 upstream. The lens assembly 184 includes, for example, a cover slip 186 and optical units 188 , 190 with apertures that the image sensors 180 , 182 assigned. The optical units 188 , 190 define the respective field of view of the image sensors 180 , 182 . Each of the two image sensors 180 , 182 is an observation channel 194 , 196 assigned. One of the two observation channels 194 , 196 can be referred to as the right channel and the other as the left channel. Accordingly, one of the two image sensors 180 , 182 can be referred to as the right sensor and the other as the left sensor.

In the embodiment according to 6th the term observation channel does not necessarily refer to an optical observation channel. Instead, the channels are 194 , 196 regularly as channels for the transmission of image data to the control device 150 formed by the right and left image sensors 180 , 182 are recorded. The control device 150 serves to process and prepare the image data. At the output of the control device 150 there are also two output channels 200 , 202 ready, which in turn serves as the right and left output channel 200 , 202 can be designated. If now via both output channels 200 , 202 Image information (for example half images for the right and left eye) are provided, so a display unit suitable for this can be used 160 for stereoscopic playback (3D playback). 6th illustrates such a 3D rendering of the observation object 164 . The object of observation 164 is also related to the location reference 166 aligned.

7th and 8th illustrate a front view of an exemplary embodiment of an image acquisition unit 130 with two image sensors 180 , 182 . In the exemplary embodiment shown, the image acquisition unit is 130 in the shaft 114 of an instrument 110 recorded. However, this is not to be understood as restrictive. The two image sensors 180 , 182 are by a measure 216 at a distance from each other, that of the stereo base (for example distance between the apertures of the optical units 188 , 190 ) is adjusted. The relative position of the two image sensors 180 , 182 also defines a sensor horizon 218 . When the sensor horizon 218 of the two image sensors 180 , 82 with the location reference 166 (in particular with the artificial horizon), an upright 3D display is possible.

8th illustrates one opposite the orientation in 7th twisted state of the image capture unit 130 . The (new) sensor horizon is accordingly 220 compared to the old sensor horizon 218 at a certain angle 222 twisted. However, if the previous sensor horizon 218 parallel to the artificial horizon of the position reference 166 is oriented, this is the case with the new sensor horizon 220 no longer the case. Next to the twisted base 216 it is also clear that there is a height offset designated by 226.

The twist / misalignment in 8th complicates the 3D display, especially when an upright image is desired, i.e. an "upright" playback image 164 even with a rotating image capture unit 130 . This is all the more true for instruments 110 with an oblique viewing direction (in relation to the longitudinal axis of the shaft 114 , which is also the roll axis, for example).

The present disclosure addresses these disadvantages with different display modes. On the one hand, emphasis is placed on the 3D representation - if possible. If, however, based on the given rotational position of the image acquisition unit, no 3D representation appears sensible, the observer is offered a 2D representation. In the 2D mode, in turn, the (digital) erection of the image can be guaranteed. The 2D mode is suitable for applications in which the orientation of the view from the observer's point of view is more important than stereoscopic observation in 3D mode.

In this context, the 9-14 Referenced. the 9-14 illustrate using an angle scale 230 by a position arrow drawn in it 232 the currently selected rotation orientation of the observation instrument 110 . Also show the 9-14 the resulting presentation and orientation for the reproduced image 164 of the observation object at the reproduction unit 160 in terms of a location reference 166 (such as an artificial horizon), provided the image processing with the control device 150 takes place in accordance with an exemplary embodiment of the present disclosure. It goes without saying that the representations of the tilt angle in the 9-14 are merely exemplary in nature. Larger, but also smaller areas with 3D representation are conceivable.

The rotation angle scale 230 covers different areas 240 , 242 , 244 , 246 of which the areas 240 and 244 are assigned by way of example to the first playback mode with 3D display. The areas 242 , 246 are assigned to the second playback mode with 2D representation. The size of each area 240 , 242 , 244 , 246 is exemplary in nature. Between the areas 240 , 242 , 244 , 246 The transition between the different playback modes takes place at a switching angle.

For example, the 0 ° position corresponds to that in 7th Rotation angle state of the image capture unit shown 130 . In the 0 ° position, the horizon corresponds 218 by the two sensors 180 , 182 or which is defined upstream of these observation optics, the artificial horizon (horizontal) of the position reference 166 . This enables a 3D image of the observation object 164 are issued. An erection of the image is not necessary to the extent that the observation object 164 is displayed in the desired orientation even without additional measures.

If now the instrument 110 or the image acquisition unit 130 is rotated, so it is with small rotation angles (compare 10 ) still conceivable, the observation object 164 reproduce stereoscopically (3D reproduction). There is no longer a perfect alignment between the horizon 218 and the location reference 166 . The resulting deviation (compare example 8th ) is however with the in 10 angle of rotation shown is still acceptable. Accordingly, the observation object can 164 continue to be displayed stereoscopically.

In 10 becomes the object of observation 164 with a tilt angle in the area 240 displayed, the one on the scale 230 indicated angle of the pointer 232 is equivalent to. This tilting, i.e. the abandonment of the image erection, is still acceptable within limits. The advantage lies in the preservation of the 3D functionality. In principle, however, it is also conceivable, even in the first playback mode (areas 240 , 244 ) the observation object reproduced in the image 164 to stand up, analogous to the orientation in 9 ). On the one hand, this would have the advantage of straightening the image. On the other hand, however, this can be associated with further impairments in the 3D display. Nevertheless, this is conceivable within acceptable limits.

On the other hand, the main embodiments focus on the fact that the Observation object 164 at least in the area 240 is not straightened up. This is (regardless of the possibly required flip function) in principle also in the opposite area 244 conceivable.

Between the representations in 10 and 11th there is a transfer from the area 240 in the area 242 . In other words shows 10 the representation in the first playback mode (3D). 11th shows the representation in the second playback mode ( 2D) . As already explained above, the electronic image erection can be implemented more easily in the second playback mode.

One would get the picture of the observation object 164 for example with the in 11th indicated angle of rotation position abruptly at the transition between the areas 240 , 242 straighten up, so there would be a noticeable “jump”, comparable to the direct result of the representation of the observation object 164 in 10 and the representation in 12th . Instead, within the scope of the present disclosure, at least in exemplary configurations, the image of the observation object is proposed 164 starting from the "orientation" transferred when changing from the first playback mode to the second playback mode, gently and not abruptly straighten up. Such a transition movement makes the change between the two playback modes more pleasant for the observer. In other words, the reproduced image is interpolated between two orientations (partially rotated and erected).

In a similar way, when changing from the second playback mode to the first playback mode, it is conceivable to correct the image upright somewhat in the 2D display in order to achieve a smooth transition to the expected tilting of the displayed object to be observed 164 to anticipate when entering the first playback mode. Here the image is interpolated between the upright orientation and the partially rotated orientation. Compare with the one below in 13th and 14th described transition.

12th illustrates that then within the area 242 , in which the second playback mode is used, image erection is enabled. The erection of the image can be done with the 2D image of the observation object 164 can be carried out in a manner that is already known in principle. This is how digital image erection is known. Furthermore, the erection of the image by moving the image sensor is known.

The pointer 232 is located in 12th at a critical angle in the range 242 . In the exemplary embodiment, the critical angle is approximately 90 °. In the opposite area 246 , which is also assigned to the second playback mode, another critical angle is approximately 270 °, for example. At least when the instrument has been rotated towards the critical angle, that is to say by approximately 90 ° or by 270 °, the exemplary embodiment provides for complete image erection. The object of observation 164 is perfectly or at least approximately aligned with respect to the position reference at the critical angle. Between the orientation at the switching angle (transition between the range 240 and the area 242 , compare also 11th ) and the orientation at the critical angle according to 12th can be a gradual, angle-dependent erection of the observation object 164 take place.

Based on the orientation in 12th illustrated 13th a state in which the pointer 232 although still in the area 242 is, but the area 244 approaching. Starting from the second playback mode, there is a transition to the first playback mode. Hence, in 13th the orientation of the reproduced observation object 164 already adapted to the expected orientation when entering the first playback mode.

14th shows such a state. The instrument 110 or the image acquisition unit 130 is rotated almost 180 °. Nevertheless, a 3D representation is now possible again. Furthermore, the reproduced object is observation 164 at least at the correct height in relation to the original arrangement ( 9 ) aligned. Based on the given orientation of the instrument or the image acquisition unit 130 so finds in 14th a so-called image flip takes place, comprising, on the one hand, a rotation of the individual images by 180 ° and, on the other hand, an exchange of the two channels.

Overall, even when the instrument or the image capturing unit is rolling, the observer can see an image that can be captured easily and has a comprehensible orientation. The 3D display can always be used if this appears possible under the given conditions. It can be switched automatically between 3D and 2D. Manual switching is also conceivable, at least as an additional option.

It goes without saying that the sequence of the representations in 9 until 14th concerns an instrument with a straight line of sight. However, the above statements can also be transferred to instruments with an oblique / inclined viewing direction. It goes without saying that, in the case of instruments with an oblique viewing direction, the field of view migrates in such a way that different objects to be observed appear in the field of view when the instrument is rotating. Regarding the alignment, however, the above statements can be transferred to this. The benefit arises at the latest when in addition to Observation instrument other instruments (pliers, tweezers and the like) can be used and appear at least partially in the field of vision.

Based on the schematically greatly simplified representation of the 15th and 16 becomes the aforementioned image flip when the instrument is rotated 110 described by 180 °. In 15th and 16 is the instrument 110 each in one of the areas 240 , 244 in which a 3D rendering is useful and desired. In 15th there is a state that is approximately in accordance with the state 9 is equivalent to. Each of the two sensors 180 , 182 captures an image for the two channels 194 , 196 . The control device 150 includes a block that connects the two channels 194 , 196 schematically the output channels 200 , 202 assigns. In the configuration according to 15th already has the recorded image of the two sensors 180 , 182 the desired orientation. Accordingly, only the signal from the two sensors 180 , 182 can be combined to form a 3D representation of the reproduced observation object 164 to enable.

In contrast, the two sensors record 180 , 182 due to the rotation by 180 ° in the meantime, the image according to the configuration in 16 upside down. Now, however, the image data of both channels would 194 , 196 just be rotated (compare the blocks 250 , 252 ), then the first channel and the second channel would still be swapped. Therefore, in the embodiment according to 16 suggested the two (partial) images of the two sensors 180 , 182 on the one hand to rotate in order to erect the respective individual image. Furthermore, in this exemplary embodiment, the signal of the two channels 194 , 196 when processing and forwarding to the output channels 200 , 202 exchanged (crossed), exemplarily illustrated by the block 254 . This is how the image flip (also referred to as a 180 ° flip) is implemented. In the area 244 an approximately upright display in 3D mode is possible, even if the instrument 110 or the image acquisition unit 130 rotated by 180 °. A 3D representation is also desired near the 180 ° position.

It goes without saying that the blocks 250 , 252 , 254 functional blocks of the control device 150 could be. The blocks 250 , 252 , 254 can be implemented in terms of software and / or hardware. The blocks 250 , 252 , 254 can be designed as discrete blocks with specific individual functions or as universal function blocks.

Based on the 9-16 illustrated functionality of the instrument 110 can through the control device 150 can be controlled in an assigned operating mode. In an exemplary embodiment, the instrument is 110 with the control device 150 also enabled for other operating modes.

A first operating mode includes, for example, operating the instrument 110 in a pure 2D mode, that is, without stereoscopic reproduction, and in which image erection is also dispensed with. The displayed image is therefore analogous to the rotation of the instrument 110 or rotate the image capture unit. A second operating mode includes, for example, operating the instrument 110 in a stereo mode with stereoscopic playback. This goes hand in hand with the fact that the image is not erected.

A third mode of operation includes operating the instrument 110 in 2D mode, with continuous straightening in relation to a reference horizon. Ideally, the rotational orientation of the displayed image therefore does not change when the instrument is rotated. For the sake of completeness, it should be noted that the image content (the field of view) changes with angled-view instruments when the instrument is rotated. However, the displayed image retains its rotational orientation. The fourth operating mode is the combined operation with stereo playback (first playback mode) and 2D playback (second playback mode), the image being at least partially erected at least in the second playback mode.

With reference to 17th is based on an angle scale 330 , which in principle according to the angle scale 230 according to the 9-16 corresponds, illustrates the transition between the first playback mode and the second playback mode as well as an image transformation (interpolation) conceivable in the second playback mode as part of the image erection. The angle scale 330 represents a full circle, which corresponds to a complete rotation around the roll axis. There are sub-areas 340 , 344 provided, which are assigned to the first playback mode. There are also sub-areas 342 , 346 provided, which are assigned to the second playback mode. The respective transitions are switching angles 350 , 352 , 354 , 356 assigned. The switching angle 350 , 352 , 354 , 356 are for example 45 °, 135 °, 225 ° and 315 °. In an alternative, in 17th embodiment not shown are the switching angles 350 , 352 , 354 , 356 at 30 °, 150 °, 210 ° and at 330 °.

The pointer 360 illustrates the rotational position of the instrument 110 or from its image acquisition unit 130 when rotating in the first playback mode (areas 340 , 344 ) and in the second playback mode (areas 342 , 346 ). In 17th are also critical angles 362 , 364 in the respective sub-area 342 , 346 of the second playback mode. The critical angles are exemplary 362 , 364 respectively in the middle of the sub-area 342 , 346 . Thus the critical angle is 362 for example at 90 °. The critical angle 364 is, for example, at 270 °.

17th also contains a corresponding diagram 370 , which is a development of the angle scale 330 includes. One axis 372 describes the current angle of rotation / roll angle of the instrument 110 or from its image acquisition unit 130 and thus corresponds to the respective position of the pointer 360 when rotating. Another axis 374 describes a resulting tilt angle, i.e. a resulting rotational orientation of the displayed image.

In the embodiment according to 17th cover the sub-areas 340 , 344 for example ranges from 0 ° +/- 45 ° and 180 ° +/- 45 °. Stereo playback is conceivable in these areas; the instrument can be operated in the first playback mode. No extensive electronic image erection is provided in the first reproduction mode. Accordingly, the tilt of the displayed image with respect to the position reference (artificial horizon) develops proportionally or even directly proportionally to the angle of rotation of the instrument 110 or from its image acquisition unit 130 . This is shown in the diagram 370 illustrated by the line labeled 380, compare the sections between 0 ° and 45 °, between 135 ° and 225 °, and between 315 ° and 360 °. So if the instrument 110 is rotated by 30 ° with respect to the position reference, the reproduced image is also rotated by 30 °.

The instrument is in the section between 135 ° and 225 ° 110 essentially turned upside down in its rotational orientation, so that the reproduced image is analogous to that in 16 illustrated embodiment can be adapted to enable a static quasi-erecting. The image of the two observation channels is then also turned upside down so that the output image has at least approximately the desired orientation.

In the other sub-areas 342 , 346 on the other hand, if a 2D representation based on an observation channel is used, compare the sections between 45 ° and 135 ° and between 225 ° and 315 °. In the second playback mode, electronic image erection is possible. This allows the orientation to be adapted or the orientation of the displayed image to be decoupled from the current angle of rotation of the instrument. In exemplary embodiments, however, the output image in the second reproduction mode is not directly and constantly set up to an ideal orientation. Such an ideal orientation corresponds, for example, to the 0 ° position on the axis 374 in the diagram 370 . Such a function would have the consequence that the picture immediately when passing one of the switching angles 350 , 352 , 354 , 356 rotates suddenly, in the exemplary embodiment by 45 ° with respect to the 0 ° deflection. This can be particularly true when operating in the vicinity of the respective switching angle 350 , 352 , 354 , 356 are perceived as disadvantageous.

In order to counteract such "jumping" in the displayed image, it is proposed to interpolate the orientation of the image in the second display mode as a function of the angle so that, on the one hand, a sufficiently stable upright position and good orientation in the image is possible and, on the other hand, jumps in rotation angle are minimized or avoided.

The curves 382 , 384 , 386 illustrate exemplary angular orientations of the displayed image as a function of the real rotational orientation of the instrument 110 or the image acquisition unit 130 in the second playback mode. These curves / lines 382 , 384 , 386 are in the sub-areas 342 , 346 of the second playback mode. The curves 382 , 384 , 386 represent a transition between the separated areas in the second playback mode 340 , 344 of the first playback mode, in particular without a jump in the angle of rotation.

The curve 382 behaves essentially proportionally or inversely proportional to the current angle of rotation of the instrument 110 . For example, the diagram extends 370 a linear section between the 45 ° position at the switching angle 350 and the 135 ° position for the switching angle 352 . As the angle of rotation of the instrument increases 110 the displayed image is rotated in the opposite direction as the angle of rotation increases. The situation is similar between the switching angles in the exemplary embodiment 354 and 356 , i.e. between 225 ° and 315 °. A boundary condition in the embodiment according to 17th is the passage through the 0 ° position at the limit angles 362 , 364 , i.e. at 90 ° and at 270 ° rotational position of the instrument. This results in the symmetrical, in particular mirror-symmetrical with respect to the vertical, design of the angular ranges 340 , 342 , 344 , 346 roughly in the middle of the respective sub-area 342 , 346 of the second playback mode, a complete erection of the displayed image.

The curve 384 is basically based on the course of the curve 382 ajar. The course of the curve 384 however, follows the goal, not just directly at the critical angles 362 , 364 but also to carry out a complete straightening of the image in their surroundings (in the example approx. +/- 20 °). This results in an area in which the displayed image is sufficiently stable and is not rotated or rotated only insignificantly. Nevertheless, the curve contains 384 Ramps that allow a smooth transition to the first playback mode.

The curve 386 is designed, for example, as a spline (polynomial train), with the basic course on the curve 384 oriented. In this way, "kinks" can be avoided when going through the curve.

It goes without saying that the curves 382 , 384 , 386 for the second playback mode with the curve 380 for the first playback mode can be combined to implement the desired behavior in the first and second playback modes.

With reference to 18th an exemplary embodiment of a method for stereo observation is illustrated using a schematic block diagram, in particular a method for stereo observation with image erection, at least with partial image erection.

The method includes a first step S10 , which concerns the provision of an observation instrument with stereo functionality. The instrument can be, for example, a stereo endoscope or a stereo exoscope. The instrument is usually provided with an image acquisition unit which can acquire first image data and second image data. For this purpose, the image acquisition unit can have a first sensor and a second sensor offset therefrom. In this way, two observation channels (right and left) are formed. In this way, the first image data and the second image data can be combined for stereo observation. However, this brings challenges with the desired image alignment when the instrument is rotated, so that the stereo basis of the instrument changes with respect to an (ideal) reference horizon.

There follows a step S12 which includes position monitoring or detection of a rotational angle position (roll position) of the instrument or of its image acquisition unit. The recorded position or the recorded rotation indicates whether the given orientation of the image recording unit makes a 3D display without image erection useful, or whether a switch should be made to a 2D display using only one of the two observation channels, in order to achieve an electronic image erection to be able to use.

Depending on the detected angle of rotation, a step S14 in which a first playback mode is activated. The first playback mode includes 3D playback. Alternatively, a step S16 in which a second playback mode is activated. The second playback mode includes 2D playback.

The step S18 a step follows S18 , which includes a reproduction using both observation channels (right and left) for 3D reproduction with an impression of depth. At least in exemplary embodiments of the method, it is provided that the reproduced image of the observation object is not continuously erected in the first reproduction mode. This rotates the displayed image together with the rotation of the instrument. However, since the first playback mode can only be used meaningfully in limited rotation angle ranges, the rough image alignment is still correct, so that an orientation in the image is possible. Nevertheless, the first playback mode has the advantage of stereo observation.

The first playback mode usually also includes a state in which the instrument or its image acquisition unit is rotated by approx. 180 °. Then namely the stereo base (for example defined by the position or orientation of the apertures of the observation optics) is again aligned parallel or almost parallel to the reference horizon. So that the displayed image is not upside down, a so-called 180 ° flip takes place. This includes, for example, a 180 ° rotation of the two channels and an exchange (right with left, and vice versa). In this way, a quasi-straightening in the state rotated by 180 ° can be realized together with the 3D representation. However, there is no continuous, closely tracked image erection.

However, if the method according to step S16 executed in the second playback mode, then in the following step S20 only one of the two observation channels is provided or used for playback. This has the advantage that there is no longer any need to take the stereo basis into account, since, to a certain extent, only a mono signal is processed for output.

Accordingly, the image to be output can be digitally / electronically erected. The image is erected in step S22 . Depending on the rotational angle position that is still recorded, tracking or continuous image erection can now be provided. For the observer, the general alignment of the image does not change or only changes within defined limits when the instrument is rotated.

The process concludes in step S24 with the reproduction of the image, in the exemplary embodiment either a 3D reproduction without direct image erection, except when the instrument is aligned upside down, or a 2D reproduction with image erection.

With reference to 19th an exemplary embodiment of a method for stereo observation is described on the basis of a further block diagram. In principle, the method can also be designed and described as a method for controlling an imaging system.

The process starts with one step S50 relating, for example, to activation of the imaging system. There follows a step S52 , which relates to the acquisition of a current rotational position (rolling position) of an instrument or of its image acquisition unit. At least one corresponding sensor can be provided for this purpose. The acquisition allowed in step S54 monitoring of the detected angle. In step P.56 it is determined in which of three (global) angular ranges the instrument or its image acquisition unit is currently located. For example, a position in relation to a reference horizon is used as a basis, with the stereo base being able to be used as an instrument-internal reference for this purpose.

Depending on which angular range is present, the instrument can subsequently be in a first playback mode (step S58 ), a second playback mode (step S60 ), or a third playback mode (step S62 ), also known as transition mode.

The step S58 aims at a 3D rendering using both image channels. The step S60 aims at a 2D display using only one image channel and preferably also at a continuous image erection. The step S62 aims to enable a transition between the first playback mode and the second playback mode.

The step S58 is a step S64 downstream, which includes image manipulations adapted to the first playback mode for 3D observation. The step S64 includes, for example, a so-called 180 ° flip, whereby a 180 ° rotation of the instrument is taken into account. If the instrument with a stereo base is rotated by approx. 180 °, a 3D observation is basically possible again. In order for the image to appear upright to the observer, image manipulations are required, as described in step S64 be performed. At least in one exemplary embodiment of the method, apart from the possible 180 ° flip, there is no provision in the first playback mode to digitally erect the image.

The step S60 is a step S66 downstream, which comprises image manipulations adapted to the second playback mode. The image is made available for 2D rendering. At least in some exemplary embodiments, this includes a continuous erection of the image, so that the reproduced image appears upright to the observer, even if the instrument is rotated. The erection of the image can have a static goal, i.e. exactly one goal orientation. The image erection can, however, also take place as a function of the angle of rotation, in particular to avoid large jumps when changing between the first playback mode and the second playback mode.

The step S62 is a step S68 downstream, which includes image manipulations adapted to the transition mode. The transition mode is primarily used to enable a smooth transition between the first playback mode and the second playback mode. As an example, the transition between the display of two slightly offset fields (stereo) in the first playback mode and the display of only one image ( 2D) , i.e. the data of only one observation channel, in the second playback mode. It is conceivable to fade out or fade in one of the two fields in stereo mode in a certain period of time (for example a defined number of individual images) so that the other of the two fields becomes the dominant image. This will smooth the transition between 2D and 3D. In this way, sudden changes between successive individual images are avoided.

In step S68 an adjustment or modification of the erection takes place so that the image for the observer shifts smoothly between the different modes, whereby instead of a "jump" a gentle transformation / rotation based on interpolations is visible, which is perceived as more pleasant when looking at it.

It takes place in the subsequent step S70 a playback taking into account the respective mode. The monitoring (step S54 ) occurs continuously when the instrument is in operation, if a corresponding operating mode is selected. The transition mode S62 allows a visually pleasant transition between the first playback mode according to the step S58 and the second playback mode according to step S60 .

The step S72 concludes the procedure and includes, for example, deactivating the imaging system.

Claims (19)

Stereo imaging system, in particular a medical imaging system, comprising: - an observation instrument (10, 50, 110) with an image acquisition unit (30, 70, 130) for acquisition first image data and second image data which can be combined for stereo observation, at least one position sensor (144, 146) for detecting an orientation, in particular a rotational orientation, of the instrument (10, 50, 110) in relation to a position reference (166), and - a control device (150) which can be operated in a first playback mode and a second playback mode as a function of the orientation of the instrument (10, 50, 110), the control device (150) being designed to use the image signal to output images in to set up the second playback mode depending on the orientation, characterized in that the control device (150) is designed to output an image signal which in the first playback mode comprises a stereo signal based on the first image data and the second image data, and in the second playback mode a mono signal Signal based on the first image data or the second image data, and the control device (150) in a The first angle of rotation range of the instrument (10, 50, 110) can be operated in the first playback mode and in a second angle of rotation range of the instrument (10, 50, 110) in the second playback mode. Imaging system according to Claim 1 , wherein the control device (150) is designed to align output images in the second playback mode so that the orientation of the displayed image with respect to a display horizon does not change or changes within defined limits. Imaging system according to Claim 1 or 2 , wherein the control device (150) is designed to output images to be output in the first playback mode without being erected, so that changes in the orientation of the instrument (10, 50, 110) are accompanied by changes in the orientation of the images to be output. Imaging system according to one of the Claims 1 until 3 , wherein the first angle of rotation range has two sections offset from one another by 180 °. Imaging system according to Claim 4 , wherein the control device (150) is designed to swap the first image signal and the second image signal and to rotate the first image signal and the second image signal by approximately 180 °. Imaging system according to one of the Claims 1 until 5 wherein the second angle of rotation range comprises at least one position of the instrument (10, 50, 110) rotated by 90 ° with respect to the position reference (166). Imaging system according to one of the Claims 1 until 6th , wherein the first angle of rotation range, based on an angle scale, at which 0 ° describes an ideal orientation of the instrument (10, 50, 110) with respect to the position reference (166), comprises a first section which has a range with a first limit between 310 ° and 350 ° and the second limit between 10 ° and 50 °. Imaging system according to Claim 7 , wherein the first angle of rotation range has a second section offset by 180 ° with respect to the first section. Imaging system according to one of the Claims 1 until 8th , wherein the control device (150) is further operable to enable an adapted transition when changing between the first playback mode and the second playback mode, and wherein the transition is preferably an adaptation between the orientation of the mono signal in the second playback mode and the orientation of the Includes stereo signal in the first playback mode at a switching angle between the first playback mode and the second playback mode. Imaging system according to one of the Claims 1 until 9 , wherein the control device (150) is designed to align images output in the second playback mode dependent on the angle of rotation, so that a transition between the first playback mode and the second playback mode is preferably carried out with little or no jump. Imaging system according to Claim 10 wherein the control device (150) in the second playback mode interpolates the images to be output as a function of the angle of rotation between the non-erect state and the erected state. Imaging system according to Claim 10 or 11th , wherein the control device (150) is designed to output images in the second playback mode between a switching angle, which is assigned to the transition between the first playback mode and the second playback mode, and in which the image has in particular an orientation corresponding to the switching angle, and a critical angle or to set up critical angle range of the instrument in the second playback mode. Imaging system according to Claim 12 , the switching angle, based on an angle scale at which 0 ° describes an ideal alignment of the instrument (10, 50, 110) with respect to the position reference (166), is 25 ° to 50 °, preferably 30 ° to 45 ° °. Imaging system according to Claim 12 or 13th , wherein the control device (150) to is designed to rotate images to be output in the second playback mode between the critical angle or critical angle range and a further switching angle in such a way that the orientation of the displayed image is adapted to the further switching angle when the instrument is rotated towards the further switching angle. Imaging system according to Claim 14 , the further switching angle, based on an angle scale at which 0 ° describes an ideal alignment of the instrument (10, 50, 110) with respect to the position reference (166), is 130 ° to 155 °, preferably 135 ° up to 150 °, and the Grenzwin angle is 90 °. Imaging system according to one of the Claims 1 until 15th , wherein the observation instrument (10, 50, 110) is designed as an instrument with an inclined viewing direction. Imaging system according to one of the Claims 1 until 16 wherein the observation instrument (10, 50, 110) carries the image acquisition unit (30, 70, 130), and wherein the image acquisition unit (30, 70, 130) has a stereo image sensor or two individual sensors (180, 182) offset from one another. A method of stereo observation comprising the following steps: - Provision of an observation instrument (10, 50, 110) with an image acquisition unit (30, 70, 130) for acquisition of first image data and second image data, which can be combined for stereo observation, - Detecting an orientation, in particular a rotational orientation, of the instrument (10, 50, 110) in relation to a position reference (166), and - Operating the imaging system (100) as a function of the orientation of the instrument (10, 50, 110) in a first playback mode or a second playback mode, comprising: in the first playback mode output of an image signal which comprises a stereo signal based on the first image data and the second image data, in the second playback mode, outputting a picture signal which comprises a mono signal, and Erecting the images to be output as a function of the captured orientation, at least in the second playback mode, wherein the imaging system (100) is operated in a first angle of rotation range of the instrument (10, 50, 110) in the first playback mode and in a second angle of rotation range of the instrument (10, 50, 110) in the second playback mode. Computer program having program code which is designed to implement an imaging system according to one of the Claims 1 until 17th cause to follow the steps of the procedure below Claim 18 when the computer program is executed on the control device (150) of the imaging system (100).

Images