DE102020104486A1 - Method and device for stereoscopic imaging - Google Patents

Abstract

Method for stereoscopic imaging, in particular for use in operations on the human eye. A first image and a second image of an object (25) are recorded, the first image having a different angle of view of the object (25) than the second image. Image processing is carried out with which the first image and the second image are each separated into a first image section (33) and a second image section (34). A first composite image (35) is put together from the first image section (33) of the first image and the second image section (34) of the second image. A second composite image (36) is combined from the second image section (34) of the first image and the first image section (33) of the second image. The first composite image (35) and the second composite image (36) are displayed. The invention also relates to a device for stereoscopic imaging.

Description

The invention relates to a method and a device for stereoscopic imaging, in particular for use in operations on the human eye.

In the case of stereoscopic imaging, two images of an object recorded from different viewing directions are displayed in order to give the observer a spatial impression of the object. When imaging the human eye, there are situations in which the fundus and front of the eye are displayed at the same time. The image then comprises a first image section which shows the fundus of the eye and a second image section which shows the front of the eye.

The image of the fundus can be generated by placing an ophthalmoscopic lens a short distance in front of the cornea. The ophthalmoscopic lens forms an inverted intermediate image of the fundus. It is known from the prior art that a new inversion is necessary before such an image section can be combined with another image section to form a meaningful whole, WO 91/15150 A1 , WO 2017/063714 A1 , DE 10 2017 121 085 B3 .

The invention is based on the object of creating a device and a method for stereoscopic imaging in which the spatial representation is improved. Based on the stated prior art, the object is achieved with the features of the independent claims. Advantageous embodiments are specified in the subclaims.

In the method according to the invention, a first image and a second image of an object are recorded, the first image being recorded from a different angle of view of the object than the second image. Image processing is carried out with which the first image and the second image are each separated into a first image section and a second image section. A first composite image is combined from the first image section of the first image and the second image section of the second image. A second composite image is combined from the second image section of the first image and the first image section of the second image. The first composite image and the second composite image are displayed.

The invention has recognized that the inversion that takes place when the images are recorded is accompanied by a reversal of the spatial representation. Elements that are closer in the object appear further away in the stereoscopic representation and vice versa. The invention proposes to improve the spatial representation by separating each of the images into two image sections and alternately combining the image sections of the two images with one another.

The first image and the second image can be recorded with one or more image sensors. The beam path between the object and the image sensor can be designed in such a way that the object is imaged onto the image sensor. Imaging in this sense is used when the image plane coincides with the plane of the image sensor or when the distance between the image plane and the plane of the image sensor is small enough that the image sensor still records a usable image.

A first section of the object can be at a greater distance from the image sensor than a second section of the object. In the beam path between the object and the image sensor, a lens can be arranged which is traversed by a part of the beam path and is not traversed by another part of the beam path. The lens can image the first section of the object in an intermediate image plane. The intermediate image plane can be arranged in a plane conjugate to the image sensor. The lens can be an ophthalmoscopic lens, which is arranged at a small distance from the cornea of an observed eye. The method can be carried out such that the first section of the first image and / or the first section of the second image correspond to the first section of the object.

The first image section of the first image and / or of the second image can be a central image section. The second image section of the first image and / or of the second image can be a peripheral image section which surrounds the central image section. On the image sensor, the first image section can be inverted relative to the second image section. Image processing can be carried out with which the first image section is inverted relative to the second image section prior to the generation of the composite image, in order to bring the two image sections into a matching alignment. For this purpose, the first image section can be mirrored relative to the second image section about two mutually orthogonal axes, or point mirroring can be carried out. It does not matter whether the orientation of the first image section is retained and the orientation of the second image section is changed in this step, or vice versa. The alignment step can be performed so that the composite image is directly in the correct alignment for the viewer's perspective. It is also possible to reverse the generated composite image again in order to bring it into the correct orientation for the viewer. There is none Restriction of the order in which several image processing steps according to the invention are carried out.

An objective lens, through which the beam path of the first image section and the second image section passes, can be arranged between the object and the image sensor. This can apply to the first picture and the second picture. A magnification system with an adjustable magnification factor can be arranged between the objective lens and the image sensor. The magnification system can be an afocal magnification system. The magnification system can be set up in such a way that, when the magnification factor is low, the first image section and the second image section impinge on the image sensor and that, when the magnification factor is high, only the first image section impinges on the image sensor.

The first composite image and / or the second composite image can be displayed in such a way that the alignment of the first image section and the second image section relative to one another corresponds to the actual alignment in the object. The first composite image and the second composite image can be displayed so that the viewer can see both images at the same time. In particular, the first composite image and the second composite image can be displayed in such a way that the viewer sees the first composite image with one eye and the second composite image with his other eye.

The image data can be transmitted electronically between the image sensor and the display, for example via a cable or via a radio link. A data memory can be provided in which the image data are stored. This opens up the possibility of displaying the composite images at a point in time that is independent of the recording of the images. In one embodiment, the image data are transmitted to the display without delay, so that the user can follow changes in the object immediately.

In one embodiment, the first composite image and the second composite image are displayed in the eyepieces of a surgical microscope. The surgical microscope can be designed in such a way that the eyepieces are each equipped with a display, the first display showing the first composite image and the second display showing the second composite image. The surgical microscope can have a continuous beam path that extends from the object to the image sensor. The image data can be transmitted electronically between the image sensor and the display, so that there is no direct optical beam path on this route.

It is also possible for the first composite image and the second composite image to be displayed in a display unit that the viewer wears on the body. For example, the display unit can be VR glasses. In this embodiment, too, each eye of the viewer can be assigned its own display.

The first composite image and the second composite image can also be shown on a single display. The spatial impression for the viewer can be produced in a suitable manner, for example by means of holography. The use of stereoscopic displays (auto-stereoscopic displays, 3D displays for use with polarized filter glasses, 3D displays for use with shutter glasses) is also possible. Such a representation can be helpful as a primary visualization or, for example, also for training purposes. The primary visualization can also be done classically, i.e. with eyepieces and a continuous optical view of the object levels, and an additional or alternative visualization can be carried out using the procedure described here.

The possibility can be provided of changing the brightness with which the first image section and the second image section of the composite image are displayed relative to one another. The change in the relative brightness can be triggered by a user input. A first mode can be provided in which the first image section and the second image section are displayed with the same brightness. A second mode can be provided in which the first image section is displayed with a different brightness than the second image section. In particular, the first image section representing the fundus can be displayed with greater brightness than the second image section representing the surroundings. The brightness of the second image section can be reduced to such an extent that it is no longer noticeable compared to the first image section. It can also be possible to completely deactivate the display of the second image section. The first composite image and the second composite image can be synchronized with one another so that the two first image sections are displayed with the same brightness and / or that the two second image sections are displayed with the same brightness.

All features that are described in connection with one of the elements present in pairs can apply to the first element and the second element.

The invention also relates to a device for stereoscopic imaging with an image sensor for recording a first image and a second image of an object, wherein the first image is recorded from a different angle of view of the object than the second image. An image processing device is designed to separate the first image and the second image into a first image section and a second image section. The image processing device combines a first composite image from the first image section of the first image and the second image section of the second image and combines a second composite image from the second image section of the first image and the first image section of the second image. The first composite image and the second composite image are shown on a display.

The device can comprise a first image sensor for recording the first image and a second image sensor for recording the second image. The device can comprise a first display for displaying the first composite image and a second display for displaying the second composite image. The image processing unit can be designed as a spatial unit or distributed among its spatially separate components.

The device can be developed with further features which are described in connection with the method according to the invention. The method can be developed with further features which are described in connection with the device according to the invention.

• 1: eine erste Ausführungsform einer erfindungsgemäßen Vorrichtung;
• 2: ein menschliches Auge während einer augenchirurgischen Operation;
• 3: ein Bild des Auges aus 2;
• 4: ein von dem Auge aus 2 aufgenommenes Bild;
• 5: das Bild aus 4 nach einer ersten Bildbearbeitung;
• 6: zwei erfindungsgemäße Composite-Bilder;
• 7: eine zweite Ausführungsform einer erfindungsgemäßen Vorrichtung;
• 8: eine dritte Ausführungsform einer erfindungsgemäßen Vorrichtung.

The invention is described below by way of example with reference to the accompanying drawings on the basis of advantageous embodiments. Show it:

• 1 : a first embodiment of a device according to the invention;
• 2 : a human eye during an ophthalmic surgical operation;
• 3 : a picture of the eye out 2 ;
• 4th : one from the eye 2 captured image;
• 5 : the picture off 4th after a first image processing;
• 6th : two composite images according to the invention;
• 7th : a second embodiment of a device according to the invention;
• 8th : a third embodiment of a device according to the invention.

One in 1 The device according to the invention shown is constructed similarly to a surgical microscope. The device comprises a main lens 14th and an afocal magnification system 15th . With the main lens 14th and the afocal magnification system 15th a beam path is formed in which an object is imaged from the object plane to infinity. The afocal magnification system is used for a stereoscopic display 15th a first observation beam path, which is intended for the left eye of the viewer, and a second observation beam path, which is intended for the right eye of the viewer. With suitable optical elements 16 become the first observation beam path onto a first image sensor 17th and the second observation beam path onto a second image sensor 18th pictured.

The one with the image sensors 17th , 18th recorded image data become an image processing unit 19th which is set up to store, process and / or forward the image data.

The device includes an insight 20th , the one with two eyepieces 21 , 22nd Is provided. Any of the eyepieces 21 , 22nd is a display 23 , 24 assigned so that a user can see his through the eyepieces 21 , 22nd sets them up on the displays 23 , 24 the images shown. As the images of the object with the image sensors 17th , 18th were taken from slightly different angles, the through the eyepieces 21 , 22nd looking observer get a spatial impression of the object.

Between the main lens 14th and the object in the form of a human eye 25th is an attachment module 26th arranged that an ophthalmoscopic lens 27 and an additional lens 28 includes. With the ophthalmoscopic lens 27 becomes the fundus 28 in an intermediate image plane 29 pictured. With the additional lens 28 the beam path is adjusted so that the intermediate image plane 29 one to the image sensors 17th , 18th conjugate plane forms.

The ophthalmoscopic lens 27 is dimensioned so that it only covers the central part of the anterior segment of the eye, through which the fundus passes 28 coming light passes through. In the vicinity of the ophthalmoscopic lens 27 the beam path runs on the ophthalmoscopic lens 27 passing so that there is a direct ray path between the front 30th of the eye 25th and the image sensors 17th , 18th results. Because the front 30th of the eye 25th a small axial distance to the intermediate image plane 29 has arises on the image sensors 17th , 18th a slightly blurred image of the front 30th of the eye 25th .

The ones in the 1 , 7th , 8th The lenses shown can be singlet lenses or also doublet lenses, triplet lenses or lens systems.

In 1 a pure video microscope is shown. Variations such as a video hybrid microscope are also possible, in which imaging with video technology and conventional microscope imaging (ie with a continuous optical view from the eyepieces down to the object planes) can take place simultaneously or alternatively. In this case, for example, a beam splitter would be added above the zoom system, part of the light from the object plane would be directed to the camera via a lens or a lens system and another part of the light from the object plane would be directed to an optical unit via conventional tube lenses and optional optical inverters Eyepiece. The reference number 16 would in this case designate an optical system consisting of a beam splitter and two lenses or lens systems. In this case, the video 3D visualization would take place via 3D displays, for example.

In 2 is the eye 25th shown during an ophthalmic operation using a surgical instrument 31 in the interior of the eye 25th is introduced. An observer who looks at the eye from the front without aids 25th a picture would be similar to in 3 see. That through the pupil 32 visible front end of the surgical instrument 31 would be out of focus due to the optical elements in the anterior segment of the eye.

On the first image sensor 17th creates a picture like in 4th indicated, in which a peripheral image section 34 a direct view of the front 30th of the eye and a central section of the image 33 the one with the ophthalmoscopic lens 27 generated image of the fundus 38 shows. In the peripheral image section 34 The view is enlarged but otherwise compared to that in FIG 3 shown direct view unchanged. In the central section of the picture 33 the view is also enlarged but due to the intermediate image plane 29 (or the lens 27 ) additionally inverted (compared to an image alignment without an ophthalmoscope lens). In the sense of the invention, the central image section forms 33 a first image section and the peripheral image section 34 a second image section. On the second image sensor 18th a corresponding image is created that differs from the image on the first image sensor 17th differs only slightly due to the different viewing angle.

In the image processing unit 19th is an image processing for those with the image sensors 17th , 18th recorded images carried out with each of the central image section 33 from the peripheral image portion 34 is separated. The central section of the picture 33 becomes relative to the peripheral image portion 34 mirrored about two mutually orthogonal axes A, B, so that the central image section 33 one to the peripheral image portion 34 has a suitable alignment, see 5 .

From the two central sections of the picture 33 and the two peripheral image sections 34 resulting from the two image sensors 17th , 18th Recorded images result, are in the image processing unit 19th two composite images 35 , 36 formed by the central image section 33 and the peripheral image portion 34 can be alternately combined with one another. So it becomes the central image section 33 the one with the first image sensor 17th recorded image combined with the peripheral image section 34 the one with the second image sensor 18th captured image. The central image section 33 the one with the second image sensor 18th recorded image is combined with the peripheral image section 34 the one with the first image sensor 17th captured image.

The two composite images 35 , 36 will be like in 6th shown on the displays 23 , 24 shown so that the composite images 35 , 36 have the same orientation for the viewer as in 3 . Depending on the design of the optical system 16 it may be necessary to use the composite images 35 , 36 to mirror or invert electronically. The peripheral image section becomes 34 from the first image sensor 17th on the first display 23 and the peripheral image section 34 from the second image sensor 18th on the second display 24 shown. The central image section 33 from the first image sensor 17th will appear on the second display 24 shown. The central image section from the second image sensor 18th will be on the first display 23 shown. This results in a representation that shows the viewer the image with a suitable orientation and spatial impression.

In the alternative embodiment according to 7th are the two displays 23 , 24 in a display unit 37 arranged in the form of VR glasses, which are connected to the user's head and worn by the user. The image data are transmitted via a radio link 39 from the image processing unit 19th to the displays 23 , 24 of the device 37 transfer.

The image processing unit 19th includes an operating element 38 with which the brightness with which the peripheral image sections 34 the composite images 35 , 36 on the displays 23 , 24 are represented relative to the brightness of the central image section 33 can be reduced. The actual operation takes place inside the eye, which the surgeon does with the central section of the image 33 is pictured. The peripheral portion of the image 34 is of interest, for example, when using a surgical instrument 31 is introduced. In such a phase it is advantageous if the central image section 33 and the peripheral image section 34 can be displayed with the same brightness. During the actual operation, it may be more comfortable for the surgeon if the peripheral portion of the image 34 is displayed darker or is completely hidden.

The alternative embodiment according to 8th includes a data store 40 , in which the with the image processing unit 19th edited image data can be saved. To the data store 40 is a holographic screen 41 connected that from the two composite images 35 , 36 creates a three-dimensional image that appears on the screen 41 is shown. The reproduction of the image data on the screen 41 can take place at any time interval from the recording of the image data.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• WO 9115150 A1 [0003]
• WO 2017/063714 A1 [0003]
• DE 102017121085 B3 [0003]

Claims (10)

Procedure for stereoscopic imaging with the following steps. a. Recording a first image and a second image of an object (25), the first image being recorded from a different viewing angle to the object (25) than the second image; b. Carrying out an image processing with which the first image and the second image are each separated into a first image section (33) and a second image section (34); c. Merging a first composite image (35) from the first image section (33) of the first image and the second image section (34) of the second image and merging a second composite image (36) from the second image section (34) of the first image and the first image portion (33) of the second image; d. Displaying the first composite image (35) and the second composite image (36). Procedure according to Claim 1 , characterized in that the first image is recorded with a first image sensor (17) and the second image is recorded with a second image sensor (18). Procedure according to Claim 1 or 2 , characterized in that a lens (27) is arranged in the beam path between the object (25) and the image sensor (17, 18), which is traversed by part of the beam path and which is not traversed by another part of the beam path. Method according to one of the Claims 1 until 3 , characterized in that the first image section (33) is a central image section and the second image section (34) is a peripheral image section. Method according to one of the Claims 1 until 4th , characterized in that before the composite image (35, 36) is generated, the first image section (33) is inverted relative to the second image section (34). Method according to one of the Claims 1 until 5 , characterized in that the first composite image (35) and the second composite image (36) are displayed such that the viewer has the first composite image (35) with one eye and the second composite image with his other eye sees. Method according to one of the Claims 1 until 5 , characterized in that the first composite image (35) and the second composite image (36) are shown on a display (41). Method according to one of the Claims 1 until 7th , characterized in that in the displayed composite image (35, 36) the brightness of the second image section (34) is changed relative to the brightness of the first image section (33). Method according to one of the Claims 1 until 8th , characterized in that the first composite image (35) and the second composite image (36) are synchronized with one another, so that the two first image sections (33) are displayed with the same brightness and / or that the two second image sections (34 ) can be displayed with the same brightness. Device for stereoscopic imaging with an image sensor (17, 18) for recording a first image and a second image of an object (25), the first image being recorded from a different angle of view of the object (25) than the second image, with a Image processing device (19) which separates the first image and the second image into a first image section (33) and a second image section (34) and which comprises a first composite image (35) from the first image section (33) of the first image and merges the second image section (34) of the second image and merges a second composite image (36) from the second image section (34) of the first image and the first image section (33) of the second image, and with a display (17, 18, 41) for displaying the first composite image (35) and the second composite image (36).

Images