DE10237470A1 - Method and device for generating light microscopic, three-dimensional images - Google Patents

Abstract

The invention relates to a device for imaging a three-dimensional object (22) as an object image (30), which comprises an imaging system, especially a microscope for imaging the object (22) and a computer. Actuators change the position of the object (22) in the x, y and z direction in a specific and rapid manner. A recording device records an image stack (26) of individual images (24) in different focal levels of the object (22). A control device controls the hardware of the imaging system, and an analytical device produces a three-dimensional elevation relief image (28) and a texture (29) from the image stack (24). A control device combines the three-dimensional elevation relief image (28) with the texture (29).

Description

The invention relates to a method for representing a three-dimensional Object according to the preamble of claim 1 and a device for this purpose according to the preamble of claim 17.

Known devices of this type such as microscopes, macroscopes, etc. serve the viewing of an object with the help of physical Basic laws. Limits in sharpness and depth, viewing angle and Time dependency must be accepted despite good technology.

To the depth of field and the physical limits There are already a number of ways to improve microscopic imaging Equipment and processes. Such devices can, for example, all types of Be light microscopes. This includes, for example, a confocal Scanning microscope. This is a sample with the focus of a light beam scanned point by point in one plane, so that an image, but with less Depth of field, this level is obtained. By including one Plurality of different levels and by a corresponding one The object can then be displayed three-dimensionally using image processing. Such a thing confocal scanning microscopy method is for example from the US 6,128,077. The ones used in confocal scanning microscopy However, optical components are very expensive and require one good technical understanding of the operating personnel a lot Adjustment work.

From US 6,055,097 is also a method for Luminescence microscopy known. Here, dyes are introduced into a sample, which are under suitable lighting conditions fluoresce so that with the radiation the localization of the dyes in the sample is possible. For generation a spatial image becomes a number of images in different Focus planes added. Each of these images contains image information that come directly from the focus level, as well as such Image information that comes from spatial sections of the object that are outside the Focus plane. To determine a sharp image, it is necessary to use the to eliminate parts of the image that do not originate from the focal plane. This will proposed to provide the microscope with an optical system that it allows the sample to be illuminated with a special lighting field, such as one standing wave or to illuminate a non-periodic excitation field. These usual microscopic images are limited due to the Depth of field of the imaging process optically limited and because of the Art and way of viewing, d. H. through the angle of view, in their Limited representation. Microscopic images can be partially out of focus. This Uncertainty can be explained. a. through non-planar objects since the Object surface is often not completely in the respective focal plane. With usual Imaging systems is as a result of through the microscope or the macroscope given the direction of view of the object, no other point of view possible on the object (e.g. tangential to the object surface) without elaborate new preparation and new adjustment of the object itself.

With all these light-optical processes, the imaging accuracy is through limited depth of field limitation.

The unpublished DE 101 49 357.6 describes a method and a device for generating a sharp three-dimensional Surface imaging of microscopic objects. This is the surface profile of the object in a three-dimensional coordinate system (x, y, z) optically measured. In this process, different focus levels become one microscopic object by a CCD camera digital or analog recorded. This creates an image for each focal plane, so that a "Image stack" is created. This stack of images is composed of images that are made up of the different focus levels one during recording object immobile under the microscope. Each of these pictures in the image stack contains areas of sharp image structures with high Sharpness of detail and areas when the picture is taken outside the focal plane and were therefore out of focus and without a high level of detail in the image. In In this sense, an image can be understood as a set of image subareas high level of detail (in focus) and low level of detail (outside the Focus). Using image analysis methods, every image of the image stack can the subsets of the image areas are extracted in high detail is present. In a result image, all extracted subsets are now each Image with a high level of detail, i.e. image areas in focus, into one new overall picture put together. This creates a new, complete one detailed picture.

Since the relative position of the focal planes to each other are known, from which the Subset of each image with high detail sharpness is also the Known distance of the images in the image stack. Thus, a three-dimensional surface profile of the microscopic object can be created.

So it's a deep image and a three-dimensional one To be able to carry out surface reconstruction of the recorded object area, requires a previously created sequence of images from different focus levels of the Object.

So far, the change in the focus level has been made by adjusting the Microscope stage height, i.e. by adjusting the distance between the Object and the lens, by mechanical adjustment of the support table reached. Due to the high table mass and the associated inertia of the Overall system could set certain speed limits for the Image acquisition in different focus levels must not be undercut.

The unpublished DE 101 44 709.4 describes this improved process and apparatus for rapid production more precise image recordings of the individual images of the image stack in the various Focus levels using piezo actuators in conjunction with stepper motor-controlled and / or servomotor-controlled methods. In this procedure, a Focus plane adjustment, by changing the Distance between lens and object as well as an object position adjustment in the x-y plane by various actuators such as piezo lenses, Piezo preparation tables, combinations of piezo actuators and standard Stepper motor adjustments or any other table adjustments achieved. By the The use of piezo actuators increases the precision and delicacy of the adjustment elevated. Furthermore, the adjustment speed by piezo actuators accelerated. It is also described as by suitable insertion or Applying deconvolution techniques to image quality and Evaluation quality can be further increased.

So rasterized by means of automatically adjustable object holders However, surfaces do not allow a holistic, deep view of the Object itself. Also is a three-dimensional representation of the whole Scanning area not possible. Furthermore, the representation in space cannot rotated and viewed from different angles.

The object of the present invention is therefore a method and a Device for generating light microscopic, three-dimensional images indicate which works with less technical effort and at the same time provides an improved image quality in the three-dimensional representation.

This task is accomplished by a method for representing a three-dimensional object with the features of claim 1 and by a Device with the features according to claim 10 solved.

According to the invention, an image stack is thus obtained from a real object, which consists of light microscopic images. With a suitable one The image stack then becomes a method, in particular a software method obtained a surface relief image, which is then combined with a texture is that an image of the object is created. To combine the texture with the It is particularly advantageous to add a texture to the height relief image Project elevation relief image. The texture can in turn be derived from the data of the Image stack can be obtained.

With this method, a virtual image of a real object can be created are generated that meet all requirements placed on a virtual image are sufficient. This means that this object image can also be used for virtual images possible manipulations are processed. Generally, the virtual Reality tries through appropriate procedures, particularly in one Computer program can be realized through reality as best as possible map virtual objects that have been properly calculated. Through virtual Lamps and shadows, by simulating physical Basic laws and properties such as refractive index settings, simulation of Elasticities of objects, gravitational effects, tracking of a virtual light beam in virtual space under the influence of matter, the so-called Ray tracing and many more properties are becoming increasingly real Replicas of reality on the computer.

Usually, the scenes and processes in the virtual rooms are Designer completely re-created or it is based on existing one Resources. In contrast, in the present invention, a real one Imaging system, especially a microscope, used to view the Generation of a virtual image of the data necessary for reality to generate. This data can then be processed so that it is possible automatically display a virtual three-dimensional structure. A A special feature here is that an elevation relief from the real object determined and this relief is provided with a texture, which preferably consists of the data obtained from the object is determined. Thereby with the Projection of the texture on the relief image gives particularly good results achieved.

A major advantage of the invention is to be seen in that Use of the method according to the invention, the usual light microscopy and light macroscopy is expanded by the fact that the real Light imaging systems such as light microscopes or light macroscopes obtained Raw data such as, static three-dimensional surface information or blurred image information can be combined into a new image. Consequently can all or any combination or subset of the real partial information obtained can be displayed simultaneously.

Another advantage is that individual or temporal successive, in-depth calculated multi-focus images with also obtained associated three-dimensional surface information with each other to be merged. This amalgamation process is realized by that the in-depth multi-focus image as the surface texture of an associated one three-dimensional surface. By projecting this The surface texture on the three-dimensional surface becomes the Merging process reached.

The new three-dimensional virtual image obtained according to the invention thus bears both information at the same time, the three-dimensional Surface information and the completely sharp image information. This image display can with "Virtual Reality 3D Light Microscopy" because the data merging described in "real" microscopes not performed can be.

To generate the image stack, which consists of individual images that are in different focal planes of the object can be recorded Process steps already described in more detail above are carried out In particular, that described in DE 101 49 357.6 can be used for this purpose Method for creating a three-dimensional surface reconstruction be used. These provide two data records in the form of an image. The one encodes the height information of the microscopic object and is in the called another mask image.

The second data set represents a high-contrast, completely sharp Microscope image and is called multifocus image below. This Multi-focus image is created with the aid of the mask image by the Mask image gray values are used to make the level of the sharpest Identify pixels and the corresponding pixel of the layer in the image stack to copy into a summary multifocus image.

As described above, the method steps can, for example as described in DE 101 44 709.4 can be used by Piezo technology on lenses and / or specimen tables and mask images Multi-focus images in higher resolution in the direction of the focus planes (z direction) and by scanning the object over larger areas in the respective Create focal plane (x, y direction).

The mask image thus carries the height information and the multifocus image pure, sharp image information. The mask image then becomes a three-dimensional relief image (pseudo image) created. This arises in that the mask image is shown as a relief in height. The pseudo picture carries no direct image information other than the height information. therefore the three-dimensional pseudo image represents a so-called height relief. In Another step is the three-dimensional pseudo image with the real one Provide the texture of the sharp parts of the image in the image stack. This will be the Pseudo image and mask image appropriately aligned so that the Height information of the pseudo image and image information of the Mask image, i.e. the texture is superimposed pixel-perfect. So that every pixel of the multifocus texture image exactly to its associated pixel in the three-dimensional pseudo image, so that a virtual image of the real object arises.

Previously common light microscopic methods for imaging objects are characterized by an abundance of physical limitations in their Limited display options. The invention largely overcomes these limitations and gives the user many new opportunities to add microscopic objects examine and present.

A suitable user interface can also be used to apply the invention be defined, which allows the user to use the invention without use special technical knowledge. Furthermore, the invention is also applicable for three-dimensional display of large areas. Through the Real microscopic or macroscopic image Image information in a "virtual reality room" is the usual light microscopy Access to the entire technology of virtual worlds opened. The Compared to conventional light microscopy, the resulting images convey a much clearer and more meaningful microscopic image and put the user in a position to do so far known mapping processes and manipulation processes of virtual reality apply.

The virtual image has no limitation in the sharpness that can be seen in usual object images due to the limited depth of field of the used Optics results. So it is perfectly sharp. The virtual image also carries the complete depth information. This creates a completely sharp, three-dimensional, lifelike virtual image of a real microscopic object.

In a preferred embodiment of the invention, the illustration can be realized virtually in a computer. Here are all the options image display and manipulation available with virtual images can be used. These range from the overlays real microscopic surfaces and purely virtual surfaces up to the possibility supervision from any angle to a depth of field three-dimensional surface. The surfaces can be virtually animated, illuminated or are otherwise modified. Time dependencies such as temporal Changes to the surface of the microscopic object can be made simultaneously in-depth image information and three-dimensional Surface topologies are mapped.

This opens up completely new possibilities for light microscopy, Limitations of the image quality due to physical limitations compensated.

• 1. Mikroskop mit notwendigem Zubehör (Objektive, etc.) oder ein anderes geeignetes Abbildungssystem wie etwa ein Makroskop
• 2. Computer mit geeignetem Zubehör wie Monitor, etc.
• 3. Aktoren zur gezielten, schnellen Positionsveränderung eines Objektes in x, y, und z-Richtung wie etwa Piezo, Schrittmotortisch usw.
• 4. Eine Kamera, insbesondere eine analoge oder digitale CCD Kamera mit notwendigem oder vorteilhaftem Zubehör wie Grabber, Firewire, Hotlink, USB-Port, Bluetooth zur Funkdatenübertragung, Netzwerkkarte zur Bilddatenübertragung via. Netz, etc.
• 5. Eine Steuerungseinrichtung zur Steuerung der Hardware des Mikroskops, insbesondere des Präparatetisches, der Kamera, und der Beleuchtung.
• 6. Eine Analyseeinrichtung zum Erstellen der Multifokusbilder, der Maskenbilder, der Mosaikbilder und der Erstellung der "virtuell Reality 3D Licht Mikroskopbilder". Steuer- und Analyseverfahren sind bevorzugt als Software implementiert.
• 7. Eine Einrichtung zur Darstellung, Berechnung und Manipulation der erstellten "virtuell Reality 3D Licht Mikroskopbilder" wie z. B. Drehung im Raum, Beleuchtungsänderungen etc. Diese wird bevorzugt wiederum als Darstellungssoftware implementiert.

In an advantageous embodiment of the invention, the following components are used:

• 1. Microscope with necessary accessories (lenses, etc.) or another suitable imaging system such as a macroscope
• 2. Computer with suitable accessories such as a monitor, etc.
• 3. Actuators for the targeted, rapid change in position of an object in the x, y, and z direction, such as piezo, stepper motor table, etc.
• 4. A camera, in particular an analog or digital CCD camera with necessary or advantageous accessories such as grabber, firewire, hotlink, USB port, Bluetooth for radio data transmission, network card for image data transmission via. Network, etc.
• 5. A control device for controlling the hardware of the microscope, in particular the preparation table, the camera, and the lighting.
• 6. An analysis device for creating the multi-focus images, the mask images, the mosaic images and the creation of the "virtual reality 3D light microscope images". Control and analysis methods are preferably implemented as software.
• 7. A device for displaying, calculating and manipulating the created "virtual reality 3D light microscope images" such. B. rotation in space, changes in lighting, etc. This is in turn preferably implemented as display software.

Software used in a computer controls the microscope Preparation table in x, y, and z direction, possible piezo actuators, lighting, Camera image acquisition, other microscope hardware. The procedure for Generation of the mask and multifocus images and the creation of a "virtual Reality 3D microscope image "can be controlled.

By using a piezo-controlled lens or one piezo-controlled object holder or the combination of a piezo-controlled Lenses with a piezo-controlled object holder can achieve a very fast, reproducible and precise positioning of an object in all three Spatial directions take place. In combination with the depth of field enhancing image analytical processes and the possibilities for 3D reconstruction succeed a fast, 3D reconstruction of microscopic surfaces. Let also quickly generate picture mosaics that are both in focus and can create a dimensional surface profile. The single images are recorded by a suitable CCD camera. An unfolding of the Single images with a suitable apparatus profile before the subsequent one Sharpening and 3D reconstruction also allow generation High-resolution, apparatus profile-corrected microscope images of high Depth of field.

In a further advantageous embodiment of the invention, in Several image stacks were recorded in chronological order. The one described above Implementation of these temporal individual recordings of the image stacks too temporally successive Virtual Reality 3D images enables three-dimensional, totally Sharpness of time in an animation, such as a movie.

Another advantageous embodiment of the invention is obtained from the Use of so-called morphing, in which several images in one Animation can be merged. It is a Interpolation between pictures, such that based on a known start picture and of a known final image further, previously unknown intermediate images be calculated. If you arrange the start, the intermediate and the end picture then suit each other and then play the familiar and the interpolated images sequentially in time, so the impression is created a continuous transition between the start and the end image.

Morphing accelerates the described method in that only a few pictures were taken in real time and in space Need to become. All other images for a virtual display are through Interframe interpolation calculated.

It is a particular advantage of the present invention to produce one "Virtual Reality 3D Light Microscopy Image", real data from light optical Imaging systems such as light microscopes or light macroscopes use. It should be noted that the in light macroscopes by the Imaging optics caused distortions to be mathematically corrected become. According to the invention, the virtual reality becomes automatic, semi-automatic or generated manually from the underlying real data. The Possibility of any connection of the obtained data of the "Virtual Reality 3D Light microscopy "using already known techniques of virtual reality generated purely virtually, i.e. without the direct influence of real physical data Data is another advantage of the invention.

Another advantage of the invention is the possibility of using the data of the "Virtual Reality 3D Light Microscopy" 3D measurements such as B. Volume measurements, surface measurements, etc.

A further advantageous embodiment of the invention offers the possibility the projection of image analytically influenced and / or changed Texture images on the 3D surface as described above. This is one further "extended perception" through the "virtual reality 3D light Microscopy "possible because the changed textures are spot-on on the 3D surface be projected. This enables using image analysis results to connect three-dimensional surface data and display them simultaneously. This also applies to time series of shots in the picture that are influenced by image analysis sense above.

Another advantage of the invention is the method for mosaic images apply so that defined surface sub-areas of an object be scanned. These drawing files are put together in depth and together with the associated 3D object surface data to form a "Virtual Reality 3D Light microscopy image "of the scanned object area.

The invention is particularly notable for its advantages from that a significant expansion in microscopic perception Facts on the object becomes possible. This is done by displaying a completely sharp image on a microscopic image three-dimensional surface. Through the virtual 3D reality of the Microscope image and its compatibility of the virtual representation too Standard programs and procedures succeed in integrating the whole Knowledge and all the possibilities that have so far generated for virtual reality were.

The images generated with the method according to the invention correspond more the actual conditions in the sample than those with conventional microscopes. Because in the "Virtual Reality 3D light microscope image "is in addition to complete sharpness three-dimensional information of the object clearly produced.

In addition, "Virtual Reality 3D light microscope image" can be used under different Solid angles can be viewed by moving the image into any position rotates. In addition, the object image can be seen through transparencies and others Standard procedures can be manipulated to any other microscopic To reinforce or weaken details.

The meaningfulness and one much closer to human perception upcoming three-dimensional representation of a microscopic object enable completely new analysis methods. Image mosaics that are called "Virtual Reality 3D light microscope image "are shown, also expand the Representability.

The possibility of fully automating the processes mentioned Generation of a "virtual reality 3D light microscope image" or several "virtual Reality 3D light microscope images "through automatic time series only low demands on the technical know-how of the users.

Combinations of the "virtual reality 3D light microscope image", which consists of real recorded basic data was generated with the possibilities of Fade in of purely virtual objects such as Platonic basic bodies or other more complex bodies, give new didactic possibilities of Knowledge transfer. The combination of the data of "Virtual Reality 3D Light microscope image "with 3D cyberspace glasses allows a view of microscopic objects with unprecedented precision and holism.

Because the data of the "Virtual Reality 3D light microscope image" in a computer can be stored is a representation of the data on others Systems, data transfer via computer networks such as the intranet or the Internet, and the representation of the "Virtual Reality 3D light microscope image data" via Web browser possible. There is also a three-dimensional image analysis possible.

Virtual microscopy, i. that is, microscopy by users "without" Microscope, that is, only on the basis of the obtained and / or stored "virtual Reality 3D light microscope image data "allows a separation between real microscopy and evaluation of the data obtained.

The use of standard light microscopes with a standard Standard lighting is possible for generating the 3D image according to the invention and therefore inexpensive.

Further advantages and advantageous embodiments of the invention are Subject of the following figures and their descriptions, in their Representation for the sake of clarity on a scale Playback was waived.

They show in detail:

Fig. 1 shows a schematic flow of the method according to the invention.

Fig. 2 shows a schematic sequence of the method according to the invention using an example

Fig. 3 shows a schematic flow of the process of the invention by way of example

Fig. 4a example of a pseudo image

Fig. 4b example of a structured pseudo image

Fig. 5 combination of a texture with a pseudo image using an example

Fig. 6 Schematic automated process flow

In Fig. 1, the basic sequence of the method according to the invention is shown schematically, which is again illustrated in Figs. 2 and 3 using a schematic example. Starting from an object 22 ( FIG. 2), an image stack 24 is generated in method step 10 in that individual images 26 are recorded manually or fully automatically from a plurality of focus planes of the object 22 . The distance between the individual images is suitably dimensioned in order to be able to reconstruct a three-dimensional image with a sharp focus and is preferably kept equidistant. Each of the individual images 26 has sharp and unsharp image areas, the image spacing and the total number of individual images 26 being known. After recording, the images are first stored in uncompressed form or compressed using a data loss-free compression process in process step 12 . The individual images 26 can be color images or grayscale images. The color or gray value resolution (8 bit, 24 bit etc.) can be any.

When creating the image stack, it can also be done in such a way that several images lie side by side in a focal plane (in the x, y direction) and are reassembled with pixel accuracy so that a so-called mosaic image of the focal plane is created. It is also possible to generate an image stack 24 from the mosaic images. After an individual image has been recorded in each desired focal plane (z plane), an image stack 24 with a series of individual images 26 is available for further image processing. The z-planes are preferably equidistant from one another.

An imaging system can be used to create the image stack 24 , in particular using a microscope or a macroscope. However, a suitably attached camera system with a lens can also be used. The entire illumination range of a specimen from near UV to far IR can be used, provided the imaging system allows this.

The recording system can generally be any analog or digital CCD camera have, with all types of CCD cameras, in particular Line scan cameras, color cameras, gray image cameras, IR cameras, integrating cameras, Cameras with multi-channel plates, etc. can be used.

In a further method step 14 , a multi-focus image 15 and a mask image 17 are now obtained from the data obtained from the image stack 24 , it being possible in particular to use the methods according to DE 101 49 357.6 and DE 101 44 709.4. Due to the depth of field of the microscope, each of the individual images contains 26 sharp and unsharp areas. The respective sharp areas in the individual images 26 are determined according to certain criteria and assigned to the corresponding coordinate points (x, y) with their plane number. The assignment of level numbers and coordinate points (x, y) is stored in a memory and represents the mask image 17. When the mask image 17 is processed, the level numbers stored in the mask image can be viewed as gray values.

In the multi-focus image 15 all blurred image portions of the frames of the previously recorded and stored image stack 24 are removed so that a completely sharp deep sharp image. The multifocus image ( 15 ) can also be created from a stack of mosaic images in such a way that a plurality of mosaic images from different focus planes are calculated to form a multifocus image ( 15 ).

In the mask image 17 , all gray values of the pixels identify the number of the origin level of the sharpest pixel. The mask image can thus also be represented as a three-dimensional height relief 28 . The three-dimensionality results from the x, y positions of the mask image pixels and the amount of the gray value of a pixel, which characterizes the focal plane position of the three-dimensional data set. The mask image 17 can also be created from a stack of mosaic images, wherein a plurality of mosaic images from different focal planes are calculated into the mask image 17 .

Now that the mask image 17 is available, a so-called three-dimensional pseudo image 28 can be created from it. For this purpose, the mask image 17 is represented as a height relief in method step 16 . Apart from the height information, this image does not carry any direct image information. For this purpose, the mask image 17 is represented by a suitable software as a dimensional height relief. The software can be developed, for example, on the basis of the well-known software libraries OpenGL or Direct3D (Microsoft). Other commercially available software packages are also suitable for displaying, creating, animating and manipulating 3D scenes such as Cinema 4D (Maxon), MAYA 3.0, 3D Studio MAX or Povray.

So-called splines are used to generate this representation. Splines are essentially sequences of vertices lying in three-dimensional space, which are connected with each other by lines. Splines are well known in mathematics for the technology to create three-dimensional objects. In a way, they represent contour lines on a map. The support points are supplied by the gray values of the mask image, such that the coordinates (X, Y, Z) of the support points for a spline interpolation correspond to the following mask image data
The base coordinate X corresponds to the mask image pixel coordinate X
The base coordinate Y corresponds to the mask image pixel coordinate Y
The reference coordinate Z corresponds to the gray value at X, Y of the mask image 17 .

The course of the spline curves is determined by the so-called interpolation. The course of the spline curves is calculated by an interpolation between the support points of the splines (polynomial fit of a nth order polynomial by a predetermined number of points in space, such as by Bezier polynomials or Bernstein polynomials etc.), so that the spline curves are created. Depending on the type of interpolation function used and the number of interpolation points, curve adaptations to the given interpolation points can be made with more or less detail. By not considering all mask image points as support points for splines but only a suitably selected subset of the mask image points, the number of support points can be varied. Here, for. B. only every fourth pixel of the mask image 17 can be used. A subsequent interpolation between the lower number of reference points would represent the object surface in a lower resolution. By adjusting the number of reference points, you have the option of displaying surfaces with a greater or lesser degree of detail and thus filtering out various surface artifacts. Fewer support points therefore lead to a smoothing effect on the three-dimensional surface.

In the present invention, the previously calculated mask image forms the Base database. The bases are in a 3D space and must thus be described by three spatial coordinates. The three Spatial coordinates (x, y, z) of each base for splines is given by the x, y Pixel positions of the mask image pixels and the gray value of each mask pixel (z position) formed. Since the gray values in a mask image anyway Correspond to the height information of the underlying microscope image, you can use the 3D pseudo image as a representation of the height of the interpret the underlying microscope image.

If you therefore specify an array of support points which contains all or a suitable subset of the mask image points and mask image point coordinates, a spline network of selectable density can be placed over the base point array. In Fig. 4a obtained in this manner three-dimensional pseudo image 28 is shown.

As shown in FIG. 4b, a fine structure can then be placed over this surface by suitable triangularization and shading, such as the so-called Gouraud shading. With ray tracing algorithms, surface reflections and shadows can also lead to very realistic surfaces 28 '.

Furthermore, it is necessary to link the three-dimensional pseudo image 28 with a texture 29 . Texture 29 is understood to be a basic element for the surface design of virtual structures, the aim being to give surfaces a natural and realistic appearance. A texture 29 is thus generated in method step 18 from the previously provided multifocus image 15 . For this purpose, for example, the previously calculated deep-focus multifocus image 15 is now used as a texture image.

In order to link the otherwise determined information, which is present in particular in the multifocus image 15 , with the three-dimensional pseudo image 28 , the three-dimensional pseudo image 28 is linked with the texture 29 in method step 20 , as shown in FIGS. 1-3.

The texture 29 is understood to mean, as is usual in virtual reality, in particular an image which is suitably projected onto the surface of a virtual three-dimensional object by three-dimensional projection methods. To achieve the desired effect, the texture image must be appropriately aligned on the surface of virtual objects. For a suitable alignment, it is necessary to assign the texture 29 to the three-dimensional pseudo image 28 so that the affiliations of the pixel coordinates (x, y) of the mask image 17 and the multifocus image 15 are not destroyed. Each mask pixel with the gray value at the location (x _i , y _j ) is therefore assigned its corresponding multifocus pixel with the gray value at the exact same location (x _i , y _j ). If the multifocus image 15 has previously been altered by image analysis or by other image manipulation, then care must be taken that the affiliations of the pixel coordinates (x, y) of the mask image and the image analysis or any other manipulation of the multifocus image modified in any way is not lost.

In order to link the pseudo image 28 with the texture 29, the texture 29 is advantageously appropriately projected onto the three-dimensional pseudo image 28 . As a result, the two resources are merged in such a way that a three-dimensional object image 30 of the object 22 is produced. This object image 30 represents a virtual image in the sense of virtual reality.

As shown in the example according to FIG. 5, the basis for the texturing according to the invention is formed by the multifocus image itself, which was previously calculated. The already realistic-looking pseudo image 28 and the mask image 17 are suitably aligned in such a way that the height information of the pseudo image 28 and the image information of the mask image 17 , ie the texture, are superimposed on one another with pixel accuracy. The multifocus texture image, that is to say the texture 29 , is projected onto the three-dimensional pseudo image 28 , so that each pixel of the multifocus texture image 29 is imaged exactly on its associated pixel in the three-dimensional pseudo image 28 . As a result, a focussed object image 30 of the object 22 is generated as a virtual image by merging virtual and real imaging techniques.

With the flow of the process schematically illustrated in FIGS. 1-3 according to the invention novel illustration is based on real measured values determined an actually existing object 20, which are merged so that a virtual real three-dimensional illustration of the light-microscopic data is created. In comparison with conventional virtual techniques, a real image of an object 22 is carried out in the present invention. Data on the image sharpness, the topology of the object and the exact position of sharp partial areas of an image in three-dimensional space are recorded on the real object 22 . This real data then serves as the basis for the generation of a virtual image in a three-dimensional space. The virtual image, which determines data from the real images, such as image information, sharpness and three-dimensionality and at the same time maps them, creates a clear added value compared to conventional light microscopy.

A new type of light microscopy is thus in accordance with the invention proposed, the core properties of the detection of real z. B. more light microscopic Object data and their summary presentation in one is three-dimensional virtual space. In this sense, the invention Methods can be referred to as "virtual reality 3D light microscopy". In the "Virtual Reality 3D light microscope" can generate the Images of reality (3D, sharpness, etc.) are also currently available known and future developed methods and procedures of virtual Imaging techniques are influenced.

After in the previous embodiment, the manual and fully automatic generation of a "virtual reality three-dimensional Light microscopy image "was described in a further embodiment Process for the visualization, manipulation and analysis of "Virtual Reality three-dimensional light microscopy images ".

In order to visualize the object image data 30 due to the transformation of real microscopic data into a virtual space, the microscopic data of the object image 30 are now available as three-dimensional, in-depth images.

The surface of the object image 30 can now be illuminated by virtual lamps in order to visually highlight certain details of the microscopic data. The virtual lamps can be placed anywhere in the virtual room and the properties of the virtual lamps such as radiation characteristics or light color can be set flexibly.

This procedure allows the creation of much better and permanent preserved microscope images for teaching and documentation purposes.

The images can be viewed in space using rotation and translation operators rotate and scale as required. This operation allows viewing the Images from angles that are impossible in a normal microscope. By gradually changing the orientation of a "Virtual Reality 3D Light microscopy image "and saving these individual images can also Animation sequences are created that represent a movement of "Virtual Reality pretend three-dimensional light microscopy image ". By saving of these individual images as a film sequence (e.g. in AVI, MOV data format, etc.) these animation sequences can be played.

The data can also be manipulated. The image of the three-dimensional pseudo-image lie as bases for one three-dimensional spline interpolation. Through Gouraud shading and ray tracing you can finally get this three-dimensional data three-dimensional assign effective surface.

A central role for data manipulation, the z. B. for measurement purposes or can be used to emphasize certain details play the x, y, z bases.

A multiplication of the z values by a number would mean, for example, an extension or compression of the height relief. Through targeted manipulation of individual support points, certain parts of the 3D profile of the three-dimensional pseudo image 28 can be manipulated individually.

Through image analytical manipulations of the projected multifocus Textured images can also provide analytical results such as marking individual ones Image objects, edge highlighting, object classifications, binary images, Image improvements etc. are projected on. This is done by Image-analytically changed initial image (multifocus texture image) as a new one "manipulated" multifocus texture image is used and the new image as texture on the three-dimensional surface of the 3D pseudo image projected. In this way can also use image analysis to manipulate images (new textures) three-dimensional pseudo image can be merged.

There are both possibilities of 3D data manipulation such. B. the manipulation of the bases for the spline interpolation as well Possibility of manipulating the multifocus image using image analysis methods.

A fusion of these two representations can lead to an added value in the microscopic image representation, since the object images 30 carry, in addition to the three-dimensional representation, a location-specific overlay of the image-analytically manipulated multifocus images.

Because of the transformation of the data of the real object 22 into data which are present in a virtual space, the three-dimensional data can now be measured with regard to their volume, the surface or the roughness etc.

Another improvement allows the combination of image analysis measurement results obtained on the multifocus image and the three-dimensional Data measurements. By logically linking the three-dimensional data with suitable other three-dimensional objects, you can also create a Perform a variety of calculations on three-dimensional data.

Thus, the mere type of representation makes the two-dimensional Image analysis around a third dimension of image analysis and a topological Dimension of data analysis expanded.

By recording time series, ie by recording images of the object 22 at different successive times according to the described method, a further dimension is added to the data analysis, the temporal dimension. This then enables the representation of a time process, such as the change over time in object 22 in slow motion or in fast motion.

The method according to the invention is also suitable for stereo image generation and stereo image animation. Since the data of the object image 30 are three-dimensional, two views of a virtual microscope image can be calculated from any viewing angle. This allows a visualization of the "Virtual Reality 3D light microscope image" in the sense of a classic stereo image.

A visualization of the "Virtual Reality 3D light microscope image" can also be used output on a monitor, printer or plotter also through a Polarization shutter glasses or in anaglyph technique or by illustration above 3D cyberspace glasses are used.

By a perspective separated for the right and left eye Animation of a series of different views of the "Virtual Reality 3D Light microscope image ", can be one of those described above Visualization process also a moving stereo image one from real ones Microscopy data generated "Virtual Reality 3D light microscope image" are generated.

Since the data can be available in three dimensions for the left and right eye a perspective correct view of the "Virtual Reality 3D Light microscope image "can be calculated. Thus, the" Virtual Reality 3D Light microscope images "also on 3D output units such as 3D stereo LCD monitors or but cyberspace glasses are issued.

In the case of a 3D LCD stereo monitor, the current one is checked using an image analysis Measure the eye position of the viewer. This data then becomes the respective viewpoint calculated. From this follow the data on perspective View of the "Virtual Reality 3D light microscope image" for the right and left Viewer's eye. These two perspectives are calculated and based on the 3D LCD stereo monitor shown. For the viewer it is the Impression creates that the "Virtual Reality 3D light microscope image" in front of the room floating on the monitor. It is thus possible to record real microscopy data like this depict a three-dimensional spatial representation of reality arises. By temporally and perspectively correct picture sequences also spatially animated three-dimensional images more real Realize microscope images.

With cyberspace glasses, an image is separated into each eye for technical reasons displayed in perspective correct view. The viewer's brain creates a three-dimensional impression. By time and Perspectively correct image sequences can also be spatially animated here Realize three-dimensional images of real microscope images.

In a further embodiment of the invention, it is possible to combine the data obtained from the "virtual reality 3D light microscopy" with one another in such a way that time-changing processes can also be animated and visualized in the "virtual reality 3D light microscopy". In addition to the three spatial coordinates X, Y, Z, data manipulation can also be carried out for this purpose, which relates to the texture 29 - pure, sharply calculated image information of the object (multifocus image) or to temporal changes in the surface and / or the texture (time series of images) ,

As in the previously described methods, changes over time in microscopic objects can also be recorded at different times by repeatedly recording the same image stack in the z direction (in the direction of the optical axis of the imaging system). This creates a series of image stacks that correspond to the conditions in object 22 at different times. Both the three-dimensional microscopic surface structures and the microscopic image data itself can change over time.

Through a time series always of the same microscopic range, a series of mask images and associated multi-focus images, which follow one another in time, arises in such a way that

Mask [t ₁ ] → Mask [t ₂ ] →. , , → mask [t _n ] → mask [t _{n + 1} ]

and belonging

Multifocus [t ₁ ] → Multifocus [t ₂ ] →. , , →. , , → multifocus [t _n ] → multifocus [t _{n + 1} ]

In the event of changes in the topology over time

Mask [t _n ] not equal to mask [t _{n + 1} ] {n = 1, 2, 3, 4,. , .}

and applies to visual changes

Multifocus [t _n ] not equal to Multifocus [t _{n + 1} ] {n = 1, 2, 3, 4,. , .}

These time series can be generated both manually and automatically become.

By recording time sequences of mosaic multifocus images and the associated mosaic mask images is also a temporal kinetics of Surface changes and I or of image changes possible.

As shown in FIG. 6, the method sequence for generating an animation can be integrated into the method sequences already known from DE 101 49 357.6 and DE 101 44 709.4, so that a fully automatic sequence can also be implemented. For this purpose, the process sequence already known from these documents is supplemented by further automatable process steps. If the process sequence for creating a virtual reality object image 30 is started, a virtual reality object image 30 can be generated in step 32 as described above. This object image 30 can be animated as desired in step 34 . The animated image is then preferably stored in step 36 . In this way, mosaic images, mask images and mosaic multifocus images can be created and saved at specific times. These mask and multifocus images then serve as the starting basis for a combination of the associated mask and multifocus images.

In a second step, the associated masks and Multi-focus images for single images in "Virtual Reality 3D light microscopy" be put together.

The result is a time series of individual "Virtual Reality 3D light microscopy images". Each simultaneously contains the 3D information of the mask image and the projected texture of the multifocus image. If the object 22 changes over time, the individual images can differ in the three-dimensional topological appearance and / or in changes in the texture 29 .

A sequence of the individual pictures allows a temporal Animation of the recordings with the possibilities of "Virtual Reality 3D Light microscopy ".

Three-dimensional surface information can thus be changes over time of the surfaces, calculated in depth Show multifocus images and changes over time of these multifocus images.

The required mask images 17 and multi-focus images 15 can also be understood as a mosaic mask image and a multi-focus mosaic image created by repeated scanning of a surface of the object 22 at certain times.

By rotating these "Virtual Reality 3D Light Microscopy" images, the Simultaneously mapped features, such as three-dimensional Surface information, temporal changes in surfaces, in-depth calculations Multifocus images and changes over time of these multifocus images, also under other viewing angles. The record, the one describes three-dimensional surface, becomes a suitable, three-dimensional Subjected to transformation.

• - X, Y, Z - reine dreidimensionale Oberflächeninformationen des Objektes 22
• - Die Textur 29, also reine, scharfgerechnete Bildinformation des Objektes 22
• - Die zeitliche Veränderungen der Oberfläche und/oder der Textur als Zeitserie von Bildern

Bezugszeichenliste 10 Erzeugung eines Bildstapels eines Objektes
12 Speichern der Bilder des Bildstapels
14 Erzeugen eines Multifokusbildes und eines Maskenbildes
15 Multifokusbild
16 Erzeugen eines dreidimensionalen Pseudobildes
17 Maskenbild
18 Bereitstellen einer Textur
20 Verknüpfen der Textur mit dem Pseudobild
22 Objekt
24 Bildstapel
26 Einzelbild einer Fokusebene
28 dreidimensionales Pseudobild
28' dreidimensionales Pseudobild mit Oberflächenstruktur
29 Textur
30 Objektbild
32 Erzeugen eines virtual Reality Bildes
34 Erstellen einer Animation
36 Speichern des Bildes
In summary, the described image of "virtual reality 3D light microscopy" can be understood as a simultaneous image of five dimensions of microscopic data of an object 22 . The five dimensions are:

• - X, Y, Z - pure three-dimensional surface information of the object 22
• - The texture 29 , that is to say pure, sharply calculated image information of the object 22
• - The temporal changes in the surface and / or texture as a time series of images

REFERENCE NUMERALS 10 producing an image of an object stack
12 Save the images in the image stack
14 Creation of a multifocus image and a mask image
15 multifocus image
16 Generation of a three-dimensional pseudo image
17 mask image
18 Providing a texture
20 Link the texture to the pseudo image
22 object
24 image stacks
26 Single image of a focus plane
28 three-dimensional pseudo image
28 'three-dimensional pseudo image with surface structure
29 texture
30 object image
32 Creation of a virtual reality image
34 Creating an animation
36 Saving the image

Claims (20)

1. A method for displaying a three-dimensional object ( 22 ) as an object image ( 30 ), wherein the object ( 22 ) first records an image stack ( 24 ) from a plurality of images ( 26 ) in different focal planes, characterized in that from the image stack ( 24 ) a three-dimensional height relief image ( 28 ) is generated and the three-dimensional height relief image ( 28 ) is combined with a texture ( 29 ), in particular the texture ( 29 ) is projected onto the three-dimensional height relief image ( 28 ). 2. The method according to claim 1, characterized in that the texture and the three-dimensional height relief image ( 28 ) are obtained from the data of the individual images ( 6 ) of the image stack ( 24 ). 3. The method according to claim 1 or 2, characterized in that the three-dimensional height relief image ( 28 ) is formed by contour lines which run through support points which are connected to one another by interpolation. 4. The method according to any one of claims 1 to 3, characterized in that the texture ( 29 ) is projected onto the three-dimensional height relief image ( 28 ). 5. The method according to claim 4 characterized in that the texture (29) is aligned to project the three-dimensional height relief image (28) pixel precision on the three-dimensional height image relief (28). 6. The method according to any one of claims 1 to 5, characterized in that the three-dimensional height relief image ( 28 ) changes before combining with the texture ( 29 ), in particular by triangulation, shading and / or ray tracing algorithms is provided with a virtual surface. 7. The method according to any one of claims 1 to 6, characterized in that the texture ( 29 ) is obtained from the data of a multifocus image ( 15 ) which carries the in-depth information of the object ( 22 ). 8. The method according to any one of claims 1 to 7, characterized in that the three-dimensional height relief image ( 28 ) is obtained from the data of a mask image ( 17 ) which carries the height information of the focus planes. 9. The method according to claim 3, characterized in that the three-dimensional height relief image ( 28 ) is changed by stretching or compressing the support points of the contour lines before or after the combination with the texture ( 29 ). 10. The method according to any one of claims 1 to 9, characterized in that the object image ( 30 ) manipulated image analysis, in particular combined with another texture. 11. The method according to any one of claims 1 to 10, characterized in that the generation of the object image ( 30 ) is started manually or automatically after the recording of the individual images ( 26 ) of the image stack ( 24 ). 12. The method according to any one of claims 1 to 7, characterized in that a plurality of temporally successive object images ( 30 ) are generated. 13. The method according to any one of claims 1 to 12, characterized in that the object image ( 30 ) is output on an output device, in particular a monitor, plotter, printer, LCD monitor or cyberspace glasses. 14. The method according to any one of claims 1 to 13, characterized in that the object image ( 30 ) changes before output, in particular illuminated with a virtual lamp, processed by rotation or translation operators and / or exposed to virtual physical laws. 15. The method according to any one of claims 1 to 14, characterized in that a plurality of object images ( 30 ) as a chronological sequence of object images ( 30 ), in particular as a film or animation, are output to the output device. 16. The method according to claim 15, characterized in that the plurality of object images ( 30 ) are converted into one another by morphing. 17. Device for displaying a three-dimensional object ( 22 ) as an object image ( 30 ) with an imaging system, in particular a microscope for imaging the object ( 22 ), a computer, actuators for targeted, rapid change in position of the object ( 22 ) in x, y, and z-direction, a recording device, in particular an analog or digital CCD camera for recording an image stack ( 26 ) of individual images ( 24 ) in different focal planes of the object ( 22 ), a control device for controlling the hardware of the imaging system, characterized in that a Analysis device for generating a three-dimensional height relief image ( 28 ) and a texture ( 29 ) from the individual images ( 26 ) of the image stack ( 24 ) and a control device for combining the three-dimensional height relief image ( 28 ) with the texture ( 29 ) is provided. 18. The apparatus according to claim 17, characterized in that an output device for outputting the object image ( 30 ), in particular a monitor, plotter, printer, LCD monitor or cyberspace glasses, is provided. 19. The apparatus of claim 17 or 18, characterized in that a device for data analysis of the object image ( 30 ) is provided. 20. Device according to one of claims 17-19, characterized in that the analysis, control device and / or the device for data analysis of the object image ( 30 ) are realized in the computer.