US20100201784A1 - Method for the microscopic three-dimensional reproduction of a sample - Google Patents
Method for the microscopic three-dimensional reproduction of a sample Download PDFInfo
- Publication number
- US20100201784A1 US20100201784A1 US12/680,056 US68005608A US2010201784A1 US 20100201784 A1 US20100201784 A1 US 20100201784A1 US 68005608 A US68005608 A US 68005608A US 2010201784 A1 US2010201784 A1 US 2010201784A1
- Authority
- US
- United States
- Prior art keywords
- sample
- image information
- image
- reference object
- light sheet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/24—Base structure
- G02B21/241—Devices for focusing
- G02B21/244—Devices for focusing using image analysis techniques
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
Definitions
- the invention is directed to a method for the three-dimensional imaging of a sample in which image information from different depth planes of the sample is acquired from at least one detection direction and is stored in a spatially resolved manner, i.e., correlated to its spatial coordinates X, Y, Z, and in which the three-dimensional image of the sample is subsequently reconstructed electronically from this image information.
- the image quality depends substantially on the position and aperture size of the measurement diaphragm relative to the position of the pinhole diaphragm, i.e., the pinhole diaphragm should be aligned in such a way that it is imaged on the measurement diaphragm with great accuracy.
- SPIM single plane illumination microscopy
- SPIM technology In contrast to confocal laser scanning microscopy (LSM), SPIM technology is based on widefield microscopy and makes it possible to generate three-dimensional images of a sample based on optical sections through different planes of the sample.
- LSM confocal laser scanning microscopy
- SPIM SPIM
- fluorophores which are contained inherently in the sample or are introduced into the sample specifically for contrasting are excited by laser light, the laser radiation being shaped to form a light sheet, as it is called.
- a selected depth plane in the sample is illuminated by the light sheet, and image information is obtained from this sample plane in the form of an optical section by imaging optics.
- the excitation beam path is separated from the detection beam path.
- the length and width of the light sheet extend perpendicular to the optical axis of the microscope objective in coordinates X and Y and are adapted to the sample field to be analyzed and the light sheet has a thickness extending in direction of the optical axis of the microscope objective within a range of a few ⁇ m.
- the sample to be analyzed for example, a medaka embryo, is embedded in an aqueous gel in which it continues to live for a certain period of time.
- the light sheet can either be generated directly, e.g., through cylindrical optics, or as a quasi light sheet by scanning a line-shaped beam over the object area.
- a mirror is generally used for this purpose. This mirror is arranged in the sample space in place of the sample and reflects the light sheet toward the detection objective. An adjustment can be carried out by means of reference structures on the mirror (e.g., in the form of scratches) and based on the thickness and position of the light stripe which is generated by the light sheet and visible on the mirror.
- the reference object should have a structure serving as a reference structure so that the stored spatial coordinates X, Y, Z of this reference structure can be used to obtain transformation operators.
- overlapping is meant within the meaning of the invention that the image information which has the same origin in the reference structure and in the sample but which does not overlap in the individual image stacks because of the different detection directions is brought into alignment in the registered three-dimensional view.
- the transformation operators which are related to their respective spatial coordinates X, Y, Z in the form of a rule for translation, rotation and deformation of the image information which is acquired from the individual detection directions and stored are used for this purpose. This rule is then taken as a basis for the reconstruction of the three-dimensional image of the sample.
- An object preferably in the form of a pin, a needle, a fiber or a thread, can be used as a reference object. It is also possible to use particles, e.g., fluorescing or non-fluorescing beads or microspheres.
- the object of the invention is met by a method of the type mentioned above in a second variant which relates to the above-mentioned method of selective plane illumination microscopy (SPIM) and in which the three-dimensional imaging:
- the sample is obtained from a sample comprising a fluorescing substance or from a sample to which a fluorescing substance is added, the sample is illuminated by an excitation beam which is shaped as a light sheet, and the image information from the sample is obtained based on the emission radiation coming from the sample.
- SPIM selective plane illumination microscopy
- this light sheet is brought into the focus plane of the microscope objective. Tilting of the light sheet relative to the focus plane is compensated by orienting the light sheet on the basis of the homogeneity of the imaging sharpness over the entire image field.
- This adjustment of the light sheet preceding the acquisition of the image information is carried out on the basis of the reference structure in-situ, i.e., while the sample remains in the sample space.
- the precise orientation of the light sheet in the focus plane is an essential condition for a higher-quality three-dimensional imaging of the sample.
- the illumination light is structured, for example, in that a grating is introduced in a plane of the illumination beam path conjugate to the object plane of the detection objective so that the light sheet is modulated in a grating-shaped manner in the focal plane.
- a fluorescing sample is located in the area of the light sheet, the fluorescence is modulated correspondingly over the image field and the grating is made visible as a reference structure.
- the reference object when applying selective plane illumination microscopy (SPIM), should have a structure which is usable as a reference structure so that the spatial coordinates X, Y, Z which are acquired from this reference structure and stored can advantageously be used to obtain the transformation operators.
- SPIM selective plane illumination microscopy
- the reference structure impressed on the illumination light is referred to in the following as an indirect reference structure and the structure of the reference object is referred to as a direct reference structure.
- the direct reference structure instead of an indirect reference structure for adjusting the light sheet.
- the light sheet is displaced in the detection direction until the direct reference structure is sharply imaged in the observation plane.
- the light sheet is oriented in such a way that the imaging of the direct reference structure is carried out homogeneously over the image field.
- the advantage of using a reference structure to assist in adjusting the light sheet and in acquiring the image information consists in that the reference structure is not dependent on the sample structure and is therefore less subject, or not at all subject, to the artifacts predominantly caused by the sample itself.
- An object for example, in the form of a pin, a needle, a fiber or a thread, or the like can also conceivably be used as reference objects in connection with selective plane illumination microscopy (SPIM). However, they must be made at least partially from a fluorescing material.
- SPIM selective plane illumination microscopy
- a plurality of reference objects in the form of particles which fluoresce when excited are positioned together with the sample in a gel.
- at least three particles should be used if these particles are to be used for registration.
- more particles are advantageous for adjusting the light sheet.
- the sample itself can be dyed, for example, with enhanced green fluorescent protein (EGFP) so that it fluoresces green when excited.
- EGFP enhanced green fluorescent protein
- Agarose gel can advantageously be used as a gel for embedding the sample.
- the detection direction is preferably changed by rotating the sample relative to the detection direction defined by the detection objective.
- the purpose of this procedure is to avoid shadow effects or to achieve a more homogeneous resolution.
- the image information originating from the sample is not considered at all at first; rather, a comparison of structures is carried out based only on the direct reference structure.
- filtering is carried out by means of hardware or software, i.e., the image information of the reference structure is separated from the image information of the sample. This is accomplished, for example, with the help of different shapes of the reference structure or based on different emission spectra or polarization characteristics.
- the amount of image information of the reference structure is less by orders of magnitude than the amount of image information of the sample. This substantially reduces the resources required for processing.
- the transformation operators are obtained in the form of a rule for translation, rotation and deformation so that the image information which is acquired from the individual detection directions and stored in image stacks can be overlapped with the image information of a different detection direction.
- This rule is given, for example, in the form of a matrix which renders the translation, rotation and possibly also deformation operations to be applied to the coordinates of the voxels of the image stack.
- the matrix obtained on the basis of the image information of the reference structure is now applied to the image information of the actual sample.
- the image information of the sample and the image information of the at least one reference object are acquired either simultaneously or successively in time.
- the image information of the sample and reference structure is acquired simultaneously through different detectors, for example, through spectral splitting when the sample and reference object have different emission characteristics.
- the light sheet is moved into the focus plane of the microscope objective when it is displaced in the detection direction, while tilting of the light sheet relative to the focus plane is compensated by the orientation of the light sheet based on the homogeneity of the imaging sharpness over the entire image field.
- the essential advantages of using reference structures which originate from reference objects and which can also be impressed upon the illumination light consist in that, first, a simple in-situ adjustment of the light sheet is possible when applying selective plane illumination microscopy (SPIM) and, second, the computing effort in the reconstruction of the three-dimensional image of the sample is substantially reduced compared to the prior art when reference objects are used.
- SPIM selective plane illumination microscopy
- the orientation of the light sheet can be carried out automatically in that the light sheet is moved through the sample space in z-direction during which an image stack is recorded, an evaluation of this stack is carried out with respect to the contrast of the individual images, the optimal light sheet position relative to the focus plane of the microscope objective is determined based on the contrast, and the light sheet is moved into this position by means of an adjusting device.
- the method steps for adjusting the light sheet and the method steps for reducing computation in reconstructing the three-dimensional image of the sample based on the image information acquired from the sample are based on a sample structure suitable as a reference structure instead of on a reference structure originating from a reference object.
- the sample simultaneously takes on the function of a reference object within the meaning of the invention.
- FIG. 1 a schematic view illustrating the principle of microscopic recording of a three-dimensional image of a sample using the single plane illumination microscopy (SPIM) method;
- SPIM single plane illumination microscopy
- FIG. 2 a first embodiment example of the method according to the invention in which a sample in the form of a zebrafish embryo which is dyed with enhanced green fluorescent protein (EGFP) and therefore fluoresces green when excited is embedded in agarose gel and fluorescing particles are used as reference structure;
- EGFP enhanced green fluorescent protein
- FIG. 3 a second embodiment example with a reference structure which is impressed on the light sheet by means of a grating
- FIG. 4 a third embodiment example in which the reference structure is generated in the form of a structured quasi light sheet by the movement of a scanner mirror and temporal beam modulation;
- FIG. 5 a fourth embodiment example illustrating the light sheet adjustment based on a light sheet on which a reference structure is impressed.
- a sample 1 is embedded in a gel which is shaped to form a circular cylinder 2 .
- the circular cylinder 2 is located in a sample chamber 4 and is supported inside the sample chamber 4 so as to be rotatable around an axis of rotation 3 and displaceable translationally in coordinate directions X, Y, Z together with the sample 1 .
- the sample 1 is a three-dimensional object, for example, a biological substance.
- the circular cylinder 2 with the enclosed sample 1 is filled with an immersion medium, for example, water or PBS (phosphate buffered saline).
- the sample chamber 4 is open at its upper side so that the gel with the enclosed sample 1 and the immersion medium can be introduced from the top.
- the gel can be made of a mixture of approximately 98% water and 2% agarose.
- other materials can be used for embedding the sample 1 ; for example, glass cylinders filled with a liquid can also be used.
- the walls of the sample chamber 4 can be made of glass or some other transparent material with a suitable refractive index.
- a laser light source (not shown) radiating a light bundle at a wavelength of, e.g., 488 nm is provided for illuminating the sample. It is advantageous when the radiation intensity in the cross section of this light bundle is initially homogenized with suitable optical means before proceeding to be shaped as a light sheet.
- the shaping of the light sheet is carried out either directly, e.g., through cylindrical optics, or as a quasi light sheet through the scanning of a line-shaped beam over the object field.
- the illumination light shaped as the light sheet passes through the sample chamber wall 6 in direction of arrow B into the immersion medium and must be positioned in such a way that it there illuminates the focus plane of a microscope objective (not shown).
- the invention allows the adjustment to be carried out based on a known reference structure which is introduced into the sample space next to the sample indirectly or directly.
- a plane of the sample 1 namely, the plane located in the focus plane, is illuminated.
- image information is initially acquired from the detection direction indicated by arrow D from the plane of the sample 1 that is actually located in the focus plane by the microscope, whose functional part is the microscope objective.
- the sample 1 is then repeatedly displaced by a given amount or continuously in direction of coordinate Z.
- a spatially resolving optoelectronic converter preferably a CCD camera
- image information is acquired from the planes of the sample 1 which are located in the illuminated focus plane and is digitized and subsequently electronically stored so as to be correlated with its spatial coordinates X, Y, X.
- a three-dimensional image of the sample is already obtained at this stage; however, its quality is generally still severely limited by the shadow effects and aberrations caused by the sample.
- the resolution is generally inhomogeneous because it is determined laterally by the numerical aperture of the detection objective and axially by the light sheet thickness.
- the detection direction is changed repeatedly by rotating the sample 1 around the axis of rotation 3 , and image information is acquired again from each changed detection direction from different depth planes of the sample 1 and is stored in a spatially resolved manner.
- a three-dimensional image of the sample 1 which is substantially free from image artifacts such as shadows caused by opaque constituents of the sample and aberrations is electronically reconstructed by registration and fusion from the stored image information that was obtained from all of the given detection directions. At the same time, an extensively uniform resolution is achieved in this method.
- an image stack of 1000 ⁇ 1000 ⁇ 1000 image pixels is recorded in coordinates X, Y, Z at 16 bits by a commercial CCD camera for every given detection direction in order to obtain image information.
- the amount of data to be acquired and processed already amounts to 16 GB.
- the amount of data to be acquired and processed is 64 GB. If the difficulties mentioned above with respect to evaluation are still to be overcome, the total expenditure is so great that conventional commercially available computer systems are overtaxed.
- the invention provides that, in addition to the image information from the sample 1 , image information of one or more reference objects positioned next to the sample 1 or in the sample 1 is also acquired from each of the given detection directions and is stored in a spatially resolved manner together with the image information of the sample 1 , and the spatial coordinates X, Y, Z of the image information obtained from the reference object or reference objects is taken as a basis for the reconstruction of the three-dimensional image of the sample 1 .
- the registration is carried out exclusively on the basis of the substantially smaller amount of image information obtained from the reference structure, and a transformation rule in the form of a matrix is determined therefrom and is then applied to the image information of the actual sample.
- the image stacks of the sample structure which are registered after applying the transformation are then fused to form a three-dimensional picture.
- the sample in the form of a zebrafish embryo which is dyed with enhanced green fluorescent protein (EGFP) and accordingly fluoresces green when excited is embedded in an agarose gel.
- EGFP enhanced green fluorescent protein
- a three-dimensional image of this zebrafish embryo is to be recorded by means of the above-described method of single plane illumination microscopy (SPIM).
- SPIM single plane illumination microscopy
- red-fluorescing particles for example, small polystyrene beads tagged with BODIPY 630 dye are embedded in the agarose gel along with the zebrafish embryo.
- concentration of these particles is high enough that they are distributed over the sample space within the sample chamber virtually homogeneously.
- the light sheet can be adjusted in a simple manner as is illustrated in FIG. 2 .
- the light sheet is moved over the sample in z-direction and irradiates the fluorescing particles as well as the sample.
- the adjustment steps can take place automatically.
- the light sheet is moved through the sample space in z-direction and an image stack is recorded. This stack is then evaluated, for example, with respect to the contrast of the individual images, so that the optimal light sheet position can be determined quickly.
- the actual image recording is carried out.
- image information from a plurality of planes lying at different depths in the zebrafish embryo is first acquired from a given detection direction along with the image information of the particles, and this image information is stored in a spatially resolved manner.
- the detection direction is then changed repeatedly and image information is in turn acquired from each changed detection direction from planes of different depths in the zebrafish embryo and particles and is stored in a spatially resolved manner.
- the computing capacity required for processing the image information of the sample acquired in this way to arrive at a three-dimensional image of the sample can be reduced according to the invention as will be explained in the following.
- the data in the ideal case of a monochrome picture of these particles is reduced to 100 bits per detection direction.
- the total image which generally initially contains the image information of the sample and particles, must first be segmented i.e., the particles structures are separated from the sample structures by filtering, the coordinates of the particle positions are acquired and are then stored as a monochrome model.
- the filtering can be carried out by spectral separation using hardware.
- the resulting relatively small amount of data can easily be processed in real time by commercial computers, especially as the particle emission should be substantially unaffected by image artifacts caused by the sample.
- the particles form 100 fixing points on the basis of which displacements, rotations and deformation of the sample substance are calculated from one detection direction to the other and then corrected computationally.
- the size of the particles is advantageously selected in such a way that they do not have a scattering effect and also do not act like spherical lenses in the sample, in this case, the zebrafish embryo, and therefore cannot result in excessive artifacts.
- the size of the particles is preferably 100 nm to 1 ⁇ m.
- the gel with the enclosed zebrafish embryo and the enclosed particles is first placed in the sample chamber in a first viewing direction of the microscope objective on the zebrafish embryo and the particles, and image information of a first image stack is recorded.
- the gel with the zebrafish embryo and the particles is then rotated by 90° around the axis of rotation, and image information of a second image stack is recorded. Similarly, a 90-degree rotation is then carried out twice and a third and fourth image stack is recorded.
- the light sheet is adjusted, advantageously automatically, based on the particles in that the light sheet is moved over the sample space in z-direction and the images obtained in so doing are evaluated for maximum sharpness, the sample is focused in a region of interest (RIO) by displacement in z-direction, a detection filter is selected so that the particles stand out against the background with a sufficiently good signal, a first image stack is acquired from the first detection direction (0°), where the image stack is recorded once for green fluorescence and immediately thereafter a second time for red fluorescence so that image information of the sample (green) and image information of the particles (red) is acquired, or both color channels, red and green, are recorded simultaneously by means of a detection system suitable for spectral splitting, the sample is rotated by 90° for orientation in three additional detection directions which consequently enclose angles of 90°, 180° and 270° with the first detection direction; another image stack is recorded for the green fluorescence and for the red fluorescence from each of these detection directions, wherein the image information overlaps in the image stacks.
- the three-dimensional image of the sample is reconstructed from the resulting image information, for example, in that a segmentation is carried out for the red fluorescence for purposes of data reduction in such a way that a three-dimensional monochrome model is formed.
- This is followed by the registration of the image stacks obtained with the red fluorescence, i.e., the necessary transformation operators are determined for each image stack, e.g., in the form of a matrix, for translation, rotation and deformation which are suitable for making the image information of each image stack overlap with the image information of the other image stacks.
- the transformation operators determined in this way are then applied to the green channel, i.e., to the image stack obtained with the green fluorescence, which results in registered image stacks with image information of the sample.
- the latter are combined by a fusion step resulting in an individual high-quality three-dimensional image of the sample.
- a dyed polyester or polystyrene thread instead of the fluorescing particles is embedded in agarose gel next to the sample, for example, a zebrafish embryo.
- the thread For adjusting the light sheet, it must be ensured that a portion of the thread is located in the focus plane of the detection objective.
- the thread can first be moved in z-direction together with the sample by evaluation of a simple transmitted light contrast until it is located at least partially in the focus plane, which can be judged based on the sharpness of its imaging.
- the light sheet is then oriented in such a way that the portion of the thread located in the focus plane is also sharply imaged in SPIM mode.
- image stacks are recorded from different detection directions by the method steps described above such that at least one identical portion of the fiber always appears in the image stacks. Since the amount of image information of the thread as well as the amount of image information of the particles in the first application example is substantially less than the amount of image information of the sample, a substantial reduction in the data volume to be processed is also achieved in this case.
- the three-dimensional image of the sample is reconstructed in that the same portion of the thread is first separated from the sample structure for every detection direction, which can be accomplished, for example, simply by cropping the image.
- the registration for the thread images takes place and a transformation rule in the form of a matrix is determined and then applied to the sample information that is actually of interest. The fusion is then carried out, resulting in an individual high-quality three-dimensional image of the sample.
- a needle is embedded as reference object in an agarose gel.
- the tip of the needle is advantageously arranged in such a way that it is located exactly in the axis of rotation 3 (see FIG. 1 ).
- sample is also positioned in the immediate vicinity of the needle tip, which can be achieved, e.g., by pinning it on the latter.
- the needle tip For light sheet adjustment, it must be ensured that the needle tip is located in the focus plane of the detection objective. For this purpose, it can be moved together with the sample in z-direction by evaluation of a simple transmitted light contrast until it is sharply imaged.
- the region of interest (RIO) to be recorded is selected in such a way that the tip of the needle appears in the image stacks.
- the image recording is greatly facilitated when the needle tip is positioned in the axis of rotation 3 and the sample is placed in its immediate vicinity because the sample generally does not move out of the image field during a rotational movement. Further, it is clear that the absolute position of the needle tip has not changed during the rotational movement and, therefore, presents a fixing point. That is, when the positions of the sample structures for each image stack are determined relative to the needle tip, only a simple rotational movement is required for the registration, because the needle tip does not undergo any translational movement. Therefore, the registration algorithms proceed very efficiently.
- Another advantage in using a needle tip consists in that it can be used to calibrate the drives provided for an automatic adjustment of the light sheet.
- an indirectly generated reference structure in the form of a structured light sheet is used.
- the reference structure can be generated, for example, as is shown in FIG. 3 .
- a grating 9 is positioned in a plane 8 conjugate to the object space. Different diffraction orders 11 then result in the pupil plane 10 of the illumination objective, and an image of the grating is in turn formed in the sample space.
- This grating causes an intensity distribution which is modulated over the width of a light sheet 12 .
- the intensity distribution of the light sheet 12 is shown in a side view on the object plane of a detection objective (not shown), the light sheet 12 being oriented perpendicular to the drawing plane, and the reference structure 13 (see FIG. 3 b ) lies in the range of the depth of field of the illumination objective.
- the dashed line in the light sheet 12 in FIG. 3 a represents the range of the depth of field.
- the intensity distribution of the light sheet 12 causing the reference structure 13 is shown in FIG. 3 b in a top view of the object plane.
- a comparable intensity distribution is also yielded, for example, by an arrangement or process as shown in FIG. 4 .
- the light sheet 12 is generated here as a quasi light sheet through the movement of a scanner mirror 14 in a pupil plane of the illumination beam path.
- a substantially line-shaped beam is accordingly moved over the image field of the detection objective.
- the integration period of the image recording is greater than or equal to the scanner period and is synchronized as far as possible with the scanner movement in order to prevent temporal intensity modulations in the image recording.
- a deliberate temporal modulation can be introduced in addition, e.g., by using an acousto-optical modulator 15 , so that a reference structure 13 results in the sample space due to a reseau pattern in the intensity distribution.
- the fluorescing sample When the fluorescing sample is illuminated by the structured light sheet, the fluorescence proceeding from the sample is also spatially modulated.
- the grating is then clearly visible in the image recording when the light sheet overlaps with the focus plane of the detection objective as is shown on the right-hand side of FIG. 5 .
- the grating is virtually undetectable as is illustrated on the left-hand side of FIG. 5 .
- the grating image can be used for very accurate adjustment of the light sheet.
- the sample is not sufficient by itself because it has a defined structure only in exceptional cases.
- the advantage of using an indirectly generated reference structure over a directly generated reference structure is that the sample can be acquired after adjustment without interference from any reference objects in the normal SPIM mode either by simply removing the grating from the beam path or by halting the temporal intensity modulation during the scanner movement.
- a disadvantage compared to the use of reference objects consists in that indirect structuring cannot be used to reduce the computing capacity in the three-dimensional imaging of the sample.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Microscoopes, Condenser (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
A method for the three-dimensional imaging of a sample in which image information from different depth planes of the sample is stored in a spatially resolved manner, and the three-dimensional image of the sample is subsequently reconstructed from this stored image information is provided. A reference structure is applied to the illumination light, at least one fluorescing reference object is positioned next to or in the sample, images of the reference structure of the illumination light, of the reference object are recorded from at least one detection direction and evaluated. The light sheet is brought into an optimal position based on the results and image information of the reference object and of the sample from a plurality of detection directions is stored. Transformation operators are obtained on the basis of the stored image information and the reconstruction of the three-dimensional image of the is based on these transformation operators.
Description
- The present application claims priority from PCT Patent Application No. PCT/EP2008/007686 filed on Sep. 16, 2008, which claims priority from German Patent Application No. 10 2007 045 897.7 filed on Sep. 26, 2007, the disclosure of which is incorporated herein by reference in its entirety.
- 1. Field of the Invention
- The invention is directed to a method for the three-dimensional imaging of a sample in which image information from different depth planes of the sample is acquired from at least one detection direction and is stored in a spatially resolved manner, i.e., correlated to its spatial coordinates X, Y, Z, and in which the three-dimensional image of the sample is subsequently reconstructed electronically from this image information.
- 2. Description of Related Art
- It is known to obtain microscopic three-dimensional images, for example, using the principle of laser scanning microscopy (LSM). For this purpose, a pinhole diaphragm which is irradiated by laser light is projected in the sample and illuminates this sample with the intensity distribution of an Airy disk. The image of this illuminated sample point occurs on a measurement diaphragm which is conjugate to the illumination diaphragm with respect to position and size. To acquire an expanded sample field, this sample field is scanned by means of optical scanning. The image information which is obtained in so doing is stored electronically and the three-dimensional image is reconstructed from the stored data. In laser scanning microscopy (LSM), the image quality depends substantially on the position and aperture size of the measurement diaphragm relative to the position of the pinhole diaphragm, i.e., the pinhole diaphragm should be aligned in such a way that it is imaged on the measurement diaphragm with great accuracy.
- Another principle used for obtaining microscopic three-dimensional images which was discovered only recently is known as selective plane illumination microscopy or single plane illumination microscopy (SPIM).
- In contrast to confocal laser scanning microscopy (LSM), SPIM technology is based on widefield microscopy and makes it possible to generate three-dimensional images of a sample based on optical sections through different planes of the sample.
- The advantages of SPIM include faster acquisition of image information, reduced risk of bleaching out biological samples, and the possibility of adjusting the horizontal resolution of the image independent from the depth resolution.
- Basically, in the SPIM technique fluorophores which are contained inherently in the sample or are introduced into the sample specifically for contrasting are excited by laser light, the laser radiation being shaped to form a light sheet, as it is called. A selected depth plane in the sample is illuminated by the light sheet, and image information is obtained from this sample plane in the form of an optical section by imaging optics. For this purpose, the excitation beam path is separated from the detection beam path.
- The length and width of the light sheet extend perpendicular to the optical axis of the microscope objective in coordinates X and Y and are adapted to the sample field to be analyzed and the light sheet has a thickness extending in direction of the optical axis of the microscope objective within a range of a few μm.
- An example of the SPIM method is known from WO2004/053558 A1. The sample to be analyzed, for example, a medaka embryo, is embedded in an aqueous gel in which it continues to live for a certain period of time. The light sheet can either be generated directly, e.g., through cylindrical optics, or as a quasi light sheet by scanning a line-shaped beam over the object area.
- Since the excitation beam path in the SPIM method is decoupled from the detection beam path in a different way on principle compared to LSM, an additional adjustment step is required which involves the precise orientation of the light sheet on the focus plane of the detection objective that is used. According to the prior art, a mirror is generally used for this purpose. This mirror is arranged in the sample space in place of the sample and reflects the light sheet toward the detection objective. An adjustment can be carried out by means of reference structures on the mirror (e.g., in the form of scratches) and based on the thickness and position of the light stripe which is generated by the light sheet and visible on the mirror.
- One problem with this is that measurement is not carried out directly at the sample; another is that the adjustment is carried out at a laser wavelength which does not correspond to the emission wavelength coming from the sample, which is actually the wavelength of interest, so that any chromatic longitudinal errors, e.g., of the detection objective, are not taken into account.
- The adjustment of the light sheet based directly on the sample is desirable, but a suitable method for this is not known from the prior art.
- Another problem with the SPIM method becomes apparent when reconstructing three-dimensional images of a sample from image information acquired from different detection directions. The amount of data to be processed is so extensive that efficient algorithms are required for overlapping and fusing the stack of images obtained from the different detection directions through comparison of structures through translational, rotational and deformation operations within a suitable time period. This is all the more true when the comparison is based on structures presented by the sample itself, which may be more or less pronounced depending on the detection direction and rotational angle because, e.g., shadow effects, may play a role. The registration, i.e., the acquisition of image information and the necessary operation rules, is then very costly and time-consuming, and commercially available computers are usually not suitable for handling the required computing capacity.
- Therefore, it is the object of the invention to provide method steps which ensure an enhanced imaging quality in the three-dimensional imaging of a sample.
- This object is met by a method of the type mentioned above in a first variant which is applicable, for example, in microscopic three-dimensional imaging of samples by laser scanning microscopy (LSM) in that:
-
- at least one reference object is positioned next to or in the sample, the detection direction from which the image information is acquired is changed repeatedly relative to the sample and relative to the reference object,
- from each detection direction given by a change:
- image information is acquired from the reference object and is stored in a spatially resolved manner in spatial coordinates X, Y, Z, and
- image information is acquired from different depth planes of the sample and is stored in a spatially resolved manner in spatial coordinates X, Y, Z,
- based on the spatial coordinates X, Y, Z of the stored image information from the reference object, transformation operators can be obtained for translation, rotation and deformation, by means of which image information from the same source from the different detection directions can be overlapped, and
- subsequently, the reconstruction of the three-dimensional image of the sample from the spatial coordinates X, Y, Z of the image information obtained from the sample is based on these transformation operators.
- The reference object should have a structure serving as a reference structure so that the stored spatial coordinates X, Y, Z of this reference structure can be used to obtain transformation operators.
- By overlapping is meant within the meaning of the invention that the image information which has the same origin in the reference structure and in the sample but which does not overlap in the individual image stacks because of the different detection directions is brought into alignment in the registered three-dimensional view.
- The transformation operators which are related to their respective spatial coordinates X, Y, Z in the form of a rule for translation, rotation and deformation of the image information which is acquired from the individual detection directions and stored are used for this purpose. This rule is then taken as a basis for the reconstruction of the three-dimensional image of the sample.
- An object, preferably in the form of a pin, a needle, a fiber or a thread, can be used as a reference object. It is also possible to use particles, e.g., fluorescing or non-fluorescing beads or microspheres.
- The object of the invention is met by a method of the type mentioned above in a second variant which relates to the above-mentioned method of selective plane illumination microscopy (SPIM) and in which the three-dimensional imaging:
- is obtained from a sample comprising a fluorescing substance or from a sample to which a fluorescing substance is added,
the sample is illuminated by an excitation beam which is shaped as a light sheet, and
the image information from the sample is obtained based on the emission radiation coming from the sample. - According to the invention, it is provided that:
-
- at least one reference object comprising a fluorescing substance or a reference object having a fluorescing substance is positioned next to or in the sample,
- the detection direction from which the image information is acquired is repeatedly changed relative to the sample and relative to the reference object, wherein
- from each detection direction given by a change:
- image information is acquired from the reference object and is stored in a spatially resolved manner in spatial coordinates X, Y, Z, and
- image information is acquired from different depth planes of the sample and is stored in a spatially resolved manner in spatial coordinates X, Y, Z,
- based on the spatial coordinates X, Y, Z of the stored image information from the reference object, transformation operators can be obtained for translation, rotation and deformation, by means of which image information from the same source from the different detection directions can be overlapped, and
- subsequently, the reconstruction of the three-dimensional image of the sample from the spatial coordinates X, Y, Z of the image information obtained from the sample is based on these transformation operators.
- As was already mentioned above, it is important in selective plane illumination microscopy (SPIM) to correctly align the light sheet in the focus plane of the respective detection objective being used. Therefore, in a particularly preferred embodiment of the second variant of the invention mentioned above:
-
- a reference structure in the form of an irregular intensity distribution is applied to the illumination light, and
- prior in time to the acquisition of the image information of the reference object and of the sample from at least one of the given detection directions:
- images of the reference structure of the illumination light, of the reference structure of the reference object, or of a sample structure which is suitable as a reference structure are recorded and evaluated with respect to the sharpness of their imaging, then
- the light sheet is displaced in this detection direction into the position in which the sharpest imaging of the reference structure is achieved in an observation plane, and/or
- the light sheet is oriented in such a way that the reference structure is sharply imaged in a homogeneous manner over the entire image field.
- By displacing the light sheet in the detection direction, this light sheet is brought into the focus plane of the microscope objective. Tilting of the light sheet relative to the focus plane is compensated by orienting the light sheet on the basis of the homogeneity of the imaging sharpness over the entire image field.
- This adjustment of the light sheet preceding the acquisition of the image information is carried out on the basis of the reference structure in-situ, i.e., while the sample remains in the sample space. The precise orientation of the light sheet in the focus plane is an essential condition for a higher-quality three-dimensional imaging of the sample.
- The illumination light is structured, for example, in that a grating is introduced in a plane of the illumination beam path conjugate to the object plane of the detection objective so that the light sheet is modulated in a grating-shaped manner in the focal plane. When a fluorescing sample is located in the area of the light sheet, the fluorescence is modulated correspondingly over the image field and the grating is made visible as a reference structure.
- Another possibility is presented when the light sheet is generated, in effect, by the scanning of a line-shaped beam. In this case, a simple temporal modulation of the illumination intensity is sufficient for generating a spatial modulation in the sample space.
- Also, when applying selective plane illumination microscopy (SPIM), the reference object should have a structure which is usable as a reference structure so that the spatial coordinates X, Y, Z which are acquired from this reference structure and stored can advantageously be used to obtain the transformation operators.
- For the sake of clarity and to help distinguish between the two, the reference structure impressed on the illumination light is referred to in the following as an indirect reference structure and the structure of the reference object is referred to as a direct reference structure.
- It also lies within the scope of the invention to use the direct reference structure instead of an indirect reference structure for adjusting the light sheet. For this purpose, in a manner analogous to the above-described procedure based on the indirect reference structure, the light sheet is displaced in the detection direction until the direct reference structure is sharply imaged in the observation plane. Also, the light sheet is oriented in such a way that the imaging of the direct reference structure is carried out homogeneously over the image field.
- The advantage of using a reference structure to assist in adjusting the light sheet and in acquiring the image information consists in that the reference structure is not dependent on the sample structure and is therefore less subject, or not at all subject, to the artifacts predominantly caused by the sample itself.
- An object, for example, in the form of a pin, a needle, a fiber or a thread, or the like can also conceivably be used as reference objects in connection with selective plane illumination microscopy (SPIM). However, they must be made at least partially from a fluorescing material.
- It is particularly advantageous to use a needle with a fluorescing needle tip and to position the sample on the tip of the needle as will be explained in more detail in the following referring to an application example.
- Alternatively, it is conceivable that a plurality of reference objects in the form of particles which fluoresce when excited are positioned together with the sample in a gel. In so doing, at least three particles should be used if these particles are to be used for registration. Considerably more particles are advantageous for adjusting the light sheet.
- The sample itself can be dyed, for example, with enhanced green fluorescent protein (EGFP) so that it fluoresces green when excited. Agarose gel can advantageously be used as a gel for embedding the sample.
- In both variants of the invention, the detection direction is preferably changed by rotating the sample relative to the detection direction defined by the detection objective. The purpose of this procedure is to avoid shadow effects or to achieve a more homogeneous resolution.
- The computational electronic reconstruction will be described in more detail in the following. In both variants of the invention, this reconstruction consists in that
-
- on the one hand, the image information obtained from the different detection directions from different depth planes of the sample are registered in the form of image stacks, and
- on the other hand, these image stacks are then fused to form the three-dimensional image of the sample.
- When registering the image information, the image information originating from the sample is not considered at all at first; rather, a comparison of structures is carried out based only on the direct reference structure.
- To this end, filtering is carried out by means of hardware or software, i.e., the image information of the reference structure is separated from the image information of the sample. This is accomplished, for example, with the help of different shapes of the reference structure or based on different emission spectra or polarization characteristics.
- The amount of image information of the reference structure is less by orders of magnitude than the amount of image information of the sample. This substantially reduces the resources required for processing.
- Based on the direct reference structure, i.e., based on spatial coordinates X, Y, Z of the reference object, the transformation operators are obtained in the form of a rule for translation, rotation and deformation so that the image information which is acquired from the individual detection directions and stored in image stacks can be overlapped with the image information of a different detection direction.
- This rule is given, for example, in the form of a matrix which renders the translation, rotation and possibly also deformation operations to be applied to the coordinates of the voxels of the image stack. The matrix obtained on the basis of the image information of the reference structure is now applied to the image information of the actual sample.
- To acquire the image information from a plurality of different detection directions, it is possible to rotate the sample several times by 90° until a revolution of 360° is executed and image information from four detection directions is stored. Of course, any other rotational angle and a corresponding quantity of detection directions also lie within the scope of the invention.
- Further, according to the invention, the image information of the sample and the image information of the at least one reference object are acquired either simultaneously or successively in time. In an advantageous manner, the image information of the sample and reference structure is acquired simultaneously through different detectors, for example, through spectral splitting when the sample and reference object have different emission characteristics.
- In a particularly advantageous embodiment of the invention relating to the orientation of the sheet-shaped illumination light relative to the focus plane of a microscope objective when applying the SPIM method:
-
- a reference structure in the form of an inhomogeneous intensity distribution is applied to the illumination light and/or
- at least one reference object which is made of a fluorescing substance or has a fluorescing substance and which has a structure which can be used as a reference structure is positioned next to or in the sample, and/or
- a characteristic structure of the sample is determined as a reference structure,
- images of the reference structure of the illumination light, of the reference structure of the reference object or of the reference structure of the sample are recorded from at least one given detection direction and are evaluated with respect to the sharpness of their image, and then
- the light sheet is displaced in this detection direction into the position in which the sharpest imaging of the reference structure is achieved in an observation plane, and/or
- the light sheet is oriented in such a way that the reference structure is imaged in a uniformly sharp manner over the entire image field.
- As was already mentioned, the light sheet is moved into the focus plane of the microscope objective when it is displaced in the detection direction, while tilting of the light sheet relative to the focus plane is compensated by the orientation of the light sheet based on the homogeneity of the imaging sharpness over the entire image field.
- The essential advantages of using reference structures which originate from reference objects and which can also be impressed upon the illumination light consist in that, first, a simple in-situ adjustment of the light sheet is possible when applying selective plane illumination microscopy (SPIM) and, second, the computing effort in the reconstruction of the three-dimensional image of the sample is substantially reduced compared to the prior art when reference objects are used.
- In another embodiment, the orientation of the light sheet can be carried out automatically in that the light sheet is moved through the sample space in z-direction during which an image stack is recorded, an evaluation of this stack is carried out with respect to the contrast of the individual images, the optimal light sheet position relative to the focus plane of the microscope objective is determined based on the contrast, and the light sheet is moved into this position by means of an adjusting device.
- It is also conceivable and likewise lies within the scope of the invention that the method steps for adjusting the light sheet and the method steps for reducing computation in reconstructing the three-dimensional image of the sample based on the image information acquired from the sample are based on a sample structure suitable as a reference structure instead of on a reference structure originating from a reference object. In this case, the sample simultaneously takes on the function of a reference object within the meaning of the invention.
-
FIG. 1 a schematic view illustrating the principle of microscopic recording of a three-dimensional image of a sample using the single plane illumination microscopy (SPIM) method; -
FIG. 2 a first embodiment example of the method according to the invention in which a sample in the form of a zebrafish embryo which is dyed with enhanced green fluorescent protein (EGFP) and therefore fluoresces green when excited is embedded in agarose gel and fluorescing particles are used as reference structure; -
FIG. 3 a second embodiment example with a reference structure which is impressed on the light sheet by means of a grating; -
FIG. 4 a third embodiment example in which the reference structure is generated in the form of a structured quasi light sheet by the movement of a scanner mirror and temporal beam modulation; and -
FIG. 5 a fourth embodiment example illustrating the light sheet adjustment based on a light sheet on which a reference structure is impressed. - It is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the present invention, while eliminating, for purposes of clarity, many other elements which are conventional in this art. Those of ordinary skill in the art will recognize that other elements are desirable for implementing the present invention. However, because such elements are well known in the art, and because they do not facilitate a better understanding of the present invention, a discussion of such elements is not provided herein.
- The present invention will now be described in detail on the basis of exemplary embodiments.
- As can be seen from
FIG. 1 , asample 1 is embedded in a gel which is shaped to form acircular cylinder 2. Thecircular cylinder 2 is located in asample chamber 4 and is supported inside thesample chamber 4 so as to be rotatable around an axis ofrotation 3 and displaceable translationally in coordinate directions X, Y, Z together with thesample 1. - The
sample 1 is a three-dimensional object, for example, a biological substance. - The
circular cylinder 2 with theenclosed sample 1 is filled with an immersion medium, for example, water or PBS (phosphate buffered saline). Thesample chamber 4 is open at its upper side so that the gel with theenclosed sample 1 and the immersion medium can be introduced from the top. - If water is provided as immersion medium, the gel can be made of a mixture of approximately 98% water and 2% agarose. When a different immersion medium is used, other materials can be used for embedding the
sample 1; for example, glass cylinders filled with a liquid can also be used. The walls of thesample chamber 4 can be made of glass or some other transparent material with a suitable refractive index. - A laser light source (not shown) radiating a light bundle at a wavelength of, e.g., 488 nm is provided for illuminating the sample. It is advantageous when the radiation intensity in the cross section of this light bundle is initially homogenized with suitable optical means before proceeding to be shaped as a light sheet.
- The shaping of the light sheet is carried out either directly, e.g., through cylindrical optics, or as a quasi light sheet through the scanning of a line-shaped beam over the object field.
- The illumination light shaped as the light sheet passes through the sample chamber wall 6 in direction of arrow B into the immersion medium and must be positioned in such a way that it there illuminates the focus plane of a microscope objective (not shown).
- In the prior art, this orientation of the light sheet relative to the focus plane is difficult when the sample is already located in the sample space because a so-called in-situ adjustment is required in that case. In so doing, it cannot be assumed that the plane of the sample which overlaps the focus plane of the objective following a rough preliminary positioning contains high-contrast structures on whose basis the overlapping between the light sheet and focus plane can be determined beyond doubt when the light sheet passes through in z-direction.
- In contrast to the prior art in which an adjustment of this type is carried out by means of mirrors, the invention allows the adjustment to be carried out based on a known reference structure which is introduced into the sample space next to the sample indirectly or directly.
- When using a reference structure generated directly by means of reference objects, it must be ensured that at least a portion of this reference structure is already situated in the focus plane of the detection objective before the adjustment of the light sheet is carried out. This is accomplished either by using a large number of reference objects which are distributed substantially uniformly over the sample space or by a separate adjustment by means of another contrasting method, e.g., in conventional transmitted light microscopy.
- When using an indirect reference structure in which the light of the light sheet is structured, a method step of this kind is not required.
- If the sample is then positioned in a desired manner relative to the focus plane and accordingly also relative to the adjusted light sheet, a plane of the
sample 1, namely, the plane located in the focus plane, is illuminated. - In so doing, a portion of the illumination light that is reflected or scattered by the
sample 1 passes through thesample chamber wall 7 out of thesample chamber 4 as detection light in the detection direction designated by arrow D and is collected by the microscope objective. - In the prior art, image information is initially acquired from the detection direction indicated by arrow D from the plane of the
sample 1 that is actually located in the focus plane by the microscope, whose functional part is the microscope objective. Thesample 1 is then repeatedly displaced by a given amount or continuously in direction of coordinate Z. In so doing, by means of a spatially resolving optoelectronic converter, preferably a CCD camera, image information is acquired from the planes of thesample 1 which are located in the illuminated focus plane and is digitized and subsequently electronically stored so as to be correlated with its spatial coordinates X, Y, X. A three-dimensional image of the sample is already obtained at this stage; however, its quality is generally still severely limited by the shadow effects and aberrations caused by the sample. Further, the resolution is generally inhomogeneous because it is determined laterally by the numerical aperture of the detection objective and axially by the light sheet thickness. - According to the invention, the detection direction is changed repeatedly by rotating the
sample 1 around the axis ofrotation 3, and image information is acquired again from each changed detection direction from different depth planes of thesample 1 and is stored in a spatially resolved manner. - A three-dimensional image of the
sample 1 which is substantially free from image artifacts such as shadows caused by opaque constituents of the sample and aberrations is electronically reconstructed by registration and fusion from the stored image information that was obtained from all of the given detection directions. At the same time, an extensively uniform resolution is achieved in this method. - In this case, there would still be the disadvantage of the relatively high requirement for computing capacity resulting from the fact that the image artifacts affect different areas of the sample depending on the detection direction, which hampers registration based on comparison of structures. Further, the amount of image information of the sample is extremely extensive and, therefore, direct processing is possible only to a limited extent.
- For example, an image stack of 1000×1000×1000 image pixels is recorded in coordinates X, Y, Z at 16 bits by a commercial CCD camera for every given detection direction in order to obtain image information. In so doing, the amount of data to be acquired and processed already amounts to 16 GB.
- When detection is carried out from only four different directions, for which the
sample 1 is rotated by 90° in each instance around the axis ofrotation 3, the amount of data to be acquired and processed is 64 GB. If the difficulties mentioned above with respect to evaluation are still to be overcome, the total expenditure is so great that conventional commercially available computer systems are overtaxed. - For this purpose, the invention provides that, in addition to the image information from the
sample 1, image information of one or more reference objects positioned next to thesample 1 or in thesample 1 is also acquired from each of the given detection directions and is stored in a spatially resolved manner together with the image information of thesample 1, and the spatial coordinates X, Y, Z of the image information obtained from the reference object or reference objects is taken as a basis for the reconstruction of the three-dimensional image of thesample 1. - Specifically to this end, the registration is carried out exclusively on the basis of the substantially smaller amount of image information obtained from the reference structure, and a transformation rule in the form of a matrix is determined therefrom and is then applied to the image information of the actual sample. The image stacks of the sample structure which are registered after applying the transformation are then fused to form a three-dimensional picture.
- In a first application example of the method according to the invention which will be explained in the following, the sample in the form of a zebrafish embryo which is dyed with enhanced green fluorescent protein (EGFP) and accordingly fluoresces green when excited is embedded in an agarose gel. A three-dimensional image of this zebrafish embryo is to be recorded by means of the above-described method of single plane illumination microscopy (SPIM).
- According to the invention, red-fluorescing particles, for example, small polystyrene beads tagged with BODIPY 630 dye are embedded in the agarose gel along with the zebrafish embryo. The concentration of these particles is high enough that they are distributed over the sample space within the sample chamber virtually homogeneously.
- Using these particles, the light sheet can be adjusted in a simple manner as is illustrated in
FIG. 2 . To this end, the light sheet is moved over the sample in z-direction and irradiates the fluorescing particles as well as the sample. - As long as the light sheet radiates into the sample chamber outside the focus plane, there will be a weak, blurred image of the particles located therein as is shown on the left-hand side of
FIG. 2 . The gel surrounding the sample is not shown in the drawing for the sake of clarity. - This changes as soon as the light sheet radiates into the focus plane as is shown on the right-hand side of
FIG. 2 . The particles are sharply imaged and the light sheet is adjusted at this location. - In an advantageous manner, the adjustment steps can take place automatically.
- For this purpose, in the simplest embodiment form, the light sheet is moved through the sample space in z-direction and an image stack is recorded. This stack is then evaluated, for example, with respect to the contrast of the individual images, so that the optimal light sheet position can be determined quickly.
- After the adjustment of the light sheet, the actual image recording is carried out. For this purpose, image information from a plurality of planes lying at different depths in the zebrafish embryo is first acquired from a given detection direction along with the image information of the particles, and this image information is stored in a spatially resolved manner. The detection direction is then changed repeatedly and image information is in turn acquired from each changed detection direction from planes of different depths in the zebrafish embryo and particles and is stored in a spatially resolved manner.
- The computing capacity required for processing the image information of the sample acquired in this way to arrive at a three-dimensional image of the sample can be reduced according to the invention as will be explained in the following.
- For example, if one hundred the particles are acquired microscopically for each detection direction, the data in the ideal case of a monochrome picture of these particles is reduced to 100 bits per detection direction. For a data reduction of this kind, the total image, which generally initially contains the image information of the sample and particles, must first be segmented i.e., the particles structures are separated from the sample structures by filtering, the coordinates of the particle positions are acquired and are then stored as a monochrome model.
- In the case of a sample fluorescing in the green spectral region, the filtering can be carried out by spectral separation using hardware. The resulting relatively small amount of data can easily be processed in real time by commercial computers, especially as the particle emission should be substantially unaffected by image artifacts caused by the sample.
- The particles form 100 fixing points on the basis of which displacements, rotations and deformation of the sample substance are calculated from one detection direction to the other and then corrected computationally.
- The size of the particles is advantageously selected in such a way that they do not have a scattering effect and also do not act like spherical lenses in the sample, in this case, the zebrafish embryo, and therefore cannot result in excessive artifacts. The size of the particles is preferably 100 nm to 1 μm.
- To obtain the image stack from, e.g., four detection directions, the gel with the enclosed zebrafish embryo and the enclosed particles is first placed in the sample chamber in a first viewing direction of the microscope objective on the zebrafish embryo and the particles, and image information of a first image stack is recorded. The gel with the zebrafish embryo and the particles is then rotated by 90° around the axis of rotation, and image information of a second image stack is recorded. Similarly, a 90-degree rotation is then carried out twice and a third and fourth image stack is recorded.
- To summarize, substantially the following method steps are carried out:
- the light sheet is adjusted, advantageously automatically, based on the particles in that the light sheet is moved over the sample space in z-direction and the images obtained in so doing are evaluated for maximum sharpness,
the sample is focused in a region of interest (RIO) by displacement in z-direction,
a detection filter is selected so that the particles stand out against the background with a sufficiently good signal,
a first image stack is acquired from the first detection direction (0°), where the image stack is recorded once for green fluorescence and immediately thereafter a second time for red fluorescence so that image information of the sample (green) and image information of the particles (red) is acquired, or both color channels, red and green, are recorded simultaneously by means of a detection system suitable for spectral splitting,
the sample is rotated by 90° for orientation in three additional detection directions which consequently enclose angles of 90°, 180° and 270° with the first detection direction; another image stack is recorded for the green fluorescence and for the red fluorescence from each of these detection directions, wherein the image information overlaps in the image stacks. - As a result, there are four image stacks each for the green fluorescence and the red fluorescence from viewing directions on the sample which are offset by an angle of 90° relative to one another.
- The three-dimensional image of the sample is reconstructed from the resulting image information, for example, in that a segmentation is carried out for the red fluorescence for purposes of data reduction in such a way that a three-dimensional monochrome model is formed. This is followed by the registration of the image stacks obtained with the red fluorescence, i.e., the necessary transformation operators are determined for each image stack, e.g., in the form of a matrix, for translation, rotation and deformation which are suitable for making the image information of each image stack overlap with the image information of the other image stacks.
- The transformation operators determined in this way are then applied to the green channel, i.e., to the image stack obtained with the green fluorescence, which results in registered image stacks with image information of the sample. The latter are combined by a fusion step resulting in an individual high-quality three-dimensional image of the sample.
- In a second application example of the method according to the invention, a dyed polyester or polystyrene thread instead of the fluorescing particles is embedded in agarose gel next to the sample, for example, a zebrafish embryo.
- For adjusting the light sheet, it must be ensured that a portion of the thread is located in the focus plane of the detection objective. To this end, the thread can first be moved in z-direction together with the sample by evaluation of a simple transmitted light contrast until it is located at least partially in the focus plane, which can be judged based on the sharpness of its imaging. The light sheet is then oriented in such a way that the portion of the thread located in the focus plane is also sharply imaged in SPIM mode.
- Subsequently, image stacks are recorded from different detection directions by the method steps described above such that at least one identical portion of the fiber always appears in the image stacks. Since the amount of image information of the thread as well as the amount of image information of the particles in the first application example is substantially less than the amount of image information of the sample, a substantial reduction in the data volume to be processed is also achieved in this case.
- In this case, also, as was explained with reference to the first application example, the three-dimensional image of the sample is reconstructed in that the same portion of the thread is first separated from the sample structure for every detection direction, which can be accomplished, for example, simply by cropping the image. In the latter case, it is advantageous when the thread is positioned just below the sample, for example, in y-direction. After this, the registration for the thread images takes place and a transformation rule in the form of a matrix is determined and then applied to the sample information that is actually of interest. The fusion is then carried out, resulting in an individual high-quality three-dimensional image of the sample.
- In a third application example, a needle is embedded as reference object in an agarose gel. The tip of the needle is advantageously arranged in such a way that it is located exactly in the axis of rotation 3 (see
FIG. 1 ). - It is likewise advantageous when the sample is also positioned in the immediate vicinity of the needle tip, which can be achieved, e.g., by pinning it on the latter.
- For light sheet adjustment, it must be ensured that the needle tip is located in the focus plane of the detection objective. For this purpose, it can be moved together with the sample in z-direction by evaluation of a simple transmitted light contrast until it is sharply imaged.
- The region of interest (RIO) to be recorded is selected in such a way that the tip of the needle appears in the image stacks.
- To start with, the image recording is greatly facilitated when the needle tip is positioned in the axis of
rotation 3 and the sample is placed in its immediate vicinity because the sample generally does not move out of the image field during a rotational movement. Further, it is clear that the absolute position of the needle tip has not changed during the rotational movement and, therefore, presents a fixing point. That is, when the positions of the sample structures for each image stack are determined relative to the needle tip, only a simple rotational movement is required for the registration, because the needle tip does not undergo any translational movement. Therefore, the registration algorithms proceed very efficiently. - Another advantage in using a needle tip consists in that it can be used to calibrate the drives provided for an automatic adjustment of the light sheet.
- In a fourth application example, an indirectly generated reference structure in the form of a structured light sheet is used. The reference structure can be generated, for example, as is shown in
FIG. 3 . - For this purpose, a
grating 9 is positioned in aplane 8 conjugate to the object space.Different diffraction orders 11 then result in thepupil plane 10 of the illumination objective, and an image of the grating is in turn formed in the sample space. - This grating causes an intensity distribution which is modulated over the width of a
light sheet 12. InFIG. 3 a, the intensity distribution of thelight sheet 12 is shown in a side view on the object plane of a detection objective (not shown), thelight sheet 12 being oriented perpendicular to the drawing plane, and the reference structure 13 (seeFIG. 3 b) lies in the range of the depth of field of the illumination objective. The dashed line in thelight sheet 12 inFIG. 3 a represents the range of the depth of field. - The intensity distribution of the
light sheet 12 causing thereference structure 13 is shown inFIG. 3 b in a top view of the object plane. - A comparable intensity distribution is also yielded, for example, by an arrangement or process as shown in
FIG. 4 . - The
light sheet 12 is generated here as a quasi light sheet through the movement of ascanner mirror 14 in a pupil plane of the illumination beam path. A substantially line-shaped beam is accordingly moved over the image field of the detection objective. - In this process, it must always be ensured that the integration period of the image recording is greater than or equal to the scanner period and is synchronized as far as possible with the scanner movement in order to prevent temporal intensity modulations in the image recording.
- In order to generate the spatial structuring, a deliberate temporal modulation can be introduced in addition, e.g., by using an acousto-
optical modulator 15, so that areference structure 13 results in the sample space due to a reseau pattern in the intensity distribution. - When the fluorescing sample is illuminated by the structured light sheet, the fluorescence proceeding from the sample is also spatially modulated. The grating is then clearly visible in the image recording when the light sheet overlaps with the focus plane of the detection objective as is shown on the right-hand side of
FIG. 5 . However, when the light sheet is out of alignment and does not overlap with the focus plane, the grating is virtually undetectable as is illustrated on the left-hand side ofFIG. 5 . - Therefore, the grating image can be used for very accurate adjustment of the light sheet. The sample is not sufficient by itself because it has a defined structure only in exceptional cases.
- As was mentioned above, two variants are possible according to the invention for adjusting the light sheet; namely, on the basis of a directly generated reference structure by means of reference objects for one, and, for another, based on an indirectly generated reference structure in the form of a given intensity distribution.
- The advantage of using an indirectly generated reference structure over a directly generated reference structure is that the sample can be acquired after adjustment without interference from any reference objects in the normal SPIM mode either by simply removing the grating from the beam path or by halting the temporal intensity modulation during the scanner movement.
- However, a disadvantage compared to the use of reference objects consists in that indirect structuring cannot be used to reduce the computing capacity in the three-dimensional imaging of the sample.
-
- 1 sample
- 2 circular cylinder
- 3 axis of rotation
- 4 sample chamber
- 5 upper side
- 6, 7 sample chamber wall
- 8 plane conjugate to the object space
- 9 grating
- 10 pupil plane
- 11 diffraction orders
- 12 light sheet
- 13 reference structure
- 14 scanner mirror
- 15 acousto-optical modulator
- B, D arrow direction
Claims (18)
1. A Method for microscopic three-dimensional imaging of a sample in which image information from different depth planes of the sample is acquired and is stored in a spatially resolved manner so as to be correlated to its spatial coordinates X, Y, Z, and in which the three-dimensional image of the sample is reconstructed electronically from the stored image information, comprising the steps of:
positioning at least one reference object is next to or in the sample;
changing the detection direction from which the image information is acquired relative to the sample and relative to the reference object;
acquiring image information from the reference object from each detection direction given by a change;
storing the acquired image information in a spatially resolved manner in spatial coordinates X, Y, Z;
acquiring image information from different depth planes of the sample from each detection direction given by a change;
storing the image information acquired from different planes in a spatially resolved manner in spatial coordinates X, Y, Z;
obtaining transformation operators for translation, rotation and deformation based on the spatial coordinates X, Y, Z of the stored image information from the reference object so that image information from the same source from the different detection directions can be overlapped; and
reconstructing a three-dimensional image of the sample from the spatial coordinates X, Y, Z of the image information obtained from the sample based on these transformation operators.
2. The Method according to claim 1 ,
wherein the reference object is a pin, a needle, a fiber, a thread, or small particles.
3. The Method according to claim 1 , wherein the structure of the reference object is used as the reference structure, and the spatial coordinates X, Y, Z which are acquired on the basis of the reference structure and stored are used to obtain transformation operators.
4. The Method according to claim 1 ,
wherein the transformation operators are related to their respective spatial coordinates X, Y, Z in the form of a rule for translation, rotation and deformation of the image information which is acquired from the individual detection directions and stored, and
wherein the rule is used in the reconstruction of the three-dimensional image of the sample.
5. A Method for microscopic three-dimensional imaging of a sample in which image information from different depth planes of the sample is acquired and is stored in a spatially resolved manner so as to be correlated to its spatial coordinates X, Y, Z, and in which the three-dimensional image of the sample is reconstructed electronically from the stored image information, comprising the steps of: obtaining a three-dimensional image of a sample comprising a fluorescing substance or of a sample to which a fluorescing substance is added by selective plane illumination microscopy;
illuminating the sample by an excitation beam which is shaped as a light sheet;
obtaining an image information of the sample based on the emission radiation coming from the sample,
positioning at least one reference object comprising a fluorescing substance or a reference object having a fluorescing substance next to or in the sample;
acquiring the detection direction from which the image information; acquired
changing the detection direction relative to the sample and relative to the reference object,
acquiring image information from the reference object from each detection direction given by a change;
storing the image information from the reference object from each detection direction given by a change in a spatially resolved manner in spatial coordinates X, Y, Z;
acquiring image information from different depth planes of the sample from each detection direction given by a change;
storing the image information from different depth planes in a spatially resolved manner in spatial coordinates X, Y, Z;
obtaining transformation operators for translation, rotation and deformation based on the spatial coordinates X, Y, Z of the stored image information from the reference object so that image information from the same source from the different detection directions can be overlapped;
reconstructing the three-dimensional image of the sample from the spatial coordinates X, Y, Z of the image information obtained from the sample based on these transformation operators.
6. The Method according to claim 5 , further comprising the steps of:
applying a reference structure in the form of an irregular intensity distribution to the illumination light;
recording and evaluating images of the reference structure of the illumination light, of the reference structure of the reference object, or of a sample structure which is suitable as a reference structure prior in time to the acquisition of the image information of the reference object and of the sample from at least one of the given detection directions, with respect to the sharpness of their imaging, and
displacing the light sheet in the detection direction into the position in which the sharpest imaging of the reference structure is achieved in an observation plane, and/or the light sheet is oriented in such a way that the reference structure is sharply imaged in a homogeneous manner over the entire image field.
7. The Method according to claim 5 ,
wherein the image information of the reference object and the image information of the sample are acquired based on the emission radiation whose wavelengths differ from one another.
8. The Method according to claim 5 ,
wherein the transformation operators are related to their respective spatial coordinates X, Y, Z in the form of a rule for translation, rotation and deformation of the image information which is acquired from the individual detection directions and stored, and
wherein the rule is used in the reconstruction of the three-dimensional image of the sample.
9. The Method according to claim 1 ,
wherein the image information of the sample and the image information of the at least one reference object are acquired simultaneously or successively in time.
10. Method according to claim 1 ,
wherein the detection direction is changed by rotating the sample by 90° in each instance,
wherein the axis of rotation is oriented perpendicular to the detection directions.
11. The Method for orientation of an illumination light shaped as a light sheet relative to a focus plane of a microscope objective in three-dimensional imaging of a sample using a method of selective plane illumination microscopy (SPIM), comprising the steps of:
applying a reference structure in the form of an inhomogeneous intensity distribution to the illumination light;
positioning at least one reference object which is made of a fluorescing substance or has a fluorescing substance and which has a structure that is usable as a reference structure next to or in the sampler;
determining a characteristic structure of the sample as reference structure;
recording images of the reference structure of the illumination light, of the reference structure of the reference object or of the reference structure of the sample from at least one given detection direction;
evaluating the images of the reference structure of the illumination light, of the reference structure of the reference object or of the reference structure of the sample with respect to the sharpness of their image;
displacing the light sheet in the at least one given detection direction into a position in which the sharpest imaging of the reference structure is achieved in an observation plane; and,
orienting the light sheet in a way that the reference structure is imaged in a uniformly sharp manner over the entire image field.
12. The Method according to claim 6 , further comprising the steps of:
moving the orientation of the light sheet in a way that that the light sheet is moved through the sample space in a z-direction;
recording an image stack during the moving step; and
evaluating the image stack with respect to the contrast of each of a plurality of individual images of the image stack,
wherein the optimal light sheet position relative to the focus plane of the microscope objective is determined based on the contrast, and the light sheet is moved into this position by means of an adjusting device.
13. The Method according to claim 5 ,
wherein the orientation of the light sheet is carried out automatically based on the results of the evaluation of the contrast of individual images of the image stack.
14. The Method according to claim 5 ,
wherein an object, comprised at least partially of fluorescing material, is used as reference object.
15. The Method according to claim 5 ,
wherein a plurality of reference objects in the form of fluorescing particles, are used.
16. The Method according to claim 15 ,
wherein at least three fluorescing particles in the form of small polystyrene beads which are marked with a dye and have a diameter in the range of 100 nm to 1 μm are used and are embedded together with the sample in a.
17. The Method according to claim 5 ,
wherein the sample is dyed with enhanced green fluorescent protein causing the sample to fluoresce green when excited.
18. The Method according to claim 5 ,
wherein the sample and the at least one reference object are illuminated by an excitation beam with a wavelength of 488 nm.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102007045897A DE102007045897A1 (en) | 2007-09-26 | 2007-09-26 | Method for the microscopic three-dimensional imaging of a sample |
DE102007045897.7 | 2007-09-26 | ||
PCT/EP2008/007686 WO2009043472A2 (en) | 2007-09-26 | 2008-09-16 | Method for the microscopic three-dimensional reproduction of a sample |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2008/007686 A-371-Of-International WO2009043472A2 (en) | 2007-09-26 | 2008-09-16 | Method for the microscopic three-dimensional reproduction of a sample |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/705,237 Division US9697605B2 (en) | 2007-09-26 | 2012-12-05 | Method for the microscopic three-dimensional reproduction of a sample |
Publications (1)
Publication Number | Publication Date |
---|---|
US20100201784A1 true US20100201784A1 (en) | 2010-08-12 |
Family
ID=40091778
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/680,056 Abandoned US20100201784A1 (en) | 2007-09-26 | 2008-09-16 | Method for the microscopic three-dimensional reproduction of a sample |
US13/705,237 Active 2029-12-09 US9697605B2 (en) | 2007-09-26 | 2012-12-05 | Method for the microscopic three-dimensional reproduction of a sample |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/705,237 Active 2029-12-09 US9697605B2 (en) | 2007-09-26 | 2012-12-05 | Method for the microscopic three-dimensional reproduction of a sample |
Country Status (5)
Country | Link |
---|---|
US (2) | US20100201784A1 (en) |
EP (1) | EP2193398B1 (en) |
JP (1) | JP5485156B2 (en) |
DE (1) | DE102007045897A1 (en) |
WO (1) | WO2009043472A2 (en) |
Cited By (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090317044A1 (en) * | 2002-09-05 | 2009-12-24 | Nanosys, Inc. | Nanocomposites |
US20100177190A1 (en) * | 2008-12-16 | 2010-07-15 | Ann-Shyn Chiang | Microscopy system with revolvable stage |
US20100239138A1 (en) * | 2007-10-09 | 2010-09-23 | Helmut Lippert | Method for positioning biological samples in a microscopic arrangement |
US20110212844A1 (en) * | 2010-02-26 | 2011-09-01 | Massachusetts Institute Of Technology | High-throughput platform for in-vivo sub-cellular screens on vertebrate larvae |
US20120098949A1 (en) * | 2010-10-22 | 2012-04-26 | Leica Microsystems Cms Gmbh | Spim microscope with a sted light sheet |
US20120206798A1 (en) * | 2011-02-14 | 2012-08-16 | European Molecular Biology Laboratory (Embl) | Light-pad microscope for high-resolution 3d fluorescence imaging and 2d fluctuation spectroscopy |
WO2012146720A1 (en) * | 2011-04-29 | 2012-11-01 | Peira Bvba | Stereo-vision system |
US20130094755A1 (en) * | 2007-09-26 | 2013-04-18 | Carl Zeiss Microlmaging Gmbh | Method for the microscopic three-dimensional reproduction of a sample |
WO2014005866A1 (en) * | 2012-07-03 | 2014-01-09 | Carl Zeiss Microscopy Gmbh | Method for preparing for and carrying out the recording of image stacks of a sample from various orientation angles |
US8792162B2 (en) | 2009-09-24 | 2014-07-29 | Carl Zeiss Microscopy Gmbh | Microscope with illumination switching for capturing sample images during detector integration time |
US20140347447A1 (en) * | 2008-12-05 | 2014-11-27 | Unisensor A/S | Optical sectioning of a sample and detection of particles in a sample |
US20150168706A1 (en) * | 2013-12-18 | 2015-06-18 | Carl Zeiss Microscopy Gmbh | Microscope and Method for SPIM Microscopy |
JP2015537236A (en) * | 2012-10-12 | 2015-12-24 | カール ツァイス マイクロスコピー ゲーエムベーハーCarl Zeiss Microscopy Gmbh | Microscope and method for SPIM microscopy |
US9297760B2 (en) | 2011-03-11 | 2016-03-29 | Hellma Gmbh & Co., KG | Apparatus for analysing a small amount of liquid |
US20160161410A1 (en) * | 2012-10-09 | 2016-06-09 | Howard Hughes Medical Institute | Multiview Light-Sheet Microscopy |
US20160180509A1 (en) * | 2014-12-17 | 2016-06-23 | Casio Computer Co., Ltd. | Commodity identification device and commodity recognition navigation method |
WO2016138003A1 (en) * | 2015-02-23 | 2016-09-01 | The Research Foundation For The State University Of New York | Method and apparatus for tiling light sheet selective plane illumination microscopy with real-time optimized light sheet |
US20160320301A1 (en) * | 2013-12-17 | 2016-11-03 | Leica Microsystems Cms Gmbh | Method and apparatus for investigating a sample by means of optical projection tomography |
US9715095B2 (en) | 2011-10-11 | 2017-07-25 | Carl Zeiss Microscopy Gmbh | Microscope and method for SPIM microscopy |
US9880378B2 (en) | 2013-04-05 | 2018-01-30 | Riken | Microscope, focusing unit, fluid holding unit, and optical unit |
US10379335B2 (en) * | 2016-05-09 | 2019-08-13 | Olympus Corporation | Illumination setting method, light sheet microscope apparatus, and recording medium |
CN110501315A (en) * | 2018-05-18 | 2019-11-26 | 卡尔蔡司显微镜有限责任公司 | The method of general view image is provided |
US10539772B2 (en) | 2013-10-09 | 2020-01-21 | Howard Hughes Medical Institute | Multiview light-sheet microscopy |
US20200158633A1 (en) * | 2017-07-26 | 2020-05-21 | Hamamatsu Photonics K.K. | Sample observation device and sample observation method |
US11320640B2 (en) | 2016-06-24 | 2022-05-03 | Howard Hughes Medical Institute | Automated adjustment of light sheet geometry in a microscope |
US11422503B2 (en) * | 2014-08-01 | 2022-08-23 | The Regents Of The University Of California | Device and method for iterative phase recovery based on pixel super-resolved on-chip holography |
US11561384B2 (en) | 2017-12-22 | 2023-01-24 | Leica Microsystems Cms Gmbh | Method and device for adjusting the focus or determining the refractive index of a sample medium, non-volatile computer-readable storage medium and microscope |
US11586027B2 (en) | 2018-02-01 | 2023-02-21 | Leica Microsystems Cms Gmbh | Method for imaging a sample by means of a light-sheet microscope |
US11808931B2 (en) | 2019-02-06 | 2023-11-07 | Evident Corporation | Image pickup apparatus with rotation unit |
Families Citing this family (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102009022912B4 (en) * | 2009-05-27 | 2016-11-17 | Carl Zeiss Ag | Correlative optical and particle beam microscopy |
DE102010013223B4 (en) * | 2010-03-29 | 2016-05-12 | Lavision Biotec Gmbh | Method and arrangement for microscopy |
PT3034608T (en) | 2010-06-22 | 2019-05-28 | Regeneron Pharma | Mice expressing an immunoglobulin hybrid light chain |
KR101535918B1 (en) * | 2011-05-11 | 2015-07-13 | 한양대학교 산학협력단 | A Method for the Toxicity Assessments of Nano-Materials using Selective Plane Illumination Microcopy |
DE102011000835C5 (en) | 2011-02-21 | 2019-08-22 | Leica Microsystems Cms Gmbh | Scanning microscope and method for light microscopic imaging of an object |
DE102011051042B4 (en) | 2011-06-14 | 2016-04-28 | Leica Microsystems Cms Gmbh | Scanning microscope and method for light microscopic imaging of an object |
JP5892410B2 (en) * | 2011-10-03 | 2016-03-23 | 株式会社ニコン | Scanning microscope |
US10545075B2 (en) | 2012-08-09 | 2020-01-28 | The Board Of Trustees Of The Leland Stanford Junior University | Methods and compositions for preparing biological specimens for microscopic analysis |
DE102013009042A1 (en) * | 2013-05-28 | 2014-12-04 | Carl Zeiss Microscopy Gmbh | luminescence |
US10746981B2 (en) | 2014-05-30 | 2020-08-18 | The Board Of Trustees Of The Leland Stanford Junior University | Methods and devices for imaging large intact tissue samples |
DE102014116174A1 (en) * | 2014-11-06 | 2016-05-12 | Carl Zeiss Microscopy Gmbh | Method for generating an image of a sample |
JP6510239B2 (en) * | 2015-01-08 | 2019-05-08 | オリンパス株式会社 | Fluorescence microscope |
WO2016125281A1 (en) * | 2015-02-05 | 2016-08-11 | 株式会社ニコン | Structured illumination microscope, observation method, and control program |
US10989661B2 (en) * | 2015-05-01 | 2021-04-27 | The Board Of Regents Of The University Of Texas System | Uniform and scalable light-sheets generated by extended focusing |
US11254974B2 (en) | 2016-02-10 | 2022-02-22 | The Board Of Trustees Of The Leland Stanford Junior University | RNA fixation and detection in clarity-based hydrogel tissue |
US10509215B2 (en) * | 2016-03-14 | 2019-12-17 | Olympus Corporation | Light-field microscope |
WO2017180680A1 (en) | 2016-04-12 | 2017-10-19 | The Board Of Regents Of The University Of Texas System | LIGHT-SHEET MICROSCOPE WITH PARALLELIZED 3D lMAGE ACQUISITION |
JP2018004777A (en) | 2016-06-28 | 2018-01-11 | オリンパス株式会社 | Light sheet microscope and light sheet microscope control method |
US10310246B2 (en) * | 2016-09-28 | 2019-06-04 | SCREEN Holdings Co., Ltd. | Converter, illuminator, and light sheet fluorescence microscope |
WO2018204814A1 (en) * | 2017-05-04 | 2018-11-08 | Intelligent Imaging Innovations, Inc. | A method to keep the excitation light sheet in focus in selective plane illumination microscopy |
CN111051954A (en) | 2017-07-20 | 2020-04-21 | 维微恩提斯显微镜有限责任公司 | Microscope, method of operating a microscope and method of imaging a sample |
GB2566115B (en) * | 2017-09-22 | 2020-04-01 | Univ Court Univ St Andrews | Imaging of a sample through a scattering medium |
DE102017123510A1 (en) * | 2017-10-10 | 2019-04-11 | Carl Zeiss Microscopy Gmbh | Digital microscope and method for picking up a stack of microscopic images of a sample |
DE102017009804A1 (en) * | 2017-10-20 | 2019-04-25 | Vermicon Ag | Method for evaluating microscopic samples and apparatus for carrying out this method |
CN108226120B (en) * | 2018-01-23 | 2020-10-16 | 哈尔滨工业大学 | Device and method for measuring size and energy distribution of sheet laser beam |
CN111381357B (en) * | 2018-12-29 | 2021-07-20 | 中国科学院深圳先进技术研究院 | Image three-dimensional information extraction method, object imaging method, device and system |
WO2021041807A1 (en) * | 2019-08-30 | 2021-03-04 | The Regents Of The University Of California | Systems and methods for image cytometry |
CN113409270B (en) * | 2021-06-18 | 2022-04-29 | 深圳市宏申工业智能有限公司 | Coarse aggregate particle size distribution analysis method, device and system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6114695A (en) * | 1990-10-12 | 2000-09-05 | Hitachi, Ltd. | Scanning electron microscope and method for dimension measuring by using the same |
US20050047640A1 (en) * | 2000-10-24 | 2005-03-03 | Oncosis Llc | Method and device for selectively targeting cells within a three-dimensional specimen |
US20050259864A1 (en) * | 2004-05-05 | 2005-11-24 | Mary Dickinson | Four-dimensional imaging of periodically moving objects via post-acquisition synchronization of nongated slice-sequences |
Family Cites Families (43)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE3739223A1 (en) | 1987-11-19 | 1989-06-01 | Reichert Optische Werke Ag | Method for autofocusing of microscopes, and microscopes with an autofocusing system |
US5109276A (en) * | 1988-05-27 | 1992-04-28 | The University Of Connecticut | Multi-dimensional multi-spectral imaging system |
US6882473B2 (en) * | 1995-03-02 | 2005-04-19 | Carl Zeiss Jena Gmbh | Method for generating a stereoscopic image of an object and an arrangement for stereoscopic viewing |
KR100504261B1 (en) | 1997-04-04 | 2005-07-27 | 아이시스이노베이션리미티드 | Microscopy imaging apparatus and method |
US6650357B1 (en) * | 1997-04-09 | 2003-11-18 | Richardson Technologies, Inc. | Color translating UV microscope |
US6229562B1 (en) * | 1997-07-08 | 2001-05-08 | Stanley H. Kremen | System and apparatus for the recording and projection of images in substantially 3-dimensional format |
JP3721425B2 (en) * | 1998-02-20 | 2005-11-30 | ライカ ミクロジュステムス ツェーエムエス ゲーエムベーハー | Laser scanning microscope calibration equipment |
US6081577A (en) * | 1998-07-24 | 2000-06-27 | Wake Forest University | Method and system for creating task-dependent three-dimensional images |
US6867851B2 (en) * | 1999-11-04 | 2005-03-15 | Regents Of The University Of Minnesota | Scanning of biological samples |
US6784982B1 (en) * | 1999-11-04 | 2004-08-31 | Regents Of The University Of Minnesota | Direct mapping of DNA chips to detector arrays |
AU785290B2 (en) * | 1999-11-30 | 2006-12-21 | Intrexon Corporation | Method and apparatus for selectively targeting specific cells within a cell population |
US7546210B2 (en) * | 2000-06-08 | 2009-06-09 | The Regents Of The University Of California | Visual-servoing optical microscopy |
DE10108240B4 (en) * | 2001-02-21 | 2018-06-28 | Leica Microsystems Cms Gmbh | Method for imaging and measuring microscopic three-dimensional structures |
US7105795B2 (en) * | 2001-07-06 | 2006-09-12 | Palantyr Research, Llc | Imaging system, methodology, and applications employing reciprocal space optical design |
US20050117118A1 (en) * | 2001-10-05 | 2005-06-02 | David Miller | Digital ophthalmic workstation |
WO2003030763A1 (en) * | 2001-10-05 | 2003-04-17 | Boston Innovative Optics, Inc. | A system and method of providing visual documentation during surgery |
WO2003036566A2 (en) * | 2001-10-22 | 2003-05-01 | Leica Microsystems Wetzlar Gmbh | Method and device for producing light-microscopy, three-dimensional images |
JP2005508016A (en) * | 2001-10-24 | 2005-03-24 | ニューローケイ・エルエルシー | Projecting 3D images |
US7634129B2 (en) * | 2001-12-28 | 2009-12-15 | Rudolph Technologies, Inc. | Dual-axis scanning system and method |
EP1467655A1 (en) * | 2002-01-14 | 2004-10-20 | Cadent Ltd. | Method and sytem for imaging a patient's teeth arrangement |
US20040002154A1 (en) * | 2002-04-19 | 2004-01-01 | Palsson Bernhard O. | Methods for preparing libraries of unique tags and related screening methods |
JP4377811B2 (en) * | 2002-06-21 | 2009-12-02 | オリンパス株式会社 | Biomolecule analyzer and biomolecule analysis method |
ATE285590T1 (en) * | 2002-10-25 | 2005-01-15 | Evotec Technologies Gmbh | METHOD AND APPARATUS FOR CAPTURING THREE-DIMENSIONAL IMAGERY OF FLOATED MICROOBJECTS USING HIGH-RESOLUTION MICROSCOPY |
US7200252B2 (en) * | 2002-10-28 | 2007-04-03 | Ventana Medical Systems, Inc. | Color space transformations for use in identifying objects of interest in biological specimens |
DE10257423A1 (en) * | 2002-12-09 | 2004-06-24 | Europäisches Laboratorium für Molekularbiologie (EMBL) | Microscope used in molecular biology comprises a focussing arrangement producing an extended planar object illumination region, a detection device, and a movement arrangement |
JP2005017805A (en) | 2003-06-27 | 2005-01-20 | Mitsutoyo Corp | Focus detecting method of image measuring device, focus detection mechanism, and image measuring device equipped with the focus detection mechanism |
US7372985B2 (en) * | 2003-08-15 | 2008-05-13 | Massachusetts Institute Of Technology | Systems and methods for volumetric tissue scanning microscopy |
US7609289B2 (en) * | 2003-09-25 | 2009-10-27 | Omnitek Partners, Llc | Methods and apparatus for capturing images with a multi-image lens |
DE102004034979A1 (en) * | 2004-07-16 | 2006-02-16 | Carl Zeiss Jena Gmbh | Method for detecting at least one sample area using a light-scanning microscope with punctiform light source distribution |
ITMI20041448A1 (en) * | 2004-07-20 | 2004-10-20 | Milano Politecnico | APPARATUS FOR THE MERGER AND NAVIGATION OF ECOGRAPHIC AND VOLUMETRIC IMAGES OF A PATIENT USING A COMBINATION OF ACTIVE AND PASSIVE OPTICAL MARKERS FOR THE LOCALIZATION OF ECHOGRAPHIC PROBES AND SURGICAL INSTRUMENTS COMPARED TO THE PATIENT |
DE102005027077C5 (en) | 2004-11-04 | 2021-01-28 | Leica Microsystems Cms Gmbh | Light disk microscope |
US7480402B2 (en) * | 2005-04-20 | 2009-01-20 | Visionsense Ltd. | System and method for producing an augmented image of an organ of a patient |
US8164622B2 (en) * | 2005-07-01 | 2012-04-24 | Aperio Technologies, Inc. | System and method for single optical axis multi-detector microscope slide scanner |
US7933056B2 (en) * | 2007-09-26 | 2011-04-26 | Che-Chih Tsao | Methods and systems of rapid focusing and zooming for volumetric 3D displays and cameras |
DE102007045897A1 (en) * | 2007-09-26 | 2009-04-09 | Carl Zeiss Microimaging Gmbh | Method for the microscopic three-dimensional imaging of a sample |
WO2009085218A1 (en) * | 2007-12-21 | 2009-07-09 | President And Fellows Of Harvard College | Sub-diffraction limit image resolution in three dimensions |
WO2009115108A1 (en) * | 2008-03-19 | 2009-09-24 | Ruprecht-Karls-Universität Heidelberg | A method and an apparatus for localization of single dye molecules in the fluorescent microscopy |
US8440952B2 (en) * | 2008-11-18 | 2013-05-14 | The Regents Of The University Of California | Methods for optical amplified imaging using a two-dimensional spectral brush |
US8620065B2 (en) * | 2010-04-09 | 2013-12-31 | The Regents Of The University Of Colorado | Methods and systems for three dimensional optical imaging, sensing, particle localization and manipulation |
WO2013086452A1 (en) * | 2011-12-09 | 2013-06-13 | California Institute Of Technology | Multi-view difraction granting imaging with two-dimensional displacement measurement for three-dimensional deformation or profile output |
US9041793B2 (en) * | 2012-05-17 | 2015-05-26 | Fei Company | Scanning microscope having an adaptive scan |
US9823458B2 (en) * | 2012-07-31 | 2017-11-21 | Georgia Tech Research Corporation | Imaging system and method for multi-scale three-dimensional deformation or profile output |
US10795144B2 (en) * | 2014-12-06 | 2020-10-06 | Howard Hughes Medical Institute | Microscopy with structured plane illumination and point accumulation for imaging and nanoscale topography |
-
2007
- 2007-09-26 DE DE102007045897A patent/DE102007045897A1/en not_active Withdrawn
-
2008
- 2008-09-16 JP JP2010526192A patent/JP5485156B2/en active Active
- 2008-09-16 WO PCT/EP2008/007686 patent/WO2009043472A2/en active Application Filing
- 2008-09-16 EP EP08802221.5A patent/EP2193398B1/en active Active
- 2008-09-16 US US12/680,056 patent/US20100201784A1/en not_active Abandoned
-
2012
- 2012-12-05 US US13/705,237 patent/US9697605B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6114695A (en) * | 1990-10-12 | 2000-09-05 | Hitachi, Ltd. | Scanning electron microscope and method for dimension measuring by using the same |
US20050047640A1 (en) * | 2000-10-24 | 2005-03-03 | Oncosis Llc | Method and device for selectively targeting cells within a three-dimensional specimen |
US20050259864A1 (en) * | 2004-05-05 | 2005-11-24 | Mary Dickinson | Four-dimensional imaging of periodically moving objects via post-acquisition synchronization of nongated slice-sequences |
Cited By (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090317044A1 (en) * | 2002-09-05 | 2009-12-24 | Nanosys, Inc. | Nanocomposites |
US20130094755A1 (en) * | 2007-09-26 | 2013-04-18 | Carl Zeiss Microlmaging Gmbh | Method for the microscopic three-dimensional reproduction of a sample |
US9697605B2 (en) * | 2007-09-26 | 2017-07-04 | Carl Zeiss Microscopy Gmbh | Method for the microscopic three-dimensional reproduction of a sample |
US20100239138A1 (en) * | 2007-10-09 | 2010-09-23 | Helmut Lippert | Method for positioning biological samples in a microscopic arrangement |
US8228499B2 (en) * | 2007-10-09 | 2012-07-24 | Carl Zeiss Microimaging Gmbh | Method for positioning biological samples in a microscopic arrangement |
US9841593B2 (en) * | 2008-12-05 | 2017-12-12 | Koninklijke Philips N.V. | Optical sectioning of a sample and detection of particles in a sample |
US20140347447A1 (en) * | 2008-12-05 | 2014-11-27 | Unisensor A/S | Optical sectioning of a sample and detection of particles in a sample |
US20100177190A1 (en) * | 2008-12-16 | 2010-07-15 | Ann-Shyn Chiang | Microscopy system with revolvable stage |
US8792162B2 (en) | 2009-09-24 | 2014-07-29 | Carl Zeiss Microscopy Gmbh | Microscope with illumination switching for capturing sample images during detector integration time |
US9506912B2 (en) | 2010-02-26 | 2016-11-29 | Massachusetts Institute Of Technology | High-throughput platform for in-vivo sub-cellular screens on vertebrate larvae |
US20110212844A1 (en) * | 2010-02-26 | 2011-09-01 | Massachusetts Institute Of Technology | High-throughput platform for in-vivo sub-cellular screens on vertebrate larvae |
US8865630B2 (en) * | 2010-02-26 | 2014-10-21 | Massachusetts Institute Of Technology | High-throughput platform for in-vivo sub-cellular screens on vertebrate larvae |
US9116354B2 (en) * | 2010-10-22 | 2015-08-25 | Leica Microsystems Cms Gmbh | SPIM microscope with a STED light sheet |
CN102455500A (en) * | 2010-10-22 | 2012-05-16 | 徕卡显微系统复合显微镜有限公司 | SPIM microscope with STED light sheet |
US11762182B2 (en) | 2010-10-22 | 2023-09-19 | Leica Microsystems Cms Gmbh | SPIM microscope with a sequential light sheet |
US9057879B2 (en) | 2010-10-22 | 2015-06-16 | Leica Microsystems Cms Gmbh | SPIM microscope with a sequential light sheet |
US11520132B2 (en) | 2010-10-22 | 2022-12-06 | Leica Microsystems Cms Gmbh | SPIM microscope with a sequential light sheet |
US20120098949A1 (en) * | 2010-10-22 | 2012-04-26 | Leica Microsystems Cms Gmbh | Spim microscope with a sted light sheet |
CN102455500B (en) * | 2010-10-22 | 2015-10-28 | 徕卡显微系统复合显微镜有限公司 | There is the SPIM microscope of STED light sheet |
US10288862B2 (en) | 2010-10-22 | 2019-05-14 | Leica Microsystems Cms Gmbh | SPIM microscope with a sequential light sheet |
US10908403B2 (en) * | 2011-02-14 | 2021-02-02 | European Molecular Biology Laboratory (Embl) | Light-pad microscope for high-resolution 3D fluorescence imaging and 2D fluctuation spectroscopy |
US20120206798A1 (en) * | 2011-02-14 | 2012-08-16 | European Molecular Biology Laboratory (Embl) | Light-pad microscope for high-resolution 3d fluorescence imaging and 2d fluctuation spectroscopy |
US9297760B2 (en) | 2011-03-11 | 2016-03-29 | Hellma Gmbh & Co., KG | Apparatus for analysing a small amount of liquid |
CN103703339A (en) * | 2011-04-29 | 2014-04-02 | 佩拉私人有限公司 | Stereo-vision system |
WO2012146720A1 (en) * | 2011-04-29 | 2012-11-01 | Peira Bvba | Stereo-vision system |
US9445078B2 (en) | 2011-04-29 | 2016-09-13 | Thrombogenics Nv | Stereo-vision system |
US9715095B2 (en) | 2011-10-11 | 2017-07-25 | Carl Zeiss Microscopy Gmbh | Microscope and method for SPIM microscopy |
US20170299853A1 (en) * | 2011-10-11 | 2017-10-19 | Carl Zeiss Microscopy Gmbh | Microscope and Method for SPIM Microscopy |
WO2014005866A1 (en) * | 2012-07-03 | 2014-01-09 | Carl Zeiss Microscopy Gmbh | Method for preparing for and carrying out the recording of image stacks of a sample from various orientation angles |
US10001634B2 (en) | 2012-07-03 | 2018-06-19 | Carl Zeiss Microscopy Gmbh | Method for preparing for and carrying out the acquisition of image stacks of a sample from various orientation angles |
US10386302B2 (en) * | 2012-10-09 | 2019-08-20 | Howard Hughes Medical Institute | Multiview light-sheet microscopy |
US10739266B2 (en) | 2012-10-09 | 2020-08-11 | Howard Hughes Medical Institute | Multiview light-sheet microscopy |
US20160161410A1 (en) * | 2012-10-09 | 2016-06-09 | Howard Hughes Medical Institute | Multiview Light-Sheet Microscopy |
JP2015537236A (en) * | 2012-10-12 | 2015-12-24 | カール ツァイス マイクロスコピー ゲーエムベーハーCarl Zeiss Microscopy Gmbh | Microscope and method for SPIM microscopy |
US9645378B2 (en) | 2012-10-12 | 2017-05-09 | Carl Zeiss Microscopy Gmbh | Microscope and method for SPIM microscopy |
US10712547B2 (en) | 2013-04-05 | 2020-07-14 | Riken | Microscope, focusing unit, fluid holding unit, and optical unit |
US9880378B2 (en) | 2013-04-05 | 2018-01-30 | Riken | Microscope, focusing unit, fluid holding unit, and optical unit |
US10222601B2 (en) | 2013-04-05 | 2019-03-05 | Riken | Microscope, focusing unit, fluid holding unit, and optical unit |
US10539772B2 (en) | 2013-10-09 | 2020-01-21 | Howard Hughes Medical Institute | Multiview light-sheet microscopy |
US10429301B2 (en) * | 2013-12-17 | 2019-10-01 | Leica Microsystems Cms Gmbh | Method and apparatus for investigating a sample by means of optical projection tomography |
US20160320301A1 (en) * | 2013-12-17 | 2016-11-03 | Leica Microsystems Cms Gmbh | Method and apparatus for investigating a sample by means of optical projection tomography |
US20150168706A1 (en) * | 2013-12-18 | 2015-06-18 | Carl Zeiss Microscopy Gmbh | Microscope and Method for SPIM Microscopy |
US9791687B2 (en) * | 2013-12-18 | 2017-10-17 | Carl Zeiss Microscopy Gmbh | Microscope and method for SPIM microscopy |
US11422503B2 (en) * | 2014-08-01 | 2022-08-23 | The Regents Of The University Of California | Device and method for iterative phase recovery based on pixel super-resolved on-chip holography |
US20160180509A1 (en) * | 2014-12-17 | 2016-06-23 | Casio Computer Co., Ltd. | Commodity identification device and commodity recognition navigation method |
US9811816B2 (en) * | 2014-12-17 | 2017-11-07 | Casio Computer Co., Ltd | Commodity identification device and commodity recognition navigation method |
US10222600B2 (en) | 2015-02-23 | 2019-03-05 | The Research Foundation For The State University Of New York | Method and apparatus for tiling light sheet selective plane illumination microscopy with real-time optimized light sheet |
WO2016138003A1 (en) * | 2015-02-23 | 2016-09-01 | The Research Foundation For The State University Of New York | Method and apparatus for tiling light sheet selective plane illumination microscopy with real-time optimized light sheet |
US10379335B2 (en) * | 2016-05-09 | 2019-08-13 | Olympus Corporation | Illumination setting method, light sheet microscope apparatus, and recording medium |
US11320640B2 (en) | 2016-06-24 | 2022-05-03 | Howard Hughes Medical Institute | Automated adjustment of light sheet geometry in a microscope |
US20200158633A1 (en) * | 2017-07-26 | 2020-05-21 | Hamamatsu Photonics K.K. | Sample observation device and sample observation method |
US11874224B2 (en) * | 2017-07-26 | 2024-01-16 | Hamamatsu Photonics K.K. | Sample observation device and sample observation method |
US11561384B2 (en) | 2017-12-22 | 2023-01-24 | Leica Microsystems Cms Gmbh | Method and device for adjusting the focus or determining the refractive index of a sample medium, non-volatile computer-readable storage medium and microscope |
US11586027B2 (en) | 2018-02-01 | 2023-02-21 | Leica Microsystems Cms Gmbh | Method for imaging a sample by means of a light-sheet microscope |
CN110501315A (en) * | 2018-05-18 | 2019-11-26 | 卡尔蔡司显微镜有限责任公司 | The method of general view image is provided |
US11808931B2 (en) | 2019-02-06 | 2023-11-07 | Evident Corporation | Image pickup apparatus with rotation unit |
Also Published As
Publication number | Publication date |
---|---|
DE102007045897A1 (en) | 2009-04-09 |
EP2193398A2 (en) | 2010-06-09 |
EP2193398B1 (en) | 2014-04-23 |
WO2009043472A2 (en) | 2009-04-09 |
JP5485156B2 (en) | 2014-05-07 |
JP2010540994A (en) | 2010-12-24 |
WO2009043472A3 (en) | 2009-07-30 |
US20130094755A1 (en) | 2013-04-18 |
US9697605B2 (en) | 2017-07-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9697605B2 (en) | Method for the microscopic three-dimensional reproduction of a sample | |
CN106842529B (en) | Quick three-dimensional micro imaging system | |
US9201011B2 (en) | Increased depth-resolution microscopy | |
US10901194B2 (en) | Large field of view, high resolution microscope | |
US7196843B2 (en) | Confocal microscope apparatus | |
US8350230B2 (en) | Method and optical assembly for analysing a sample | |
JP6453220B2 (en) | Microscope and method for SPIM microscopy | |
JP7009619B2 (en) | Double-pass macro image | |
US20130087718A1 (en) | Confocal fluorescence lifetime imaging system | |
Shin et al. | Oblique scanning 2‐photon light‐sheet fluorescence microscopy for rapid volumetric imaging | |
NL2008873C2 (en) | Method and apparatus for multiple points of view three-dimensional microscopy. | |
JP6928757B2 (en) | Systems and methods for image processing in light microscopy | |
EP1806575B1 (en) | Examination apparatus | |
Temprine et al. | Three-dimensional photoactivated localization microscopy with genetically expressed probes | |
US20020030680A1 (en) | Method and system for generating a three-dimensional object | |
Pitkeathly et al. | Auto‐Align–Multi‐Modality Fluorescence Microscopy Image Co‐registration | |
Dixon et al. | Confocal scanning of genetic microarrays |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CARL ZEISS MICROIMAGING GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LIPPERT, HELMUT;RADT, BENNO;KEMPE, MICHAEL;AND OTHERS;SIGNING DATES FROM 20100331 TO 20100430;REEL/FRAME:024478/0587 |
|
AS | Assignment |
Owner name: CARL ZEISS MICROSCOPY GMBH, GERMANY Free format text: CHANGE OF NAME;ASSIGNOR:CARL ZEISS MICROIMAGING GMBH;REEL/FRAME:030610/0269 Effective date: 20120403 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |