WO2017085618A1 - Superresolved synthetic aperture microscope - Google Patents

Superresolved synthetic aperture microscope Download PDF

Info

Publication number
WO2017085618A1
WO2017085618A1 PCT/IB2016/056861 IB2016056861W WO2017085618A1 WO 2017085618 A1 WO2017085618 A1 WO 2017085618A1 IB 2016056861 W IB2016056861 W IB 2016056861W WO 2017085618 A1 WO2017085618 A1 WO 2017085618A1
Authority
WO
WIPO (PCT)
Prior art keywords
apertures
prism
optical system
resolution
configuration
Prior art date
Application number
PCT/IB2016/056861
Other languages
French (fr)
Inventor
Theo Lasser
Daniel SZLAG
Tomas LUKES
Adrien DESCLOUX
Original Assignee
Ecole Polytechnique Federale De Lausanne (Epfl)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ecole Polytechnique Federale De Lausanne (Epfl) filed Critical Ecole Polytechnique Federale De Lausanne (Epfl)
Publication of WO2017085618A1 publication Critical patent/WO2017085618A1/en

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/58—Optics for apodization or superresolution; Optical synthetic aperture systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052—Optical details of the image generation
    • G02B21/0056—Optical details of the image generation based on optical coherence, e.g. phase-contrast arrangements, interference arrangements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052—Optical details of the image generation
    • G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/10—Beam splitting or combining systems

Definitions

  • the present invention relates to super-resolution imaging methods and related devices.
  • SOFI microscopy as described in patent application PCT/US2010/037099 represents a super-resolved imaging concept exploiting the statistical blinking behavior of the fluorescent markers without any prior knowledge of the loci of emitters within a diffraction- limited area.
  • SOFI is based on a pixel-wise auto- or cross-cumulant analysis, which yields a resolution enhancement growing with the cumulant order in all three dimensions. Uncorrelated noise, stationary background as well as out-of-focus light are greatly reduced by the cumulant analysis.
  • SOFI as a super-resolved imaging concept is conceived as a widefield microscopy method. The originally proposed SOFI does not need additional hardware and can be integrated by those skilled in the art in a classical widefield microscope.
  • Patent application WO 2012/172440 Al describes an extension to the SOFI method and a microscopy system related to super-resolved imaging.
  • Three-dimensional super- resolution in x-y-z (with z along the optical axis and xy for the lateral dimensions) can be easily realized with the disclosed prism configuration, while imaging simultaneously multiple planes onto array detectors.
  • This invention furthermore provides a method for the spectral unmixing of multi- color samples using spectral cross-cumulants.
  • Patent application WO 2015/044819 Al discloses an optical prism, which divides an incoming beam into at least two beams on the exit side.
  • This invention allows a multiplane configuration where each voxel is imaged simultaneously.
  • SOFI super resolution fluctuation imaging
  • This prism as well as the mentioned cited work is considered as prior art.
  • This prism as disclosed in this patent can also be used for multi-color 3-dimensional imaging by placing inside the prism pair a corresponding dichroic interface.
  • the axial color must be compensated by additional optical means, which ends up in a complicated optical configuration with the further limitation, that for any novel fluorophore combinations a new prism configuration may be necessary.
  • additional optical means which ends up in a complicated optical configuration with the further limitation, that for any novel fluorophore combinations a new prism configuration may be necessary.
  • the color splitting with this prism configuration becomes complicated. More than two colors to be imaged on the same detector might be truly a complicated endeavor as this prism was originally designed for 3-dimensional imaging.
  • PSF point-spread function
  • An objective of the invention is to solve at least the above-mentioned problems and/or disadvantages and to provide the advantages described herewith in the following.
  • the invention aims to enhance the optical resolution and to improve the signal to background ratio while generating multiple aperture copies and exploiting their interference in the detector plane.
  • a following post-processing using higher-order statistics for localizing single point-like emitters known by those skilled in the art as cross-cumulant analysis may result in a further enhancement of the optical resolution and an improvement of the signal to background ratio.
  • Another objective of the invention is to apply the SYNAPS principle for widefield microscopy for enhancing the resolution and improving the SNR.
  • An additional objective of this invention is the use of SYNAPS principle for generating an improved resolution for scanning confocal or scanning multi-photon microscopy.
  • a further objective of this invention is the use of SYNAPS principle for generating an improved resolution for scanning microscopy and exploiting the fast single photon emission of the fluorescence process for a fast cumulant analysis.
  • An objective of this invention to extend the SYNAPS principle for exploiting the speckle effect i.e. the dynamic stochastic response of a natural or induced dynamic speckle field.
  • a further objective of this invention is to provide microscope configurations allowing an optical sectioning for discriminating the emitted optical signal from a defined object plane inside an object.
  • a further objective of this invention is to provide microscope configurations allowing a lightsheet configuration combined with superresolved imaging.
  • the present invention relies on combining the principles of synthetic apertures with a following cumulant analysis.
  • a very robust and reliable approach for generating an aperture splitting necessary for synthetic apertures can be realized by using beam spitting means, as for example a 30-60-90 degree prism combination but not limited hereto.
  • the beam splitting mean may contain a basic prism as the well-known 30-60-90 degree prism or as an alternative prism as 45-90-45 degree prisms.
  • the combination of at least two of these prisms (known for those skilled in the art as a Koester prism) offers a simple and very flexible way to generate at least 2 apertures or precisely an aperture field splitting of identical amplitude. This combination of these basic prism combinations is also used for imaging at least two spectrally distinct fluorophores located in the same object plane.
  • An alternative prism combination based on a 45-90-45 can also be used for generating a similar aperture splitting.
  • An additional advantage for imaging different depth planes in the object space onto the same detector element can easily be achieved while adjusting or displacing the basic prism elements in an appropriate way.
  • This imaging splitting approach can be used to provide at least a two plane configuration where two axially displaced object planes are imaged on the same array detector.
  • a lens combination or an equivalent mean for example a liquid lens
  • the object and its fluorescently marked sites are illuminated by a synchronously moving lightsheet illumination.
  • a self- interference configuration can be designed which leads to a further essential narrowing or sharpening of the central peak of the point spread function when compared with the classical Airy profile. This self-interference causes an improvement of resolution and contrast.
  • a further improvement of this self-interference based resolution enhancement and the suppression of related side bands or sidelobe can be achieved when using a cumulant analysis in a post-processing step.
  • a fast hardware-processing unit can be designed to realize this data or signal processing.
  • a feature of the invention leading to a further improvement of the PSF narrowing and SNR improvement is based on a combination of cumulant analysis, i.e. a cross-cumulant analysis of the generated interference pattern originating from at least two apertures. An essential contribution is already achieved when using the co-variance between adjacent pixels as a simple algorithm for the sideband suppression.
  • the sharpening of the PSF in dependence of the aperture constellation entails pronounced sidelobes, which are strongly supressed by the cross-cumulant analysis.
  • Cross-cumulant analysis is related to cross- correlation i.e. a non-linear operation which strongly supresses the sidelobes, eliminates Gaussian noise and can depending on the optical configuration be used for optical sectioning based on a cross-cumulant analysis of emitted signals originating from object planes in different depths.
  • the cumulant analysis is known to suppress noise, which results in a further improvement of the signal-to-noise ratio SNR.
  • SYNAPS Interfering SYNthetic APertureS and cumulant analysis.
  • the interfering apertures and their various constellations can be analysed and described easily based on the so-called Array theorem.
  • A ⁇ ( ⁇ — ⁇ ⁇ ) ( ⁇ is the Dirac function).
  • ⁇ ⁇
  • ⁇ the Dirac function
  • A the location and ⁇ the amplitude and phase of a single aperture.
  • This convolution integral is ideal for the description of Fraunhofer diffraction pattern i.e. the fundamental point spread function.
  • A' as a Fourier transform of Dirac functions causes a cosinusoidal modulation of the field distribution and the expected narrowing of the point spread function.
  • Dirac functions causes a cosinusoidal modulation of the field distribution and the expected narrowing of the point spread function.
  • aperture constellation can be given for a generating a narrowed point spread function.
  • the Fourier transform ng the oscillation in the field.
  • the final intensity is
  • Microscopes as instruments can be categorized into widefield and point or line scanning instruments.
  • the SYNAPS concept applies to all microscope categories with an improvement in resolution and contrast.
  • the fluorescence process is when analysed at a single fluorescent molecule level a stochastic emission of single photons.
  • This stochastic single photon emission process offers when recorded by state of the art single photon counting detectors a fast stochastic of at least 2- state process such fulfilling the condition for a cumulant analysis as describe in detail in PCT/US2010/0370.
  • An advanced microscope configuration based on this concept offers a fast scanning microscopy concept for classical confocal as well as for multi-photon microscopy.
  • the coherence requirement or the size of the coherence volume given by the coherence length for the axial extends offers a "coherence gating" resulting in an optical sectioning. Emitters located within this coherence volume are interfering, the cumulant analysis will preferentially amplify this interference enhanced intensities and further supress the lower non-interfering contributions i.e. the non-interfering background.
  • SYNAPS principle as a general concept can be applied for widefield as well scanning microscopy concepts, including multiphoton microscopy concepts.
  • Point-like emitter Small particle (with respect to the point-spread function) that absorbs, reemits or scatters photons.
  • “Stochastic blinking” A fluctuation in measurable optical properties (e.g. emission, absorption or scattering) between at least two states of the point-like emitter. It can be intrinsic or provoked by external means and has to be independent among the individual emitters. For example, the fluctuation can be caused by a transition between two or more molecular energy levels or by a reorientation or conformational change.
  • “Synthetic aperture” the aperture field of a microscope in particular of the objective is splitted into several apertures. Preferentially all the apertures have a copy of this original aperture field (nevertheless with an amplitude divided by the number of these apertures, also called sub-apertures). These apertures are imaged onto a detector which might be an array detector (widefield microscopy) or an assembly of few detectors (scanning microscopy) for acquiring the interfering fields or intensities originating from the said apertures. It is possible to add a phase or appropriate amplitude filter to the sub-apertures for increasing the resolution or the contrast or the signal to noise ratio.
  • beam splitting mean (or synonym to image splitter or aperture splitting mean) the aperture of a microscope is ideally split into several identical apertures which obey several conditions as
  • the beamsplitting mean can be built using a diffractive or refractive or reflective or a mixed technology mean for generating the aimed pupil splitting.
  • This beam spitting mean is the core element of the synthetic aperture generating unit for generating at the detector plane a self interference between the fields originating from the sub apertures.
  • Beam beam means a propagating electromagnetic field carrying an image information. This electromagnetic is split by the beam splitting mean into different fields preferentially copies of the initial aperture field.
  • Fig. 1A shows a basic configuration i.e. (30 ⁇ -60 ⁇ -90°) prism 100.
  • the surfaces 1, 2, 3 are the surfaces of said prism.
  • Fig. IB shows a combination of 2 prisms 100 and 101 as described above constituting a basic prism combination 1000. Both identical prisms 100, 101 are in contact at surface 2 and 6.
  • the entry beam 20 at surface 3 is perpendicular to surface 3.
  • Surface 2 and or surface 6 are optically coated to split the incoming beam 20 into 2 beams preferentially with the same field amplitude and phase, which will hit or surface 5 or after reflection at the interface 2 the surface 3.
  • the beam at surface 5 undergoes a total reflection and exits the prism 101 perpendicular to surface 4.
  • the reflected beam at interface 2 will undergo a total reflection at surface 3 and exit prism 100 perpendicular to surface 4.
  • Two parallel beams are generated 21 and 22, from the initial beam 20.
  • this prism configuration 100 101 is called a Koester prism
  • the optical coating at surface 2 of prism 100 or at surface 6 of prism 101 can be designed to adjust the relative beam intensity of beam 21 and 22.
  • an appropriate dichroic coating can be used for a spectral separation of beam 20 into two spectrally distinct beams 21 and 22.
  • the prism 101 and 100 can be fabricated with different glass material for compensating chromatic aberrations.
  • Fig. 1C shows a combination of 2 prisms 100 and 101 as described in Fig. IB constituting a basic prism combination 1001.
  • the intensity split or spectral separation is realized with a separated optical mean comprising two optical flats with appropriate optical coatings.
  • Fig. 2a shows a combination of 2 prisms 100 and 101 constituting a basic prism combination 1111.
  • the prisms 100, 101 are displaced. Obviously this relative displacement results in a continuously adjustable optical path difference for the beams 21 and 22.
  • Fig 2b shows an alternative prism configuration consisting of 2 (45 ⁇ -90 ⁇ -45°) prisms 2902 and 2903 forming the configuration 2911.
  • the incoming beam 290 is split at the common interface of said prisms into a first upward directed beam, which is reflected by the mirror or reflected layer 2901 and is forming a first beam (or aperture copy) 291.
  • the splitting at the common interface of said prism leads to a second beam (or aperture copy) 292.
  • this prism configuration allows a large range of aperture separation by simply shifting the incoming beam 290.
  • a further property of this prism configuration is the identical optical path length of both beam 291 and 292 independent of the beam height of the incoming beam 290.
  • a preferential size of this prism configuration is the length of the hypotenuse of prism 2902 is matching the kathete of prism 2903. However this is not a necessary requirement.
  • the common interface between said prism has normally a 50%:50% coating for ensuring an equal field distribution in both sub-apertures. It is obvious for those skilled in the art that this coating can also be used for a spectral split of the incoming beam 290.
  • Fig. 2c shows an almost identical prism configuration 2912 as configuration 2911, but with slightly displaced prisms 2902 and 2912 to introduce an optical path length difference.
  • Fig. 2d shows a further prism configuration 2111 consisting of 4 prisms.
  • Prism 2100 and 2101 form a classical beamsplitter which is matched at its sidefaces with a (45 ⁇ -90 ⁇ -45°) prism for directing the incoming beam 220 splitted at the beamsplitters interface in an upward traveling beam which is after 2 total reflections reversed and forms the beam 222.
  • the transmitted beam at the beamsplitters interface crosses the optical flat 2104 and is backreflected by the coating 2112 and further reflected at the interface forming the beam 221.
  • Fig. 2e shows an almost identical configuration 2811 as disclosed in Fig. 2d.
  • the prism 2100 and 2102 as shown in Fig. 2d are replaced by a single prism 2800 fulfilling the same beam steering then the former prism configuration 2100 combined with 2102.
  • Fig. 3A shows a combination of two Koester prisms as described above and shown in Fig. IB and 1C.
  • the entry beam 20 has a broad spectrum as indicated in 320.
  • the coating 62, 63 are made in an appropriate way to decompose the entry spectrum 320, into 321 for the exit beam 21, into 322 for the exit beam 22 and into 323 for the exit beam 23.
  • This spectral splitting serves to channel a multi spectral object into 2 images and a registration on 2 array detectors (or single point detectors) with a spectral separation as just described.
  • Fig. 3B shows an identical optical configuration as disclosed in Fig. 3A but used as a 3 channel beam combiner.
  • this reciprocal configuration allows to combine spectrally distinct beams 21, 22, 23 to be combined into a single output beam 20 with the help of the disclosed optical configuration shown in Fig. 3B.
  • the configuration is not limited to 3 channels but can be decreased or increased by a cascade of additional Koester prims to more channels.
  • Fig. 4a discloses a diffraction grating based configuration 450 for splitting an incoming beam 411 into 4 identical beams 412, 413, 414 and 415.
  • the cross gratings 421 and cross-grating 422 are designed to build a diffractive telescope.
  • the glass plate 425 serves as a carrier for the grating and ensures a distance between the gratings in such a way the aperture separation is optimized for the SYNAPS requirements and lead to the search resolution enhancement.
  • Fig. 4b discloses a refraction based configuration 4800 for splitting an incoming beam 461 into 4 identical beams 462, 463, 464 and 465.
  • This configuration 4800 is realized with 2 Koester prisms (Koester prism 1000) as shown in Fig. IB.
  • the Koester prisms 1000 are aligned along the axis 4651 and are in an orthogonal orientation.
  • prism 4611 is oriented upright in the z-x plane whereas prism 4612 is oriented downright in the y-z plane.
  • this configuration is a refractive embodiment for generating 4 apertures.
  • For visualisation purposes we added representative rays for this configuration 4800.
  • Fig. 4c is identical to Fig. 2b and reminds the prism configuration 2911.
  • Fig 4d uses two configurations 2911 in a 90 degree reoriented way to generate a 4 sub-aperture configuration.
  • Fig. 4e discloses a prism configuration consisting of 2 elements 2911 as described in Fig. 4c.
  • the incoming beam 1290 is split into 4 beams in a square equidistant beam configuration indicated by the beams 1291, 1292, 1293, 1294.
  • Fig. 4f represents an equivalent compact prism configuration, which allows a square equidistant beam configuration indicated by the beams 1291, 1292, 1293, 1294.
  • Fig. 4g represents a beam splitting mean, which is built by a classical beam splitter 4600 and 4601. The beam are reversed by the triple prism 4603 (45-90-45 ) and 4604 and exit the prism 4601 on its free side in a parallel beam configuration.
  • Fig. 4h shows 2 prisms 4700 and 4701, which redirect the incoming beam 470 into 2 beams 476 and 472. Basically the same beam constellation as in Fig. 4g is achieved. The shown configuration can be used to adjust the optical path difference.
  • Fig. 4i shows a beamsplitting mean consisting of 3 prisms 4800, 4801 and 4802. Only for visualization purposes the prism 4802 is displaced into position 4803 showing the flexibility of this beamsplitting mean for adjusting the distance between the outgoing beams 486 and 482.
  • the optical mean 4805 serve as a ghost imaging suppression mean for avoiding distortions for the self-interference.
  • Fig. 4j discloses a beamsplitting mean based on the basic 30-90-60 prism.
  • the prism 4904 is identical to prism 4902.
  • the prisms 4903 and 4901 are identical and only different in size to 4904 or 4902.
  • the incoming is split at the interface between prism 4904 and 4901 into 2 beam of equal intensity.
  • the underlying concept of the Koester prisms ensures a parallel exit indicated by the beam 492 and 493.
  • a coating 4902 with a reflection of 100% or close too, ensures almost identical beam intensity and almost no loss during propagation, splitting and total reflexion during the propagation inside this beam splitting mean.
  • this beamsplitting mean 4911 can be built based on only two identical prisms, where each of said prism integrates a prism 4904 and 4903.
  • Fig. 5 describes a 4 channel - 2-array detector microscope, which comprises a classical widefield microscope by the optical means of 2002, 2003 and for each color channel a tubelens 53 or 54 or 51 or 52. These optical means are imaging the object plane 2001 onto the array detector 152 or 151.
  • the explicit function of the microscope 2000 is further described in detail in Fig. 8 and the associated description.
  • Fig. 6 describes a two-plane super-resolved lightsheet microscope comprising several subsystems similar to the classical widefield microscope 2000, the beamsplitter 3000, a lightsheet illumination system 4000 and an array detector 151.
  • the classical widefield microscope 2000 comprises a multiple lens system 2004, 2004' for a continuously adaptable focal length of this said lens system.
  • the beam splitting mean 3000 comprises two basic prisms slightly displaced. The such induced optical path difference allows to image simultaneously the object planes 2001 and 2001' onto the array detector 151.
  • the focal length variation caused by the axial displacement of the lens combination 2004 and 2004' causes a continuous axial displacement of the object planes 2001 and 2001'. This axial displacement is synchronized with the axial displacement of the lateral illumination light sheet 4070.
  • the beam splitting mean 3000 can also be realized by the configuration shown in 2b until 2e or for 4 apertures in the realizations shown in Fig. 4a,b,c,d.
  • a superresolved configuration could be achieved be simply interchanging the beam splitting mean before and the flexible tubelens after.
  • the illumination subsystem 4000 as indicated in Fig. 6 comprises a beam splitting element 4010 and an optical mean 4031 for deviating the light beam perpendicular to the optical axis 2006.
  • the optical mean 4031 comprises a cylindrical lens for generating the said light sheet illumination.
  • the illumination system 4000 comprises a symmetrical lens combination 4030 for generating a homogeneous lightsheet.
  • the optical mean 4060 comprises a flat element 4060, which when turned causes a lateral beamshift which is translated into an axial displacement of the lightsheet synchronized with the axial object plane displacement as described above.
  • Fig. 7A describes a general microscope configuration integrating a synthetic aperture module 8500 consisting either of a refractive as for example the module 4800 or a diffractive module 450 but is not limited to these examples.
  • This general optical set-up images an object in plane 2001 across the objective 2002 and back focal aperture 2003 across this general module 8500.
  • Module 8500 is followed by a tubelens 2004, which generates a self-interference between the contributing apertures.
  • the array sensor is preferentially positioned in the focal plane of lens 2004.
  • Fig. 7B describes a realization of the general configuration shown in Fig. 7A.
  • the module 8500 is realized by module 4800 or the prism configurations disclosed in Fig. 4d.
  • the generated four apertures This general optical set-up images an object in plane 2001 across the objective 2002 and back focal aperture 2003 across this general module 8500.
  • Module 8500 is followed by a tubelens 2004, which generates a self-interference between the contributing apertures.
  • the array sensor is preferentially positioned in the focal plane of lens 2004.
  • Fig 8A, 8B and 8C demonstrate and visualize the resolution enhancement due to the SYNAPS principle.
  • Fig. 8A shows the intensity distribution 1201 and the corresponding footprint of the point spread function in 1202 of a single aperture the typical point spreadfunction known from widefield microscopy.
  • the intensity distribution 1203 is showing the achievable narrowing of the point spread function when using a "4 - aperture" configuration. Obviously this intensity distribution shows some sidelobes.
  • the intensity distribution 1204 shows a point spread function (footprint) and its corresponding footprint in 1205. It is obvious that this is a strong improvement for the resolution due to the cumulant analysis is achieved. This resolution improvement can be achieved with a scanning as well as a widefield microscopy system.
  • Fig 9 discloses the application of the SYNAPS principle for a widefield microscopy.
  • the image shown in Fig. 9A is an average of 1000 widefield images.
  • Fig. 9B shows a clear improvement for the resolution for this widefield setting due to the SynAps principle.
  • Fig. 10 shows a scanning microscope with 2021 the object plane, 2002 the objective,
  • the infinity space between the back-aperture 2003 and the tubelens 2004 contains an optical mean for scanning the object in plane 2021 and general beam splitting 2015 which can be realized as shown in Fig. 4A with a diffractive unit 450 or a refraction based configuration 4800 or the prism configuration in Fig. 4d but not limited to these configurations.
  • the following optical mean is a tubelens 2004, which is forming an image in the image plane 2016. In this plane 2016 is also positioned a fast few element array detector or the entry port of a fibre bundle. If the point image overlaps over several detector elements as shown in 2011 or for the fibre bundle in 2012 the SYNAPS principle can be used for a strong resolution enhancement.
  • Fig. 11A describes a classical widefield microscope 2000, a so-called ICS-System (Infinity Color Corrected System).
  • the objective 2002 and the tubelens 2004 are imaging the object plane 2001 into the image plane 2005. All optical means 2002, the aperture 2003 and
  • the object plane 2001 is in the front focal plane of the objective, whereas the aperture plane 2003 is in the back focal of this said objective 2002. For those skilled in the art this represent a telecentric system in the object space.
  • a further subassembly is shown as 2200, which contains the tubelens 2004.
  • Objective 2002 combined with the tubelens 2004 image the objectplane 2001 into the image plane 2005.
  • Fig. 11B is a general microscope 2400 containing the subassembly 2100 and the subassembly 2200 the optical means for imaging the object plane 2001 in the image plane 2005.. Between the objective subassembly 2100 and the tubelens subassembly 2200 is placed the SYNAPS module for splitting the entry aperture into several exit aperture and such realizing the SYNAPS principle, which allows to further increase the resolution.
  • This general configuration applies to widefield microscopy, scanning microscopy and multiphoton microscopy but is not limited hereto.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

Optical system comprising a synthetic aperture generating unit; said synthetic aperture generating unit being adapted for generating several apertures, which are correlated by interference for improving contrast and/or resolution based on the interference pattern generated by said apertures. The invention also concerns a method for improving the contrast and/or resolution of an optical system comprising the generation of synthetic apertures.

Description

Superresolved synthetic aperture microscope Field of the invention
[0001] The present invention relates to super-resolution imaging methods and related devices.
State of the art
[0002] SOFI microscopy as described in patent application PCT/US2010/037099 represents a super-resolved imaging concept exploiting the statistical blinking behavior of the fluorescent markers without any prior knowledge of the loci of emitters within a diffraction- limited area. SOFI is based on a pixel-wise auto- or cross-cumulant analysis, which yields a resolution enhancement growing with the cumulant order in all three dimensions. Uncorrelated noise, stationary background as well as out-of-focus light are greatly reduced by the cumulant analysis. SOFI as a super-resolved imaging concept is conceived as a widefield microscopy method. The originally proposed SOFI does not need additional hardware and can be integrated by those skilled in the art in a classical widefield microscope.
[0003] Patent application WO 2012/172440 Al describes an extension to the SOFI method and a microscopy system related to super-resolved imaging. Three-dimensional super- resolution in x-y-z (with z along the optical axis and xy for the lateral dimensions) can be easily realized with the disclosed prism configuration, while imaging simultaneously multiple planes onto array detectors. This invention furthermore provides a method for the spectral unmixing of multi- color samples using spectral cross-cumulants.
[0004] Patent application WO 2015/044819 Al discloses an optical prism, which divides an incoming beam into at least two beams on the exit side. This invention allows a multiplane configuration where each voxel is imaged simultaneously. For those skilled in the art such a prism is of interest as it allows to perform super-resolved imaging in three dimensions based on super resolution fluctuation imaging (SOFI). This prism as well as the mentioned cited work is considered as prior art. This prism as disclosed in this patent can also be used for multi-color 3-dimensional imaging by placing inside the prism pair a corresponding dichroic interface. As disclosed in this patent the axial color must be compensated by additional optical means, which ends up in a complicated optical configuration with the further limitation, that for any novel fluorophore combinations a new prism configuration may be necessary. When imaging a planar object as for example a fluorescently labeled DNA strand the color splitting with this prism configuration becomes complicated. More than two colors to be imaged on the same detector might be truly a complicated endeavor as this prism was originally designed for 3-dimensional imaging.
[0005] Another method known for achieving super-resolution is based on so-called pupil plane filters. These filters aim a sharpening of the point-spread function (PSF). In essence the consists of arrays of rings or of a continuously varying transmittance for achieving a wavefront modification such that the mentioned pupil modulations will sharpen the point spread function. Many of these concepts lead to a resolution enhancement up to a factor of two. However this wavefront manipulation shows very often additional sidebands or side lobes which results in undesirable artefacts.
[0006] Another concept known as synthetic apertures has been applied to astronomy for enhancing the resolution. This concept has been used in various configurations and fields including acoustics, stellar interferometry but are not limited hereto. In essence this concept is based on generating several apertures, which are correlated by interference for improving contrast and/or resolution. However as in the former concept on pupil plane filters undesirable artefacts like pronounced sidelobes may limit the fields of application.
General description of the invention
[0007] An objective of the invention is to solve at least the above-mentioned problems and/or disadvantages and to provide the advantages described herewith in the following.
[0008] The invention aims to enhance the optical resolution and to improve the signal to background ratio while generating multiple aperture copies and exploiting their interference in the detector plane. A following post-processing using higher-order statistics for localizing single point-like emitters known by those skilled in the art as cross-cumulant analysis may result in a further enhancement of the optical resolution and an improvement of the signal to background ratio.
[0009] Another objective of the invention is to apply the SYNAPS principle for widefield microscopy for enhancing the resolution and improving the SNR.
[0010] An additional objective of this invention is the use of SYNAPS principle for generating an improved resolution for scanning confocal or scanning multi-photon microscopy.
[0011] A further objective of this invention is the use of SYNAPS principle for generating an improved resolution for scanning microscopy and exploiting the fast single photon emission of the fluorescence process for a fast cumulant analysis.
[0012] An objective of this invention to extend the SYNAPS principle for exploiting the speckle effect i.e. the dynamic stochastic response of a natural or induced dynamic speckle field.
[0013] A further objective of this invention is to provide microscope configurations allowing an optical sectioning for discriminating the emitted optical signal from a defined object plane inside an object.
[0014] A further objective of this invention is to provide microscope configurations allowing a lightsheet configuration combined with superresolved imaging.
Summary of the invention
[0015] The present invention relies on combining the principles of synthetic apertures with a following cumulant analysis. A very robust and reliable approach for generating an aperture splitting necessary for synthetic apertures can be realized by using beam spitting means, as for example a 30-60-90 degree prism combination but not limited hereto. The beam splitting mean may contain a basic prism as the well-known 30-60-90 degree prism or as an alternative prism as 45-90-45 degree prisms. The combination of at least two of these prisms (known for those skilled in the art as a Koester prism) offers a simple and very flexible way to generate at least 2 apertures or precisely an aperture field splitting of identical amplitude. This combination of these basic prism combinations is also used for imaging at least two spectrally distinct fluorophores located in the same object plane.
[0016] Taking more than 2 of the elementary prisms 30-60-90 allows to design and built a beam splitting mean where all generated apertures have an optical path difference = 0, the same image orientation and comparable optical field distributions in order to generate a self-interference at the detector plane of all contributing apertures.
[0017] An alternative prism combination based on a 45-90-45 can also be used for generating a similar aperture splitting.
[0018] An additional advantage for imaging different depth planes in the object space onto the same detector element can easily be achieved while adjusting or displacing the basic prism elements in an appropriate way.
[0019] This imaging splitting approach can be used to provide at least a two plane configuration where two axially displaced object planes are imaged on the same array detector. Providing a lens combination or an equivalent mean (for example a liquid lens) as an continuously adjustable tubelens generating a continuous fast axial displacement or depth scanning of the object planes. For such a configuration the object and its fluorescently marked sites are illuminated by a synchronously moving lightsheet illumination.
[0020] Based on the prism configurations and its aperture splitting property a self- interference configuration can be designed which leads to a further essential narrowing or sharpening of the central peak of the point spread function when compared with the classical Airy profile. This self-interference causes an improvement of resolution and contrast.
[0021] A further improvement of this self-interference based resolution enhancement and the suppression of related side bands or sidelobe can be achieved when using a cumulant analysis in a post-processing step. A fast hardware-processing unit can be designed to realize this data or signal processing. [0022] A feature of the invention leading to a further improvement of the PSF narrowing and SNR improvement is based on a combination of cumulant analysis, i.e. a cross-cumulant analysis of the generated interference pattern originating from at least two apertures. An essential contribution is already achieved when using the co-variance between adjacent pixels as a simple algorithm for the sideband suppression. The sharpening of the PSF in dependence of the aperture constellation entails pronounced sidelobes, which are strongly supressed by the cross-cumulant analysis. Cross-cumulant analysis is related to cross- correlation i.e. a non-linear operation which strongly supresses the sidelobes, eliminates Gaussian noise and can depending on the optical configuration be used for optical sectioning based on a cross-cumulant analysis of emitted signals originating from object planes in different depths. In addition the cumulant analysis is known to suppress noise, which results in a further improvement of the signal-to-noise ratio SNR. We name this principal SYNAPS (Interfering SYNthetic APertureS and cumulant analysis).
[0023] The interfering apertures and their various constellations can be analysed and described easily based on the so-called Array theorem. When using identical apertures and representing its amplitude and phase distribution as ψ(ξη) and a collection of these elementary apertures located at different locations ξη then the total instrument apertures consisting of this aperture collection is described as a sum of ψ(ξ) =∑± ψ(ξ— ξη) where the sum over the elementary apertures extends over n, representing the number of these identical apertures.
The location of these apertures can be describes in general as a sum of dirac functions giving the locations of said elementary apertures as A(a) = δ(α— ξη) (δ is the Dirac function). This allows to describe the total instrument aperture as a convolution given as Ψ(ξ) = / άα Ψ(ξ— α)Α(α). The term A describes the location and Ψ the amplitude and phase of a single aperture. This convolution integral is ideal for the description of Fraunhofer diffraction pattern i.e. the fundamental point spread function. The diffraction pattern is given as a Fourier transform of this expression leading to the generated diffraction pattern as Γ(χ) = ψ'(χ)Α'(χ). The term A' as a Fourier transform of Dirac functions causes a cosinusoidal modulation of the field distribution and the expected narrowing of the point spread function. For the skilled the art several aperture constellation can be given for a generating a narrowed point spread function.
[0024] For a configuration with two circular apertures, the elementary aperture is given in its intensity as Ieie(r) = ip(r)ip*(r) = π2α Α2 \2J(kar/f)/(kar/f) \2 with k the wavevector and a, the radius of the opening, A the field amplitude and f the focal length. The position of two of these openings is given as Α(ξ) = δ(ξ— b) + δ(ξ— b) with 2b the separating distance between the centre of these elementary opening. The Fourier transform ng the oscillation in the field. The final intensity is
Figure imgf000008_0001
w'tn r = *2 + y2 ·
Obviously this expression shows the interference leading to an oscillating point spread function.
[0025] The above given analysis is based on identical apertures. This is in now case a requirement when searching for a narrowing of the point spread function and applying a synthetic aperture concept. Different elementary apertures including individual phase modulations may lead to a narrowing of the point spread function and in consequence to a further image resolution.
[0026] For those skilled in the art these calculation can be easily expanded to more than two openings and even further generalized for the interference of non-identical apertures integrating phase variations and/or phase or amplitude filters for a further modification of the wavefront in these individual openings and finally a narrowed point spread function.
[0027] The use of a synthetic aperture leads to a strong amplification of the central peak.
At this privileged location a minimum optical path difference lead to this strong intensity peak with a strongly narrowed PSF. The sideband generation depends on the spatial coherence of the lightfield in the aperture. A low spatial coherence as for example of a fluorescent emitter but not limited hereto will lead due to the limited spatial coherence to a further sideband suppression with no need for further cumulant processing. [0028] This general and central teaching leads to a PSF narrowing per image acquisition and does not need to acquire sequences of images or signals for a further postprocessing of the acquired signal.
[0029] Microscopes as instruments can be categorized into widefield and point or line scanning instruments. The SYNAPS concept applies to all microscope categories with an improvement in resolution and contrast.
[0030] An additional concept based on the fluorescence emission process can be realized.
The fluorescence process is when analysed at a single fluorescent molecule level a stochastic emission of single photons. This stochastic single photon emission process offers when recorded by state of the art single photon counting detectors a fast stochastic of at least 2- state process such fulfilling the condition for a cumulant analysis as describe in detail in PCT/US2010/0370.
[0031] If these photons are detected across a fast array detector or a fiber bundle guiding these photons to a fast photon counting array a consequent photon registration and cumulant analysis leads to a further enhancement of the resolution and contrast.
[0032] An advanced microscope configuration based on this concept offers a fast scanning microscopy concept for classical confocal as well as for multi-photon microscopy. The coherence requirement or the size of the coherence volume given by the coherence length for the axial extends offers a "coherence gating" resulting in an optical sectioning. Emitters located within this coherence volume are interfering, the cumulant analysis will preferentially amplify this interference enhanced intensities and further supress the lower non-interfering contributions i.e. the non-interfering background.
[0033] In summary, the combination of a synthetic aperture combined with a cumulant analysis offers solutions for
• An microscope system with increased resolution and an enhanced contrast can be advised when using the self-interference • A further improvement of resolution and a strong suppression of sidelobes can be achieved when using the self-interference and a cumulant analysis
• A lightsheet configuration with a three-dimensional resolution enhancement has been disclosed
• A microscope set-up integrating a synthetic aperture for an increased resolution.
• The so-called SYNAPS principle as a general concept can be applied for widefield as well scanning microscopy concepts, including multiphoton microscopy concepts.
• The fluorescence emission by itself when captured with a fast array detector or a fibre bundle linked to ultrafast single photon detectors allows to use this emission process for integrating the SYNAPS principle and a cumulant analysis for a resolution enhancement. This concept applies in particular to confocal and multiphoton microscopy.
Definitions and terms
[0034] "Point-like emitter": Small particle (with respect to the point-spread function) that absorbs, reemits or scatters photons.
[0035] "Stochastic blinking": A fluctuation in measurable optical properties (e.g. emission, absorption or scattering) between at least two states of the point-like emitter. It can be intrinsic or provoked by external means and has to be independent among the individual emitters. For example, the fluctuation can be caused by a transition between two or more molecular energy levels or by a reorientation or conformational change.
[0036] "Synthetic aperture": the aperture field of a microscope in particular of the objective is splitted into several apertures. Preferentially all the apertures have a copy of this original aperture field (nevertheless with an amplitude divided by the number of these apertures, also called sub-apertures). These apertures are imaged onto a detector which might be an array detector (widefield microscopy) or an assembly of few detectors (scanning microscopy) for acquiring the interfering fields or intensities originating from the said apertures. It is possible to add a phase or appropriate amplitude filter to the sub-apertures for increasing the resolution or the contrast or the signal to noise ratio.
[0037] "beam splitting mean": (or synonym to image splitter or aperture splitting mean) the aperture of a microscope is ideally split into several identical apertures which obey several conditions as
• conserved image orientation at least in one spatial direction preferentially in all lateral dimensions
• Optical path difference equal or close to zero but less than the coherence length of the fluorescence emitters
• Comparable intensities or amplitude field distributions for a good modulation depth of the self-interference at the detector array
• No polarisation mismatch between the different apertures
These conditions are useful but in now way a limitation for generating a self- interference at the detector plane. Symmetric or asymmetric phase manipulation may provide potential solutions for the self-interference.
The beamsplitting mean can be built using a diffractive or refractive or reflective or a mixed technology mean for generating the aimed pupil splitting.
This beam spitting mean is the core element of the synthetic aperture generating unit for generating at the detector plane a self interference between the fields originating from the sub apertures.
[0038] "Beam": beam means a propagating electromagnetic field carrying an image information. This electromagnetic is split by the beam splitting mean into different fields preferentially copies of the initial aperture field. [0039]
Detailed description of the invention
[0040] Fig. 1A shows a basic configuration i.e. (30ο-60ο-90°) prism 100. The surfaces 1, 2, 3 are the surfaces of said prism.
[0041] Fig. IB shows a combination of 2 prisms 100 and 101 as described above constituting a basic prism combination 1000. Both identical prisms 100, 101 are in contact at surface 2 and 6. The entry beam 20 at surface 3 is perpendicular to surface 3. Surface 2 and or surface 6 are optically coated to split the incoming beam 20 into 2 beams preferentially with the same field amplitude and phase, which will hit or surface 5 or after reflection at the interface 2 the surface 3. The beam at surface 5 undergoes a total reflection and exits the prism 101 perpendicular to surface 4. The reflected beam at interface 2 will undergo a total reflection at surface 3 and exit prism 100 perpendicular to surface 4. Two parallel beams are generated 21 and 22, from the initial beam 20. For those skilled in the art this prism configuration 100, 101 is called a Koester prism The optical coating at surface 2 of prism 100 or at surface 6 of prism 101 can be designed to adjust the relative beam intensity of beam 21 and 22. Obviously an appropriate dichroic coating can be used for a spectral separation of beam 20 into two spectrally distinct beams 21 and 22. The prism 101 and 100 can be fabricated with different glass material for compensating chromatic aberrations.
[0042] Fig. 1C shows a combination of 2 prisms 100 and 101 as described in Fig. IB constituting a basic prism combination 1001. In this prism combination the intensity split or spectral separation is realized with a separated optical mean comprising two optical flats with appropriate optical coatings.
[0043] Fig. 2a shows a combination of 2 prisms 100 and 101 constituting a basic prism combination 1111. In this prism combination 1111 the prisms 100, 101 are displaced. Obviously this relative displacement results in a continuously adjustable optical path difference for the beams 21 and 22. [0044] Fig 2b shows an alternative prism configuration consisting of 2 (45ο-90ο-45°) prisms 2902 and 2903 forming the configuration 2911. The incoming beam 290 is split at the common interface of said prisms into a first upward directed beam, which is reflected by the mirror or reflected layer 2901 and is forming a first beam (or aperture copy) 291. The splitting at the common interface of said prism leads to a second beam (or aperture copy) 292. It is worth noting that this prism configuration allows a large range of aperture separation by simply shifting the incoming beam 290. A further property of this prism configuration is the identical optical path length of both beam 291 and 292 independent of the beam height of the incoming beam 290. A preferential size of this prism configuration is the length of the hypotenuse of prism 2902 is matching the kathete of prism 2903. However this is not a necessary requirement. The common interface between said prism has normally a 50%:50% coating for ensuring an equal field distribution in both sub-apertures. It is obvious for those skilled in the art that this coating can also be used for a spectral split of the incoming beam 290.
[0045] Fig. 2c shows an almost identical prism configuration 2912 as configuration 2911, but with slightly displaced prisms 2902 and 2912 to introduce an optical path length difference.
[0046] Fig. 2d shows a further prism configuration 2111 consisting of 4 prisms. Prism 2100 and 2101 form a classical beamsplitter which is matched at its sidefaces with a (45ο-90ο-45°) prism for directing the incoming beam 220 splitted at the beamsplitters interface in an upward traveling beam which is after 2 total reflections reversed and forms the beam 222. The transmitted beam at the beamsplitters interface crosses the optical flat 2104 and is backreflected by the coating 2112 and further reflected at the interface forming the beam 221.
[0047] Fig. 2e shows an almost identical configuration 2811 as disclosed in Fig. 2d. The prism 2100 and 2102 as shown in Fig. 2d are replaced by a single prism 2800 fulfilling the same beam steering then the former prism configuration 2100 combined with 2102. [0048] Fig. 3A shows a combination of two Koester prisms as described above and shown in Fig. IB and 1C. The entry beam 20 has a broad spectrum as indicated in 320. The coating 62, 63 are made in an appropriate way to decompose the entry spectrum 320, into 321 for the exit beam 21, into 322 for the exit beam 22 and into 323 for the exit beam 23. This spectral splitting serves to channel a multi spectral object into 2 images and a registration on 2 array detectors (or single point detectors) with a spectral separation as just described.
[0049] Fig. 3B shows an identical optical configuration as disclosed in Fig. 3A but used as a 3 channel beam combiner. For those skilled in the art this reciprocal configuration allows to combine spectrally distinct beams 21, 22, 23 to be combined into a single output beam 20 with the help of the disclosed optical configuration shown in Fig. 3B. The configuration is not limited to 3 channels but can be decreased or increased by a cascade of additional Koester prims to more channels.
[0050] Fig. 4a discloses a diffraction grating based configuration 450 for splitting an incoming beam 411 into 4 identical beams 412, 413, 414 and 415. The cross gratings 421 and cross-grating 422 are designed to build a diffractive telescope. The glass plate 425 serves as a carrier for the grating and ensures a distance between the gratings in such a way the aperture separation is optimized for the SYNAPS requirements and lead to the search resolution enhancement.
[0051] Fig. 4b discloses a refraction based configuration 4800 for splitting an incoming beam 461 into 4 identical beams 462, 463, 464 and 465. This configuration 4800 is realized with 2 Koester prisms (Koester prism 1000) as shown in Fig. IB. The Koester prisms 1000 are aligned along the axis 4651 and are in an orthogonal orientation. As an example but not limited hereto prism 4611 is oriented upright in the z-x plane whereas prism 4612 is oriented downright in the y-z plane. Along the former saying, this configuration is a refractive embodiment for generating 4 apertures. For visualisation purposes we added representative rays for this configuration 4800.
[0052] Fig. 4c is identical to Fig. 2b and reminds the prism configuration 2911. [0053] Fig 4d uses two configurations 2911 in a 90 degree reoriented way to generate a 4 sub-aperture configuration.
[0054] Fig. 4e discloses a prism configuration consisting of 2 elements 2911 as described in Fig. 4c. The incoming beam 1290 is split into 4 beams in a square equidistant beam configuration indicated by the beams 1291, 1292, 1293, 1294.
[0055] Fig. 4f represents an equivalent compact prism configuration, which allows a square equidistant beam configuration indicated by the beams 1291, 1292, 1293, 1294.
[0056] Fig. 4g represents a beam splitting mean, which is built by a classical beam splitter 4600 and 4601. The beam are reversed by the triple prism 4603 (45-90-45 ) and 4604 and exit the prism 4601 on its free side in a parallel beam configuration.
[0057] Fig. 4h shows 2 prisms 4700 and 4701, which redirect the incoming beam 470 into 2 beams 476 and 472. Basically the same beam constellation as in Fig. 4g is achieved. The shown configuration can be used to adjust the optical path difference.
[0058] Fig. 4i shows a beamsplitting mean consisting of 3 prisms 4800, 4801 and 4802. Only for visualization purposes the prism 4802 is displaced into position 4803 showing the flexibility of this beamsplitting mean for adjusting the distance between the outgoing beams 486 and 482. The optical mean 4805 serve as a ghost imaging suppression mean for avoiding distortions for the self-interference.
[0059] Fig. 4j discloses a beamsplitting mean based on the basic 30-90-60 prism. The prism 4904 is identical to prism 4902. The prisms 4903 and 4901 are identical and only different in size to 4904 or 4902. There are basically 2 input ports 490 and 491 (not used at the same time!). As an example the incoming is split at the interface between prism 4904 and 4901 into 2 beam of equal intensity. The underlying concept of the Koester prisms ensures a parallel exit indicated by the beam 492 and 493. A coating 4902 with a reflection of 100% or close too, ensures almost identical beam intensity and almost no loss during propagation, splitting and total reflexion during the propagation inside this beam splitting mean. It is obvious for those skilled in the art, that this beamsplitting mean 4911 can be built based on only two identical prisms, where each of said prism integrates a prism 4904 and 4903. [0060] Fig. 5 describes a 4 channel - 2-array detector microscope, which comprises a classical widefield microscope by the optical means of 2002, 2003 and for each color channel a tubelens 53 or 54 or 51 or 52. These optical means are imaging the object plane 2001 onto the array detector 152 or 151. The explicit function of the microscope 2000 is further described in detail in Fig. 8 and the associated description. The image splitter 3000 containing a prism configuration 3001 as described in detail in Fig. 3 and ensures a spectral decomposition. This image splitter 3000 can also be built based on the alternative prism configurations (see Fig. 2 b etc.).
[0061] Fig. 6 describes a two-plane super-resolved lightsheet microscope comprising several subsystems similar to the classical widefield microscope 2000, the beamsplitter 3000, a lightsheet illumination system 4000 and an array detector 151. The classical widefield microscope 2000 comprises a multiple lens system 2004, 2004' for a continuously adaptable focal length of this said lens system. The beam splitting mean 3000 comprises two basic prisms slightly displaced. The such induced optical path difference allows to image simultaneously the object planes 2001 and 2001' onto the array detector 151. The focal length variation caused by the axial displacement of the lens combination 2004 and 2004' causes a continuous axial displacement of the object planes 2001 and 2001'. This axial displacement is synchronized with the axial displacement of the lateral illumination light sheet 4070. As before the beam splitting mean 3000 can also be realized by the configuration shown in 2b until 2e or for 4 apertures in the realizations shown in Fig. 4a,b,c,d. A superresolved configuration could be achieved be simply interchanging the beam splitting mean before and the flexible tubelens after.
[0062] The illumination subsystem 4000 as indicated in Fig. 6 comprises a beam splitting element 4010 and an optical mean 4031 for deviating the light beam perpendicular to the optical axis 2006. The optical mean 4031 comprises a cylindrical lens for generating the said light sheet illumination. As shown in Fig. 6 the illumination system 4000 comprises a symmetrical lens combination 4030 for generating a homogeneous lightsheet. The optical mean 4060 comprises a flat element 4060, which when turned causes a lateral beamshift which is translated into an axial displacement of the lightsheet synchronized with the axial object plane displacement as described above.
[0063] Fig. 7A describes a general microscope configuration integrating a synthetic aperture module 8500 consisting either of a refractive as for example the module 4800 or a diffractive module 450 but is not limited to these examples. This general optical set-up images an object in plane 2001 across the objective 2002 and back focal aperture 2003 across this general module 8500. Module 8500 is followed by a tubelens 2004, which generates a self-interference between the contributing apertures. The array sensor is preferentially positioned in the focal plane of lens 2004.
[0064] Fig. 7B describes a realization of the general configuration shown in Fig. 7A. The module 8500 is realized by module 4800 or the prism configurations disclosed in Fig. 4d. The generated four apertures This general optical set-up images an object in plane 2001 across the objective 2002 and back focal aperture 2003 across this general module 8500. Module 8500 is followed by a tubelens 2004, which generates a self-interference between the contributing apertures. The array sensor is preferentially positioned in the focal plane of lens 2004.
[0065] Fig 8A, 8B and 8C demonstrate and visualize the resolution enhancement due to the SYNAPS principle. Fig. 8A shows the intensity distribution 1201 and the corresponding footprint of the point spread function in 1202 of a single aperture the typical point spreadfunction known from widefield microscopy. The intensity distribution 1203 is showing the achievable narrowing of the point spread function when using a "4 - aperture" configuration. Obviously this intensity distribution shows some sidelobes. Finally as shown in Fig. 8C the intensity distribution 1204 shows a point spread function (footprint) and its corresponding footprint in 1205. It is obvious that this is a strong improvement for the resolution due to the cumulant analysis is achieved. This resolution improvement can be achieved with a scanning as well as a widefield microscopy system. [0066] Fig 9 discloses the application of the SYNAPS principle for a widefield microscopy. The image shown in Fig. 9A is an average of 1000 widefield images. Fig. 9B shows a clear improvement for the resolution for this widefield setting due to the SynAps principle.
[0067] Fig. 10 shows a scanning microscope with 2021 the object plane, 2002 the objective,
2003 the backfocal-aperture or back-aperture in a classical 2f-configuration. The infinity space between the back-aperture 2003 and the tubelens 2004 contains an optical mean for scanning the object in plane 2021 and general beam splitting 2015 which can be realized as shown in Fig. 4A with a diffractive unit 450 or a refraction based configuration 4800 or the prism configuration in Fig. 4d but not limited to these configurations. The following optical mean is a tubelens 2004, which is forming an image in the image plane 2016. In this plane 2016 is also positioned a fast few element array detector or the entry port of a fibre bundle. If the point image overlaps over several detector elements as shown in 2011 or for the fibre bundle in 2012 the SYNAPS principle can be used for a strong resolution enhancement.
[0068] Fig. 11A describes a classical widefield microscope 2000, a so-called ICS-System (Infinity Color Corrected System). The objective 2002 and the tubelens 2004 are imaging the object plane 2001 into the image plane 2005. All optical means 2002, the aperture 2003 and
2004 are centered on the optical axis and are often forming a tubelens subassembly indicated as 2100. The object plane 2001 is in the front focal plane of the objective, whereas the aperture plane 2003 is in the back focal of this said objective 2002. For those skilled in the art this represent a telecentric system in the object space. A further subassembly is shown as 2200, which contains the tubelens 2004. Objective 2002 combined with the tubelens 2004 image the objectplane 2001 into the image plane 2005.
[0069] Fig. 11B is a general microscope 2400 containing the subassembly 2100 and the subassembly 2200 the optical means for imaging the object plane 2001 in the image plane 2005.. Between the objective subassembly 2100 and the tubelens subassembly 2200 is placed the SYNAPS module for splitting the entry aperture into several exit aperture and such realizing the SYNAPS principle, which allows to further increase the resolution. This general configuration applies to widefield microscopy, scanning microscopy and multiphoton microscopy but is not limited hereto.

Claims

1. Optical system comprising a synthetic aperture generating unit; said synthetic aperture generating unit being adapted for generating several apertures, which are correlated by interference for improving contrast and/or resolution based on the interference pattern generated by said apertures.
2. Optical system according to claim 1 furthermore comprising a processing unit which is adapted to carry out a cumulant analysis based on the interference pattern generated by said apertures.
3. Optical system according to claim 1 or 2 wherein said synthetic aperture generating unit comprises a refractive, diffractive or refractive-diffractive beam splitting mean.
4. Optical system according to claim 3 wherein said combination is made of at least two identical prisms, which are in contact at a common prism surface.
5. Optical system according to claim 4 wherein said two prisms are slightly displaced from each other, along said common lateral surface for introducing a definite optical path difference.
6. Optical system according to anyone of the previous claims which is adapted to use the fluorescence photon emission captured with a fast array detector or a fibre bundle linked to ultrafast single photon detectors for using this photon emission process in combination with the said synthetic aperture and a cumulant analysis for a resolution and contrast enhancement.
7. Optical system according to anyone of the previous claims for use as a microscope.
8. Method for improving the contrast and/or resolution of an optical system comprising the generation of synthetic apertures followed by a cumulant analysis of the interference pattern generated by said apertures.
9. Method according to claim 8 wherein said synthetic apertures are obtained with a 30-60- 90 degree prism combination.
10. Method according to claim 8 wherein said synthetic apertures are obtained with a 45-90- 45 degree prism combination.
PCT/IB2016/056861 2015-11-16 2016-11-15 Superresolved synthetic aperture microscope WO2017085618A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IB2015058836 2015-11-16
IBPCT/IB2015/058836 2015-11-16

Publications (1)

Publication Number Publication Date
WO2017085618A1 true WO2017085618A1 (en) 2017-05-26

Family

ID=57583387

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2016/056861 WO2017085618A1 (en) 2015-11-16 2016-11-15 Superresolved synthetic aperture microscope

Country Status (1)

Country Link
WO (1) WO2017085618A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109374133A (en) * 2018-09-29 2019-02-22 中国科学院光电技术研究所 Asymmetric spatial heterodyne spectrometer based on improved K star-ster prism
DE102020205849A1 (en) 2020-05-08 2021-11-11 Technische Universität Dresden Arrangement of optical elements for the formation of structural patterns
DE102024200909B3 (en) * 2024-01-31 2025-07-10 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein Beam splitter, beam splitter system and use of a beam splitter

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4061425A (en) * 1976-07-06 1977-12-06 Martin Marietta Corporation High resolution alignment interferometer
DE19624167A1 (en) * 1995-06-23 1997-01-16 Zeiss Carl Jena Gmbh Coherence biometry and tomography with dynamic coherent focus
WO2008008774A2 (en) * 2006-07-10 2008-01-17 The Board Of Trustees Of The University Of Illinois Interferometric synthetic aperture microscopy
US20100303386A1 (en) * 2009-06-02 2010-12-02 Enderlein Joerg Superresolution Optical Fluctuation Imaging (SOFI)
WO2012172440A1 (en) 2011-06-14 2012-12-20 Ecole Polytechnique Federale De Lausanne (Epfl) Cumulant microscopy
WO2015044819A1 (en) 2013-09-26 2015-04-02 Ecole Polytechnique Federale De Lausanne (Epfl) Optical prism for creating multiple views of an image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4061425A (en) * 1976-07-06 1977-12-06 Martin Marietta Corporation High resolution alignment interferometer
DE19624167A1 (en) * 1995-06-23 1997-01-16 Zeiss Carl Jena Gmbh Coherence biometry and tomography with dynamic coherent focus
WO2008008774A2 (en) * 2006-07-10 2008-01-17 The Board Of Trustees Of The University Of Illinois Interferometric synthetic aperture microscopy
US20100303386A1 (en) * 2009-06-02 2010-12-02 Enderlein Joerg Superresolution Optical Fluctuation Imaging (SOFI)
WO2012172440A1 (en) 2011-06-14 2012-12-20 Ecole Polytechnique Federale De Lausanne (Epfl) Cumulant microscopy
WO2015044819A1 (en) 2013-09-26 2015-04-02 Ecole Polytechnique Federale De Lausanne (Epfl) Optical prism for creating multiple views of an image

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109374133A (en) * 2018-09-29 2019-02-22 中国科学院光电技术研究所 Asymmetric spatial heterodyne spectrometer based on improved K star-ster prism
DE102020205849A1 (en) 2020-05-08 2021-11-11 Technische Universität Dresden Arrangement of optical elements for the formation of structural patterns
DE102024200909B3 (en) * 2024-01-31 2025-07-10 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung eingetragener Verein Beam splitter, beam splitter system and use of a beam splitter

Similar Documents

Publication Publication Date Title
US10401609B2 (en) Embedded pupil function recovery for fourier ptychographic imaging devices
US10419665B2 (en) Variable-illumination fourier ptychographic imaging devices, systems, and methods
US10591417B2 (en) Systems and methods for 4-D hyperspectral imaging
EP3394579B1 (en) Systems and methods for determining an identity of a probe in a target based on colors and locations of two or more fluorophores in the probe and in the target
JP5536650B2 (en) System and method for self-interfering fluorescence microscopy and associated computer-accessible media
US5311021A (en) Spectroscopic sampling accessory having dual measuring and viewing systems
CN107490562B (en) Ultra-high-speed three-dimensional refractive index image shooting and fluorescent structured light illuminating microscope system using wave surface shaper and using method thereof
US8558998B2 (en) Fourier domain sensing
CN108254909B (en) Optical microscope and method for recording images using an optical microscope
US8405059B2 (en) Method and apparatus for improving the resolution and/or sectioning ability of an imaging system
Martínez-Corral et al. The resolution challenge in 3D optical microscopy
JPWO2018070451A1 (en) Digital holographic microscope
US20220381695A1 (en) Focus scan type imaging device for imaging target object in sample that induces aberration
EP2649421A1 (en) Image mapped optical coherence tomography
EP3184992B1 (en) Raman spectroscopy microscope and method for observing raman scattering
JP2006518854A (en) Interferometric confocal microscope observation method incorporating a pinhole array beam splitter.
US8098374B2 (en) Highly sensitive spectroscopic unit
CN108344508A (en) Wide-spectrum-range asymmetric spatial heterodyne spectrometer
WO2020039283A2 (en) Fourier-transform hyperspectral imaging system
Wicker et al. Single-shot optical sectioning using polarization-coded structured illumination
WO2017085618A1 (en) Superresolved synthetic aperture microscope
JP2007286379A (en) Real-time confocal microscope using dispersive optics
JP7534226B2 (en) Systems and methods for improved axial resolution in microscopes using photoswitching and standing wave illumination techniques - Patents.com
RU2673784C1 (en) Two-component general track interferometer
CN120195857B (en) Dual confocal image scanning microscope based on turntable and imaging method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16815643

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16815643

Country of ref document: EP

Kind code of ref document: A1