WO2007109861A1 - Super resolution microscopy - Google Patents

Super resolution microscopy Download PDF

Info

Publication number
WO2007109861A1
WO2007109861A1 PCT/AU2007/000407 AU2007000407W WO2007109861A1 WO 2007109861 A1 WO2007109861 A1 WO 2007109861A1 AU 2007000407 W AU2007000407 W AU 2007000407W WO 2007109861 A1 WO2007109861 A1 WO 2007109861A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
images
ensemble
standing wave
sample
Prior art date
Application number
PCT/AU2007/000407
Other languages
French (fr)
Inventor
Brad Littleton
Dennis Longstaff
Vassilios Sarafis
Original Assignee
The University Of Queensland
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2006901595A external-priority patent/AU2006901595A0/en
Application filed by The University Of Queensland filed Critical The University Of Queensland
Publication of WO2007109861A1 publication Critical patent/WO2007109861A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/06Means for illuminating specimens
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/16Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/18Arrangements with more than one light path, e.g. for comparing two specimens

Definitions

  • This invention relates to microscopy.
  • it relates to super- microscopy that achieves resolution below the classical diffraction limit.
  • the spatial resolution limit is commonly called the diffraction limit. It is known that with some a priori knowledge about an object it is possible to encode information from saturated channels of the imaging system into unsaturated ones, and thereby surpass the physical resolution limit in a desired dimension.
  • Laterally structured illumination microscopy achieves increased lateral resolution by sending different subsets of the high frequency information about the object through the system at different times.
  • Higher spatial frequencies in the object are aliased into the passband of the optical system by a carrier with a given phase, and a number of images at different phases can be post-processed to give an image with greater resolution.
  • the a priori information about the object is therefore the assumption that it changes slowly with respect to the data acquisition rate, and this allows spatial-frequency information to be encoded into the time domain.
  • This is a wide-field imaging technique that discards little of the light available from a scene, as compared to other (point-wise) techniques that exhibit lateral super-resolution, such as confocal microscopy.
  • the invention resides in a microscope comprising: a source of coherent radiation; optical elements directing the coherent radiation to establish a phase- variable transverse volumetric standing wave field at a sample; a lens collecting radiation emanating from the sample; a camera periodically capturing an ensemble of images from the lens; and a processor converting the ensemble of images to a super-resolution image.
  • the optical elements suitably include a variable delay line to adjust the phase of the standing wave field.
  • the images are coherent images resulting from reflection, transmission or coherent scattering.
  • the invention resides in a method of imaging a sample with resolution below the diffraction limit including the steps of: illuminating the sample with a phase-variable transverse volumetric standing wave; capturing an ensemble of images of radiation emanating from the sample at various phases of the standing wave; and processing the ensemble of images by: forming a first image by halving an average of the ensemble of images; forming a second image by multiplying each image in the ensemble of images by the respective standing wave field and calculating an average; and doubling the addition of the first image and the second image.
  • FIG 1 shows an apparatus for super microscopy below the diffraction limit
  • FIG 2 shows a sketch of a sample for demonstrating super microscopy
  • FIG 3 shows a sketch of images of the sample taken at different phases of the standing wave illumination pattern
  • FIG 4 shows a sketch of a cross-section through the sample at different frame numbers of the acquisition
  • FIG 5 shows a sketch of a super-resolved image of the sample
  • FIG 6 shows a comparison between cross-section measurements of the sample through super-resolved and diffraction-limited images
  • FIG 7 shows the effect of structured illumination on the spectrum of an image
  • FIG 8 is a flow chart of the general signal processing steps; and FIG 9 shows how greater than double resolution is obtainable.
  • FIG. 1 An apparatus for super-microscopy below the diffraction limit is depicted schematically in FIG 1.
  • the demonstration apparatus is indicative of an apparatus for super-microscopy according to the invention, however the specific elements will change to suit the particular application.
  • the beam 2 is split by beamsplitter 4 into a first beam 5 and a second beam 6 which counter-propagate at a sample 7 to produce a transverse standing wave field.
  • the first beam 5 is directed by mirror 8 and mirror 9 to illuminate the sample from a first direction.
  • the second beam 6 is directed by mirror 10 to piezo-modulated mirror 11 which directs the beam 6 at the sample from an opposite direction to the first beam 5. Due to space restrictions the sample was mounted on a glass block 12 and the beams 5, 6 were incident at the sample 7 from below at ⁇ 48° to the horizontal.
  • Refractive index matching oil was used between the block 12 and the sample 7, and between the sample 7 and an objective lens 13 of the microscope.
  • an index matching oil By using an index matching oil the first beam 5 and second beam 6 are coupled into the bulk of the sample block 12 to create a volumetric transverse standing wave rather than an evanescent transverse standing wave as in the prior art.
  • the illumination pattern wavelength is 320 nm
  • Super-resolution with this system therefore exceeds the classical resolution limit.
  • structured and non-structured illumination was performed with beams occupying the same input numerical aperture so the system has the same bandwidth in both cases, and the resolution in each case can therefore be directly compared. Images of the scene were captured via a CCD camera, and subsequently processed by computer.
  • the relative phase of the two beams 5, 6 is controlled via the position of the piezo-electrically mounted mirror 11 in the second beam path.
  • An arbitrary function generator HP33120A was used to control the piezo driver, and hence the phase of the illumination pattern.
  • a portion of the light from each beam was split off by beamsplitters 14, 15 and recombined by mirror 16 and beamsplitter 17 onto a photodiode 18, which could then be used to monitor the stability and phase of the system; although this arrangement required the assumption that the difference between the optical path lengths from the beamsplitters to the photodiode, and from the beamsplitters to the sample, remained fixed throughout the experiment.
  • the sample 7 was a test object made by etching a trench 20 (using a focused ion beam) through an opaque platinum film 21 that had been vapour deposited onto a glass coverslip 22.
  • a negative sketch of a scanned electron micrograph (SEM) of the object is shown in FIG 2.
  • the trench 20 was -170 nm wide, and etched all the way through the Pt film 21 , which was ⁇ 500 nm thick.
  • the width of the trench 20 was therefore less than the classical resolution limit of the system, which under ideal He- Ne illumination was around 240 nm (and for the illumination geometry used here was approximately 320 nm).
  • Images of the sample 7 were acquired at 25 frames/s while the piezo-mounted mirror 11 was driven in a sawtooth pattern with an amplitude of four times the standing wave wavelength.
  • the illumination pattern therefore went through four complete cycles for each leg of the sawtooth waveform.
  • Sketched images of the object corresponding to two different phases of the illumination are shown in FIG 3: the beams are incident from the top and bottom of the figures, and the interference of the illumination pattern wavefronts with the obliquely oriented trench can be seen to change with the phase of the standing wave.
  • FIG 4 shows how the cross-section through the sample changes with the frame number over a 10s acquisition.
  • the discontinuities near frames 120 and 240
  • the discontinuities are due to the change in direction of the sawtooth driving function.
  • the Fourier transform of a subset of the data of FIG 4 was used to determine the starting phase of the standing wave, and the direction of its wavevector (via the spatial frequency along the object. This approach is, however, not valid for an unknown object, in which case the starting phase can be found in post-processing by maximizing the integrated signal in the final image.
  • Each frame of the acquisition could then be associated with a phase of the illumination, with 28 frames spanning one cycle of the standing wave.
  • FIG 5 A sketch of the resulting super-resolved image is shown in FIG 5. This was formed by adding the S and T images calculated via equations (10), using as input raw images from which the background had been subtracted. The apparent width of the trench is much less than in the classically resolved images sketched in FIG 3 (as the standing wavevector is parallel to the y-axis, the image is super-resolved in the y-direction). The resolution improvement was further demonstrated by taking cross- sections of the image along the y-axis (and accounting for the change in peak position due to the object not being parallel to the x-axis). It is apparent from the mean cross-section shown in FIG 6 that the resolution is improved by approximately a factor of two over the diffraction limited image.
  • the imaging system shown in FIG 1 involves illuminating a specimen transversally with two opposed, collinearly polarised, coherent beams. This generates a volumetric standing wave along the illumination axis.
  • the high spatial frequency of the standing wave modulates the spatial frequencies in the scene, and high frequencies are "mixed down" to become low frequencies that are within the passband of the optical system.
  • By recording images at different phases of the standing wave it is possible to pass enough information through the optical system to completely specify the high frequency components of the object, up to the spatial frequency of the standing wave.
  • Orienting two sets of beams along each of the transverse directions would allow the technique to be applied in two dimensions, however for simplicity the invention is described in one dimension only with two opposing beams.
  • the phase of the standing wave is changed by moving the piezo-electrically mounted mirror (PZM) 11.
  • Laterally structured illumination can also be generated by illuminating through the objective lens although this limits the spatial carrier frequency to the bandwidth limit of the optics, as the numerical aperture (NA) of the system limits the beams' angles of incidence.
  • NA numerical aperture
  • the analysis is restricted to the case where the spatial carrier frequency equals the classical bandwidth in order to be able to draw direct comparisons between the super-resolution image and a simple equivalent system with twice the NA.
  • FIG 7(a) depicts conventional (axial) illumination.
  • the spectrum (solid line) is enveloped by the passband of the system (broken line).
  • the passband here is equivalent to the coherent transfer function with cutoff frequency f c .
  • FIG 7(b) depicts structured illumination that modulates the object and therefore shifts the object spectrum.
  • FIG 7(c) shows that an image decoded from the structured illumination spectrum has effectively twice the bandwidth of the conventional image.
  • O(m,n) is the Fourier transform of the object function
  • c(m,n) is the in-focus coherent transfer function (the CTF, which is equivalent to the pupil function of the system)
  • M is the magnification.
  • the illumination function is:
  • O(m, ⁇ ) cos( ⁇ t)[ ⁇ sxp(-i ⁇ 12) ⁇ (m + k/2 ⁇ ,n) + exp(i ⁇ 12) ⁇ (m ⁇ k/2 ⁇ , n)] ® A(m, ⁇ )
  • I(x,y, ⁇ ) 2F[A(m, n)c(-)]F[A(m, n)c(+)] + exp[-i( ⁇ - 2kx)]F[A(m, ⁇ )c(-)f
  • equation (2) From three or more images described by equation (1). If the phase of the illumination pattern is known, the I(x,y, ⁇ i) can be added with appropriate coefficients, p,-, to cause the oscillatory terms to cancel. To simplify the notation, we define S, T and U such that equation (1) becomes,
  • T X q, cos( ⁇ , - 2kx)(I(x, y, ⁇ ,) - 2S) (6)
  • the system tends to be sufficiently stable over a single acquisition that the relative phases are well known, as these are set by the voltage on the piezo mounted mirrors and not altered by drifts in the optical system. Between acquisitions, however, the interferometer can drift such that the starting phase of the standing wave is not known a priori. This can instead be accounted for by performing the processing steps with a range of different starting phases, and comparing the integrated intensity in each of the resulting images. The image with the largest integrated signal corresponds to the correct value of the standing wave starting phase, and is therefore the super- resolved image.
  • the frequency and direction of the carrier signal i.e. the standing wave
  • This can change across the image due to imaging aberrations and refraction by specimens.
  • the effect of aberrations on the standing wave parameters can be accounted for, to some degree, through post-processing.
  • One possible way to do this is by calibrating the field of view by observing scatter of the standing wave off an extended known object, such as a grating. If the grating and standing wave wavevectors are similar then the difference frequency of the two patterns is a low frequency Moire pattern that can be observed across the field of the view. Determining how the frequency and direction of the pattern changes across the image gives information on the apparent changes to the standing wavevector due to aberrations (though if aberrations are minimised through good optical design then it is expected that such a calibration will not be necessary).
  • the image processing steps are outlined briefly in FIG 8.
  • An ensemble of images are captured at a range of phases of the standing wave.
  • the phase of the standing wave is adjusted by the piezo-electrically mounted mirror 11 which forms a delay line as described earlier.
  • An average image is calculated from the ensemble of images and halved to give the 'image' referred to as S in equation (10).
  • each image is multiplied by the standing wave that applied at the time of capture of the image and these are averaged to give the 'image' T in equation (10).
  • the 'images' S and T are added and doubled according to equation (7) to give the super-resolved image.
  • FIG 9(a) depicts conventional (axial) illumination.
  • the spectrum (solid line) is enveloped by the passband of the system (broken line).
  • the passband here is equivalent to the coherent transfer function with cutoff frequency f c (the region of the spectrum within the passband is denoted in FIG 9 by a thicker line).
  • the standing wave has frequency k.
  • FIG 9(b) depicts structured illumination that modulates the object and therefore shifts the object spectrum by k. Compared to FIG 7 the illumination angle is closer to horizontal which therefore generates a shorter illumination pattern wavelength (higher spatial frequency).
  • the higher spatial frequency information from the wings of the spectrum is combined with the information from conventional illumination up to frequency k-f c (thickest line in FIG 9(a) and 9(c)), to obtain a resolution that is greater than twice the diffraction limit (>2f c ).
  • FIG 10. One method of signal processing to achieve higher resolution is depicted in FIG 10.
  • the process involves illuminating the scene with one beam blocked and recording an image (a static image as compared to the ensemble of images recorded with the standing wave generated by counter-propagating beams), doubling the amplitude of the image, computing the Fourier transform, and cropping the transform at frequency k-fc.
  • the inventor envisages two alternate schemes to complete the processing.
  • One option is to compute the inverse transform and add the result to the image, I, computed using the scheme explained by reference to FIG 8 (dash-dot lines in FIG 10).
  • the other option is to add the cropped transform to the Fourier Transform of I and then calculate the inverse transform of the result (dotted line in FIG 10).
  • the inventors envisage the apparatus and method having best effect for coherent imaging (reflection, transmission and coherent scattering), it is recognized that it can also be used for incoherent imaging (fluorescence) to achieve resolution greater than twice the diffraction limit.
  • the system transfer function is the incoherent optical transfer function (OTF), which is equal to the autocorrelation of the CTF, and which therefore decreases in amplitude towards the cutoff frequency. If unaccounted for, this leads to attenuation of low frequency components of the object when processing incoherent structured illumination images.
  • OTF incoherent optical transfer function

Landscapes

  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

A microscope is described that achieves resolution below the diffraction limit by establishing a phase-variable transverse volumetric standing wave field at a sample. An ensemble of images of radiation emanating from the sample is recorded and processed to produce a super- resolution image. The processing involves: forming a first image by halving an average of the ensemble of images; forming a second image by multiplying each image in the ensemble of images by the respective standing wave field; and calculating an average; and doubling the addition of the first image and the second image.

Description

SUPER RESOLUTION MICROSCOPY
This invention relates to microscopy. In particular it relates to super- microscopy that achieves resolution below the classical diffraction limit.
BACKGROUND TO THE INVENTION
For an imaging system with a given numerical aperture there is a physical limit to the temporal and spatial resolution that can be achieved. For a classical microscope, the spatial resolution limit is commonly called the diffraction limit. It is known that with some a priori knowledge about an object it is possible to encode information from saturated channels of the imaging system into unsaturated ones, and thereby surpass the physical resolution limit in a desired dimension.
Laterally structured illumination microscopy achieves increased lateral resolution by sending different subsets of the high frequency information about the object through the system at different times. Higher spatial frequencies in the object are aliased into the passband of the optical system by a carrier with a given phase, and a number of images at different phases can be post-processed to give an image with greater resolution. The a priori information about the object is therefore the assumption that it changes slowly with respect to the data acquisition rate, and this allows spatial-frequency information to be encoded into the time domain. This is a wide-field imaging technique that discards little of the light available from a scene, as compared to other (point-wise) techniques that exhibit lateral super-resolution, such as confocal microscopy.
Super-resolution microscopy via laterally structured illumination has thus far been applied to fluorescence microscopy, where the light emitted from the object is incoherent. [M G L Gustaffson (2000) Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy J. Microsc. 198(2):82-7 and R Heintzmann and C Cremer (1999) Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating Proc. SPIE 3568:185-95]. A method and apparatus appropriate for coherent illumination has not been developed. Furthermore, resolution better than double the diffraction limit has not been achieved.
Shemer et al [A Shemer, D Mendlovic, Z Zalevsky, J Garcia and P G Martinez (1999) Superresolving optical system with time multiplexing and computer decoding Appl. Opt. 38(35):7245-51] have described an approach to (non-microscopic) coherent super-resolution that requires moving gratings: one at the object, and one (which can be either real, or implemented in post-processing) at the detector. However, it is not feasible to make physical gratings fine enough to approach the diffraction limit in microscopic applications, and unless binary phase gratings (Dammann gratings) are used, these methods lead to unequal weighting of different regions of the output image spectrum.
Resolution below the diffraction limit has been described for imaging along the optical axis in United States patent number 5394268 assigned to Carnegie Mellon University. The technique uses axial standing waves to improve the resolution looking into a sample but does not improve resolution across the field of view.
Improved resolution across the field of view is described in United States patent number 6255642 (assigned to Massachusetts Institute of Technology). An evanescent standing wave is formed and an image is recorded. The evanescent standing wave can only image a very shallow portion of a sample at the edge of the sample volume and is therefore of limited usefulness.
OBJECT OF THE INVENTION
It is an object of the present invention to provide an apparatus and method to achieve super-resolution microscopy. Further objects will be evident from the following description.
DISCLOSURE OF THE INVENTION
In one form, although it need not be the only or indeed the broadest form, the invention resides in a microscope comprising: a source of coherent radiation; optical elements directing the coherent radiation to establish a phase- variable transverse volumetric standing wave field at a sample; a lens collecting radiation emanating from the sample; a camera periodically capturing an ensemble of images from the lens; and a processor converting the ensemble of images to a super-resolution image.
The optical elements suitably include a variable delay line to adjust the phase of the standing wave field. Preferably the images are coherent images resulting from reflection, transmission or coherent scattering.
In another form the invention resides in a method of imaging a sample with resolution below the diffraction limit including the steps of: illuminating the sample with a phase-variable transverse volumetric standing wave; capturing an ensemble of images of radiation emanating from the sample at various phases of the standing wave; and processing the ensemble of images by: forming a first image by halving an average of the ensemble of images; forming a second image by multiplying each image in the ensemble of images by the respective standing wave field and calculating an average; and doubling the addition of the first image and the second image. BRIEF DETAILS OF THE DRAWINGS
To assist in understanding the invention preferred embodiments will now be described with reference to the following figures in which : FIG 1 shows an apparatus for super microscopy below the diffraction limit;
FIG 2 shows a sketch of a sample for demonstrating super microscopy;
FIG 3 shows a sketch of images of the sample taken at different phases of the standing wave illumination pattern; FIG 4 shows a sketch of a cross-section through the sample at different frame numbers of the acquisition;
FIG 5 shows a sketch of a super-resolved image of the sample;
FIG 6 shows a comparison between cross-section measurements of the sample through super-resolved and diffraction-limited images; FIG 7 shows the effect of structured illumination on the spectrum of an image;
FIG 8 is a flow chart of the general signal processing steps; and FIG 9 shows how greater than double resolution is obtainable.
DETAILED DESCRIPTION OF THE DRAWINGS
In describing different embodiments of the present invention common reference numerals are used to describe like features.
An apparatus for super-microscopy below the diffraction limit is depicted schematically in FIG 1. The demonstration apparatus is indicative of an apparatus for super-microscopy according to the invention, however the specific elements will change to suit the particular application.
A He-Ne laser 1 generates a beam 2 (A=632nm) which is horizontally polarised by polariser 3. The beam 2 is split by beamsplitter 4 into a first beam 5 and a second beam 6 which counter-propagate at a sample 7 to produce a transverse standing wave field. The first beam 5 is directed by mirror 8 and mirror 9 to illuminate the sample from a first direction. The second beam 6 is directed by mirror 10 to piezo-modulated mirror 11 which directs the beam 6 at the sample from an opposite direction to the first beam 5. Due to space restrictions the sample was mounted on a glass block 12 and the beams 5, 6 were incident at the sample 7 from below at ~48° to the horizontal. Refractive index matching oil was used between the block 12 and the sample 7, and between the sample 7 and an objective lens 13 of the microscope. By using an index matching oil the first beam 5 and second beam 6 are coupled into the bulk of the sample block 12 to create a volumetric transverse standing wave rather than an evanescent transverse standing wave as in the prior art. A Leitz 100χ oil-immersion objective lens 13 with NA=1.3 was used, however with the geometry of the demonstration apparatus the cone of input light was less than the NA of the objective lens. The system's effective numerical aperture was therefore reduced, and was calculated to be equal to 1.0. With the illumination beams 5, 6 incident at ~48° the illumination pattern wavelength is 320 nm, and the theoretical super- resolution limit is -160 nm, which is less than the 240 nm ideal diffraction limited resolution for a system with NA= 1.3, illuminated with 632 nm light. Super-resolution with this system therefore exceeds the classical resolution limit. For the purpose of comparison, structured and non-structured illumination was performed with beams occupying the same input numerical aperture so the system has the same bandwidth in both cases, and the resolution in each case can therefore be directly compared. Images of the scene were captured via a CCD camera, and subsequently processed by computer. The relative phase of the two beams 5, 6 is controlled via the position of the piezo-electrically mounted mirror 11 in the second beam path. An arbitrary function generator (HP33120A) was used to control the piezo driver, and hence the phase of the illumination pattern. A portion of the light from each beam was split off by beamsplitters 14, 15 and recombined by mirror 16 and beamsplitter 17 onto a photodiode 18, which could then be used to monitor the stability and phase of the system; although this arrangement required the assumption that the difference between the optical path lengths from the beamsplitters to the photodiode, and from the beamsplitters to the sample, remained fixed throughout the experiment. Data was acquired much faster than the typical drift within the system, hence this was considered a valid assumption (and in any case the phase of the standing wave was calculated from the images during processing, rather than measured with the photodiode). The entire arrangement was shielded from air currents and constructed on a vibrational^ isolated optical table (not shown), to minimise noise and drift in the relative phases of the beams.
The sample 7 was a test object made by etching a trench 20 (using a focused ion beam) through an opaque platinum film 21 that had been vapour deposited onto a glass coverslip 22. A negative sketch of a scanned electron micrograph (SEM) of the object is shown in FIG 2. The trench 20 was -170 nm wide, and etched all the way through the Pt film 21 , which was ~500 nm thick. The width of the trench 20 was therefore less than the classical resolution limit of the system, which under ideal He- Ne illumination was around 240 nm (and for the illumination geometry used here was approximately 320 nm). In the original image small scale granularity can be seen outside the test specimen (where the film hasn't been etched) due to the morphology of the platinum film. These structures were not deep or wide enough to allow light to pass; hence background noise due to detector noise alone is evident in the original images but is removed from the sketch of FIG 3 (and subsequent figures). As the test object was essentially formed by transmission through a mask (rather than by scatter off an object), we observe no speckle effects in the images. It is worth noting that we expect speckle due to scatter from outside the focal plane to be reduced due to the optical sectioning effect of structured illumination that has been described by Neil [M A A Neil, R Juskaitis and T Wilson (1997) Method of obtaining optical sectioning by using structured light in a conventional microscope Opt. Lett. 22(24): 1905-7].
Images of the sample 7 were acquired at 25 frames/s while the piezo-mounted mirror 11 was driven in a sawtooth pattern with an amplitude of four times the standing wave wavelength. The illumination pattern therefore went through four complete cycles for each leg of the sawtooth waveform. Sketched images of the object corresponding to two different phases of the illumination (φ=12° and φ=192°) are shown in FIG 3: the beams are incident from the top and bottom of the figures, and the interference of the illumination pattern wavefronts with the obliquely oriented trench can be seen to change with the phase of the standing wave.
FIG 4 shows how the cross-section through the sample changes with the frame number over a 10s acquisition. The discontinuities (near frames 120 and 240) are due to the change in direction of the sawtooth driving function. In this case, the Fourier transform of a subset of the data of FIG 4 was used to determine the starting phase of the standing wave, and the direction of its wavevector (via the spatial frequency along the object. This approach is, however, not valid for an unknown object, in which case the starting phase can be found in post-processing by maximizing the integrated signal in the final image. Each frame of the acquisition could then be associated with a phase of the illumination, with 28 frames spanning one cycle of the standing wave. Summing the phasors corresponding to 28 consecutive frames resulted in a phasor with an amplitude of 0.05; it was therefore possible to cancel the oscillating terms in equations 4 and 9 to within 5%, simply by applying equations 10 over the 28 consecutive images (see equations 4, 9 and 10 below).
A sketch of the resulting super-resolved image is shown in FIG 5. This was formed by adding the S and T images calculated via equations (10), using as input raw images from which the background had been subtracted. The apparent width of the trench is much less than in the classically resolved images sketched in FIG 3 (as the standing wavevector is parallel to the y-axis, the image is super-resolved in the y-direction). The resolution improvement was further demonstrated by taking cross- sections of the image along the y-axis (and accounting for the change in peak position due to the object not being parallel to the x-axis). It is apparent from the mean cross-section shown in FIG 6 that the resolution is improved by approximately a factor of two over the diffraction limited image.
IMAGE PROCESSING
In order to describe the process for obtaining the image displayed in FIG 5 it is useful to consider a theoretical description of the super- resolved imaging system.
The imaging system shown in FIG 1 involves illuminating a specimen transversally with two opposed, collinearly polarised, coherent beams. This generates a volumetric standing wave along the illumination axis. The high spatial frequency of the standing wave modulates the spatial frequencies in the scene, and high frequencies are "mixed down" to become low frequencies that are within the passband of the optical system. By recording images at different phases of the standing wave it is possible to pass enough information through the optical system to completely specify the high frequency components of the object, up to the spatial frequency of the standing wave. Orienting two sets of beams along each of the transverse directions would allow the technique to be applied in two dimensions, however for simplicity the invention is described in one dimension only with two opposing beams. For the arrangement shown in FIG 1 , the phase of the standing wave is changed by moving the piezo-electrically mounted mirror (PZM) 11. Laterally structured illumination can also be generated by illuminating through the objective lens although this limits the spatial carrier frequency to the bandwidth limit of the optics, as the numerical aperture (NA) of the system limits the beams' angles of incidence. The essential points of the theory are the same in either case, however if the standing wave frequency exceeds the system bandwidth, low frequency information will be lost from the image and must be restored in the post-processing.
The analysis is restricted to the case where the spatial carrier frequency equals the classical bandwidth in order to be able to draw direct comparisons between the super-resolution image and a simple equivalent system with twice the NA.
A diagram of the effect of structured illumination in the frequency domain is shown in FIG 7. FIG 7(a) depicts conventional (axial) illumination. The spectrum (solid line) is enveloped by the passband of the system (broken line). The passband here is equivalent to the coherent transfer function with cutoff frequency fc. FIG 7(b) depicts structured illumination that modulates the object and therefore shifts the object spectrum. FIG 7(c) shows that an image decoded from the structured illumination spectrum has effectively twice the bandwidth of the conventional image.
It can be seen that higher object frequencies are within the bandwidth of the system once the object spectrum has been shifted by modulation with the standing wave. If the spectrum can be unfolded and reconstructed, the result is an image equivalent to a system with twice the bandwidth.
As the beams are temporally and spatially coherent over the field of view, light scattered into the microscope from different locations on the object have a definite phase relationship. The theory of coherent imaging must therefore be used, as field amplitudes must be added at the image plane before the intensity measured by the detector can be calculated. We begin with the theory for in-focus coherent imaging of a thin object (i.e. we neglect out-of-focal-plane contributions to the image), through a system for which the numerical aperture is not large. Deviation from the paraxial approximation at high NA does not change the following argument, however aberrations will complicate the processing method. The intensity, /, in the image plane due to a thin object at the focus, illuminated with coherent light is given by:
Figure imgf000011_0001
where O(m,n) is the Fourier transform of the object function, c(m,n) is the in-focus coherent transfer function (the CTF, which is equivalent to the pupil function of the system), and M is the magnification. I(xy) can also be
written as,
Figure imgf000011_0002
where F[..] denotes the Fourier transform, and o(x\γ) the object function. If the transmittance (or reflectance) of the object is given by
Figure imgf000011_0003
and the illumination field by
Figure imgf000011_0004
we have; o(x',/) = /(x\/)α(x',/)
For simplicity, we only treat the case of super-resolution in one dimension. For the case of illumination in the x,y-plane, parallel to the x- axis, with two counter-propagating coherent beams of unity amplitude, the illumination function is:
l(x, y) = cos(ωt - kx - φ/2) + cos(ωt + kx + φ/2)
= cos(ωt)[exp[-i(bc + φ/2)] + exp[i(kx + φ/ 2)]]
We have omitted the z component of / here as it merely gives rise to a global phase factor in the image amplitude, which goes to unity when the intensity is measured. Applying the convolution theorem then gives,
F[o(x',/)] = F[Z(X1,/)] ® F[O(X1,/)]
where <8> denotes the convolution operation. Letting uppercase symbols
denote the transform of lowercase ones, this becomes,
O(m, ή) = cos(ωt)[<sxp(-iφ 12)δ(m + k/2π,n) + exp(iφ 12)δ(m ~k/2π, n)] ® A(m, ή)
and / becomes,
J(x, y, φ) = |exp(-/ø / 2)F[_4(JB + k/2π,n)c(m, «)] + exp(iφ 12)F[A(m -kl2π,ή)c(m, ή)f
We have ignored the rapidly oscillating time dependent term as it is integrated at the detector over time t » Λlω. In the above equation, high spatial frequencies in the x-direction have now been aliased to low (difference) frequencies due to the shift of the A(m,n) in the Fourier domain. Applying the shift theorem, and using the Hermitian properties of real transforms, this expands to,
I(x,y, φ) = 2F[A(m, n)c(-)]F[A(m, n)c(+)] + exp[-i(φ - 2kx)]F[A(m, ή)c(-)f
+ exp[i(φ - 2kx)]F[A(m,n)c(+)]2
where c(±)=c(m ± k/2π,n). Noting that F[A(m,ή)c(+)] = F*[A(m,ή)c(-)] (if
the object is real, such that A is Hermitian), we then have,
I(x,y,φ) = 2F[A(m,ή)c(-)]E[A(m,ή)c(+)]
+ 2 cos(φ - 2kx)!Re{F[A(m, ή)c(+)f } (1 )
+ 2ήn(φ - 2kx)5m{F[A(miή)c(+)]2}
We can follow a similar procedure for the case of axial illumination through a system with double the aperture, i.e. with the coherent transfer function C2 = c(-) + c(+), and with l(x,y) = 1. By doing so we have effectively chosen k to be the same as the cutoff frequency of the CTF. In this case the intensity is:
J2x <X y) = |F[^O, n)c2 (m, ή)f
= 2¥[A(m, ή)c(-)]F[A(m, ή)c(+)] + 2yie{F[A(m, ή)c(+)f }
By taking a number of images I(xy) with different phase, φ, it is possible to fully recover the term <Αe{F[A(m,n)c(+)f} , and construct
f(Ii(x,y,φi)) = Iu(x,y) . In practice this is done by incrementing the standing wave pattern across the object using the piezo-electric mirror. It is possible to form k* exactly (in principle, at least) without explicitly taking account of the system transfer function because the coherent transfer function (CTF) has constant amplitude within the cutoff frequency
Building an image with super-resolution in a given direction requires determining equation (2) from three or more images described by equation (1). If the phase of the illumination pattern is known, the I(x,y, φi) can be added with appropriate coefficients, p,-, to cause the oscillatory terms to cancel. To simplify the notation, we define S, T and U such that equation (1) becomes,
I(x, y, φ) = 2S + 27 cos(φ - 2kx) + TXJ sin(^ - 2fcc) . (3)
We can find S by forming
S = ±∑p,I(x,y,φ,) (4)
where,
Figure imgf000014_0001
and the phases φ, are arbitrary and distinct. We can recover T through analogy with homodyne (or quadrature) detection, by multiplying the oscillatory terms in (3) by cosinusoids of the same frequency and phase. This results in a 'dc' term (the demodulated signal), and oscillatory terms with phase 2$ and frequency 4k for each quadrature. Again, these can be summed to zero with the proper coefficients. We can therefore find T by forming T = X q, cos(φ, - 2kx)(I(x, y,φ,) - 2S) (6)
with the coefficients q,- found by solving equations (5) with q' = p\2φ\, 2<k, 2φ£) (this adds an additional constraint on the φ, that the difference between any two of the three phases cannot be π). The image with double resolution (in one direction) is then simply:
J2x = 2(S + T) (7)
Note that it is not possible to remove the oscillatory terms by filtering in the Fourier domain (as is often done in homodyne/quadrature detection schemes), as the 4/c carrier frequency of the modulated terms corresponds to the superresolved bandwidth. Spectral information due to T is therefore mixed with information due to the 4/c modulated terms right up to the synthetic bandlimit of the system. Simply attenuating higher frequency components is consequently not sufficient to separate the signals.
For the special case of the phases being chosen to be 2^/3 apart, the coefficients become,
p, = q, = 1/3, (i = 1, 2,3)
and the expression
∑ q, costø - 2kx) =∑ q, cos(2^, - 4kx) = 0 (8)
is satisfied. Terms oscillating at both 2/c and 4/c will then simultaneously cancel and equation (6) simplifies to:
T = ∑q, costø -2kx)I(x,y,φt) . (9) This is a particular case of the general result that if we use Λ/>3 phases that uniformly span the unit circle, then;
p, = q, = \/N, (i = UJV)
and equations (4), (7) and (8) are satisfied. The equations for S and T are then;
Figure imgf000016_0001
(10)
Figure imgf000016_0002
It is worth noting that the oscillatory terms also tend to zero if many images with random phases are used, i.e.:
1 N lim Y l(x,yΛ) = S w→ 2N tr
(11)
1 N lim— ∑costø -2kx)I(x,y,φ,) = T
with random φ,-. However, these expressions converge rather slowly; numerical simulations have indicated that under ideal conditions (without noise, and with accurately known φ,- and k) it is necessary to average more than 100 images before the computed S and T resemble the correct functions. Nevertheless, equations (10) imply that errors in S and T due to random errors in φ,- and k will tend to zero as the number of images N is increased.
In order to extract the signal from the encoded data using the techniques described above it is necessary to accurately know the absolute phase φ\ of the standing waves. The system tends to be sufficiently stable over a single acquisition that the relative phases are well known, as these are set by the voltage on the piezo mounted mirrors and not altered by drifts in the optical system. Between acquisitions, however, the interferometer can drift such that the starting phase of the standing wave is not known a priori. This can instead be accounted for by performing the processing steps with a range of different starting phases, and comparing the integrated intensity in each of the resulting images. The image with the largest integrated signal corresponds to the correct value of the standing wave starting phase, and is therefore the super- resolved image. Furthermore, it is necessary to accurately know the frequency and direction of the carrier signal (i.e. the standing wave). This can change across the image due to imaging aberrations and refraction by specimens. The effect of aberrations on the standing wave parameters can be accounted for, to some degree, through post-processing. One possible way to do this is by calibrating the field of view by observing scatter of the standing wave off an extended known object, such as a grating. If the grating and standing wave wavevectors are similar then the difference frequency of the two patterns is a low frequency Moire pattern that can be observed across the field of the view. Determining how the frequency and direction of the pattern changes across the image gives information on the apparent changes to the standing wavevector due to aberrations (though if aberrations are minimised through good optical design then it is expected that such a calibration will not be necessary).
The image processing steps are outlined briefly in FIG 8. An ensemble of images are captured at a range of phases of the standing wave. The phase of the standing wave is adjusted by the piezo-electrically mounted mirror 11 which forms a delay line as described earlier. An average image is calculated from the ensemble of images and halved to give the 'image' referred to as S in equation (10). Simultaneously each image is multiplied by the standing wave that applied at the time of capture of the image and these are averaged to give the 'image' T in equation (10). The 'images' S and T are added and doubled according to equation (7) to give the super-resolved image.
HIGHER RESOLUTION
The foregoing explanation and example achieves a super- resolution double the diffraction limit. The inventor envisages even greater resolution being achievable as depicted in FIG 9. FIG 9(a) depicts conventional (axial) illumination. The spectrum (solid line) is enveloped by the passband of the system (broken line). The passband here is equivalent to the coherent transfer function with cutoff frequency fc (the region of the spectrum within the passband is denoted in FIG 9 by a thicker line). The standing wave has frequency k. FIG 9(b) depicts structured illumination that modulates the object and therefore shifts the object spectrum by k. Compared to FIG 7 the illumination angle is closer to horizontal which therefore generates a shorter illumination pattern wavelength (higher spatial frequency). As shown in FIG 9(b) and FIG 9(c), the higher spatial frequency information from the wings of the spectrum is combined with the information from conventional illumination up to frequency k-fc (thickest line in FIG 9(a) and 9(c)), to obtain a resolution that is greater than twice the diffraction limit (>2fc).
One method of signal processing to achieve higher resolution is depicted in FIG 10. The process involves illuminating the scene with one beam blocked and recording an image (a static image as compared to the ensemble of images recorded with the standing wave generated by counter-propagating beams), doubling the amplitude of the image, computing the Fourier transform, and cropping the transform at frequency k-fc. The inventor envisages two alternate schemes to complete the processing. One option is to compute the inverse transform and add the result to the image, I, computed using the scheme explained by reference to FIG 8 (dash-dot lines in FIG 10). The other option is to add the cropped transform to the Fourier Transform of I and then calculate the inverse transform of the result (dotted line in FIG 10).
INCOHERENT IMAGING
Although the inventors envisage the apparatus and method having best effect for coherent imaging (reflection, transmission and coherent scattering), it is recognized that it can also be used for incoherent imaging (fluorescence) to achieve resolution greater than twice the diffraction limit. For the case of incoherent imaging, the system transfer function is the incoherent optical transfer function (OTF), which is equal to the autocorrelation of the CTF, and which therefore decreases in amplitude towards the cutoff frequency. If unaccounted for, this leads to attenuation of low frequency components of the object when processing incoherent structured illumination images.
Other methods that do not require directly multiplying the data by the carrier can be used to recover a modulated signal, however they typically require finding the norm or square of processed quantities, and therefore assume the signal is non-negative. Such techniques are not applicable here, as T (and U) in equation (3) can, in general, be negative. The other methods are however applicable in the case of incoherent imaging. CONCLUSION
An expression for a coherent structured illumination image has been derived, and a processing scheme devised to allow construction of a super-resolved image. It has been verified experimentally that the resolution of the apparatus is improved by a factor of two over a diffraction-limited image. Greater resolution is possible. To facilitate ease of description an example is given of imaging in one dimension however persons skilled in the art will appreciate that the apparatus and method can be easily extended to image in two dimensions with a further pair of counter-propagating beams to establish two perpendicular transverse volumetric standing waves. Furthermore, the example has been given of imaging in a transmission mode but again persons skilled in the art will appreciate that a scatter mode is also encompassed by the description.
Throughout the specification the aim has been to describe the invention without limiting the invention to any particular combination of alternate features.

Claims

1. A microscope comprising: a source of coherent radiation; optical elements directing the coherent radiation to establish a phase- variable transverse volumetric standing wave field at a sample; a lens collecting radiation emanating from the sample; a camera periodically capturing an ensemble of images from the lens; and a processor converting the ensemble of images to a super-resolution image.
2. The microscope of claim 1 wherein the optical elements include a variable delay line to adjust the phase of the standing wave field.
3. The microscope of claim 2 wherein the variable delay line comprises a piezo-modulated mirror.
4. The microscope of claim 2 wherein the variable delay line is driven with a sawtooth pattern.
5. The microscope of claim 1 further comprising means for monitoring the phase of the standing wave field.
6. The microscope of claim 1 wherein the images are coherent images resulting from reflection, transmission or coherent scattering.
7. The microscope of claim 1 wherein the images are incoherent images resulting from fluorescence or incoherent scattering.
8. The microscope of claim 1 wherein the source of coherent radiation is a laser.
9. The microscope of claim 1 wherein the processor performs the steps of: forming a first image by halving an average of the ensemble of images; forming a second image by multiplying each image in the ensemble of images by the respective standing wave field and calculating an average; and doubling the addition of the first image and the second image.
10. A method of imaging a sample with resolution below the diffraction limit including the steps of: illuminating the sample with a phase-variable transverse volumetric standing wave; capturing an ensemble of images of radiation emanating from the sample at various phases of the standing wave; and processing the ensemble of images by: forming a first image by halving an average of the ensemble of images; forming a second image by multiplying each image in the ensemble of images by the respective standing wave field and calculating an average; and doubling the addition of the first image and the second image.
11. The method of claim 10 wherein the transverse volumetric standing wave is formed by counter-propagating coherent optical beams.
12. The method of claim 11 wherein a phase relationship between the beams is controlled by a variable delay line.
13. The method of claim 10 wherein there are at least three images in the ensemble of images.
14. The method of claim 13 wherein each image in the ensemble of images is recorded with a different phase relationship between the beams such that the phases uniformly span a unit circle.
15. The method of claim 10 wherein there are about 100 images in the ensemble of images and each image in the ensemble of images is recorded with a random phase relationship between the beams.
16. The method of claim 10 further including the steps of: illuminating the sample with a coherent optical beam; capturing a static image of radiation emanating from the sample; and processing the static image with the ensemble of images.
17. The method of claim 16 wherein the transverse volumetric standing wave is formed by splitting the coherent optical beam and counter- propagating the split coherent optical beams.
18. The method of claim 16 further including the steps of: doubling the amplitude of the static image; computing a Fourier transform; and cropping the transformed image.
19. The method of claim 18 further including the steps of: computing an inverse Fourier transform of the cropped image; and adding the resultant image to the image of claim 10.
20. The method of claim 18 further including the steps of: adding the cropped transformed image to the image of claim 10; and computing an inverse transform of the resultant image.
PCT/AU2007/000407 2006-03-29 2007-03-29 Super resolution microscopy WO2007109861A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2006901595A AU2006901595A0 (en) 2006-03-29 Super resolution microscopy
AU2006901595 2006-03-29

Publications (1)

Publication Number Publication Date
WO2007109861A1 true WO2007109861A1 (en) 2007-10-04

Family

ID=38540732

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2007/000407 WO2007109861A1 (en) 2006-03-29 2007-03-29 Super resolution microscopy

Country Status (1)

Country Link
WO (1) WO2007109861A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2317362A1 (en) * 2009-10-28 2011-05-04 Carl Zeiss MicroImaging GmbH Microscopic method and microscope with improved resolution
CN102985866A (en) * 2010-06-23 2013-03-20 浜松光子学株式会社 Image generation device
US20130120563A1 (en) * 2010-06-23 2013-05-16 Hamamatsu Photonics K.K. Image generation device
CN110515192A (en) * 2019-09-17 2019-11-29 苏州睿仟科技有限公司 The fast automatic scanning imaging system of super-resolution and method based on water immersion objective
CN113544572A (en) * 2019-03-07 2021-10-22 Z思快尔有限公司 Spatial encoding system, decoding system, imaging system and method thereof

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4621911A (en) * 1985-03-12 1986-11-11 Carnegie-Mellon University Standing wave luminescence microscopy
US5394268A (en) * 1993-02-05 1995-02-28 Carnegie Mellon University Field synthesis and optical subsectioning for standing wave microscopy

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4621911A (en) * 1985-03-12 1986-11-11 Carnegie-Mellon University Standing wave luminescence microscopy
US5394268A (en) * 1993-02-05 1995-02-28 Carnegie Mellon University Field synthesis and optical subsectioning for standing wave microscopy

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
FREIMANN ET AL.: "Development of a standing-wave fluorescence microscope with high nodal plane flatness", JOURNAL OF MICROSCOPY, vol. 187, no. 3, 1997, pages 193 - 200 *
LITTLETON ET AL.: "Coherent super-resolution microscopy via laterally structured illumination", MICRON., vol. 38, no. 2, 2007, pages 150 - 157, XP005802321 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2317362A1 (en) * 2009-10-28 2011-05-04 Carl Zeiss MicroImaging GmbH Microscopic method and microscope with improved resolution
US8705172B2 (en) 2009-10-28 2014-04-22 Carl Zeiss Microscopy Gmbh Microscopy method and microscope with enhanced resolution
CN102985866A (en) * 2010-06-23 2013-03-20 浜松光子学株式会社 Image generation device
US20130100283A1 (en) * 2010-06-23 2013-04-25 Hamamatsu Photonics K.K. Image generation device
US20130120563A1 (en) * 2010-06-23 2013-05-16 Hamamatsu Photonics K.K. Image generation device
CN113544572A (en) * 2019-03-07 2021-10-22 Z思快尔有限公司 Spatial encoding system, decoding system, imaging system and method thereof
CN110515192A (en) * 2019-09-17 2019-11-29 苏州睿仟科技有限公司 The fast automatic scanning imaging system of super-resolution and method based on water immersion objective
CN110515192B (en) * 2019-09-17 2024-04-30 苏州睿仟科技有限公司 Super-resolution rapid automatic scanning imaging system and method based on water immersion objective lens

Similar Documents

Publication Publication Date Title
US8019136B2 (en) Optical sectioning microscopy
US8558998B2 (en) Fourier domain sensing
US9998658B2 (en) Variable-illumination fourier ptychographic imaging devices, systems, and methods
US8570650B2 (en) Method and system for fast three-dimensional structured-illumination-microscopy imaging
US8184279B2 (en) Fourier domain sensing
US6376818B1 (en) Microscopy imaging apparatus and method
US10310246B2 (en) Converter, illuminator, and light sheet fluorescence microscope
EP0842497B1 (en) Imaging measurement system
US8841591B2 (en) Grating-enhanced optical imaging
JP6033798B2 (en) System and method for illumination phase control in fluorescence microscopy
US20090219607A1 (en) Method and apparatus for enhanced resolution microscopy of living biological nanostructures
Wicker et al. Single-shot optical sectioning using polarization-coded structured illumination
US9194811B1 (en) Apparatus and methods for improving defect detection sensitivity
KR20050119672A (en) Apparatus and method for joint measurement of fields of scattered/reflected or transmitted orthogonally polarized beams by an object in interferometry
JPWO2006109448A1 (en) Image creation method and microscope apparatus
WO2021155363A1 (en) Method and apparatus for high performance wide field photothermal infrared spectroscopy and imaging
JP5085608B2 (en) Wide-field super-resolution optical microscope using a spatial light modulator
WO2007109861A1 (en) Super resolution microscopy
Littleton et al. Coherent super-resolution microscopy via laterally structured illumination
US11947098B2 (en) Multi-focal light-sheet structured illumination fluorescence microscopy system
CN107923735B (en) Method and device for deducing the topography of an object surface
WO2007121175A2 (en) High-resolution optical imager
CN110515192B (en) Super-resolution rapid automatic scanning imaging system and method based on water immersion objective lens
Jackson et al. Demonstration of Interference Pattern Structured Illumination Imaging
CN115711866B (en) Quantitative phase contrast layer analytical microscopy device and method based on annular scanning illumination

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07718654

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07718654

Country of ref document: EP

Kind code of ref document: A1