WO2009059378A1 - Differential aberration correction microscopy (dac) - Google Patents

Abstract

This invention concerns the measurement of distances in the order of nanometers using fluorescence microscopy or other imaging techniques where the invention is applicable. In a first aspect the invention is a method for measuring the nanometer distances. In another aspect the invention is a method for data processing image data, and a further aspect is software for performing the method. In particular the invention involves exciting first reference probes in a sample and acquiring a first channel image showing the locations of the first reference probes in the sample. Exciting second reference probes, at known locations relative to the first probes, in a sample and acquiring a second channel image showing the locations of the second reference probes. Determining with nanometer accuracy the geometric center of first and second probes in the first and second channel images. Generating a set of displacement vectors from the measured location of each first reference probe in the first channel and the corresponding second reference probe in the second channel. Interpolating the displacement vectors across the sample to generate a deformation field across the image area. And, using the deformation field to correct the location of a third interrogation probe in the same sample as the reference probes.

Description

Title

Differential Aberration Correction Microscopy (DAC)

This application is related to Australian provisional application No. 2007906176, filed on 9 November 2007 the contents of which are incorporated herein by reference.

Technical Field

This invention concerns the measurement of distances in the order of nanometers using fluorescence microscopy or other imaging techniques where the invention is applicable. In a first aspect the invention is a method for measuring the nanometer distances. In another aspect the invention is a method for processing image data, and a further aspect is software for performing the method.

Background Art

A fluorescence microscope relies on fluorescence or phosphorescence rather than reflection and absorption. The specimen is generally labeled with a fluorescent molecule, or fluorophore, a type of "probe", and when it is illuminated with light of a wavelength that is absorbed by the probe, it then emits light having a different wavelength. The illuminating light is then separated from the much weaker emitted light by fluorescence filters. As a result, the weak fluorescent signal can be imaged with good signal to noise ratio.

Conventional microscopy is governed by the Abbe diffraction limit (or the Rayleigh criterion), which dictate the smallest distance that can be measured. This limit is approximately 0.3 micrometers for a fluorescence microscope.

Fluorescence Resonance Energy Transfer (FRET) is a different technique that can be used to measure much smaller distances, in the order of Angstroms to several nanometers. However, there is an unmet need to measure distances between the micrometer and nanometer ranges, for instance to investigate the conformation and function of macromolecular complexes and large proteins.

A number of investigators have succeeded in making measurements with a fluorescence mircoscope below the Abbe diffraction limit of 0.3 micrometers, using a range of complex procedures:

Reference [1] describes the measuring of nanometer changes in position of a single fluorophore probe.

Reference [2] describes the use of a piezo stage to measure nanometer distances between two fluorophores. This technique is dependant on the two probes being in the very centre of the field of view (where there are no optical aberrations). A lengthy scanning procedure then produces an image that can be used for nanometer distance measurements. References [3] , [4], and [5] describe other arrangements with similar drawbacks.

Reference [6] WO2004090617 describes a method for measuring small distances using a complicated photo-bleaching scheme. Similar schemes based on photo-activatable probes have been developed recently.

Reference [7] US2005048539-A1 describes the use of Fluorescence Resonance Energy Transfer (FRET).

Other techniques to improve on the optical resolution limit of microscopy include: deconvolution techniques, scanning near field optical microscopies (SNOM), and 4PI microscopy, see References [10, 1 1 and 12].

There have been a number of earlier attempts to measure chromatic aberrations and control them. Notably, Patwardham and Manders describe a confocal instrument with sequential laser excitation of multicolor beads that shows minimal crosstalk between channels and improved resolution; see Reference [8]. They essentially develop a procedure to align the microscope using fluorescence bead probes, a procedure that is still in use today. However, they have not developed a framework capable of nanometer resolution.

Other investigators have measured chromatic aberrations towards significantly improving deconvolution techniques by integrating this information in the algorithm to constrain the deconvolution process; see reference [9].

Reference [13] describes a technique for high resolution colocalization between two image channels where first, second and third reference emitter probes are used. The first emitter probes are imaged in a first channel, and the second emitter probes are imaged in a second channel. The third reference emitter probe is scanned using a piezo stage in a preliminary process to form a grid which is visible in both channels. The grid is then used to register the images from each channel to each other.

Disclosure of the Invention

The invention is a method suitable for measuring distances in the order of nanometers using fluorescence microscopy (and other imaging techniques for which the method is applicable). The method comprises the steps of:

Exciting first reference probes in a sample and acquiring a first channel image showing the locations of the first reference probes in the sample. Exciting second reference probes, at known locations (ideally, the same position) relative to the first probes, in a sample and acquiring a second channel image showing the locations of the second reference probes.

Determining with nanometer accuracy the geometric center of first and second probes in the first and second channel images. Generating a set of displacement vectors from the measured location of each first reference probe in the first channel and the corresponding second reference probe in the second channel.

Interpolating the displacement vectors across the sample to generate a deformation field across the image area.

Using the deformation field to correct the location of a third interrogation probe in the same sample as the reference probes.

The method is able to measure distances, accurate to the order of nanometers, between two nearby interrogation probes in a sample by subtracting the deformation field from the measured location of the interrogation probes to correct their location. The method is able to measure such distances over the whole field of view very quickly. The method can also be used to investigate the co-localisation of two interrogation probes rather than measuring their relative distance.

The instrument used may be a confocal microscope, a wide field fluorescent microscope, or a transmission optical microscope.

When the probe is an emitter, the mechanism of emission may be fluorescence, phosphorescence, chemiluminescence or any other types of luminescence, including radioactive emission.

The excitation of the probes may be a single photon process, or a multi-photon process. Excitation may be achieved using one or more lasers, or pulsed lasers.

The reference probes may be exactly co-localized.

The probes may be distinguished from each other using any property including:

Physical properties, such as by having distinctive emissions, including different wavelengths (different colours) and extinction coefficient. Chemo-physical characteristics such as absorption, polarization, lifetime, or the relative intensity between two fluorophore species.

The coherence of the various displacement vectors (automated outlier detection methods may be used in conjunction with this technique). Other measurable phenomena, for instance that might be measured using a nonzero fluorescence resonant energy transfer. Or,

The signal exploited for fitting the positions of single molecules may not be due to emission but to the absorption or depletion of some radiation.

First reference emitters may be arranged symmetrically around a single second reference emitter, such that the geometric centre of each are at the same position. For example, a quantum dot functionalised with avidin could be labelled with an excess of fluorescently labelled biotin to obtain such a probe.

The first and second reference emitters may comprise a short double stranded stretch of DNA tagged at the 5' ends with two different organic dyes. In this case the dye SYBR can be used as DNA-binding dye in order to distinguish an interrogation emitter from the reference emitters.

A reference emitter may comprise a short single strand of DNA labelled at position 5' with alexa 488, and two additional strands complementary to the first and second half of that strand, and labelled with Alexa 594 and Alexa 488, such that Alexa 594 is located approximation midway between the two Alexa 488 emitters by virtue of hybridisation. In this arrangement, a fluorescence resonance energy transfer equal to approximately 50% can distinguish the reference emitter from other emitters and whether the geometric centres of the first and second reference probes are exactly at the same positon.

The probes may comprise quantum dots, organic dyes, nano-diamonds or a mixture of different probes. The excitation of the first and second reference probes, and the image acquisitions, may take place sequentially or simultaneously.

Detection of the emitted light may be achieved using: a photodiode, a photodiode array or a CCD camera.

The precise locations of the probes may be determined by fitting the expected theoretical intensity profiles, such as the point spread function (PSF) to the image data.

Drift correction may be performed following the generation of the displacement vectors. The overall shift between the two images may be evaluated using a drift correction method. For example, imposing the condition that the vectorial sum of all displacement vectors for the reference probe is equal to zero after the correction, or by considering a parameterized deformation model and maximizing the correlation coefficient between the deformed image of the first channel and the second channel.

The deformation field may be derived using kernel based filtering methods, such as averaging. The estimation of the chromatic deformation field may be achieved using algorithms different from kernel based algorithms. The deformation field is able to correct for all aberrations of the microscope.

The deformation field may be estimated and used in 3-Dimensions to measure distances in 3-Dimensions.

Distance measurements or co-localisation measurements may be performed in time-lapse mode.

In another aspect the invention is a data processing method for measuring distances within an imaged sample in the order of nanometers; the method uses: First channel image data defining the locations of first reference probes. Second channel image data defining the locations of second reference probes, at known locations relative to the first probes. And,

Third channel image data defining the location of an interrogation probe. The method comprises the steps of: Determining with nanometer accuracy the geometric center of first and second reference probes in the first and second channel image data.

Generating a displacement vector from the measured location of each first reference probe in the first channel and the corresponding second reference probe in the second channel. Interpolating the displacement vectors across the sample to generate a deformation field across the image area.

Using the deformation field to correct the location of the third interrogation probe.

In a further aspect the invention is computer software for performing the method defined above.

Brief Description of the Drawings

An example of the invention will now be described with reference to the accompanying drawings, in which:

Fig. 1 is a diagram showing the general arrangement of an epi-fluorescence microscope.

Fig. 2 is a first channel inverted contrast image showing the locations of first emitter probes. Fig. 2(a) is a magnified part of the image of Fig. 2 showing two of the first emitter probes with their centres marked.

Fig. 3 is a second channel image showing the locations of second emitter probes. Fig. 3(a) is a magnified part of the image of Fig. 3 showing 3D representations segmented from images of the second emitter probes with their centres marked. Fig. 4 is an image of displacement vectors measured between the first and second channel images; the vectors are scaled x30 to increase visibility. Fig. 5 is an image of a deformation field interpolated from Fig. 4 and plotted to a grid; the vectors are scaled xlO to increase visibility.

Best Modes of the Invention

Referring first to Fig. 1 the general arrangement of a fluorescence microscope 10 involves a light source 12, and an excitation filter 14 to extract the wavelength 16 that will excite the fluorescent molecule. A dichroic beamsplitter 18 deflects the excitation wavelength down through the objective 20 to the specimen 22. The specimen absorbs the excitation wavelength and emits another wavelength 24 that passes back through the objective 20, through the beamsplitter 18, through an emission filter 26 to arrive at a detector 28.

In the following experiments the sample was observed using a BX61 upright Olympus microscope using a 6OX air objective. Excitation of the sample was achieved using an HBO 100 W lamp (Osram). Two different configurations of the microscope were used, with different filter sets, to obtain images from two difference types of fluorescent molecules (probes) colocated at the same locations in the specimen. The first filter set was suited for DAPI (model U-MWU2, Omega optical). The second filter set was suited for CY3 (model U-MWIG2, Omega optical).

Tetraspeck multicolor fluorescent beads (0.1 micrometer in diameter) from Invitrogen were vortexed and diluted 1/100 in ethanol. They were sonicated for 30 s and one microliter of the solution was placed to spread and dry on a slide. The sample was then covered using a coverslip and sealed with nail polish.

Images were acquired using a 12 bit, low noise, cooled CCD camera (Evolution QEi, Qimaging). In each of the microscope configurations, Image Pro (Media Cybernetics) was used to perform averaging over 5 frames to create an image; this procedure reduces the noise in the resulting image. Fig. 2 is an image obtained using a first configuration of the microscope with a filter set adapted to the dye DAPI. The insert Fig. 2(a) shows two individual probes 50 and 52, with their geometrical centres indicated by crosses. These centres were determined by fitting the expected point spread function (PSF) to the image data.

Fig. 3 is an image obtained using the second configuration of the microscope with a filter set adapted to the dye CY3. Again the geometrical centres of the probes is indicated by a cross, one of which is labelled 60. The insert Fig. 3(a) illustrates that exactly the same methodology can be used in 3D where the spots have been segmented using a standard segmentation algorithm, and precision of the order of 10 nm has been archived.

Because the instrument configuration is switched mechanically between the two configurations on this particular microscope, significant lateral shifts may occur between the two frames shown in Figs. 2 and 3.

Other sources of registration errors between the two images include drifts due to temperature changes, chromatic aberrations, optical drifts due to alignment mismatch and mechanical artifacts.

A displacement vector was generated from the measured location of each first reference probe in the first channel to the corresponding second reference probe in the second channel. To correct the displacement vectors for sources of drift, the displacement vectors were required to satisfy the requirement that the sum of all displacement vectors after drift correction should be equal to zero. The resulting recalculated displacement vectors, scaled x30 times to increase their visibility, are shown in Fig. 4. The vectors can be seen to converge towards the centre of the field following a regular pattern.

In this case the lateral shift, after drift correction, was found to amount to about three pixels at the boundary after drift correction. From the displacement vectors it is possible to calculate a deformation field across the entire image area that will correct for all sources optical aberrations in the image. The deformation field is explicitly calculated by interpolation. A simple kernel-based filter can be used for the interpolation, for instance where the kernel is a Gaussian function of a Half Width at Half Maximum (HWHM) equal to 200 pixels and a support equal to 300 pixels. Fig. 5 shows the interpolated deformation field plotted on a grid with the vectors scaled xlO times to increase visibility.

The beads (probes) from which the deformation field is calculated are called "reference probes". A third bead that emits light which is distinguishable from the reference probes, is called a "interrogation probe", and the technique can be used to measure distances between interrogation probes in the order of a few nanometers. The light emitted by the interrogation probes is at a different wavelength (although it could be distinguished in other ways, such as by polarization, lifetime, or extinction coefficient).

Ideally, the interrogation probes are distanced from the reference probes so that they do not interfere with ultrahigh precision localization of the individual point spread functions (PSF) of the interrogation probes. The measurement methodology involves generating the deformation field using two reference probes as described above. Then determining the corrected positions of the interrogation probe using the deformation field. Finally, measuring the distance between the interrogation probes.

Validation

In this example beads 1, 2, and 3 shown in Fig. 3 will each considered successively to consist of pairs of interrogation probes. Since these probes have been used as reference probes, it is necessary when considering each of them to be an interrogation probe, to obtain the deformation field based only all the other reference probes. As a result the processing described above is repeated three times.

The drift-corrected displacement vectors, from the deformation field, at positions 1, 2, and 3 in Fig. 3 were respectively equal to: (0.673, -0.0162), (0.318, -0.0874), and (0.295, 0.12), in pixel units,

where the first coordinate corresponds to horizontal displacements.

The deformation fields calculated for these positions were respectively equal to: (0.597,-0.087), (0.518, -0.059), and (0.293, 0.039), in pixel units.

This means that the error corrected measured distance was equal to: ((0.673-0.597)²+(-0.0162-0.087)²)^1/2 = 0.1039 pixels

0.2020 pixels, and 0.0810 pixels, respectively.

As the pixel size in this object space was equal to 0.1 micron, the precision of the method is about 10 nm in this example.

Although the invention has been described with reference to a particular example, it should be appreciated that it could be exemplified in many other forms and in combination with other features not mentioned above. For instance, even greater precision may be achieved by using better instrumentation. Theoretical considerations indicate that subnanometer precisions should be possible using more appropriate filter sets, better probes such as the Alexa dyes or quantum dots, and a more sensitive, lower noise camera. Additionally, a better isolation stage on the microscope would assist.

References

The following references are used to explain and describe the prior art and the invention. The contents of all these references are hereby incorporated by reference in their entirety.

[1] Van Oijen AM. Single-molecule studies of complex systems: the replisome. MoI Biosyst. 2007 Feb;3(2):l 17-25.

[2] Xavier Michaleta, Thilo D. Lacostea, Fabien Pinauda, Daniel S. Chemlaa,b, A. Paul, Alivisatosa,c, Shimon Weissa,b, Ultrahigh Resolution Multicolor Colocalization of Single Fluorescent Nanocrystals.

[3] Gordon MP, Ha T, Selvin PR. Single-molecule high-resolution imaging with photobleaching. Proc Natl Acad Sci U S A. 2004 Apr 27;101(17):6462-5. [4] Xiaohui Qu, David Wu, Laurens Mets, and Norbert F. Scherer. Nanometer-localized multiple single-molecule fluorescence microscopy. PNAS 2004 101 : 1 1298-1 1303 [5] Patent Number(s): US2002064789-A1 ; US6844150-B2; Title: Fluorescent species colocalization method in biological sample analysis, involves computing distance between geometric centers of point spread function excitation for two species. Weiss [6] WO2004090617-A2; US2004212799-A1; DE10325460-A1; Method for high- resolution, three-dimensional imaging of labeled structures in specimens uses substance which can be reversibly converted using radiation between states with different optical properties. Hell

[7] US2005048539-A1. Assessing distance between molecules by exposing samples having donor/ acceptor fluorophore labeled molecules to an excitation source, measuring fluorescence lifetime, calculating distance between fluorophores and resolving donor proportion. HYMAN B T, BEREZOVSKA O, BACSKAI B, et. Al [8] Patwrdham, A & Manders, E.M.M. (1996) Bioimaging 4, 17-24. [9] Scalettar, B.A., Swedlow, J.R., Sedat, J. W. & Agard, D.A. (1996) J. Microsc. 182, 50-60. [10] Huntington ST, Ladouceur F. Evanescent fields - direct measurement, modeling, and application. Microsc Res Tech. 2007 Mar;70(3): 181-5. [11] Lang MC, Engelhardt J, Hell SW. 4Pi microscopy with linear fluorescence excitation. Opt Lett. 2007 Feb 1;32(3):259-61.

[12] Shaevitz JW, Fletcher DA. Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function. J Opt Soc Am A Opt Image Sci Vis. 2007 Sep;24(9):2622-7.

[13] Churchman LS, Okten Z, Rock RS, Dawson JF, Spudich JA. Single molecule high- resolution colocalization of Cy3 and Cy5 attached to macromolecules measures intramolecular distances through time.

Claims

Claims

l . A method for measuring distances in the order of nanometers using fluorescence microscopy; the method comprising the steps of: exciting first reference probes in a sample and acquiring a first channel image showing the locations of the first reference probes in the sample; exciting second reference probes, at known locations relative to the first probes, in a sample and acquiring a second channel image showing the locations of the second reference probes; determining with nanometer accuracy the geometric center of first and second probes in the first and second channel images; generating a set of displacement vectors from the measured location of each first reference probe in the first channel and the corresponding second reference probe in the second channel; interpolating the displacement vectors across the sample to generate a deformation field across the image area; and, using the deformation field to correct the location of a third interrogation probe in the same sample as the reference probes.

2. A method according to claim 1 , wherein the instrument is a confocal microscope.

3. A method according to claim 1, wherein the instrument is a wide field fluorescent microscope.

4. A method according to claim 1 , wherein the instrument is an optical microscope.

5. A method according to any preceding claim, wherein the probe is an emitter.

6. A method according to claim 5, wherein the mechanism of emission is one of fluorescence, phosphorescence, chemiluminescence or radioactive emission.

7. A method according to claim 5 or 6, wherein the excitation of the probes is a single photon process.

8. A method according to claim 1, wherein excitation is achieved using lasers.

9. A method according to any preceding claim, wherein the reference probes are exactly co-localized.

10. A method according to any preceding claim, wherein the probes are distinguished from each other using any physical or chemical property.

11. A method according to claim 1 , wherein the signal exploited for fitting the positions of single molecules is due to the absorption or depletion of some radiation.

12. A method according to claim 1, wherein first reference emitters are arranged symmetrically around a single second reference emitter.

13. A method according to claim 1 , wherein the first and second reference emitters comprise a short double stranded stretch of DNA tagged at the 5' ends with two different organic dyes.

14. A method according to claim 1, wherein a reference emitter comprises a short single strand of DNA labelled at position 5' with alexa 488, and two additional strands complementary to the first and second half of that strand, and labelled with Alexa 594 and Alexa 488, such that Alexa 594 is located approximation midway between the two Alexa 488 emitters by virtue of hybridisation.

15. A method according to claim 1, wherein the probes comprise quantum dots, organic dyes, nano-diamonds or a mixture of different probes.

16. A method according to claim 1, wherein the precise locations of the probes is determined by fitting the expected theoretical intensity profiles to the image data.

17. A method according to claim 1, wherein drift correction is performed following the generation of the displacement vectors.

18. A method according to claim 1, wherein the deformation field is derived using kernel based filtering methods.

19. A data processing method for measuring distances within an imaged sample in the order of nanometers; the method uses: first channel image data defining the locations of first reference probes; second channel image data defining the locations of second reference probes, at known locations relative to the first probes; and, third channel image data defining the location of an interrogation probe; wherein the method comprises the steps of: determining with nanometer accuracy the geometric center of first and second reference probes in the first and second channel image data; generating a displacement vector from the measured location of each first reference probe in the first channel and the corresponding second reference probe in the second channel; interpolating the displacement vectors across the sample to generate a deformation field across the image area; and, using the deformation field to correct the location of the third interrogation probe.

20. Computer software comprising computer readable data on a computer readable medium for performing the method according to claim 19.