NL2012169C2 - Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy. - Google Patents
Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy. Download PDFInfo
- Publication number
- NL2012169C2 NL2012169C2 NL2012169A NL2012169A NL2012169C2 NL 2012169 C2 NL2012169 C2 NL 2012169C2 NL 2012169 A NL2012169 A NL 2012169A NL 2012169 A NL2012169 A NL 2012169A NL 2012169 C2 NL2012169 C2 NL 2012169C2
- Authority
- NL
- Netherlands
- Prior art keywords
- localizations
- img
- emitters
- determining
- spatial correlation
- Prior art date
Links
- 230000004807 localization Effects 0.000 title claims description 138
- 238000000386 microscopy Methods 0.000 title description 29
- 238000000034 method Methods 0.000 claims description 64
- 230000006870 function Effects 0.000 claims description 54
- 238000012545 processing Methods 0.000 claims description 45
- 238000004061 bleaching Methods 0.000 claims description 32
- 230000001186 cumulative effect Effects 0.000 claims description 16
- 238000009826 distribution Methods 0.000 claims description 16
- 230000015654 memory Effects 0.000 claims description 15
- 230000000694 effects Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 12
- 238000005314 correlation function Methods 0.000 claims description 7
- 230000006399 behavior Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 5
- 230000004913 activation Effects 0.000 description 15
- 238000001994 activation Methods 0.000 description 15
- 230000004397 blinking Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 12
- 230000003287 optical effect Effects 0.000 description 12
- 238000003860 storage Methods 0.000 description 12
- 230000027455 binding Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 238000003384 imaging method Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 6
- 239000000758 substrate Substances 0.000 description 6
- 239000000975 dye Substances 0.000 description 5
- 238000002372 labelling Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 4
- 239000007850 fluorescent dye Substances 0.000 description 3
- 238000004445 quantitative analysis Methods 0.000 description 3
- 108010032595 Antibody Binding Sites Proteins 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000007621 cluster analysis Methods 0.000 description 2
- 238000010219 correlation analysis Methods 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000001000 micrograph Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 230000002186 photoactivation Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000010869 super-resolution microscopy Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 239000012114 Alexa Fluor 647 Substances 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004624 confocal microscopy Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000007865 diluting Methods 0.000 description 1
- 102000034287 fluorescent proteins Human genes 0.000 description 1
- 108091006047 fluorescent proteins Proteins 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000003834 intracellular effect Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 229920002521 macromolecule Polymers 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000009870 specific binding Effects 0.000 description 1
- 210000004895 subcellular structure Anatomy 0.000 description 1
- 238000010857 super resolution fluorescence microscopy Methods 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 230000004304 visual acuity Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/16—Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/58—Optics for apodization or superresolution; Optical synthetic aperture systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Engineering & Computer Science (AREA)
- Optics & Photonics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Description
NL20342-Vi/td
Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy
Field of the invention
The invention relates to determining the average number of localizations per emitter in localization microscopy and/or and the average number of emitters associated with one site, and, in particular, though not exclusively, to methods and systems for determining the average number of localizations per emitter and/or the average number of emitters associated with one site, an image-processing module using said method, a localization microscopy system comprising such image-processing module and a computer program product for implementing such method.
Background of the invention
The invention relates to the field of microscopy, in particular to the field of super-resolution fluorescence microscopy. In ordinary (optical) microscopy the resolving power is limited to an amount on the order of the ratio between the wavelength of the light that is used in the image formation and the numerical aperture of the microscope. Different techniques for circumventing this so-called diffraction limit ("super-resolution methods") have emerged over the last 10-15 years. An important class of superresolution microscopy techniques is the family of localization microscopy techniques, such as Photo-Activation Localization Microscopy (PALM, Betzig et al. "Imaging Intracellular Fluorescent Proteins at Nanometer Resolution", Science 313: 1643-1645, 2006) and STochastic Optical Reconstruction Microscopy (STORM, Rust et al., "Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)",
Nature Methods 3: 793-795, 2006).
In these techniques the sample of interest is labeled with a set of fluorescent molecules which can be stochastically switched between a light emitting state ("on" state) and a dark state ("off"-state) such that at any instant in time only a sparse subset of all fluorophores is in the on-state. This has the consequence that the individual fluorophores can be individually identified in the camera image. The position of the individual emitters in the plane imaged onto the camera can then be determined with a precision, which is typically on the order of 10 nm depending on photon count, about 10-20 times smaller than the diffraction limit. Repeating this process, i.e. recording many time frames of such sparse images and analysing all frames for the emitter locations, finally gives an image of the sample with details on the order of the localization precision rather than on the order of the diffraction limit.
Typically, fluorescent emitters are bound to the structure of interest by primary and/or secondary antibody labelling to ensure specific labeling via the antibody. On one primary/secondary antibody there are typically a few (~5) binding sites for fluorescent dyes. This results that often many (5-20) fluorescent dye molecules are bound to one binding site (epitope) accessible to an antibody. This stoichiometry is very hard to analyse experimentally as well as to control precisely in the sample preparation phase. The emitters within one epitope or attached to one anti-body are in general not resolvable as their mutual distances are below the localization precision, which is typically 10 nm. The distance between epitopes can vary from 1-100 nm depending on the structure of interest and labelling strategy.
In all of the existing embodiments of this superresolution technology, the same single fluorescent emitter may contribute many times to the final super-resolution image (even in PALM). A single emitter can go through many on/off cycles during the acquisition process (e.g. Annibale et al., "Identification of clustering artifacts in photoactivated localization microscopy." Nature Methods, 8: 527-528, 2011 or Dempsey et al. "Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging"
Nature Methods, 8: 1027-1031, 2011). Consequently, the reconstructed final super-resolution image shows the density of observed localization events, which does not give a trustworthy representation of the more relevant actual density of emitters or density of epitopes. This is different from traditional imaging (wide-field and confocal microscopy) where there is a known linear relationship between the recorded intensity and the density of emitters, but also to other type of super-resolution imaging such as Stimulated Depletion microscopy (STED). Quantitative knowledge of the density of emitters or density of epitopes is very much desired for studying the sample next to structural details alone. One could obtain this quantitative information if the number of times an emitter is localized would be known. A number of methods have been disclosed that deal with this problem (e.g. Annibale et al. "Quantitative photo activated localization microscopy: Unraveling the effects of photoblinking" PLoS ONE, 6:e22678, 2011; Veatch et al. "Correlation functions quantify super-resolution images and estimate apparent clustering due to over-counting" PloS ONE, 7:e31457, 2012; Lee et al. "Counting single photoactivatable fluorescent molecules by photoactivated localization microscopy (PALM). Proceedings of the National Academy of Sciences USA, 109:17436-17441, 2012; Nieuwenhuizen et al. "Measuring image resolution in optical nanoscopy" Nature Methods, 10:557-562, 2013) .
These methods can be divided into three categories: 1) calibration based techniques (Annibale et al.), 2) Pair-correlation techniques (Veatch et al.) and 3) techniques that employ statistical blinking models. Methods in the first category make use of a calibration measurement that is acquired before the actual acquisition but that is supposed to represent the actual measurement as much as possible. In this prior experiment the blinking statistics, namely the on and off-times of the fluorescent emitters are measured. This is done typically by diluting the dyes sufficiently such that single emitters are visible in a wide-field acquisition and subsequent analyses of a time-trace of these emitters. Methods in the second category employ post-processing on the set of localized emitters positions such that they cluster emitters location via spatial correlation functions. However, a-prior knowledge about the imaged structure is needed for this method. Methods in the third category employ a photo-physical model on the switching kinetics between the on and off-states of the emitters. This prior knowledge is combined with the evaluation of spatial correlations to estimate the number of emitters from the localizations in the case of STORM type of super-resolution imaging.
The actual counting technology comprised by the three categories of methods is confronted with different problems. Methods in the first category rely on the assumption that the calibration measurement is representative for the actual measurement. This is problematic as for each dye and experimental condition such as illumination power and buffer conditions the switching times can vary, which in turn compromises the certainty of the outcome. For real-time count estimation during acquisition this requires that many calibration experiments are performed prior to the experiment and stored online. Methods in the second category rely on prior knowledge or assumptions on the spatial structure of the imaged sample. If the actual structure does not conform to this assumption, the estimated count of emitters is biased. Methods in the third category rely on a photo-physical switching model. If this model does not describe the actual blinking kinetics the estimate of the number of emitters will be not trustworthy. Currently for STORM-type experiments only a two-state model (on/off) is employed. This neglects the important effect of photo-bleaching, i.e. an irreversible light-induced chemical reaction to a non-fluorescent state. In addition, the uncertainty of estimation of all methods is not well characterized.
Currently there is no method available in the prior art that may be used to reliable determine, or at least reliable estimate, the average number of localizations per emitter and/or the number of emitters associated with one anti-body binding site or epitope without either a calibration experiment or prior knowledge on the imaged structure. Hence, there is a need in the art for improved methods and system that allows such quantitative analysis in localization microscopy without any prior knowledge of the spatial sample and/or photo-physics of the emitters.
In this disclosure the term "site" defines a spatial domain with the extend of the localization precision of the localization of single emitters. Typically the size of a "site" is smaller than the distance between epitopes and larger than the distance of fluorescent emitters on one antibody and an epitope itself.
Summary of the invention
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit," "module" or "system." Functions described in this disclosure may be implemented as an algorithm executed by a microprocessor of a computer. Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied, e.g., stored, thereon.
Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non- exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber, cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java(TM), Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor, in particular a microprocessor or central processing unit (CPU), of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer, other programmable data processing apparatus, or other devices create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks .
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It is an objective of the invention to reduce or eliminate at least one of the drawbacks known in the prior art. In particular it is an object of the invention to provide a method for measuring the average number of emitters contributing to a localization microscopy image, thus calibrating the linear relationship between the actual emitter density and the visualized intensity derived from the density of localizations. Additionally, it is an object of the invention to use localization microscopy with reversibly switchable fluorophores as a quantitative imaging technique by measuring the average number of activation events per site as a function of time from the build-up of spatial image correlations during image acquisition.
In an aspect the invention may relate to a computer-implemented method for enabling to determine the average number of localizations per site and/or the average number of emitters per site using a localization microscope wherein said localization microscope is adapted to generate one or more super-resolution images of emitters in a sample.
In an embodiment, said method may comprise: measuring the cumulative number of localizations in a super-resolution image captured by said localization microscope and fitting the measured cumulative number of localizations with an exponential function for determining a bleaching rate kbi; and, determining a spatial correlation parameter Q(t) on the basis of measured localizations in at least one of said one or more super-resolution images and fitting the measured spatial correlation parameter Q(t) with a spatial correlation parameter function Q ( kbi, M«>, μ; t) derived from a photo-physical blinking model for modelling the photo-physical behaviour of said emitters, including the effect of photo-switching and photo-bleaching, for determining the asymptotic number of localizations per emitter Moo and the stoichiometry parameter μ.
In an embodiment, the method may comprise determining the average number of emitters per site on the basis of said stoichiometry parameter μ. In another embodiment, the method may comprise determining the average number of localizations per site <M> on the basis of the fitted values of kbi, Moo and μ.
Hence, the invention is based on an analysis method of the spatial correlations in a super-resolution image that do not arise from the sample structure itself but rather from having multiple localizations per site. These latter correlations can be characterized by a single parameter Q, which can be measured as a function of time. This measured correlation parameter Q and the cumulative number of localizations as a function of time can be fitted using a photo-physical blinking model including the effects of switching and photo-bleaching. The fitted model may allow both the determination, or at least an accurate estimation, of the average number of localizations per site as well as a statistical parameter μ indicating the number of emitters from the same site (stoichiometry) may be obtained.
The method according to the invention enables quantitative analysis in localization microscopy without any prior knowledge of the spatial sample and/or photo-physics of the emitters.
In an embodiment, said exponential function may have the form y=a(1-exp (-bt)) .
In an embodiment, said spatial correlation parameter function Q (kbi, M«>, μ; t) may be given by:
In an embodiment, said stoichiometry parameter μ may be given by:
wherein S is the number of emitters per site.
In an embodiment, the average number of localizations per site <M> may be given by:
The derivation of the spatial correlation function Q for multiple emitters per binding site on the basis of a photo-physical blinking model for modelling the photo-physical behaviour of said emitters, including the effect of photoswitching and photo-bleaching, is not trivial and allows determination of valuable quantitative information of the sample, including the average number of localizations per emitter and the number of emitters associated with one site.
The spatial correlation parameter Q as a function of t is determined by three parameters: the effective bleaching rate kbi, the asymptotic value of the number of activations per label Μ- and the stoichiometry parameter μ, which may be easily determined on the basis of information that is derivable from super-resolution images of localizations.
In an embodiment, measuring a spatial correlation parameter Q as a function of time t on the basis of localizations in at least one of said one or more superresolution images may comprise: determining a set of localizations by measuring localizations in one or more images over a predetermined time; dividing the set of localizations in at least two statistically independent sets fi and f2; determining a spatial correlation as a function of the q by computing:
wherein is q the spatial frequency and the caret Λ denoting the Fourier transformation of f(q); and, determining Q on the basis of said spatial correlation.
In an embodiment, S = 1 and μ = 0 so that the spatial correlation parameter function Q(t) only depends on the bleaching rate kbi and the asymptotic number of localizations per molecule Moo.
In an embodiment, a site may comprise one or two emitters and wherein the probability that a site comprises one emitters may be 1-p and the probability that a site comprises two emitters may be p so that <S>=l+p and μ = 2p/(1+p) and so that a one-to-one correspondence between μ and <S> is provided.
In an embodiment S may follow a Poisson-distribution μ=<3> so that both parameters M~ and μ may be determined on the basis of said measured spatial correlation parameter Q(t).
In a further aspect, the invention may relate to an image processing module for use in a localization microscope, wherein said image processing module may be configured for determining the average number of localizations per site and/or the average number of emitters per site of localizations and wherein said localization microscope may be adapted to generate one or more super-resolution images of emitters in a sample.
In an embodiment, said image processing module further being configured for: measuring the cumulative number of localizations in a super-resolution image captured by said localization microscope and fitting the measured cumulative number of localizations with an exponential function, preferably an exponential function of the form y=a(1-exp (-bt)), for determining a bleaching rate kbi; determining a spatial correlation parameter Q(t) on the basis of measured localizations in at least one of said one or more superresolution images and fitting the measured spatial correlation parameter Q(t) with a spatial correlation parameter function Q( kbi, Μ™, μ;t) derived from a photo-physical blinking model for modelling the photo-physical behaviour of said emitters, including the effect of photo-switching and photo-bleaching, for determining the asymptotic number of localizations per emitter Moo and the stoichiometry parameter μ.
In an embodiment, said image processing module may be configured for determining the average number of emitters per site on the basis of said stoichiometry parameter μ.
In an embodiment, said image processing module may be configured for determining the average number of localizations per site <M> on the basis of the fitted values of kbi, Moo and μ.
In an embodiment, said spatial correlation parameter function Q (kbi, Μ,», μ; t) may be given by:
wherein μ is given by:
wherein S is the number of emitters per site; and, wherein the average number of localizations per site <M> may be given by:
In an further embodiment, wherein measuring a spatial correlation parameter Q as a function of time t on the basis of localizations in at least one of said one or more superresolution images by said image-processing module may comprise: determining a set of localizations by measuring localizations in one or more images over a predetermined time; dividing the set of localizations in at least two statistically independent sets fi and f2; determining a spatial correlation as a function of the q by computing:
wherein is q the spatial frequency and the caret Λ denoting the Fourier transformation of f(q); and, determining Q on the basis of said spatial correlation.
In another aspect, the invention may relate to the use of the image-processing module as described above in a microscopy system, preferably a localization microscopy system.
In yet another aspect, the invention may relate to a microscopy system comprising an image-processing module according as described above.
The invention may also relate to a computer program product, comprising software code portions configured for, when run in the memory of a computer, executing the method steps according to any of the method steps described above.
The invention will be further illustrated with reference to the attached drawings, which schematically will show embodiments according to the invention. It will be understood that the invention is not in any way restricted to these specific embodiments.
Brief description of the drawings
Fig. 1 depicts a microscopy system using an emitter counter module according to an embodiment of the invention.
Fig. 2 depicts a high-level schematic of a process for forming an image of a sample using localization microscopy;
Fig. 3 depicts the scaled correlation function as a function of the spatial frequency q;
Fig. 4 depicts the experimental validation of the activation-bleaching switching model according to an embodiment of the invention;
Fig. 5 depicts histograms at different points in time of the number of localizations per cluster and the predicted probability distribution of the used photo-physical blinking model;
Fig. 6 is a block diagram illustrating an exemplary data processing system that may be used in systems and methods .
Detailed description
Fig. 1 depicts a microscopy system using an emitter counter module according to an embodiment of the invention. In particular, Fig. 1 depicts a schematic of a system 100 that may be used for super-resolution microscopy and in particular for localization microscopy. The microscopy system may relate to any type of localization microscopy system including the Photo-Activation Localization Microscopy (PALM) system and STochastic Optical Reconstruction Microscopy (STORM) system.
The system may comprise a microscope objective 106 comprising an optical lens system over a substrate holder (not shown) comprising a substrate 102. A sample 104 positioned on the substrate may comprise a biological or molecular structure that is selectively labeled with one or more emitters 103, typically fluorescent molecules such as fluorescent dyes of fluorophores, which can be stochastically switched between a light emitting and dark state such that at any instant in time a sparse subset of all emitters is in the on state. An emitter may be positioned in the sample at a location (x,y,z), wherein x,y are the coordinates in the plane of the substrate and the axial coordinate z is the coordinate normal to the plane of the substrate.
The optical lens system of the microscope object may be configured to focus light from one or more light sources 108,110 onto the sample and to image light originating from an emitter onto a camera 118. The first light source may be a switching light source for exposing the sample with light while the second light source may be used to expose the sample with (laser) light for exciting the emitters. In another embodiment, one switching light source may be used that both exposes the sample and excites the emitters. Semi-transparent mirrors may be used for reflecting light from the first and second light source to the microscope objective and transmitting light originating from an emitter to the camera.
In case localization of emitters in the sample is required, the emitted light originating from excited emitters is imaged by the optical system of the microscope objective and one or more further optical elements (not shown) onto the image plane of a digital camera 118. Δη image-acquisition and processing module 126 (in short an image-processing module) on a computer 120 that is connected to the camera may instruct the camera to take multiple images (a movie) of the randomly activated light-emitting emitters. The image-processing module may record the images and process the images in order to render an image of all emitters, wherein individual emitters are imaged with a resolution down to the localization precision of typically 10 nm.
Fig. 2 depicts a high-level schematic of a process for forming an image of a sample using localization microscopy. In this process (that may be executed by the image-processing module connected to the camera of the microscope), a sequence of images 202 is taken, wherein each image comprises a subsets of light-emitting emitters (step 202). Because only a (small) subset of the whole set of emitters is emitting, the individual light-emitting emitters are visible in each image. These images may be subsequently processed by the image-processing module (step 204), wherein for each image the individual locations 206i-4 of emitting emitters may be localized. Repeating this process for a large number of images may render precise localization of all emitters. In this way a super-resolution image of localized emitters may be obtained that has details on the order of the localization precision rather than the diffraction limit. As one or more emitters are selectively attached to specific binding sites (so-called epitopes) of the underlying structure (e.g. a macro-molecule, DNA, subcellular structure etc.), the shape of the labelled structure may become visible in the emitter localization image as can be seen in the picture of Fig. 2 that is obtained on the basis of the STORM imaging technique .
The image-processing module may be further configured to execute a quantitative analysis method on the spatial correlations of the imaged emitters in the super-resolution image that do not arise from the sample structure itself but rather from having multiple localizations, due to repeated activations of the same emitter pertaining to a site and/or having multiple independent emitters per. The spatial correlations of the localizations in the super-resolution image may be characterized by a single correlation parameter Q(t), which can be measured as a function of time. As will be discussed hereunder in detail, the spatial correlation parameter Q that may be measured on the basis of a superresolution image and the cumulative number of localizations as a function of time may be fitted with a photo-physical blinking model in which the effects of switching and photo-bleaching are taken into account. In this way, a reliable determination, or at least a reliable estimation, of the average number of localizations per site may be obtained as well as a statistical parameter μ that provides an indication of S, the number of emitters associated with the same site (the stoichiometry).
The photo-physical blinking model including the effects of switching and photo-bleaching may be based on the switching between three states, the on-state, the off-state, and the bleached state. The switching between the on-states and off-states is modelled using the Poisson distribution:
where r = kactt wherein t represents time and kact the activation rate, which is related to the lifetimes of the on-states and off-states by l/kact = ion + i0ff ·
The photo-bleaching is governed by a geometric distribution, namely the probability for bleaching at the m-th switching cycle is:
where b is the probability for bleaching during one cycle. Intuitively, for small time scales the statistics will be close to the activation dominated Poisson-model, whereas for large times it will be close to the bleaching dominated geometric distribution. For intermediate times t, the probability for m activation cycles is the sum of two terms.
The first is the product of the probability pact of having m switching cycles and the probability (1 - b)m that the emitter has m not bleached in the m switching cycles. The second term is the product of the probability pbim of bleaching during the m-th switching cycle and the probability of having at least m switching cycles. In mathematical terms (for m > 1) :
For m = 0 bleaching does not play a role, so the probability is then given by the Poisson term only:
Further, it may be verified that:
so that conservation of probability is satisfied.
The probability distribution of the number of activation cycles m is equivalent to the distribution of the minimum of two random variables mPOisson and mgeometric, where mpoisson is Poisson distributed with expectation value r and mgeometric follows a geometric distribution with expectation value 1/b. The moments of this probability distribution may be calculated from the moment-generating function:
The moments follow from the derivatives of this function at a = 0:
giving the spatial correlation parameter Q as a function of time :
If the bleaching rate is defined as kbi = bkac then the results for the average number of activations and for the correlation parameter Q for a single label (emitter) per site may be written as:
The effect of labeling stoichiometry, i.e. multiple labels (emitters) per state, may be determined as follows. Suppose there are K anti-body binding sites with Si (i = l, 2,..., K) fluorescent labels per anti-body which have M±j activations (j=l, 2,..., Si) . The number of activations per binding site is then given by:
The statistics of the number of labels per site are considered to be independent of the site and has moments <S> and <S2>. Furthermore, the statistics of the number of activations of each label are considered to be independent of label and binding site and gives rise to moments according to the three-state model:
for all i and j and with M- = kact/kbi· The spatial correlation parameter Q determined from the spatial correlation analysis is given by:
with: and:
Combining all results gives the following analytical expression for the spatial correlation parameter Q:
with:
Here μ represents the stoichiometry parameter, i.e. a statistical parameter related to S, the number of emitters associated with the same site. The second term of the above expression for Q is now compared to the expression of Q when considering the statistics of a single label. The derivation of the spatial correlation function Q for multiple emitters per binding site is not trivial and - as will be described hereunder in more detail - allows determination of valuable quantitative information of the sample, including the average number of localizations per emitter and the number of emitters associated with one site.
Hence, from the above, it follows that, there are three parameters that determine the spatial correlation parameter Q as a function of t: the effective bleaching rate kbi, the asymptotic value of the number of activations per label M~ and the stoichiometry parameter μ. The expected total number of activations is given by:
These parameters may be obtained by analysis of one or more super-resolution images of localized emitters (a localization image) that are obtained using a localization microscope as described with reference to Fig. 1 and 2. The parameters obtained from the analysis may then be used to determine the average number of localizations per emitter and the number of emitters associated with one binding site.
In a first step a set of localizations may be determined by capturing a movie (i.e. multiple images) of the localizations up to a time point t. Then, the set of localizations that is acquired up to a time point t is divided into at least two statistically independent sets fi and f2. In an embodiment, division into two sets may be realized by dividing the acquisition time series into blocks of 500 frames and randomly assigning these blocks to the two image halves. From the sets fi and f2, a spatial correlation may be computed as :
with q the spatial frequency and the caret Λ denoting the Fourier transformation. The spatial correlation is multiplied with a factor exp(4n2q2o2) where σ is the average localization uncertainty. Preferably, a suitably weighted sum of factors exp(4n2q2o2) over the distribution of different σ values may be used. The resulting curve as a function of q comprises a plateau as shown in Fig. 3. The value of the plateau of the spatial correlation is equal to the experimentally measured Q, i.e. the (spatial) image correlation parameter arising from having multiple localizations per emitter.
As already discussed above in detail, the photophysical model with three states (on, off, bleached) and one label (emitter) per site predicts the relation between Q and the average number of times a fluorescent emitter switches <M> as :
wherein kbi is the effective rate constant. From this model the effective rate constant kbi may be found from a fit to the cumulative number of localization as a function of time in the form y=a(1-exp (-bt) ), where the fit of b gives the effective rate constant kbi. The cumulative number of localization as a function of time may be easily determined by the image-processing module determined during formation of a superresolution image of localized emitters.
Having this rate constant, the parameter M~ may be obtained from the experimentally measured Q as a function of t as described above. Then, <M> (the average number switching cycles per emitter) may be determined on the basis of the measured spatial correlation parameter Q. Dividing the total, cumulative number of localizations by <M> gives the desired average number of emitters contributing to the image.
The above-mentioned analytical relation between Q and the average number of times a fluorescent emitter switches <M> only holds for one emitter per site. As already described above in detail, in case of multiple emitters per site, the analytical expression for the spatial correlation parameter function Q ( kbi,Μ«,,μ;t) derived from the photo-statistical blinking model is given by:
wherein μ is a number characterizing the statistics of S, i.e. the number of labels per site (the number of emitters with spatial separation less than the localization uncertainty site) (stoichiometry):
Hence, the average number of emitters per binding site may be obtained from the stoichiometry statistics parameter μ. In an embodiment, (the most simple case) that there is only one dye per epitope, so that S = 1 and μ = 0, and only the bleaching rate kbi and the asymptotic number of localizations per molecule Moo appear as parameters. In another embodiment, if there are known to be either one or two dyes (with probability 1-p and p, respectively) then <S>=l+p and μ = 2p/(l+p), giving a one-to-one correspondence between μ and <S>. In yet another embodiment, S may follow a Poisson-distribution μ=<Ξ> so that both parameters M~ and <S>may be estimated from the measured curve Q(t).
Hence, from the above it follows that both the average number of localizations per emitter and the number of emitters contributing to the localization microscopy image can be obtained on the basis of a three-state model (on, off and bleached state) where the on-off switching is characterized by an activation rate kact, and the photo-bleaching by a bleaching rate kbi and an analysis of the spatial correlations in a super-resolution image that do arise from multiple localizations per site. The analysis process may be performed by the image-processing module of a localization microscope wherein the image-processing module is configured to execute the following steps: - measuring the cumulative number of localizations in a super-resolution image determined by a localization microscope and fitting the measured cumulative number of localizations with an exponential function, preferably in the form of y=a(1-exp (-bt), for determining a bleaching rate kbi; - measuring a spatial correlation parameter Q as a function of time t for at least part of said localizations in said super-resolution image and fitting the measured spatial correlation parameter Q with the spatial correlation parameter Q derived from a photo-physical blinking model that includes the effects of photo-switching and photo-bleaching, for determining the asymptotic number of localizations per molecule M«, and the stoichiometry parameter μ; - determining the average number of emitters per site on the basis of said stoichiometry parameter μ; and/or, - determining the average number of localizations per binding site <M> on the basis of the fitted values of kbi Moo and μ.
The average number of emitters per site may be determined on the basis of μ if the labelling strategy is chosen such that there is an predetermined relationship between the mean and the variance of the distribution of the number of emitters per binding site.
Fig. 4 depicts the experimental validation of the activation-bleaching switching model according to an embodiment of the invention. The model was validated by experiments on isolated (sparsely distributed) clusters of DNA oligomers labelled with single Alexa Fluor 647 dyes on a glass substrate as shown in Fig. 4(A). Clearly recognizable isolated clusters may provide a ground truth for the distribution of localizations per emitter. First order switching kinetics is confirmed by an exponential on-time and off-time distribution giving τ0η = 27 ms and x0ff = 26 s.
Fig. 4(B) depicts the fitting of the cumulative number of localizations with a single exponential with bleaching rate kbi of 4.8><10~3/s. These data show that the parameter Q from the spatial correlation analysis may be well fitted with the expression for Q giving rise to Mm = 10.8 and with the ground truth value obtained from the cluster analysis .
The prediction for the expected number of localizations per emitter according to the expression of Q agrees well with the ground truth value obtained from the cluster analysis, thus validating the final step of the model. The measured τ0η and x0ff lead to an activation rate kact = 3.9xl0~2/s, which is agreement with the value kact = 5.2><10“2/s obtained from the fitted bleaching rate and asymptotic number of localizations per cluster.
Fig. 5 depicts histograms at different points in time of the number of localizations per cluster and the predicted probability distribution of the used photo-physical blinking model showing that the measured number of localizations match the model with the estimated parameters quite accurately.
The counting method according to the invention assumes constant and uniform rates, which may not apply for varying illumination power or for the depth dependent intensity in TIRF-imaging. These effects may be compensated for by recording the power as a function of time and inclusion of time dependent switching rates into the model.
The counting method according to the invention assumes that three states are sufficient for an effective description of the switching and bleaching behaviour of the emitters. In particular multiple long-lived dark states may alter the relation between the average number of localizations per emitter and the spatial correlation parameter Q.
Fig. 6 is a block diagram illustrating an exemplary data processing system that may be used in systems and methods as described with reference to Fig. 1-5. Data processing system 600 may include at least one processor 602 coupled to memory elements 604 through a system bus 606. As such, the data processing system may store program code within memory elements 604. Further, processor 602 may execute the program code accessed from memory elements 604 via system bus 606. In one aspect, data processing system may be implemented as a computer that is suitable for storing and/or executing program code. It should be appreciated, however, that data processing system 600 may be implemented in the form of any system including a processor and memory that is capable of performing the functions described within this specification.
Memory elements 604 may include one or more physical memory devices such as, for example, local memory 608 and one or more bulk storage devices 610. Local memory may refer to random access memory or other non-persistent memory device(s) generally used during actual execution of the program code. A bulk storage device may be implemented as a hard drive or other persistent data storage device. The processing system 600 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the number of times program code must be retrieved from bulk storage device 610 during execution.
Input/output (I/O) devices depicted as input device 612 and output device 614 optionally can be coupled to the data processing system. Examples of input device may include, but are not limited to, for example, a keyboard, a pointing device such as a mouse, or the like. Examples of output device may include, but are not limited to, for example, a monitor or display, speakers, or the like. Input device and/or output device may be coupled to data processing system either directly or through intervening I/O controllers. A network adapter 616 may also be coupled to data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and/or remote storage devices through intervening private or public networks. The network adapter may comprise a data receiver for receiving data that is transmitted by said systems, devices and/or networks to said data and a data transmitter for transmitting data to said systems, devices and/or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapter that may be used with data processing system 650.
As pictured in FIG. 6, memory elements 604 may store an application 618. It should be appreciated that data processing system 600 may further execute an operating system (not shown) that can facilitate execution of the application. Application, being implemented in the form of executable program code, can be executed by data processing system 600, e.g., by processor 602. Responsive to executing application, data processing system may be configured to perform one or more operations to be described herein in further detail.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Claims (14)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| NL2012169A NL2012169C2 (en) | 2014-01-30 | 2014-01-30 | Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy. |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| NL2012169 | 2014-01-30 | ||
| NL2012169A NL2012169C2 (en) | 2014-01-30 | 2014-01-30 | Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy. |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| NL2012169C2 true NL2012169C2 (en) | 2015-08-06 |
Family
ID=50483444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| NL2012169A NL2012169C2 (en) | 2014-01-30 | 2014-01-30 | Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy. |
Country Status (1)
| Country | Link |
|---|---|
| NL (1) | NL2012169C2 (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6259524B1 (en) * | 1997-04-25 | 2001-07-10 | The University Of Amsterdam | Photobleachable luminescent layers for calibration and standardization in optical microscopy |
| US20090067458A1 (en) * | 2007-09-12 | 2009-03-12 | Na Ji | Nonlinear imaging using passive pulse splitters and related technologies |
| US20100303386A1 (en) * | 2009-06-02 | 2010-12-02 | Enderlein Joerg | Superresolution Optical Fluctuation Imaging (SOFI) |
| US20120313012A1 (en) * | 2010-02-23 | 2012-12-13 | Selvin Paul R | Photobleaching and Intermittency Localization Microscopy |
-
2014
- 2014-01-30 NL NL2012169A patent/NL2012169C2/en not_active IP Right Cessation
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6259524B1 (en) * | 1997-04-25 | 2001-07-10 | The University Of Amsterdam | Photobleachable luminescent layers for calibration and standardization in optical microscopy |
| US20090067458A1 (en) * | 2007-09-12 | 2009-03-12 | Na Ji | Nonlinear imaging using passive pulse splitters and related technologies |
| US20100303386A1 (en) * | 2009-06-02 | 2010-12-02 | Enderlein Joerg | Superresolution Optical Fluctuation Imaging (SOFI) |
| US20120313012A1 (en) * | 2010-02-23 | 2012-12-13 | Selvin Paul R | Photobleaching and Intermittency Localization Microscopy |
Non-Patent Citations (3)
| Title |
|---|
| D. T. BURNETTE ET AL: "Bleaching/blinking assisted localization microscopy for superresolution imaging using standard fluorescent molecules", PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES, vol. 108, no. 52, 13 December 2011 (2011-12-13), pages 21081 - 21086, XP055141041, ISSN: 0027-8424, DOI: 10.1073/pnas.1117430109 * |
| ROBERT NIEUWENHUIZEN ET AL: "Measuring image resolution in optical nanoscopy", NATURE METHODS, 1 June 2013 (2013-06-01), United States, pages 557, XP055140742, Retrieved from the Internet <URL:http://search.proquest.com/docview/1357394150> [retrieved on 20140917], DOI: 10.1038/nmeth.2448 * |
| ROBERT P. J. NIEUWENHUIZEN ET AL: "Image resolution in optical nanoscopy", PROCEEDINGS OF SPIE, vol. 8815, 20 September 2013 (2013-09-20), pages 881508, XP055140750, ISSN: 0277-786X, DOI: 10.1117/12.2025352 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sage et al. | Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software | |
| Mahecic et al. | Event-driven acquisition for content-enriched microscopy | |
| Conrad et al. | Micropilot: automation of fluorescence microscopy–based imaging for systems biology | |
| Venkataramani et al. | SuReSim: simulating localization microscopy experiments from ground truth models | |
| JP5870851B2 (en) | Information processing apparatus, information processing method, and program | |
| JP5834584B2 (en) | Information processing apparatus, information processing method, program, and fluorescence spectrum intensity correction method | |
| JP2022500744A (en) | Computer implementation methods, computer program products and systems for analysis of cell images | |
| Rowley et al. | Robust Bayesian fluorescence lifetime estimation, decay model selection and instrument response determination for low-intensity FLIM imaging | |
| van Teeffelen et al. | Image analysis in fluorescence microscopy: bacterial dynamics as a case study | |
| Mailfert et al. | A theoretical high-density nanoscopy study leads to the design of UNLOC, a parameter-free algorithm | |
| Munro et al. | Accelerating single molecule localization microscopy through parallel processing on a high‐performance computing cluster | |
| JP2016520798A (en) | 3D image processing for localization of nanoparticles in biological and non-biological media | |
| Bryan IV et al. | Diffraction-limited molecular cluster quantification with Bayesian nonparametrics | |
| Pylvänäinen et al. | Live-cell imaging in the deep learning era | |
| Chen et al. | MS-PyCloud: An open-source, cloud computing-based pipeline for LC-MS/MS data analysis | |
| Shepherd et al. | PySTACHIO: Python Single-molecule TrAcking stoiCHiometry Intensity and simulatiOn, a flexible, extensible, beginner-friendly and optimized program for analysis of single-molecule microscopy data | |
| JP2022505252A (en) | Microscope workflow optimization | |
| Shroff et al. | Live-cell imaging powered by computation | |
| Nino et al. | FOCAL3D: A 3-dimensional clustering package for single-molecule localization microscopy | |
| Schrangl et al. | Kinetic analysis of single molecule FRET transitions without trajectories | |
| Wanninger et al. | Deep-LASI: deep-learning assisted, single-molecule imaging analysis of multi-color DNA origami structures | |
| Gole et al. | OpenSegSPIM: a user-friendly segmentation tool for SPIM data | |
| NL2012169C2 (en) | Determining the average number of localizations per emitter and the average number of emitters associated with one site in localization microscopy. | |
| Rees et al. | A transfer function approach to measuring cell inheritance | |
| Caldas et al. | iSBatch: a batch-processing platform for data analysis and exploration of live-cell single-molecule microscopy images and other hierarchical datasets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM | Lapsed because of non-payment of the annual fee |
Effective date: 20170201 |