Classifications

• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
  • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
  • G02B21/0052—Optical details of the image generation
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
  • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
  • G02B21/0052—Optical details of the image generation
  • G02B21/0064—Optical details of the image generation multi-spectral or wavelength-selective arrangements, e.g. wavelength fan-out, chromatic profiling
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
  • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
  • G02B21/0052—Optical details of the image generation
  • G02B21/0072—Optical details of the image generation details concerning resolution or correction, including general design of CSOM objectives
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
  • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
  • G02B21/0052—Optical details of the image generation
  • G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/002—Scanning microscopes
  • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
  • G02B21/008—Details of detection or image processing, including general computer control

Definitions

• the invention relates to forming images using scanning microscopy and, in particular, though not exclusively, to systems and methods for forming images of a sample using scan ⁇ ning microscopy, a computer system for controlling such systems and computer program product for executing such meth- ods .
• Rescan confocal microscopy is a known super- resolution microscopy technique for obtaining high-resolution images using conventional refractive and reflective optical elements.
• the principles of RCM are explained in the article by De Luca et al . , "Re-scan confocal microscopy: scanning twice for better resolution” , Biomed Opt Express; 2013 Nov 1 ; 4(11) : 2644-2656.
• a high resolution colour image of an area of a sample can be obtained by scanning the area line by line with an illumination light beam of two different wavelengths and rescanning the light originating from the area onto an image sensor so that an image of the excited area is obtained.
• a problem related to the colour rescan confocal microscopy technique as described by De Luca is that it is less suitable for time-critical measurements which typically require (almost) simultaneous acquisition of different images associated with different optical information.
• the article by De Luca reported reduction of the time difference between two colour scans to the time of a single scanning line, the fact that the line images are not of the same time instance, imaged features and/or objects in different images may have moved and/or changed so that combining the images will cause artefacts in the final colour image.
• fluorescence or luminescence lifetime imaging microscopy the luminescence lifetime properties of a sample can be obtained by exposing a sample with modulated light and by measuring multiple images of the modulated emission light.
• the line by line scanning technique described in De Luca however is not suitable for reliably measuring lifetime properties of a sample .
• WO 2015/164843 discloses a multifocal structured illumination microscopy system having two scanning mirrors and two cameras.
• a sample that is positioned at a focus is scanned with excitation light using a scanning mirror.
• the sample emits emission light back onto the scanning mirror which reflects the emission light onto a second scanning mirror, which scans the emission light onto the two cameras.
• the emission light from the second scanning mirror passes through a dichroic mirror that splits the emission light towards the two cameras. This enables each camera to separately detect a different light color.
• a problem related to this system is that movement of the emission light beam with respect to the dichroic mirror causes the emission light to be incident on the mirror under a varying angle. This varying angle influences the optical interactions between the dichroic mirror and the light beam, which negatively impacts the quality of the images captured by the cameras.
• scanning microscopy that allows for increasing the amount of information that can be obtained with one scan cycle. This for example allows to reduce the number of required scans and to increase scan speed. It may also beneficially eliminate a time gap between measurements.
• the invention may relate to a method for forming images of a sample using scanning microscopy.
• the method may include controlling a scanning mirror of a first optical system to move (scan) one or more focused illumination light signals over (or through) an optical active sample, wherein the one or more focused
• illumination light signal cause one or more optical
• the first optical system forming at least one emission light signal from
• emission light originating from the one or more optical excitations and directing the emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system; a light splitter in the second optical system splitting the at least one emission light signal in at least a first emission light signal comprising first optical information and second
• emission light signal comprising second optical information, preferably the first optical information being different from the second optical information; and, the second optical system directing the first and second emission light signals onto a rescanning mirror, the rescanning mirror being controlled to move (scan) a focused emission light beam signals over the imaging plane of the imaging system for simultaneously forming a first image associated with the first optical information and second image associated with the second optical
• the invention enables a scanning microscope to determine image data, i.e. multiple different images,
• the invention thus enables a scanning microscope to measure and image time-critical processes, such as color images, time-dependent processes and/or life-time
• the angle under which light falls onto the light splitter can be kept constant. Consequently, the optical interactions between the incident light and the splitter are also constant.
• the cutoff wavelength of the dichroic mirror remains the same during an experiment, which ensures that any variation in light intensity detected by the imaging system corresponds to an intensity variation of the light emission signal and not to a variation of the cutoff wavelength of the dichroic mirror. In the latter case, a formed image would become inaccurate as it would contain intensity variations that do not correspond with variations in density of
• fluorophores in the sample may reduce such distortions and increase the quality of the images that are generated by the scanning microscope in one scanning cycle.
• the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band.
• the first optical information may be associated with the emission of the first wavelength or a first wavelength band and the second optical information may be associated with the emission of the second wavelength or second wavelength band.
• the method further comprise: the capturing first image data associated with the first emission light and a second image data associated with the second emission light; and, forming an image of at least part of the sample by comprising at least part of the first and second image data.
• the method may further comprise: modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, wherein the modulation of the illumination light signal may cause a modulation of the intensity of the first and second emission light signal.
• the method may further comprise phase-shifting the modulated second emission light signal with respect to the modulated first emission light signal.
• the phase-shifting may include directing the modulated second emission light signal through an optical delay line.
• the modulation frequency may be selected between 1 and 200 MHz.
• the method may further comprise: the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal.
• the method may further comprise a processor determining a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
• the method may comprise focusing at least part of the emission light signal upon a pinhole
• the focusing may include using at least one first optical element for focusing the emission light signal originating from the scanning mirror onto the pinhole aperture and at least one second optical element configured to form a collimated emission light signal of the emission light that passes pinhole and to direct the
• the one or more focused image is displayed in an embodiment.
• illumination light signals may comprise at least a first focused illumination light spot at a first position of a focusing plane in the sample and a second focused illumination light spot at a second position in the focusing plane in the sample, preferably the first illumination light being
• the first emission light signal comprises emission light of the first optical
• the second emission light beam comprises emission light of the second optical excitation.
• the one or more focused image is displayed in an embodiment.
• illumination light signals may comprise at least a first focused illumination light spot at a first position in a first focusing plane in the sample and a second focused illumination light spot at a second position in a second focusing plane in the sample.
• controlling the scanning and the rescanning mirror may include rotating the scanning mirror and the rescanning mirror back and forth over a predetermined first angular amplitude and a second angular amplitude
• the second angular amplitude of the rescanning mirror may be selected larger than the first angular amplitude of the scanning mirror.
• the second angular amplitude may be selected as two times the first angular amplitude.
• the invention may relate to a method for forming images using scanning microscopy
• the first optical system forming a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system; a light splitter in the second optical system splitting the modulated emission light signal in at least a first modulated emission light signal and second modulated emission light signal representing a phase-shifted version of the first modulated light signal; and, the light splitter directing the first and second modulated emission light signals onto a rescanning mirror, which is controlled to move (scan) focused emission light signals over the imaging plane of the imaging system in order to simultaneously form a first image of the first modulated emission light signal and second image of the second modulated emission light signal, while the one or more focused modulated illumination light signals are scanned over (or through) the sample.
• the first optical system forming a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via
• the invention may relate to a scanning microscopy system for forming images of a sample
• the system may comprise: a light source, preferably one or more lasers, configured for generating light for illuminating the sample; a first optical system comprising a scanning mirror, the optical system being configured: to focus illumination light of the light source as one or more
• a second optical system comprising a
• the light splitter being configured: to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical
• second optical system being configured: to direct the first and second emission light signals towards the rescanning mirror and to focus the first and second emission light signals onto an imaging plane;
• an imaging system configured to receive first and second emission light signals originating from the rescanning mirror; and, a computer system being adapted to: control the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample; control the rescanning mirror of the second optical system to move (scan) a focused emission light beam signal over the imaging plane of the imaging system; and, control the imaging system for simultaneously capturing first image data associated with the first optical information and second image data associated with the second optical
• the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band; and, wherein the first optical
• the second optical information is associated with the emission of the second wavelength or second wavelength band.
• the system may further comprise: a modulator for modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a modulation of the intensity of the first and second emission light signal; a phase-shifter, preferably an optical delay line, configured to phase-shift the modulated second emission light signal with respect to the modulated first emission light signal.
• a modulator for modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz
• the modulation of the illumination light signal causing a modulation of the intensity of the first and second emission light signal
• a phase-shifter preferably an optical delay line, configured to phase-shift the modulated second emission light signal with respect to the modulated first emission light signal.
• the system may further comprise the imaging system further being configured to simultaneously capture first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal; and, the computer system further being adapted to: determine a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
• the invention may relate to a computer system for controlling scanning microscopy system for forming images of a sample
• the system comprising a light source, preferably one or more lasers, configured for generating light for illuminating the sample; a first optical system comprising a scanning mirror, the optical system being configured to focus illumination light of the light source as one or more illumination light signals via the scanning mirror onto the sample, the one or more focused illumination light signals causing one or more optical excitations in the optical active sample; to form an emission light signal from emission light originating from the one or more optical excitations in the sample; and, to direct the emission light signal onto the scanning mirror; a second optical system comprising a
• the light splitter being configured to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical
• second emission light signal comprising second optical information, the second optical system being
• the computer system comprises a computer readable storage medium having computer readable program code embodied
• a processor preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the first computer readable program code, the processor is configured to perform executable operations comprising:
• controlling the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample
• controlling the imaging system to simultaneously first image data associated with the first optical information and second image data associated with the second optical information, while the sample is scanned with the one or more illumination light signals.
• the processor may be further configured to perform an executable operation comprising:
• modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a
• the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with a time-shifted version of the modulated first emission light signal;
• the invention also relates to computer program product for carrying out the methods described herein, as well as a non-transitory computer readable storage-medium storing the computer program.
• a computer program may, for example, be downloaded (updated) to an existing control device of the scanning microscopy system or be stored upon manufac ⁇ turing of this control device.
• Fig. 1 shows a scanning microscopy system according to an embodiment of the invention
• FIG. 2A-2C schematically depict the scanning of an illumination light spot and emission light spots by a scanning microscopy system according to an embodiment of the invention
• Fig. 3 depicts a flow diagram of a scanning process according to an embodiment of the invention
• FIG. 4 depicts a scanning microscopy system according to another embodiment of the invention.
• Fig. 5A-5C depicts a data acquisition process for a scanning microscopy system according to an embodiment of the invention . ;
• Fig. 6 depicts a flow diagram of a scanning process according to an embodiment of the invention.
• Fig. 7 depicts a scanning microscopy system according to another embodiment
• Fig. 8 depicts a scanning microscopy system according to yet another embodiment
• Fig. 9 depicts a scanning microscopy system according to a further embodiment
• DETAILED DESCRIPTION OF THE DRAWINGS Fig. 1 shows a scanning microscopy system according to an embodiment of the invention.
• the system may comprise a light source 102 for generating illumination light that is directed onto a sample 116 located on a sample holder (not shown) .
• Illumination light 106 may be directed onto a sample using a first optical system 132, which may include one or more refractive and/or reflective optical elements.
• the first optical system may include one or more collimating and/or focusing lenses
• a lens 106 may be used in order to from a collimated beam of illumination light 106 which may be reflected via a set of mirrors 108,110,112 towards a lens 114 in order to form a focused illumination light spot 117 on the sample.
• the focused illumination light spot may excite local optical excitations, which generate emission light 118.
• the emission light may pass through lens 114 and may be directed via a second optical system 136 comprising one or more
• refractive and/or reflective optical elements to an imaging system 124.
• the second optical system may include a lens 114 for forming a collimated beam of the emission light that is reflected by mirror 112.
• the beam of emission light may pass one or more optical elements, e.g. dichroic mirrors 110,128, for splitting the emission light beams into different emission light beams before it is reflected by a further mirror 120, which directs the emission light beams to a focussing lens 122.
• the focussing lens may focus the emission light beams into focussed emission light spots 126a, 126b onto the imaging plane of imaging system 124.
• the imaging system may include one or more image sensors, e.g. one or more CMOS image sensors or one or more CCDs.
• Dichroic mirror 110 may be configured such that it functions as a reflector for
• Mirror 112 which reflects the illumination light towards the sample and the emission light towards the further mirror 120 may be configured as a rotatable mirror which can be controlled by a computer system (not shown) .
• This mirror may be referred to as a scanning mirror.
• the scanning mirror may be configured to move the focused illumination light spot 117 over or through the sample 116.
• scanning mirror 112 may be configured to rotate back and forth over a first angular amplitude Al 138 in order to move (“scan") the focused illumination light spot over or through sample causing local optical illumination along its way.
• the rotatable scanning mirror may be controlled by a number of computer-controlled actuators. This way the rotatable scanning mirror is used in order to scan illumination light spot over the surface of the sample in accordance to a particular scanning pattern.
• a scanning pattern may be a meandering pattern. The scanning pattern may be configured so that after scanning a particular area, the scanned area is fully exposed by the moving illumination light spot .
• emission light originating from the moving illumination light spot is split into multiple
• collimated beams may be reflected via the scanning mirror 112 towards the further mirror 120, for example a rotatably mounted mirror, which is controlled to reflect the light beams towards the focusing lens in order form multiple moving emission light spots 126a, 126b onto the imaging plane of the imaging system 124.
• the further mirror 120 for example a rotatably mounted mirror, which is controlled to reflect the light beams towards the focusing lens in order form multiple moving emission light spots 126a, 126b onto the imaging plane of the imaging system 124.
• the further mirror 120 may be configured to rotate back and forth over a second angular amplitude A2 140 in order to move (scan) the focused emission light spots over the imaging plane of the image sensor, while the focussed
• mirror 120 may hereafter be referred to as the rescanning mirror.
• the actuators and/or electro-motors of the scanning and rescanning mirror may be controlled by a computer system so that the mirrors can be moved in sync.
• the frequency of the back and forth rotation of the scanning mirror is identical to the frequency of the back and forth rotation of the rescanning mirror. This way, when the scanning mirror is controlled by the computer system to scan an area of the sample by the moving
• the rescanning mirror is controlled by the computer system to scan associated areas of pixels of the imaging plane of the image sensor so that the pixels are exposed by moving emission lights spots. This way, when an area of a sample is optically excited by the illumination beam, emission light originating from the excited area is imaged onto the imagining plane.
• the second angular amplitude of the rescanning mirror may be larger than the first angular
• the second angular amplitude may be twice the first angular amplitude, so that the resolution of the image is improved by a factor with respect to the resolution that may be obtained when the second angular amplitude is chosen to be equal to the first angular
• the light source 102 may comprise a plurality of light sources, e.g. a plurality of lasers, and one or more light filters.
• the light sources and filters may be used to control which wavelengths the illumination light comprises.
• the light source may be configured to generate white light.
• the light source may be configured to generate light consists of two or more
• the light source may generate light of a first wavelength selected from the blue band of the visible spectrum and light of a second wavelength selected from the yellow band of the visible spectrum.
• the optical elements of optical system 132 may be configured to control different wavelengths.
• mirror 108 may reflect the illumination light onto dichroic mirror 110 that may be configured to reflect light of a first group of wavelengths, e.g. blue light and yellow light.
• Dichroic mirror 110 may be further configured to pass light of a second group of wavelengths, e.g. red and green light. This way, the illumination light may be reflected by dichroic mirror 110 onto scanning mirror 112. Scanning mirror may reflect the illumination light onto lens 114 which may focus the illumination light onto the sample 116 creating a focused illumination light spot 117.
• a second group of wavelengths e.g. red and green light.
• the illumination light may comprise light of a first wavelength, e.g. yellow light, which may cause a first type of optical excitations in the sample, the first type of excitations generating first emission light.
• a first wavelength e.g. yellow light
• the illumination light may comprise light of a second wavelength, e.g. blue light, which may cause a second type of optical excitations in the sample, the second type of
• first group of wavelengths which may be reflected by dichroic mirror 110, may comprise said first wavelength and said second wavelength.
• sample may be sample, e.g. a biological material, that is imaged using a reflective
• sample 116 may be an optically active sample.
• the sample may be a material, e.g. a biological material, comprising one or more types of fluorescent /luminescent materials, such as
• the optically active sample may be non-luminescent optical active material.
• the sample may be a material that can be imaged on the basis of second harmonics generation (SHG) microscopy or third harmonics generation (THG) microscopy.
• the sample may be a material that can be imaged using a Rahman microscopy technique, such as Coherent Anti- Stokes Rahman Scattering (CARS) microscopy.
• the sample comprise at least a first type of fluorophores, e.g. red fluorophores, and a second type of fluorophores, e.g. green fluorophores .
• the light of the first wavelength of the illumination light may cause the first fluorophores in the sample to emit the first emission light and the light of the second wavelength may cause the second fluorophores in the sample to emit second emission light.
• the emission light may comprise light that is emitted as a result of photoluminescence, preferably
• a blue light component of the illumination light may cause green fluorophores at the focused illumination light spot to emit green light.
• yellow light component of the illumination light may cause red fluorophores at the focused illumination light spot to emit red light.
• the emission light thus comprises green light and red light.
• the path of the illumination light that travels from dichroic mirror to the sample may coincide with the path of the emission light that travels from the sample to the
• the emission light may pass through dichroic mirror, since it may be configured to pass light of the second group of wavelength, (e.g. red and green light) .
• the second group of wavelengths may thus comprise the
• the emission light originating from the sample may include optical
• emission light beam 118 may be split into a first emission light beam 118a and second emission light beam 118b.
• forming the first and second emission light beams may include spitting the emission light beam in a first and second light beam of different wavelengths .
• the first and second emission light beams may be formed on the basis of a variety of optical properties of the emission light.
• the emission light beams may be formed on the basis of the wavelength, wherein the first emission light beam comprises a first wavelength and the second light beam a different second wavelength.
• the emission light beams may be formed on the basis of polarization, wherein the first light beam comprises a first polarization and the second light beam a different second polarization.
• the second optical system of the scanning microscopy system may comprise an optical splitter that is configured to split the emission light into different emission light beams on the basis of at least one optical properties, such as wavelength,
• a light splitter such as an optical splitter 134 for splitting the emission light into different emission light beams, e.g. a first emission light 118a and second emission light 118b.
• the light splitter 128 may include at least a dichroic mirror 128 and a mirror 130 in the path of the emission light beam 118
• a dichroic mirror instead of a dichroic mirror other optical splitter elements may be used including but not limited to a polarization splitter, a grey splitter or an acousto-optical tunable splitter.
• the emission light beam 118 may be incident on dichroic mirror 128, which may be configured to reflect the first emission light, e.g. green light, and to let pass the second emission light, e.g. red light.
• dichroic mirror 128, may be configured to reflect the first emission light, e.g. green light, and to let pass the second emission light, e.g. red light.
• dichroic mirror 128 may split the emission light into a first emission light beam 118a, which may comprise the first
• the first emission light beam may comprise first optical information, e.g. information relating to red fluorophores in the sample
• the second emission light beam 118b may comprise second optical information, e.g. information relating to green fluorophores in the sample.
• the first emission light beam 118a may be incident onto rescanning mirror 120, while the second emission light beam 118b may be directed onto mirror 130, which may reflect the second emission light beam 118b onto rescanning mirror 120.
• the light splitter may be configured so that the split emission light beams, e.g. first emission light beam 118a and second light beam 118b, may be incident on rescanning mirror 120 under different angles.
• Rescanning mirror may reflect the emission light beams onto different areas of the image plane of the imaging system.
• Rotatable rescanning mirror may reflect both emission light beams 118a, 118b onto the focusing lens which may focus the light beams onto an image plane of imaging system. This way, when rescanning the focused emission light spots onto the imaging plane, at least a first and a second image are formed simultaneously.
• the first emission light beam may cause a first illumination spot at a first position 126a, while the second emission light beam may cause a second illumination spot at a second position 126b that is different from the first position.
• filters such as color filters, e.g. a first color filter 123a and second color filter 123b, may be positioned in front of the imaging system.
• a color filter may be configured as a bandpass filter that is configured to pass light in a predetermined band of wavelengths.
• the first color filter 123a may be configured to pass first
• the second color filter 123b may be configured to pass second emission light, e.g.
• the scanning mirror may rotate over a smaller angular amplitude than the rescanning mirror. Due to the rotation of the scanning mirror, the focused illumination light spot may move over or through the sample. Due to the rotation of the rescanning mirror, the focused emission light beams, and thus the emission light spots and, may move over the image area of imaging system 124 thereby exposing pixels to the emission light.
• Moving the scanning mirror may comprise rotating the scanning mirror over a first angular amplitude and moving the rescanning mirror may comprise rotating the rescanning mirror over a second angular amplitude.
• the first angular amplitude may be different from the second angular amplitude, in particular the first angular amplitude may be smaller than the second angular amplitude, in particular the first angular amplitude may be approximately half the second angular amplitude.
• Fig. 1 depicts an example wherein the emission light beam is split in two emission light beams for simultaneous imaging two images of a sample area comprising different optical information
• this scheme may be extended to embodiments that are configured to split the emission light beam in multiple (more than two) emission light beams so that multiple images comprising different optical information.
• scanning microscopy system depicted in Fig. 1 is capable simultaneously capturing super-resolution images of excited areas of a sample and combining these images into a super-resolution colour image.
• the system is suitable of colour imaging time- critical processes of an area of a sample with super- resolution using conventional refractive and reflective optical elements.
• Certain reflective elements in the scanning microscopy system such as mirrors 108,130 and dichroic mirrors 110,128 may be implemented as static optical elements. Such optical elements cannot be controlled by the computer system. Other reflective elements may be implemented as rotatable optical elements that can be controlled by the computer system.
• the path of illumination light 106 from light source 102 towards the scanning mirror 112 may be controlled using static optical elements. During a scan, the scanning mirror 112 may rotate (i.e. rotatable moving back and forth) over an angular amplitude which causes the path of the illumination light 106 from scanning mirror 112 to the sample 110 (and thus the focused execution light spot) to move over the surface of the sample in accordance with the movements of the scanning mirror.
• the path of the illumination light 106 from scanning mirror 112 to sample 116 is said to be dynamic in the sense that it is moving with respect to the sample and/or light detector during scanning. Consequently the point of illumination in the sample is moving during scanning and therefore also the point where emission signal comes from from the sample is moving during scanning.
• the scanning mirror both directs the illumination light signal from the light source 102 towards the sample 116 as well as the emission light signal from the sample 116 towards the rescanning mirror 120. This way, the movement of the emission light signal originating from the sample is neutralized by the movement of the scanning mirror.
• the path of emission light from scanning mirror to rescanning mirror (the paths of light beams 118a and 118b from dichroic mirror 128 to rescanning mirror) is static.
• the scanning mirror 112 "descans" the emission light 118 and reflects the emission light as a static emission light beam towards the rescanning mirror.
• the path of the emission light between rescanning mirror and light detector may be dynamic during scanning due to the movement of the rescanning mirror.
• the optical beam splitter comprising e.g. a dichroic mirror, may be positioned in a static path of emission light.
• FIG. 2A-2C schematically depict a top view of the scanning movements of an illumination light spot 217 and one or more associated emission light spots 226a, 226b of a
• Fig. 2A-2C depict the movement of a focused illumination light spot 217 over or through optical active sample 216 and the movement of the associated focused first and second emission light spots 226a, 226b over the image area of the imaging system 224 at three different moments in time TO, Tl and T2.
• the illumination light may comprise multiple wavelengths, e.g. a first and a second wavelength, for exciting different optical illumination sites in the sample.
• Fig. 2A depicts the start of the scanning process for both the illumination light spot and the associated emission light spots.
• Fig. 2B depicts the movement of the illumination light spot and the associated focused first and second
• time instance Tl emission light spots over the image area of the imaging system 224 at an intermediate time instance Tl.
• TO may mark the start of a scan wherein the illumination light spot will start exposing a predetermined area by moving
• the scanning pattern may be a meandering pattern.
• the light spots may be moved by a computer system controlling the scanning and descanning mirror using actuators or the like as described with reference to
• emission light originating from the sample is split into different emission light beams which are focused as emission light spots, in this example a first and second emission light spot 226a, 226b, onto the imaging plane of the imaging system wherein each emission light beam carries a different type of optical information of the area of the sample that is excited by the illumination light spot.
• emission light spots a first and second emission light spot 226a, 226b
• each emission light beam carries a different type of optical information of the area of the sample that is excited by the illumination light spot.
• Fig. 2C depicts the scanning pattern at a second time instance T2 (the end of a scan) wherein a full scan of the scanned sample area 228 may results in two scanned areas 230a, 230b of the imaging plane of the imaging system wherein each scanned area of the imaging plane may result in an image of the scanned sample, wherein each image may include
• the imaging area of the imaging system may be any imaging area of the imaging system.
• an image sensor may include one or more pixel arrays that are configured to simultaneously image the different illumination light beams.
• An image sensor may be implemented as a CCD image sensor or a CMOS image sensor .
• the speed with which the focused illumination light spot moves over or through optical active sample 216 may be lower than the speed with which the first and second emission light beams move over the image area of the imaging system.
• said speed of the focused illumination light spot 217 may be approximately half the speed of the emission light beams, in particular of the first and second illumination spots.
• Fig. 3 depicts a flow diagram of a scanning process according to an embodiment of the invention.
• the figure depicts a scanning process that may be executed by a scanning microscope system as described with reference to Fig. 1.
• the process may be executed by a computer system configured to control the light source and the actuators for controlling the movements of the scanning and rescanning mirror .
• the computer system may control the scanning mirror of a first optical system to move (scan) one or more focused illumination light spots over (or through) an optical active sample, wherein the one or more focused illumination light spots cause local optical excitations in the optical active sample.
• Local optical excitations may generate emission light which in a second step 304 may be picked up by the first optical system in order to form an emission light beam from the emission light originating from the optical excitations.
• the first optical system may direct the emission light beam onto the scanning mirror which may reflect the emission light beam via a second optical system to an imaging system (step 304) .
• the second optical system may include a light
• the light splitter for splitting the emission light beam in at least a first emission light beam comprising first optical information and second emission light beam comprising second optical information (step 306) . Then, the light splitter may direct the first and second emission light beams onto a computer- controlled rescanning mirror, which is used to move (can) focused emission light beam spots over the imaging plane of the image system (step 308) in order to form a first image comprising the first optical information and second image comprising the second optical information, while the sample is scanned with the illumination light.
• a computer- controlled rescanning mirror which is used to move (can) focused emission light beam spots over the imaging plane of the image system (step 308) in order to form a first image comprising the first optical information and second image comprising the second optical information, while the sample is scanned with the illumination light.
• the computer system may be configured to
• the system is capable of simultaneously generating multiple high-resolution images of a sample area that is excited with one or more illumination light spots. This way, the rescan microcopy system allow the imaging of time-critical processes .
• FIG. 4 depicts a scanning microscopy system according to another embodiment of the invention.
• Fig. 4 depicts a scanning microscopy system that is similar to the scanning microscopy system of Fig. 1, comprising a light source 402, a first optical system 432 comprising a scanning mirror 412 that is configured to guide an illumination light beam from the light source towards the scanning mirror and to expose a sample 416 with a moving illumination light beam, a second optical system 436 comprising a rescanning mirror 420
• Fig. 4 The embodiment of Fig. 4 is configured to measure lifetime luminescence properties. As will be explained
• the properties can be derived from a phase difference between a modulated illumination light signal and an emission light signal, because the phase difference depends on the lifetime properties of the sample as e.g.
• the emission light signal may be integrated by the imaging system over a number of very short time intervals.
• the Nyquist criterion stipulates that for mono-exponential life time measurements at least two time intervals or phase ranges are required per period (e.g. 0-180 and 180-360 degrees) .
• more than two time intervals or phase ranges e.g. 0-90, 90-180, 180-270, 270-360 degrees
• integrating the time-varying intensity of the emission light in such short phase ranges is very difficult for an image sensor, because of the very short exposure times associated with these phase ranges.
• the second optical system 436 of the scanning microscopy system may be configured to split the modulated emission light beam into a first modulated emission light beam and into one or more further modulated emission light beams, wherein the one or more further modulated emission light beams are phase- shifted with respect to the first modulated emission light beam.
• the phase shift may be 90 degrees. Both modulated emission light signals are then used in order to determine the emission light signal.
• Fig. 5A-5C depicts a data acquisition process for a scanning microscopy system according to an embodiment of the invention.
• the figures depict a data
• Fig. 5A illustrates an illumination light signal that is modulated using a frequency in a range selected between 1 kHz and 100 MHz, typically 40 MHz. Due to the modulation of the illumination signal, the emission light signals will be modulated as well.
• Fig. 5B depicts the first modulated emission light signal which has a certain amplitude and phase shift with respect to the illumination signal.
• Fig. 5C depicts a second modulated emission light signal which represents a 90 degrees phase-shifted version of the first modulated emission light signal .
• first images of the first modulated emission light signal may be captured by the imaging system at predetermined time instances TO , Tl , T2 , T3 , wherein the exposure time of the imaging system is defined by an integration window T0-T1.
• the time instances are synchronized with the modulation frequency in the sense that at time instances T0,T2,T4 the phase angle of the signal is
• pixel values of the first images generated during integration window in which the phase of the signal is 0-180 degrees represent an average value of the intensity of the first modulated emission signal in that window.
• pixel values of images generated during integration window in which the phase of the signal is 180-360 degrees represent an average value of the intensity of the first modulated emission signal in that window.
• second images are generated on the basis of the second modulated emission signal in the integration window in which the phase of the signal ranges 90- 270 degrees (region III in Fig. 5C) and 270-90 degrees (region IV in Fig. 5C) . Based on the average signal values of the four regions, the original response of the emission light signal may be obtained and subsequently the phase difference with the driving modulated illumination signal may be determined.
• microscopy system may comprise an optical source for
• the optical source may be configured to vary the intensity of the one or more focused illumination light spots with a modulation frequency, e.g. a sinusoidal modulation frequency, so that also the intensity of the first and second emission light spot will vary as a function of time.
• a modulation frequency e.g. a sinusoidal modulation frequency
• the second optical system of the microscopy system of Fig. 4 may comprise an optical splitter 427, e.g. in the form of reflective mirror, for splitting the emission light into different emission light beams, e.g. a first emission light beam and a second emission light beam.
• the second optical system may further include an optical delay line 434 in order to introduce phase shift in the modulated second emission light signal. This way, a predetermined phase difference between the time-varying intensity of the first emission light spot 426b and the second emission light spot 426a may be introduced .
• the optical delay line may comprise one or more mirrors, e.g. a first mirror 438 and a second mirror 440, in order to control the length of the optical path.
• the optical delay line shown in Fig. 4 may introduce a predetermined phase shift in the modulation signal of the second emission light spot 426b with respect to that of the first emission light spot 426a.
• the phase shift may be at least 45 degrees, preferably by at least 90 degrees, most preferably approximately 90 degrees (as e.g. depicted in Fig. 5B and 5C) .
• the phase shifted emission light beams are directed towards the rescanning mirror, which is controlled to move (scan) focused emission light beam signals over the imaging plane of the image system in order to simultaneously form a first image associated with the first emission light beam and a second image associated with the second emission light beam, while the sample is scanned with the illumination light.
• the optical splitter and the optical delay line enable the scanning microscopy system to simultaneously capture an image of the modulated emission light signal and an image of a 90 degrees phase-shifted version of the modulated emission light signal.
• an image generated during an integration window in which the phase of the signal is 0-180 degrees may be captured simultaneously with an image of the signal in an integration window in which the phase of the signal is 90-270 degrees (region III in Fig. 5C) .
• the scanning microscopy system according the invention allows lifetime measurements using a RCM super- resolution technique.
• Fig. 4 may be configured to produce more than two phase shifted emission light beam, using e.g. a beam splitter that splits the emission light beam into multiple emission light beams and one or more optical delay lines configured to introduce different phase shifts in each of the multiple emission light beams.
• a beam splitter that splits the emission light beam into multiple emission light beams and one or more optical delay lines configured to introduce different phase shifts in each of the multiple emission light beams.
• the invention allows lifetime luminescence properties using a scanning microscope.
• the invention eliminates the need for an image sensor that has on-chip phase-sensitive detection functionality.
• the lifetime imaging may also be based on other well-known techniques such as a pulsed illumination technique.
• the light source may be configured as a pulsed light source, e.g. one or more pulsed lasers.
• Fig. 6 depicts a flow diagram of a scanning process according to an embodiment of the invention.
• flow diagram depicts a scanning process that may be executed by a scanning microscope system as described with reference to Fig. 4 and 5.
• the process may be executed by a computer system configured to control the light source and the actuators for controlling the movements of the scanning and rescanning mirror .
• the computer system may control a scanning mirror of a first optical system to move (scan) one or more focused modulated illumination light signals over (or through) an optical active sample, wherein the one or more focused modulated illumination light spots cause optical excitations in the optical active sample.
• Local optical excitations may generate modulated emission light which in a second step 604 may be picked up by the first optical system in order to form a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system.
• the light splitter may direct the first and second modulated emission light signals onto a rescanning mirror, which is controlled to move (scan) focused emission light signals over the imaging plane of the image system in order to simultaneously form a first image of the first modulated light signal and second image of the second
• multiple differently phase-shifted modulated emission light signals may be used so that instead of two images a plurality of images of different parts of the modulated emission light signal may be obtained.
• the original emission light signal can be constructed and luminescent lifetimes can be determined.
• the computer system may be configured to
• Fig. 7 depicts a scanning microscopy system according to yet another embodiment.
• Fig. 7 depicts a scanning microscopy system similar to the system described with reference to Fig. 1.
• the system in Fig. 7 comprises a second optical system 736 including an optical arrangement 739 for enabling confocal measurements wherein the optical arrangement may comprise a pinhole 744.
• the scanning microscopy may be configured to focus illumination light spot 717 in a plane of interest P.
• the system may be configured to arranged to focus emission light from the plane of interest P onto plane P ' .
• the system may comprise one or more optical elements, e.g.
• Plane P' thus may be a confocal conjugate plane of plane of interest P.
• the optical arrangement may comprise a further optical element, e.g. lens 742, arranged to form a collimated light beam of the emission light 718 that passes pinhole 744.
• Emission light from the plane of interest in the sample may pass through the pinhole while emission light 718 that is emitted from other planes than the plane of interest is substantially blocked by the pinhole so that the signal-to- noise ratio of the emission light signal that is focused onto the imaging plane of the imaging system may be substantially improved.
• Fig. 8 depicts a scanning microscopy system according to yet another embodiment.
• Fig. 8 depicts a scanning microscopy system comprising a first optical system 832 comprising a number of optical elements, e.g. refractive and/or diffractive optical elements, arranged to form a first focused illumination light spot 817a at a first position in or on the sample and a second focused illumination light spot 817b at a second position in or on the sample.
• the first and second position may have a different lateral position.
• the first optical system further comprises at least one scanning reflector 812 for moving the light spots over or through the optical active sample 816.
• the first illumination light spot 817a may be formed using a first illumination light beam 806a and the second illumination light spot 817b may be formed using a second illumination light beam 806b.
• the first illumination light beam may comprise light of a first
• the second illumination light beam may comprise light of a second wavelength, e.g. blue light.
• the first and second illumination light beams may be formed on the basis of light of light source.
• the first and second light spots may be arranged close together so that the spots illuminate a small area very close to each other.
• Light of two or more wavelengths may be split using a first optical splitter 846, for example dichroic mirror.
• the optical splitter 46 may be configured to reflect light of a second wavelength, e.g. blue light, and let pass light of a first wavelength, e.g. yellow light.
• first illumination beam 806a may comprise the first wavelength and the second illumination beam 806b may comprise the second wavelength.
• the illumination beams may be directed on mirror 808 that may reflect both light beams onto mirror 10.
• Mirror 10 may reflect both the first and second illumination light beam onto scanning mirror 812.
• the scanning mirror may reflect the two illumination light beams onto sample 16 which causes local optical excitations generating emission light at the two focused illumination light spots 817a and 817b.
• a yellow illumination light spot 817a may cause emission of red emission light and a blue focused illumination light spot causes emission of green emission light.
• the system may comprise a second optical system for directing the emission light via a rescanning mirror onto an imaging system that is similar to the second optical system of Fig. 1.
• the emission light both the first and second emission light, may be incident on scanning mirror 812 and may pass through dichroic mirror 810.
• Dichroic mirror 828 may reflect the second emission light, e.g. the green light, and let pass the first emission light, e.g. the green emission light.
• the first emission light beam 818a comprising the first emission light and the second emission light beam 818b
• Both emission light beams may be directed onto rescanning mirror 820 which may direct the two beams onto the image sensor.
• a first image and second image are formed by focusing with lens 822 the first and respective second
• the two emission light beams 18a and 18b cause first
• this embodiment allows to simultaneously illuminate and image an area of the sample using different illumination light signals.
• the first optical information in the first emission light beam and the second optical information in the second emission light beam may relate to the same area of the sample.
• Fig. 9 depicts a scanning microscopy system according to another embodiment.
• the scanning microscopy system may be configured such that the first and second illumination light spot may be at different depths into the sample.
• an optical splitter 947 may direct part, e.g. 50%, of the incoming illumination light 906 towards mirror 948 and may let pass part, e.g. 50%, of the incoming illumination light.
• the two illumination light beams 906a and 906b may comprise the same wavelengths of light. Again, the two illumination light beams may be directed onto scanning mirror 912.
• the first illumination light beam may be directed via a first lens 907a onto lens 914, thereby focusing the first illumination light beam 906a onto the sample creating a first focused illumination light spot 917a at a first focus plane PI in the sample, which causes emission of first
• the second illumination light beam may be directed via a second lens 907b onto lens 914 thereby focusing the second light beam onto the sample 916.
• illumination light beam 906b causes emission of second
• the first illumination light spot may cause emission of emission light from a first plane of interest PI and the second illumination light spot may cause emission of emission light from a second plane of interest P2.
• the step of forming the first and second emission light beams on the basis of the emission light may comprise directing the first emission light through a first optical path and directing the second emission light through a second optical path, wherein the first optical path is different from the second optical path.
• the first and second emission light beams are formed on the basis of the emission light in the sense that the second emission light is directed on mirror 930 whereas the first emission light is not.
• the first light beam 18a which comprises the first emission light may be focused upon a pinhole 944a.
• the pinhole 944a is placed in a conjugate focal plane PI' to plane of interest PI.
• the second emission light beam 18b is focused upon pinhole 944b.
• Pinhole 944b is in a conjugate focal plane P2 ' to plane of interest P2.
• the pinholes 944a and 944b provide images of the emission light with an improved signal-to-noise ratio.
• Various embodiments of the invention may be imple ⁇ mented as a program product for use with a computer system, where the program(s) of the program product define functions of the embodiments (including the methods described herein) .
• the program(s) can be contained on a vari ⁇ ety of non-transitory computer-readable storage media, where, as used herein, the expression "non-transitory computer readable storage media" comprises all computer-readable media, with the sole exception being a transitory, propagating signal.
• the program(s) can be contained on a vari ⁇ ety of transitory computer-readable storage media.
• Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read- only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid- state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., flash memory, floppy disks within a diskette drive or hard- disk drive or any type of solid-state random-access semicon ⁇ ductor memory) on which alterable information is stored.
• the computer program may be run on the processor 102 described herein .

Landscapes

• Physics & Mathematics (AREA)
• Chemical & Material Sciences (AREA)
• Analytical Chemistry (AREA)
• General Physics & Mathematics (AREA)
• Optics & Photonics (AREA)
• Engineering & Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Engineering & Computer Science (AREA)
• Spectroscopy & Molecular Physics (AREA)
• Microscoopes, Condenser (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=58455598

Family Applications (1)

Country Status (3)

Families Citing this family (2)

Family Cites Families (5)

• 2017
  • 2017-01-05 NL NL2018134A patent/NL2018134B1/en not_active IP Right Cessation
• 2018
  • 2018-01-05 EP EP18700268.8A patent/EP3566089A1/de not_active Withdrawn
  • 2018-01-05 WO PCT/EP2018/050280 patent/WO2018127566A1/en unknown

Also Published As

Similar Documents

Legal Events