NL2024861B1 - Method for recording and correlating light and charged particle microscopy images - Google Patents
Method for recording and correlating light and charged particle microscopy images Download PDFInfo
- Publication number
- NL2024861B1 NL2024861B1 NL2024861A NL2024861A NL2024861B1 NL 2024861 B1 NL2024861 B1 NL 2024861B1 NL 2024861 A NL2024861 A NL 2024861A NL 2024861 A NL2024861 A NL 2024861A NL 2024861 B1 NL2024861 B1 NL 2024861B1
- Authority
- NL
- Netherlands
- Prior art keywords
- charged particle
- view
- image
- sample
- images
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01J—ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS
- H01J37/00—Discharge tubes with provision for introducing objects or material to be exposed to the discharge, e.g. for the purpose of examination or processing thereof
- H01J37/02—Details
- H01J37/22—Optical or photographic arrangements associated with the tube
- H01J37/226—Optical arrangements for illuminating the object; optical arrangements for collecting light from the object
- H01J37/228—Optical arrangements for illuminating the object; optical arrangements for collecting light from the object whereby illumination and light collection take place in the same area of the discharge
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01J—ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS
- H01J37/00—Discharge tubes with provision for introducing objects or material to be exposed to the discharge, e.g. for the purpose of examination or processing thereof
- H01J37/26—Electron or ion microscopes; Electron or ion diffraction tubes
- H01J37/28—Electron or ion microscopes; Electron or ion diffraction tubes with scanning beams
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01J—ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS
- H01J2237/00—Discharge tubes exposing object to beam, e.g. for analysis treatment, etching, imaging
- H01J2237/22—Treatment of data
- H01J2237/226—Image reconstruction
Landscapes
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
- Analysing Materials By The Use Of Radiation (AREA)
- Microscoopes, Condenser (AREA)
Abstract
The invention relates to a method for recording and correlating light and charged particle microscopy images of a sample or one of more sections of a sample, wherein the method comprises the steps of: recording a light microscopy image of a first field of view; recording a first charged particle microscopy image of a second field of view at least partially overlapping with the first field of view, wherein the first charged. particle microscopy image has a first magnification; recording' multiple second. charged. particle microscopy images of third fields of view, wherein each of the multiple second charged particle microscopy images has a second magnification which is higher than the first magnification, and wherein the third fields of view are sub— fields of the second field of view; registering the light microscopy image to the first charged particle microscopy image; and registering the multiple second charged particle microscopy images to the first charged particle microscopy image.
Description
P137251NLO00. Method for recording and correlating light and charged particle microscopy images
BACKGROUND The invention relates to a method for recording and correlating light and charged particle microscopy images.
The aim of large-scale or serial section imaging with an electron microscope is in general to obtain a high- resolution, for example in the range of 1-10 nm, map or three- dimensional volume reconstruction of the structure, for example, of a biological specimen, Such a map or three- dimensional volume construction can be compared to an atlas for the respective biological specimen.
A known method for obtaining a high-resolution map or three-dimensional volume reconstruction involves scanning a sample multiple times by means of an electron microscope.
The first scan is made at low magnification, and the first low magnification scan is used to select regions to be scanned at a higher magnification. These steps are repeated up to the point where one defines the regions for scanning at highest magnification, resulting in a resolution in the range of 1- 10 nm. This method has been applied for imaging the brain of a zebrafish embryo.
Another known method involves the use of a light microscope, wherein the sample to be scanned is equipped with one or more fluorescence labels. The fluorescence labels are arranged at on those regions of the sample, of which, for example, a high-resolution map is required. Additionally, by detecting in-situ whether or not fluorescence is present, it can be decided in an automated fashion which regions of the sample not to scan or to scan in low magnification, and which regions to scan in high magnification. Yet another method involves scanning a sample by means of a so-called multi-beam electron microscope. In such a microscope, a sample is scanned by means of multiple electron beams. As this leads to a higher throughput, it means that the first entire scan of the sample can be conducted at a higher magnification than in the first mentioned method where only a single beam is used.
SUMMARY OF THE INVENTION A disadvantage of the known method that involves scanning a sample multiple times by means of an electron microscope is that the total collection time for scanning the sample up to the point where one defines the regions for scanning at highest magnification is a very time-consuming method. In case of the mentioned example of imaging the brain of a zebrafish embryo, the total collection time including the multiple scans consisted of almost a year. Thus, the throughput of this method is too low in order to collect the required data within an experimentally realistic time. A disadvantage of the method involving the use of a light microscope for obtaining a fluorescence image is that fluorescence and high magnification electron microscopy images have a very different magnification and scale. This results in that several high magnification electron microscopy images have to be recorded and stitched together to fill the field of view of one fluorescence image. Determination of the registration of each of the high magnification electron microscopy images to the fluorescence image is very time-consuming, therewith lowering the throughput of the method. Additionally, in order to achieve a seamless large- scale image, the electron microscopy images have to be stitched together at high precision, preferably comparable to the electron microscope resolution.
The registration accuracy to the fluorescence image is much lower and, therefore, the fluorescence image cannot be used as a template for registration of the electron microscopy images to the fluorescence image.
The registration of the electron microscopy images to the fluorescence image, however, must be maintained during stitching together the electron microscopy images.
In respect of the method including scanning a sample by means of a so-called multi-beam electron microscope, it is noted that a first magnification needs to be selected by the user with which the entire sample is scanned by the multi-beam electron microscope.
This magnification and the corresponding resolution may be limited by the allowed pitch between the individual beams in the respective microscope and the number of pixels per image.
When it is desired to scan specific regions at a higher resolution, it is also required to adjust the magnification of the multi-beam microscope locally on the sample, which may require switching to a single-beam operation.
This reduces the throughput of the multi-beam electron microscope adversely.
To maintain the highest possible throughput, it would be beneficial to select the regions for high magnification acquisition automatically during scanning.
This could be done based on for instance light or fluorescence microscopy images, but then it is again essential to maintain the registration.
It is an object of the present invention toat least partially obviate or eliminate at least one of the disadvantages of the methods mentioned above, or to provide an alternative method for recording and correlating light and charged particle microscopy images.
According to a first aspect, the invention provides a method for recording and correlating light and charged particle microscopy images of a sample or one or more sections of a sample, wherein the method comprises the steps of: — recording a light microscopy image of a first field of view; = recording a first charged particle microscopy image of a second field of view at least partially overlapping with the first field of view, wherein the first charged particle microscopy image has a first magnification; — recording multiple second charged particle microscopy images of third fields of view, wherein each of the multiple second charged particle microscopy images has a second magnification which is higher than the first magnification, and wherein the third fields of view are sub- fields of the second field of view; ~ registering the light microscopy image to the first charged particle microscopy image; and — registering the multiple second charged particle microscopy images to the first charged particle microscopy image.
During applying the method according to the invention, the light microscopy image is registered to the first charged particle microscopy image having the first magnification, which is usually a low magnification.
Furthermore, the multiple second charged particle images having the second magnification, which is usually a high magnification, are registered to the first charged particle microscopy image.
As a result, a registration between the light microscopy image and the multiple second charged particle microscopy images 1s accomplished.
The difference between the magnification of the light microscopy image and the first magnification of the first charged particle microscopy image is relatively small, or at least much smaller than the difference between the magnification of the light microscopy image and each of the multiple second charged particle images.
This same applies for the difference between the first magnification of the first charged particle image and the second magnification of each of the multiple second charged particle microscopy images.
As mentioned above,
determination of the registration of multiple high magnification charged particle microscopy images to a light microscopy image is time-consuming and lowers the throughput.
As each of the second charged particle microscopy images 5 contains similar features in similar contrast, albeit at higher magnification, as part of the first charged particle microscopy image, these can be registered using standard image recognition software.
As a result, determination of the registration for each of the second charged particle microscopy images to the first charged particle microscopy image is less time-consuming then determining the registration of each second charged particle microscopy image to the light microscopy image.
As the time-consuming registration between light and charged particle microscopy images now only has to be conducted once for the low magnification image, the method according to the invention results in a higher throughput.
Additionally, the method has the advantage that the first charged particle microscopy image can be used as a template for stitching together the multiple second charged particle microscopy images.
The registration accuracy to the first charged particle images is sufficient to be used as a template for registration of the multiple second charged particle microscopy images.
Advantageously, the registration Of the second charged particle microscopy images to the light microscopy image can be maintained during stitching together the second charged particle microscopy images via the first charged particle microscopy image.
Moreover, registration of the multiple second charged particle microscopy images with the second magnification to the first charged particle microscopy image with the first magnification may allow for a smaller overlap region between the second charged particle microscopy images.
In the context of the present patent application, this overlap region has to be understood as some extra image area around the second charged particle microscopy images that overlaps with the images of the neighboring third fields of view.
This overlap region is used to stitch together all second magnification images and is usually not seen in the final image. A smaller overlap region results in less area to be scanned, therewith increasing the throughput of the method according to the invention.
In an embodiment, the method comprises the steps of dividing a sample to be imaged into multiple areas, and for each of the multiple areas performing the following steps: - recording a light microscopy image of a first field of view; - recording a first charged particle microscopy image of a second field of view at least partially overlapping with the first field of view, wherein the first charged particle microscopy image has a first magnification; - recording multiple second charged particle microscopy images of third fields of view, wherein each of the multiple second charged particle microscopy images has a second magnification which is higher than the first magnification, and wherein the third fields of view are sub- fields of the second field of view; - registering the light microscopy image to the first charged particle microscopy image; and = registering the multiple second charged particle microscopy images to the first charged particle microscopy image. An advantage of this embodiment is that a sample can be imaged within several repetitions of the given steps, such that it is possible to image the sample completely and at the desired/needed magnification(s).
In an embodiment, the light microscopy images of two or more neighboring areas of the sample to be imaged are recorded before recording the first and second charged particle microscopy images of the respective two or more neighboring areas of the sample to be imaged.
In an embodiment, the first field of view covers the second field of view completely. In an embodiment thereof, the second field of view is smaller than the first field of view, so as to leave one or more borders of the first field of view unexposed to charged particles. During imaging a sample by means of at least one charged particle beam, fluorescence present within the sample will be destroyed by means of charged particles of the at least one charged particle beam. This embodiment has the advantage that, if a section or sample is divided into multiple areas to be imaged, the light microscopy images of the multiple areas can be stitched together by means of the one or more borders of the first field of view unexposed to charged particles.
In an embodiment, when light microscopy images of neighboring areas of the sample to be imaged are recorded, the method comprises the further steps of recording a first charged particle microscopy image of a fourth field of view including one of the one or more borders of the first field of view unexposed to charged particles between the neighboring areas; and recording multiple second charged particle microscopy images of fifth fields of view each including a portion of the one border of the first field of view unexposed to charged particles between the neighboring areas. In an embodiment thereof, the method comprises the step of registering the multiple second charged particle microscopy images of the fifth fields of view with the first charged particle microscopy image of the fourth field of view. An advantage of this embodiment is that first and second charged particle microscopy images are obtained for a complete sample or one of more sections of a sample, while maintaining the possibility to register and stitch together multiple light microscopy images of the sample or the one or more sections of the sample.
In an embodiment, when the method comprises the steps of dividing a sample to be imaged into multiple areas, and for each of the multiple areas performing the following steps: recording a light microscopy image of a first field of view; recording a first charged particle microscopy image of a second field of view at least partially overlapping with the first field of view, wherein the first charged particle microscopy image has a first magnification; recording multiple second charged particle microscopy images of third fields of view, wherein each of the multiple second charged particle microscopy images has a second magnification which is higher than the first magnification, and wherein the third fields of view are sub-fields of the second field of view; registering the light microscopy image to the first charged particle microscopy image; and registering the multiple second charged particle microscopy images to the first charged particle microscopy image, the method comprises the step of stitching together the light microscopy images of the imaged multiple areas of the sample. An advantage of this embodiment is that a large-area composite light microscopy image of a sample or of at least one section of a sample can be obtained.
In an embodiment, the method comprises the step of stitching together the multiple second charged particle microscopy images in order to create a large area composite image, preferably before registering the multiple second charged particle microscopy images to the first charged particle microscopy image.
In a further embodiment thereof, adjacent ones of the multiple second charged particle microscopy images have overlapping parts due to partially overlapping third fields of view, and wherein stitching together the multiple second charged particle microscopy images comprises combining the multiple second charged particle microscopy images with the overlapping parts in order to create the large area composite image. In an embodiment thereof, the method comprises the step of, in order to create the large area composite image, using data of the multiple second charged particle microscopy images, in particular of the overlapping parts thereof, which has been acquired first in time. In the context of the present patent application it is noted that contrast in charged particle, such as electron, images may depend on the total applied charged particle, such as electron, dose. Therefore, each of the overlapping parts of adjacent second charged particle microscopy images used to stitch together the adjacent images has a different contrast. The different contrasts may deteriorate the appearance of the final composite image. By using data of the multiple second charged particle microscopy images, in particular of the overlapping parts thereof, which has been acquired first in time, the contrast differences between different parts of the composite image can be decreased, therewith improving the quality of the composite image and the registration accuracy to the first charged particle microscopy image. Thus, the large area composite image has substantially the same contrast over its complete surface, In an embodiment, the method comprises the steps of determining a first drift of at least one light microscopy beam to a sample from previous light microscopy images, determining a second drift of at least one charged particle beam to a sample from a sequence of charged particle images, and/or using the determined first drift and/or second drift for correcting a registration matrix for drift. The registration matrix is used for registration, for example, of the light microscopy image to the first charged particle microscopy image. This embodiment has the advantage that by including a correction for the drift of the at least one light microscopy beam to the sample into the registration matrix, registration and correction for drift can be performed in a single event.
In an embodiment, the method comprises, before the step of recording a first charged particle microscopy image, the steps of recording one or more high-resolution or super- resolution, preferably fluorescence, light microscopy images of a sixth field of view equivalent to or smaller than the first field of view, and registering the one or more high- resolution or super-resolution images to the light microscopy image. In an embodiment thereof, the method comprises the steps of detecting a predetermined fluorescence signal on a sample, and the step of recording a high-resolution or super- resolution, preferably fluorescence, light microscopy image comprises recording a high-resolution or super-resolution,
preferably fluorescence, light microscopy image of those areas of the sample where the predetermined fluorescence signal has been detected. An advantage of this embodiment is that at least of areas of the sample where the predetermined fluorescence signal has been detected, light microscopy images can be taken with a higher resolution than the resolution imposed by the diffraction limit. As a result, high-resolution light microscopy images of the sample can be taken and, subsequently, registered indirectly to the second charged particle microscopy images having a high magnification.
In an embodiment, light microscopy image(s), first charged particle microscopy image(s) and second charged particle microscopy images are recorded for multiple sections of a sample and/or the method is repeated for one or more following sections. In an embodiment thereof, these sections have been consecutively cut from a 3D sample and thus data acquired for each section of the one or more sections of the sample is used to create a 3D stack of registered, composite images.
In an embodiment, the steps of recording the light microscopy image(s) and of recording the first charged particle microscopy image(s) are performed on an integrated microscope assembly, and the step of recording the multiple second charged particle microscopy images is performed on a multi-beam charged particle system. A multi-beam charged particle microscopy system has the possibility to scan large areas at once, because of the multiple beams. The time required for recording the multiple second charged particle microscopy images can be shortened in comparison with a single beam charged particle microscopy. Thus, this embodiment reduces the time needed for recording and correlating light and charged particles images of a sample or one or more sections of a sample.
In another embodiment, the step of recording the first charged particle microscopy image(s) is performed on a multi-beam scanning electron microscope (SEM) and the step of recording the multiple second charged particle microscopy images is performed on a single-beam SEM, or the multi-beam SEM operating in single-beam mode, or the multi-beam SEM operating in a higher resolution or smaller pitch mode.
In an embodiment, the method comprises the step of determining registration of the light microscopy image to the first charged particle microscopy image by means of a previously determined registration procedure. This allows a further increase in throughput as the time-consuming light to charged particle microscopy registration does not need to be conducted for each pair of first and second or fourth fields of view, but can for instance be calculated from the previously determined registration procedure and the known sample stage or charged particle beam translation coordinates and the determination of the drift. Using a previously determined registration procedure may also allow a light microscope, to be separate from a charged particle microscope. This allows the method according to this embodiment to be performed on conventional light and charged particle microscopes.
According to a second aspect, the invention provides an integrated microscopy assembly for imaging a sample, comprising: - at least one optical microscope configured to observe one or more regions of interest on the sample; - at least one charged particle microscope, such as an ion or electron microscope, to emit and to focus at least one charged particle beam on one or more positions on the sample; and = an image processing device, wherein the image processing device is configured for performing the steps of the method according to any one of the preceding claims.
The integrated microscopy assembly according to the invention has at least the same advantages as described in relation to the method according to the first aspect of the invention. The various aspects and features described and shown in the specification can be applied, individually, wherever possible. These individual aspects, in particular the aspects and features described in the attached dependent claims, can be made subject of divisional patent applications.
BRIEF DESCRIPTION OF THE DRAWINGS The invention will be elucidated on the basis of an exemplary embodiment shown in the attached drawings, in which: Figure 1 shows a schematic overview of an integrated microscopy system according to an embodiment of the invention; Figure 2 shows a schematic overview of a method for recording and imaging light and charged particle microscopy images according to a first embodiment of the invention; Figures 3A and 3B schematically show a top view of a sample to be imaged with indicated a first and second field of view, and thirds fields of view; and a top view of two adjacent recorded charged particle images, respectively; and Figure 4 shows a schematic overview of a method for recording and imaging light and charged particle microscopy images according to a second embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION In order to elucidate the context of the present invention, a principle set-up of a so-called optical SEM combination is provided by way of example. With reference to figure 1, the basic design of an integrated microscopy system 1 is explained. It is noted that the shown example of the integrated microscopy system 1 is only included for the purpose of exemplifying the principle of the integrated microscopy system 1 and does not exclude any alternative or unknown variations of such an integrated microscopy system
1. Specifically, the system may be of a non-confocal type, for instance wide-field type, it may include a light beam to excite luminescence in the sample, or it may have multiple individual beams.
Figure 1 shows an integrated microscopy system 1 comprising at least an optical microscope 2, 3, 4 to observe one or more regions of interest on a sample 10 on a non-shown sample holder, and a charged particle microscope, such as an ion or electron microscope 7, 8 to emit and to focus at least one charged particle beam 9 on one or more positions on the sample 10 within the one or more regions of interest on the sample 10.
The charged particle microscope is provided with a charged particle source 7 for emitting a primary charged particle beam 9 and to focus said charged particle beam 9 on the sample 10 within the one or more regions of interest. Further, the charged particle microscope comprises a detector 8 for detection of secondary radiation or secondary charged particles 11 backscattered from the sample 10, or emitted, transmitted, or scattered from the sample 10 and possibly induced by the primary beam 9. The charged particle microscope 7, 8 is arranged within a vacuum chamber 13.
The optical microscope 2, 3, 4 is equipped with a light collecting device 2 to receive in use luminescence light 12 emitted by the sample and/or substrate 10 and induced by the primary beam 9 and to focus the luminescence light on a photon-detector 4. The light collecting device 2 may be an objective, a mirror or a glass fiber. It may also consist of a plurality of devices to arrange for collecting and focusing of the concerning luminescence light 12 that is emitted by the sample 10, e.g. using a known per se CCD camera.
In the present example the optical microscope 2, 3, 4 is of the confocal type having a pinhole 3 between the light collecting device 2 and the photon-detector 4. In this embodiment the optical microscope 2, 3, 4 is placed entirely inside the vacuum chamber 13 of the charged particle microscope 7, 8. The closed dashed line 14 encircles those parts of the integrated microscopy assembly 1 according to the invention that may be mounted on the (replaceable) door of the vacuum chamber 13, notably the sample holder for the sample 10, the light collecting device 2, the pinhole 3, and the photon-detector 4.
As shown in figure 1, the integrated microscopy system 1 is provided with an image processing device 30, alternatively denoted as controller or processor. The image processing device 30 is useable as an automation unit and may be in the form of a computer, including a personal computer provided with dedicated software. The control unit 30 may have with one or more screens, e.g. one screen or screen part for depicting the recorded optical image, and another screen or another part of the same screen depicting an image, in particular of the same sample or substrate, recorded via the charged particle part of the integrated microscopy assembly
1. In general, the image processing device is among others configured for implementing one or more methods of use and controlling the integrated microscopy assembly 1. In particular, the image processing device 30 is configured for performing a method for recording and correlating light and charged particle microscopy images according to an embodiment Of the invention.
Figure 2 shows a schematic overview of the steps of the method for recording and correlating light and charged particle microscopy images according to an embodiment of the invention. As a first step, the method comprises the step of arranging S1 a sample holder with the sample 10 placed thereon within the integrated microscopy assembly 1, for example such as shown in figure 1. Subseguently, the method comprises the step of recording S2, by means of the optical microscope 2, 3, 4, a light microscopy image of a first field of view 35 on the sample 10, as schematically indicated in figure 3A and of which the right border is indicated by means of the dashed line. The recorded light microscopy image is transmitted to the image processing device 30 and, optionally, shown on at least one of the one or more screens thereof. After the light microscopy image has been recorded, the step of recording S3 a first charged particle image of a second field of view 36 on the sample 10 is performed. The first charged particle image is recorded at a first magnification, in particular a low magnification. As schematically indicated in figure 3A, the second field of view 36 overlaps with the first field of view 35 and is slightly smaller than the first field of view 35, such that an unexposed border of fluorescence 37 remains. The unexposed border of fluorescence 37, for example, can be used for stitching adjacent light microscopy images of a set of light microscopy images.
The step of recording S3 a first charged particle image is followed by a step of recording S4 multiple second charged particle microscopy images of third fields of view
38. Each of the multiple second charged particle microscopy images is recorded at a second magnification, in particular a high magnification. In the context of the present patent application, it implies that the higher the magnification, the higher the number of details within the recorded images. The third fields of view 38 are located within the second field of view 36 and are sub-fields of the second field of view 36. The third fields of view 38 may fill the second field of view 36 completely or partially, depending on a region of interest of the sample 10.
After acquiring the images described above, the method comprises the step of registering S5 the light microscope image to the first charged particle image. The difference in magnification of the light microscope image and the magnification of the first charged particle image is relatively small.
Additionally, the method also comprises the step of stitching together S6 the multiple second charged particle images to generate a single high-magnification image, followed by registering S7 the single high-magnification image to the first charged particle image. During stitching together the multiple second charged particle images, adjacent images with overlapping parts 39 due to partially overlapping third fields of view 38, as is schematically indicated in figure 3B, are combined in order to generate the single high-magnification image of the multiple second charged particle images. In order to create the multiple second charged particle images which can be stitched together, the overlapping parts 39 of the third fields of view 38, as indicated in figure 3B, are exposed to the at least one charged particle beam 9 multiple times, usually two times. Preferably the overlapping part 39 of each of the second charged particle images 39 which is exposed to the at least one charged particle beam 9 only once is used for generating the single high-resolution image.
Furthermore, the difference between the magnification of the first charged particle image and the magnification of the single high-magnification image formed from the multiple second charged particle images is relatively small. This contributes advantageously to the registration result of the single high-magnification image, i.e. the stitched together multiple second charged particle images, to the first charged particle image.
As a result of the method according to this embodiment of the invention the stitched together multiple charged particle images are registered to the light microscopy image by means of the first charged particle image with a sufficient registration accuracy. Additionally, since registration of the multiple second charged particle images to the first charged particle image is less time-consuming than registration of the multiple second charged particle images to the light microscopy image, the method according to this embodiment of the invention requires less time for acquiring the registration result.
Optionally, the sample 10 to be imaged has to be imaged in multiple steps in order to cover the complete surface of the sample 10 or to acquire high-resolution images.
In that case, steps S2 —- S57 are repeated for two or more areas, as schematically indicated by the first fields of view 35 in figure 3A, wherein the facing unexposed borders of fluorescence 37 of adjacent first fields of view 35 form a fluorescence overlapping part 40. The fluorescence overlapping part(s) 40 can be used for stitching together all light microscopy images which are made of the sample 10, such that a large-area light microscopy image of the sample 10 can be obtained.
When steps S2 — 87 are completed for adjacent first fields of view 35, the fluorescence overlapping part 40 is imaged by means of the charged particle microscope 7,9. In particular, firstly a first charged particle image of the respective fluorescence overlapping part 40 is recorded, as in step S3, followed by recording, as in step $54, multiple second charged particle images of the respective fluorescence overlapping part 40. The first charged particle image of the respective fluorescence overlapping part 40 is stitched together to the first charged particle images of the adjacent areas in the same manner as described in relation to step 56, in order to realize a single low-magnification charged particle image of the two or more areas. The multiple second charged particle images of the respective fluorescence overlapping part 40 are stitched together and to the multiple second particle images of the one or more adjacent areas, as in step S86, in order to create a single high-magnification charged particle image. Following thereon, the single high- magnification charged particle image is registered to the low-magnification charged particle image and to the large- area light microscopy image. As a result, the single high- magnification charged particle image is registered to the large-area light microscopy image.
Figure 4 shows a schematic overview of the steps of the method for recording and correlating light and charged particle microscopy images according to another embodiment of the invention. The method comprises substantially the same steps as the method schematically indicated in figure 2. For the sake of brevity, refrained is from reintroducing similar steps and, therefore, similar steps are indicated by the same reference number increased with 100.
The method schematically indicated in figure 4 differs from the previous described method in that this method comprises the step of recording S108 one or more high- or super-resolution light microscopy images of at least a part of the sample 10, in particular of those parts of the sample where a predetermined fluorescence signal has been 10 detected in the light microscopy image. It is noted that this step S108 is performed after step $5102 and before step S103. Furthermore, the recorded one or more high- or super- resolution light microscopy images are registered S109 to the light microscopy image(s) and, therewith, to the multiple second charged particle images.
It is noted that the steps of the embodiments of the method according to the invention can be performed on various sections, for example, of a biological specimen to be imaged, wherein each section corresponds to the sample 10 as described above. When each of the various sections is imaged, the recorded and registered images can be used for creating a 3D stack of registered, composite images.
In the context of the present application, it is noted that alternative to the integrated microscopy system 1 as shown in figure 1, the first charged particle image(s) can be acquired by means of a multi-beam scanning electron microscope (SEM), and the second charged particle images can be acquired by means of a single-beam SEM, the multi-beam SEM operating in a single-beam mode, or the multi-beam SEM operating in a higher resolution or smaller pitch mode in comparison with recording the first charged particle image (s).
It is to be understood that the above description is included to illustrate the operation of the preferred embodiments and is not meant to limit the scope of the invention. From the above discussion, many variations will be apparent to one skilled in the art that would yet be encompassed by the scope of the present invention.
Claims (20)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
NL2024861A NL2024861B1 (en) | 2020-02-07 | 2020-02-07 | Method for recording and correlating light and charged particle microscopy images |
EP21708080.3A EP4100985A1 (en) | 2020-02-07 | 2021-02-05 | Method for recording and correlating light and charged particle microscopy images |
PCT/NL2021/050075 WO2021158111A1 (en) | 2020-02-07 | 2021-02-05 | Method for recording and correlating light and charged particle microscopy images |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
NL2024861A NL2024861B1 (en) | 2020-02-07 | 2020-02-07 | Method for recording and correlating light and charged particle microscopy images |
Publications (1)
Publication Number | Publication Date |
---|---|
NL2024861B1 true NL2024861B1 (en) | 2021-09-13 |
Family
ID=70155270
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
NL2024861A NL2024861B1 (en) | 2020-02-07 | 2020-02-07 | Method for recording and correlating light and charged particle microscopy images |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4100985A1 (en) |
NL (1) | NL2024861B1 (en) |
WO (1) | WO2021158111A1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040217297A1 (en) * | 2000-12-01 | 2004-11-04 | Yeda Research And Development Co. Ltd. | Device and method for the examination of samples in a non vacuum environment using a scanning electron microscope |
US20100091362A1 (en) * | 2008-10-15 | 2010-04-15 | Kabushiki Kaisha Topcon | Complex type microscopic device |
US20180053627A1 (en) * | 2011-05-13 | 2018-02-22 | Fibics Incorporated | Microscopy imaging method and system |
-
2020
- 2020-02-07 NL NL2024861A patent/NL2024861B1/en active
-
2021
- 2021-02-05 WO PCT/NL2021/050075 patent/WO2021158111A1/en unknown
- 2021-02-05 EP EP21708080.3A patent/EP4100985A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040217297A1 (en) * | 2000-12-01 | 2004-11-04 | Yeda Research And Development Co. Ltd. | Device and method for the examination of samples in a non vacuum environment using a scanning electron microscope |
US20100091362A1 (en) * | 2008-10-15 | 2010-04-15 | Kabushiki Kaisha Topcon | Complex type microscopic device |
US20180053627A1 (en) * | 2011-05-13 | 2018-02-22 | Fibics Incorporated | Microscopy imaging method and system |
Non-Patent Citations (3)
Title |
---|
NALAN LIV ET AL: "Simultaneous Correlative Scanning Electron and High-NA Fluorescence Microscopy", PLOS ONE, vol. 8, no. 2, 8 February 2013 (2013-02-08), pages e55707, XP055666091, DOI: 10.1371/journal.pone.0055707 * |
SCHORB MARTIN ET AL: "New hardware and workflows for semi-automated correlative cryo-fluorescence and cryo-electron microscopy/tomography", JOURNAL OF STRUCTURAL BIOLOGY, ACADEMIC PRESS, UNITED STATES, vol. 197, no. 2, 28 June 2016 (2016-06-28), pages 83 - 93, XP029894438, ISSN: 1047-8477, DOI: 10.1016/J.JSB.2016.06.020 * |
VAN DRIEL L F ET AL: "Tools for correlative cryo-fluorescence microscopy and cryo-electron tomography applied to whole mitochondria in human endothelial cells", EUROPEAN JOURNAL OF CELL BIOLOGY, WISSENSCHAFLICHE VERLAGSGESELLSCHAFT, STUTTGART, DE, vol. 88, no. 11, 1 November 2009 (2009-11-01), pages 669 - 684, XP026585829, ISSN: 0171-9335, [retrieved on 20090902], DOI: 10.1016/J.EJCB.2009.07.002 * |
Also Published As
Publication number | Publication date |
---|---|
WO2021158111A1 (en) | 2021-08-12 |
EP4100985A1 (en) | 2022-12-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8314837B2 (en) | System and method for imaging with enhanced depth of field | |
US20110090327A1 (en) | System and method for imaging with enhanced depth of field | |
EP1762878B1 (en) | Observation apparatus and observation method | |
EP2273302B1 (en) | Image acquiring apparatus, image acquiring method and image acquiring program | |
US7885447B2 (en) | Image acquiring apparatus including macro image acquiring and processing portions, image acquiring method, and image acquiring program | |
JP7252190B2 (en) | System for generating enhanced depth-of-field synthetic 2D images of biological specimens | |
US20110091125A1 (en) | System and method for imaging with enhanced depth of field | |
EP2834832B1 (en) | Integrated optical and charged particle inspection apparatus | |
US9378921B2 (en) | Integrated optical and charged particle inspection apparatus | |
NL2013262B1 (en) | Method for inspecting a sample using an assembly comprising a scanning electron microscope and a light microscope. | |
EP2618197A1 (en) | Device for observing three-dimensional confocal point, and observation focal plane displacement/correction unit | |
US7709773B2 (en) | Scanning optical device which sets hardware in association with a depth of a focus position based on hardware set values stored in association with depths of focus positions | |
AU2014236055A1 (en) | Referencing in multi-acquisition slide imaging | |
US20200379232A1 (en) | Optical scanning arrangement and method | |
JP4883936B2 (en) | Image processing method and apparatus for scanning cytometer | |
JP6131568B2 (en) | Microscope device and image forming method | |
JP5963487B2 (en) | Laser microscope and observation method | |
NL2024861B1 (en) | Method for recording and correlating light and charged particle microscopy images | |
EP2263070B1 (en) | Synchronized imaging by way of an optical method and atomic force microscopy | |
JP7287957B2 (en) | Radiation detection device, computer program and positioning method | |
JP4083193B2 (en) | electronic microscope | |
Crosby et al. | Multi-beam SEM technology for high throughput imaging | |
Woo et al. | High-throughput high-dynamic range imaging by spatiotemporally structured illumination | |
US20220335580A1 (en) | Method for producing a corrected image and apparatus | |
WO2018131173A1 (en) | Image processing device, microscope system, image processing method, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PD | Change of ownership |
Owner name: DELMIC IP B.V.; NL Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), ASSIGNMENT; FORMER OWNER NAME: TECHNISCHE UNIVERSITEIT DELFT Effective date: 20230118 |