WO2025155992A1

WO2025155992A1 - Systems and methods of deep-learning enhanced dark-field microscopy imaging

Info

Publication number: WO2025155992A1
Application number: PCT/US2025/012474
Authority: WO
Inventors: Zhaowei Liu; Ming Lei; Junxiang Zhao
Original assignee: University of California Berkeley; University of California San Diego UCSD
Current assignee: University of California Berkeley; University of California San Diego UCSD
Priority date: 2024-01-19
Filing date: 2025-01-21
Publication date: 2025-07-24
Anticipated expiration: 2026-07-19

Abstract

In some implementations, there is provided a method of imaging an object using dark-field microscopy, the method comprising acquiring a ground truth image of the object, simulating a plurality of dark field images of the object based on the ground truth image, training a machine learning model using the plurality of simulated dark field images and the ground truth image, acquiring a dark field image of the object, inputting the dark field image of the object into the trained machine learning model, and outputting, from the trained machine learning model, a highresolution dark field image of the object.

Description

SYSTEMS AND METHODS OF DEEP-LEARNING ENHANCED DARK-FIELD

MICROSCOPY IMAGING

Technical Field

[0001] The present disclosure relates to methods of deep-learning enhanced dark-field microscopy imaging.

Cross-Reference to Related Applications

[0002] This application claims the benefit of priority under 35 U.S.C. § 119(e) to U.S. Provisional application number 63/622,907, filed January 19, 2024 and titled “SUPER RESOLUTION LABEL-FREE DARK-FIELD MICROSCOPY BY DEEP LEARNING”, the contents of which are hereby incorporated by reference in their entirety.

Background

[0003] Dark-field microscopy (DFM) is a powerful label-free and high-contrast imaging technique due to its ability to reveal features of transparent specimens with inhomogeneities. However, owing to the Abbe diffraction limit, fine structures at sub -wavelength scale are difficult to resolve. Herein, a single-image super-resolution DFM scheme is described using a convolutional neural network (CNN). A U-net-based CNN is trained with a dataset which is numerically simulated based on the forward physical model of the DFM. The forward physical model described by the parameters of the imaging setup connects the object ground truths and dark field images. With the trained network, there is described herein the super-resolution dark-field imaging of various test samples that attains twice the resolution of traditional imaging methods. The disclosed technique illustrates a promising deep learning approach to double the resolution of DFM with little (if any) any hardware modification.

[0004] Label-free imaging techniques have been developed for many decades and are widely applied in the life sciences due to their non-invasive and non-toxic approaches to biological samples. Dark-field microscopy (DFM) is an important label-free imaging method used in biology, material science, and other disciplines. Dark-field microscopy offers high-contrast imaging for a wide range of unstained specimens. In DFM, the illumination light beam becomes hollow and oblique after passing through the dark-field ring and a condenser, in which case only the light scattered by the objects can be detected. As a result, compared with bright-field microscopy, DFM emphasizes high spatial frequency information and enhances image contrast by highlighting the discontinuities or edges of the object, while the uniform regime remains dark. Due to its simple configuration and effective imaging performance, DFM is widely used in revealing outlines of unstained objects. However, the resolution of traditional DFM is diffractionlimited by the detection optics.

[0005] Although many excellent super-resolution imaging methods have emerged in the past few decades, most of them focus on fluorescent imaging and cannot be applied to scattering DFM. Structured illumination microscopy (SIM) is a successful super resolution fluorescent imaging technique using a series of excitation light patterns to encode the high-resolution information into the observed images. Label-free SIM has also been developed in recent years for super resolution imaging but with much less impressive resolution improvement. Fourier ptychographic microscopy using a light emitting diode array to illuminate the label-free sample with different angles achieves resolution improvement with synthesized aperture, which could be potentially applied to super-resolution DFM. However, both of them require large modification of the DFM setup and need multiple images to reconstruct a super-resolution image.

Summary

[0006] In some example embodiments, there may be provided a super resolution label-free dark-field microscopy by machine learning, such as neural networks, deep learning, and the like, that allows for reconstruction of a high-resolution image of a target based on a single shot low- resolution image of the target.

[0007] According to an embodiment, there is method of imaging an object using dark-field microscopy, the method includes acquiring a ground truth image of the object, simulating a plurality of dark field images of the object based on the ground truth image, training a machine learning model using the plurality of simulated dark field images and the ground truth image, acquiring a dark field image of the object, inputting the dark field image of the object into the trained machine learning model, and outputting, from the trained machine learning model, a high- resolution dark field image of the object.

[0008] In some implementations, acquiring the ground truth image of the object includes simulating the ground truth image of the object. In certain implementations, the ground truth image includes a super-resolution image of the object. In further implementations, the ground truth image of the object is used to determine a manner of interaction between the object and light. [0009] In some implementations, the plurality of dark field images of the object include low- resolution images of the object and high-resolution images of the object, and wherein the plurality of dark field images are simulated based on a plurality of parameters describing an imaging system. [0010] In certain implementations, the machine learning model is a neural network. In further implementations, training the machine learning model using the plurality of simulated dark field images and the ground truth images includes adjusting at least a first weight corresponding to a layer of a plurality of layers of the machine learning model. In some implementations, acquiring the dark field image of the object includes imaging the object using an imaging system, the imaging system comprising a substrate, a light-emitting layer, and a metal layer.

[0011] In certain implementations, the metal layer is configured to generate surface plasmonic polaritons, and the surface plasmonic polaritons are configured to illuminate the object such that the dark field image of the object encodes high-frequency information that permits the outputting of a high-resolution dark field image of the object.

[0012] In further implementations, the machine learning model includes a plurality of layers, training the machine learning model includes adjusting a weight of a layer of the plurality of layers of the machine learning model, and the weight of the layer of the plurality of layers of the machine learning model corresponds to a weight of a feature of the high-resolution dark field image of a particular scale.

[0013] Non-transitory computer program products (e g., physically embodied computer program products) are also described that store instructions, which when executed by one or more data processors of one or more computing systems, causes at least one data processor to perform operations herein. Similarly, computer systems are also described that may include one or more data processors and memory coupled to the one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. In addition, methods may be implemented by one or more (data) processors either within a single computing system or distributed among two or more computing systems. Such computing systems may be connected and may exchange data and/or commands or other instructions or the like via one or more connections, including a connection over a network (e.g., the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, or the like), via a direct connection between one or more of the multiple computing systems, etc. [0014] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims.

[0015] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims.

Brief Description of the Drawings

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the disclosed implementations. In the drawings,

[0017] FIG. 1 A illustrates an imaging system in accordance with some embodiments described herein;

[0018] FIG. IB illustrates a substrate system in accordance with some embodiments described herein;

[0019] FIG. 2 illustrates a process used to image a target in accordance with some embodiments described herein;

[0020] FIG. 3 illustrates a Fourier space representation of the numerical aperture of various imaging systems in accordance with some embodiments described herein;

[0021] FIG. 4 illustrates, for a plurality of numerical aperture ratios, a captured first image, a captured second image, and an output image in accordance with embodiments described herein; and

[0022] FIG. 5 illustrates a computing system configured to implement methods of imaging in accordance with some embodiments described herein.

Detailed Description

[0023] In recent years, deep learning (DL) has dramatically influenced the optical imaging field and gained great success in solving the complex inverse problem. In contrast to traditional reconstruction methods (which typically require explicit mathematical models to find an analytical solution), the DL neural network relies on large datasets to learn by itself to solve the complex inverse problems. Well-designed neural networks have been successfully implemented on reconstructions of many super-resolution imaging techniques.

[0024] Artificial neural network accelerated photoactivated localization microscopy (PALM) reconstructs a PALM image using a smaller number of sub-images with similar performance, which improved the imaging speed by 26 times. A generative adversarial network (GAN) based network transforms diffraction-limited input images into super-resolved ones. These neural networks improve the performance of existing imaging methods in either speed or resolution.

[0025] Nevertheless, in order to optimize their weight and bias parameters for better performance, these neural networks require large training datasets which often can only be acquired via laborious experiment work. It’s also worth noting that researchers have proposed neural networks with no pre-training requirement in certain imaging situations. For example, by embedding the forward physical model of imaging process, PhysenNet reconstructs a phase image from a single diffraction pattern. Ghost imaging using deep neural network constraint (GIDC) was proposed to increase the spatial resolution in ghost imaging. These physics-informed neural networks avoid the training process at the expense of slow reconstruction speed due to iteration processes, which makes real-time super resolution difficult to achieve.

[0026] Disclosed herein is a machine learning model-based framework specifically designed to improve the resolution of single-frame DFM images in real time by retrieving the high spatial frequency information acquired by large-wavevector illumination without any modification to a standard DFM setup. The framework described herein may use, for example, a convolutional neural network (CNN) to improve the resolution of single-frame DFM images. Using a physicsbased DFM forward process, the training dataset are numerically generated based on the parameters of the optical setup and require no experimental images. The network is trained on the simulated dataset to retrieve the high spatial frequency information which is encoded in the dark field image. By applying this framework to single-frame images, reconstruction of high-resolution images can be attained with increased speed. This framework could be applied to any DFM setup for super-resolution with only the knowledge of its illumination and detection parameters, which provides an appealing method for real-time label-free super resolution dark field imaging.

[0027] FIG. 1A illustrates an imaging system 100 that may be used capture dark-field microscopy images of a target 120. The imaging system 100 may comprise a light source 102, a dark field ring 104, a condenser 106, a cover slip 108, a slide 110, an objective 112, and a charge- coupled device (CCD) 114. The imaging system 100 may further comprise a sample or target 120 disposed between the cover slip 108 and the slide 110. The imaging system is configured to illuminate, via the light source 102, the target 120, and is further configured to detect an image, via the charge-coupled device 114, of the target 120.

[0028| J In some implementations, the light source 102 comprises a laser. For example, the light source 102 may comprise a 532 nm laser guided by a multi-mode fiber. In some implementations, the CCD 114 may comprise a camera. For example, the CCD 114 may comprise an iXon 897 by Andor to record or detect images.

[0029] In DFM, the central part of the illumination is blocked by the dark field ring, as shown in Fig. 1 A. After passing through the condenser, the illumination light becomes oblique and strikes the sample at an illumination angle 113, in which case the objective can only collect the scattered light from the object. Following the idea of synthetic aperture, the angled illumination pattern corresponding to a ring in Fourier space shifts the high spatial frequency information of the object into the detection bandwidth of the objective, as discussed with respect to FIG. 3.

[0030] FIG. IB illustrates an imaging system 150 that may be used to image a target 120. The target 120 of the imaging system 150 may be placed on a substrate system 151. The substrate system 151 comprises a substrate 152, a light-emitting layer 153, and a metal layer 154. The lightemitting layer may be disposed on a surface of the substrate 152. The metal layer 154 may be disposed on a surface of the light-emitting layer 153. The target 120 may be disposed on the metal layer 154. The substrate system 151 can be used to image a target 120 using an effective numerical aperture that is greater than the numerical aperture limit of standard imaging systems, such as standard microscopes and the imaging system 100 of FIG. 1A.

[0031] The substrate 152 may comprise glass. The light-emitting layer 153 may comprise a material that is configured to transmit incident light 160 to the metal layer 154. For example, the light-emitting layer 153 may comprise a fluorophore configured to re-emit light upon excitation by incident light 160. The fluorophore can absorb light at a first frequency and re-emit the light at a second frequency. The second frequency may be lower than the first frequency. In some implementations, the light-emitting layer 153 comprises a rhodamine 6G (R6G) polymer. In certain implementations, the light-emitting layer 153 comprises a light-emitting diode (LED). In further implementations, the light-emitting layer 153 comprises an OLED. The light-emitting layer 153 comprises a material configured to transmit incident light 160 towards the metal layer 154 so that surface excitations can be generated in the metal layer 154.

[0032] In some implementations, the metal layer 154 comprises a metal film. In further implementations, the metal layer 154 comprises gold. In certain implementations, the metal layer 154 comprises silver. In further implementations, the metal layer 154 comprises an alloy of gold and silver. In some implementations, the metal layer 154 comprises a plurality of layers, each of the plurality of layers comprising at least one of gold, silver, or an alloy of gold and silver. In some implementations, the plurality of layers of the metal layer 154 comprise alternating layers of gold and silver. The metal layer 154 comprises any material in which surface plasmonic polaritons (SPPs) can be generated. Imaging the target 120 using surface plasmonic polaritons, rather than using incident light, allows for the effective numerical aperture of the imaging system 150 to be increased relative to the numerical aperture of conventional imaging systems (e.g., imaging system 100 of FIG. 1A).

[0033] As shown in FIG. IB, incident light 160 is emitted towards the substrate system 151. The incident light 160 may be emitted from an illumination source. The incident light 160 may be incident on the substrate 152 at a first illumination angle 163.

[0034] The substrate 152 guides the incident light 160 through the light-emitting layer 153 towards the metal layer 154. The incident light 160 is configured to excite the light-emitting layer 153. Light-emitting layer 153 might be in form of an organic light-emitting diode (OLED), which can be excited with electrical power and does not require incident light 160. The excited lightemitting layer transmits the excitation to the metal layer 154 to generate surface plasmon polaritons (e.g., surface plasmonic waves) in the metal layer 154. The geometry of the metal layer (e.g., the thickness of the metal layer or the inclusion of alternating metals within the film) can be configured to allow for the excitation of surface plasmonic waves of a particular mode.

[0035] The surface plasmonic waves can be used to illuminate the target 120. By using the surface plasmonic waves, rather than visible light and lenses, to illuminate the target 120, the numerical aperture of the imaging system 150 comprising the substrate system 151 is increased relative to the numerical aperture of a standard microscope. The target 120 can then be imaged by, for example, a charge-coupled device, such as CCD 114 of FIG. 1A.

[0036] The incident light 160 may be configured via selection of the appropriate parameters (e.g., wavelength, first illumination angle 163, intensity, pulse duration, and pulse frequency) to excite surface plasmonic waves in the metal layer 154 having a particular wavevector. The wavevector of the surface plasmonic waves in the metal layer 154 is proportional to the numerical aperture of the imaging system 150. By selecting light of the appropriate parameters, the wavevector of the surface plasmonic waves can be made to be greater than the numerical aperture limit of standard microscopes. This allows for higher resolution imaging (e.g., via surface plasmonic waves) of the target than is attainable via imaging with visible light. The effective numerical aperture, and thus the image resolution, can be further improved by using a higher illumination lateral wavevector to extract higher spatial frequency information of the target 120. For example, localized plasmonic structures may be used to excite the target 120 with a higher wavevector for super-resolution imaging. In other words, for super-resolution imaging, the target 120 may be illuminated by surface plasmonic polaritons having large wavevectors.

[0037] The environment of the substrate system 151 may also be selected so as to permit the generation of surface plasmonic waves of a particular mode. For example, the substrate system 151 may be disposed in, e.g., water or air, so that surface plasmonic waves of a particular mode can be used to image the target 120.

[0038] The substrate system 151 can be used to perform single-shot imaging of the target 120, rather than using multiple captured images of the target 120. By using a machine learning model to minimize a difference between a simulated ground truth image and an image captured using the substrate system 151, the embodiments described herein allow for a high-resolution image to be constructed while capturing only a single image. This allows the high-resolution image of the target 120 to be reconstructed with a faster speed than would be possible if multiple images of the target 120 needed to be captured. Further, this allows for the reconstruction of an image having a higher resolution than that which is attainable by use of the imaging system 100 of FIG. 1 A.

[0039] FIG. 2 illustrates a process 200 in accordance with some embodiments described herein. The process 200 may be used to image a target using an imaging system. For example, the imaging system 150 may be configured to image a target 120 using process 200.

[0040] At 201 of the process 200, a ground truth image of a target is acquired. The ground truth image of the target may be numerically simulated. The ground truth image of the target that is numerically simulated may be a high-resolution image of the target. As an example, a ground truth image of the target 120 of FIG. IB may be simulated or acquired. [0041] The ground truth image of the target can be used to determine a manner in which the light emitted by the illumination source of the imaging system interacts with the target. Knowledge of the interaction between target and the light emitted by the illumination source can be used to construct a high-resolution image and a low-resolution image of the target.

[0042] Collecting a sufficient training image dataset is crucial for ensuring the quality of network training. However, this process can be very time-consuming for many imaging technologies. In super-resolution microscopy, obtaining ground truth images is particularly challenging. Given that the forward imaging model, including the manner of interaction between light and the target, can be known from the imaging setup and the ground truth image of the target that is acquired at 201 of the process 200, it is possible to employ a simulation method to generate training data that can be used to estimate the low resolution plasmonic dark-field image of any given object.

[0043] To this end, at 202 of the process 200, a plurality of images of the target are simulated. The plurality of images of the target may be numerically simulated. The plurality of images of the target may comprise both low-resolution images of the target and high-resolution images of the target. The plurality of images of the target may be simulated dark field microscopy images of the target. The plurality of images of the target may be simulated based on the ground truth image of the target that is simulated at 201 of the process 200. The plurality of images of the target may be simulated using a simulation method that considers parameters of an imaging system used to image the target. For example, an imaging system used to image a target may be described by a plurality of parameters (e.g., parameters describing the geometry or optical configuration of the imaging system). The plurality of images of the target may be simulated using a simulation that considers a plurality of parameters of the imaging system. For example, the plurality of high-resolution images of the target may include images simulated at all possible illumination angles of the target for a given imaging system.

[0044] In some implementations, the plurality of images of the target may be simulated based on the optical and geometric parameters of the imaging system 100. In certain implementations, the plurality of images of the target may be simulated based on the optical and geometric parameters of the imaging system 150. In further implementations, the plurality of images of the target may be simulated based on the optical and geometric parameters of both of the imaging systems 100 and 150. The plurality of images of the target may be simulated based on the illumination numerical aperture and the detection numerical aperture of the imaging system. Based on the illumination numerical aperture and the detection numerical aperture, object information can be shifted from the high-frequency range to the low-frequency range during simulation. This facilitates the simulation of low-resolution dark field images. For high-resolution target images, the high-frequency range defined by the combined illumination numerical aperture and the detection numerical aperture can be utilized directly to simulate high-resolution images.

[0045] For example, the imaging system 100 of FIG. 1A may comprise a 45° illumination angle and a 50X/0.55 NA objective. Under incoherent light illumination, the dark field image produced by imaging system 100 can be regarded as a superposition of multiple off-axis images generated by each point source of illumination on the illumination dark field ring. Each angled illumination shifts the corresponding high-spatial frequency information of the object into the detection bandwidth of the objective, which is a low-pass filter in the Fourier plane. In implementations in which the substrate system used to image the target comprises a fluorophore, incoherent illumination can be considered so that the final low-resolution dark field image ILR can be regarded as a sum of images under each angled illumination and approximately calculated as: where Eiii_u,i=8 f_Xii> f_y, is the Fourier transform of the illumination electrical field of the /-th angle, f_x and f_y are coordinates in Fourier space, f_{x i} and f_{y i} are illumination position in Fourier space, m-3Q is the total number of illumination angles, <8> represents convolution operation, O is the object spatial information, F¹ represents the inverse Fourier transform, CTF is the coherent transfer function of the detection optics and n is the additive noise. The larger the number m is utilized, the better approximation of real experiment is achieved.

[0046] In a similar way, the super-resolution dark field image ISR containing all the detectable high-k information of the object, can be calculated as:

[0047] The effective optical transfer function (OTF) of the super-resolved dark field image is a combination of a series of OTFs which are generated by angled illumination. Thus, the highest spatial frequency in the super resolution image corresponds to the combination of the NA of illumination and detection. For each object, simulated JLR and JSR are used as input and label images respectively to train the neural network.

[0048] At 203 of the process 200, a machine learning model is trained using the simulated plurality of images of the target. The machine learning model may be, for example, a convolutional neural network (CNN). The neural network may comprise a plurality of layers. The plurality of layers of the neural network may have a corresponding weight. The weight of a layer of the neural network may correspond to a weight of a feature of the image of a particular scale. Training the machine learning model using the plurality of images of the target may comprise adjusting the weights of the layers of the neural network so that an image output from the machine learning model has improved resolution. In particular, the image output from the trained machine learning model has a resolution that is close to the resolution of the high-resolution simulated image. The resolution of the image output from the trained machine learning model may be a high-resolution dark field image of a target object.

[0049] White Gaussian noise may be added to the images that are numerically simulated. For example, the signal-to-noise ratio of the simulated images may be 35. Adding noise to the simulated images ensures that the simulated images more closely resemble images taken experimentally before they are input to the neural network.

[0050] At 204 of the process 200, a dark field image of the target is acquired. The dark field image of the target may be captured experimentally using, for example, the imaging system 100 of FIG. 1 A or the imaging system 150 of FIG. IB. In particular, the dark field image of the target may comprise an image of the target that is captured when the object is placed on a plasmonic substrate system, such as the substrate system 151 of FIG. IB. Surface plasmon polaritons (SPPs), rather than free-space propagating visible light, can be used to illuminate the target when the target is placed on a plasmonic substrate. A detector can capture the image of the target when it is illuminated by surface plasmon polaritons. As discussed with respect to FIG. 3 below, using SPPs to image a target increases the numerical aperture of the imaging system relative to the numerical aperture of a standard microscope.

[0051] At 205 of the process 200, the dark field image of the target acquired at 204 of the process 200 is input to the trained machine learning model. At 206 of the process 200, a high- resolution dark field image of the target is output from the machine learning model. The trained machine learning model may reconstruct the output high-resolution image based on the dark field image that is input to the machine learning model at 205 of the process 200. The trained machine learning model may be configured to solve an inverse problem of image reconstruction. In particular, the machine learning model may be configured to output the high-resolution dark field image based on the known forward process determined from the ground truth image and based on the plurality of high- and low-resolution simulated images. In this way, the trained machine learning model permits the reconstruction of a high-resolution image of the target.

[0052] Following the idea of synthetic aperture, the angled illumination pattern corresponding to a ring in Fourier space shifts the high spatial frequency information of the object into the detection bandwidth of the objective, as shown in Fig. 3. The highest spatial frequency information attainable by the optical system is restricted by the combination of numerical aperture (NA) of both the illumination and detection objective. The high spatial frequency information is encoded in the diffraction limited dark field image. Thus, a super resolution image may be reconstructed from the detected low resolution one by retrieving the encoded high spatial frequency information. Since all the high spatial frequency information of the object overlay and form a single diffraction limited dark field image, retrieving the high-resolution image is an extremely ill-posed problem, which is very difficult to solve by traditional optimization methods.

[0053] To retrieve the high-resolution information, CNNs can be a good choice because of their great performance in solving complex inverse problems. In general case, to train a CNN, a large amount of data from either experiment or simulation is required to ensure its performance. Acquiring enough experimental images for training could be time consuming. Collecting ground truth images is even more difficult in most cases. Here, we propose a simulation method based on the synthetic aperture idea to generate pairs of low resolution and super resolution DFM images with the knowledge of the optics including the objective NA, the illumination angle and the working wavelength.

[0054] Herein, we use a U-net structure to solve the complex inverse problem of reconstructing a high-resolution dark field image of an object given low-resolution input images. The input low resolution images passing through a down-sampling (encoder) part and an up-sampling (decoder) part are transformed into high-resolution output images. Rectified linear units (ReLU) are used between each convolutional layers to avoid gradient vanishing. We also use adaptive moment estimation (Adam) as an optimizer to compute an adaptive learning rate for each parameter of the machine learning model, with the learning rate set to 0.0001. The output super resolution images are compared with the ground truth images using both multi-scale structure similarity index measure (MS-SSIM) and Lf-norm. The combined loss function of both MS-SSIM loss and LI loss is given as: fl-aj-L¹’ (3) with a = 0.8, which may be determined after parameter optimization. In some implementations, the difference between the first image and the second image that is minimized is the combined loss function.

[0055] After training the neural network with simulated dataset which consists of 3,000 pairs of low resolution and super resolution images, the unseen dataset is processed by the trained network as a test. By comparing the network output with ground truth image, it is obvious to notice that the diffraction limited features can be clearly resolved in the output image, which match well with the ground truth. We used structure similarity index measure (SSIM) to evaluate the quality of network reconstructed super resolution image, which gives a SSIM of 0.982±0.013 compared to ground truth.

[0056] FIG. 3 represents a Fourier-space representation 300 of images captured using various different methods of imaging. A Fourier-space representation 300 of an image may be computed according to the equation

, 2?r fc₀ = — A , where 2 is the wavelength of the light. In this manner, the wavevectors k plotted in the Fourier- space representation of FIG. 3 serve as proxies for the numerical aperture NA of the image systems used to capture the images.

[0057] Circles in FIG. 3 having larger radii (e.g., larger wavevectors in Fourier space) correspond to images taken of a target with a larger numerical aperture imaging system, assuming a constant wavelength of light used to illuminate the target. The outermost circle 302 labeled “object information” represents the maximum amount of information regarding a target than can be acquired according to theory.

[0058] The circle 304 having a radius denoted ko represents the theoretical maximum numerical aperture of traditional lenses placed in air. The circle 306 labeled “detection” represents the numerical aperture of lenses in air used to experimentally detect an image of a target. As shown in FIG. 3, the numerical aperture of the lenses used for an experimental detection is always less than the theoretical maximum numerical aperture of such lenses, as shown by the circle 304 having a larger radius than the circle 306.

[0059] The circle 308 labeled “SPP mode” represents the numerical aperture of an illumination system, such as substrate system 151 of FIG. 1, that illuminates a target using surface plasmonic polaritons. As seen by the circle 308 having a larger radius than the circle 304 in FIG. 3, illuminating a target with surface plasmonic polaritons allows for a higher numerical aperture to be obtained than is maximally permitted by theory when imaging with lenses in air.

[0060] Further, the embodiments described herein acquire an amount of information upon imaging an object that approaches the theoretical limit shown by the outermost circle 302 more closely than does imaging with traditional lenses in air.

[0061] The resolution of an images captured by an imaging system increases with the numerical aperture of the imaging system. Accordingly, by using an imaging system, such as substrate system 151 of FIG. 1, that can achieve a higher effective numerical aperture via imaging with surface plasmonic polaritons, higher resolution images can be captured than can be captured with traditional lenses in air.

[0062] FIG. 4 illustrates, for a plurality of numerical aperture (NA) ratios, simulated ground truth images (in columns 1 and 4), captured DFM images (in columns 2 and 5), and output reconstructed images (in columns 3 and 6). In Figure 4, the numerical aperture ratio is defined as the sum of the illumination numerical aperture and the detection numerical aperture, the sum then divided by the detection numerical aperture, which reflects the improvement of the resolution. For example, the plurality of simulated high-resolution images of 202 of the process 200 may comprise the simulated ground truth images in columns 1 and 4 of FIG. 4. The captured second images of the target of 204 of process 200 may comprise the dark-field (DF) images of columns 2 and 5 of FIG. 4. The output images of 206 of the process 200 may comprise the output images of columns 3 and 6 of FIG. 4. A machine learning model, such as a convolutional neural network, may be training based on the high-resolution ground truth images similar to columns 1 and 4 of FIG. 4. The trained machine learning model may be configured to minimize a difference between a first image (that may have a low resolution) and a second image. The second image may comprise an input dark-field microscopy image. The input dark-field microscopy image may be captured by placing a target on a substrate system, such as the substrate system 151 of FIG. IB. The trained machine learning model may be configured to minimize the difference between the first image and the second image based on the simulated high-resolution ground truth images. The trained machine learning model may be configured to minimize the difference between the first image and the second image based on a manner of interaction between light and a target that can be determined from the first image. The trained machine learning model is configured to output a high-resolution image by minimizing the difference between the first and second images based on the high-resolution ground truth images.

[0063] One or more aspects or features of the subject matter described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs, field programmable gate arrays (FPGAs) computer hardware, firmware, software, and/or combinations thereof. These various aspects or features can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. [0064] In some implementations, the current subject matter may be configured to be implemented in a system 500, as shown in FIG. 5. For example, aspects disclosed herein may be at least in part physically comprised on system 500. To illustrate further system 500 may further include an operating system, a hypervisor, and/or other resources, to provide virtualize physical resources (e.g., via virtual machines). The system 500 may include a processor 510, a memory 520, a storage device 530, and an input/output device 540. Each of the components (e.g., 510, 520, 530 and 540) may be interconnected using a system bus 550. The processor 510 may be configured to process instructions for execution within the system 500. In some implementations, the processor 510 may be a single-threaded processor. In alternate implementations, the processor 510 may be a multi -threaded processor.

[0065] The processor 510 may be further configured to process instructions stored in the memory 520 or on the storage device 530, including receiving or sending information through the input/output device 540. The memory 520 may store information within the system 500. In some implementations, the memory 520 may be a computer-readable medium. In alternate implementations, the memory 520 may be a volatile memory unit. In yet some implementations, the memory 520 may be a non-volatile memory unit. The storage device 530 may be capable of providing mass storage for the system 500. In some implementations, the storage device 530 may be a computer-readable medium. In alternate implementations, the storage device 530 may be a floppy disk device, a hard disk device, an optical disk device, a tape device, non-volatile solid state memory, or any other type of storage device. The input/output device 540 may be configured to provide input/output operations for the system 500. In some implementations, the input/output device 540 may include a keyboard and/or pointing device. In alternate implementations, the input/output device 540 may include a display unit for displaying graphical user interfaces.

[0066] These computer programs, which can also be referred to as programs, software, software applications, applications, components, or code, include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object- oriented programming language, and/or in assembly/machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus and/or device, such as for example magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs), used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor. The machine-readable medium can store such machine instructions non-transitorily, such as for example as would a non-transient solid-state memory or a magnetic hard drive or any equivalent storage medium. The machine-readable medium can alternatively or additionally store such machine instructions in a transient manner, such as for example, as would a processor cache or other random access memory associated with one or more physical processor cores.

[0067] To provide for interaction with a user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as for example a cathode ray tube (CRT) or a liquid crystal display (LCD) or a light emitting diode (LED) monitor for displaying information to the user and a keyboard and a pointing device, such as for example a mouse or a trackball, by which the user may provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, such as for example visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive track pads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and the like.

[0068] In the descriptions above and in the claims, phrases such as “at least one of’ or “one or more of’ may occur followed by a conjunctive list of elements or features. The term “and/or” may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases “at least one of A and B;” “one or more of A and B;” and “A and/or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation is also intended for lists including three or more items. For example, the phrases “at least one of A, B, and C;” “one or more of A, B, and C;” and “A, B, and/or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Use of the term “based on,” above and in the claims is intended to mean, “based at least in part on,” such that an unrecited feature or element is also permissible.

[0069] The subject matter described herein can be embodied in systems, apparatus, methods, and/or articles depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and/or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of several further features disclosed above. In addition, the logic flows depicted in the accompanying figures and/or described herein do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Other implementations may be within the scope of the following claims.

Claims

What is claimed:

1. A method of imaging an object using dark-field microscopy, the method comprising: acquiring a ground truth image of the object; simulating a plurality of dark field images of the object based on the ground truth image; training a machine learning model using the plurality of simulated dark field images and the ground truth image; acquiring a dark field image of the object; inputting the dark field image of the object into the trained machine learning model; and outputting, from the trained machine learning model, a high-resolution dark field image of the object.

2. The method of claim 1, wherein acquiring the ground truth image of the object comprises simulating the ground truth image of the object.

3. The method of claim 2, wherein the ground truth image comprises a superresolution image of the object.

4. The method of claim 1, wherein the ground truth image of the object is used to determine a manner of interaction between the object and light.

5. The method of claim 1, wherein the plurality of dark field images of the object comprise low-resolution images of the object and high-resolution images of the object, and wherein the plurality of dark field images are simulated based on a plurality of parameters describing an imaging system.

6. The method of claim 1, wherein the machine learning model is a neural network.

7. The method of claim 6, wherein training the machine learning model using the plurality of simulated dark field images and the ground truth images comprises adjusting at least a first weight corresponding to a layer of a plurality of layers of the machine learning model.

8. The method of claim 1, wherein acquiring the dark field image of the object comprises imaging the object using an imaging system, the imaging system comprising a substrate, a light-emitting layer, and a metal layer.

9. The method of claim 8, wherein the metal layer is configured to generate surface plasmonic polaritons, and wherein: the surface plasmonic polaritons are configured to illuminate the object such that the dark field image of the object encodes high-frequency information that permits the outputting of a high- resolution dark field image of the object.

10. The method of claim 1, wherein: the machine learning model comprises a plurality of layers; training the machine learning model comprises adjusting a weight of a layer of the plurality of layers of the machine learning model; and the weight of the layer of the plurality of layers of the machine learning model corresponds to a weight of a feature of the high-resolution dark field image of a particular scale.

11. A system comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, causes operations comprising: acquiring a ground truth image of an object; simulating a plurality of dark field images of the object based on the ground truth image; training a machine learning model using the plurality of simulated dark field images and the ground truth image; acquiring a dark field image of the object; inputting the dark field image of the object into the trained machine learning model; and outputting, from the trained machine learning model, a high-resolution dark field image of the object.

12. The system of claim 11, wherein acquiring the ground truth image of the object comprises simulating the ground truth image of the object.

13. The system of claim 12, wherein the ground truth image comprises a superresolution image of the object.

14. The system of claim 11, wherein the ground truth image of the object is used to determine a manner of interaction between the object and light.

15. The system of claim 11, wherein the plurality of dark field images of the object comprise low-resolution images of the object and high-resolution images of the object, and wherein the plurality of dark field images are simulated based on plurality of parameters describing an imaging system.

16. The system of claim 11, wherein the machine learning model is a neural network.

17. The system of claim 16, wherein training the machine learning model using the plurality of simulated dark field images and the ground truth images comprises adjusting at least a first weight corresponding to a layer of a plurality of layers of the machine learning model.

18. The system of claim 11, wherein acquiring the dark field image of the object comprises imaging the object using an imaging system, the imaging system comprising a substrate, a light-emitting layer, and a metal layer.

19. The system of claim 18, wherein the metal layer is configured to generate surface plasmonic polaritons, and wherein: the surface plasmonic polaritons are configured to illuminate the object such that the dark field image of the object encodes high-frequency information that permits the outputting of a high- resolution dark field image of the object.

20. Anon-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, causes operations comprising: acquiring a ground truth image of an object; simulating a plurality of dark field images of the object based on the ground truth image; training a machine learning model using the plurality of simulated dark field images and the ground truth image; acquiring a dark field image of the object; inputting the dark field image of the object into the trained machine learning model; and outputting, from the trained machine learning model, a high-resolution dark field image of the object.