TWI414818B - Wide-field super-resolution optical sectioning microscopy using a spatial light modulator - Google Patents

Abstract

Systems and methods for optical sectioning microscopy with structured illumination are provided. A light source generates a light beam with a spatial pattern for successively illuminating a sample at each phase of a plurality of phases. A detector detects a first set of images of the sample at a first axial resolution and a first lateral resolution, each image being associated with a respective phase of the plurality of phases of the illumination. A processor processes the first set of images to generate an enhanced sectioned image of the sample. More specifically, the processor generates data representing a second set of images at a second axial resolution greater than the first axial resolution; and subsequently, performs spectral analysis on the data representing the second set of images to form data representing the enhanced sectioned image of the sample at a second lateral resolution greater than the first lateral resolution.

Description

Ultra-resolution wide-field light sectioning microscope using a spatial light modulator

The invention relates to a microscope system and a method of use, in particular to a microscopic system and a method for using super-resolution wide-field light sectioning.

In the field of far field optical microscopy, the limit of "spatial resolution" has always been an important issue for breakthroughs. Far-field optical microscopy techniques with ultra-resolution capability of spatial resolution, such as Stimulated Emission Depletion (STED) microscopy, Photoactivated Localization Microscopy (PALM), and presumptive Techniques such as Stochastic Optical Reconstruction Microscopy require imaging with a scanning optical system.

Since the system complexity and system cost formed by the foregoing techniques are quite high, and if a three-dimensional image is required, the image capturing time becomes quite long. In addition, the recursive algorithm can reduce the diffraction effect of the optical image and improve the spatial resolution. Such a technique can successfully improve the resolution on high-contrast images; however, it takes a long time to perform calculations; To improve the observation resolution of a dynamic movie, it takes longer to perform calculations.

Structured Illumination, or Pattern Excitation, is another type of super-resolution optical technology. Usually, the incident light of the optical microscope is modulated by a two-dimensional mesh pattern, and the mesh pattern is moved to different positions in the x direction and the y direction of the vertical optical axis, and at least five images are taken, and then solved by an equation. A microscopic image with a lateral resolution exceeding the diffraction limit is solved. The advantage is that it can be taken directly from the traditional wide-field optical system without the need for a scanning mechanism; and the time to calculate the image is much shorter than the recursive algorithm. This technique can be excited in a linear manner or in a nonlinear manner. In addition, structural illumination super-resolution full-reflection fluoroscopy microscopy with liquid crystal spatial light modulator has been proposed; however, the two-dimensional structured illumination microscopy technology has not yet provided optical sectioning (Optical Sectioning). Ability.

Structured illumination can also achieve optical slicing in wide-field optics. In this technique, only one-dimensional periodic grid pattern is used for modulation. Moving the grid pattern in the direction of the vertical optical axis to the spatial phase of 0, 2p/3, 4p/3, etc. of the grating period, and obtaining the images I ₁ , I ₂ , I ₃ at the three positions, a simple algebraic operation By reorganizing the three images, a wide-field optical slice image can be obtained.

In addition, wide-field optical sectioning microscopy with one-dimensional modulation of liquid crystal spatial light modulator has also been proposed; using the concept of differential measurement, the longitudinal resolution of wide-field optical section microscopy can be further improved. The nanometer scale can be achieved; the horizontal resolution can also be improved by using a recursive algorithm. However, the one-dimensional modulation of wide-field optical section microscopy technology can not achieve the effect of directly improving the lateral resolution.

The present invention is a method of optical image processing. The method includes: continuously illuminating a sample in a spatial pattern on each phase of the plurality of phases; obtaining data of the first set of images representing the sample at the first longitudinal resolution and the first lateral resolution; Each image is associated with an individual phase of a plurality of phases of the illumination; and the obtained data is processed to produce an enhanced sliced image sample. Further processing the obtained data, including processing the obtained data to generate data representing the second set of images at a second longitudinal resolution greater than the first longitudinal resolution; and, representing the second set of images Spectral analysis is performed on the data to form enhanced slice image data samples at a second lateral resolution greater than the first lateral resolution.

According to the invention, the first set of images is mathematically combined to generate a second set of images according to the individual phases of the first set of images.

For each image of the second set of images, a Fourier analysis is performed to form the spectrum of the image in the spatial spectrum. The spectra of each image are combined to form a composite spectrum of enhanced sliced image samples. The inverse spectral Fourier transform is represented using a composite spectrum to produce an enhanced slice image sample.

In some examples, combining the individual portions of each image spectrum, the spectrum of the subset of the second set of images is transferred to the origin of the spectrum determined by the vector, based on the physical characteristics of the spatial pattern. The physical characteristics of the spatial pattern may include the periodicity of the pattern.

In some examples, the spatial pattern displays periodicity in a one-dimensional space. In other examples, the spatial pattern displays periodicity in a two-dimensional or multi-dimensional space.

In the plane of the vertical optical axis, the spatial pattern is transferred to each of a series of positions in a linear manner according to the periodicity of the spatial pattern. Alternatively, the illumination sample can be illuminated while the spatial pattern is rotated at each angle of a series of angles in the plane of the vertical optical axis.

In some examples, the light is focused to the first depth of the sample to produce a slice image of the sample at the first depth. Also, the light can be continuously focused to a depth of one of the series of samples to produce a series of slice images of the sample at each individual depth.

The present invention is a method of optical image processing. A light source capable of producing a spatial pattern to continuously illuminate a sample over each phase of a plurality of phases. A detector that detects the first set of images in the first vertical resolution and the first horizontal resolution. A processor receives and processes the first set of images to produce an enhanced slice image of the sample. In particular, the processor is configured to process the first set of images to produce data representative of the second set of images at a second longitudinal resolution greater than the first longitudinal resolution; and, in the representative On the data of the two sets of images, spectrum analysis is performed to form data representing the enhanced slice image of the sample at a second lateral resolution higher than the first lateral resolution.

The present invention includes an optical assembly to focus the beam at a first depth of the sample.

The invention can also include a controller coupled to the optical assembly, and the module can control the optical assembly to focus the beam at a second depth of the sample. The controller includes a piezoelectric positioner.

The invention includes a mask having a first periodicity in a one-dimensional space, the mask having a second periodicity in the two-dimensional space.

The light source of the present invention further includes a modulator coupled to the mask, and which is capable of adjusting the phase of the illumination with a module that changes the mask. The module of the mask includes a position of the mask and a direction of the mask.

A further configuration of the processor can be calculated based on the individual phases of the illumination corresponding to each image of the first set of images, and the first set of images is combined to produce a second set of images. For each image of a subset of the second set of images, the processor performs a Fourier analysis to form a spectrum of the image in a spatial spectrum. The processor combines each individual portion of the spectrum to form a composite spectrum of the enhanced slice image of the sample. The processor then performs an inverse Fourier transform of the composite spectrum to produce an enhanced slice image.

The module of the light source itself can produce the spatial pattern, such as the light source can include an LED array structure to produce a movable two-dimensional optical pattern.

Using a single optical modulator to modulate the spatial phase of the illumination light improves the resolution of the wide-field optical microscope in three dimensions. When the light is modulated using the fast-switching optoelectronic technology, the image capture rate may be as fast as an image with a super-resolution (eg, an image frame that processes five patterns per second).

The system and method of the present invention can be easily and cost effectively mounted to a variety of optical devices, including conventional fluorescent microscopes. In addition, more than one excitation source can be easily installed in the system, which can be useful in the study of cell function applications. And because the image of the super-resolution of one sample can be obtained at a high speed, the dynamic analysis of the structure inside the living cell can be performed.

The present invention requires only fast Fourier transform (FFT) and algebraic calculations, and a separate processor can be used to process the image. The stand-alone processor can process images faster than other software and can be integrated into existing PCs like a network card (PCI).

Therefore, the advantages and spirit of the present invention will be further understood from the following detailed description of the invention.

1. System Overview

Referring to Fig. 1, a specific embodiment of a light sectioning microscope 100 having structured illumination is shown. The light sectioning microscope 100 has several components of a conventional microscope, including a light source 110 to produce a beam 112, and a beam splitter 140 to reflect the beam 112 to a sample 160 for image processing. A set of optical components includes an objective lens 150 for focusing the beam 112 onto the image plane selected by the sample 160. The light reflected (or emitted) by the sample 160 is first received by the objective lens 150 and, after passing through the beam splitter 140, detected by a detector 170 (e.g., a CCD camera). The detector 170 converts the detected optical signal into an electrical signal and passes it to a processor 180 (such as a computer) to form a numerical image of the sample 160.

The sample 160 (or the objective lens 150) is mounted on a positioning table 162 for linear movement in the lateral and longitudinal directions. In this description, the term "lateral" generally describes a direction along the optical axis of light propagation (as shown in Figure 1 for the z-axis), while the term "longitudinal" generally describes a plane perpendicular to the optical axis ( Such as xy-plane). For a "thick" sample 160 (such as an object having a height greater than 1 μm), the distance of z can be adjusted between sample 160 and objective 150 to form a slice image of the sample at various depths, also known as the ability of a microscope to slice light. .

The spatial resolution of conventional wide-field microscopes is limited by the diffraction limit of light. Here, the term "spatial resolution" or "optical resolution" generally describes the ability of an optical component of an image processing system to resolve details within a sample for image processing. In other words, the "space resolution rate" is equivalent to the smallest spatial distance at which two distinguishable (resolvable) points within a sample are separated. Therefore, an image processing system with a "larger" or "enhanced" spatial resolution can display a "thinner" structure within a sample, or can distinguish adjacent points separated by a smaller spatial distance.

One way to improve the spatial resolution of wide field microscopes is to use a spatially modulated pattern to illuminate the sample, essentially as described below, to perform a mixing of the spatial frequencies of a harmonic.

In a particular embodiment, a light modulator 120 and light source 110 are provided to project a spatial pattern (e.g., a substantially periodic spatial pattern) onto a sample 160 through a pattern pattern beam 112. An example of a light modulator 120 includes a mask 130 (as in the form of a two-dimensional grid) and a control unit (not shown) to adjust the spatial structure of the mask to enable segmental or continuous change of the pattern pattern beam 112. Spatial phase. The mask 130 can have a local one-dimensional or multi-dimensional periodicity (as shown in Figure 1 on a plane perpendicular to the beam 112 along the p _x and p _y axes). The spatial structure of the mask 130 can be varied, for example, to move the mask 130 spatially or in a rotational manner.

There is no theoretical limitation here, and at least the resolution enhancement capability of the structured illumination microscope 100 can be understood from the following sections.

Let I ₀ be the intensity of a uniform illumination on the focal plane of objective lens 150, and M ₀ is a general image produced by a sample through uniform illumination. Currently, samples are considered in a spatial spectrum (also referred to as inverted space) with a Fourier transform of the sample. For each optical system, according to the optical transfer function (OTF), only the spatial frequency of a sample below a cutoff frequency threshold can be transferred through the system. Therefore, the image M ₀ formed by the optical system contains only optical information emitted by samples belonging to the spatial band of the transfer ("pass band"). In other words, the sample information outside the passband will be lost in M ₀ and cannot be recovered.

When a modulated two-dimensional (2D) spatial pattern is projected onto the sample, the intensity I( x , y ) of the structured illumination on the focal plane becomes:

I ( x , y )= I ₀ [2+cos( ux -ΔΦ _x )+cos( uy -ΔΦ _y )] (1)

Where u is the spatial frequency of the modulation pattern in the inverted space, and ΔΦ _x and ΔΦ _y are the spatial phase constants of the modulation pattern relative to the movement of the sample in the x and y directions, respectively. The spatial frequency u can be expressed as:

u =(4π n )sin(α)/λ (2)

Where n is the refractive index of the glass, α is the angle between the beam and the optical axis of the optical system, and λ is the vacuum wavelength of the illumination.

As a result of structured illumination, the image M ( x , y ) of the sample can now be described as:

Wherein M ₀ is the sample images generally uniform illumination, and M _{x ± (x,} y) and M _{Y ±} (x, y) is the image four ingredients, which are along the center of the spectrum k in the positive and negative directions _{The x} or k _y axis is offset by the spatial frequency u in the inverted space. And use ~ (corrugated symbol) to represent a two-dimensional Fourier transform of a variable, To represent the spectrum of the image M ₀ , and ( k _x ± u , k _y ) and ( k _x , k _y ± u ) to represent the spectrum of M _{x ±} ( x , y ) and M _{Y ±} ( x , y ).

One of the main effects of structured illumination is that by moving the extra high-frequency region of the spatial spectrum into the passband of the optical system, it can be restored to the traditional wide-field microscope within the reconstruction of the image M ( x , y ). Information that is not accessible, thus improving the spatial resolution of the optical system.

According to equation (3), the currently observed image M ( x , y ) is a combination of a general image and four spectra, and its origin is offset by an additional component of + u or - u . Since the total coefficients of these components in equation (3) are related to the phase of the illumination (ie, ΔΦ _x and ΔΦ _y ), an image of one of the samples can be recorded at different illumination phases, and arithmetic can be performed on the recorded image. Operate to extract these ingredients. After separating these components, information within these components can be used to reconstruct an image of the sample at greater than the longitudinal and lateral resolution of a conventional wide-field microscope, as described in detail below.

2. Operation

Referring to Figure 2, an exemplary process 200 is provided for use with microscope 100 to perform a light slice of a three dimensional (3D) sample. Using this procedure, longitudinal slice images of samples with larger resolutions than conventional wide field microscopes can be obtained in both the lateral and longitudinal directions.

Step 210: First find a preliminary "interest area of interest" (ROI) for image processing. The "interested area" is typically defined by a two-dimensional (x- and y-) region at a selected z-depth of the sample 160. By moving the sample 160 in the longitudinal and lateral directions relative to the movement of the objective lens 150, the incident beam can be focused onto the "interesting region", for example, by moving the positioning stage 162.

Step 220: Continuously illuminate the sample 160 in an optical pattern over each of a series of spatial phases. In the present embodiment, the optical system to the beam pattern 112 is generated through the mask 130, and by moving the mask 130 p _x p and _y-axis along a manner as to adjust the optical spatial phase pattern.

For example, referring to Figures 3A through 3E, a two-dimensional (2D) periodic optical pattern is displayed over five spatial phases of a set. In Fig. 3A, the center of the optical pattern is at the origin, and its line pitch is repeated over a space period of one of T _x and T _y along the p _x and p _y axes, respectively. In this example, the structure of T _x and is the same as T _y . In some other examples, the structure of a two-dimensional optical pattern can also be set in a particular local periodicity in each of the two directions.

In FIGS. 3B and 3C, the optical patterns are offset by 120° (or T/3) along the p _x axis in the negative and positive directions, respectively. Similarly, in FIGS. 3D and 3E, the optical patterns are shifted by 120° (or T/3) along the p _y axis in the negative and positive directions, respectively.

3A to 3E show that the five pattern spatial phase constants φ _mn can be expressed as φ _mn = (2π / 3) ‧ ( m , n ), where ( m , n ) = (0, 0), (1,0), (2,0), (0,1), and (0,2).

Step 230: Under structured illumination, five images of the "area of interest" in the sample 160 may be continuously obtained by the detector 170, and each image may be displayed and formed in five spatial phases illuminated. Every phase of it. According to Equation (3) an image in M _{φ mn} phase φ _mn obtained can be expressed as:

At this step, the lateral and longitudinal resolution of the image M _{φ mn} obtained can be compared with the conventional wide field microscope resolution.

Step 240: Process the obtained five images M _{φ mn} to form a longitudinal slice image in the real space. In particular, based on the special phase constants used in the illumination in this example, M ₀ can be extracted from the five images obtained as follows:

The longitudinal slice images M _X± and M _Y± can be obtained as follows:

As described above, each of the four longitudinal slice images M _X± and M _Y± provides a larger longitudinal resolution than a conventional wide-field microscope, as in Neil, et al., December 15, 1997. A reference cited in the "Methods of Using Structured Light to Obtain Light Slices in Conventional Microscopy", published in the Optics Letters publication.

Step 250: In order to further improve the lateral resolution of the sample 160 image, the four longitudinal slice images M _X± and M _{Y± are} transferred into a spatial spectrum (inverted space) by two-dimensional Fourier transform. Therefore, respectively To represent the obtained spectrum of M _{x ±} ( x , y ) and M _{Y ±} ( x , y ).

Step 260: Within the spatial spectrum, along k _x and k _y , with a size of u , The four offset spectral images M _x± ( k _x , k _y ) and M _y± ( k _x , k _y ) are generated by their original positional offsets.

Step 270: Process the four offset spectral images to form a composite spectrum M _super , for example, "joining" the individual portions of the offset spectral image to create an ultra-overlapping spectral image. Preferably, in the process of creating a super-overlapping spectral image, the signal attenuation caused by the optical transfer function of the microscope is also compensated.

Step 280: Reconstruct a super-resolution image M _{super of} an "interested area" by performing inverse Fourier transform on the composite spectrum. Through the spectrum analysis from step 250 to step 280, the reconstructed super-resolution image M _super will provide a larger lateral resolution than the vertical slice images M _X± and M _Y± . Therefore, the super-resolution image M _{super is} sometimes also considered to be a reinforced slice image, which has a larger resolution in the horizontal and vertical directions than the conventional wide-field microscope.

Step 290: After obtaining the super-resolution image M _{super of the} current "interested area", select the next "interested area" sample at different z depths, for example, in a predetermined increment/decrement in the vertical direction. Move sample 160 up. Thus, steps 220 through 280 can be repeated at each z-depth to continuously form a super-resolution image of the sample over a set of z-depths.

In this example, a light slice of a three-dimensional sample is described in a series of ways. In particular, prior to the illumination step 220 beginning with the next depth, and at a selected depth, the sample is completed at image reconstruction step 280. In other examples, image acquisition steps 220 through 230, and image analysis steps 240 through 280 may be independently processed in a parallel manner. For example, a group of five images can be taken without interruption, via a set of sample depths, and executed in a later step to obtain a spectral analysis of the image.

Referring now to Figures 4A through 4F, the above-described step 200 can be further illustrated in the following description. For the sake of simplicity, the two-dimensional optical pattern of FIG. 3A is projected onto a plane for image processing.

Figure 4A shows a two-dimensional optical pattern of Figure 3A, an image M _{φ 00} that has been projected onto a plane (as obtained in step 230).

Figure 4B shows a spectrum formed by two-dimensional Fourier transform of the image M _{φ 00} spectrum in the spatial spectrum. . The spectrum An Airy pattern contained at the origin and four offsets of the origin along the k _x or k _y axis respectively become four additional Airy patterns. This compensation distance is inversely proportional to the line spacing of the optical pattern of Figure 3A.

Figure 4C shows a two-dimensional optical pattern being projected onto a longitudinal slice image M _{X -} on the plane (as obtained at step 240).

Figure 4D shows the spectral image of M _{X -} in the spatial spectrum (as obtained at step 250). Within the spectrum image, only the frequency components to the right of the two dashed lines are used to establish a composite spectrum. .

Figure 4E shows the connection of four offset spectrum images with Individual parts, into a spectrum to obtain a composite spectrum (as obtained at step 270). The frequency component enclosed by the dotted line of Fig. 4D forms the composite spectrum One quarter of the.

Figure 4F is a super-resolution image reconstructed from the image of Figure 4A. (Not shown in the same scale).

3. Examples

The following sections provide examples of the above systems and usage methods. In some examples, the results of structured illumination are compared to uniform illumination to illustrate the enhanced effects of structured illumination.

3.1 Example I

Referring to FIGS. 5A to 5C, image processing is performed using the light sectioning microscope 100, including a sample having a diameter of 100-nm fluorescent spheres. The fluorescent sphere has an emission wavelength of about 560 nm. At this emission wavelength, the theoretical lateral resolution of uniform illumination is about 263 nm. Therefore, a uniformly illuminated image of a 100-nm fluorescent sphere will have a viewing width of about 280 nm. Here, the term "width" refers to the full width (FWHM) of half of the maximum value when an individual particle in an image is a Gaussian intensity distribution.

Figure 5A shows an image of two 100-nm fluorescent spheres under uniform illumination. Figure 5B shows an ultra-resolution image of two fluorescent spheres on a focal plane using structured illumination with a 750-nm periodic two-dimensional mesh pattern. Figure 5C shows two intensity distribution curves for a selected phosphor sphere, the dashed curve is a sample along the dashed line of Figure 5A, and the solid curve is a sample along the dashed line of Figure 5B.

Based on the 5C plot, the "full width at half of the maximum value" of the two intensity distribution curves, and the lateral resolution of the uniform illumination image of Fig. 5A, is about 325 nm. By comparison, the lateral resolution of the structured illumination image of FIG. 5B is about 180 nm, so the improvement of the display resolution is about 2 times.

3.2 Example II

Referring to Figures 6A through 6C, three 200-nm fluorescent spheres are scanned along the z-axis to estimate the longitudinal resolution of the microscope 100 with structured illumination. In the first example, a larger diameter ratio of the fluorescent sphere is chosen to avoid coupling between longitudinal and lateral intensity variations, which typically occurs when the imaging of the object is much smaller than the lateral resolution of the optical system.

The longitudinal strength curves of the two mesh patterns of different periods are measured and compared. In Figure 6C, the dashed curve is obtained by projecting a 750 nm period mesh pattern (measured on the focal plane) onto the sample, while the solid line curve is a projection of a 500 nm period mesh pattern to the sample. Obtained on it. The "full width at half the maximum value" of the solid line curve is about 290 nm, which corresponds to a longitudinal resolution of 210 nm (about 0.38 λ).

Figures 6A and 6B are images of the fluorescent spheres recorded at two different lateral positions using a 750-nm mesh pattern. These two images show that the lateral resolution has been improved so that the three spheres are clearly resolved and the intensity of the fluorosphere is reduced as the sample is moved away from the focal plane.

It is worth noting that the flank of the longitudinal intensity curve obtained with the 500-nm mesh pattern still has 20% of the peak intensity, which may be caused by the longitudinal chromatic aberration of the objective lens. Such flank may affect the quality of the image for the stacked samples, so in some instances, a 750-nm mesh pattern may be better at observing the internal structure of a biological sample, such as a cell. In some applications, certain filtering techniques (such as longitudinal apodization filters) and modulation patterns can be used to reduce the size of the flanks.

3.3 Example III

The light-slicing microscope 100 described above and the flow of use can be used in biological applications, such as viewing intracellular structures, even when cells are stacked together.

Fluorescence images of actin filaments (using Alexa Fluor 488 phalloidin staining technique) in fixed fibroblasts obtained under various image processing conditions with reference to Figures 7A through 7E.

Figure 7A is an image of actin filaments under uniform illumination. The image was obtained under a low magnification (20x) objective having a numerical aperture (NA) of 0.4. Within this image, actin microfilaments in several stacked cells can be seen.

Figure 7B is an enlarged image of the selected area (the area enclosed by the square dotted line) of one of the 7A maps under uniform illumination. The image was obtained under a high magnification (100x) objective with a numerical aperture of 1.3.

The 7C and 7B maps have the same area, but are images obtained under structural illumination. Fig. 7D is an out-of-focus image obtained having the same area as that of Fig. 7C and after the objective lens is moved by 500 nm in the longitudinal direction. In Figure 7C, the improved lateral resolution allows the number of microfilaments blurred in Figure 7B to be clearly distinguished. Thus, this technique can be used to accurately quantify the cytoskeleton and/or other intracellular structures. Further, the intensity of the 7D image appears to be lower than that of the 7C chart, and is therefore positively correlated with the slicing ability of the present invention.

Figure 7E shows the intensity distribution curve along the dashed line of Figure 7B (the intensity curve of the dashed line) and the 7C chart (the intensity curve of the solid line). Based on the two distribution curves, the width of the actin filaments was observed to be about 330 nm and about 200 nm, respectively, indicating that structured illumination can reveal a more subtle structure in a biological sample compared to uniform illumination.

4 other specific embodiments

The following description may provide various other specific embodiments of the above systems and methods.

Referring to Figure 8, a diagram of another embodiment of a light sectioning microscope 800 is shown. In this embodiment, a spatial light modulator 820 includes a reflective germanium liquid crystal (LCoS) panel 824 coupled to a polarizer 822 for two-dimensional modulation of illumination light. The reflective 矽 liquid crystal panel 824 is composed of 1024 x 768 pixels and has a pixel size of 11.3 x 11.3 μm ² . A drive signal can be rapidly changed on the two-dimensional reflective 矽 liquid crystal 824 at an update rate of up to 60 Hertz (Hz) to move the sinusoidal modulation pattern projected onto a sample 860.

A light source 810 (e.g., a 50 mW, 475 nm diode pumped solid state laser) produces a beam that passes through a spatial filter 812 and then is expanded by lens 814 to a "maximum of about 8 mm diameter" The full width" is used to illuminate the spatial light modulator 820. The mesh pattern on spatial light modulator 820 is then projected through lens 880 and objective lens 850 onto sample 860 (e.g., an oil immersion objective having a 1.3 numerical aperture). Sample 860 is an image captured by CCD camera 870. A piezoelectric (PZT) converter 852 is coupled to the objective lens 850 to control the relative height of the focal plane to the sample 860.

Referring again to FIG. 1, other examples for light source 110 include: various lamps (such as LED lamps and xenon arc lamps) and lasers (such as single and multiple wavelength lasers). Light source 110 can also include a set of optical components such as lenses, mirrors, and filters (not shown) to control the characteristics (e.g., intensity, wavelength, and direction) of the output beam 112. Examples of detectors 170 include CCD cameras and other CMOS detectors.

In addition to the use of a mask, the pattern produced by the light source itself can be used to modulate the light. For example, an array of LEDs can be constructed to enable the creation of a movable two-dimensional mesh pattern.

Various forms of space patterns can be used. For example, a one-dimensional sinusoidal pattern or a two-dimensional sinusoidal pattern of symmetric or asymmetric periods can be used. At the same time, the patterning of the spatial phase can be easily changed by the optical pattern.

Referring to Figures 9A through 9F, a graphical example of the spatial phase of a one-dimensional (1D) grid modulated illumination is shown. In this example, the one-dimensional grid is rotated to a series of angular positions, each of which can be relative to one of the series of spatial phases.

There may be additional ways to create a spatial pattern. In some examples, a laser speckle technique can be used to create a spatial pattern. For example, if a piece of frosted glass or light diffuser is mounted along a light illumination path, a pattern of fixed spots can also be created for illuminating the sample. In some cases, due to the speckle pattern, a number of frames can be processed to obtain a slice image with an enhanced resolution.

The present invention can be used in a computer program product to perform operations and signal processing; the present invention can use a programmable processor and method to execute a machine readable storage device; the present invention can input data and generate output through a programmable processor . The present invention can be implemented by coupling one or more spatial light modulators, detectors, light sources, processors, and other components of the system to a controller within the computer program; the programmable system of the present invention includes at least one A programmable processor that receives and transmits data and instructions, a data storage system, at least one input device, and at least one output device.

The above is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the present invention should be included in the following. Within the scope of the patent application.

100. . . Light sectioning microscope

110. . . light source

112. . . beam

140. . . Beam splitter

150. . . Objective lens

160. . . sample

162. . . Positioning table

170. . . Detector

180. . . processor

810‧‧‧Light source

812‧‧‧Space filter

814‧‧‧ lens

820‧‧‧Space light modulator

880‧‧‧ lens

850‧‧‧ objective lens

860‧‧‧ sample

870‧‧‧CCD camera

852‧‧‧Piezoelectric converter

Figure 1 is a diagram of a specific embodiment of a light sectioning microscope with structured illumination.

Fig. 2 is a flow chart showing the image processing used in the light section microscopy shown in Fig. 1.

3A to 3E are diagrams for modulating a two-dimensional periodic pattern example in a sequence of phases according to the flowchart of FIG. 2.

Figure 4A is an optical image of a two-dimensional periodic pattern projected onto a plane, Figure 3A.

Figure 4B is a spectral image of a 2D periodic pattern that converts Figure 4A into a spatial spectrum.

Fig. 4C is an image M _{X -} which is a slice of the two-dimensional periodic pattern of Fig. 3A.

The 4D image is a spectral image M _{X -} ( k _-u , k _y ) of M _{X -} in a spatial spectrum.

Figure 4E is a composite spectrum image generated by combining four offset spectral images M _x± ( k _x , k _y ) and My± ( k _x , k _y ) .

Figure 4F is a reconstructed image of a two-dimensional periodic pattern projected onto a plane .

Figure 5A is an image of two 100 nanometer (nm) fluorescent spheres under uniform illumination.

Figure 5B is an ultra-resolution image reconstructed from two fluorescent spheres using structured illumination with a periodic two-dimensional mesh pattern of 750 nm.

Fig. 5C is a diagram of a first intensity distribution curve (a curve of a broken line) along a broken line of Fig. 5A, and a second intensity distribution curve (a curve of a solid line) along a broken line of Fig. 5B.

Figures 6A and 6B show images of the three balls recorded at different lateral positions using a 750 nm periodic mesh pattern.

Figure 6C is a plot of the first intensity distribution curve (dashed curve) obtained using a 750 nm periodic mesh pattern along the optical axis, and a second obtained using a 500 nm periodic mesh pattern. A plot of intensity distribution curves (solid curves).

Figure 7A is an image of a group of fibroblast actin filaments (NA = 0.4) under uniform illumination with a 20X objective.

Figure 7B is a detailed image of the area enclosed by the square dotted line of Figure 7A under uniform illumination with a 100X objective (NA = 1.3).

Figure 7C is a detailed image of the same area as in Figure 7B (NA = 1.3) under a structured illumination with a 100X objective.

Fig. 7D is a detailed image (NA = 1.3) of the same region as Fig. 7C under a structured illumination with a 100X objective lens, which is longitudinally shifted by 500 nm from the position of Fig. 7C.

Fig. 7E is a diagram of a first intensity distribution curve (dashed curve) along a broken line of Fig. 7B, and a second intensity distribution curve (curve of a solid line) along a broken line of Fig. 7C.

Fig. 8 is a view showing another specific embodiment of the light sectioning microscope shown in Fig. 1.

Figures 9A through 9F are diagrams for modulating another one-dimensional periodic pattern in a sequence of phases according to the flow chart of Figure 2.

100. . . Light sectioning microscope

110. . . light source

112. . . beam

140. . . Beam splitter

150. . . Objective lens

160. . . sample

162. . . Positioning table

170. . . Detector

180. . . processor

Claims (24)

A method for forming an optical image, comprising: illuminating a sample continuously in a spatial pattern on each phase of a plurality of phases; and obtaining a data representing a first set of images of the sample at a first longitudinal resolution And a first lateral resolution, each image being associated with a different phase of the plurality of phases of the illumination; and processing the obtained data to produce an enhanced slice image of the sample, comprising: processing the obtained data to And the second longitudinal resolution of the first longitudinal resolution is greater, generating data representing the second set of images; and performing spectral analysis on the data representing the second set of images to be transverse to the first The enhanced slice image data representing the sample is formed at a second lateral resolution of the resolution. The method of forming an optical image according to claim 1, wherein the processing obtains data to generate data representing the second set of images, comprising: determining an individual phase of the illumination according to each image of the first set of images And calculating the first set of images to generate the second set of images. The method of optical image formation according to claim 1, wherein the spectrum analysis performed on behalf of the second set of image data comprises: performing Fourier analysis on each image of the subset of the second set of images for Within a spatial spectrum, a spectrum of the image is formed. The method of forming an optical image according to claim 3, wherein the spectrum analysis performed on the second set of image data further comprises: combining each individual part of the spectrum to form the sample. Enhance the composite spectrum of sliced images. The method of forming an optical image according to claim 4, wherein each of the individual portions of the combined spectrum representation includes: shifting a spectrum of the second set of image subsets according to a physical characteristic of the spatial pattern, An origin of the spectrum determined by a vector. The method of forming an optical image according to claim 5, wherein the physical characteristic of the spatial pattern comprises a periodicity of the spatial pattern. The method of optical image formation according to claim 4, wherein the spectrum analysis performed on behalf of the second set of image data further comprises: using an inverse Fourier transform to generate the enhanced slice image to process the composite spectrum . The method of forming an optical image according to claim 1, wherein the spatial pattern exhibits periodicity in a one-dimensional space. The method of forming an optical image according to claim 1, wherein the spatial pattern displays periodicity in a two-dimensional space. The method of optical image formation according to claim 1, wherein a sample is continuously illuminated in a spatial pattern on each phase of the plurality of phases, including: in a plane of a vertical optical axis, using a linear manner The spatial pattern is transferred to determine each position of a series of positions based on the periodicity of the spatial pattern. The method of optical image formation according to claim 1, wherein a sample is continuously illuminated in a spatial pattern on each phase of the plurality of phases, including: in a plane of a vertical optical axis, in a series The space pattern is rotated at each angle of the angle. The method of optical image formation of claim 1, wherein continuously scanning a sample in a spatial pattern over each of the plurality of phases comprises: focusing the illumination to a first depth of the sample To produce a slice image of the sample at the first depth. The method of optical image formation of claim 12, further comprising: continuously focusing the illumination to a depth of a series of the samples to produce a series of slice images of the sample at each individual depth. A system for forming an optical image, comprising: a light source that generates a light beam and a spatial pattern, continuously illuminating a sample in each phase of the plurality of phases; and a detector that detects the sample at the first a first set of images of a longitudinal resolution and a first lateral resolution, each image being associated with an individual phase of a plurality of phases of the illumination; and a processor processing the first set of images to produce an enhanced slice of the sample Image, the processor module is: processing the first set of images to generate a second set of image data at a second longitudinal resolution greater than the first longitudinal resolution; and A spectrum analysis is performed on the second set of image data to form the enhanced slice image data of the sample at a second lateral resolution greater than the first lateral resolution. The optical image forming system of claim 14, further comprising: an optical component to focus the beam at a first depth of the sample. The optical image forming system of claim 15, further comprising: a controller coupled to the optical component, and a module that controls the beam to focus to a second depth of the sample. The optical image forming system of claim 16, wherein the controller comprises a piezoelectric positioner. The optical image forming system of claim 14, wherein the light source comprises a mask having a first periodicity in a one-dimensional space. The optical image forming system of claim 18, wherein the mask is in a two-dimensional space, and further has a second periodicity. The optical image forming system of claim 18, wherein the light source further comprises: a modulator coupled to the mask and changing a module of the mask to adjust the illumination Phase. The optical image forming system of claim 20, wherein the mask module comprises a position of the mask. The optical image forming system of claim 20, wherein the mask module comprises a direction of the mask. The optical image forming system of claim 14, wherein the processor is further configured to: calculate and combine the first set of images to generate the second set of images according to the first set of images Each image of the image is associated with the individual phase of the illumination; for each image of a subset of the second set of images, Fourier analysis is performed to form a spectrum of the image in a spatial spectrum; each of the spectra is combined The individual portions are combined to form a composite spectrum of the enhanced slice image of the sample; and an inverse Fourier transform is used to generate the enhanced slice image to process the composite spectrum. The optical image forming system of claim 14, wherein the light source comprises an LED array structure to produce a movable light pattern.