CN113670878A - Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction - Google Patents

Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction Download PDF

Info

Publication number
CN113670878A
CN113670878A CN202110985056.7A CN202110985056A CN113670878A CN 113670878 A CN113670878 A CN 113670878A CN 202110985056 A CN202110985056 A CN 202110985056A CN 113670878 A CN113670878 A CN 113670878A
Authority
CN
China
Prior art keywords
super
image
resolution
frequency domain
illumination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110985056.7A
Other languages
Chinese (zh)
Other versions
CN113670878B (en
Inventor
汪召军
雷铭
梁言生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CN202110985056.7A priority Critical patent/CN113670878B/en
Publication of CN113670878A publication Critical patent/CN113670878A/en
Application granted granted Critical
Publication of CN113670878B publication Critical patent/CN113670878B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458—Fluorescence microscopy
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6402—Atomic fluorescence; Laser induced fluorescence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6428—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6428—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
    • G01N2021/6439—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" with indicators, stains, dyes, tags, labels, marks
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N2021/6463—Optics
    • G01N2021/6471—Special filters, filter wheel
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N2021/6463—Optics
    • G01N2021/6478—Special lenses

Landscapes

  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Immunology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Pathology (AREA)
  • Optics & Photonics (AREA)
  • Chemical Kinetics & Catalysis (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

The invention discloses a super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction, which comprises the following steps: 1) generating a striped illumination field; 2) illuminating and exciting a sample to be detected to generate a fluorescent signal under the conditions of different stripe directions and different phase shift steps by utilizing a stripe-shaped illuminating light field; 3) collecting fluorescence signals to obtain original fluorescence images excited by different stripe-shaped illumination light fields; 4) if the original fluorescent image does not comprise the out-of-focus background, processing the original fluorescent image without the out-of-focus background by using a hybrid reconstruction algorithm of a space-frequency domain to obtain a super-resolution image; if the original fluorescence image comprises the out-of-focus background, the original fluorescence image comprising the out-of-focus background is processed by utilizing a space-frequency domain hybrid reconstruction algorithm, and a super-resolution image with a chromatography effect is obtained. The invention reduces the calculated amount required in the reconstruction process by more than two times, and can increase the reconstruction speed to more than 80 times of the prior super-resolution reconstruction algorithm.

Description

Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction
Technical Field
The invention belongs to the technical field of optics, relates to a light illumination microscopic imaging method, and particularly relates to a super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction, which can be widely applied to the research in the fields of biology, medicine, microelectronics, material science and the like.
Background
The spatial resolution of the conventional optical microscope is limited by the diffraction limit of light, and can only reach half the wavelength order of light, thereby greatly limiting the application range of the optical microscope. How to realize imaging with higher spatial resolution has been one of the important research topics in the field of optical microscopy. Overcoming the diffraction limit is essential in enabling the system to distinguish between fluorescent molecules in the diffraction limited region and can generally be achieved by two types of methods. The first category of methods achieves the goal of distinguishing between different fluorescent molecules by randomly activating and localizing a Single fluorescent Molecule within a diffraction limited region at different time points, including Photoactivated Localization Microscopy (PALM) and random Optical Reconstruction Microscopy (STORM), both collectively referred to as Single Molecule Localization Microscopy (SMLM). The second category is to differentiate fluorescent molecules in a diffraction-limited region by modulating their Emission signals with a special Illumination light field, including Stimulated Emission Depletion (STED) and Super-resolution Structured Illumination Microscopy (SR-SIM). The highest resolution of the current super-resolution fluorescence microscopic imaging technology is close to the resolution level of an electron microscope, a powerful tool is provided for modern biomedicine, and related researches are pushed to a new depth. Among many super-resolution imaging methods, the structured light illumination microscope system SIM has the highest imaging rate and the lowest excitation power density (1W/cm)2) Therefore, the dynamic super-resolution observation can be carried out for a long time. In addition, the linear SIM is compatible with the traditional fluorescent molecules and fluorescent dyes, special optical switch dyes or proteins are not needed, and the application range of super-resolution imaging is greatly expanded. These advantages make SIM attractive for dynamic behavior observation of organelles, biological macromolecules and assemblies thereof.
In the imaging process of the SR-SIM, the imaging speed and the user experience are determined by the speed of the super-resolution reconstruction speed. So far, most SR-SIM reconstruction algorithms are based on the frequency domain reconstruction methods constructed by Heintzmann and Gustafsson. The method comprises the steps of firstly, transforming all collected original images into a frequency domain, and solving through a group of linear equations to obtain a zeroth-order component (low-frequency information) and positive and negative first-order components (high-frequency information) of a sample. Then, as a preparatory step to the deconvolution process, each component is multiplied by the conjugate of the optical transfer function, respectively, and moved to their true positions. Finally, by superimposing these components and dividing their sum by the square of the extended optical transfer function plus a small constant, the spectrum is inverse transformed back into real space to obtain a super-resolved image and the deconvolution operation (which can typically be done with wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution, Hessian deconvolution, etc.) is completed. Furthermore, for thicker biological samples, to suppress background fluorescence and periodic cellular artifacts, the spectral components are typically multiplied by an empirical attenuation function to obtain super-resolution images with light slicing capability. The specific flow is shown in fig. 2. The reconstruction approach described above is very effective in improving spatial resolution and slicing capability and thus finds wide application, including in commercial SR-SIM devices and in Open source toolkits such as fair-SIM, Open-SIM and SIMToolbox. However, the mainstream SIM reconstruction algorithm is based on frequency domain reconstruction, and involves a large amount of complex operations, so the reconstruction process is time-consuming and is not favorable for real-time super-resolution imaging.
Unlike the frequency domain reconstruction method described above, Dan et al propose reconstruction in real space rather than Fourier space, thereby greatly simplifying the reconstruction workflow (hereinafter referred to as spatial domain reconstruction, SDR, patent No.: ZL 201911238624.6). This approach reduces most of the major steps in the conventional reconstruction protocol (including fourier transform, spectral separation, spectral shift, spectral concatenation, inverse fourier transform, etc.) to simple multiplication and summation operations in real space. Thereby greatly shortening the execution time of reconstruction. However, the reconstruction result of SDR is different from that of the frequency domain reconstruction method in nature, and is equivalent to a reconstruction protocol without deconvolution in nature, and the spectral components O (k) H (k), O (k) H (k + k)0) Reacting O (k) H (k-k)0) And (4) directly superposing, and converting the superposed frequency spectrum back to a real space without OTF compensation to obtain an intermediate super-resolution image without deconvolution. This intermediate image is finally treated as a new image taken with a reduced PSF for conventional wiener deconvolution. This work flow has realized 7 times rebuild speed and has promoted, has the problem: 1) since the SDR reconstruction method lacks the OTF compensation step, its spatial resolution is lower than the frequency domain reconstruction method, as shown in fig. 4 (b). 2) The SDR reconstruction method does not consider a defocused background, so that the method is only suitable for two-dimensional super-resolution imaging of a thin sample with the thickness of micron order, and cannot image a thicker sample.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction, which has more concise reconstruction steps, can reduce the calculation amount required by the reconstruction process by more than two times without sacrificing any image quality, has higher parallelism and high reconstruction speed, and can properly solve three-dimensional imaging.
In order to achieve the purpose, the invention adopts the following technical scheme:
a super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction is characterized in that: the super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction comprises the following steps:
1) generating a striped illumination field;
2) illuminating and exciting a sample to be detected to generate a fluorescent signal under the conditions of different stripe directions and different phase shift steps by utilizing the stripe-shaped illuminating light field obtained in the step 1);
3) collecting fluorescence signals to obtain original fluorescence images excited by different stripe-shaped illumination light fields;
4) if the original fluorescence image does not comprise the defocused background, executing step 5); if the original fluorescence image comprises the defocused background, executing step 6);
5) processing an original fluorescence image without an out-of-focus background by using a hybrid reconstruction algorithm of an air-frequency domain to obtain a super-resolution image;
6) and processing the original fluorescence image comprising the defocused background by using a space-frequency domain hybrid reconstruction algorithm to obtain a super-resolution image with a tomography effect.
The intensity distribution of the striped illumination light field in the step 1) is expressed as formula (1):
Figure BDA0003230370000000041
wherein:
r is the coordinate of a two-dimensional plane;
d is different stripe directions, and d is 1, 2, 3;
i is different phase shift steps, i is 1, 2, 3;
Idaverage light intensity for the striped illumination field;
mdthe modulation degree of the fringe-like illumination light field;
kdwave vectors that are fringes;
Figure BDA0003230370000000042
the corresponding phase of the light field is illuminated for the current stripe.
The specific implementation manner of the step 2) is as follows:
2.1) setting the initial phase of the fringe-like illumination field
Figure BDA0003230370000000043
2.2) controlling the rotation and the movement of the strip-shaped illumination light field in the plane of the sample to be detected, wherein the rotation directions are 0 degree, 120 degrees and 240 degrees, each direction is three-step phase shift, and the movement amount of each step is
Figure BDA0003230370000000044
Generating 3 × 3 different striped illumination fields in total;
2.3) illuminating by using different stripe-shaped illuminating light fields obtained in the step 2.2) and exciting the sample to be detected to generate a fluorescence signal.
The specific implementation manner of the step 3) is as follows: respectively collecting the fluorescence signals generated in the step 2) by an area array digital camera to obtain an original fluorescence image D excited by different striped illumination light fieldsd，i(r) wherein: r is the coordinate of a two-dimensional plane; d is the different stripe direction, d is 1, 2, 3; i indicates the number of different phase shift steps, i ═ 1, 2, 3.
The specific implementation manner of the step 5) is as follows:
5.1) calculating the weight image w corresponding to each original fluorescence imaged,i(r)：
According to the modulation degree m of each stripe-shaped illumination light fielddWave vector kdAnd the phase of each of the striped illumination fields
Figure BDA0003230370000000051
Calculating the weight image w corresponding to each original fluorescence imaged,i(r)；
5.2) filtering the original fluorescence image:
setting the optical transfer function of the optical system as H (k);
respectively using the conjugate H of the optical transfer function of the optical system to the original fluorescence image*(k) Performing low-pass filtering on the filter to obtain a filtered image;
5.3) dot product:
respectively connecting the filtered images with the weighted image wd,i(r) phase point multiplication, and superposing the multiplication results to obtain a non-deconvoluted super-resolution image ISR_woDeconv(r)；
5.4) deconvolution:
using non-deconvoluted super-resolution images ISR_woDeconv(r) and square of extended optical transfer function
Figure BDA0003230370000000052
Completing deconvolution operation to obtain a final super-resolution image;
square of the extended optical transfer function
Figure BDA0003230370000000053
Is prepared by reacting | H (k) & gtY2According to different wave vectors kdAnd translating and superposing the components.
The weighted image w corresponding to the original fluorescence image in the step 5.1) aboved,iThe expression for (r) is:
Figure BDA0003230370000000054
wherein:
mdthe modulation degree of the fringe-like illumination light field;
kdis a wave vector;
Figure BDA0003230370000000061
an initial phase of the illumination field for the fringe pattern;
Figure BDA0003230370000000062
is the amount of phase shift;
the expression of the optical transfer function of the optical system in the step 5.2) as h (k) is:
Figure BDA0003230370000000063
wherein:
Figure BDA0003230370000000064
kc＝2NA/λ；
the specific implementation manner of performing the low-pass filtering in the step 5.2) is as follows: respectively low-pass filtering the original fluorescence image shot in the step 3) by taking the conjugate distribution H (k) of the optical transfer function H (k) of the optical system as a filter to obtain a filtered image Dfd，i(r)；
Dfd，i(r)＝ifft(fft(Dd，i)·H*(k))；
The deconvolved super-resolution image I in the step 5.3)SR_woDeconvThe expression for (r) is:
Figure BDA0003230370000000065
wherein: w is ad,i(r) is a weighted image corresponding to the original fluorescence image;
the deconvolution operation in the step 5.3) is wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution or Hessian deconvolution;
when the deconvolution operation is wiener deconvolution, the expression of the wiener deconvolution is as follows:
Figure BDA0003230370000000066
wherein:
a (k) is an apodization function;
k represents a two-dimensional coordinate of the spectrum space;
alpha is a wiener deconvolution parameter;
the expression of the apodization function is:
Figure BDA0003230370000000071
wherein k isdA frequency corresponding to the maximum range of the spread spectrum;
expanding the square of the optical transfer function in said step 5.4)
Figure BDA0003230370000000072
The expression of (1) is:
Figure BDA0003230370000000073
the specific implementation manner of the step 6) is as follows:
6.1) calculating the weight image w corresponding to each original fluorescence imaged,i(r)：
According to the modulation degree m of each stripe-shaped illumination light fielddWave vector kdAnd the phase of each of the striped illumination fields
Figure BDA0003230370000000076
Calculating the weight image w corresponding to each original fluorescence imaged,i(r)；
6.2) filtering the original fluorescence image:
setting the optical transfer function of the optical system as H (k), and introducing an attenuation function [1-a (k) ], wherein a (k) is a Gaussian distribution; performing band-pass filtering on the original fluorescence image shot in the step 3) by respectively taking conjugate H (k) 1-a (k) of the attenuated optical transfer function as a filter to obtain a filtered image;
6.3) dot product:
respectively connecting the filtered images with the weighted image wd,i(r) phase point multiplication, and superposing the multiplication results to obtain a non-deconvoluted super-resolution image ISR_woDeconv_att(r)；
6.4) deconvolution:
using non-deconvoluted super-resolution images ISR_woDeconv_att(r) and square of the expanded attenuated optical transfer function
Figure BDA0003230370000000074
Completing deconvolution operation to obtain a final super-resolution image with a chromatography effect;
the square of the extended attenuated optical transfer function
Figure BDA0003230370000000075
Is prepared by reacting | H (k) & gtY2[1-a(k)]According to different wave vectors kdAnd translating and superposing the components.
The weighted image w corresponding to the original fluorescence image in the step 6.1) aboved,iThe expression for (r) is:
Figure BDA0003230370000000081
wherein:
mdthe modulation degree of the fringe-like illumination light field;
kdis a wave vector;
Figure BDA0003230370000000082
an initial phase of the illumination field for the fringe pattern;
Figure BDA0003230370000000083
is the amount of phase shift;
the expression of the optical transfer function of the optical system in step 6.2) as h (k) is:
Figure BDA0003230370000000084
wherein:
Figure BDA0003230370000000085
kc＝2NA/λ；
the expression for the decay function in step 6.2) is:
Figure BDA0003230370000000086
wherein:
aattthe attenuation amplitude parameter is a value range from 0 to 1;
kσis an adjustable empirical parameter;
the specific implementation manner of performing the band-pass filtering in the step 6.2) is as follows: optical transfer function of attenuation by optical system H (k) [1-a (k)]Conjugate distribution of (1-a (k))]Is a filter; respectively carrying out band-pass filtering on the original fluorescence images shot in the step 3) to obtain filtered images Dfd，i(r)；
Dfd，i(r)＝ifft(fft(Dd，i)·H*(k)[1-a(k)])；
The deconvolved super-resolution image I in the step 6.3)SR_woDeconv_attThe expression for (r) is:
Figure BDA0003230370000000091
the deconvolution operation in the step 6.4) is wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution or Hessian deconvolution; when the deconvolution operation is wiener deconvolution, the expression of the wiener deconvolution is as follows:
Figure BDA0003230370000000092
wherein:
a (k) is an apodization function;
k represents a two-dimensional coordinate of the spectrum space;
alpha is a wiener deconvolution parameter;
the expression of the apodization function is
Figure BDA0003230370000000093
K isdA frequency corresponding to the maximum range of the spread spectrum;
the expression for the square of the attenuated optical transfer function expanded in said step 6.4) is:
Figure BDA0003230370000000094
an imaging system for implementing the super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as described above, characterized in that: the imaging system comprises a memory, wherein the memory stores the super-resolution structured light illumination microscopic imaging method based on the space-frequency domain hybrid reconstruction, and the imaging system executes the imaging method in the memory when in operation.
A computer-readable storage medium characterized by: the computer readable storage medium stores a computer program capable of executing the super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as described above.
Compared with the prior art, the invention has the advantages that:
in order to fundamentally solve the problems that the traditional reconstruction method is low in reconstruction speed and cannot perform three-dimensional imaging, the invention provides a super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction (the flow is shown in figure 3), and the reconstruction result obtained by the method is completely equivalent to a classical frequency domain reconstruction algorithm. Compared with the classical super-resolution reconstruction algorithm, the space-frequency domain hybrid reconstruction algorithm has more concise reconstruction steps, can reduce the calculation amount required by the reconstruction process by more than two times without sacrificing any image quality, and has higher parallelism. In addition, under the acceleration of the GPU, the reconstruction speed can be increased to more than 80 times of that of a classical super-resolution reconstruction algorithm by the aid of the algorithm. The simple and rapid reconstruction method provides an important means for realizing real-time super-resolution microscopic reconstruction and result display.
Compared with the classic frequency domain reconstruction algorithm, the space-frequency domain hybrid reconstruction algorithm only comprises filtering, dot multiplication, summation and final deconvolution operation, so that the super-resolution reconstruction speed can be remarkably increased. In order to prove the speed advantage of the space-frequency domain hybrid reconstruction algorithm, the execution time of the classical frequency domain reconstruction algorithm and the space-frequency domain hybrid reconstruction algorithm under different input image sizes in the Matlab 2020a environment is tested. Wherein each result is represented by the mean and standard deviation of 2000 replicates. The parameter estimation times of both algorithms are excluded, considering that the light field parameters can be reused in hundreds of consecutive super-resolution frames. In addition, some static calculations that are not related to sample distribution are not time consuming. As can be seen, for input images with the size of 256 multiplied by 256 to 1024 multiplied by 1024, the execution speed of the space-frequency domain hybrid reconstruction algorithm provided by the method under the CPU is improved by 2 to 3 times compared with that of a classical frequency domain reconstruction algorithm.
In addition, the reconstruction speed improvement brought by the spatial domain processing is perfectly compatible with parallel computing tools such as GPU acceleration and the like. Moreover, as the space domain processing process comprises more similar calculations than the classical frequency domain reconstruction algorithm, the difference between the operation speeds of the two under the GPU operation environment is further expanded. To confirm this, the speed of operation of the two algorithms in the GPU environment was tested on the same graphics device (Intel Core i7-9700H @3.6GHz, DRR 43200 MHz 16GB, NVIDIA GeForce GTX 16606 GB). The result shows that the growth rate of the classical frequency domain reconstruction algorithm and the space-frequency domain hybrid reconstruction algorithm is 20 and 32 respectively. That is, the GPU acceleration further expands the reconstruction speed advantage of the space-frequency domain hybrid reconstruction algorithm. Finally, the GPU-assisted space-frequency domain hybrid reconstruction algorithm can increase the reconstruction speed to be more than 80 times that of the traditional frequency domain reconstruction algorithm. Due to the fact that the reconstruction speed is high, the space-frequency domain hybrid reconstruction algorithm can achieve real-time reconstruction under all common image sizes. In contrast, classical frequency domain reconstruction algorithms fail to meet the maximum acquisition speed at most image sizes under similar circumstances.
Drawings
FIG. 1 is a light path diagram of an interferometric fringe SIM super-resolution microscope system based on spatial light modulator SLM modulation and laser illumination;
the reference numbers in the figures are: the system comprises a 1-laser illumination source, a 2-polarization beam splitter, a 3-half wave plate, a 4-spatial light modulator SLM, a 5-quarter wave plate, a 6-first lens, a 8-second lens, a 9-third lens, a 7-spatial filter, a 10-dichroic mirror, an 11-reflector, a 12-objective lens, a 13-sample and objective table, a 14-emission filter, a 15-barrel lens and a 16-area array digital camera.
FIG. 2 is a flow chart of a classical frequency domain reconstruction algorithm;
FIG. 3 is a flow chart of a space-frequency domain hybrid reconstruction algorithm according to the present invention;
FIG. 4 is a mitochondrial super-resolution image effect comparison reconstructed by a classical frequency domain reconstruction algorithm and a space-frequency domain hybrid reconstruction algorithm;
fig. 5 is a super-resolution imaging result diagram of tubulin in three-dimensional distribution reconstructed by the space-frequency domain hybrid reconstruction algorithm.
Detailed Description
The invention is further described with reference to the following figures and specific embodiments.
The invention can be directly implemented on a mainstream SIM super-resolution microscope system, including but not limited to an interference fringe SIM super-resolution microscope system based on SLM modulation and laser illumination, a projection fringe SIM super-resolution microscope system based on DMD modulation and LED illumination, a projection fringe SIM super-resolution microscope system based on a lattice diffraction grating and LED illumination and the like, and basically, no change is needed on hardware. The following will take a self-built interferometric fringe illumination SIM super-resolution microscope system based on spatial light modulator modulation and laser illumination as an example, and a detailed description will be given to a specific implementation of the method.
The schematic diagram of the optical path of the system is shown in FIG. 1: the device comprises a laser illumination light source 1 after beam expansion and collimation, a beam splitter 2 arranged after beam expansion and collimation laser beams, a half-wave plate 3 and a spatial light modulator 4 which are sequentially arranged on a transmission light path of a polarization beam splitter 2, a quarter-wave plate 5 and a first lens 6 which are arranged on a reflection light path of the polarization beam splitter 2, a spatial filter 7 arranged at the rear end of the first lens 6, a telescopic system which is arranged behind the spatial filter 7 and consists of a second lens 8 and a third lens 9, a dichroic mirror 10 arranged behind the telescopic system, a reflecting mirror 11 arranged on a transmission light path of the dichroic mirror 10, a microscope objective 12, a sample and objective table 13, a transmitting light filter 14 and a barrel lens 15 which are arranged on a reflection light path of the dichroic mirror 10, and a scientific grade CMOS camera 16 arranged behind the barrel lens 15. The spatial light modulator 6 is a reflective ferroelectric liquid crystal spatial light modulator. As with the mainstream structured illumination microscopy, rotation and phase shifting of the fringes is achieved by switching the pattern on the spatial light modulator. And original images under different illumination stripes are collected by the area-array camera and transmitted to the PC. Hardware synchronous control, image acquisition and processing software related to the system are designed and developed by self.
Example 1
In this embodiment, the interferometric fringe illumination SIM super-resolution microscope system based on spatial light modulator SLM modulation and laser illumination is used to image mitochondria in COS7 cells, and since the distribution of mitochondria in COS7 cells is relatively dispersed, a local area can be processed according to a background without defocus, which is specifically realized by the following steps:
step 1, using the interferometric fringe SIM super-resolution microscope system based on spatial light modulator SLM modulation and laser illumination shown in fig. 1, making 488nm wavelength laser after collimation and beam expansion incident to a polarization beam splitter 2, penetrating a half-wave plate 3, vertically irradiating a spatial light modulator SLM4 and returning in an original path, and generating a vertically linear polarization diffraction light beam on a reflection light path of the polarization beam splitter 2. The vertically polarized diffracted beam is converted to circularly polarized light by the quarter-wave plate 5 and converged by the lens 6, the converged multi-order diffracted zero-order and other higher-order diffracted beams are blocked by the customized spatial filter 7 (shown in fig. 1), and only the ± 1-order diffracted beams are transmitted, including two beams. Subsequently, the two beams are relayed by the telescopic system to the objective lens back pupil and enter the microscope objective lens 12, and the two tilted circularly polarized light beams interfere with each other at the focal plane and form a fringe-like illumination light field for illuminating the sample.
The intensity of the fringe illumination light field can be expressed in the form of a cosine function, satisfying the distribution of equation (1):
Figure BDA0003230370000000131
where r is the coordinate of the two-dimensional plane, the subscript d (1, 2, 3) indicates different stripe directions, the subscript I (1, 2, 3) indicates different phase shift steps, IdMean intensity, m, of a fringe-like illumination fielddFor illuminating the modulation, k, of the light field in stripesdThe wave vector representing the fringes is then calculated,
Figure BDA0003230370000000132
the phase corresponding to the current fringe light field;
step 2, placing the COS7 cell sample on an objective table, adjusting the COS7 cell sample to an objective focal plane, and exciting marked mitochondria to emit fluorescence by utilizing a striped illumination light field for illumination; the fluorescence is collected by the objective lens, transmitted through the emission optical filter 14, the tube lens 15, and finally imaged on the sensor of the area-array digital camera 16.
Assuming an initial phase of the light field of
Figure BDA0003230370000000141
And controlling a Spatial Light Modulator (SLM) to rotate and translate the striped illumination pattern through loading and refreshing so that the striped illumination light field rotates and moves in a sample plane, wherein the three directions (0 degrees, 120 degrees and 240 degrees) are provided, and each direction is three-step phase shift. Usually, equal-spaced phase shifts are used, with the amount of shift per step being
Figure BDA0003230370000000142
The phase shift amounts of the three illumination patterns in each direction are sequentially set to be 0, 2 pi/3 and 4 pi/3 and are sequentially used for illuminating and exciting the sample to generate a fluorescence signal.
Step 3, the area array digital camera 16 respectively collects the corresponding 9 original fluorescence images, and the images are marked as D1，1，D1，2，D1，3，D2，1，D2，2，D2，3，D3，1，D3，2，D3，3. Assuming a magnification ratio of 1 between the object space and the image space, the raw fluorescence images taken by the camera can be written as: d11(r)，D12(r)，D13(r)、D21(r)，D22(r)，D23(r)、D31(r)，
D32(r)，D33(r); these raw fluorescence images are stored in computer memory, hard disk, or floppy disk.
Step 4, since the distribution of mitochondria in COS7 cells is relatively dispersed, the original fluorescence image without the defocused background can be processed, and the processing result is shown in fig. 4. The specific treatment method comprises the following steps:
firstly, according to the modulation degree of the structured light field in each stripe directionmdWave vector kdInitial phase of
Figure BDA0003230370000000145
Figure BDA0003230370000000146
And amount of phase shift
Figure BDA0003230370000000143
Calculating the weight image w corresponding to each original fluorescence imaged,i(r), the weighted image is calculated by:
Figure BDA0003230370000000144
assuming that the optical transfer function of the optical system is h (k), which is generally a rotationally symmetric distribution, it can be obtained by measuring the point spread function of the optical system and performing fourier transform. H (k) has in theory a number of representations, typically:
Figure BDA0003230370000000151
wherein
Figure BDA0003230370000000152
kc2NA/λ. Passing | H (k) through2Wave vectors + -k corresponding to different stripe directionsdThe square of the extended optical transfer function can be obtained by translation and superposition
Figure BDA0003230370000000153
Figure BDA0003230370000000154
Then, taking conjugate distribution H (k) of optical transfer function H (k) of the optical system as a filter; low-pass filtering the 9 original fluorescence images shot in the step 2) respectivelyWave to obtain a filtered image Dfd，i(r)；
Dfd，i(r)＝ifft(fft(Dd，i)·H*(k)) (5)
The 9 filtered images are respectively compared with the weighted image wd,i(r) phase point multiplication, and superposing the multiplication results to obtain a non-deconvoluted super-resolution image ISR_woDeconv(r)；
Figure BDA0003230370000000155
Finally, the deconvolved super-resolution image I is usedSR_woDeconv(r) and square of extended optical transfer function
Figure BDA0003230370000000156
And (4) completing deconvolution operation (usually adopting wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution, Hessian deconvolution and the like) to obtain a final super-resolution image.
Taking wiener deconvolution as an example:
Figure BDA0003230370000000157
wherein A (k) is an apodization function, k represents a two-dimensional coordinate of the spectrum space, and α is a wiener deconvolution parameter, assuming kdFor frequencies corresponding to the maximum range of the spread spectrum, the apodization function is expressed as
Figure BDA0003230370000000158
Fig. 4 is a wide field illumination image and a striped SIM super-resolution image contrast image of mitochondria within COS7 cells obtained from an interferometric striped illuminated SIM super-resolution microscopy system based on spatial light modulator SLM modulation and laser illumination. The experiment used a 100 × microscope objective with a numerical aperture NA of 1.49. Fig. 4(a) is a general wide-field fluorescence image, fig. 4(b) is a super-resolution image obtained using a spatial domain reconstruction method proposed by Dan et al, fig. 4(c) is a super-resolution image obtained using a space-frequency domain hybrid reconstruction method proposed by the present invention, and fig. 4(d) is a super-resolution image obtained using a classical frequency domain reconstruction method. As can be seen by comparison, the resolution of the image obtained by the method is obviously higher than that of a common wide-field image and a space domain reconstruction algorithm, and the effect is completely the same as that of the traditional classical frequency domain reconstruction method. Meanwhile, under the support of the GPU, the reconstruction speed of the space-frequency domain hybrid reconstruction-based method is increased by more than 80 times compared with the original method (see table 1).
TABLE 1 comparison of the execution times of two reconstruction algorithms for different original image sizes
Figure BDA0003230370000000161
Example 2
In this example, the same set of system was used to image tubulin in COS7 cells, which has a continuous three-dimensional structure, so that it was necessary to scan layer by layer to collect data, and then process the data according to the original fluorescence image with out-of-focus background. The method is realized by the following steps:
step 1, using the interferometric fringe SIM super-resolution microscope system based on spatial light modulator SLM modulation and laser illumination shown in fig. 1, making the collimated and expanded 488nm wavelength laser enter the polarization beam splitter 2, vertically irradiating the spatial light modulator SLM4 through a half-wave plate and returning in the original path, and generating a vertically linear polarization diffraction beam on the reflection light path of the polarization beam splitter. The vertically polarized light beam is converted into circularly polarized light by the quarter-wave plate 5 and converged by the lens 6, the converged multi-order diffracted zero-order and other high-order diffracted light beams are blocked by the customized spatial filter 7 (shown in fig. 1), and only the ± 1-order diffracted light beams are transmitted, including two light beams. Subsequently, the two beams are relayed by the telescopic system to the objective lens back pupil and enter the microscope objective lens 12, and the two tilted circularly polarized light beams interfere with each other at the focal plane and form a fringe-like illumination light field for illuminating the sample.
The intensity of the fringe illumination light field can be expressed in the form of a cosine function, satisfying the distribution of equation (1):
Figure BDA0003230370000000171
where r is the coordinate of the two-dimensional plane, the subscript d (1, 2, 3) indicates different stripe directions, the subscript I (1, 2, 3) indicates different phase shift steps, IdMean intensity, m, of a fringe-like illumination fielddFor illuminating the modulation, k, of the light field in stripesdThe wave vector representing the fringes is then calculated,
Figure BDA0003230370000000172
the phase corresponding to the current fringe light field;
step 2, placing the COS7 cell sample on an objective table, adjusting the COS7 cell sample to an objective focal plane, illuminating by using a stripe-shaped illumination light field, and exciting a marked microtube to emit fluorescence; the fluorescence is collected by the objective lens, transmitted through the emission optical filter 14, the tube lens 15, and finally imaged on the sensor of the area-array digital camera 16. For an original fluorescence image with an out-of-focus background, a sample needs to be scanned by using an electric translation table in the shooting process, each layer of sample is imaged respectively, and the imaging process and the image reconstruction process of each layer are completely the same.
Assuming that the initial phase of the fringe illumination field is
Figure BDA0003230370000000173
And controlling a Spatial Light Modulator (SLM) to rotate and translate the striped illumination pattern through loading and refreshing so that the striped illumination light field rotates and moves in a sample plane, wherein the three directions (0 degrees, 120 degrees and 240 degrees) are provided, and each direction is three-step phase shift. Usually, equal-spaced phase shifts are used, with the amount of shift per step being
Figure BDA0003230370000000174
The phase shift amounts of the three illumination patterns in each direction are sequentially set to be 0, 2 pi/3 and 4 pi/3 and are sequentially used for illuminating and exciting the sample to generate a fluorescence signal.
Step 3, taking any layer of the images as an example, the area array digital camera respectively collects corresponding 9 original fluorescence images, and the images are recorded as D1，1，D1，2，D1，3，D2，1，D2，2，D2，3，D3，1，D3，2，D3，3. Assuming a magnification ratio of 1 between the object space and the image space, the raw fluorescence images taken by the camera can be written as: d11(r)，D12(r)，D13(r)、D21(r)，D22(r)，D23(r)、
D31(r)，D32(r)，D33(r); these raw fluorescence images are stored in computer memory, hard disk, or floppy disk. And after the data of the layer is acquired, moving the translation stage, and repeating the acquisition operation until the whole region of interest is traversed and covered. In the acquisition process, the scanning step length is 200nm, 35 layers are acquired in total, and the microtubules are seen to be distributed in the depth range of 7 μm.
Step 4, because tubulin in the COS7 cell has a continuous three-dimensional structure, the treatment can be carried out layer by layer according to a thick sample, and the treatment mode of each layer is completely the same.
Firstly, according to the modulation degree m of the structured light field in each fringe directiondWave vector kdInitial phase of
Figure BDA0003230370000000186
Figure BDA0003230370000000187
And amount of phase shift
Figure BDA0003230370000000188
Calculating the weight image w corresponding to each original fluorescence imaged,i(r), the weighted image is calculated by:
Figure BDA0003230370000000181
assuming that the optical transfer function of the optical system is h (k), which is generally a rotationally symmetric distribution, it can be obtained by measuring the point spread function of the optical system and performing fourier transform. H (k) has in theory a number of representations, typically:
Figure BDA0003230370000000182
wherein
Figure BDA0003230370000000183
kc2NA/λ. Introducing attenuation function
Figure BDA0003230370000000184
Wherein a isattFor the attenuation amplitude parameter, the value ranges from 0 to 1 (usually 0.99), and k isσIs an adjustable empirical parameter. Passing | H (k) through2[1-a(k)]Wave vectors + -k corresponding to different stripe directionsdThe square of the attenuated expanded optical transfer function can be obtained by translation and superposition
Figure BDA0003230370000000185
Figure BDA0003230370000000191
Then, the attenuation optical transfer function H (k) [1-a (k)]Conjugate distribution of (1-a (k))]Is a filter; respectively carrying out band-pass filtering on the 9 fluorescence images shot in the step 2) to obtain a filtered image Dfd，i(r)；
Dfd，i(r)＝ifft(fft(Dd，i)·H*(k)[1-a(k)]) (13)
The 9 filtered images are respectively compared with the weighted image wd,i(r) phase point multiplication, and superposing the multiplication results to obtain a non-deconvoluted super-resolution image ISR_woDeconv_att(r)；
Figure BDA0003230370000000192
Finally, the deconvolved super-resolution image I is usedSR_woDeconv_att(r) and the square of the attenuated expanded optical transfer function
Figure BDA0003230370000000193
And (4) completing deconvolution operation (which can be generally implemented by wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution, Hessian deconvolution and the like) to obtain a final super-resolution image. Taking wiener deconvolution as an example:
Figure BDA0003230370000000194
wherein A (k) is an apodization function, k represents a two-dimensional coordinate of the spectrum space, and α is a wiener deconvolution parameter, assuming kdFor frequencies corresponding to the maximum range of the spread spectrum, the expression of the apodization function is:
Figure BDA0003230370000000195
fig. 5 is an image of COS7 intracellular tubulin obtained from an interferometric fringe illumination SIM super-resolution microscopy system based on spatial light modulator SLM modulation and laser illumination. The experiment used a 100 × microscope objective with a numerical aperture NA of 1.49. Fig. 5(a) is a three-dimensional rendering of a multi-layer image obtained by using the space-frequency domain reconstruction algorithm proposed in the present invention, and fig. 5(b) -5(d) are wide-field images or super-resolution images of a certain layer in the multi-layer image, wherein fig. 5(b) is the wide-field image; fig. 5(c) is a super-resolution image obtained using the space-frequency domain hybrid reconstruction method proposed by the present invention, and fig. 5(d) is a super-resolution image obtained using a classical frequency domain reconstruction method. It can be easily found that the resolution of the super-resolution image of the thick sample obtained by the method is obviously higher than that of the common wide-field image, and the effect is completely the same as that of the traditional classical frequency domain reconstruction method. Meanwhile, under the support of the GPU, the reconstruction speed of the space-frequency domain hybrid reconstruction-based method is increased by more than 80 times compared with the original method (see table 1).
The invention also provides a super-resolution imaging system using the stripe-shaped illumination light field for illumination, which comprises a processor and a memory, wherein the memory stores a computer program, and the super-resolution image reconstruction processing method of the step 4 is executed when the computer program runs in the processor.
The present application also provides a computer-readable storage medium storing a program which, when executed, implements the above-described super-resolution image reconstruction processing method. In some possible embodiments, the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the method part of the description above, when said program product is run on the terminal device.
A program product for implementing the above method, which may employ a portable compact disc read only memory (CD-ROM) and include program code, may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in the present invention, the computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Claims (10)

1. A super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction is characterized in that: the super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction comprises the following steps:
1) generating a striped illumination field;
2) illuminating and exciting a sample to be detected to generate a fluorescent signal under the conditions of different stripe directions and different phase shift steps by utilizing the stripe-shaped illuminating light field obtained in the step 1);
3) collecting fluorescence signals to obtain original fluorescence images excited by different stripe-shaped illumination light fields;
4) if the original fluorescence image does not comprise the defocused background, executing step 5); if the original fluorescence image comprises the defocused background, executing step 6);
5) processing an original fluorescence image without an out-of-focus background by using a hybrid reconstruction algorithm of an air-frequency domain to obtain a super-resolution image;
6) and processing the original fluorescence image comprising the defocused background by using a space-frequency domain hybrid reconstruction algorithm to obtain a super-resolution image with a tomography effect.
2. The super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in claim 1, wherein: the intensity distribution of the striped illumination light field in the step 1) is expressed as formula (1):
Figure FDA0003230369990000011
wherein:
r is the coordinate of a two-dimensional plane;
d is different stripe directions, and d is 1, 2, 3;
i is different phase shift steps, i is 1, 2, 3;
Idaverage light intensity for the striped illumination field;
mdthe modulation degree of the fringe-like illumination light field;
kdwave vectors that are fringes;
Figure FDA0003230369990000021
the corresponding phase of the light field is illuminated for the current stripe.
3. The super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in claim 2, wherein: the specific implementation manner of the step 2) is as follows:
2.1) setting the initial phase of the fringe-like illumination field
Figure FDA0003230369990000022
2.2) controlling the rotation and the movement of the strip-shaped illumination light field in the plane of the sample to be detected, wherein the rotation directions are 0 degree, 120 degrees and 240 degrees, each direction is three-step phase shift, and the movement amount of each step is
Figure FDA0003230369990000023
Generating 3 × 3 different striped illumination fields in total;
2.3) illuminating by using different stripe-shaped illuminating light fields obtained in the step 2.2) and exciting the sample to be detected to generate a fluorescence signal.
4. The super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in claim 3, wherein: the specific implementation manner of the step 3) is as follows: respectively collecting the fluorescence signals generated in the step 2) by an area array digital camera to obtain an original fluorescence image D excited by different striped illumination light fieldsd，i(r) wherein: r is the coordinate of a two-dimensional plane; d is the different stripe direction, d is 1, 2, 3; i indicates the number of different phase shift steps, i ═ 1, 2, 3.
5. The super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in claim 4, wherein: the specific implementation manner of the step 5) is as follows:
5.1) calculating the weight image w corresponding to each original fluorescence imaged,i(r)：
According to the modulation degree m of each stripe-shaped illumination light fielddWave vector kdAnd the phase of each of the striped illumination fields
Figure FDA0003230369990000024
Calculating the weight image w corresponding to each original fluorescence imaged,i(r)；
5.2) filtering the original fluorescence image:
setting the optical transfer function of the optical system as H (k);
respectively using the conjugate H of the optical transfer function of the optical system to the original fluorescence image*(k) Performing low-pass filtering on the filter to obtain a filtered image;
5.3) dot product:
respectively connecting the filtered images with the weighted image wd,i(r) phase point multiplication, and superposing the multiplication results to obtain a non-deconvoluted super-resolution image ISR_woDeconv(r)；
5.4) deconvolution:
using non-deconvoluted super-resolution images ISR_woDeconv(r) and square of extended optical transfer function
Figure FDA0003230369990000031
Completing deconvolution operation to obtain a final super-resolution image;
square of the extended optical transfer function
Figure FDA0003230369990000032
Is prepared by reacting | H (k) & gtY2According to different wave vectors kdAnd translating and superposing the components.
6. The super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in claim 5, wherein: the above-mentionedStep 5.1) weight image w corresponding to the original fluorescence imaged,iThe expression for (r) is:
Figure FDA0003230369990000033
wherein:
mdthe modulation degree of the fringe-like illumination light field;
kdis a wave vector;
Figure FDA0003230369990000034
an initial phase of the illumination field for the fringe pattern;
Figure FDA0003230369990000035
is the amount of phase shift;
the expression of the optical transfer function of the optical system in the step 5.2) as h (k) is:
Figure FDA0003230369990000041
wherein:
Figure FDA0003230369990000042
kc＝2NA/λ；
the specific implementation manner of performing the low-pass filtering in the step 5.2) is as follows: respectively low-pass filtering the original fluorescence image shot in the step 3) by taking the conjugate distribution H (k) of the optical transfer function H (k) of the optical system as a filter to obtain a filtered image Dfd，i(r)；
Dfd，i(r)＝ifft(fft(Dd，i)·H*(k))；
The deconvolved super-resolution image I in the step 5.3)SR_woDeconvThe expression for (r) is:
Figure FDA0003230369990000043
wherein: w is ad,i(r) is a weighted image corresponding to the original fluorescence image;
the deconvolution operation in the step 5.3) is wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution or Hessian deconvolution;
when the deconvolution operation is wiener deconvolution, the expression of the wiener deconvolution is as follows:
Figure FDA0003230369990000044
wherein:
a (k) is an apodization function;
k represents a two-dimensional coordinate of the spectrum space;
alpha is a wiener deconvolution parameter;
the expression of the apodization function is
Figure FDA0003230369990000045
Wherein k isdA frequency corresponding to the maximum range of the spread spectrum;
expanding the square of the optical transfer function in said step 5.4)
Figure FDA0003230369990000051
The expression of (1) is:
Figure FDA0003230369990000052
7. the super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in claim 4, wherein: the specific implementation manner of the step 6) is as follows:
6.1) calculating the weight image w corresponding to each original fluorescence imaged,i(r)：
According to the modulation degree m of each stripe-shaped illumination light fielddWave vector kdAnd the phase of each of the striped illumination fields
Figure FDA0003230369990000055
Calculating the weight image w corresponding to each original fluorescence imaged,i(r)；
6.2) filtering the original fluorescence image:
setting the optical transfer function of the optical system as H (k), and introducing an attenuation function [1-a (k) ], wherein a (k) is a Gaussian distribution; performing band-pass filtering on the original fluorescence image shot in the step 3) by respectively taking conjugate H (k) 1-a (k) of the attenuated optical transfer function as a filter to obtain a filtered image;
6.3) dot product:
respectively connecting the filtered images with the weighted image wd,i(r) phase point multiplication, and superposing the multiplication results to obtain a non-deconvoluted super-resolution image ISR_woDeconv_att(r)；
6.4) deconvolution:
using non-deconvoluted super-resolution images ISR_woDeconv_att(r) and square of the expanded attenuated optical transfer function
Figure FDA0003230369990000053
Completing deconvolution operation to obtain a final super-resolution image with a chromatography effect;
the square of the extended attenuated optical transfer function
Figure FDA0003230369990000054
Is prepared by reacting | H (k) & gtY2[1-a(k)]According to different wave vectors kdAnd translating and superposing the components.
8. The null-based according to claim 7The super-resolution structured light illumination microscopic imaging method of frequency domain hybrid reconstruction is characterized in that: the weight image w corresponding to the original fluorescence image in the step 6.1)d,iThe expression for (r) is:
Figure FDA0003230369990000061
wherein:
mdthe modulation degree of the fringe-like illumination light field;
kdis a wave vector;
Figure FDA0003230369990000062
an initial phase of the illumination field for the fringe pattern;
Figure FDA0003230369990000063
is the amount of phase shift;
the expression of the optical transfer function of the optical system in step 6.2) as h (k) is:
Figure FDA0003230369990000064
wherein:
Figure FDA0003230369990000065
kc＝2NA/λ；
the expression of the decay function in said step 6.2) is
Figure FDA0003230369990000066
Wherein:
aattthe attenuation amplitude parameter is a value range from 0 to 1;
kσis an adjustable empirical parameter;
the specific implementation manner of performing the band-pass filtering in the step 6.2) is as follows: optical transfer function of attenuation by optical system H (k) [1-a (k)]Conjugate distribution of (1-a (k))]Is a filter; respectively carrying out band-pass filtering on the original fluorescence images shot in the step 3) to obtain filtered images Dfd，i(r)；
Dfd，i(r)＝ifft(fft(Dd，i)·H*(k)[1-a(k)])；
The deconvolved super-resolution image I in the step 6.3)SR_woDeconv_attThe expression for (r) is:
Figure FDA0003230369990000071
the deconvolution operation in the step 6.4) is wiener deconvolution, Richardson-Lucy deconvolution, Total Variation deconvolution or Hessian deconvolution; when the deconvolution operation is wiener deconvolution, the expression of the wiener deconvolution is as follows:
Figure FDA0003230369990000072
wherein:
a (k) is an apodization function;
k represents a two-dimensional coordinate of the spectrum space;
alpha is a wiener deconvolution parameter;
the expression of the apodization function is
Figure FDA0003230369990000073
K isdA frequency corresponding to the maximum range of the spread spectrum;
the expression for the square of the attenuated optical transfer function expanded in said step 6.4) is:
Figure FDA0003230369990000074
9. an imaging system for implementing the super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction as claimed in any one of claims 1 to 8, characterized in that: the imaging system comprises a memory, wherein the memory stores the super-resolution structured light illumination microscopic imaging method based on the space-frequency domain hybrid reconstruction as claimed in any one of claims 1 to 8, and the imaging system executes the imaging method in the memory when in operation.
10. A computer-readable storage medium characterized by: the computer readable storage medium stores a computer program capable of executing the method for super-resolution structured light illumination microscopic imaging based on spatial-frequency domain hybrid reconstruction as claimed in any one of claims 1 to 8.
CN202110985056.7A 2021-08-25 2021-08-25 Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction Active CN113670878B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110985056.7A CN113670878B (en) 2021-08-25 2021-08-25 Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110985056.7A CN113670878B (en) 2021-08-25 2021-08-25 Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction

Publications (2)

Publication Number Publication Date
CN113670878A true CN113670878A (en) 2021-11-19
CN113670878B CN113670878B (en) 2022-09-27

Family

ID=78546452

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110985056.7A Active CN113670878B (en) 2021-08-25 2021-08-25 Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction

Country Status (1)

Country Link
CN (1) CN113670878B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114219715A (en) * 2021-12-17 2022-03-22 广州超视计生物科技有限公司 Quality enhancement method and system for structured illumination light super-resolution microscope image
CN115839936A (en) * 2022-12-12 2023-03-24 之江实验室 Phase-locked detection-based structured light illumination super-resolution microscopic imaging reconstruction method
CN116380408A (en) * 2023-04-10 2023-07-04 南京航空航天大学 Three-dimensional super-resolution flow field measurement method and system based on structured light and light field imaging
CN116402678A (en) * 2022-12-19 2023-07-07 中国科学院苏州生物医学工程技术研究所 Spectrum optimization direct reconstruction method of super-resolution structured light illumination microscope
CN117030674A (en) * 2023-10-09 2023-11-10 中国科学院生物物理研究所 Structured light illuminated three-dimensional super-resolution imaging system and method
WO2024051079A1 (en) * 2022-09-05 2024-03-14 中国科学院苏州生物医学工程技术研究所 Super-resolution microscopic imaging method and system based on active-structured illumination
WO2024160074A1 (en) * 2023-01-31 2024-08-08 深圳市真迈生物科技有限公司 Base calling method and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111077121A (en) * 2019-12-06 2020-04-28 中国科学院西安光学精密机械研究所 Rapid method and system for directly reconstructing structured light illumination super-resolution image in space domain
CN111458318A (en) * 2020-05-12 2020-07-28 西安交通大学 Super-resolution imaging method and system utilizing square lattice structure light illumination

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111077121A (en) * 2019-12-06 2020-04-28 中国科学院西安光学精密机械研究所 Rapid method and system for directly reconstructing structured light illumination super-resolution image in space domain
CN111458318A (en) * 2020-05-12 2020-07-28 西安交通大学 Super-resolution imaging method and system utilizing square lattice structure light illumination

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114219715A (en) * 2021-12-17 2022-03-22 广州超视计生物科技有限公司 Quality enhancement method and system for structured illumination light super-resolution microscope image
WO2024051079A1 (en) * 2022-09-05 2024-03-14 中国科学院苏州生物医学工程技术研究所 Super-resolution microscopic imaging method and system based on active-structured illumination
CN115839936A (en) * 2022-12-12 2023-03-24 之江实验室 Phase-locked detection-based structured light illumination super-resolution microscopic imaging reconstruction method
CN115839936B (en) * 2022-12-12 2023-07-18 之江实验室 Structured light illumination super-resolution microscopic imaging reconstruction method based on phase-locked detection
CN116402678A (en) * 2022-12-19 2023-07-07 中国科学院苏州生物医学工程技术研究所 Spectrum optimization direct reconstruction method of super-resolution structured light illumination microscope
CN116402678B (en) * 2022-12-19 2023-10-20 中国科学院苏州生物医学工程技术研究所 Spectrum optimization direct reconstruction method of super-resolution structured light illumination microscope
WO2024130824A1 (en) * 2022-12-19 2024-06-27 中国科学院苏州生物医学工程技术研究所 Spectrum optimization direct reconstruction structured illumination microscopy method for super-resolution structured illumination microscopy
WO2024160074A1 (en) * 2023-01-31 2024-08-08 深圳市真迈生物科技有限公司 Base calling method and system
CN116380408A (en) * 2023-04-10 2023-07-04 南京航空航天大学 Three-dimensional super-resolution flow field measurement method and system based on structured light and light field imaging
CN116380408B (en) * 2023-04-10 2024-06-07 南京航空航天大学 Three-dimensional super-resolution flow field measurement method and system based on structured light and light field imaging
CN117030674A (en) * 2023-10-09 2023-11-10 中国科学院生物物理研究所 Structured light illuminated three-dimensional super-resolution imaging system and method

Also Published As

Publication number Publication date
CN113670878B (en) 2022-09-27

Similar Documents

Publication Publication Date Title
CN113670878B (en) Super-resolution structured light illumination microscopic imaging method based on space-frequency domain hybrid reconstruction
CN111077121B (en) Rapid method and system for directly reconstructing structured light illumination super-resolution image in space domain
Wang et al. High-speed image reconstruction for optically sectioned, super-resolution structured illumination microscopy
US8019136B2 (en) Optical sectioning microscopy
Poon Optical scanning holography-a review of recent progress
JP6424313B2 (en) Holographic microscope and data processing method for high resolution hologram images
CN110954521B (en) Wide-field super-resolution microscopic imaging method and system thereof
US20090219607A1 (en) Method and apparatus for enhanced resolution microscopy of living biological nanostructures
EP2788820B1 (en) Apparatus for producing a hologram
CN106885796B (en) Super-resolution fluorescence digital holographic tomography microscopic imaging system and method
CN111458318B (en) Super-resolution imaging method and system utilizing square lattice structure light illumination
WO2013018024A1 (en) Apparatus and method for quantitative phase tomography through linear scanning with coherent and non-coherent detection
CN114594588B (en) Structured light illumination microscopic device and method based on grating projection and SLM phase shift
CN115308185A (en) High-speed artifact-free super-resolution structured light illumination microscopic imaging method
JP5085608B2 (en) Wide-field super-resolution optical microscope using a spatial light modulator
JP2022046447A (en) Method for super-resolution of microscope image illuminated in a structured manner and microscope having structured illumination
US11867894B2 (en) Method for SIM microscopy
Joseph et al. Improving the space-bandwidth product of structured illumination microscopy using a transillumination configuration
JP2022527827A (en) Advanced sample imaging using structured illumination microscopy
Dan et al. Super-resolution and optical sectioning integrated structured illumination microscopy
CN113568294B (en) Holographic optical tweezers fusion structure light illumination microscopic system and method
Shabani et al. Simultaneous optical sectioning and super resolution in 2D-SIM using tunable structured illumination
Inochkin et al. Increasing the space-time product of super-resolution structured illumination microscopy by means of two-pattern illumination
Fiolka Seeing more with structured illumination microscopy
Nguyen et al. 3D high speed characterization of phase objects using the transport of intensity equation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant