WO2021227292A1 - 一种直接结构光照明超分辨显微重建方法 - Google Patents

一种直接结构光照明超分辨显微重建方法 Download PDF

Info

Publication number
WO2021227292A1
WO2021227292A1 PCT/CN2020/110154 CN2020110154W WO2021227292A1 WO 2021227292 A1 WO2021227292 A1 WO 2021227292A1 CN 2020110154 W CN2020110154 W CN 2020110154W WO 2021227292 A1 WO2021227292 A1 WO 2021227292A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
modulation
extracted
frequency
stack
Prior art date
Application number
PCT/CN2020/110154
Other languages
English (en)
French (fr)
Inventor
席鹏
姜杉
乔晖
戴琼海
Original Assignee
北京大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京大学 filed Critical 北京大学
Priority to US17/998,484 priority Critical patent/US20230214961A1/en
Publication of WO2021227292A1 publication Critical patent/WO2021227292A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N21/6456Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458Fluorescence microscopy
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/6408Fluorescence; Phosphorescence with measurement of decay time, time resolved fluorescence
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/6486Measuring fluorescence of biological material, e.g. DNA, RNA, cells
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/16Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • G02B21/367Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4084Scaling of whole images or parts thereof, e.g. expanding or contracting in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • G06T7/521Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N2021/6463Optics
    • G01N2021/6467Axial flow and illumination
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10064Fluorescence image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10141Special mode during image acquisition
    • G06T2207/10148Varying focus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Definitions

  • the invention relates to fluorescence microscopy imaging technology, in particular to a direct structured light illumination super-resolution microscopy reconstruction method.
  • the spatial resolution of super-resolution microscopy can reach 10-100 nm, and it can observe the ultrastructure of molecular sites and nerve cells.
  • the intensity of the extinction light of the stimulated emission depletion microscope (STED) is too high and the point scanning time resolution is low, and the single-molecule positioning microscope (SMLM) needs to collect thousands of original images for time resolution.
  • the low rate and high illumination intensity so these two techniques are not suitable for imaging live cells.
  • the structured light illumination super-resolution microscope SIM has the lowest illumination intensity, fast imaging speed, and multi-color imaging capability, which is suitable for rapid imaging of living cells.
  • the present invention proposes a direct structured light illumination super-resolution microscopy reconstruction algorithm, which is different from traditional SIM and image scanning microscopes (ISM, AiryScan, SD-SIM).
  • the present invention proposes a new reconstruction method that can achieve a two-fold higher resolution and no artifact super-resolution microscopy method, and named it as direct structured light illumination super-resolution Microscopy (dSIM, Direct structured illumination microscopy).
  • dSIM direct structured light illumination super-resolution Microscopy
  • Structured light is an illuminating light with periodic stripes.
  • the process of illuminating the sample with different phases is the process of modulating the sample, with different modulation directions, and a different phase in each modulation direction.
  • Structured light illumination can make the high-frequency information of the sample produce a frequency shift, enter the observable optical transfer function OTF, and use the method to move the high-frequency information to the correct position in the frequency domain, thereby expanding the OTF of the optical system, and Obtain super-resolution microscopy.
  • the original image obtained by the structured light microscope is a 3D original image, a 2D original image or a TIRF SIM image of a total internal reflection microscope.
  • the original image is a 3D original image and includes the following steps:
  • the structured light irradiates the sample with N modulation directions, each modulation direction has M phases, N and M are both natural numbers ⁇ 2, and an original image is collected for each phase in each modulation direction to obtain N ⁇ M 3D original images, forming an original image stack;
  • the wavelet packet frequency separation method is used to extract the first modulation frequency K1, the second modulation frequency K2, and the zero frequency K0 of each pixel;
  • interpolation is performed by zero padding in the spatial frequency domain FFT, so that the sampling frequency becomes larger and expanded to more than twice to obtain the first and second extracted images after interpolation Image stack
  • each pixel of the extracted complex modulation image stack calculate the autocorrelation accumulation of the complex modulation signal.
  • M phases of the first modulation frequency K1 generate one autocorrelation image, and obtain N images
  • the auto-correlation image and the second modulation frequency K2 generate an auto-correlation image to obtain N auto-correlation images, thereby using the auto-correlation accumulation of signals of different spatial positions to generate super-resolution images, and each group of original image stacks corresponds to 2N images image;
  • the dSIM intermediate processing results on the first and second modulation frequencies in different modulation directions are added together to generate a final dSIM image.
  • step 2) d) the denoising process uses the spatial frequency domain image to perform Butterworth low-pass filtering.
  • step 3) b) extracting the complex modulation signal from the time evolution of the intensities of the first modulation frequency K1 and the second modulation frequency K2 includes the following steps: first extract the real signal of the modulation frequency using wavelet packets, and then use filtering The generator extracts the complex modulated signal from the modulated signal.
  • step 4 the accumulation of autocorrelation is calculated at each pixel.
  • the M phases of the first and second modulation frequencies generate an autocorrelation image, including the following steps: Calculate the M phases in the same modulation direction In the complex modulation image stack, the autocorrelation accumulation amount of the complex modulation signal at each pixel is generated to generate a result image to obtain the autocorrelation image.
  • the original image is a 2D original image or a total internal reflection microscope image, including the following steps:
  • the structured light irradiates the sample with N modulation directions, each modulation direction has M phases, N and M are both natural numbers ⁇ 2, and an original image is collected for each phase in each modulation direction to obtain N ⁇ M 2D original images or total internal reflection microscope images to form an original image stack;
  • interpolation is performed through spatial frequency domain FFT zero padding, so that the sampling frequency becomes larger and expanded to more than twice to obtain the extracted image after interpolation Stack
  • the autocorrelation accumulation is calculated at each pixel.
  • an autocorrelation image is generated for M phases, and N autocorrelation images are obtained.
  • the image is a super-resolution image, so that the auto-correlation accumulation of signals from different spatial locations is used to generate a super-resolution image;
  • the dSIM intermediate processing results in different modulation directions are summed to generate a final dSIM image.
  • step 2) d) the denoising process uses the spatial frequency domain image to perform Butterworth low-pass filtering.
  • step 3) b) extracting the complex modulation signal from the time evolution of the intensity of the modulation frequency K extracted from each pixel includes the following steps: firstly extract the real signal of the modulation frequency by using wavelet packet, and then use the filter Extract the complex modulated signal from the modulated signal.
  • step 4 the accumulation of autocorrelation is calculated at each pixel, and an autocorrelation image is generated with M phases in each modulation direction, including the following steps: Calculate the complex modulation of M phases in the same modulation direction In the image stack, the autocorrelation accumulation amount of the complex modulation signal at each pixel is generated to generate a result image to obtain the autocorrelation image.
  • the super-resolution microscopy reconstruction algorithm of dSIM of the present invention first extracts time-domain modulation signals through wavelet, converts incoherent signals into coherent signals, and then calculates each pixel The accumulated amount of position uses the correlation between signals of different spatial positions to generate super-resolution images; the autocorrelation algorithm of dSIM is not sensitive to the errors of reconstruction parameters.
  • dSIM bypasses the complex frequency domain operations in SIM image reconstruction, and also avoids The artifacts caused by the parameter error in each step of the frequency domain operation; at the same time, dSIM retains the advantages of SIM imaging, with simple sample preparation process, twice the spatial resolution, high temporal resolution, live cell imaging, and multi-color imaging Other advantages; the dSIM method is highly adaptable, and can use laboratory SIM, nonlinear SIM imaging systems or commercial systems (GE, Nikon, Zeiss) for experiments.
  • Figure 1 is an image of the evolution of the space and frequency domains obtained according to an embodiment of the direct structured light illumination super-resolution microscopy reconstruction method of the present invention, in which (a) ⁇ (c) are respectively in different modulation directions
  • the wide-field WF modulated image under structured light illumination, the original data is provided by the author of Reference [1]
  • (d) ⁇ (f) are the frequency domain images of (a) ⁇ (c)
  • (g) ⁇ (i) ) Are the unidirectional dSIMs of the first modulation frequency K1 in different modulation directions
  • (m) ⁇ (o) are the unidirectional dSIMs of the second modulation frequency K2 in different modulation directions
  • (p) ⁇ (q ) Are the frequency domain images of (m) ⁇ (o) respectively;
  • Fig. 2 is a dSIM result diagram obtained according to an embodiment of the direct structured light illumination super-resolution microscopy reconstruction method of the present invention, where (a) is a dSIM result diagram of all modulation directions of the first modulation frequency K1, (b) Is the dSIM result diagram of all modulation directions of the second modulation frequency K2, (c) the final dSIM result diagram, (d) is the RL SIM result diagram with the same original image, (e) ⁇ (h) are respectively (a) ⁇ (d) Fourier domain image;
  • Figure 3 is the direct structured light illumination super-resolution microscopy reconstruction method according to the present invention and the actin results obtained by the existing SIM, where (a) is the SIM result reconstructed by the traditional SIM, (b) is the dSIM result , (C) ⁇ (f) are the enlarged images marked in (a) and (b) respectively;
  • Fig. 4 is a flow chart of the direct structured light illumination super-resolution microscopy reconstruction method of the present invention.
  • the original image is a 3D original image, as shown in FIG. 4, including the following steps:
  • the structured light irradiates the sample with three modulation directions, namely the first modulation direction, the second modulation direction and the third modulation direction.
  • Each modulation direction has five phases, and each phase in each modulation direction is collected.
  • One original image, 15 original 3D images are obtained, and two modulation frequencies in space form an original image stack;
  • interpolation is performed through the spatial frequency domain FFT zero padding, so that the sampling frequency becomes larger and expanded to more than twice to obtain the first and second extracted images after interpolation Image stack
  • the preprocessed extracted image stack constitutes the preprocessed image stack
  • the wavelet family is Fejer-Korovkin, from the time sequence signal on each pixel of each image in the image stack after the first preprocessing, the first modulation frequency K1 is extracted, and Extract the second modulation frequency K2 from the time sequence signal on each pixel of each image in the image stack after the second preprocessing;
  • the autocorrelation accumulation is calculated at each pixel.
  • M phases of the first modulation frequency K1 Generate an auto-correlation image, get N auto-correlation images, after auto-correlation is a real number, the phase information is cancelled, and the second modulation frequency K2 produces an auto-correlation image, and N auto-correlation images are obtained, so that each 3D original
  • the image corresponds to six autocorrelation images, as shown in Figure 1 (a) ⁇ (f), where the first and second modulation frequencies are given in (g) ⁇ (l) and (m) ⁇ (r), respectively From the spatial and frequency domain images of the processing results of the three modulation directions of K1 and K2, it can be seen that the second modulation frequency K2 has a larger frequency domain range, but due to the limitation of the optical transfer function OTF, the second frequency is higher.
  • Each autocorrelation image has unidirectional modulation to provide a super-resolution image with unidirectional modulation.
  • the resolution will be isotropically
  • the direct summation in each direction may greatly reduce the resolution.
  • deconvolution processing is applied to the results of each direction.
  • For each autocorrelation image Perform RL deconvolution processing. RL deconvolution is used to enhance the higher frequency components within the cut-off frequency.
  • the square root of each pixel value in the deconvolution processed autocorrelation image is calculated to obtain the intermediate processing result of dSIM,
  • the square root calculation can improve the linearity of the result, but at the same time it will reduce the resolution of dSIM;
  • the original image is a 2D original image and includes the following steps:
  • the structured light irradiates the sample with three modulation directions, each modulation direction has three phases, and one original image is collected for each phase in each modulation direction, so that nine 2D original images are obtained to form an original image stack ;
  • interpolation is performed through spatial frequency domain FFT zero padding, so that the sampling frequency becomes larger and expanded to more than twice, to obtain the extracted image stack after interpolation;
  • the autocorrelation accumulation is calculated at each pixel, and one autocorrelation image is generated in each modulation direction and three phases to obtain three autocorrelation images, using different The correlation between spatial position signals produces super-resolution images;
  • the intermediate dSIM processing results on modulation frequencies in different modulation directions are summed to generate a final dSIM image.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Biochemistry (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Optics & Photonics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

一种直接结构光照明超分辨显微重建方法(dSIM),首先通过小波提取时间域调制信号,将非相干信号转化成相干信号,再计算每个像素位置上的积累量,利用不同空间位置信号之间的相关性产生超分辨图像。dSIM的自相关算法对重建参数的误差不敏感,绕过了SIM图像重建中复杂的频域操作,避免了每步频域操作中的参数误差带来的伪影。dSIM算法的适应性较强,能够利用实验室SIM、非线性SIM成像系统或商用系统进行实验。

Description

一种直接结构光照明超分辨显微重建方法 技术领域
本发明涉及荧光显微成像技术,具体涉及一种直接结构光照明超分辨显微重建方法。
背景技术
相比传统显微技术,超分辨显微技术的空间分辨率可以达到10~100nm,能够观测到分子位点和神经细胞的超微结构。目前主流的三种超分辨技术当中,受激发射损耗显微镜(STED)的消激发光强度过高且点扫描时间分辨率低,单分子定位显微镜(SMLM)需要采集上千张的原始图像,时间分辨率较低且照明光强高,因此这两种技术不适合对活细胞进行成像。其中结构光照明超分辨显微镜(SIM)的照明光强最低,成像速度快,有多色成像能力,适合进行活细胞快速成像。
传统的SIM方法中,通过空间频率分离,将高频信息搬移到频域的正确位置,再返回到空域产生超分辨图像。SIM重建效果严重依赖算法,实验中的条纹方向、相位、调至深度等参数对误差极其敏感,容易产生伪影,影响超分辨图像的质量。而实际情况下,整个视场当中的实际参数是非均一的,统一设置的重建参数与实际局部参数的差异,进而导致SIM重建图像中普遍出现伪影,并将妨碍超分辨图像的分析结果。
发明内容
针对以上现有技术中存在的问题,本发明提出了一种直接结构光照明超分辨显微重建算法,与传统SIM和图像扫描显微镜(ISM,AiryScan,SD-SIM)不同。
基于结构光照明显微平台,本发明提出了一种全新的重建方法,能够实现提高两倍的分辨率且无伪影的超分辨率显微方法,并将其命名为直接结构光照明超分辨显微(dSIM,Direct structured illumination microscopy)。
结构光是具有周期性条纹的照明光,不同的相位照射样品的过程就是对样品的调制过程,具有不同的调制方向,并且在每一个调制方向上具有不同的相位。结构光照明能够使得样本的高频信息产生频移,进入到可观测的光学传递函数OTF内,并通过方法使得高频信息搬移到在频域中的正确位置,从而扩大光学系统的OTF,并获得超分辨显微效果。结构光照明显微镜得到的原始图像为3D原始图像、2D原始图像或全内反射显微镜TIRF SIM 图像。
本发明的直接结构光照明超分辨显微重建方法,原始图像为3D原始图像,包括以下步骤:
1)得到原始图像栈:
结构光对样品的照射具有N个调制方向,每一个调制方向具有M个相位,N和M均为≥2的自然数,在每一个调制方向上的每一个相位均采集一张原始图像,从而得到N×M张3D原始图像,构成原始图像栈;
2)对原始图像栈进行预处理:
a)对原始图像栈中的每一张3D原始图像,采用小波包频率分离方法提取每个像素的第一调制频率K1、第二调制频率K2以及零频率K0;
b)对每一张3D原始图像,通过组合第一调制频率K1与零频率K0生成第一提取图象,得到第一提取图象栈,以及通过组合第二调制频率K2与零频率K0生成第二提取图象,得到第二提取图象栈,从而由一张3D原始图象生成两张提取图像,构成包括第一提取图象栈和第二提取图象的提取图像栈,提取图像栈中图像的数量倍增,即2N×M张;
c)对第一和第二提取图象栈中的每张图像,通过空间频域FFT零填充进行插值,使得采样频率变大,扩大至两倍以上,得到插值后的第一和第二提取图象栈;
d)对插值后的第一和第二提取图象栈中的提取图像进行去噪处理,得到去噪后的第一和第二提取图象栈;
e)对去噪后的第一和第二提取图象栈中的每一张去噪后的提取图像通过理查森-露西(RL,Richardson Lucy algorithm)算法进行反卷积,提高高频信号的相对强度,得到第一和第二预处理后的提取图象栈,构成预处理后的图象栈,共2N×M张;
3)对预处理后的图象栈分别提取第一调制频率K1和第二调制频率K2:
a)利用小波包滤波器,从第一预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出第一调制频率K1,以及从第二预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出第二调制频率K2;
b)利用低通滤波器从每个像素上提取出的第一调制频率K1和第二调制频率K2的强度的时间演化中分别提取复调制信号,从而将3D原始图像的非相干实数信号转化成相干复调制信号;
c)对提取的复调制信号进行FFT插值以增加采样频率,采样频率扩大至两倍以上,得到复调制图象栈;
4)各个像素上进行自相关运算:
对提取出的复调制图象栈的每个像素处,计算复调制信号的自相关积累量,在每一个调制方向,第一调制频率K1的M个相位产生一张自相关图像,得到N张自相关图像,以及第二调制频率K2产生一张自相关图像,得到N张自相关图像,从而利用不同空间位置信号的自相关积累量生成超分辨图像,每一组原始图像栈对应产生2N张图像;
5)自相关图像后处理:
首先,对每一张自相关图像进行RL反卷积处理;然后,对反卷积处理后的自相关图像中的每个像素值计算平方根,得到dSIM中间处理结果,通过平方根计算能够提高结果的线性度,但同时会降低dSIM的分辨率;
6)dSIM图像融合:
将不同调制方向上的第一和第二调制频率上的dSIM中间处理结果进行加和,从而生成一张最终的dSIM图像。
在步骤2)的d)中,去噪处理采用空域频域图像进行巴特沃斯Butterworth低通滤波。
在步骤3)的b)中,从第一调制频率K1和第二调制频率K2的强度的时间演化中提取复调制信号,包括以下步骤:首先利用小波包提取调制频率的实数信号,再利用滤波器提取调制信号中的复调制信号。
在步骤4)中,在每个像素处计算自相关积累量,第一和第二调制频率的M个相位产生一张自相关图像,包括以下步骤:计算同一个调制方向上的M个相位的复调制图像栈中,每个像素处的复调制信号的自相关积累量,生成一张结果图像得到自相关图像。
本发明的直接结构光照明超分辨显微重建算法,原始图像为2D原始图像或全内反射显微镜图像,包括以下步骤:
1)得到原始图像栈:
结构光对样品的照射具有N个调制方向,每一个调制方向具有M个相位,N和M均为≥2的自然数,在每一个调制方向上的每一个相位均采集一张原始图像,从而得到N×M张2D原始图像或全内反射显微镜图像,构成原始图像栈;
2)对原始图像栈进行预处理:
a)对原始图像栈中的每一张2D原始图像或全内反射显微镜图像,通过空间频域 FFT零填充进行插值,使得采样频率变大,扩大至两倍以上,得到插值后的提取图象栈;
b)对插值后的提取图象栈中的提取图像进行去噪处理,得到去噪后的提取图象栈;
c)对去噪后的提取图象栈中的每一张去噪后的提取图像通过理查森-露西RL算法进行反卷积,提高高频信号的相对强度,得到N×M张预处理后的图象,构成预处理后的图象栈;
3)对预处理后的图象栈提取调制频率K:
a)利用小波包滤波器,从预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出调制频率K;
b)利用低通滤波器从每个像素上提取出的调制频率K的强度的时间演化中提取复调制信号,将2D原始图像或全内反射显微镜图像的非相干实数信号转化成相干复调制信号;
c)对提取的复调制信号中的复振幅信号进行FFT插值以增加采样频率,采样频率扩大至两倍以上,得到复调制图象栈;
4)各个像素上进行自相关运算:
对复调制图象栈中提取出的复调制信号,在每个像素处计算自相关积累量,在每一个调制方向,M个相位产生一张自相关图像,得到N张自相关图像,自相关图像为超分辨图像,从而利用不同空间位置信号的自相关积累量生成超分辨图像;
5)自相关图像后处理:
对每一张自相关图像进行RL反卷积处理,然后,对反卷积处理后的自相关图像中的每个像素值计算平方根,得到dSIM中间处理结果,通过平方根计算能够提高结果的线性度,但同时会降低dSIM的分辨率;
6)dSIM图像融合:
将不同调制方向上的dSIM中间处理结果进行加和,从而生成一张最终的dSIM图像。
其中,在步骤2)的d)中,去噪处理采用空域频域图像进行巴特沃斯Butterworth低通滤波。
在步骤3)的b)中,从每个像素上提取出的调制频率K的强度的时间演化中提取复调制信号,包括以下步骤:首先利用小波包提取调制频率的实数信号,再利用滤波器提取调制信号中的复调制信号。
在步骤4)中,在每个像素处计算自相关积累量,在每一个调制方向上M个相位产生一张自相关图像,包括以下步骤:计算同一个调制方向上的M个相位的复调制图像栈中,每个像素处的复调制信号的自相关积累量,生成一张结果图像得到自相关图像。
本发明的优点:
与传统的SIM和ISM(AiryScan,SD-SIM)不同,本发明的dSIM的超分辨显微重建算法,首先通过小波提取时间域调制信号,将非相干信号转化成相干信号,再计算每个像素位置上的积累量,利用不同空间位置信号之间的相关性产生超分辨图像;dSIM的自相关算法对重建参数的误差不敏感,dSIM绕过了SIM图像重建中复杂的频域操作,也避免了每步频域操作中的参数误差带来的伪影;同时dSIM保留了SIM成像的优点,具有样本制作流程简单、空间分辨率提高两倍、时间分辨率高、活细胞成像、多色成像等优点;dSIM方法的适应性较强,能够利用实验室SIM、非线性SIM成像系统或商用系统(GE、Nikon、Zeiss)进行实验。
附图说明
图1为根据本发明的直接结构光照明超分辨显微重建方法的一个实施例得到的空域和频域的演变的图像,其中,(a)~(c)分别为在具有不同的调制方向的结构光照明下的宽场WF调制图像,原始数据由参考文献[1]的作者提供,(d)~(f)分别为(a)~(c)的频域图像,(g)~(i)分别为不同的调制方向上的第一调制频率K1的单向dSIM,(m)~(o)分别为不同的调制方向上的第二调制频率K2的单向dSIM,(p)~(q)分别为(m)~(o)的频域图像;
图2为根据本发明的直接结构光照明超分辨显微重建方法的一个实施例得到的dSIM结果图,其中,(a)为第一调制频率K1的所有调制方向的dSIM结果图,(b)为第二调制频率K2的所有调制方向的dSIM结果图,(c)最终的dSIM结果图,(d)为具有相同原始图像的RL SIM结果图,(e)~(h)分别为(a)~(d)的傅立叶域图像;
图3为根据本发明的直接结构光照明超分辨显微重建方法以及现有的SIM得到的肌动蛋白结果,其中,(a)为用传统的SIM重建的SIM结果,(b)为dSIM结果,(c)~(f)分别为(a)和(b)中标记的放大图;
图4为本发明的直接结构光照明超分辨显微重建方法的流程图。
具体实施方式
下面结合附图,通过具体实施例进一步阐述本发明。
实施例一
本实施例的本发明的直接结构光照明超分辨显微重建方法,原始图像为3D原始图像,如图4所示,包括以下步骤:
1)得到原始图像栈:
结构光对样品的照射具有三个调制方向,分别为第一调制方向、第二调制方向和第三调制方向,每一个调制方向具有五个相位,在每一个调制方向上的每一个相位均采集一张原始图像,从而得到15张3D原始图像,空间上两个调制频率,构成原始图像栈;
2)对原始图像栈进行预处理:
a)从3D原始图像中,在图1(a)~(f)中,观察到两个频率搬移即第一和第二调制频率K1和K2,这两个调制频率的频移与条纹调制方向相同,对原始图像栈中的每一张3D原始图像,采用小波包频率分离方法提取每个像素的第一调制频率K1、第二调制频率K2以及零频率K0;
b)对每一张3D原始图像,通过组合第一调制频率K1与零频率K0生成第一提取图象,得到第一提取图象栈,以及通过组合第二调制频率K2与零频率K0生成第二提取图象,得到第二提取图象栈,从而由一张3D原始图象生成两张提取图像,构成包括第一提取图象栈和第二提取图象的提取图像栈,提取图像栈中图像的数量倍增,共30张;
c)对第一和第二提取图象栈中的每张图像,通过空间频域FFT零填充进行插值,使得采样频率变大,扩大至两倍以上,得到插值后的第一和第二提取图象栈;
d)对插值后的第一和第二提取图象栈中的提取图像采用Butterworth低通滤波进行去噪处理,得到去噪后的第一和第二提取图象栈;
e)对去噪后的第一和第二提取图象栈中的每一张提取图像通过理查森-露西RL算法对图像进行反卷积,提高高频信号的相对强度,得到第一和第二预处理后的提取图象栈,构成预处理后的图象栈;
3)对预处理后的图象栈分别提取第一调制频率K1和第二调制频率K2:
a)利用小波包滤波器,小波族是Fejer-Korovkin,从第一预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出第一调制频率K1,以及从第 二预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出第二调制频率K2;
b)利用sigmoid低通滤波器从每个像素上提取出的第一调制频率K1和第二调制频率K2的强度的时间演化中分别提取复调制信号,将非相干信号转化成相干信号;
c)对提取的复调制信号中的复振幅信号进行FFT插值以增加采样频率,采样频率扩大至两倍以上,得到复调制图象栈;
4)各个像素上进行自相关运算:
这是dSIM的第一步核心步骤,对复调制图象栈中提取出的复调制信号,在每个像素处计算自相关积累量,在每一个调制方向,第一调制频率K1的M个相位产生一张自相关图像,得到N张自相关图像,自相关后是实数,相位信息抵消,以及第二调制频率K2产生一张自相关图像,得到N张自相关图像,从而每一张3D原始图像对应产生六张自相关图像,如图1(a)~(f)所示,其中(g)~(l)和(m)~(r)中分别给出了第一和第二调制频率K1和K2三个调制方向的处理结果的空间域和频率域图像,可以看出第二调制频率K2拥有更大的频域范围,但由于受到光学传递函数OTF的限制,频率更高的第二调制频率K2的调制信号强度远低于第一调制频率K1,导致第二调制频率K2的自相关图像强度低于第一调制频率K1的自相关图像;
5)自相关图像后处理:
这是dSIM的第二步核心步骤,每一张自相关图像具有单方向调制,从而提供具有单方向调制的超分辨率图像,当三个方向的结果相加时,分辨率将各向同性地提高,但是,在各个方向上直接求和可能会大大降低分辨率,为了在单个方向上保持较高的分辨率,对每个方向的结果都应用了去卷积处理对每一张自相关图像进行RL反卷积处理,RL反卷积用于增强截止频率内的较高频率分量,然后,对反卷积处理后的自相关图像中的每个像素值计算平方根,得到dSIM中间处理结果,通过平方根计算能够提高结果的线性度,但同时会降低dSIM的分辨率;
6)dSIM图像融合:
将不同调制方向上的第一调制频率上的dSIM中间处理结果加在一起,结果如图2(a)所示,以及将不同调制方向上的第二调制频率上的dSIM中间处理结果加在一起,结果如图2(b)所示,与图1中单向处理结果相比,在图2(e)和2(f) 中,第一和第二调制频率K1和K2的求和结果的截止频率几乎各向同性,最后一步,通过将第一和第二调制频率K1和K2的所有方向结果相加得出dSIM的最终结果,即图2(a)与图2(b)相加的结果,如图2(c)所示。
在图2(g)中,与传统的SIM频域不同,dSIM的频域更平滑的,在其中几乎看不到频移造成的尖峰,因此,dSIM几乎不会产生传统SIM中的蜂窝伪影。为了方便比较,通过对传统SIM的频域进行插值,使频域范围和像素大小与dSIM一致,图中的白色虚线框是SIM重建结果的频率范围。在图2(g)和2(h)中可以看出,dSIM的截止频率略高于SIM的频率边界。
在图3中,给出了使用SIM和dSIM对肌动蛋白实验成像的更多细节,并将结果在离焦背景消除和波纹伪影方面进行了比较。在图3(c)和(d)的对比中可以看出,传统结构光照明算法中如图(d)所示,出现了明显的离焦伪影,如箭头所示表现为强烈的三个方向上的条纹伪影。这是由于传统SIM算法无法消除离焦信号的影响,在频率搬移时也将离焦搬移到高频上,从而产生条纹状离焦伪影。而在直接结构光照明方法的处理结果中,基本没有出现离焦伪影,能够清楚的看到焦面上的微丝蛋白。从图3(f)中可以看出,能够看出由于局部调制深度等参数的变化,如箭头所示在正常的微丝蛋白周围出现了波纹伪影,这种伪影会影响对样本真实结构的判断。而在图3(e)中,没有出现明显的波纹伪影,这说明直接结构光照明方法对调制深度等参数的变化不敏感,不易产生传统算法由于参数估计误差产生的伪影。
实施例二
本实施例的直接结构光照明超分辨显微重建算法,原始图像为2D原始图像,包括以下步骤:
1)得到原始图像栈:
结构光对样品的照射具有三个调制方向,每一个调制方向具三个相位,在每一个调制方向上的每一个相位均采集一张原始图像,从而得到九张2D原始图像,构成原始图像栈;
2)对原始图像栈进行预处理:
a)对原始图像栈中的每一张2D原始图像,通过空间频域FFT零填充进行插值,使得采样频率变大,扩大至两倍以上,得到插值后的提取图象栈;
b)对插值后的提取图象栈中的提取图像进行去噪处理,得到去噪后的提取图象栈;
c)对去噪后的提取图象栈中的每一张提取图像通过RL算法对图像进行反卷积, 提高高频信号的相对强度,得到预处理后的图象栈;
3)对预处理后的图象栈提取调制频率K:
a)利用小波包滤波器,从预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出调制频率K;
b)利用Sigmoid低通滤波器从每个像素上提取出的调制频率K强度的时间演化中分别提取复调制信号,将非相干信号转化成相干信号;
c)对提取的复调制信号中的复振幅信号进行FFT插值以增加采样频率,采样频率扩大至两倍以上,得到复调制图象栈;
4)各个像素上进行自相关运算:
对复调制图象栈中提取出的复调制信号,在每个像素处计算自相关积累量,在每一个调制方向,三个相位产生一张自相关图像,得到三张自相关图像,利用不同空间位置信号之间的相关性产生超分辨图像;
5)自相关图像后处理:
对每一张自相关图像进行RL反卷积处理,然后,对反卷积处理后的自相关图像中的每个像素值计算平方根,得到dSIM中间处理结果,通过平方根计算能够提高结果的线性度,但同时会降低dSIM的分辨率;
6)dSIM图像融合:
将不同调制方向上的调制频率上的dSIM中间处理结果进行加和,从而生成一张最终的dSIM图像。
最后需要注意的是,公布实施例的目的在于帮助进一步理解本发明,但是本领域的技术人员可以理解:在不脱离本发明及所附的权利要求的精神和范围内,各种替换和修改都是可能的。因此,本发明不应局限于实施例所公开的内容,本发明要求保护的范围以权利要求书界定的范围为准。
参考文献:
[1]Muller,M.,et al.,Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ.Nature Communications,2016.7:p.10980.

Claims (8)

  1. 一种直接结构光照明超分辨显微重建方法,原始图像为3D原始图像,其特征在于,所述直接结构光照明超分辨显微重建方法包括以下步骤:
    1)得到原始图像栈:
    结构光对样品的照射具有N个调制方向,每一个调制方向具有M个相位,N和M均为≥2的自然数,在每一个调制方向上的每一个相位均采集一张原始图像,从而得到N×M张3D原始图像,构成原始图像栈;
    2)对原始图像栈进行预处理:
    a)对原始图像栈中的每一张3D原始图像,采用小波包频率分离方法提取每个像素的第一调制频率K1、第二调制频率K2以及零频率K0;
    b)对每一张3D原始图像,通过组合第一调制频率K1与零频率K0生成第一提取图象,得到第一提取图象栈,以及通过组合第二调制频率K2与零频率K0生成第二提取图象,得到第二提取图象栈,从而由一张3D原始图象生成两张提取图像,构成包括第一提取图象栈和第二提取图象的提取图像栈,提取图像栈中图像的数量倍增,即2N×M张;
    c)对第一和第二提取图象栈中的每张图像,通过空间频域FFT零填充进行插值,使得采样频率变大,扩大至两倍以上,得到插值后的第一和第二提取图象栈;
    d)对插值后的第一和第二提取图象栈中的提取图像进行去噪处理,得到去噪后的第一和第二提取图象栈;
    e)对去噪后的第一和第二提取图象栈中的每一张去噪后的提取图像通过理查森-露西RL算法进行反卷积,提高高频信号的相对强度,得到第一和第二预处理后的提取图象栈,构成预处理后的图象栈,共2N×M张;
    3)对预处理后的图象栈分别提取第一调制频率K1和第二调制频率K2:
    a)利用小波包滤波器,从第一预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出第一调制频率K1,以及从第二预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出第二调制频率K2;
    b)利用低通滤波器从每个像素上提取出的第一调制频率K1和第二调制频率K2的强度的时间演化中分别提取复调制信号,从而将3D原始图像的非相干实数信号转化成相干复调制信号;
    c)对提取的复调制信号进行FFT插值以增加采样频率,采样频率扩大至两倍以上, 得到复调制图象栈;
    4)各个像素上进行自相关运算:
    对提取出的复调制图象栈的每个像素处,计算复调制信号的自相关积累量,在每一个调制方向,第一调制频率K1的M个相位产生一张自相关图像,得到N张自相关图像,以及第二调制频率K2产生一张自相关图像,得到N张自相关图像,从而利用不同空间位置信号的自相关积累量生成超分辨图像,每一组原始图像栈对应产生2N张图像;
    5)自相关图像后处理:
    首先,对每一张自相关图像进行RL反卷积处理;然后,对反卷积处理后的自相关图像中的每个像素值计算平方根,得到直接结构光照明超分辨显微dSIM中间处理结果;
    6)dSIM图像融合:
    将不同调制方向上的第一和第二调制频率上的dSIM中间处理结果进行加和,从而生成一张最终的dSIM图像。
  2. 如权利要求1所述的直接结构光照明超分辨显微重建方法,其特征在于,在步骤2)的d)中,去噪处理采用空域频域图像进行巴特沃斯低通滤波。
  3. 如权利要求1所述的直接结构光照明超分辨显微重建方法,其特征在于,在步骤3)的b)中,从第一调制频率K1和第二调制频率K2的强度的时间演化中提取复调制信号,包括以下步骤:首先利用小波包提取调制频率的实数信号,再利用滤波器提取调制信号中的复调制信号。
  4. 如权利要求1所述的直接结构光照明超分辨显微重建方法,其特征在于,在步骤4)中,在每个像素处计算自相关积累量,每个调制频率的M个相位产生一张自相关图像,包括以下步骤:计算同一个调制方向上的M个相位的复调制图像栈中,每个像素处的复调制信号的自相关积累量,生成一张结果图像得到自相关图像。
  5. 一种直接结构光照明超分辨显微重建方法,原始图像为2D原始图像或全内反射显微镜图像,其特征在于,所述直接结构光照明超分辨显微重建方法包括以下步骤:
    1)得到原始图像栈:
    结构光对样品的照射具有N个调制方向,每一个调制方向具有M个相位,N和M均为≥2的自然数,在每一个调制方向上的每一个相位均采集一张原始图像,从而得到N×M张2D原始图像或全内反射显微镜图像,构成原始图像栈;
    2)对原始图像栈进行预处理:
    a)对原始图像栈中的每一张2D原始图像或全内反射显微镜图像,通过空间频域FFT零填充进行插值,使得采样频率变大,扩大至两倍以上,得到插值后的提取图象栈;
    b)对插值后的提取图象栈中的提取图像进行去噪处理,得到去噪后的提取图象栈;
    c)对去噪后的提取图象栈中的每一张去噪后的提取图像通过理查森-露西RL算法进行反卷积,提高高频信号的相对强度,得到N×M张预处理后的图象,构成预处理后的图象栈;
    3)对预处理后的图象栈提取调制频率K:
    a)利用小波包滤波器,从预处理后的图象栈中的每一张图像的每一个像素上的时序信号中,提取出调制频率K;
    b)利用低通滤波器从每个像素上提取出的调制频率K的强度的时间演化中提取复调制信号,将2D原始图像或全内反射显微镜图像的非相干实数信号转化成相干复调制信号;
    c)对提取的复调制信号中的复振幅信号进行FFT插值以增加采样频率,采样频率扩大至两倍以上,得到复调制图象栈;
    4)各个像素上进行自相关运算:
    对复调制图象栈中提取出的复调制信号,在每个像素处计算自相关积累量,在每一个调制方向,M个相位产生一张自相关图像,得到N张自相关图像,自相关图像为超分辨图像,从而利用不同空间位置信号的自相关积累量生成超分辨图像;
    5)自相关图像后处理:
    对每一张自相关图像进行RL反卷积处理,然后,对反卷积处理后的自相关图像中的每个像素值计算平方根,得到dSIM中间处理结果;
    6)dSIM图像融合:
    将不同调制方向上的dSIM中间处理结果进行加和,从而生成一张最终的dSIM图像。
  6. 如权利要求5所述的直接结构光照明超分辨显微重建方法,其特征在于,在步骤2)的d)中,去噪处理采用空域频域图像进行巴特沃斯低通滤波。
  7. 如权利要求5所述的直接结构光照明超分辨显微重建方法,其特征在于,从每个像素上提取出的调制频率K的强度的时间演化中提取复调制信号,包括以下步骤:首先利 用小波包提取调制频率的实数信号,再利用滤波器提取调制信号中的复调制信号。
  8. 如权利要求5所述的直接结构光照明超分辨显微重建方法,其特征在于,在步骤4)中,在每个像素处计算自相关积累量,在每一个调制方向上M个相位产生一张自相关图像,包括以下步骤:计算同一个调制方向上的M个相位的复调制图像栈中,每个像素处的复调制信号的自相关积累量,生成一张结果图像得到自相关图像。
PCT/CN2020/110154 2020-05-12 2020-08-20 一种直接结构光照明超分辨显微重建方法 WO2021227292A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/998,484 US20230214961A1 (en) 2020-05-12 2020-08-20 Direct structured illumination microscopy reconstruction method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010396335.5 2020-05-12
CN202010396335.5A CN111458317B (zh) 2020-05-12 2020-05-12 一种直接结构光照明超分辨显微重建方法

Publications (1)

Publication Number Publication Date
WO2021227292A1 true WO2021227292A1 (zh) 2021-11-18

Family

ID=71682979

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/110154 WO2021227292A1 (zh) 2020-05-12 2020-08-20 一种直接结构光照明超分辨显微重建方法

Country Status (3)

Country Link
US (1) US20230214961A1 (zh)
CN (1) CN111458317B (zh)
WO (1) WO2021227292A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092331A (zh) * 2021-11-24 2022-02-25 北京大学 一种结合锁相放大预处理的结构光超分辨重建方法
WO2024051079A1 (zh) * 2022-09-05 2024-03-14 中国科学院苏州生物医学工程技术研究所 一种主动结构光照明的超分辨显微成像方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111458317B (zh) * 2020-05-12 2021-04-30 北京大学 一种直接结构光照明超分辨显微重建方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140226165A1 (en) * 2009-12-14 2014-08-14 Academia Sinica Height measurement by correlating intensity with position of scanning object along optical axis of a structured illumination microscope
CN106770147A (zh) * 2017-03-15 2017-05-31 北京大学 一种结构光照明超分辨显微成像系统及其成像方法
CN108319009A (zh) * 2018-04-11 2018-07-24 中国科学院光电技术研究所 基于结构光调制的快速超分辨成像方法
CN109712072A (zh) * 2018-12-15 2019-05-03 浙江大学 基于全内反射的条纹照明傅里叶域迭代更新超分辨显微成像方法
CN111077121A (zh) * 2019-12-06 2020-04-28 中国科学院西安光学精密机械研究所 空域中直接重构结构光照明超分辨图像的快速方法及系统
CN111458317A (zh) * 2020-05-12 2020-07-28 北京大学 一种直接结构光照明超分辨显微重建方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101236646B (zh) * 2007-01-30 2011-09-14 宝利微系统控股公司 在频率域检测与估计图像显著的强相关方向的方法和系统
CN108007386B (zh) * 2016-11-02 2021-04-20 光宝电子(广州)有限公司 基于结构光的三维扫描方法及其装置与系统
KR102013647B1 (ko) * 2017-11-24 2019-10-21 한국항공우주연구원 위성 영상처리방법 및 기록매체
CN109068024B (zh) * 2018-06-29 2020-07-21 北京大学 一种对时空信号进行滤波的方法
CN109191509A (zh) * 2018-07-25 2019-01-11 广东工业大学 一种基于结构光的虚拟双目三维重建方法
CN110595387B (zh) * 2019-08-01 2022-05-13 佛山市南海区广工大数控装备协同创新研究院 一种基于多频率结构光的三维重建系统标定方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140226165A1 (en) * 2009-12-14 2014-08-14 Academia Sinica Height measurement by correlating intensity with position of scanning object along optical axis of a structured illumination microscope
CN106770147A (zh) * 2017-03-15 2017-05-31 北京大学 一种结构光照明超分辨显微成像系统及其成像方法
CN108319009A (zh) * 2018-04-11 2018-07-24 中国科学院光电技术研究所 基于结构光调制的快速超分辨成像方法
CN109712072A (zh) * 2018-12-15 2019-05-03 浙江大学 基于全内反射的条纹照明傅里叶域迭代更新超分辨显微成像方法
CN111077121A (zh) * 2019-12-06 2020-04-28 中国科学院西安光学精密机械研究所 空域中直接重构结构光照明超分辨图像的快速方法及系统
CN111458317A (zh) * 2020-05-12 2020-07-28 北京大学 一种直接结构光照明超分辨显微重建方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
XIE ZHONGYE; HU SONG; TANG YAN; LIU XI; LIU JUNBO; HE YU: "3D Super-resolution Reconstruction Using Microsphere-assisted Structured Illumination Microscopy", IEEE PHOTONICS TECHNOLOGY LETTERS, vol. 31, no. 22, 15 November 2019 (2019-11-15), pages 1783 - 1786, XP011757519, ISSN: 1041-1135, DOI: 10.1109/LPT.2019.2946793 *
ZHOU XING;DAN DAN;QIAN JIA;YAO BAOLI;LEI MING: "Super-Resolution Reconstruction Theory in Structured Illumination Microscopy", ACTA OPTICA SINICA, vol. 37, no. 3, 10 March 2017 (2017-03-10), pages 10 - 21, XP055866980 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092331A (zh) * 2021-11-24 2022-02-25 北京大学 一种结合锁相放大预处理的结构光超分辨重建方法
CN114092331B (zh) * 2021-11-24 2024-05-24 北京大学 一种结合锁相放大预处理的结构光超分辨重建方法
WO2024051079A1 (zh) * 2022-09-05 2024-03-14 中国科学院苏州生物医学工程技术研究所 一种主动结构光照明的超分辨显微成像方法及系统

Also Published As

Publication number Publication date
CN111458317B (zh) 2021-04-30
US20230214961A1 (en) 2023-07-06
CN111458317A (zh) 2020-07-28

Similar Documents

Publication Publication Date Title
WO2021227292A1 (zh) 一种直接结构光照明超分辨显微重建方法
CN111145089B (zh) 高保真图像重构方法、系统、计算机设备和存储介质
JP4865930B2 (ja) 構造化された照射および均一な照射の両方を用いて光学的に切片化された画像を生成するためのシステムおよび方法
CN108319009B (zh) 基于结构光调制的快速超分辨成像方法
CN111308682B (zh) 基于结构光照明的超分辨重构方法
CN105204311B (zh) 一种基于高斯切趾的光学扫描全息边缘检测方法
US8041142B2 (en) Method and device for reconstructing images
CN106991647B (zh) 一种低信噪比图像重构方法和系统
CN109949329B (zh) 基于相位一致的相位拉伸变换pst图像边缘检测方法
Forster et al. Extended depth-of-focus for multi-channel microscopy images: a complex wavelet approach
Ponti et al. Image restoration using gradient iteration and constraints for band extrapolation
CN115839936B (zh) 一种基于锁相探测的结构光照明超分辨显微成像重构方法
JP6040469B2 (ja) デジタルホログラフィ装置
CN114092331B (zh) 一种结合锁相放大预处理的结构光超分辨重建方法
CN118762125A (zh) 一种高速光层切三维结构光照明显微成像方法
Lin et al. SEM resolution improvement using semi-blind restoration with hybrid L1-L2 regularization
Lagunas et al. Human eye visual hyperacuity: Controlled diffraction for image resolution improvement
Xu et al. Interferogram blind denoising using deep residual learning for phase-shifting interferometry
CN116228547B (zh) 一种合成孔径成像过程的仿真方法
Homem et al. Poisson noise reduction in deconvolution microscopy
Hu et al. Hybrid method for accurate phase retrieval based on higher order transport of intensity equation and multiplane iteration
CN110400257B (zh) 一种基于比值改正项的多次曝光图像反混叠方法
Cheng et al. An improved multi-focus image fusion method based on Wavelet transform
Zheng et al. High-speed wide-field optical-sectioning fluorescence microscopy based on one-shot structured illumination
Jignasha et al. Learning based single frame super-resolution using lorentzian error norm & gabor filter

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20935009

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20935009

Country of ref document: EP

Kind code of ref document: A1