WO2021143707A1 - Système et procédé d'imagerie microscopique à double modalité - Google Patents

Système et procédé d'imagerie microscopique à double modalité Download PDF

Info

Publication number
WO2021143707A1
WO2021143707A1 PCT/CN2021/071393 CN2021071393W WO2021143707A1 WO 2021143707 A1 WO2021143707 A1 WO 2021143707A1 CN 2021071393 W CN2021071393 W CN 2021071393W WO 2021143707 A1 WO2021143707 A1 WO 2021143707A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
optical diffraction
light
sample
diffraction tomography
Prior art date
Application number
PCT/CN2021/071393
Other languages
English (en)
Chinese (zh)
Inventor
施可彬
陈良怡
董大山
黄小帅
李柳菊
毛珩
王爱民
Original Assignee
北京大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京大学 filed Critical 北京大学
Publication of WO2021143707A1 publication Critical patent/WO2021143707A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0036—Scanning details, e.g. scanning stages
    • G02B21/0048—Scanning details, e.g. scanning stages scanning mirrors, e.g. rotating or galvanomirrors, MEMS mirrors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458—Fluorescence microscopy
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052—Optical details of the image generation
    • G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/06—Means for illuminating specimens
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/361—Optical details, e.g. image relay to the camera or image sensor

Definitions

  • This application relates to the field of microscopic imaging technology, and in particular to a dual-modality microscopic imaging system and method.
  • optical microscopy is a powerful tool for modern molecular biology research. Its development promotes the continuous progress of human observation and understanding of life phenomena.
  • An optical microscope is a device that uses the interaction of light and matter to present the microstructure of an object. Since the invention of the optical microscope, it has been the most commonly used tool in the biomedical field.
  • Traditional lenses and visible light illumination support about 80% of microscopic research. Due to the optical transparency of cells, only optical microscopy can achieve non-invasive, label-free imaging of living cells.
  • Fluorescence microscopy imaging is the main method of molecular biology research.
  • the total number of imaging is limited, so the interaction and dynamic processes of the internal organelles in the cell have not been fully exposed.
  • High-resolution long-term imaging of living cells is still a huge challenge in biological research.
  • Due to the limitation of axial scanning speed, three-dimensional fluorescence imaging requires a larger excitation photon flux, and the photobleaching effect greatly limits the three-dimensional The total duration of imaging.
  • fluorescence imaging generally can only label a limited number of molecules at the same time.
  • auxiliary imaging methods such as electron microscopy can observe a variety of organelles, they can only provide static snapshots as an aid.
  • Optical diffraction tomography microscopy is a technique for non-invasive, label-free three-dimensional imaging of cells and tissues. Because of its combination of quantitative phase imaging technology and scattering theory, optical diffraction tomography microscopic imaging technology can sense nano-scale morphological changes, and can perform long-term high-resolution non-damage imaging of living cells. As a powerful tool for observing the dynamic changes of cells, it has great application prospects in cell metabolism, pathology and tumor diagnosis. However, due to the relatively complex optical architecture of the diffraction tomography technology and the immature algorithm, it has not been applied to biomedical research on a large scale. Diffraction tomography microscopy has two problems that need to be solved urgently.
  • the amount of diffraction tomography data is large and the calculation process is complicated, which makes it difficult to use for continuous observation of life phenomena.
  • the chemical selective imaging capabilities of optical diffraction tomography are limited, morphological characterization lacks chemical specificity, and convincing power is limited.
  • Optical diffraction tomography microscopic imaging has the characteristics of low luminous flux and low phototoxicity, which can effectively solve the problems encountered in fluorescence imaging.
  • the previous work lacks fluorescence imaging as an aid.
  • Most of the structures in the diffraction tomography image lack calibration and can only be analyzed by morphology.
  • traditional optical diffraction tomography only lipid droplets, chromosomes and mitochondria were identified and calibrated combined with wide-field fluorescence imaging.
  • One of the embodiments of the present application provides a dual-modality microscopic imaging system, including an optical diffraction tomography imaging subsystem and a structured light illumination fluorescence imaging subsystem; the optical diffraction tomography imaging subsystem is used for performing processing based on a first laser Label-free optical diffraction tomography to obtain an optical diffraction tomographic image of the sample; the structured light illumination fluorescence imaging subsystem is used to perform fluorescence imaging based on the second laser to obtain a structured light illumination fluorescence image of the sample; wherein The dual-mode microscopic imaging system includes a first light source and a second light source that are independent of each other, the first light source is used to emit the first laser light, and the second light source is used to emit the second laser light.
  • One of the embodiments of the present application provides a dual-modality microscopic imaging method, including: using mutually independent light sources to generate a first laser and a second laser respectively; using an optical diffraction tomography subsystem to acquire a sample based on the first laser The optical diffraction tomography image; the structured light illumination fluorescence imaging subsystem is used to obtain the structured light illumination fluorescence image of the sample based on the second laser; the structured light illumination fluorescence image is generated based on the optical diffraction tomography image and the structured light illumination fluorescence image The bimodal fusion image of the sample.
  • One of the embodiments of the present application provides a dual-modality microscopic imaging device, including a processor, and the processor is configured to execute the dual-modality microscopic imaging method described in any one of the embodiments of the present application.
  • One of the embodiments of the present application provides a computer-readable storage medium that stores computer instructions. After the computer reads the computer instructions in the storage medium, the computer executes the dual-mode storage medium described in any of the embodiments of the present application. State microscopy imaging method.
  • Fig. 1 is a schematic structural diagram of a dual-mode microscopic imaging system according to some embodiments of the present application
  • Fig. 2 is a block diagram of a dual-mode microscopic imaging system according to some embodiments of the present application
  • Fig. 3 is a schematic diagram of a control subsystem of a dual-mode microscopic imaging system according to some embodiments of the present application;
  • Fig. 4 is a control sequence diagram of a dual-mode microscopic imaging system according to some embodiments of the present application.
  • Fig. 5 is an exemplary flowchart of a dual-modality microscopic imaging method according to some embodiments of the present application.
  • Fig. 6 is a flow chart of a diffraction tomography reconstruction algorithm according to some embodiments of the present application.
  • Fig. 7a is an exemplary flowchart of determining a target scanning wave vector according to some embodiments of the present application.
  • FIG. 7b is a schematic diagram of using the VISA method to determine the target scanning wave vector according to some embodiments of the present application.
  • FIG. 8 is a schematic diagram of the deviation of the scanning wave vector according to some embodiments of the present application.
  • Fig. 9 is a reconstructed image without scanning wave vector iteration according to some embodiments of the present application.
  • FIG. 10 is a reconstructed image in the case of performing scanning wave vector iteration according to some embodiments of the present application.
  • Fig. 11a is a schematic diagram of an optical diffraction tomographic image reconstruction algorithm according to some embodiments of the present application.
  • Fig. 11b is a schematic diagram of a structured light illumination fluorescence image reconstruction algorithm according to some embodiments of the present application.
  • FIG. 12 is an optical diffraction tomography image of a Pelizaeus-Merzbacher Disease patient's fibroblasts obtained by using an optical diffraction tomography imaging subsystem 210 according to some embodiments of the present application;
  • Fig. 13 is an optical diffraction tomography result of fixed INS-1 cells according to some embodiments of the present application.
  • 14a and 14b are comparisons of the results of a wide-field fluorescence layer scan image and an optical diffraction tomography image of fixed primary hepatocytes according to some embodiments of the present application;
  • FIG. 15 is a continuous imaging result of COS-7 cells using the optical diffraction tomography imaging subsystem 210 according to some embodiments of the present application;
  • Fig. 16 is an optical diffraction tomography image of the pollen tube growth process according to some embodiments of the present application.
  • FIG. 17 is an optical diffraction tomography image of a nematode embryonic development process according to some embodiments of the present application.
  • Fig. 18 is an optical diffraction tomography-fluorescence co-localization image collected by using a dual-modality microscopic imaging system for six main organelles in COS-7 living cells according to some embodiments of the present application;
  • Fig. 19 is a lateral resolution characterization diagram of an optical diffraction tomography-structured light illumination fluorescence dual-mode microscopic imaging system according to some embodiments of the present application;
  • Figure 20 is a mitotic bimodal fluorescence colocalization image of COS-7 cells collected by the bimodal microscopic imaging system according to some embodiments of the present application;
  • FIG. 21 is an optical diffraction tomography-fluorescence co-localization image of organelles that cannot be resolved by optical diffraction tomography in COS-7 living cells by the dual-modality microscopic imaging system according to some embodiments of the present application;
  • Fig. 22 shows low refractive index vesicles appearing in optical diffraction tomography images of living COS-7 cells according to some embodiments of the present application
  • FIG. 23 shows low refractive index vesicles that appear in optical diffraction tomography images when the dual-modality microscopic imaging system according to some embodiments of the present application images different types of cells;
  • Fig. 24 is an optical diffraction tomography image of human bone marrow mesenchymal stem cells and the correlation between the number of low-refractive index vesicles and their cell senescence phenotypes according to some embodiments of the present application;
  • Figure 25 shows the interaction between mitochondria and other organelles in COS-7 cells according to some embodiments of the present application.
  • Fig. 26 shows the interaction between the organelles of the low refractive index vesicles in COS-7 cells according to some embodiments of the present application
  • Figure 27 is a visualization of the DBs transport pathway and its role in the interaction of tissue organelles according to some embodiments of the present application.
  • FIG. 28 is an analysis of motion artifacts that may be generated by different microscopes in the optical diffraction tomography of fast-moving lysosomes according to some embodiments of the present application;
  • Fig. 29 is a two-dimensional coherence function (CTF) measurement of a microscope according to some embodiments of the present application.
  • CTF coherence function
  • Figure 30 is an observation image of vacuoles of budding yeast by the ODT subsystem according to some embodiments of the present application.
  • Figure 31 shows the correlation between the LC3-EGFP labeling structure and DBs in COS-7 cells according to some embodiments of the present application
  • Figure 32 is a histogram of the LE/LY structure size observed in COS-7 cells overexpressing different protein markers according to the optical diffraction tomography subsystem shown in some embodiments of the present application;
  • FIG. 33 is a spatial frequency domain based on optical diffraction tomography reconstruction and restoration according to some embodiments of the present application.
  • FIG. 34 is a schematic diagram of using an optical diffraction tomography imaging subsystem to perform a three-dimensional overall observation of organelles according to some embodiments of the present application;
  • FIG. 35 is a schematic diagram of a super-resolution fluorescence assisted diffraction tomography (SR-FACT) system according to some embodiments of the present application.
  • SR-FACT super-resolution fluorescence assisted diffraction tomography
  • 100 is a dual-mode microscopy imaging system
  • 101 is the first light source
  • 102 is the first acousto-optic modulator (AOM)
  • 103 is the first half-wave plate (HWP)
  • 104 is the first polarization beam splitter (PBS)
  • 105 is a single-mode fiber (SMF)
  • 106 is a lens
  • 107 is a galvanometer (GM)
  • 108 is a tube lens
  • 109 is a microscope objective lens (OBJ)
  • 110 is a sample
  • 111 is a microscope objective
  • 112 is
  • 113 is a lens
  • 114 is a second camera
  • 115 is a single-mode fiber
  • 116 is a lens
  • 117 is a first camera
  • 118 is a polarization-independent beam splitter (BS)
  • 119 is a lens
  • 120 is a lens
  • 121 is a second dichroic
  • system is a method for distinguishing different components, elements, parts, parts, or assemblies of different levels.
  • the words can be replaced by other expressions.
  • the dual-modality microscopic imaging system may be a super-resolution fluorescence-assisted diffraction tomography (SR-FACT) system (FIG. 35).
  • the dual-modality microscopic imaging system includes an optical diffraction tomography (ODT) subsystem and a structured light illumination fluorescence imaging (SIM) subsystem.
  • ODT optical diffraction tomography
  • SIM structured light illumination fluorescence imaging
  • the optical diffraction tomography subsystem can be used to obtain the optical diffraction tomographic image of the sample
  • the structured light illumination fluorescence imaging subsystem can be used to obtain the structured light illumination fluorescence image of the sample.
  • the sample 110 may include, but is not limited to, biological tissues, biological macromolecules, proteins, cells, microorganisms, or other substances.
  • Fig. 1 is a schematic structural diagram of a dual-modality microscopic imaging system (such as a super-resolution fluorescence assisted diffraction tomography (SR-FACT) system) according to some embodiments of the present application.
  • Fig. 2 is a block diagram of a dual-mode microscopic imaging system according to some embodiments of the present application.
  • Figure 35 is a simple display of the hardware structure of the SR-FACT system.
  • the dual-modality microscopic imaging system 100 may include an optical diffraction tomography imaging subsystem 210, a structured light illumination fluorescence imaging subsystem 220, a control subsystem 230, and a processor 240.
  • the optical diffraction tomography imaging subsystem 210 may be a system that applies optical diffraction tomography (Optical Diffraction Tomography, ODT) microscopy technology for imaging.
  • ODT optical diffraction Tomography
  • Optical Diffraction Tomography microscopy is an imaging technology that reconstructs the three-dimensional distribution image of the refractive index of the sample through the inverse scattering (or diffraction) process.
  • the optical diffraction tomography subsystem may perform mark-free optical diffraction tomography based on the first laser to obtain the optical diffraction tomography image of the sample.
  • the optical diffraction tomography subsystem 210 may include a first light source 101, a first acousto-optic modulator 102, a first half-wave plate 103, a first polarization beam splitter 104, a galvanometer 107, and a polarization-independent beam splitter.
  • the prism 127 also called a beam splitter
  • the first light source 101 may be used to emit a first laser.
  • the optical diffraction tomography imaging subsystem 210 may further include one or more lenses, single-mode optical fibers, and/or couplers, and the like. In the embodiment of the present application, as shown in FIG.
  • the optical diffraction tomography subsystem 210 may include a first light source 101, a first acousto-optic modulator (AOM) 102, a first half-wave plate (HWP) 103, First polarization beam splitter (PBS) 104, single-mode fiber (SMF) 105, lens 106, galvanometer (GM) 107, tube lens 108, microscope objective lens (OBJ) 109, microscope objective lens 111, first two-way Color mirror (DM) 112, single-mode fiber 115, lens 116, first camera 117, polarization-independent beam splitter (BS) 118, lens 119, lens 120, second dichroic mirror 121, lens 122, etc.
  • AOM acousto-optic modulator
  • HWP first half-wave plate
  • PBS First polarization beam splitter
  • SMF single-mode fiber
  • GM galvanometer
  • OBJ microscope objective lens
  • DM microscope objective lens
  • DM two-way Color mirror
  • the first laser light emitted by the first light source 101 is modulated by the first acousto-optic modulator 102
  • the first laser light becomes +1-order diffracted light
  • the +1-order diffracted light passes through the first half wave
  • the sheet 103 is divided into a first light splitting and a second light splitting by the first polarization beam splitting prism 104.
  • rotating the first half-wave plate 103 can adjust the light splitting ratio of the first light splitting and the second light splitting.
  • the first light beam can be coupled into the single-mode fiber 105 through the coupler 134, and then the first light beam is output from the single-mode fiber 105 and collimated by the lens 106, and the first light beam after collimation (the first light beam is collimated)
  • the sample 110 can be illuminated from multiple angles after the galvanometer 107 is used to obtain sample light with sample information. Specifically, the first split collimated beam passes through the galvanometer 107 to obtain a two-dimensional scanning beam in the deflection direction of the collimated beam. After that, the scanning beam is focused on the back focal plane of the microscope objective lens 109 by the tube lens 108 to achieve Illumination of collimated beams on the sample 110 in different directions.
  • the microscopic objective 109 may be an illumination objective.
  • the sample light may be light that is irradiated on the sample 110 by collimated beams of different directions and transmitted (such as scattered or diffracted) through the sample 110. After the sample light is collected by the microscope objective lens 111, it is reflected by the first dichroic mirror 112, and after passing through the lens 122 and the second dichroic mirror 121, it is collimated again by the lens 120 and the lens 119 to obtain signal light. After the sample light passes through the second dichroic mirror 121, the beam of the sample light will be expanded by the pair of lenses 120 and 119.
  • the second split light is coupled into the single-mode fiber 115 through the coupler 135.
  • a delay light path can be passed through the second light splitting before coupling.
  • the second beam is the reference beam. After the second beam is output from the single-mode fiber, it is first collimated by the lens 116, and finally the collimated signal light is combined by the polarization-independent beam splitting prism 118, and forms off-axis holographic fringes at a certain off-axis angle.
  • the camera 117 receives.
  • the optical diffraction tomography subsystem may be an off-axis holographic microscope based on an MZ interferometer modified on a commercial microscope (OLYMPUS, IX73).
  • the first light source 101 can be a single longitudinal mode laser with a model of MSL-FN-561-50mW and a wavelength of 561nm manufactured by Changchun New Industry Optoelectronics Technology Co., Ltd.; Acousto-optic modulator manufactured by Optoelectronics Co., Ltd.; the model of the first half-wave plate can be Thorlabs, AHWP10M-600; the model of the first polarization beam splitter can be Thorlabs, CCM1-PBS251; the model of couplers 134 and 135 can be Thorlabs , PAF2-7A; single-mode fibers 105 and 115 can be made by Shanghai HannStar, the model can be PM460-HP HA, FC/APC polarization-maintaining
  • the optical diffraction tomography subsystem 210 may be used to obtain an optical diffraction tomography image.
  • the following description will be combined with specific experimental results.
  • FIG. 12 shows an optical diffraction tomography image of a patient's fibroblast from Pelime (Pelizaeus-Merzbacher Disease) obtained by using the optical diffraction tomography subsystem 210.
  • the left picture in Fig. 12 is the image of cells in the control group; the middle picture is the cell image of the first patient; the right picture is the cell image of the second patient.
  • the patient's cells have the characteristics of fewer mitochondria, shorter mitochondrial length, larger lysosomes and abnormal structure.
  • optical diffraction tomography imaging subsystem 210 has excellent mark-free imaging capabilities.
  • the optical diffraction tomography imaging subsystem 210 of the embodiment of the present application has better resolution and three-dimensional imaging capabilities.
  • Figure 13 shows the result of optical diffraction tomography of fixed INS-1 cells.
  • (a) is the original hologram;
  • (b) is the phase image;
  • (c) is the three-dimensional rendering of the refractive index distribution;
  • (d) is the wide-field fluorescence image;
  • (e) is the acquisition using the optical diffraction tomography subsystem 210
  • the obtained single-layer diffraction tomography image corresponding to the fluorescence image; the scale bar of (b), (d) and (e) is 10 ⁇ m.
  • FIG. 14 shows a comparison of the result of a wide-field fluorescence layer scan image of fixed primary hepatocytes and an optical diffraction tomography image.
  • (a) is the wide-field fluorescence image of different depths;
  • (b) is the single-layer diffraction tomography image of the corresponding depth;
  • the scale bars of (a) and (b) are both 10 ⁇ m.
  • the lipid droplet structure of different depths in the optical diffraction tomography image can correspond to the wide-field fluorescence image, which also verifies the three-dimensional imaging ability of the optical diffraction tomography.
  • the optical diffraction tomography imaging subsystem 210 of the embodiment of the present application also has fast and long-term non-destructive imaging capabilities. Specifically, the optical diffraction tomography imaging subsystem 210 does not need to dye and mark the cells, and therefore has low phototoxicity to the cells, and is suitable for long-term imaging of cells.
  • FIG. 15 shows the continuous imaging results of COS-7 cells with an interval of 10 seconds and a total of 83 minutes using the optical diffraction tomography subsystem 210.
  • (a) is a Z-plane image of the cell at 00:49:30, where the Z-plane is a horizontal plane perpendicular to the optical axis selected according to the content in the three-dimensional image;
  • (b) is four different moments in (a) The magnified image of the area shown by the dashed frame respectively shows the process of chromosome separation and aggregation, nuclear membrane formation, and chromatin aggregation into the nucleus;
  • (c) is another Z plane ((a) mid-plane 0.86) at 00:00 ⁇ m or less) cell image,
  • (d) is the enlarged image of the area shown by the dotted frame in (c);
  • (e) is the third Z plane ((a) mid-plane below 1.72 ⁇ m) at 00:43:10 Cell images;
  • (f) are images at two different moments in the area shown by the dashed frame in (e).
  • the tubular organelles are stretched and twisted during cell division, and the cells are arranged radially outside the nucleus after cell division.
  • (g) is an enlarged image of another cell, showing a plane of the nuclear region before cell division. Five different time points are shown. The nucleus and related nucleolus structures are clearly visible (0'00”).
  • One area of the nuclear membrane (arrow) is deformed by many incoming cellular structures (27'30”), and the nuclei in other areas
  • the scale bars in Figure 15 are 5 ⁇ m (a, c, e) and 2 ⁇ m (b, d, f, g).
  • Figure 15(a) it can be observed that there is a chromosome-like double structure in the nuclear region of dividing cells. In the process of mitosis, the chromosomes are first separated, and then two large Tightly connected high-density plaques.
  • Figure 15(b) shows the process of chromosomes being pulled apart, assembled and fused to form new nuclei during mitosis.
  • Figure 15(cd) in the cytoplasm, there are still Various structures with different shapes, densities and dynamics can be observed. For example, complex filament structures can be observed in the centrosome, while bright vesicle structures and large and dark vesicles are gathered in other areas. And black vacuole-like vesicles.
  • a tubular structure which is a tubular mitochondria.
  • the tubular mitochondria stretch and twist during the division process, and are arranged radially along the outer spindle after division.
  • the nucleus and related The nucleolus structure rotates, and then many afferent organelles will attach to an area in the cell membrane and deform. Then, the nuclear membrane disintegration is observed in the area opposite to the initial invagination position, and then the chromosome structure appears.
  • optical diffraction tomography module 210 can also be used to observe the biological processes of other cells, such as pollen tube growth (as shown in Figure 16), nematode embryonic development (as shown in Figure 17), etc. As shown in Figure 16 Optical diffraction tomography image of pollen tube growth process.
  • From left to right are the axial (for example, z-direction) extreme projections at 00:00, 03:20, 06:40, and 09:50 from left to right.
  • Figure 17 It is the optical diffraction tomography image of the embryonic development process of nematodes. From left to right are the axial extreme projections at 00:00, 04:20, 07:30 and 09:35. From the experimental results in Figure 15-17, you can It can be seen that the optical diffraction tomography imaging subsystem 210 has a fast, long-term and non-destructive imaging capability.
  • the structured light illumination fluorescence imaging subsystem 220 may be a system that applies structured light illumination technology for fluorescence imaging.
  • Traditional fluorescence microscopy imaging is limited by the imaging bandwidth of the microscope, and the resolution is limited by the optical diffraction limit.
  • the fine sample structure information corresponds to a higher spatial frequency. When this frequency exceeds the cut-off frequency of the optical transfer function of the microscope system, it cannot be imaged.
  • the structured light illumination fluorescence imaging subsystem 220 can load a certain spatial frequency to the illumination light through the grating, and the high-frequency information of the sample will be mixed with it to produce a lower spatial frequency, that is, the moiré effect.
  • the structured light illumination fluorescence imaging subsystem 220 may include linear structured light illumination imaging and/or nonlinear structured light illumination imaging.
  • the structured light illumination fluorescence imaging subsystem may be used to perform super-resolution fluorescence imaging based on the second laser to obtain the structured light illumination fluorescence image of the sample.
  • the structured light illumination fluorescence imaging subsystem 220 may include a second light source 133, a second acousto-optic modulator 132, a second polarization beam splitting prism 127, a second half-wave plate 128, a spatial light modulator 129, and a spatial light modulator 129. Filter 125, positive rotator 123, second camera 114 and other components.
  • the structured light illumination fluorescence imaging subsystem may also include a lens, a single-mode optical fiber, a dichroic mirror, and/or a coupler. In the embodiment of the present application, as shown in FIG.
  • the structured light illumination fluorescence imaging subsystem 220 may include a second light source 133, a second acousto-optic modulator 132, a coupler 136, a single-mode fiber 131, a lens 130, and a second light source.
  • the +1-order diffracted light is coupled to the single-mode fiber 131 through the coupler 136.
  • the second laser light is filtered by the single-mode fiber 131 and collimated by the lens 130.
  • the collimated linearly polarized laser light passes through the structured light system composed of the second polarization beam splitting prism 127, the second half-wave plate 128 and the spatial light modulator 129, the light field is loaded with the structured light modulation pattern on the spatial light modulator 129, That is structured light.
  • the structured light is focused on the spatial filter 125 by the lens 126 to filter out the required ⁇ 1st order diffraction, filter out the stray light generated by the spatial light modulator 129, and then collimate again by the lens 124.
  • the liquid crystal polarization rotator rotates the polarization of the illuminating light (and forms the excitation light) according to the fringe direction to keep the incident in the S polarization direction relative to the sample to form interference fringes.
  • the excitation light is reflected by the second dichroic mirror 121 to the lens 122, and is guided by the first dichroic mirror 112 to the microscope objective lens 111 and acts on the sample 110, thereby focusing the ⁇ 1st order light behind the microscope objective lens 111 On the focal plane, the diffraction-limited modulation fringes interfere with the sample 110 and excite fluorescence.
  • the excited fluorescence is collected by the microscope objective lens 111, and is guided to the lens 113 (such as the microscope with tube lens) through the first dichroic mirror 112, and is received by the second camera 114 after passing through the lens 113.
  • the structured light illumination fluorescence imaging subsystem may be an ultra-fast, long-term Hesen structured light illumination microscope system.
  • the objective lens such as the microscope objective lens 111
  • the first dichroic mirror 112 and/or the second dichroic mirror 121 in the dual-modality microscopic imaging system 100 may be used by the optical diffraction tomography subsystem 210 is shared with the structured light illumination fluorescent subsystem 220.
  • the sample light generated in the optical diffraction tomography subsystem 210 and the fluorescence generated by the structured light illumination fluorescence subsystem 220 may both be collected by the microscope objective lens 111.
  • the first dichroic mirror 112 may be used to separate sample light and fluorescence.
  • the second dichroic mirror 121 may be used to separate the sample light and the excitation light.
  • the first dichroic mirror 112 and the second dichroic mirror 121 can be used to guide the sample light to the spectroscope in the optical diffraction tomography subsystem, and to guide the excitation light in the structured light illumination fluorescence imaging subsystem. To the objective lens to act on the sample.
  • the first light source 101 and the second light source 133 may be two independent light sources in the dual-mode microscopic imaging system.
  • the first laser and the second laser may be emitted from the same combined light source (for example, the first light source 101 and the second light source 133 are combined).
  • a separate first light source 101 and a second light source 133 emit the first laser and the second laser for the optical diffraction tomography subsystem and the structured light illumination fluorescence imaging subsystem, respectively, so that fluorescence imaging can be independently performed.
  • the control effectively reduces the toxicity of the first laser and the second laser to the sample (such as cells), effectively reduces the influence of the light signal on the bleaching of the sample, and can help achieve rapid co-localization imaging observation.
  • the wavelength of the first laser and the second laser may be the same. In some embodiments, the wavelengths of the first laser and the second laser are different.
  • the first light source 101 may include a single longitudinal mode laser emitting a first laser with a wavelength of 561 nm
  • the second light source 133 may include a single longitudinal mode laser emitting a second laser with a wavelength of 488 nm.
  • the structured light illumination fluorescence imaging subsystem 220 may also acquire another structured light illumination fluorescence image of the sample based on the third laser.
  • the second laser light and the third laser light have different wavelengths.
  • the wavelength of the second laser may be 488 nm; the wavelength of the third laser may be 498 nm, 475 nm, and so on.
  • the third laser light may be emitted by the second light source.
  • the second light source can emit the second laser light or the third laser light.
  • the third laser light may also be emitted by a separate third light source.
  • the second light source may be replaced with a third light source, and the third laser light emitted by the third light source may be used for fluorescence imaging.
  • the process of acquiring another structured-light-illuminated fluorescent image of the sample based on the third laser is similar to the process of acquiring the structured-light-illuminated fluorescent image of the sample based on the second laser, and will not be repeated here.
  • the structured light illumination fluorescence imaging subsystem 220 may also acquire more structured light illumination fluorescence images of the sample based on more lasers of different wavelengths (such as the fourth laser, the fifth laser, etc.). By using lasers of different wavelengths, structured light illumination fluorescence images with different resolutions can be obtained, so that more information about the sample can be obtained, so that the observation of the sample is more comprehensive and accurate.
  • the optical diffraction tomographic image of the sample acquired by the optical diffraction tomography subsystem 210 may be a two-dimensional image and/or a three-dimensional image.
  • the structured light illumination fluorescence image of the sample acquired by the structured light illumination fluorescence imaging subsystem 220 may be a two-dimensional image and/or a three-dimensional image.
  • the optical diffraction tomography image may be a three-dimensional image
  • the structured light illumination fluorescence image may be a two-dimensional image.
  • structured light illumination fluorescence imaging can be used to assist label-free optical diffraction tomography.
  • the dual-mode microscopy imaging system 100 can be used at a speed of not less than 0.3Hz (such as 0.3Hz, 0.4Hz, 0.5Hz, 0.7Hz, 0.8Hz, etc.) to not less than 80 ⁇ m ⁇ 80 ⁇ m ⁇ 40 ⁇ m (such as 80 ⁇ m ⁇ 80 ⁇ m ⁇ 40 ⁇ m). , 100 ⁇ m ⁇ 100 ⁇ m ⁇ 50 ⁇ m, etc.) for fluorescence imaging and label-free optical diffraction tomography.
  • the lateral resolution of the obtained optical diffraction tomography image may not be worse than 200nm (such as 200nm, 190nm, 180nm, etc.), and the longitudinal resolution may not be worse than 560nm (such as 560nm, 540nm, 500nm, etc.); the resulting structured light illumination fluorescence
  • the lateral resolution of the image may not be worse than 100nm (such as 100nm, 95nm, 90nm, etc.).
  • the control subsystem 230 can be used to control each device in the dual-mode microscopic imaging system 100.
  • the control subsystem 230 can control the first light source 101, the first acousto-optic modulator (AOM) 102, the galvanometer (GM) 107, the second camera 114, the first camera 117, and the second acousto-optic modulator.
  • One or more devices such as the detector 132 and the second light source 133.
  • the control subsystem 230 can control the first light source 101 to emit the first laser.
  • the control subsystem 230 may control the second light source 133 to emit the second laser.
  • the control subsystem 230 can control the rotation of the galvanometer 107.
  • the control subsystem 230 may control the exposure of the second camera 114 and/or the first camera 117.
  • control subsystem 230 can control the working sequence of the optical diffraction tomography subsystem 210 and the structured light illumination fluorescence imaging subsystem 220, so as to realize the simultaneous or alternate execution of label-free optical diffraction tomography and fluorescence imaging. .
  • control subsystem 230 please refer to FIG. 3 and related descriptions.
  • the processor 240 may be used to process information/data in the dual-modality microscopic imaging process.
  • the processor 240 may determine the bimodal fusion image of the sample at the same location based on the optical diffraction tomography image and the structured light illumination fluorescence image.
  • the bimodal fusion image can have both the morphological information of the sample and the category labeling information.
  • the morphological information of the sample may include the size and shape of the sample.
  • the category labeling information of the sample may include the type of the labeled part of the sample (for example, what organelle is).
  • the processor 240 may include a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a central processing unit (CPU) , Graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device, and Any circuit, processor, etc. that perform one or more functions, or any combination thereof.
  • RISC reduced instruction set computer
  • ASIC application specific integrated circuit
  • ASIP application specific instruction set processor
  • CPU central processing unit
  • GPU Graphics processing unit
  • PPU physical processing unit
  • DSP digital signal processor
  • FPGA field programmable gate array
  • ARM advanced RISC machine
  • Fig. 3 is a schematic diagram of a control subsystem of a dual-mode microscopic imaging system according to some embodiments of the present application.
  • the control subsystem 230 can be implemented by a combination of one or more of a data acquisition card (DAQ), a single-chip microcomputer, and a field programmable logic gate array (FPGA).
  • the control subsystem 230 may include a control computer (PC) and a data acquisition card (DAQ).
  • the four analog output channels of the data acquisition card (DAQ) respectively control the two-axis galvanometer scanning, the Z-axis voltage level shifter and the liquid crystal polarization rotator.
  • the Z axis may indicate a direction perpendicular to the plane of the slide of the sample (such as biological cells).
  • the data acquisition card can output control voltages to the galvanometer servo circuit at the analog output ports AO0 and AO1 to control the angle of the galvanometer, and then control the deflection angle of the beam acting on the galvanometer 107 along the x and y directions.
  • the x and y directions are two orthogonal directions parallel to the sample slide.
  • the data acquisition card can output a control voltage at the AO2 port of the analog output terminal to control the Z-axis pressure level shifter, which can be used to move the sample in the Z-axis direction.
  • the data acquisition card can output the control voltage at the analog output port AO3 port to control the signal generator.
  • the liquid crystal polarization rotator is controlled by an amplitude modulated 20kHz square wave signal generated by the signal generator, and the amplitude modulated voltage is generated by the data acquisition card.
  • the spatial light modulator 129 and the two cameras are controlled by the four programmable digital output ports of the data acquisition card. Among them, PFI0.0 and PFI0.1 control the switching and triggering of the spatial light modulator 129, PFI0.2 and PFI0.3 Output camera external trigger signal. Specifically, the digital output port PFI0.2 can output a square wave signal with adjustable delay and duty cycle to the external trigger port of the camera to control the camera to synchronize exposure during the scanning of the galvanometer.
  • the scanning voltage can be a sine-cosine sawtooth voltage in the range of ⁇ 205mV, and a 5V TTL level can be used for camera triggering.
  • the camera exposure output port can output a 5V TTL exposure signal to the first acousto-optic modulator 102 to control the exposure time.
  • the dual-modality microscopic imaging system may be a super-resolution fluorescence assisted diffraction tomography (SR-FACT) system.
  • SR-FACT super-resolution fluorescence assisted diffraction tomography
  • the resolution of the dual-mode microscopy imaging system is:
  • ⁇ represents the illumination wavelength
  • NA represents the numerical aperture (NA) of the optical system.
  • rotating illumination extends the lateral spatial frequency limit.
  • the scattered field in the optical diffraction tomography system is essentially to detect the frequency shift in the frequency domain, which extends the bandwidth of the lateral frequency shift by the following equation: k
  • , odt k
  • , det respectively represent the lateral projection of the maximum numerical aperture of the illumination objective lens and the detection objective lens, and the numerical aperture of the illumination objective lens is 1.0.
  • the longitudinal spatial frequency bandwidth essentially depends on the transverse frequency. In the area where k
  • Fig. 4 is a control sequence diagram of a dual-mode microscopic imaging system according to some embodiments of the present application.
  • the control sequence of the dual-mode microscopic imaging system may be implemented by the control subsystem 230.
  • the dual-modality microscopic imaging system 100 can perform dual-modality microscopic imaging (such as long-term dynamic observation) on the sample for multiple time periods. There may be a certain time interval between two adjacent time periods (such as 0.5 second, 1 second, 3 seconds, etc.). For a time period, as shown in FIG.
  • the dual-modality microscopy imaging system 100 may first perform optical diffraction tomography (ODT) on the sample based on the optical diffraction tomography subsystem 210, and then perform optical diffraction tomography (ODT) based on the structured light illumination fluorescence imaging sub-system.
  • ODT optical diffraction tomography
  • the system 220 performs structured light illumination fluorescence imaging (SIM) on the sample.
  • the data acquisition card (DAQ) in the control subsystem 230 can respectively output the X-axis and Y-axis scanning voltages of the galvanometer 107 at the analog output ports AO0 and AO1 ports (such as within the range of ⁇ 205mV). Sine and cosine sawtooth voltage) to the galvanometer servo circuit to realize the deflection of the beam in the x and y directions.
  • the digital output port PFI0.2 of the data acquisition card can output the first camera trigger signal (such as 5V TTL level) to the first camera 117 (ODT camera), and the exposure output port of the first camera can output the exposure signal (for example, a 5V TTL exposure signal) to the first acousto-optic modulator 102 to turn on the first acousto-optic modulator 102 to release the first laser to trigger the exposure of the first camera 117 and control the exposure time.
  • the data of each time period can be composed of 240 off-axis holograms under different illumination angles.
  • the maximum frame rate of the first camera under the 1024 ⁇ 1024 frame can be 200 fps, and the actual use frame rate is 196 fps, so a single set of data acquisition takes 1.225s.
  • the illumination light of the first and last images can both be incident perpendicular to the plane of the sample; when other images are taken, the galvanometer controls the focus of the beam on the back focal plane of the objective lens to perform circular scanning.
  • the first camera 117 may be a sCOMS camera, which enters the exposure state line by line during exposure.
  • the sCMOS camera can be operated in synchronized exposure mode. Specifically, the rising edge of the camera exposure trigger is synchronized with the exposure start time of lines 1023 and 1024 in the camera. After lines 513 to 1535 are in the exposure state, the camera outputs a square wave switch signal with an adjustable time width to the first line.
  • the acousto-optic modulator 102 ie, ODTAOM in the figure
  • the sCMOS chip After the exposure, the sCMOS chip starts to read data row by row from the middle to the upper and lower sides, and immediately enters the next round of exposure sequence.
  • the opening time of the first acousto-optic modulator 102 can be set to 50 ⁇ s.
  • the vertical illumination images in each group of data collected by the optical diffraction tomography subsystem 210 can be used for timing checks during data processing to reduce data caused by occasional frame loss during high-speed camera data collection. The timing is out of order. At the same time, in order to eliminate the influence of uneven amplitude distribution and residual uneven phase of the illumination beam due to defects in the optical system, a set of background data can be collected on the sample glass slide before each sample change for data processing.
  • the data acquisition card can give the spatial light modulator 129 (SLM) a start signal, and each time the structured light illumination fluorescence imaging is performed, the SLM start signal can be given, and the SLM start signal can guarantee every The first image of the group data is correct to avoid confusion in multiple groups of data due to the lack of one or more images of a certain group of data during continuous shooting.
  • DAQ data acquisition card
  • SLM spatial light modulator 129
  • the spatial light modulator 129 can acquire multiple (such as 9) trigger signals to form multiple (such as 9) structured light modulation patterns, and sequentially obtain multiple (such as 9) structured light patterns. Illuminate the fluorescent image.
  • the data acquisition card (DAQ) can output a trigger signal to the second camera (SIM), and the exposure output port of the second camera 114 can output a trigger signal to the second acousto-optic modulator 132 In this way, the second acousto-optic modulator 132 is turned on to release the second laser, and the second camera 114 is exposed.
  • the CMOS chip of the second camera 114 reads out the data, which is composed of 9 structured-light-illuminated fluorescent images.
  • the exposure time of the second camera 114 may be longer than the exposure time of the first camera.
  • the exposure time of the second camera 114 can be set to 30 ms to ensure the signal-to-noise ratio of the fluorescence image and avoid bleaching the cell sample.
  • a different deflection voltage may be applied to the polarization rotator 123 after three exposures to change the rotation of the polarization rotator 123. angle.
  • Fig. 5 shows an exemplary flow chart of a dual-modality microscopic imaging method according to some embodiments of the present application.
  • the dual-modality microscopic imaging method 500 may be performed by the dual-modality microscopic imaging system 100.
  • the dual-modality microscopic imaging method 500 may include:
  • Step 510 using mutually independent light sources to respectively generate the first laser and the second laser.
  • the first laser light is generated by the first light source of the optical diffraction tomography imaging subsystem 210; the second laser light is generated by the second light source of the structured light illumination fluorescence imaging subsystem 220.
  • the first light source and the second light source are two independent lasers in the dual-mode microscopic imaging system 100.
  • the wavelength of the first laser and the second laser may be the same.
  • the wavelengths of the first laser and the second laser may be different.
  • the wavelength of the first laser light may be 561 nm
  • the wavelength of the second laser light may be 488 nm.
  • an optical diffraction tomography imaging subsystem is used to obtain an optical diffraction tomography image of the sample based on the first laser.
  • the optical diffraction tomography subsystem 210 may use the first laser to act on the sample to obtain a first imaging data set of the sample (the original optical diffraction tomography data as shown in FIG. 6), specifically For the implementation manner, refer to the above description of the optical diffraction tomography imaging subsystem 210, which will not be repeated here.
  • the first imaging data set of the sample may include sample information.
  • the processor 240 may use a diffraction tomography reconstruction algorithm to process the first imaging data set of the sample to generate an optical diffraction tomography image.
  • Fig. 6 is a flowchart of a diffraction tomography reconstruction algorithm according to some embodiments of the present application.
  • Fig. 11a is a schematic diagram of an optical diffraction tomographic image reconstruction algorithm according to some embodiments of the present application. As shown in Figure 6 and Figure 11a, the two sets of data of the sample-free background hologram and the sample's hologram at multiple time points are both input as the raw data of optical diffraction tomography. Among them, without a sample, one background hologram under different illumination angles can be collected.
  • the diffraction tomography reconstruction algorithm used in the sample-based first imaging data set may include: a holographic processing step, a wave vector calculation step, a sequence check step, a Rytov calculation step, a spectrum stitching step, and an inverse filtering step .
  • a holographic processing step a wave vector calculation step
  • a sequence check step a sequence check step
  • a Rytov calculation step a spectrum stitching step
  • an inverse filtering step inverse filtering step
  • the holographic processing step may be used to extract a hologram based on the first imaging data set, where the hologram may include an amplitude image and a phase image.
  • the holographic processing step may use the following techniques to extract holograms based on the first imaging data set. These techniques include, but are not limited to, coaxial digital holography, off-axis digital holography, heterodyne holography, and phase shifting. Holographic technology and so on. For example only, in the holographic processing step, off-axis digital holography technology may be used to obtain a hologram (i.e., two-dimensional light field amplitude image and phase image) based on the first imaging data set.
  • the reference light does not propagate coaxially with the sample light, but an off-axis angle ⁇ exists.
  • the off-axis angle introduces a certain space factor to the reference light on the holographic image surface ⁇ represents the wavelength, and ⁇ represents the off-axis angle.
  • the reference light of the holographic image surface can be written as:
  • r represents the amplitude of the reference light on the holographic image surface.
  • Reference light and sample light The hologram formed after interference can be expressed as:
  • o(x, y) represents the amplitude of the sample light on the holographic image surface
  • ⁇ (x, y) represents the phase distribution of the sample light on the holographic image surface
  • the restored light field generated by the original reference light is:
  • a virtual image is generated in the direction of the original sample light of the hologram, and a real image is generated in the direction of twice the off-axis angle between the back of the holographic image surface and the original sample light, and the space separation between the real image and the virtual image is realized.
  • the collected off-axis hologram can be expressed as:
  • m represents the m-th pixel in the x direction
  • n represents the n-th pixel in the y direction
  • ⁇ p represents the pixel size
  • L represents the low-pass matrix
  • DFT. and IDFT. respectively represent the discrete Fourier transform and the inverse discrete Fourier transform.
  • u represents the spatial frequency in the x direction
  • v represents the spatial frequency in the y direction.
  • ⁇ (u,v) represents the Dirac function on the spatial frequency coordinate
  • the autocorrelation operation makes the cut-off frequency of the second term in equation (6) become 2u m , and the cut-off frequency of the last two terms is still u m . It is not difficult to see that in order to ensure that the items in equation (6) do not overlap each other in the spatial frequency spectrum, the requirements for the spatial carrier frequency are:
  • the reference light and signal of each hologram can be determined based on the distance between the DC term and the modulation term measured in the frequency domain measurement of the hologram.
  • the angle between the transmitted illuminating light in the light Record the frequency domain distance obtained from the j-th hologram as The unit is pixel.
  • the reference wave vector of sub-pixel accuracy can be obtained by calculating the average value of l distances:
  • the lateral component of the illumination light wave vector at the jth illumination angle (corresponding to the jth hologram) is:
  • Equation 11 Is the initial estimation result of the lateral component of the illumination light wave vector at the jth illumination angle.
  • the wave vector calculation step may be used to determine the target scanning wave vector based on the phase image, and generate an unwrapping phase image.
  • the high-speed scanning light beam is unavoidably affected by the small tremors in the mechanical movement of the opto-mechanical device.
  • the wave vector calculation step further includes a vector iterative search algorithm, which can be used to accurately solve the scan wave vector at different angles, that is, the target scan wave vector.
  • the process 700 of using the vector iterative search algorithm to determine the target scanning wave vector in FIGS. 7a-b may include the following four steps:
  • Step 710 Multiply the phase image extracted in the holographic processing step by the digital phase shift term to obtain a frequency-shifted phase image and a preliminary estimated scanning wave vector.
  • step 710 the phase image extracted by the holographic processing step is shifted with the digital phase Multiply, you can get a frequency-shifted phase image, and a preliminary estimated sweep wave vector.
  • the preliminary estimated scan wave vector in step 710 can be used to update the scan wave vector, and how to use it will be explained in step 730.
  • Step 720 unwrap the frequency-shifted phase image by using a phase unwrapping algorithm based on least squares to obtain an unwrapped phase image with a low residual slope
  • Step 730 Perform linear fitting on the slope of the unwrapping phase image in the orthogonal direction to obtain an updated scanning wave vector.
  • linear fitting may be performed on the slope of the unwrapping phase image in the orthogonal direction, and based on the fitting result and the preliminary estimated scanning wave vector, an updated scanning wave vector can be obtained.
  • step 730 the slope of the unwrapped phase image of the hologram under the jth illumination angle in the orthogonal direction (x, y direction) Perform a linear fit and update the sweep wave vector:
  • Step 740 based on the updated scan wave vector, repeat iterations until the slope
  • the preset threshold may be set to 0.00001 ⁇ 2 ⁇ .
  • Figure 8 shows the sweep wave vector at a certain angle
  • ⁇ represents the scanning wave vector calculated from the background hologram
  • + represents the scanning wave vector calculated from 100 sets of sample holograms with an interval of 1s.
  • Wave vector calculation step 8 the average deviation of the scanning wavevector wavevector scan data with the background sample data is 0.018 (7) ⁇ m -1; offset corresponding to a 0.2% k m. Corresponding to the scan angle difference of 0.05°.
  • the standard deviation of the wave vector fluctuation of the illumination scan at 100 consecutive time points is and Fluctuation of 0.07% k m.
  • FIG. 9 is a reconstructed image when the scanning wave vector iteration is not performed
  • FIG. 10 is a reconstructed image when the scanning wave vector iteration is performed. Comparing FIG. 9 and FIG. 10, it can be seen that the scanning wave vector iteration can make the background of the reconstructed image more uniform and have better contrast.
  • the diffraction tomography reconstruction algorithm may further include a sequence check step, and the sequence check step may be executed after the wave vector calculation step.
  • the hologram may include a sample hologram and a background hologram, and the sequence checking step may include: comparing the scanning wave vector of a group of sample holograms with the scanning wave vector of the background hologram at the same time point to obtain the group of samples The scanning error of the hologram; when the scanning error is greater than the set threshold, the sequence abnormality of the group of sample holograms is marked; based on the sequence abnormality, the grouping of the first imaging data set and the spectral splicing step are controlled.
  • the scanning wave vector is obtained.
  • the scanning error of the group of sample holograms can be obtained, which can be considered as having a larger scanning error if it is greater than the set threshold.
  • the group of holograms is marked. In the process of controlling the grouping of the first imaging data set and the spectral splicing step, it is possible to choose not to use the sample hologram data of these groups marked as sequence abnormalities. In this embodiment, since file input and output errors may occur during the storage of high-throughput data, single frame loss may occasionally occur.
  • the sequence check step can use the precise scanning wave vector data corresponding to each hologram to determine whether the image sequence is abnormal under a certain difference threshold, and feed back the abnormal sequence result to the optical diffraction tomography subsystem, which can
  • the grouping and spectrum splicing steps of the first imaging data set are controlled based on sequence abnormalities to ensure that the occasional frame loss problem does not affect the long-term data reconstruction result.
  • the Rytov approximation step may be used to determine the Rytov phase field based on the amplitude image and the unwrapping phase image.
  • the light field can be described by the complex phase ⁇ (r)
  • the total light field u(r) and the incident light field u 0 (r) can be described by the complex phase ⁇ (r) and can be expressed as:
  • the complex phase ⁇ (r) of the total light field and the complex phase ⁇ 0 (r) of the incident light field have the following forms:
  • a(r) and ⁇ (r) represent the light field amplitude and phase distribution, respectively.
  • the light field amplitude can be determined based on the amplitude image, and the phase distribution can be determined based on the unwrapping phase image.
  • ⁇ s (r) is the complex phase of the scattered light field.
  • the scattered field can be expressed as:
  • Equation (19) k(r) is the wave vector that depends on the spatial distribution of refractive index, u(r) is the spatial distribution of the optical field, It is the Hamiltonian.
  • k m represents the average number of medium wave
  • f (r) represents the scattering potential
  • Equation (22) is the nonlinear and non-homogeneous differential equation of complex phase ⁇ (r).
  • the complex phase ⁇ 0 (r) of the incident light field also satisfies the homogeneous equation:
  • formula (31) is a formula for determining the Rytov phase field based on the amplitude image obtained in the whole system processing step and the unwrapping phase image obtained in the wave vector calculation step.
  • the Rytov phase field can be obtained by calculating the amplitude image obtained in the holographic processing step and the low residual slope unwrapping phase image obtained in the wave vector calculation step according to equation (31).
  • the spectral splicing step may be used for splicing in the frequency domain based on the Rytov phase field.
  • the frequency spectrum splicing step may be used to splice the two-dimensional spectrum of the Rytov phase field and the complex transmission spectrum with the target scanning wave vector as parameters to obtain a three-dimensional scattering spectrum and a three-dimensional complex transmission spectrum.
  • frequency domain interpolation (such as immediate interpolation or linear interpolation) may be used to implement spectrum splicing. Take the close-to-close interpolation method as an example.
  • the close-to-close interpolation method can directly interpolate the two-dimensional grid points of the two-dimensional spectrum to the three-dimensional grid points of the nearest three-dimensional spectrum for calculation without additional interpolation or spectrum expansion calculations. Therefore, there is basically no need to spend additional calculation time.
  • the algorithm based on the immediate interpolation method can achieve fast image processing, and it has been proven to have better calculation accuracy in the case of holographic bandwidth reorganization.
  • the inverse filtering step may be used to filter the spectral mosaic result to obtain an optical diffraction tomography image.
  • the inverse filtering step is required to restore the image.
  • the inverse filtering step can be used to divide the three-dimensional scattering spectrum and the three-dimensional complex transmission spectrum obtained in the spectrum splicing step based on the Wiener inverse filtering principle to obtain an optical diffraction tomography image.
  • the result of spectrum splicing can be multiplied by the complex conjugate of the three-dimensional complex transmission spectrum and then divided by the square of the three-dimensional complex transmission spectrum, so that the influence of noise can be eliminated as much as possible, so that the obtained optical diffraction tomography image clearer.
  • the processor 240 may be used to execute a complex deconvolutional diffraction tomography three-dimensional reconstruction algorithm to process the first imaging data set to generate an optical diffraction tomography image.
  • the complex deconvolutional diffraction tomography three-dimensional reconstruction algorithm can simultaneously splice the complex propagation spectrum in the frequency domain splicing step, and realize the normalization of the three-dimensional scattering spectrum and the three-dimensional complex transmission spectrum in the inverse filtering step.
  • G(x, y, z) is the far-field propagation term, which can be described by Green's function under scalar approximation:
  • ⁇ distinguishes between forward propagation and backward propagation.
  • equation (32) can be rewritten as:
  • FT represents Fourier transform. Since the monochromatic wave has a constant wave vector k m during the propagation process, the three-dimensional coherent transfer function C (k x , k y , k z ) can be regarded as the two-dimensional coherent transfer function C (k x , k y ) on the three-dimensional spherical surface On the mapping.
  • the scattering potential can be reconstructed based on the principle of Wiener inverse filtering.
  • the three-dimensional scattering potential can be solved as:
  • splicing and filtering can be performed in the frequency domain according to equations (37) and (39).
  • the two-dimensional frequency spectrum can be mapped on the nearest three-dimensional frequency spectrum grid point by the method of immediate interpolation.
  • the two-dimensional spectral mapping process a three-dimensional uniformly spaced lattice points in the spectrum K x and K y directions and two-dimensional lattice spacing, so Rytov approximation only interpolating the K z direction.
  • the smaller the grid point interval in the K z direction the higher the accuracy of the immediate interpolation.
  • a structured light illumination fluorescence imaging subsystem is used to obtain a structured light illumination fluorescence image of the sample based on the second laser.
  • the structured light illumination fluorescence subsystem may acquire the first imaging data of the sample based on the second laser.
  • the structured light illumination fluorescence subsystem can use a second laser to act on the sample to obtain a second imaging data set of the sample.
  • the second imaging data set of the sample may include sample information with fluorescent labels.
  • the structured light illumination fluorescence imaging subsystem may use a structured light illumination fluorescence image reconstruction algorithm to obtain a structured light illumination fluorescence image of the sample based on the second imaging data set.
  • Figure 11 is a flow chart of the algorithm for structured light illumination fluorescence image reconstruction.
  • the structured light illumination fluorescence image data first needs to calculate the reconstruction parameters based on the image data, and then the high-resolution fluorescence image is obtained through spectral component separation and splicing.
  • the structured light illumination fluorescence image reconstruction algorithm may include: calculating reconstruction parameters based on image data, spectral component separation, spectral splicing, and inverse Fourier transform to obtain a high-resolution fluorescence image.
  • coherent illumination lights in two directions interfere to form sinusoidal structured light:
  • ⁇ is the interference depth
  • p is the spatial frequency of the structured light
  • ⁇ is the initial phase of the structured light
  • the fluorescence intensity is proportional to the illumination intensity
  • the intensity image of wide-field imaging is:
  • D(r) represents the density distribution of fluorescent molecules
  • * represents the convolution operation
  • p(r) represents the light intensity point spread function of the system, according to the two-dimensional coherence function
  • p(r) can be defined as:
  • Equation (41) can be expanded in the frequency domain as:
  • the spectrum of the structured light illumination fluorescence image contains the original spectrum and the spectrum of the shift term ⁇ p, and the three spectrum information is the optical transfer function Filtered at the cutoff frequency.
  • ⁇ and perform three measurements to obtain the spectrum respectively:
  • Wiener inverse filtering can be used to solve the high-resolution fluorescence density distribution spectrum as:
  • Equation (54) shows that the final spectrum contains a total of 7 spectrum components, of which six are evenly distributed in six directions to expand a certain spectrum range.
  • the final fluorescence density distribution image (structured light illumination fluorescence image) can be obtained as:
  • Step 540 Based on the optical diffraction tomography image and the structured light illumination fluorescence image, a bimodal fusion image of the sample is generated. Specifically, step 540 may be executed by the processor 240.
  • the processor 240 may use a co-localization image fusion image processing technology to fuse the optical diffraction tomography image and the structured light illumination fluorescence image into a bimodal fusion image.
  • the bimodal fusion image can have both the morphological information of the sample and the category labeling information.
  • the morphological information of the sample may include the size and shape of the sample.
  • the category labeling information of the sample may include the type of the labeled part of the sample (for example, what organelle is).
  • the processor 240 may select a two-dimensional tomographic image that is most similar to a two-dimensional structured light illumination fluorescence image from the three-dimensional optical diffraction image for bimodal fusion.
  • the bimodal fusion image may be the result of fusion processing after marking the fluorescence information (such as fluorescently labeled organelles) displayed in the structured light illumination fluorescence image in the optical diffraction tomography image.
  • the marking method may include, but is not limited to, highlight display, display in different colors, and so on.
  • the processor 240 may process the optical diffraction tomography image, the structured light illumination fluorescence image, and/or the bimodal fusion image of the sample, such as image segmentation (for example, for each image in the image). Organelle segmentation), quantitative statistics (for example, statistics on the path information of a certain organelle), etc.
  • the processor 240 may perform online or real-time processing of the aforementioned images.
  • the processor 240 may perform offline or post-processing of the above-mentioned image.
  • the above-mentioned images may be processed by other processors other than and/or associated with the dual-modality microscopic imaging system 100.
  • the embodiment of the present application adopts a dual-mode microscopic imaging system as shown in Figs. 1-2.
  • the dual-modality microscopic imaging system includes an optical diffraction tomography imaging subsystem 210 and a structured light illumination fluorescence imaging subsystem 220.
  • the optical diffraction tomography imaging subsystem 210 and the structured light illumination fluorescence imaging subsystem 220 can at least be combined with the first dichroic mirror 112 and the second dichroic mirror 121 to form a dual-mode microscopic imaging system.
  • the dual-modality microscopic imaging system may include all the components of the optical diffraction tomography subsystem 210 and the structured light illumination fluorescence imaging subsystem 220.
  • the dual-mode microscopy imaging system can include a first light source 101, a first acousto-optic modulator (AOM) 102, a first half-wave plate (HWP) 103, a first polarization beam splitter (PBS) 104, and a single-mode fiber (SMF) 105, lens 106, galvanometer (GM) 107, tube lens 108, microscope objective lens (OBJ) 109, sample 110, microscope objective lens 111, first dichroic mirror (DM) 112, lens 113, Second camera 114, single-mode fiber 115, lens 116, first camera 117, polarization-independent beam splitter (BS) 118, lens 119, lens 120, second dichroic mirror 121, lens 122, polarization rotator (PR) 123, lens 124, spatial filter (Mask) 125, lens 126, second polarization beam splitter 127, second half-wave plate 128, spatial light modulator (SLM) 129, lens 130, single
  • the first light source 101 can be a single longitudinal mode laser with a model of MSL-FN-561-50mW and a wavelength of 561nm, manufactured by Changchun New Industry Optoelectronics Technology Co., Ltd.; the first acousto-optic modulator is China Electronics Technology Group Chongqing Acousto-optic Co., Ltd.
  • the model of the first half-wave plate is Thorlabs, AHWP10M-600; the model of the first polarization beam splitter is Thorlabs, CCM1-PBS251; the model of the couplers 134 and 135 is Thorlabs, PAF2-7A; Mode fibers 105 and 115 are made by Shanghai HannStar, the model is PM460-HPHA, FC/APC polarization maintaining single-mode fiber; the focal length of lens 106 is 40mm, specifically it can be the model AC254-040-A produced by Thorlabs in the United States Lens; the model of the galvanometer 107 is Thorlabs, GVS211/M; the focal length of the tube lens 108 is 180mm, specifically it can be the lens of the model AC508-180-A produced by Thorlabs in the United States; the model of the microscope objective 109 is OLYMPUS, LUMPlanFLN, 60x/1.0W; the model of the microscope objective 111 is OLYMPUS, ApoN 100x/1.49Oil; the model
  • the second light source is a single longitudinal mode laser with a wavelength of 488nm
  • the model is Coherent, Sapphire488LP-200
  • the model of the second half-wave plate 128 is Thorlabs, AHWP10M-600
  • the model of the second polarization beam splitter 127 is Thorlabs, CM1-PBS25
  • the model is AAOpto-Electronic, AOTF
  • the model of spatial light modulator 129 is Fourth Dimension Display, SXGA-3DM
  • the model of lens 124 Thorlabs, AC508-300-A , F 200mm
  • the second camera 114 is an sCM
  • the movement of structures in living cells may cause motion artifacts and reduce resolution.
  • LY lysosome
  • the spatial resolution must match the corresponding time resolution in order to achieve the maximum in live cell optical diffraction tomography. Resolution.
  • the dual-modality microscopic imaging system of the embodiment of the present application can perform high-speed cell imaging based on fluorescence co-localization.
  • the entire acquisition period of the structured light illumination fluorescence imaging subsystem 220 and the optical diffraction tomography imaging subsystem 210 is 1.49s, which is fast enough for cell imaging, so that the dual-modality microscopy imaging system can be used for two kinds of imaging
  • the subsystems alternately detect the same structure in living cells.
  • Figure 18 shows the optical diffraction tomography-fluorescence co-localization image collected by the dual-modality microscopic imaging system of the six main organelles in COS-7 living cells.
  • the first column is the structured light illumination fluorescence reconstruction image
  • the second column is the optical diffraction tomography image
  • the third column is the dual-mode fusion image
  • the right three columns are the first column image in turn
  • the structured light illumination fluorescence imaging subsystem 220 can analyze the internal structure of mitochondria, while the optical diffraction tomography imaging subsystem 210 can provide the three-dimensional dynamics of all mitochondria in the entire organelle at the same time, and the biodynamic information it can provide is much higher than Two-dimensional fluorescence imaging. And because the optical diffraction tomography subsystem 210 has the characteristics of non-phototoxicity and photobleaching effect that limits the imaging time, the dual-modality microscopic imaging system can perform hour-long three-dimensional continuous imaging of living cells (as shown in Figure 15). Shown).
  • the dual-modality microscopic imaging system can also observe lipid droplets, late endosomes or lysosomes, nuclear membranes and chromosomes in living cells.
  • the bright vesicle structure in the optical diffraction tomography image is confirmed to be lipid droplets by the fluorescent co-localization image of the dye Lipidspot 488 accumulated in the lipid droplets ( Figure 18(c)); at the same time, the optical diffraction tomography image
  • the darker vesicles were confirmed to be late endosomes or lysosomes by the fluorescent colocalization image of Lysoview488 used to label acid lysosomes ( Figure 18(d)); near the nucleus, continuous optical diffraction tomography images
  • the membranous structure was confirmed to be the nuclear membrane ( Figure 18(e)) due to co-localization with the fluorescent image labeled with laminA-EGFP; the irregular-like structure in the nuclear membrane was co-localized with the H2B
  • the dual-modality microscopic imaging system of the examples of the present application can also observe that the RI of dark vacuoles is even lower than the cytoplasmic RI of COS-7 cells (as shown in Figure 15(d)). Because optical diffraction tomography can measure the temporal and spatial distribution of mass density in living cells, these vacuole structures contain much less material than cytosolic gel, which is similar to vacuoles in plants and yeast.
  • yeast vacuoles observed under the same optical diffraction tomography microscope are larger and darker than those in mammalian cells, so these structures can be named dark vacuoles (DBs).
  • the dual-modality microscopic imaging system of the embodiment of the present application can observe actin filaments.
  • Figure 19 shows the lateral resolution of the optical diffraction tomography-structured light illumination fluorescence dual-mode microscopic imaging system.
  • Scale bar 5 ⁇ m.
  • Figure 20 is a dual-modal mitotic fluorescence colocalization image of COS-7 cells collected by a dual-modality microscopic imaging system.
  • the first line is the structured light illumination fluorescence reconstruction image
  • the second line is the optical diffraction tomography image
  • the third line is the dual-modal fusion image.
  • the four columns are the four moments in the imaging. Scale bar: 2 ⁇ m.
  • FIG. 34 is a schematic diagram of the three-dimensional overall observation of the organelles using the optical diffraction tomography imaging subsystem. Among them, (a) Three different Z planes of a typical COS-7 unit. The arrows indicate mitochondria, lysosomes (LYs) and lactic acid delipidated droplets (LDs), respectively. (b) In the axial volume of 0.86 ⁇ m (10Z plane), the average percentage of LDs, LYs and mitochondria is the percentage of the total area of all LDs, LYs and mitochondria in the cell (average from 13 cells). Scale bar, 5 ⁇ m. Center line, median; limit, 75% and 25%; quantity, maximum and minimum. It can be seen from Figure 34 that the phototoxicity produced by structured light illumination fluorescence imaging will block COS-7 cells in the late stage.
  • Figure 21 is an optical diffraction tomography-fluorescence co-localization image of organelles that cannot be resolved by optical diffraction tomography in COS-7 living cells by the dual-modality microscopic imaging system.
  • the first column is the structured light illumination fluorescence reconstruction image
  • the second column is the optical diffraction tomography image
  • the third column is the dual-mode fusion image
  • the right three columns are the area indicated by the dotted frame in the first column of the image.
  • the magnified structured light illumination fluorescence reconstruction image, optical diffraction tomography image and bimodal fusion image are the area indicated by the dotted frame in the first column of the image.
  • the dual-mode microscopic imaging system of the embodiment of the present application can also observe and analyze the low refractive index vesicles (DBs) that appear during the imaging process.
  • the refractive index of the low refractive index vesicles is significantly lower than that of the surrounding cytoplasm. .
  • FIG. 22 shows the low refractive index vesicles appearing in the optical diffraction tomography image of living COS-7 cells.
  • low refractive index vesicles do not co-localize with Rab7EGFP-labeled lysosomes (a) and LAMP1-EGFP-labeled late lysosomes (b); they are not co-localized with Rab5a-EGFP-labeled vesicles (c), EEA1-EGFP Marked vesicles (d), FYVE-EGFP-labeled vesicles (e) and Rab9a-EGFP-labeled vesicles (f) partially co-localized vesicles (g).
  • the first column is the structured light illumination fluorescence reconstruction image
  • the second column is the optical diffraction tomography image
  • the third column is the dual-mode fusion image
  • the right three columns are the magnification of the area shown by the dotted frame in the first column image Structured light illumination fluorescence reconstruction image, optical diffraction tomography image and bimodal fusion image. Scale bars: 5 ⁇ m (left), 1 ⁇ m (right).
  • the dual-modality microscopic imaging system is used to observe cells with different exogenously expressed fluorescent markers (Rab5a/EEA1/FYVE/Rab9a/Rab7/LAMP1), and the proteins and lipids on the DB membrane can be observed.
  • Qualitative analysis ( Figure 22 (a)- Figure 22 (g)).
  • a total of 61 ⁇ 3% of DBs are related to EE-labeled Rab5a-EGFP, and a large number of Rab5a-EGFP-labeled vesicles show RI values higher than DBs (66 ⁇ 4%, figure 22(h)).
  • the DBs co-localized with the LE/LY marker increased in size (average diameter 1.8-2.3 ⁇ m) downstream of the intracellular transport pathway.
  • Rab9a-EGFP labeled 60 ⁇ 6% of all liquefied vesicles it only accounted for 12 ⁇ 1% of all Rab9a-EGFP-labeled vesicles.
  • Approximately 31%-35% of vacuolar vesicles were labeled with Rab7-EGFP or LAMP1EGFP, and these co-localized vesicles represented a minority population (about 11%-14%) of Rab7-EGFP/LAMP1-EGFP vesicles.
  • DB may have the characteristics of LEs/LYs.
  • 61 ⁇ 3% of DBs overlapping with Rab5a-EGFP may correspond to a population similar to EE.
  • 82%-91% of vacuolar vesicles overlap with EEA1-EGFP and FYVEEGFP-labeled structures, indicating the enrichment of phosphatidylinositol 3-phosphate on DBs.
  • Figure 22(g) shows the co-localization between DBs and aquaporin, which can promote the transport of water on the plasma membrane and the endoplasmic membrane.
  • Figure 23 shows the low refractive index vesicles that appear in the optical diffraction tomography image when the dual-modality microscopic imaging system is imaging different types of cells. Among them, low refractive index vesicles appear in optical diffraction tomography images of different types of cells.
  • the second row in (a) corresponds to the moving image of low refractive index vesicles in the dotted frame area in the upper image. Scale bars: 5 ⁇ m (top), 1 ⁇ m (bottom).
  • the dual-modality microscopy imaging system can be used in human fibroblasts, human umbilical vein endothelial cells, rat insulinoma INS-1 cells, Such low refractive index vesicles were observed in mouse dorsal root ganglion nerve cells.
  • Figure 24 shows the optical diffraction tomography image of human bone marrow mesenchymal stem cells and the correlation between the number of low-refractive index vesicles and their cell senescence phenotype.
  • WT-hMSCs wild-type human bone marrow mesenchymal stem cells as a syngeneic control group
  • HGPS-hMSCs cells constitute the Hutchinson-Gilford Progeria Syndrome (HGPS)
  • HGPS Hutchinson-Gilford Progeria Syndrome
  • WRN-/- WS-hMSCs cells with WRN deficiency
  • (a-d) are optical diffraction tomography images of four kinds of cells.
  • (e) is the distribution density statistics of low refractive index vesicles in the middle layer of the cell. Scale bar: 5 ⁇ m on the left and 1 ⁇ m on the right. Mann-Whitney rank sum test: *p ⁇ 0.05, **p ⁇ 0.01, ***p ⁇ 0.001. It can be seen that low refractive index vesicles also exist in human bone marrow mesenchymal stem cells, and the number and shape of low refractive index vesicles in different types of cells are slightly different.
  • the low refractive index in bone marrow mesenchymal stem cells with progeria phenotype Compared with the control group, the number of vesicles increased significantly, indicating that its function may be related to maintaining the normal operation of cells.
  • the dual-modality microscopic imaging system of the embodiment of the present application can perform label-free rapid and long-term observation of the interaction process of organelles.
  • Organelles are the cell chambers that keep the local imprints of molecules and signals, and exchange information and materials with other organelles at the moment of formation of contact with the organelles, which are essential for many cell functions and behaviors.
  • both the endoplasmic reticulum and mitochondria are ancient eukaryotic cell inner membrane systems.
  • few studies have studied the interaction of mitochondria with different organelles. Because mitochondria are affected by phototoxicity during fluorescence imaging.
  • FIG. 25 shows the interaction between mitochondria and other organelles in COS-7 cells.
  • the nuclear membrane remains in contact with the mitochondria and interacts during the half-hour imaging;
  • the endoplasmic reticulum contacts the mitochondria and causes the mitochondria to be disconnected (b) or laterally expand (c);
  • the interaction between different modes of mitochondria and low refractive index vesicles, the contact causes the mitochondria to change shape (e) and move with the low refractive index vesicles (f) Or split (g).
  • This localized mitochondria may assist some important processes in the nucleus, such as the process of transporting mRNA to the outside of the nucleus. For another example, after lipid droplets contact with mitochondria, they are quickly pushed into mitochondria, and there is no obvious change in the morphology of the two. In contrast, contact between lysosomes and mitochondria will cause mitochondria to split ( Figure 25(d)) ).
  • the dual-mode microscopic imaging system of the embodiment of the present application can also observe the interaction process between low refractive index vesicles and mitochondria.
  • the shape of mitochondria can be observed to change ( Figure 25(e)) or split ( Figure 25(g)). Therefore, mitochondria do not form a continuous network that interacts with other organelles (such as ER) Instead, it uses a "one-to-one" contact method, through which mitochondria are tailored to interact with different organelles under various conditions. In some cases, low refractive index vesicles also drag mitochondria to move with them (Figure 25(f)).
  • the dual-modal imaging system of the embodiment of the present application can provide a complete map of organelle interaction, because the total number of organelles (such as mitochondria, LDs and LEs/LYs) that can be detected in 3D by the optical diffraction tomography subsystem 210 exceeds The total number that can be detected by a 2D microscope with only one Z plane.
  • organelles such as mitochondria, LDs and LEs/LYs
  • the green and orange lines in the top figures of (a) and (b) represent the illumination beam with the maximum NA of the illumination objective lens.
  • the spatial frequency domain bandwidth is expanded.
  • the top panel of (c) shows the horizontal spatial frequency bandwidth expansion, while the bottom panel of (c) shows the uneven distribution of the longitude frequency bandwidth.
  • det represent the lateral projection of the maximum NA of the illumination objective lens and the detection objective lens, respectively. It can be concluded from Figure 33 that although the structural dynamics inside different mitochondria can be solved by the two-dimensional Hessian structured light microscope system, this method provides information extracted from only one axial plane. In contrast, the ODT subsystem with label-free imaging capabilities provides a 3D map of bus mitochondria in cells, covering an area that is 3 times the largest mitochondrial area detectable in an axial volume of about 0.86 ⁇ m.
  • the dual-modality microscopic imaging system of the embodiment of the present application can also observe the interaction process between low refractive index vesicles and other organelles other than mitochondria.
  • Figure 26 shows the interaction of low refractive index vesicles between organelles in COS-7 cells.
  • the endoplasmic reticulum is usually a bridge between low refractive index vesicles and other organelles.
  • the low refractive index vesicles and lipid droplets contact the endoplasmic reticulum from both sides, and the three contact for more than 2 minutes ( Figure 26(c)).
  • the endoplasmic reticulum carries the low refractive index vesicles and contacts the mitochondria for more than 1 minute ( Figure 26). (d)).
  • low refractive index vesicles can also interact with multiple other organelles alone: as shown in Figure 26(e), low refractive index vesicles are in contact with the nuclear membrane at a certain depth while at another depth. Interacts with lipid droplets and mitochondria.
  • the above phenomena all illustrate the pivotal role of low refractive index vesicles and endoplasmic reticulum in cells, and the dual-modality microscopic imaging system of the embodiment of the present application provides strong support for observing the above new phenomena.
  • the dual-modality microscopic imaging system of the embodiment of the present application can also study the role of DBs in the interaction of tissue organelles.
  • Figure 27 shows the visualization of the DBs transport pathway and its role in the interaction of tissue organelles.
  • the left side shows a representative example of the conversion of DB to LE/LY in COS-7 cells, and the right side shows the corresponding intensity distribution at different time points approximated by a Gaussian function.
  • e A representative example of DB biosynthesis from a region close to the nuclear membrane (top), and then fusion of DB to the plasma membrane (bottom) after about 27 minutes. The clips of the DB at different time points are displayed on the left, and the corresponding intensity curves approximated by the Gaussian function at different time points are displayed on the right.
  • f-g Representative examples of DB-mitochondria (f, arrow indicates DB) or LD-mitochondria (g, arrow indicates LD) contact.
  • (m) A representative example of DB bridging different organelles, where the same DB interacts with the nuclear membrane in a Z plane, and at the same time interacts with LD and mitochondria outside 0.68 ⁇ m (lower plane).
  • (n) A clip of a representative example of a DB that interacts with mitochondria, LY and LD in sequence to form a DB-mitochondrial-LD-LY quaternary complex, and then dissociates from LD, mitochondria and LY in sequence.
  • a schematic diagram is shown below the optical diffraction tomography image. Scale bars, (a) 5 ⁇ m and (b-g, i, k-n) 1 ⁇ m.
  • a dual-modality microscopic imaging system can be used to perform long-term imaging of or cells to observe the characteristics of DBs.
  • large vacuoles may originate from micropinocytosis near the plasma membrane, most of the normal-sized vacuoles appear near the nuclear membrane ( Figure 27a and Figure 27b).
  • Figure 27c the nuclear membrane
  • Figure 27d the DBs (3 out of 26 from 2 cells) collapsed into the plasma membrane.
  • DBs often play a central role in the formation of multiple organelle complexes. For example, connecting DB and LD to different sides of the ER tubules formed a multi-organelle complex more than 2 minutes before separation ( Figure 27k).
  • the DB coated in the ER tubules can also attach to the mitochondria and stay on the mitochondria for at least one minute ( Figure 27i).
  • the DB itself can also connect multiple organelles at the same time.
  • DB is closely connected to the nuclear membrane, and the morphology of the two organelles changes over time; on another focal plane outside 0.68 ⁇ m, the same DB is simultaneously connected to one LD and one mitochondria on two different sides.
  • Interaction Figure 27m.
  • a DB is in contact with mitochondria (7'50”), LY (9'45”) and LD (9'50”) in sequence to form a multi-organelle complex, in LD (10'30”) , Mitochondria (11'00”) and LY (13'15”) lasted 40s before separation (Figure 27n).
  • the dual-modality microscopic imaging system of the embodiment of the present application can also be used to detect the result of co-localization between DBs and LC3-EGFP-labeled autophagosomes.
  • Figure 31 depicts the correlation between the LC3-EGFP marker structure and DBs in COS-7 cells.
  • (a-b) some LC3-EGFP-labeled structures are fluorescent rings co-localized with Lys
  • a) outer membrane or large vacuoles (b).
  • Most of the structures labeled with LC3-EGFP are fluorescent dots, which do not overlap with the transparent optical diffraction tomography structure (c), nor do they overlap with the LE/LY structure (d). All images shown in Figure 31 are representative of three batches of similar experiments.
  • Figure 32 is a histogram of the LE/LY structure size observed by the optical diffraction tomography subsystem in COS-7 cells overexpressing different protein markers. Among them, all distributions can be fitted by Gaussian functions. In LysoView 488 labeled cells, the average size of LE/LY structure is 1.77 ⁇ 0.01 ⁇ m, which is smaller than cells overexpressing LAMP1-EGFP (2.00 ⁇ 0.01 ⁇ m) and larger than cells overexpressing Rab7-EGFP (1.55 ⁇ 0.01 ⁇ m) And Rab9a-EGFP (1.67 ⁇ 0.01 ⁇ m) cells.
  • the movement of any structure in living cells may cause motion blur and resolution reduction, which is similar to SIM reconstruction. Similar.
  • the movement of LE/LY over a distance greater than the spatial resolution of the system may cause motion blur and reduced image contrast, which requires that the acquisition time for a reconstructed frame is less than 1.38s.
  • Higher spatial resolution or longer exposure time will cause the LE/LY signal to be distributed in a larger field of view, and the final structure will disappear in the background noise.
  • the spatial resolution must be matched with the corresponding temporal resolution to achieve the maximum resolution achieved in live-cell optical diffraction tomography, which was ignored in previous designs.
  • fast optical diffraction tomography microscopes must have sufficiently high sensitivity.
  • an sCMOS camera with large full well electrons and a mechanical galvanometer scanning mirror with smaller optical distortion than digital micromirror devices are also required.
  • the VISA algorithm is used to solve the problem of illumination angle jitter and ball misalignment during high-speed mechanical scanning in long-term live-cell optical diffraction tomography, and accurately determine the scanning wave vector of the illumination angle change to minimize the stitching error. These all contribute to the generation of sufficient photon flux during a short exposure period, and the optical diffraction tomography technology of the embodiment of the present application has excellent performance over the prior art.
  • optical diffraction tomography microscopes can be used to image cells, structures, and processes that are susceptible to phototoxicity (such as cell mitosis) ( Figure 15).
  • the phototoxicity caused by SR fluorescence imaging blocked COS-7 cells in the late stage ( Figure 34), which is consistent with the significant toxicity of H2EGFP in vivo imaging of H2EGFP during the embryonic development of Caenorhabditis elegans.
  • optical diffraction tomography microscopy can easily detect non-specific effects due to overexpression of foreign proteins.
  • Hesen SIM is also essential. With higher resolution and contrast, Hesen SIM provides finer details, including the interior of mitochondria and their dynamics in living cells.
  • the ability of super-resolution imaging is further enhanced to observe the organelles (such as Golgi and peroxisomes) that are not visible under the optical diffraction tomography microscope.
  • fluorescence super-resolution imaging can also highlight key proteins/lipids/molecules in the time and space of structural and dynamic changes.
  • SR-FACT can incorporate functional dynamics such as Ca 2+ , voltage, and cAMP into the cell landscape.
  • DBs may represent organelles that were previously undervalued.
  • DBs may share some endosomal markers with conventional endosomes, their vesicle cavities are pH-neutral and most of them are liquid-rich. Both of these characteristics are different from the inner body.
  • an hour-long high-resolution optical diffraction tomography revealed the biogenesis of DBs in areas rich in organelles and biomaterials around the cell nucleus, and then eventually collapsed into the plasma membrane, which is different from taking the opposite approach. The endosomal compartment of the endocytic transport route.
  • DBs may be important organizers of the organelle interaction group.
  • different organelles can interact with a DB in turn to form a multi-organelle complex, with DB as the cornerstone ( Figure 27(k)-(n)).
  • DBs may host the exchange of materials and information between different organelles, some of which may eventually be transmitted to the plasma membrane.
  • SR-FACT represents a tool that not only provides an overall view of three-dimensional organelle interactions in living cells, but also emphasizes the specific organelles/molecules/signaling pathways involved. Due to its dual-mode correlation SR imaging capability, SR-FACT can reveal phenomena that cannot be observed using one of the imaging methods alone, and often leads to unexpected observations during a well-researched process. It has minimal phototoxicity and no special requirements for labeling methods. It also represents a new generation of user-friendly super-resolution microscopes, which can generate countless structural and dynamic information, and help expand the cellular biology of living cells Understanding of the learning process.
  • cells can be cultured and prepared in the following manner.
  • COS-7 cells can be added with 10% fetal bovine serum (FBS) (GIBCO) and 1% 100mM sodium pyruvate solution (Sigma-Aldrich, S8636) in an incubator with a temperature of 37°C and a carbon dioxide concentration of 5%. It was cultured in high glucose DMEM (GIBCO, 21063029) until the degree of fusion reached about 75%.
  • FBS fetal bovine serum
  • S8636 100mM sodium pyruvate solution
  • HUVECs can be prepared by adding fiber growth cytokines, heparin and 20% FBS in an incubator with a temperature of 37°C and a carbon dioxide concentration of 5% until the fusion degree reaches about 75% in M199 medium (Thermo Fisher Scientific, 31100035) or Endothelial cell growth supplement (ECGS) and 10% FBS (GIBCO) are contained in an incubator with a temperature of 37°C and a carbon dioxide concentration of 5%, until the fusion degree reaches about 75% ECM medium is separated and cultured.
  • M199 medium Thermo Fisher Scientific, 31100035
  • ECGS Endothelial cell growth supplement
  • GEBCO 10% FBS
  • INS-1 cells can be added 10% EBS (GIBCO), 1% 100mM sodium pyruvate solution, 0.1% 55mM 2-mercaptoethanol (GIBCO, 21985023) in an incubator with a temperature of 37°C and a carbon dioxide concentration of 5%. Cultivate in RPMI1640 medium until the degree of confluence reaches 75%. Human fibroblasts can be cultured in an incubator with a temperature of 37°C and a carbon dioxide concentration of 5% with 20% FBS (GIBCO) added until the fusion degree reaches about 75% in high glucose DMEM (GIBCO, 21063029).
  • All hMSCs can be hMSCs containing 90% ⁇ -MEM+glutamine (Gibco), 10% FBS (cells, A77E01F), 1% penicillin/streptomycin (Gibco) and 1ng/mL FGF2 (Joint Protein Central) Cultivate in medium.
  • Dorsal root ganglion (DRG) neurons were isolated from P10 rats. The separated DRGs can be removed from the excess roots and digested with dispase II (Roche, 10888700)/type II collagenase (Worthington Biochemical, LS004176) at 37°C for 30 minutes, and then centrifuged again at room temperature for 35 minutes minute.
  • DRG neuron cell bodies were inoculated on 30mg/ml poly-L-ornithine-(Sigma, RNBG3346) and 5 ⁇ g/ml amine-(Roche, 11243217001) coated coverslips, and the temperature was 37°C, carbon dioxide concentration Add 2% B-27 supplement (GIBCO, A3582801), 2mM glutamine MAX (GIBCO, 35050061) and 1% penicillin/streptomycin (GIBCO, 15140122) to the 5% of the incubator. , 21103049). After 48 hours of culture in vitro, DRG neurons are ready for imaging. For SR-FACT imaging experiments, cells were seeded on coverslips (Thorlabs, CG15XH).
  • COS-7 cells in order to label mitochondria, COS-7 cells can use 250nM MitoTracker TM Green FM (Thermo Fisher Scientific ) in HBSS containing Ca 2+ and Mg 2+ but without phenol red (Thermo Fisher Scientific, 14025076). , M7514) incubate for 15 minutes, then wash and image.
  • COS7 cells can be incubated in 1 ⁇ LipidSpot TM 488 (Biotium, 70065-T) whole cell culture medium at 37°C in the dark for 30 min, then washed and imaged.
  • COS7 cells can be incubated in 1 ⁇ LysoView TM 488 (Biotium, 70067-T) whole cell culture medium at 37°C in the dark for 15-30 minutes without washing and imaging.
  • LifeAct-EGFP/KDEL-EGFP/LaminA-EGFP/H2BEGFP/LAMP1-EGFP/ ⁇ 1,4-galactosyltransferase 1 (B4GALT1)-EGFP/Pex11a-EGFP/LC3-EGFP/ Rab7-EGFP/Rab5a-EGFP/Rab9a-EGFP/FYVE-EGFP/EEA1-EGFP/AQP2-EGFP was transfected into COS-7 cells.
  • LipofectamineTM 2000 (Thermo Fisher Scientific, 11668019) was used for transfection according to the manufacturer's instructions. 24-36 hours after transfection, image the cells in a benchtop incubator.
  • the cover glass in order to clean the cover glass used for live cell imaging during the preparation of the cover glass, the cover glass may be immersed in a 10% powdered precision cleaner (Alconox, 1104-1), and the The coverslip was sonicated for 20 minutes. After rinsing in deionized water, the coverslip was sonicated in acetone for 15 minutes, and then again in 1M NaOH or KOH for 20 minutes. Finally, the coverslip can be rinsed with deionized water, and then subjected to 3 ultrasonic treatments, each for at least 5 minutes. The cleaned coverslips are stored in 95-100% ethanol at 4°C.
  • a 10% powdered precision cleaner Alconox, 1104-1
  • ImageJ can be used to analyze images during imaging data analysis and statistics.
  • DBs Figure 24
  • a threshold can be applied to each optical diffraction tomographic image stack for segmentation, and the density of DB in the Z plane of a single cell with a clearly visible nuclear cell membrane structure can be calculated.
  • the optical diffraction tomography data set and segmented LDs, LEs/LYs, and mitochondria can be manually annotated.
  • the area of LD, LEs, LYs and mitochondria in the axial volume of 0.86 ⁇ m can be calculated to match a Z plane of 2D SIM, And calculate their percentage relative to the total area of LDs, LYs and mitochondria in the whole cell.
  • MATLAB Mathworks
  • OriginPro OriginLab
  • Igor Pro Wavemetrics
  • Illustrator Adobe
  • the above-mentioned data (or image) analysis and statistical methods can be implemented by other related instructions or software modules.
  • the above-mentioned data (or image) analysis and statistical methods can be automatically implemented by invoking relevant computer instructions.
  • the computer instructions can be stored in a computer-readable storage medium. The instruction can make the dual-modality microscopic imaging system 100 automatically implement the above-mentioned method.
  • the possible beneficial effects of the embodiments of the present application include but are not limited to: (1)
  • the dual-mode microscopic imaging system can simultaneously perform label-free optical diffraction tomography with a field of view of 80 ⁇ m ⁇ 80 ⁇ m ⁇ 40 ⁇ m at a speed of at least 0.3HZ Three-dimensional imaging and super-resolution fluorescence two-dimensional imaging; (2)
  • the dual-modality microscopic imaging system fully overcomes the shortcomings of optical diffraction tomography and structured light illumination fluorescence imaging, and can perform comprehensive, rapid and long-term observation of living cells Cell metabolism research, and can observe and distinguish a variety of organelles; (3) Using a dual-mode microscopy system can also observe the existence of a new type of endosome, a low refractive index vesicle, to characterize its biochemical function and study its relationship with cells.
  • this application uses specific words to describe the embodiments of the application.
  • “one embodiment”, “an embodiment”, and/or “some embodiments” mean a certain feature, structure, or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that “one embodiment” or “one embodiment” or “an alternative embodiment” mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. .
  • some features, structures, or characteristics in one or more embodiments of the present application can be appropriately combined.
  • the computer storage medium may contain a propagated data signal containing a computer program code, for example on a baseband or as part of a carrier wave.
  • the propagated signal may have multiple manifestations, including electromagnetic forms, optical forms, etc., or a suitable combination.
  • the computer storage medium may be any computer readable medium other than the computer readable storage medium, and the medium may be connected to an instruction execution system, device, or device to realize communication, propagation, or transmission of the program for use.
  • the program code located on the computer storage medium can be transmitted through any suitable medium, including radio, cable, fiber optic cable, RF, or similar medium, or any combination of the above medium.
  • the computer program codes required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python Etc., conventional programming languages such as C language, VisualBasic, Fortran2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages.
  • the program code can be run entirely on the user's computer, or run as an independent software package on the user's computer, or partly run on the user's computer and partly run on a remote computer, or run entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (for example, via the Internet), or in a cloud computing environment, or as a service Use software as a service (SaaS).
  • LAN local area network
  • WAN wide area network
  • SaaS service Use software as a service
  • numbers describing the number of ingredients and attributes are used. It should be understood that such numbers used in the description of the embodiments use the modifier "about”, “approximately” or “substantially” in some examples. Retouch. Unless otherwise stated, “approximately”, “approximately” or “substantially” indicates that the number is allowed to vary by ⁇ 20%.
  • the numerical parameters used in the specification and claims are approximate values, and the approximate values can be changed according to the required characteristics of individual embodiments. In some embodiments, the numerical parameter should consider the prescribed effective digits and adopt the method of general digit retention. Although the numerical ranges and parameters used to confirm the breadth of the ranges in some embodiments of the present application are approximate values, in specific embodiments, the setting of such numerical values is as accurate as possible within the feasible range.

Landscapes

  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Multimedia (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Microscoopes, Condenser (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

Sont divulgués dans les modes de réalisation de la présente invention un système et un procédé d'imagerie microscopique à double modalité. Le système d'imagerie microscopique à double modalité comprend un sous-système de tomographie à diffraction optique et un sous-système d'imagerie par fluorescence à éclairage structuré. Le sous-système de tomographie à diffraction optique est utilisé pour réaliser une tomographie par diffraction optique non marquée sur la base d'un premier laser afin d'obtenir une image de tomographie par diffraction optique d'un échantillon. Le sous-système d'imagerie par fluorescence à éclairage structuré est utilisé pour réaliser une imagerie par fluorescence sur la base d'un second laser afin d'obtenir une image de fluorescence à éclairage structuré de l'échantillon. Le système d'imagerie microscopique à double modalité comprend une première source de lumière et une seconde source de lumière indépendantes l'une de l'autre. La première source de lumière est utilisée pour émettre le premier laser, et la seconde source de lumière est utilisée pour émettre le second laser.
PCT/CN2021/071393 2020-01-19 2021-01-13 Système et procédé d'imagerie microscopique à double modalité WO2021143707A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010059510.1 2020-01-19
CN202010059510.1A CN111610621B (zh) 2020-01-19 2020-01-19 一种双模态显微成像系统和方法

Publications (1)

Publication Number Publication Date
WO2021143707A1 true WO2021143707A1 (fr) 2021-07-22

Family

ID=72195888

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/071393 WO2021143707A1 (fr) 2020-01-19 2021-01-13 Système et procédé d'imagerie microscopique à double modalité

Country Status (2)

Country Link
CN (1) CN111610621B (fr)
WO (1) WO2021143707A1 (fr)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114324245A (zh) * 2021-11-15 2022-04-12 西安电子科技大学 基于部分相干结构光照明的定量相位显微装置和方法
CN114326075A (zh) * 2021-12-10 2022-04-12 肯维捷斯（武汉）科技有限公司 一种生物样品的数字显微成像系统及镜检方法
CN114813518A (zh) * 2022-02-17 2022-07-29 山东大学 一种基于单相机双模态成像的免标记流式检测装置及方法
CN115541578A (zh) * 2022-09-28 2022-12-30 佐健(上海)生物医疗科技有限公司 一种高通量超分辨宫颈细胞病理切片快速扫描分析系统
CN115901623A (zh) * 2022-11-03 2023-04-04 北京大学长三角光电科学研究院 一种高速光学衍射层析显微成像系统
CN117030674A (zh) * 2023-10-09 2023-11-10 中国科学院生物物理研究所 结构光照明三维超分辨率成像系统和方法
WO2024051079A1 (fr) * 2022-09-05 2024-03-14 中国科学院苏州生物医学工程技术研究所 Procédé et système d'imagerie microscopique à super-résolution basés sur un éclairage à structure active

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111610621B (zh) * 2020-01-19 2022-04-08 北京大学 一种双模态显微成像系统和方法
CN113009680B (zh) * 2021-03-11 2021-11-05 广东粤港澳大湾区协同创新研究院 一种用于超分辨成像的多通道成像系统和方法
CN113702288B (zh) * 2021-08-18 2022-07-01 北京大学 一种双模态显微成像系统及其成像方法
CN113777767B (zh) * 2021-09-14 2022-06-10 北京大学长三角光电科学研究院 一种快速连续旋转样品的光学层析显微成像系统及其方法
CN113779180A (zh) * 2021-09-29 2021-12-10 北京雅丁信息技术有限公司 一种地区drg分组模拟方法
CN113933277B (zh) * 2021-10-15 2023-08-22 深圳大学 一种高密度三维单分子定位超分辨显微成像系统及方法
CN113768472B (zh) * 2021-11-10 2022-03-22 华中科技大学 一种具有荧光标记的三维图像获取装置及方法
CN114190890A (zh) * 2021-11-26 2022-03-18 长沙海润生物技术有限公司 一种创面成像装置及其成像方法
CN114755200B (zh) * 2022-03-21 2022-11-08 北京大学长三角光电科学研究院 基于光动力治疗的可视化监测系统及方法
CN114820761B (zh) * 2022-05-07 2024-05-10 北京毅能博科技有限公司 基于图像显微扫描平台xy方向夹角测量与运动补偿方法
CN115015176B (zh) * 2022-05-18 2023-08-29 北京大学长三角光电科学研究院 一种光学衍射层析成像增强方法和装置
CN114668582B (zh) * 2022-05-30 2022-08-19 季华实验室 一种眼科光源手术系统
CN115980989A (zh) * 2023-01-09 2023-04-18 南开大学 基于微透镜阵列的单帧定量相位层析成像系统及方法
CN117268288B (zh) * 2023-07-26 2024-03-08 北京大学长三角光电科学研究院 光学衍射层析成像激光扫描方法、装置和电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103948399A (zh) * 2013-07-25 2014-07-30 合肥工业大学 扇形扫描方式下基于非衍射波的三维超声成像方法
CN205750291U (zh) * 2016-04-13 2016-11-30 苏州大学 一种基于空间光调制器的全息三维显示装置
CN111060485A (zh) * 2019-12-31 2020-04-24 苏州栢科昇科技有限公司 微生物多模态成像系统和微生物多模态成像检测方法
CN111610621A (zh) * 2020-01-19 2020-09-01 北京大学 一种双模态显微成像系统和方法

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010062364A1 (fr) * 2008-10-31 2010-06-03 University Of Maine System Board Of Trustees Imagerie à échelle nanoscopique de positions et d'anisotropies moléculaires
US8937722B2 (en) * 2010-03-28 2015-01-20 Ecole Polytechnique Federale De Lausanne (Epfl) Complex index refraction tomography with sub λ/6-resolution
CN101984928B (zh) * 2010-09-29 2012-06-13 北京大学 一种多模态分子层析成像系统
US10799111B2 (en) * 2014-06-10 2020-10-13 Carl Zeiss Meditec, Inc. Frequency-domain interferometric based imaging systems and methods
CN106056613B (zh) * 2016-06-02 2019-07-19 南方医科大学 一种基于像素分类和局部曲面拟合的磁共振相位解缠绕方法
KR101865624B1 (ko) * 2016-06-10 2018-06-11 주식회사 토모큐브 파면 제어기를 이용한 초고속 3차원 굴절률 영상 촬영 및 형광 구조화 조도 현미경 시스템 및 이를 이용한 방법
KR102426103B1 (ko) * 2016-07-22 2022-07-28 주식회사 내일해 개선된 홀로그래픽 복원 장치 및 방법
CN106227016B (zh) * 2016-07-28 2019-03-12 东南大学 一种非迭代复振幅调制全息投影方法
US10131133B2 (en) * 2016-10-17 2018-11-20 Purdue Research Foundation Methods for forming optically heterogeneous phantom structures and phantom structures formed thereby
US20180164562A1 (en) * 2016-10-25 2018-06-14 Stereo Display, Inc. Confocal microscopy system with vari-focus optical element
CN108169173B (zh) * 2017-12-29 2020-08-11 南京理工大学 一种大视场高分辨三维衍射层析显微成像方法
CN108665411B (zh) * 2018-03-09 2022-05-24 广州超视计生物科技有限公司 一种图像重建的方法和系统
CN208399380U (zh) * 2018-06-07 2019-01-18 中国科学院苏州生物医学工程技术研究所 一种结构光照明超分辨显微成像系统
CN209946009U (zh) * 2019-03-22 2020-01-14 中国科学院苏州生物医学工程技术研究所 光学相干层析和双光子荧光同步成像系统
CN110658195B (zh) * 2019-10-25 2020-10-30 浙江大学 一种移频无标记超分辨显微芯片及其成像方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103948399A (zh) * 2013-07-25 2014-07-30 合肥工业大学 扇形扫描方式下基于非衍射波的三维超声成像方法
CN205750291U (zh) * 2016-04-13 2016-11-30 苏州大学 一种基于空间光调制器的全息三维显示装置
CN111060485A (zh) * 2019-12-31 2020-04-24 苏州栢科昇科技有限公司 微生物多模态成像系统和微生物多模态成像检测方法
CN111610621A (zh) * 2020-01-19 2020-09-01 北京大学 一种双模态显微成像系统和方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HUANG XIAOSHUAI;LI LIUJU;FAN JUANCHAO;LIU YANMEI;TAN SHAN;CHEN LIANGYI: "Ultra-Sensitive Super Resolution Imaging with Hessian Structured Illumination Microscopy in Live Cells", BULLETIN OF NATIONAL NATURAL SCIENCE FOUNDATION OF CHINA, vol. 32, no. 4, 15 July 2018 (2018-07-15), pages 367 - 375, XP055829796, ISSN: 1000-8217, DOI: 10.16262/j.cnki.1000-8217.2018.04.005 *
SHIN SEUNGWOO, KIM DOYEON, KIM KYOOHYUN, PARK YONGKEUN: "Super-resolution three-dimensional fluorescence and optical diffraction tomography of live cells using structured illumination generated by a digital micromirror device", SCIENTIFIC REPORTS, vol. 8, no. 1, 1 December 2018 (2018-12-01), pages 1 - 8, XP055829791, ISSN: 2045-2322, DOI: 10.1038/s41598-018-27399-w *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114324245A (zh) * 2021-11-15 2022-04-12 西安电子科技大学 基于部分相干结构光照明的定量相位显微装置和方法
CN114324245B (zh) * 2021-11-15 2024-01-30 西安电子科技大学 基于部分相干结构光照明的定量相位显微装置和方法
CN114326075A (zh) * 2021-12-10 2022-04-12 肯维捷斯（武汉）科技有限公司 一种生物样品的数字显微成像系统及镜检方法
CN114326075B (zh) * 2021-12-10 2023-12-19 肯维捷斯（武汉）科技有限公司 一种生物样品的数字显微成像系统及镜检方法
CN114813518A (zh) * 2022-02-17 2022-07-29 山东大学 一种基于单相机双模态成像的免标记流式检测装置及方法
CN114813518B (zh) * 2022-02-17 2024-06-11 山东大学 一种基于单相机双模态成像的免标记流式检测装置及方法
WO2024051079A1 (fr) * 2022-09-05 2024-03-14 中国科学院苏州生物医学工程技术研究所 Procédé et système d'imagerie microscopique à super-résolution basés sur un éclairage à structure active
CN115541578A (zh) * 2022-09-28 2022-12-30 佐健(上海)生物医疗科技有限公司 一种高通量超分辨宫颈细胞病理切片快速扫描分析系统
CN115541578B (zh) * 2022-09-28 2023-10-24 佐健(上海)生物医疗科技有限公司 一种高通量超分辨宫颈细胞病理切片快速扫描分析系统
CN115901623A (zh) * 2022-11-03 2023-04-04 北京大学长三角光电科学研究院 一种高速光学衍射层析显微成像系统
CN115901623B (zh) * 2022-11-03 2023-07-11 北京大学长三角光电科学研究院 一种高速光学衍射层析显微成像系统
CN117030674A (zh) * 2023-10-09 2023-11-10 中国科学院生物物理研究所 结构光照明三维超分辨率成像系统和方法

Also Published As

Publication number Publication date
CN111610621B (zh) 2022-04-08
CN111610621A (zh) 2020-09-01

Similar Documents

Publication Publication Date Title
WO2021143707A1 (fr) Système et procédé d'imagerie microscopique à double modalité
Dong et al. Super-resolution fluorescence-assisted diffraction computational tomography reveals the three-dimensional landscape of the cellular organelle interactome
Descloux et al. Combined multi-plane phase retrieval and super-resolution optical fluctuation imaging for 4D cell microscopy
Bon et al. Self-interference 3D super-resolution microscopy for deep tissue investigations
Zhao et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy
Kim et al. Correlative three-dimensional fluorescence and refractive index tomography: bridging the gap between molecular specificity and quantitative bioimaging
Boden et al. Volumetric live cell imaging with three-dimensional parallelized RESOLFT microscopy
Zhanghao et al. High-dimensional super-resolution imaging reveals heterogeneity and dynamics of subcellular lipid membranes
US8184298B2 (en) Spatial light interference microscopy and fourier transform light scattering for cell and tissue characterization
Zhao et al. Isotropic super-resolution light-sheet microscopy of dynamic intracellular structures at subsecond timescales
Chen et al. Spatial light interference microscopy: principle and applications to biomedicine
Lue et al. Live cell refractometry using Hilbert phase microscopy and confocal reflectance microscopy
Fang et al. Confocal light absorption and scattering spectroscopic microscopy
Hedde et al. Phasor-based hyperspectral snapshot microscopy allows fast imaging of live, three-dimensional tissues for biomedical applications
Holanová et al. Optical imaging and localization of prospective scattering labels smaller than a single protein
Jin et al. High-resolution 3D reconstruction of microtubule structures by quantitative multi-angle total internal reflection fluorescence microscopy
Morishita et al. Label-free intratissue activity imaging of alveolar organoids with dynamic optical coherence tomography
Küppers et al. Confocal interferometric scattering microscopy reveals 3D nanoscopic structure and dynamics in live cells
Lévesque et al. Sample and substrate preparation for exploring living neurons in culture with quantitative-phase imaging
Hsiao et al. Spinning disk interferometric scattering confocal microscopy captures millisecond timescale dynamics of living cells
Mirsky et al. Dynamic tomographic phase microscopy by double six-pack holography
Ma et al. Label-free imaging of intracellular organelle dynamics using flat-fielding quantitative phase contrast microscopy (FF-QPCM)
Pant et al. Line-scan focal modulation microscopy
Yang et al. Pulsed stimulated Brillouin microscopy enables high-sensitivity mechanical imaging of live and fragile biological specimens
Kang et al. Mapping nanoscale topographic features in thick tissues with speckle diffraction tomography

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21741582

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21741582

Country of ref document: EP

Kind code of ref document: A1