WO2024051079A1 - 一种主动结构光照明的超分辨显微成像方法及系统 - Google Patents
一种主动结构光照明的超分辨显微成像方法及系统 Download PDFInfo
- Publication number
- WO2024051079A1 WO2024051079A1 PCT/CN2023/075140 CN2023075140W WO2024051079A1 WO 2024051079 A1 WO2024051079 A1 WO 2024051079A1 CN 2023075140 W CN2023075140 W CN 2023075140W WO 2024051079 A1 WO2024051079 A1 WO 2024051079A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- illumination
- light
- spatial
- light field
- active
- Prior art date
Links
- 238000005286 illumination Methods 0.000 title claims abstract description 190
- 238000003384 imaging method Methods 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 claims abstract description 27
- 238000009826 distribution Methods 0.000 claims abstract description 24
- 230000005284 excitation Effects 0.000 claims abstract description 11
- 241001270131 Agaricus moelleri Species 0.000 claims abstract description 9
- 238000004445 quantitative analysis Methods 0.000 claims abstract description 7
- 239000011521 glass Substances 0.000 claims description 20
- 238000010869 super-resolution microscopy Methods 0.000 claims description 16
- 238000001917 fluorescence detection Methods 0.000 claims description 15
- 238000002073 fluorescence micrograph Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 230000010287 polarization Effects 0.000 claims description 9
- 238000006073 displacement reaction Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 5
- 239000000835 fiber Substances 0.000 claims description 5
- 239000013307 optical fiber Substances 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 230000003993 interaction Effects 0.000 abstract description 4
- 238000011084 recovery Methods 0.000 abstract description 2
- 230000000638 stimulation Effects 0.000 abstract description 2
- 230000001413 cellular effect Effects 0.000 abstract 1
- 239000000523 sample Substances 0.000 description 28
- 210000004027 cell Anatomy 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 210000003463 organelle Anatomy 0.000 description 4
- 239000005262 ferroelectric liquid crystals (FLCs) Substances 0.000 description 3
- 238000000799 fluorescence microscopy Methods 0.000 description 3
- 230000008832 photodamage Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 206010034972 Photosensitivity reaction Diseases 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 239000007850 fluorescent dye Substances 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000000386 microscopy Methods 0.000 description 2
- 208000007578 phototoxic dermatitis Diseases 0.000 description 2
- 231100000018 phototoxicity Toxicity 0.000 description 2
- 241000132092 Aster Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004061 bleaching Methods 0.000 description 1
- 230000003915 cell function Effects 0.000 description 1
- 238000000701 chemical imaging Methods 0.000 description 1
- 239000003638 chemical reducing agent Substances 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010859 live-cell imaging Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000012528 membrane Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/6486—Measuring fluorescence of biological material, e.g. DNA, RNA, cells
Definitions
- the invention belongs to the field of fluorescence microscopy imaging technology, and is particularly a super-resolution microscopy imaging system combining active illumination and structured light illumination.
- the cell is a highly complex dynamic system in which dozens of organelles with different functions are formed through compartmentalization of membrane structures.
- the interaction of these organelles is key to maintaining cell function and determining cell fate.
- the study of these organelles and their interactions requires live cell imaging with high resolution below a hundred nanometers, extremely low light dose, high dynamic range, and large field of view with uniform illumination.
- the beam shaper directly converts the Gaussian propagating beam into a flat-top beam, using a pair of aspherical lens groups.
- the first lens redistributes the Gaussian beam uniformly, and the second lens redistributes it. collimated, resulting in flat field illumination.
- the working distance based on the refractive beam shaper is limited, and there are strict requirements for surface processing quality and optical arrangement.
- Optical waveguides can provide uniform illumination over a very large field of view, but because they operate in total internal reflection mode, the illumination is limited to the vicinity of the coverslip, making it impossible to switch the illumination angle to achieve different depth illumination.
- ASTER is essentially a hybrid scanning wide-field illumination device that scans a Gaussian beam according to a specific pattern and provides flat-top illumination in a time-averaged manner, so that a uniform illumination field can be scanned over a relatively wide field of view.
- the above-mentioned flat-field illumination method can only be applied to super-resolution microscopes based on single-molecule positioning, such as PALM and STORM. Its transformation to structured light illumination microscopy (SIM) based on multi-beam interference to produce cosine distribution fringes is still limited. , the reason is that when modulating the incident non-uniform Gaussian light spot, it is necessary to ensure that the frequency, constant phase difference, and polarization direction of the multiple beams are the same, so that interference on the sample surface forms illumination stripes with a high modulation degree.
- SIM structured light illumination microscopy
- the resolution of confocal, two-photon low-light-dose, high-dynamic imaging methods is limited by the optical diffraction limit.
- the lateral limit resolution is about 200nm and the axial resolution is 500nm.
- photobleaching and phototoxicity are serious and the imaging speed is slow, which cannot meet the needs of dynamic imaging of living cells.
- SIM has the characteristics of fast imaging speed, small light damage, and no special requirements for fluorescent probes. Its resolution of 100 nanometers just meets the observation requirements of important organelles in living cells, and has become the primary method for dynamic imaging of living cells. choose.
- SIM is essentially a wide-field fluorescence microscopy imaging technology, and its overall illumination intensity presents a Gaussian shape distribution on the sample. This non-uniform illumination method first reduces the available field of view size. In order to obtain a relatively uniform fluorescence image, the center area of the Gaussian spot is usually intercepted for imaging, thereby losing the edge field of view. Secondly, Gaussian illumination with a bright center and dark edges causes inconsistent photobleaching characteristics within the imaging field of view.
- the center is more easily bleached than the edges, which limits its application in quantitative analysis based on fluorescence intensity.
- this indiscriminate illumination method that does not consider the sample label density will cause underexposure of weak signals, overexposure of strong signals, and invalid exposure of background and out-of-focus signals, limiting the dynamic range and performance of the SIM imaging system. Cause unnecessary light damage.
- an active structured light illumination super-resolution imaging method (Active-SIM) based on spatial and temporal joint modulation of a spatial light modulator was invented, such as modulating the Gaussian spot into flat-field illumination, adaptively adjusting the illumination intensity according to the sample label density, and Illumination of the area of interest, etc., which is crucial to expand the application of SIM imaging technology in dynamic long-term imaging of living cells.
- Active-SIM active structured light illumination super-resolution imaging method
- the purpose of this invention is to study suborganelles and their interactions in living cells that require fluorescence imaging with high resolution below a hundred nanometers, extremely low light dose, high dynamic range, and large field of view with uniform illumination, and proposes a method based on The spatiotemporal joint intensity modulation method of spatial light modulator establishes a super-resolution microscopy imaging system based on active structured light illumination.
- the technical solution to achieve the purpose of the present invention is: a super-resolution microscopy imaging method with active structured light illumination, The method includes the following steps:
- Step 1 collect the fluorescence image of the sample after laser excitation through the camera
- Step 2 Construct the required active structured light illumination light field based on the image in step 1;
- Step 3 Convert the active structured light illumination light field in Step 2 and load it into the spatial light modulator to form an active structured light illumination light field with joint spatiotemporal intensity modulation;
- Step 4 Take out the corresponding bitmap from the holographic stripe bitmap loaded in the spatial light modulator in step 3, and start displaying it.
- the laser emits light simultaneously to illuminate the sample, and the camera synchronizes exposure to collect original image data;
- Step 5 Determine whether all holographic stripe plane images are displayed. If so, the spatial light modulator stops plane display and the camera stops image acquisition. Otherwise, switch to plane images with different spatial direction angles and phases in a preset sequence. Return to step 4 and continue to perform synchronized display and exposure.
- the active structured light illumination light field described in step 2 includes:
- the first illumination light field modulates the input non-uniform Gaussian distribution light field into a uniform flat-top illumination light field to meet the needs of quantitative analysis of fluorescence intensity and large field of view splicing imaging;
- the second illumination light field actively adjust the illumination light intensity according to the intensity distribution characteristics of the sample, including reducing the illumination dose in strong signal areas, increasing the illumination dose in weak signal areas, and no signal or background areas without illumination, to meet the required high dynamic range, low Imaging requirements for light dose;
- the third illumination light field customize the illumination at the specified position and intensity to meet the imaging needs of the user-defined illumination area of interest.
- I g (x, y, t) represents the space-time volume of the illumination intensity of the excitation light on the sample surface within a camera exposure period T.
- T the illumination intensity presents a non-uniform Gaussian distribution in space
- M i (x, y, t) represents the i-th intensity modulation function or binary bit plane loaded in the spatial light modulator
- N represents the binary bit plane loaded.
- the light intensity modulation accuracy of each pixel on the spatial light modulator can be expressed as 1/2 N ;
- t i represents the time weight function corresponding to each binary plane, and the sum of all time weights is one exposure of the camera cycle
- Point (x, y) represents any spatial coordinate in the spatial light modulator;
- m represents the modulation degree of the structured light,
- k x , k y , ⁇ represent the spatial frequency and initial phase.
- a super-resolution microscopy imaging system for active structured light illumination that implements the above method, the system includes a light source module, an active structured light illumination light field loading module, an active structured light illumination light field generation module, and a fluorescence detection module;
- the light source module is used to control at least four laser wavelengths, and can realize independent illumination of a single wavelength, multiple wavelengths, multi-color time sharing and simultaneous illumination;
- the fluorescence detection module is used to collect fluorescence images of samples excited by laser;
- the active structured light illumination light field generation module is used to generate an active structured light illumination light field based on the fluorescence image
- the active structured light illumination light field loading module is used to load the generated active structured light illumination light field into the SIM imaging system to complete active structured light illumination super-resolution imaging.
- the light source module includes a laser and a single-mode polarization-maintaining optical fiber
- the laser is used to provide lasers of multiple wavelengths, and can realize single-wavelength and multi-wavelength synchronous light emission control;
- the single-mode polarization-maintaining fiber is used to transmit multiple lasers coupled by the laser to the active structured light illumination light field loading module.
- the active structured light illumination light field loading module includes an achromatic collimated beam expander objective, a polarizing beam splitter prism, a spatial light modulator, a Fourier lens, a half glass slide, a spatial filter, and a combined half A glass plate, collimating lens, illumination tube lens; Fourier lens and collimating lens form a 4f system.
- the spatial light modulator is located at the front focal plane of the 4f system, and the spatial filter is located at the back focal plane of the 4f system.
- Spatial filtering only allows light beams of a specific angle diffracted by the spatial light modulator to pass through, blocking other diffraction orders; a half glass plate and a combined half glass plate constitute a polarization control component, which is used to control different direction angles The incident polarized light is controlled;
- the laser light emitted from the light source module is transmitted to the achromatic collimated beam expander objective lens. After collimation and beam expansion, it is incident on the polarizing beam splitting prism.
- the transmitted vertically polarized light is incident on the spatial light modulator. After being modulated by the spatial light modulator, it is mainly diffracted.
- the three outgoing beams are reflected by the polarizing beam splitter and then focused on the spatial filter through the Fourier lens and the half glass. After filtering, they are gated to the level ⁇ 1.
- the light beam is sequentially combined with a half glass slide, a collimating lens, and an illuminating tube lens and then enters the fluorescence detection module.
- the fluorescence detection module includes a three-dimensional displacement stage, a microscope objective, a dichroic mirror, an imaging lens and a camera;
- the excitation light output by the active structured light illumination light field loading module is reflected by the dichroic mirror and irradiated through the microscope objective lens to the sample placed on the three-dimensional displacement stage. After the fluorescence signal of the sample is excited, it is collected by the microscope objective lens and then passes through the two-dimensional microscope. After being transmitted through the color mirror, it is focused on the detection surface of the camera through the imaging lens.
- the active structured light illumination light field generation module includes an image acquisition unit, an image processing unit and light field generation unit;
- the image acquisition unit is used to acquire the illumination light field distribution characteristics of the sample surface detected by the camera;
- the image processing unit is used to convert the illumination light field obtained by the image acquisition unit into the active structured light illumination light field required by the user;
- the light field generation unit is used to convert the active structured light illumination light field into a set of binary bitmaps carrying different time weights based on joint spatiotemporal intensity modulation that can be loaded by the spatial light modulator, and load it into the active structured light illumination light field. field loading module in the spatial light modulator.
- the present invention does not need to add additional optical hardware. It only needs to build the required light field based on the spatio-temporal joint intensity modulation method proposed by the present invention. It can obtain rich imaging functions without increasing economic costs. Simplify system complexity.
- the present invention can increase the imaging dynamic range of traditional SIM by 40dB and reduce the light dose by 10 times without losing the spatial and temporal resolution of SIM.
- Figure 1 is a flow chart of the light field modulation method for active structured light illumination.
- Figure 2 is a schematic diagram of the active structured light illumination super-resolution microscopy imaging system.
- Figure 3 is a schematic diagram of the active structured light illumination light field modulation method.
- Figure 3(a) is a schematic diagram of pixel-level arbitrary light intensity modulation based on a ferroelectric liquid crystal spatial light modulator.
- Figure 3(b) is a schematic diagram of the spatio-temporal joint intensity modulation principle.
- the reference numbers in the figure are as follows: 1. Four-way combined beam laser; 2. Single-mode polarization-maintaining fiber; 3. Achromatic collimated beam expansion objective; 4. Polarizing beam splitter; 5. Spatial light modulator; 6. Fourier lens; 7. 1 slide; 8 spatial filters; 9 combined half glass slide; 10 collimating lens; 11 illumination tube lens; 12 three-dimensional displacement stage; 13 objective lens; 14 dichroic mirror; 15 imaging tube lens; 16 camera; 17 image acquisition unit; 18 image processing unit ; 19 light field generation unit.
- a super-resolution microscopy imaging method with active structured light illumination includes the following steps:
- Step 1 Use a camera to collect the fluorescence image of the sample excited by the laser;
- Step 2 Construct the required active structured light illumination light field based on the image in step 1;
- Step 3 Convert the active structured light illumination light field in Step 2 and load it into the spatial light modulator to form an active structured light illumination light field with joint spatiotemporal intensity modulation;
- Step 4 Take out the corresponding bitmap from the holographic stripe bitmap loaded in the spatial light modulator in step 3, and start displaying it.
- the laser emits light simultaneously to illuminate the sample, and the camera synchronizes exposure to collect original image data;
- Step 5 Determine whether all holographic stripe plane images are displayed. If so, the spatial light modulator stops plane display and the camera stops image acquisition. Otherwise, switch to plane images with different spatial direction angles and phases in a preset sequence. Return to step 4 and continue to perform synchronized display and exposure.
- the active structured light illumination light field in step 2 includes:
- the first illumination light field modulates the input non-uniform Gaussian distribution light field into a uniform flat-top illumination light field to meet the needs of quantitative analysis of fluorescence intensity and large field of view splicing imaging;
- the second illumination light field actively adjust the illumination light intensity according to the intensity distribution characteristics of the sample, including reducing the illumination dose in strong signal areas, increasing the illumination dose in weak signal areas, and no signal or background areas without illumination, to meet the required high dynamic range, low Imaging requirements for light dose;
- the third illumination light field customize the illumination at the specified position and intensity to meet the imaging needs of the user-defined illumination area of interest.
- the present invention utilizes ferroelectric liquid crystal-based spatial light modulation in which each independent pixel can be turned on and off.
- the fast switching feature enables single-pixel level control of illumination intensity within a camera integration period, as shown in Figure 3a.
- the illumination intensity on the sample surface presents a Gaussian distribution as a whole, which can be expressed as:
- the active lighting light field can be expressed as:
- the illumination intensity distribution of the modulated active structured light field in step 3 is expressed as I a (x, y):
- I 0 represents the peak intensity
- ⁇ is the beam waist radius of the Gaussian beam
- I g (x, y, t) represents the space-time volume of the illumination intensity of the excitation light on the sample surface within a camera exposure period T, in During the SIM imaging process when only one level of light beam is allowed to pass through, the illumination intensity presents a non-uniform Gaussian distribution in space
- M i (x, y, t) represents the i-th intensity modulation loaded in the spatial light modulator.
- N represents the number of loaded binary bit planes
- the light intensity modulation accuracy of each pixel on the spatial light modulator can be expressed as 1/2 N
- t i represents the corresponding The time weight function, the sum of all time weights is an exposure period of the camera
- Point (x, y) represents any spatial coordinate in the spatial light modulator
- m represents the modulation degree of the structured light
- k x , k y , ⁇ represent the spatial frequency and initial phase.
- an active structured light illumination super-resolution microscopy imaging system that implements the above method is shown in Figure 2.
- the system includes a light source module, an active structured light illumination light field loading module, an active structured light illumination light field generation module, and Fluorescence detection module;
- the light source module is used to control at least four laser wavelengths, and can realize independent illumination of a single wavelength, multiple wavelengths, multi-color time sharing and simultaneous illumination;
- the fluorescence detection module is used to collect fluorescence images of samples excited by laser;
- the active structured light illumination light field generation module is used to generate an active structured light illumination light field based on the fluorescence image
- the active structured light illumination light field loading module is used to load the generated active structured light illumination light field into the SIM imaging system to complete active structured light illumination super-resolution imaging.
- the light source module includes a laser 1 and a single-mode polarization-maintaining fiber 2;
- the laser 1 is used to provide lasers of multiple wavelengths, and can realize single-wavelength and multi-wavelength synchronous light emission control;
- the single-mode polarization-maintaining fiber 2 is used to transmit multiple lasers coupled by the laser 1 to the active structured light illumination light field loading module.
- the laser 1 provides four wavelengths of laser light, the four wavelengths being 405nm, 488nm, 561nm, and 637nm respectively.
- the active structured light illumination light field loading module includes an achromatic collimated beam expander objective 3, a polarizing beam splitter prism 4, a spatial light modulator 5, a Fourier lens 6, a half Glass slide 7, spatial filter 8, combined half glass slide 9, collimating lens 10, illumination tube lens 11; Fourier lens 6 and collimating lens 10 form a 4f system, and spatial light modulator 5 is located in the 4f system
- the front focal plane, the spatial filter 8 is located at the back focal plane of the 4f system, used for spatial filtering, only allowing the beam of a specific angle diffracted by the spatial light modulator 5 to pass, blocking other diffraction orders;
- One glass plate 7 and a combined half glass plate 9 constitute a polarization control component, which is used to control incident polarized light at different direction angles;
- the laser light emitted from the light source module is transmitted to the achromatic collimated beam expander objective lens 3. After collimation and beam expansion, it is incident on the polarizing beam splitter prism 4. The transmitted vertically polarized light is incident on the spatial light modulator 5 and modulated by the spatial light modulator. Finally, three outgoing beams of order 0 and ⁇ 1 are diffracted. The three outgoing beams are reflected by the polarizing beam splitter 4 and then focused on the spatial filter 8 through the Fourier lens 6 and the half glass plate 7. After filtering, The gated ⁇ 1-level exit beam sequentially passes through the combination of the half glass slide 9, the collimating lens 10, and the illumination tube lens 11 and then enters the fluorescence detection module.
- the fluorescence detection module includes a three-dimensional displacement stage 12, a microscope objective 13, a dichroic mirror 14, an imaging lens 15 and a camera 16;
- the active structured light field output by the active structured light illumination light field loading module is reflected by the dichroic mirror 14 and then illuminated by the microscope objective lens 13 to form the sample illumination tube lens 11 and the microscope objective lens 13 placed on the three-dimensional displacement stage 12
- Another pair of 4f systems is used to conjugate the structured light illumination light field to the sample surface, excite the sample fluorescence signal and collect it by the microscope objective lens 13. It is then transmitted by the dichroic mirror 14 and then focused to the camera 16 through the imaging lens 15. on the detection surface.
- the active structured light illumination light field generation module includes an image acquisition unit 17, an image processing unit 18 and a light field generation unit 19;
- the image acquisition unit 17 is used to acquire the illumination light field distribution characteristics of the sample surface detected by the camera 16;
- the image processing unit 18 is used to convert the illumination light field obtained by the image acquisition unit 17 into the active structured light illumination light field required by the user;
- the light field generation unit 19 is used to convert the active structured light illumination light field into a set of binary bitmaps carrying different time weights based on joint spatiotemporal intensity modulation that can be loaded by the spatial light modulator, and load it into the active structured lighting
- the bright light field is loaded into the spatial light modulator in the module.
- the spatial coordinates of the spatial light modulator and camera need to be calibrated to match the spatial coordinates of the input, output and modulated light fields, and generate an active illumination light field with pixel-level matching accuracy.
- the specific process is as follows: 1) Load "cross" intersection points composed of equally spaced distributions in the spatial light modulator; 2) The fluorescence detection module collects uniformly distributed fluorescent dye signal images; 3) Based on the images loaded by the spatial light modulator and Calculate the calibration matrix (including zoom ratio, rotation, and translation) from the images collected by the camera.
- the workflow of the system is as follows:
- One of the four lasers of 405nm, 488nm, 561nm or 673nm is gated through the laser module 1 and enters the polarization-maintaining single-mode optical fiber 2.
- the excitation light passes through the single-mode beam and transmits the beam to the achromatic collimating objective lens 3 of the light field loading module.
- the collimated and expanded beam is illuminated by the polarizing beam splitter prism 4, and the transmitted vertically polarized light (s-polarization) is illuminated by the spatial light modulator 5.
- the spatial light modulator After the beam is modulated by the spatial light modulator, it mainly diffracts into 0-order, ⁇ 1 An outgoing beam, the reflected outgoing beam passes through the polarizing beam splitter prism 4 again, and only the modulated horizontally polarized light (p polarization) can enter the imaging system; as shown in Figure 3a of the light intensity modulation principle, loaded in the spatial light modulator
- the time-weighted bit plane can achieve single-pixel light intensity control by modulating the "pixel on” and "pixel off” of the incident polarized light, such as modulating the non-uniform Gaussian distribution excitation light into a uniformly distributed flat-top beam (Fig. 3b).
- the three diffracted beams of order 0 and ⁇ 1 that have been modulated by the single pixel intensity are incident on the Fourier lens 6. After passing through the half glass plate 7 for polarization compensation, the modulated polarized beam is finally focused on the spatial filter 8. At the position of The polarization direction of the incident light is thus obtained to obtain interference fringes with the maximum modulation degree on the sample surface; after the ⁇ 1-order beam passes through the collimating lens 10, it interferes at its rear focal plane; further, the beam enters the fluorescence detection module after passing through the illumination tube lens.
- the illumination beam enters the microscope objective lens 13 after being reflected by the dichroic mirror 14 , and finally excites the fluorescent sample clamped at the displacement stage 12 .
- the fluorescent sample illuminated by actively illuminated structured light emits fluorescence with a longer wavelength than the excitation light wavelength, which is collected by the high numerical aperture objective lens 13 and enters the fluorescence detection module.
- the collected fluorescence signal is transmitted through the dichroic mirror 14 and then enters the imaging Lens 15 is finally detected by camera 16.
- the image acquisition unit of the active structured light illumination light field generation module collects the images collected by the camera 16 and sends them to the image processing unit 18.
- the image processing unit calculates the collected fluorescence images and the user's preset light field, and then generates the user's preset light field.
- the modulated light field required by the light field is further sent to the light field generation unit 19 to generate a binary plane holographic fringe pattern carrying different time weights that can be loaded by the spatial light modulator 5 .
- the light field generation module loads the generated binary bitmap reconstruction into the spatial light modulation 5 in the light field loading module, irradiates the sample through devices 6 to 11, 14, 13, and 12, and the emitted fluorescence is detected by fluorescence Modules (13, 14-15, 16) receive.
- the images collected by the camera are received by the light field generation module to generate an active structured light illumination light field, which is then sent to the spatial light modulator of the light field loading module again until the user-preset illumination light field is generated.
- the present invention can control the excitation light of the illuminating sample surface at any intensity in a specified space, and realize active structured light with a high modulation system without changing the coherence of the incident light.
- Super-resolution imaging of illumination Based on the spatio-temporal joint modulation method proposed by the present invention, the input non-uniform Gaussian distributed light field can be modulated into a uniformly illuminated flat-top light field to meet the application requirements based on quantitative analysis of fluorescence intensity; the illumination can be actively adjusted according to the spatial distribution characteristics of the sample intensity.
- Light intensity that is, the strong signal area reduces the light dose
- the weak signal area increases the light dose, and there is no signal or background area with no light to meet the application requirements of high dynamic range and low light dose; user-defined lighting area can meet the needs of light stimulation and fluorescence bleaching. Recovery and other application requirements. Without losing the SIM spatial resolution (90nm) and temporal resolution (100fps@512*512), the present invention is expected to increase the dynamic range of traditional structured light illumination microscopes by 40dB and reduce the light dose by 10 times, providing living cells with Dynamic long-term observation provides advanced new imaging technology.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Microscoopes, Condenser (AREA)
Abstract
一种主动结构光照明的超分辨显微成像方法及系统。通过时空联合强度调制方法，可对照明样本的激发光进行空间任意强度的调控，且在不改变入射光相干性的前提下，实现高调制度的主动结构光照明超分辨成像。基于这样的调制方法可实现将输入的非均匀高斯分布照明调制成均匀的平顶照明，满足基于荧光强度定量分析的应用需求；根据样本空间强度分布特征主动调整照明光强，满足高动态、低光照剂量成像需求；用户自定义照明区域，满足光刺激、荧光漂白恢复应用需求。在不损失SIM时空分辨率的前提下，可有效提高SIM的成像动态范围，减少光照剂量，满足细胞亚结构及其相互作用对百纳米以下超分辨、极低光照剂量、高动态范围的成像需求。
Description
本发明属于荧光显微成像技术领域，特别是一种主动照明与结构光照明复合的超分辨显微成像系统。
细胞是一个高度复杂的动态系统，其内部通过膜结构的区隔化形成多种功能迥异的数十种细胞器。这些细胞器的相互作用是维持细胞功能、决定细胞命运的关键。对这些细胞器及其相互作用的研究需要进行百纳米以下高分辨率、极低光照剂量、高动态范围、大视场均匀照明的活细胞成像。
在获取大视场均匀照明的研究中，光束整形器直接将高斯传播的光束转换为平顶光束，利用一对非球面透镜组，第一个透镜均匀的重新分配高斯光束，第二个透镜重新准直，从而产生平场照明。但基于折射光束整形器的工作距离有限，对表面加工质量和光学排列有严格的要求。光波导可在非常大的视场下提供均匀的照明，但由于它们工作在全内反射模式下，照明限制在盖玻片附近，无法实现切换照明角度实现不同深度照明。且固定的照明尺寸，不能和相机的成像视场进行匹配照明，全玻片照明对细胞样本造成不必要的光损伤。此外，经典方法包括使用一对微透镜阵列或者多模光纤，为了减少激光散斑，在系统中加入了散斑减速器或者振动光纤，但降低了光束的空间相干性，不适用于全内反射照明。ASTER本质上是一种混合扫描宽场照明装置，其按照特定的模式扫描高斯光束，以时间平均的方式提供平顶照明，从而可以在比较宽的视场范围内扫描出均匀照明场。然而，上述平场照明方法只能应用在基于单分子定位的超分辨显微镜中，比如PALM、STORM，其向基于多束光干涉产生余弦分布条纹的结构光照明显微镜(SIM)转化中仍受到限制，原因在于在对入射的非均匀高斯光斑进行调制时，要保障多光束之间频率相同、相位差恒定、偏振方向一直，以致在样品面干涉形成高调制度的照明条纹。
在拓展显微成像系统动态范围及减少光照剂量的研究中，Vinergoni等将摄影学中广泛应用的多曝光融合技术和基于点扫描的双光子成像技术相结合，并将其应用于内场景大动态范围的神经细胞成像中。但是连续多次曝光采集不仅增加了光照剂量，且降低了成像速度。为了克服多曝光的缺陷，发展出多探测同时成像技术，通过对同一场景进行
多级分光成像，同时采集不同曝光量的图像。然而由于采用多个成像光路同时探测，需要对不同的探测器进行校准，光学系统复杂且成本高。由此发展出主动照明技术，在成像系统中加入声光调制器，空间上单像素地对照明光场进行调控，这不仅提高了动态范围，而且减少了光照剂量。Hoebe等为了解决共聚焦显微镜有限的动态范围、光漂白及光毒性问题，研制出照明光剂量空间上可调控的显微成像技术，且在Nikon C1共聚焦显微镜上得到应用。实验验证，此成像技术不仅可以将动态范围提高2倍，而且可将光照剂量降低5倍，有效延长活细胞的存活时间。然而，基于共聚焦、双光子的低光照剂量、高动态成像方法，其分辨率受到光学衍射极限的限制。在使用大数值孔径物镜的情况下，横向极限分辨率约为200nm，轴向分辨率为500nm。此外，基于点扫描成像模式，光漂白和光毒性严重且成像速度慢，无法满足活细胞动态成像需求。
相比之下，SIM具有成像速度快、光损伤小、对荧光探针无特殊要求等特点，且百纳米的分辨率正好满足活细胞内重要细胞器的观测要求，已成为活细胞动态成像的首要选择。然而从照明方式上看，SIM本质上是一种宽场荧光显微成像技术，其整体照明强度在样品上呈现高斯形状分布。这种非均匀的照明方式，首先降低了可利用的视场尺寸，为了获得相对均匀的荧光图像，通常截取高斯光斑中心区域进行成像，从而损失了边缘视场。其次，中心亮边缘暗的高斯照明造成成像视场内光漂白特性不一致，中心比边缘更容易被漂白，限制了其在基于荧光强度定量分析中的应用。此外，对于高动态范围样本，这种不考虑样本标记密度的无差别照明方式，会造成弱信号欠曝光、强信号过曝光、背景及离焦信号无效曝光，限制了SIM成像系统的动态范围及造成不必要的光损伤。因此，发明一种基于空间光调制器时空联合调制的主动结构光照明超分辨成像方法(Active-SIM)，比如将高斯光斑调制成平场照明、根据样本标记密度自适应的调整照明强度、对任意感兴趣区域的照明等，这对于拓展SIM成像技术在活细胞动态长时程成像的应用中至关重要。
发明内容
本发明的目的在于对活细胞中亚细胞器及其相互作用的研究中需要进行百纳米以下高分辨率、极低光照剂量、高动态范围、大视场均匀照明的荧光成像需求，提出一种基于空间光调制器的时空联合强度调制方法，建立基于主动结构光照明的超分辨显微成像系统。
实现本发明目的的技术解决方案为：一种主动结构光照明的超分辨显微成像方法，
所述方法包括以下步骤：
步骤1，通过相机采集样品经激光激发的荧光图像；
步骤2，基于步骤1的图像构建所需的主动结构光照明光场；
步骤3，对步骤2的主动结构光照明光场进行转换并加载到空间光调制器中，形成时空联合强度调制的主动结构光照明光场；
步骤4，从步骤3中加载在空间光调制器的全息条纹位面图取出相应的位面图，并开始显示，激光器同步出光照明样本，相机同步曝光采集原始图像数据；
步骤5，判断全部全息条纹位面图是否显示完成，若是，空间光调制器停止位面显示，且相机停止图像采集，否则按预设的顺序切换到不同空间方向角和位相的位面图，返回步骤4，继续执行同步显示及曝光。
进一步地，步骤2中所述主动结构光照明光场包括：
第一照明光场：将输入的非均匀高斯分布光场调制成均匀的平顶照明光场，满足基于荧光强度定量分析及大视场拼接成像的需求；
或，第二照明光场：根据样本强度分布特征主动调整照明光强，包括强信号区域降低光照剂量、弱信号区域提高光照剂量、无信号或背景区域无光照，满足所需高动态范围、低光照剂量的成像需求；
或，第三照明光场：自定义指定位置和强度的照明，满足用户自定义感兴趣照明区域的成像需求。
进一步地，步骤3中调制后的主动结构照明光场的光强空间分布表示为Ia(x,y)：
式中，Ig(x,y,t)表示在一个相机曝光周期T内，激发光在样品面上照明光强的时空体积，在只允许一个级次光束通过时的SIM成像过程中，其照明强度在空间上呈现非均匀的高斯分布；Mi(x,y,t)表示加载在空间光调制器中的第i个强度调制函数或二值位面，N表示加载二值位面的个数，空间光调制器上每个像素的光强调制精度可表示为1/2N；ti表示每个二值位面对应的时间权重函数，所有时间权重求和为相机的一个曝光周期点(x,y)代表空间光调制器中任意空间坐标；m表示结构光的调制度，kx、ky、φ表示空间频率和初始相位。
实现上述方法的主动结构光照明的超分辨显微成像系统，所述系统包括光源模块、主动结构光照明光场加载模块、主动结构光照明光场生成模块以及荧光探测模块；
所述光源模块，用于实现至少四个激光波长的控制，且可实现单波长独立照明，多个波长多色分时及同时照明；
所述荧光探测模块，用于采集样本经激光激发的荧光图像；
所述主动结构光照明光场生成模块，用于基于所述荧光图像生成主动结构光照明光场；
所述主动结构光照明光场加载模块，用于将生成的主动结构光照明光场加载到SIM成像系统中，完成主动结构光照明超分辨成像。
进一步地，所述光源模块包括激光器、单模保偏光纤；
所述激光器，用于提供多个波长的激光，且可实现单波长及多波长同步出光控制；
所述单模保偏光纤，用于传输激光器耦合好的多路激光至主动结构光照明光场加载模块。
进一步地，所述主动结构光照明光场加载模块包括消色差准直扩束物镜、偏振分光棱镜、空间光调制器、傅里叶透镜、二分之一玻片、空间滤波器、组合二分之一玻片、准直透镜、照明筒镜；傅里叶透镜和准直透镜组成4f系统，空间光调制器位于4f系统的前焦面，空间滤波器位于4f系统的后焦面处，用于空间滤波，仅允许经过空间光调制器衍射出来的特定角度的光束通过，挡住其他衍射级次；二分之一玻片和组合二分之一玻片构成偏振调控组件，用于对不同方向角的入射偏振光进行调控；
所述光源模块出射的激光传输给消色差准直扩束物镜，经准直扩束后入射至偏振分光棱镜，透射的垂直偏振光入射至空间光调制器，经空间光调制器调制后主要衍射出0级、±1级三个出射光束，三个出射光束经偏振分光棱镜反射后依次经傅里叶透镜、二分之一玻片聚焦在空间滤波器上，滤波之后选通±1级出射光束依次经组合二分之一玻片、准直透镜、照明筒镜后进入荧光探测模块。
进一步地，所述荧光探测模块包括三维位移台、显微物镜、二向色镜、成像透镜和相机；
所述主动结构光照明光场加载模块输出的激发光，经二向色镜反射后通过显微物镜照射放置于三维位移台上的样本，激发样本荧光信号后由显微物镜收集，之后经二向色镜透射后再经成像透镜聚焦在相机的探测面上。
进一步地，所述主动结构光照明光场生成模块包括图像获取单元、图像处理单元和
光场生成单元；
所述图像获取单元，用于获取相机探测到的样品面的照明光场分布特征；
所述图像处理单元，用于将图像获取单元得到的照明光场转换为用户所需的主动结构光照明光场；
所述光场生成单元，用于将主动结构光照明光场转换为空间光调制器可加载的基于时空联合强度调制的一组携带不同时间权重的二值位面图，并加载至主动结构光照明光场加载模块中的空间光调制器中。
本发明与现有技术相比，其显著优点为：
1)在不破坏SIM成像系统中干涉光束的波前、偏振态及相干性，保障样品面上高调制度干涉条纹的前提下，实现主动结构光照明超分辨成像。
2)在不降低传统SIM成像系统时空分辨率的前提下，有如下优势：包括可将输入的非均匀高斯分布照明光场调制成均匀分布的平顶照明光场，满足基于荧光强度定量分析成像的应用需求；根据样本强度空间分布特征主动调整照明光强，满足高动态范围、低光照剂量成像的应用需求；自定义指定位置和强度的照明，满足用户自定义感兴趣照明区域的成像需求。通过提供这三种主动结构光照明光场，为活细胞高分辨率、极低光照剂量、高动态范围、大视场均匀照明成像提供先进的新型成像技术手段。
3)本发明在原有的SIM成像系统中，无需添加额外光学硬件，只需要基于本发明所提出的基于时空联合强度调制方法构建所需光场，在获得丰富成像功能的同时无经济成本增加，简化系统的复杂程度。
4)本发明在不损失SIM时空分辨率的前提下，可将传统SIM的成像动态范围提高40dB、光照剂量减少10倍。
下面结合附图对本发明作进一步详细描述。
图1为主动结构光照明光场调制方法流程图。
图2为主动结构光照明超分辨显微成像系统示意图。
图3为主动结构光照明光场调制方法原理图，其中图3(a)为基于铁电液晶空间光调制器像素级任意光强调制示意图，图3(b)为时空联合强度调制原理示意图。
图中附图标记如下：1四路合束激光器；2单模保偏光纤；3消色差准直扩束物镜；4偏振分光棱镜；5空间光调制器；6傅里叶透镜；7二分之一玻片；8空间滤波器；
9组合二分之一玻片；10准直透镜；11照明筒镜；12三维位移台；13物镜；14二向色镜；15成像筒镜；16相机；17图像获取单元；18图像处理单元；19光场生成单元。
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
需要说明，若本发明实施例中有涉及“第一”、“第二”等的描述，则该“第一”、“第二”等的描述仅用于描述目的，而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外，各个实施例之间的技术方案可以相互结合，但是必须是以本领域普通技术人员能够实现为基础，当技术方案的结合出现相互矛盾或无法实现时应当认为这种技术方案的结合不存在，也不在本发明要求的保护范围之内。
在一个实施例中，结合图1，提供了一种主动结构光照明的超分辨显微成像方法，所述方法包括以下步骤：
步骤1，通过相机采集样品经激光激发的荧光图像；
步骤2，基于步骤1的图像构建所需的主动结构光照明光场；
步骤3，对步骤2的主动结构光照明光场进行转换并加载到空间光调制器中，形成时空联合强度调制的主动结构光照明光场；
步骤4，从步骤3中加载在空间光调制器的全息条纹位面图取出相应的位面图，并开始显示，激光器同步出光照明样本，相机同步曝光采集原始图像数据；
步骤5，判断全部全息条纹位面图是否显示完成，若是，空间光调制器停止位面显示，且相机停止图像采集，否则按预设的顺序切换到不同空间方向角和位相的位面图，返回步骤4，继续执行同步显示及曝光。
进一步地，在其中一个实施例中，步骤2中所述主动结构光照明光场包括：
第一照明光场：将输入的非均匀高斯分布光场调制成均匀的平顶照明光场，满足基于荧光强度定量分析及大视场拼接成像的需求；
或，第二照明光场：根据样本强度分布特征主动调整照明光强，包括强信号区域降低光照剂量、弱信号区域提高光照剂量、无信号或背景区域无光照，满足所需高动态范围、低光照剂量的成像需求；
或，第三照明光场：自定义指定位置和强度的照明，满足用户自定义感兴趣照明区域的成像需求。
进一步地，在其中一个实施例中，为了实现时空联合强度调制的主动照明结构光场，本发明利用基于铁电液晶的空间光调制中每个独立的像素在“开”和“关”可进行快速的切换特性，实现照明光强在一个相机积分周期内的单像素级别的调控，如图3a所示。在SIM成像系统中，样品面上的照明光强整体上呈现高斯分布，可表示为：
由于在铁电液晶空间光调制器中每个像素上入射光的偏振态可单独调控，结合偏振分光棱镜可实现光强的“开关”操作，这样在相机的曝光周期内调整每个时间点上的照明开关，可实现任意光强的调控。因此，主动照明光场可以表示为：
为了转换为数字调制，引入数字脉宽调制方法。假设在空间光调制器中加载N个二进制位面Mi(x,y,t)，每个位面分配不同的照明时间权重ti＝(1/2i)·T，如图3b所示。通过在一个相机的曝光周期T内，连续加载N张不同时间权重的二进制位面图，可实现对入射光场单像素任意光强的调控，表示如下：
因此，步骤3中调制后的主动结构光场的照明光强分布表示为Ia(x,y)：
式中，I0表示峰值强度，σ是高斯光束的束腰半径，Ig(x,y,t)表示在一个相机曝光周期T内，激发光在样品面上照明光强的时空体积，在只允许一个级次光束通过时的SIM成像过程中，其照明强度在空间上呈现非均匀的高斯分布；Mi(x,y,t)表示加载在空间光调制器中的第i个强度调制函数或二值位面，N表示加载二值位面的个数，空间光调制器上每个像素的光强调制精度可表示为1/2N；ti表示每个二值位面对应的时间权重函数，所有时间权重求和为相机的一个曝光周期点(x,y)代表空间光调制器中任意空间坐标；m表示结构光的调制度，kx、ky、φ表示空间频率和初始相位。
在一个实施例中，实现上述方法的主动结构光照明的超分辨显微成像系统如图2所示，所述系统包括光源模块、主动结构光照明光场加载模块、主动结构光照明光场生成模块以及荧光探测模块；
所述光源模块，用于实现至少四个激光波长的控制，且可实现单波长独立照明，多个波长多色分时及同时照明；
所述荧光探测模块，用于采集样本经激光激发的荧光图像；
所述主动结构光照明光场生成模块，用于基于所述荧光图像生成主动结构光照明光场；
所述主动结构光照明光场加载模块，用于将生成的主动结构光照明光场加载到SIM成像系统中，完成主动结构光照明超分辨成像。
进一步地，在其中一个实施例中，所述光源模块包括激光器1、单模保偏光纤2；
所述激光器1，用于提供多个波长的激光，且可实现单波长及多波长同步出光控制；
所述单模保偏光纤2，用于传输激光器1耦合好的多路激光至主动结构光照明光场加载模块。
这里优选地，所述激光器1提供四个波长的激光，四个波长分别405nm、488nm、561nm、637nm。
进一步地，在其中一个实施例中，所述主动结构光照明光场加载模块包括消色差准直扩束物镜3、偏振分光棱镜4、空间光调制器5、傅里叶透镜6、二分之一玻片7、空间滤波器8、组合二分之一玻片9、准直透镜10、照明筒镜11；傅里叶透镜6和准直透镜10组成4f系统，空间光调制器5位于4f系统的前焦面，空间滤波器8位于4f系统的后焦面处，用于进行空间滤波，仅允许经过空间光调制器5衍射出来的特定角度的光束通过，挡住其他衍射级次；二分之一玻片7和组合二分之一玻片9构成偏振调控组件，用于对不同方向角的入射偏振光进行调控；
所述光源模块出射的激光传输给消色差准直扩束物镜3，经准直扩束后入射至偏振分光棱镜4，透射的垂直偏振光入射至空间光调制器5，经空间光调制器调制后衍射出0级、±1级三个出射光束，三个出射光束经偏振分光棱镜4反射后依次经傅里叶透镜6、二分之一玻片7聚焦在空间滤波器8上，滤波之后选通±1级出射光束依次经组合二分之一玻片9、准直透镜10、照明筒镜11后进入荧光探测模块。
进一步地，在其中一个实施例中，所述荧光探测模块包括三维位移台12、显微物镜13、二向色镜14、成像透镜15和相机16；
所述主动结构光照明光场加载模块输出的主动照明结构光场，经二向色镜14反射后通过显微物镜13照射放置于三维位移台12上的样本照明筒镜11和显微物镜13构成另外一对4f系统，用于将结构光照明光场共轭到样品面处，激发样本荧光信号后由显微物镜13收集，之后经二向色镜14透射后再经成像透镜15聚焦至相机16的探测面上。
进一步地，在其中一个实施例中，所述主动结构光照明光场生成模块包括图像获取单元17、图像处理单元18和光场生成单元19；
所述图像获取单元17，用于获取相机16探测到的样品面的照明光场分布特征；
所述图像处理单元18，用于将图像获取单元17得到的照明光场转换为用户所需的主动结构光照明光场；
所述光场生成单元19，用于将主动结构光照明光场转换为空间光调制器可加载的基于时空联合强度调制的一组携带不同时间权重的二值位面图，并加载至主动结构光照明光场加载模块中的空间光调制器中。
上述系统在工作前，需要对空间光调制器和相机进行空间坐标标定，用于对输入、输出光场和调制光场进行空间坐标匹配，生成像素级匹配精度的主动照明光场。具体过程如下：1)在空间光调制器中加载等间隔的分布组成的“十字”交叉点；2)荧光探测模块采集均匀分布的荧光染料信号图像；3)基于空间光调制器加载的图像和相机采集的图像，计算出标定矩阵(包括缩放比、旋转、平移)。
系统的工作流程如下：
通过激光器模块1选通405nm或488nm或561nm或673nm四路激光器的其中一路激光，进入到保偏单模光纤2中。
进一步地，激发光经过单模光束将光束传输给光场加载模块的消色差准直物镜3中。光束准直扩束后的光束照射偏振分光棱镜4，透射的垂直偏振光(s偏振)照射到空间光调制器5上，光束经过空间光调制器调制后，主要衍射出0级、±1三个出射光束，反射的出射光束再次经过偏振分光棱镜4，只有经过调制后的水平偏振光(p偏振)可以进入成像系统中；如光强调制原理图3a所示，在空间光调制器中加载时间权重的位面，通过调制入射偏振光的“像素开”、“像素关”，可实现单像素的光强控制，比如将非均匀的高斯分布激发光调制为均匀分布的平顶光束(图3b)。经过单像素强度调制的0级、±1三个衍射光束入射到傅里叶透镜6上，经过用于偏振补偿的二分之一玻片7，调制后的偏振光束最后聚焦在空间滤波器8的位置，经过空间滤波后，0级光束被阻挡，且只有±1级光束通过；±1级光束通过组合二分只玻片9，其用于旋转不同方向角的入
射光的偏振方向，从而在样品面获得最大调制度的干涉条纹；±1级光束经过准直透镜10后，在其后焦面干涉；进一步地，光束经过照明筒镜后进入荧光探测模块。
进一步地，照明光束经过二向色镜14反射后进入显微物镜13，最后激发夹持在位移台12处的荧光样本。经过主动照明的结构光照明的荧光样本发射出波长比激发光波长更长的荧光，经过高数值孔径物镜13收集，进入到荧光探测模块，收集的荧光信号经过二向色镜14透射后进入成像透镜15，最后被相机16探测。
进一步地，主动结构光照明光场生成模块的图像获取单元收集到相机16采集的图像，送入图像处理单元18，图像处理单元对收集荧光图像和用户预设的光场进行计算，进而生成用户预设光场所需要的调制光场，进一步送到光场生成单元19，生成空间光调制器5可加载的携带不同时间权重的二值位面全息条纹图。
进一步地，光场生成模块将生成的二值位面图重建加载到光场加载模块中的空间光调制5中，经过器件6至11、14、13、12照射样本，发射的荧光被荧光探测模块(13、14-15、16)接收。相机收集到的图像被光场生成模块接收，生成主动结构光照明光场，再次送入到光场加载模块的空间光调制器内，直到生成用户预设的照明光场。
综上，本发明通过发明一种时空联合强度调制方法，可对照明样本面的激发光进行指定空间任意强度的调控，且在不改变入射光相干性的前提下，实现高调制度的主动结构光照明的超分辨成像。基于本发明提出的时空联合调制方法，可将输入非均匀的高斯分布光场调制成均匀照明的平顶光场，满足基于荧光强度定量分析的应用需求；根据样本强度空间分布特征，主动调整照明光强，即强信号区域降低光照剂量、弱信号区域提高光照剂量、无信号或背景区域无光照，满足高动态范围、低光照剂量的应用需求；用户自定义照明区域，满足光刺激、荧光漂白恢复等应用需求。本发明在不损失SIM空间分辨率(90nm)和时间分辨率(100fps@512*512)的前提下，预期可将传统结构光照明显微镜的动态范围提高40dB，光照剂量减少10倍，为活细胞动态长时间观察提供先进的新型成像技术手段。
以上显示和描述了本发明的基本原理、主要特征及优点。本行业的技术人员应该了解，本发明不受上述实施例的限制，上述实施例和说明书中描述的只是说明本发明的原理，在不脱离本发明精神和范围的前提下，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
Claims (9)
- 一种主动结构光照明的超分辨显微成像方法，其特征在于，所述方法包括以下步骤：步骤1，通过相机采集样品经激光激发的荧光图像；步骤2，基于步骤1的图像构建所需的主动结构光照明光场；步骤3，对步骤2的主动结构光照明光场进行转换并加载到空间光调制器中，形成时空联合强度调制的主动结构光照明光场；步骤4，从步骤3中加载在空间光调制器的全息条纹位面图取出相应的位面图，并开始显示，激光器同步出光照明样本，相机同步曝光采集原始图像数据；步骤5，判断全部全息条纹位面图是否显示完成，若是，空间光调制器停止位面显示，且相机停止图像采集，否则按预设的顺序切换到不同空间方向角和位相的位面图，返回步骤4，继续执行同步显示及曝光。
- 根据权利要求1所述的主动结构光照明的超分辨显微成像方法，其特征在于，步骤2中所述主动结构光照明光场包括：第一照明光场：将输入的非均匀高斯分布光场调制成均匀的平顶照明光场，满足基于荧光强度定量分析及大视场拼接成像的需求；或，第二照明光场：根据样本强度分布特征主动调整照明光强，包括强信号区域降低光照剂量、弱信号区域提高光照剂量、无信号或背景区域无光照，满足所需高动态范围、低光照剂量的成像需求；或，第三照明光场：自定义指定位置和强度的照明，满足用户自定义感兴趣照明区域的成像需求。
- 根据权利要求1所述的主动结构光照明的超分辨显微成像方法，其特征在于，步骤3中调制后的主动结构照明光场的光强空间分布表示为Ia(x,y)：
式中，Ig(x,y,t)表示在一个相机曝光周期T内，激发光在样品面上照明光强的时空体积，在只允许一个级次光束通过时的SIM成像过程中，其照明强度在空间上呈现非均匀的高斯分布；Mi(x,y,t)表示加载在空间光调制器中的第i个强度调制函数或二值位 面，N表示加载二值位面的个数，空间光调制器上每个像素的光强调制精度表示为1/2N；ti表示每个二值位面对应的时间权重函数，所有时间权重求和为相机的一个曝光周期点(x,y)代表空间光调制器中任意空间坐标；m表示结构光的调制度，kx、ky、φ表示空间频率和初始相位。 - 实现权利要求1至3任意一项所述方法的主动结构光照明的超分辨显微成像系统，其特征在于，所述系统包括光源模块、主动结构光照明光场加载模块、主动结构光照明光场生成模块以及荧光探测模块；所述光源模块，用于实现至少四个激光波长的控制，且可实现单波长独立照明，多个波长多色分时及同时照明；所述荧光探测模块，用于采集样本经激光激发的荧光图像；所述主动结构光照明光场生成模块，用于基于所述荧光图像生成主动结构光照明光场；所述主动结构光照明光场加载模块，用于将生成的主动结构光照明光场加载到SIM成像系统中，完成主动结构光照明超分辨成像。
- 根据权利要求4所述的主动结构光照明的超分辨显微成像系统，其特征在于，所述光源模块包括激光器(1)、单模保偏光纤(2)；所述激光器(1)，用于提供多个波长的激光，且可实现单波长及多波长同步出光控制；所述单模保偏光纤(2)，用于传输激光器(1)耦合好的多路激光至主动结构光照明光场加载模块。
- 根据权利要求5所述的主动结构光照明的超分辨显微成像系统，其特征在于，所述激光器(1)提供四个波长的激光，四个波长分别405nm、488nm、561nm、637nm。
- 根据权利要求5所述的主动结构光照明的超分辨显微成像系统，其特征在于，所述主动结构光照明光场加载模块包括消色差准直扩束物镜(3)、偏振分光棱镜(4)、空间光调制器(5)、傅里叶透镜(6)、二分之一玻片(7)、空间滤波器(8)、组合二分之一玻片(9)、准直透镜(10)、照明筒镜(11)；傅里叶透镜(6)和准直透镜(10)组成4f系统，空间光调制器(5)位于4f系统的前焦面，空间滤波器(8)位于4f系统的后焦面处，用于空间滤波，仅允许经过空间光调制器(5)衍射出来的特定角度的光 束通过，挡住其他衍射级次；二分之一玻片(7)和组合二分之一玻片(9)构成偏振调控组件，用于对不同方向角的入射偏振光进行调控；所述光源模块出射的激光传输给消色差准直扩束物镜(3)，经准直扩束后入射至偏振分光棱镜(4)，透射的垂直偏振光入射至空间光调制器(5)，经空间光调制器调制后衍射出0级、±1级三个出射光束，三个出射光束经偏振分光棱镜(4)反射后依次经傅里叶透镜(6)、二分之一玻片(7)聚焦在空间滤波器(8)上，滤波之后选通±1级出射光束依次经组合二分之一玻片(9)、准直透镜(10)、照明筒镜(11)后进入荧光探测模块。
- 根据权利要求7所述的主动结构光照明的超分辨显微成像系统，其特征在于，所述荧光探测模块包括三维位移台(12)、显微物镜(13)、二向色镜(14)、成像透镜(15)和相机(16)；所述主动结构光照明光场加载模块输出的激发光，经二向色镜(14)反射后通过显微物镜(13)照射放置于三维位移台(12)上的样本，激发样本荧光信号后由显微物镜(13)收集，之后经二向色镜(14)透射后再经成像透镜(15)聚焦在相机(16)的探测面上。
- 根据权利要求8所述的主动结构光照明的超分辨显微成像系统，其特征在于，所述主动结构光照明光场生成模块包括图像获取单元(17)、图像处理单元(18)和光场生成单元(19)；所述图像获取单元(17)，用于获取相机(16)探测到的样品面的照明光场分布特征；所述图像处理单元(18)，用于将图像获取单元(17)得到的照明光场转换为用户所需的主动结构光照明光场；所述光场生成单元(19)，用于将主动结构光照明光场转换为空间光调制器可加载的基于时空联合强度调制的一组携带不同时间权重的二值位面图，并加载至主动结构光照明光场加载模块中的空间光调制器中。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076269.9 | 2022-09-05 | ||
| CN202211076269.9A CN115308184A (zh) | 2022-09-05 | 2022-09-05 | 一种主动结构光照明的超分辨显微成像方法及系统 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024051079A1 true WO2024051079A1 (zh) | 2024-03-14 |
Family
ID=83867000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2023/075140 WO2024051079A1 (zh) | 2022-09-05 | 2023-02-09 | 一种主动结构光照明的超分辨显微成像方法及系统 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115308184A (zh) |
| WO (1) | WO2024051079A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118392037A (zh) * | 2024-06-27 | 2024-07-26 | 厦门微亚智能科技股份有限公司 | 一种基于成像的三维测量方法及系统 |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115308184A (zh) * | 2022-09-05 | 2022-11-08 | 中国科学院苏州生物医学工程技术研究所 | 一种主动结构光照明的超分辨显微成像方法及系统 |
| FR3145983A1 (fr) * | 2023-02-17 | 2024-08-23 | Paris Sciences Et Lettres | Dispositif de localisation d’une particule individualisée dans un échantillon et procédé associé |
| CN116068745A (zh) * | 2023-02-24 | 2023-05-05 | 广东粤港澳大湾区协同创新研究院 | 一种角度连续可调超分辨显微镜照明装置及相应成像方法 |
| CN116740705A (zh) * | 2023-04-06 | 2023-09-12 | 深圳大学 | 一种基于随机森林分类算法的超亮光斑定位识别方法 |
| CN117631249B (zh) * | 2024-01-18 | 2024-05-24 | 清华大学 | 线扫共聚焦扫描光场显微成像装置及方法 |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104515759A (zh) * | 2014-12-16 | 2015-04-15 | 中国科学院苏州生物医学工程技术研究所 | 非线性结构光照明显微成像方法及系统 |
| US20150211986A1 (en) * | 2012-06-29 | 2015-07-30 | Zhejiang University | Super-resolution microscopy method and device |
| CN107389631A (zh) * | 2017-04-28 | 2017-11-24 | 中国科学院生物物理研究所 | 高速多色多模态结构光照明超分辨显微成像系统及其方法 |
| CN110954521A (zh) * | 2019-12-18 | 2020-04-03 | 深圳大学 | 一种宽场超分辨显微成像方法及其系统 |
| CN111897118A (zh) * | 2020-06-28 | 2020-11-06 | 北京大学 | 基于偏振结构光调制的多维层析荧光显微成像系统及方法 |
| CN112798564A (zh) * | 2020-12-22 | 2021-05-14 | 中国科学院苏州生物医学工程技术研究所 | 随机光学重建与结构光照明复合超分辨成像系统 |
| WO2021143707A1 (zh) * | 2020-01-19 | 2021-07-22 | 北京大学 | 一种双模态显微成像系统和方法 |
| WO2021227292A1 (zh) * | 2020-05-12 | 2021-11-18 | 北京大学 | 一种直接结构光照明超分辨显微重建方法 |
| CN113670878A (zh) * | 2021-08-25 | 2021-11-19 | 西安交通大学 | 基于空频域混合式重建的超分辨结构光照明显微成像方法 |
| WO2022120853A1 (zh) * | 2020-12-11 | 2022-06-16 | 深圳华大智造科技股份有限公司 | 超分辨检测系统及超分辨检测方法 |
| CN115308184A (zh) * | 2022-09-05 | 2022-11-08 | 中国科学院苏州生物医学工程技术研究所 | 一种主动结构光照明的超分辨显微成像方法及系统 |
-
2022
- 2022-09-05 CN CN202211076269.9A patent/CN115308184A/zh active Pending
-
2023
- 2023-02-09 WO PCT/CN2023/075140 patent/WO2024051079A1/zh unknown
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150211986A1 (en) * | 2012-06-29 | 2015-07-30 | Zhejiang University | Super-resolution microscopy method and device |
| CN104515759A (zh) * | 2014-12-16 | 2015-04-15 | 中国科学院苏州生物医学工程技术研究所 | 非线性结构光照明显微成像方法及系统 |
| CN107389631A (zh) * | 2017-04-28 | 2017-11-24 | 中国科学院生物物理研究所 | 高速多色多模态结构光照明超分辨显微成像系统及其方法 |
| CN110954521A (zh) * | 2019-12-18 | 2020-04-03 | 深圳大学 | 一种宽场超分辨显微成像方法及其系统 |
| WO2021143707A1 (zh) * | 2020-01-19 | 2021-07-22 | 北京大学 | 一种双模态显微成像系统和方法 |
| WO2021227292A1 (zh) * | 2020-05-12 | 2021-11-18 | 北京大学 | 一种直接结构光照明超分辨显微重建方法 |
| CN111897118A (zh) * | 2020-06-28 | 2020-11-06 | 北京大学 | 基于偏振结构光调制的多维层析荧光显微成像系统及方法 |
| WO2022120853A1 (zh) * | 2020-12-11 | 2022-06-16 | 深圳华大智造科技股份有限公司 | 超分辨检测系统及超分辨检测方法 |
| CN112798564A (zh) * | 2020-12-22 | 2021-05-14 | 中国科学院苏州生物医学工程技术研究所 | 随机光学重建与结构光照明复合超分辨成像系统 |
| CN113670878A (zh) * | 2021-08-25 | 2021-11-19 | 西安交通大学 | 基于空频域混合式重建的超分辨结构光照明显微成像方法 |
| CN115308184A (zh) * | 2022-09-05 | 2022-11-08 | 中国科学院苏州生物医学工程技术研究所 | 一种主动结构光照明的超分辨显微成像方法及系统 |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118392037A (zh) * | 2024-06-27 | 2024-07-26 | 厦门微亚智能科技股份有限公司 | 一种基于成像的三维测量方法及系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115308184A (zh) | 2022-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2024051079A1 (zh) | 一种主动结构光照明的超分辨显微成像方法及系统 | |
| CN107941763B (zh) | 一种共轴三维受激辐射损耗超分辨显微成像方法和装置 | |
| CN106970055B (zh) | 一种三维荧光差分超分辨显微方法及装置 | |
| US6909105B1 (en) | Method and device for representing an object | |
| US7274446B2 (en) | Method and arrangement for the deep resolved optical recording of a sample | |
| US9360428B2 (en) | Interferometric focusing of guide-stars for direct wavefront sensing | |
| CN106980174B (zh) | 一种综合性荧光超分辨显微成像装置 | |
| CN107167929A (zh) | 基于dmd的双模式光学超分辨显微成像装置及方法 | |
| CN112798564B (zh) | 随机光学重建与结构光照明复合超分辨成像系统 | |
| CN107966110A (zh) | 一种双模态数字全息显微装置 | |
| WO2023221400A1 (zh) | 超分辨单物镜光片显微成像光学系统及其成像系统 | |
| CN111024659B (zh) | 一种基于并行探测的多图像重建显微成像方法和装置 | |
| CN114460731B (zh) | 一种基于dmd的多色结构光照明超分辨显微成像方法和装置 | |
| Pozzi et al. | High-throughput spatial light modulation two-photon microscopy for fast functional imaging | |
| WO2024087615A1 (zh) | 一种荧光发射比率三维超分辨成像方法 | |
| US11947098B2 (en) | Multi-focal light-sheet structured illumination fluorescence microscopy system | |
| US20100097694A1 (en) | Method and arrangement for controlled actuation of a microscope, in particular of a laser scanning microscope | |
| US20080135790A1 (en) | Real-Time High-Magnification Stereoscopic Microscope | |
| JP2011128573A (ja) | ホログラム像投影装置 | |
| CN117137433B (zh) | 一种多波长共聚焦散斑成像系统、方法、存储介质和设备 | |
| RU2579640C1 (ru) | Конфокальный спектроанализатор изображений | |
| CN214374304U (zh) | 一种复合超分辨成像装置 | |
| CN107941777A (zh) | 一种抗漂白单分子定位三维超分辨显微系统 | |
| CN117250178B (zh) | 一种荧光各向异性探测方法及其探测系统 | |
| DE112018005412T5 (de) | Markierungsfreie Mikroskopie |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23861791 Country of ref document: EP Kind code of ref document: A1 |