CN111665227B - Laser scanning microscopic imaging method, device and system based on virtual modulation imaging - Google Patents
Laser scanning microscopic imaging method, device and system based on virtual modulation imaging Download PDFInfo
- Publication number
- CN111665227B CN111665227B CN202010510316.0A CN202010510316A CN111665227B CN 111665227 B CN111665227 B CN 111665227B CN 202010510316 A CN202010510316 A CN 202010510316A CN 111665227 B CN111665227 B CN 111665227B
- Authority
- CN
- China
- Prior art keywords
- image
- lens
- light intensity
- intensity distribution
- unscanned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 57
- 238000002310 reflectometry Methods 0.000 claims abstract description 33
- 238000005286 illumination Methods 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 7
- 238000005315 distribution function Methods 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims description 3
- 238000000034 method Methods 0.000 abstract description 8
- 230000003287 optical effect Effects 0.000 abstract description 7
- 239000000523 sample Substances 0.000 description 43
- 230000006870 function Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000004621 scanning probe microscopy Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 2
- 238000000386 microscopy Methods 0.000 description 2
- 238000000399 optical microscopy Methods 0.000 description 2
- 206010034972 Photosensitivity reaction Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001268 conjugating effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000007850 fluorescent dye Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 208000007578 phototoxic dermatitis Diseases 0.000 description 1
- 231100000018 phototoxicity Toxicity 0.000 description 1
- 102000004169 proteins and genes Human genes 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4084—Scaling of whole images or parts thereof, e.g. expanding or contracting in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Microscoopes, Condenser (AREA)
Abstract
The invention discloses a laser scanning microscopic imaging method, a device and a system based on virtual modulation imaging, wherein the method comprises the following steps: obtaining a descanned image of the sample reflected fluorescence imaged on an image plane after passing through a scanning system; calculating the light intensity distribution of the descanned image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity; carrying out image mask processing on the unscanned image to obtain an image before modulation; performing integral processing on the image before modulation to obtain a modulated image; and performing Fourier transform processing on the modulated image, and generating a super-resolution image based on the high-frequency component. The embodiment of the invention virtually modulates the image obtained from the laser scanning microscope in a digital mode, so that the laser scanning microscope breaks through the diffraction limit, improves the resolution and has better optical sectioning capability.
Description
Technical Field
The invention relates to the field of optical microscopy, in particular to a laser scanning microscopic imaging method, device and system based on virtual modulation imaging.
Background
High resolution imaging is critical for biomedical research, however the spatial resolution of conventional imaging systems is limited by the optical diffraction limit, making it difficult to detect more delicate structures. Therefore, in order to break through the limitation of optical diffraction on resolution and realize super-resolution imaging, a series of microscopes for realizing super-resolution imaging, such as a stimulated emission depletion microscope (STED), a random optical reconstruction microscope (STORM), a photo-activated positioning microscope (pram), and the like, have appeared. However, the above-mentioned imaging techniques of microscopy rely on exogenous specific fluorescent dyes or proteins and are problematic in terms of time efficiency.
In order to overcome the problems, the structural illumination obvious microtechnology of the wide-field super-resolution optical microscopy is widely applied in the field of biomedicine, and the technology has the advantages of rapidness, weak photobleaching, low phototoxicity and the like. In the structured light micro-technology, devices such as gratings are usually adopted, a structured light field with sinusoidal intensity distribution is used for modulating the surface of a sample, and the illumination direction needs to be changed for many times in the process of acquiring an image so as to modulate an objective function, so that high-frequency information is loaded, and the effect of improving the resolution is finally achieved. However, the implementation process of the structured light illumination technology requires a complicated illumination system and a possible phase error, so that the structured light illumination microscopy technology cannot be applied to a laser scanning microscope.
Disclosure of Invention
The invention aims to provide a laser scanning microscopic imaging method, a device and a system based on virtual modulation imaging, which aim to solve the technical problem that the existing structured light illumination microscopic technology cannot be applied to a laser scanning microscope to realize super-resolution imaging, and realize the effect of improving the super-resolution by digitally modulating an image obtained from the laser scanning microscope.
In order to solve the above technical problem, in a first aspect, an embodiment of the present invention provides a laser scanning microscopy imaging method based on virtual modulation imaging, including:
obtaining a descanned image of the sample reflected fluorescence imaged on an image plane after passing through a scanning system;
calculating the light intensity distribution of the descanned image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity;
carrying out image mask processing on the unscanned image to obtain an image before modulation;
integrating the image before modulation to obtain a modulated image;
and carrying out Fourier transform processing on the modulated image, and generating a super-resolution image based on high-frequency components.
Further, the calculating the light intensity distribution of the descan image and obtaining the non-descan image according to the light intensity distribution and the preset sample reflectivity specifically comprises:
calculating the light intensity distribution of the descanned image according to the light intensity distribution of the image plane; substituting the light intensity distribution and the preset sample reflectivity into a first relational expression of preset unscanned image data, the light intensity distribution and the sample reflectivity to obtain unscanned image data;
and obtaining an unscanned image from the unscanned image data.
Further, let the sampling point on the image plane be (x) 0 ,y 0 ) And (μ, v) is a value other than (x) on the image plane 0 ,y 0 ) Any point of (a), h il (μ-x 0 ,v-y 0 ) Is the sample point (x) measured at (μ, v) 0 ,y 0 ) The distribution function of the relative light intensity of (h) de (x- μ, y-v) is the light intensity distribution at (μ, v);
substituting the light intensity distribution and the preset sample reflectivity into a first relational expression of preset unscanned image data, the light intensity distribution and the sample reflectivity to obtain the unscanned image data, which specifically comprises the following steps:
setting the preset sample reflectivity as s (x, y), and then setting the sample reflectivity at (mu, v) as s (u, v);
substituting the light intensity distribution and the sample reflectivity into the first relation, wherein the first relation is as follows:
I non (x,y,x 0 ,y 0 )＝∫∫h il (μ-x 0 ,v-y 0 )s(μ,v)h de (x-u,y-v)dudv；
and calculating according to the first relational expression to obtain the unscanned image data.
Further, performing image mask processing on the unscanned image, specifically;
and multiplying the digital image mask formula in a sine form by the first relational expression to obtain an image formula before modulation.
Further, the image formula before modulation is as follows:
I mul (x,y,x 0 ,y 0 )＝I non (x,y,x 0 ,y 0 )m(x,y)；
wherein m (x, y) is the digital image mask formula for the sinusoidal form:
m(x,y)＝cos[2Πf 0 (xcosθ+ysinθ)+α)；
theta is the rotation angle of the sine stripe, alpha is the initial phase, f 0 Carrier frequency equal to cut-off frequency f c 。
Further, the integrating processing is performed on the image before modulation, specifically:
and integrating the image before modulation at the sampling point to obtain a modulated image formula.
Further, the modulated image formula is:
wherein the content of the first and second substances,
represents the convolution, m (x) 0 ,y 0 ) Is a point (x) 0 ,y 0 ) A digital mask of (a), h il (x 0 ,y 0 ) Indicating the point of illumination (x) 0 ,y 0 ) Point spread function of (c), s (x) 0 ,y 0 ) Is (x) 0 ,y 0 ) The sample reflectivity at (a).
In a second aspect, an embodiment of the present invention provides a laser scanning microscopy imaging apparatus based on virtual modulation imaging, including:
the descanning image acquisition module is used for acquiring a descanning image of the sample reflected fluorescence which is imaged on an image plane after passing through the scanning system;
the unscanned image acquisition module is used for calculating the light intensity distribution of the unscanned image and acquiring an unscanned image according to the light intensity distribution and the preset sample reflectivity;
the image mask processing module is used for carrying out image mask processing on the unscanned image to obtain an image before modulation;
the integral processing module is used for carrying out integral processing on the image before modulation to obtain a modulated image;
and the Fourier transform processing module is used for carrying out Fourier transform processing on the modulated image and generating a super-resolution image based on high-frequency components.
In a third aspect, an embodiment of the present invention further provides a laser scanning microscopy imaging system based on virtual modulation imaging, including a laser scanning microscope and a computer, where the computer is configured to execute the laser scanning microscopy imaging method based on virtual modulation imaging as described above.
Further, the laser scanning microscope comprises a laser, a collimating lens, a beam splitter, a two-dimensional scanning galvanometer, a first lens, a second lens, a reflector, an objective lens, a third lens and an image pickup device; the first lens and the second lens form a lens pair to expand;
the laser emitted by the laser sequentially passes through the collimating lens, the beam splitter, the two-dimensional scanning galvanometer, the first lens, the second lens, the reflector and the objective lens and then reaches the sample, so that the fluorescence on the surface of the sample is excited and a fluorescence signal is reflected;
the fluorescence signal sequentially passes through the objective lens, the second lens, the first lens, the two-dimensional scanning galvanometer, the beam splitter and the third lens and then is imaged on the camera device; wherein, the directions of the laser light entering from the two-dimensional scanning galvanometer and the emergent fluorescent signal are on the same straight line.
In summary, the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, a computer is used for obtaining a descanned image of a sample reflected fluorescence which is imaged on an image plane after passing through a scanning system; then calculating the light intensity distribution of the descanned image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity; carrying out image mask processing on the unscanned image to obtain an image before modulation, and then carrying out integral processing on the image before modulation to obtain a modulated image; and finally, carrying out Fourier transform processing on the modulated image, and generating a super-resolution image based on high-frequency components. The embodiment of the invention virtually modulates the image obtained from the laser scanning microscope in a digital mode, so that the light source intensity is not required to be dynamically adjusted at the illuminating end, a physical mask is not required to be added in front of the detecting end for modulation, and the virtual modulation principle is combined into the laser scanning microscope, so that the laser scanning microscope achieves the effect of improving the resolution. In addition, the embodiment modulates the acquired image in a digital mode, so that the generated super-resolution image has no artifact, has better optical slicing capability and can image deeper tissues.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a block diagram of a laser scanning microscope provided by an embodiment of the present invention;
FIG. 2 is a flowchart of a laser scanning microscopic imaging method based on virtual modulation imaging according to an embodiment of the present invention;
reference numerals: 1. the device comprises a laser, 2, a collimating lens, 3, a beam splitter, 4, a two-dimensional scanning galvanometer, 5, a first lens, 6, a second lens, 7, a reflector, 8, an objective lens, 9, a sample, 10, a third lens, 11 and an image pickup device.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
The embodiment of the invention provides a laser scanning microscopic imaging system based on virtual modulation imaging, which comprises: the laser scanning microscope system comprises a laser scanning microscope and a computer, wherein the computer is used for executing a laser scanning microscopic imaging method based on virtual modulation imaging.
As an example of the embodiment of the present invention, referring to fig. 1, the laser scanning microscope includes: the device comprises a laser 1, a collimating lens 2, a beam splitter 3, a two-dimensional scanning galvanometer 4, a first lens 5, a second lens 6, a reflector 7, an objective lens 8, a third lens 10 and an image pickup device 11; the two-dimensional scanning galvanometer 4, the first lens 5, the second lens 6, the reflector 7 and the objective lens 8 form a scanning system, and the scanning system is responsible for completing scanning operation on a sample 9; the first lens 5 and the second lens 6 form a lens pair to expand beams, and the Fourier plane of the objective lens is conjugated with the position of the scanning galvanometer. In an embodiment of the invention, the purpose of conjugating the fourier plane of the objective lens to the position of the scanning galvanometer is to reduce the vignetting effect.
In the embodiment of the invention, the imaging principle of the laser scanning microscope is as follows: the laser 1 emits laser, the laser sequentially passes through the collimating lens 2, the beam splitter 3, the two-dimensional scanning galvanometer 4, the first lens 5, the second lens 6, the reflecting mirror 7 and the objective lens rear 8 to reach a sample, and the surface fluorescence of the sample 9 is excited and a fluorescence signal is reflected. The fluorescence signal sequentially passes through the objective lens 8, the second lens 6, the first lens 5, the two-dimensional scanning galvanometer 4, the beam splitter 3 and the third lens 10, and then is imaged on an image plane of the camera device 11 to form a descanned image. The deflection of the scanning galvanometer does not affect the emitting direction of the light, so that the directions of the laser light incident from the two-dimensional scanning galvanometer 4 and the emitted fluorescent signal are on the same straight line.
A series of images collected by the image plane of the camera device 11 are virtually modulated by using an algorithm on a computer, and the algorithm principle of the virtual modulation algorithm is as follows:
suppose the point spread function of the illumination path is h il (x,y)；
Point spread function h of probe path de (x, y) under incoherent illumination, the resulting Point Spread Function (PSF) is:
wherein, J 1 Is a first order bessel function, omega =2 pi NA/lambda,
NA is the numerical aperture and λ is the source wavelength, then the resolution of a conventional laser scanning microscope is defined by the radius of the airy disk R =0.6 λ/NA, and in the fourier domain, the corresponding cut-off frequency is expressed as: f. of c =1/R, i.e. only at-f c <f<f c In the frequency range to pass through a conventional laser scanning microscope system.
Fig. 2 is a flowchart of a laser scanning microscopic imaging method based on virtual modulation imaging according to an embodiment of the present invention, where the laser scanning microscopic imaging method based on virtual modulation imaging includes steps S1-S5:
s1, obtaining a descanned image of a sample reflected fluorescence which is imaged on an image plane after passing through a scanning system.
In this embodiment, descan (descan) means that laser irradiates a sample after passing through a scanning galvanometer to excite a fluorescent signal on the surface of the sample, then a fluorescent light beam carrying information of the sample returns along the original path and passes through the scanning galvanometer again, the light beam emitted by the scanning galvanometer is represented as a descaned light beam, and an image formed on the camera device after the descaned light beam passes through the beam splitter and the third lens is a descaned image. Descanning can also be understood as that the fluorescent light beam emitted from the scanning galvanometer and the fluorescent light beam incident on the scanning galvanometer are along the same straight line and are not influenced by the deflection of the scanning galvanometer.
S2, calculating the light intensity distribution of the descanned image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity.
As an example of the embodiment of the present invention, the calculating a light intensity distribution of the descan image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity specifically includes:
calculating the light intensity distribution of the descanned image according to the light intensity distribution of the image plane; substituting the light intensity distribution and the preset sample reflectivity into a first relational expression of preset unscanned image data, the light intensity distribution and the sample reflectivity to obtain unscanned image data;
obtaining the unscanned image from the unscanned image data.
As an example of the embodiment of the present invention, let the sampling point on the image plane be (x) 0 ,y 0 ) (mu, v) is not (x) on the image plane 0 ,y 0 ) Any point of (a), h il (μ-x 0 ,v-y 0 ) Is the sample point (x) measured at (μ, v) 0 ,y 0 ) The distribution function of the relative light intensity of (h) de (x- μ, y-v) is the light intensity distribution at (μ, v);
substituting the light intensity distribution and the preset sample reflectivity into a first relational expression of preset unscanned image data, the light intensity distribution and the sample reflectivity to obtain unscanned image data, which specifically comprises the following steps:
setting the preset sample reflectivity as s (x, y), and then the sample reflectivity at (mu, v) is s (u, v);
substituting the light intensity distribution and the sample reflectivity into the first relation, wherein the first relation is as follows:
I non (x,y,x 0 ,y 0 )＝∫∫h il (μ-x 0 ,v-y 0 )s(μ,v)h de (x-u,y-v)dudv；
and calculating according to the first relational expression to obtain unscanned image data.
Wherein the distribution function of the light intensity measured at the sampling point is h il (x-x 0 ,y-y 0 ) The distribution function of the intensity measured at (mu, v) is h il (x-μ,y-v)。
And S3, carrying out image mask processing on the unscanned image to obtain an image before modulation.
As an example of the embodiment of the present invention, the performing an image mask process on the non-descan image specifically includes:
and multiplying the digital image mask formula in a sine form by the first relational expression to obtain an image formula before modulation.
As an example of the embodiment of the present invention, the image formula before modulation is:
I mul (x,y,x 0 ,y 0 )＝I non (x,y,x 0 ,y 0 )m(x,y)
wherein m (x, y) is the digital image mask formula for the sinusoidal form:
m(x,y)＝cos[2Πf 0 (xcosθ+ysinθ)+α)；
theta is the rotation angle of the sine stripe, alpha is the initial phase, f 0 The carrier frequency being equal to the cut-off frequency f c 。
In the embodiment of the present invention, it should be noted that the digital mask may also be a negative value, so that the digital mask has no dc component.
S4, integrating the image before modulation to obtain a modulated image;
in this embodiment of the present invention, as an example, the performing integration processing on the image before modulation specifically includes:
and integrating the image before modulation at the sampling point to obtain a modulated image formula.
As an example of the embodiment of the present invention, the modulated image formula is:
wherein the content of the first and second substances,
representing the convolution, m (x) 0 ,y 0 ) Is a point (x) 0 ,y 0 ) A digital mask of (a), h il (x 0 ,y 0 ) Indicating the point of illumination at (x) 0 ,y 0 ) Point spread function of (d), s (x) 0 ,y 0 ) Is (x) 0 ,y 0 ) The sample reflectivity at (a).
And S5, carrying out Fourier transform processing on the modulated image, and generating a super-resolution image based on the high-frequency component.
In the embodiment of the invention, the modulated image is subjected to Fourier transform processing according to the following formula,
wherein f is x ,f y The spatial frequency is represented by a representation of,
which is indicative of the fourier transform,
if σ is a dirac δ function, the image formula obtained after fourier transform processing is as follows:
from the image formula after the fourier transform processing, it can be known that the high frequency signal
Moved detection pass band
The frequency range that can be acquired becomes:
-2f c <f<2f c
therefore, based on the high-frequency component, the resolution can be increased by two times, and a super-resolution image can be generated.
In summary, in the embodiment of the present invention, a computer is used to obtain a descanned image of the sample reflected fluorescence, which is imaged on the image plane after passing through the scanning system; then calculating the light intensity distribution of the descanned image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity; carrying out image mask processing on the unscanned image to obtain an image before modulation, and then carrying out integral processing on the image before modulation to obtain a modulated image; and finally, carrying out Fourier transform processing on the modulated image, and generating a super-resolution image based on high-frequency components. The embodiment of the invention carries out virtual modulation on the image obtained from the laser scanning microscope in a digital mode, thereby not needing to carry out dynamic adjustment on the light source intensity at the illuminating end and not needing to add a physical mask in front of the detecting end for modulation, and combining the virtual modulation principle into the laser scanning microscope, so that the laser scanning microscope achieves the effect of improving the resolution ratio. In addition, the embodiment digitally modulates the acquired image, so that the generated super-resolution image has no artifact, has better optical slicing capability and can image deeper tissues.
In addition, an embodiment of the present invention further provides a laser scanning microscopic imaging apparatus based on virtual modulation imaging, including:
the descanning image acquisition module is used for acquiring a descanning image of the sample reflected fluorescence imaged on an image plane after passing through the scanning system;
the unscanned image acquisition module is used for calculating the light intensity distribution of the unscanned image and acquiring an unscanned image according to the light intensity distribution and the preset sample reflectivity;
the image mask processing module is used for carrying out image mask processing on the unscanned image to obtain an image before modulation;
the integral processing module is used for carrying out integral processing on the image before modulation to obtain a modulated image;
and the Fourier transform processing module is used for carrying out Fourier transform processing on the modulated image and generating a super-resolution image based on high-frequency components.
It should be noted that all technical contents and technical effects of the laser scanning microscopic imaging method based on virtual modulation imaging provided in the embodiment of the present invention, and all explanations and descriptions of the laser scanning microscopic imaging method based on virtual modulation imaging are applicable to the laser scanning microscopic imaging apparatus based on virtual modulation imaging provided in the embodiment of the present invention, and therefore, the laser scanning microscopic imaging apparatus based on virtual modulation imaging provided in the embodiment of the present invention is not described in detail herein.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (4)
1. A laser scanning microscopic imaging method based on virtual modulation imaging is characterized by comprising the following steps:
s1, acquiring a descanned image of a sample reflected fluorescence imaged on an image plane after passing through a scanning system;
s2, calculating the light intensity distribution of the descanned image, and obtaining an unscanned image according to the light intensity distribution and a preset sample reflectivity;
calculating the light intensity distribution of the descanned image according to the light intensity distribution of the image plane; substituting the light intensity distribution and the preset sample reflectivity into a first relational expression of preset non-descanned image data and the light intensity distribution and the sample reflectivity to obtain non-descanned image data; setting the preset sample reflectivity as s (x, y), and then setting the sample reflectivity at (mu, v) as s (u, v); substituting the light intensity distribution and the sample reflectivity into the first relation, wherein the first relation is as follows: i is non (x,y,x 0 ,y 0 )＝∫∫h il (μ-x 0 ,v-y 0 )s(μ,v)h de (x-u, y-v) dudv; calculating according to the first relational expression to obtain unscanned image data; obtaining the unscanned image from the unscanned image data; wherein, the sampling point on the image plane is set as (x) 0 ,y 0 ) (μ, v) is not (x) on the image plane 0 ,y 0 ) Any point of (a), h il (μ-x 0 ,v-y 0 ) Is the sample point (x) measured at (μ, v) 0 ,y 0 ) Corresponding to the distribution function of the light intensity, h de (x- μ, y-v) is the light intensity distribution at (μ, v);
s3, carrying out image mask processing on the unscanned image to obtain an image before modulation;
multiplying the digital image mask formula in a sine form by the first relational expression to obtain an image formula before modulation; the image formula before modulation is as follows: i is mul (x,y,x 0 ,y 0 )＝I non (x,y,x 0 ,y 0 ) m (x, y); wherein m (x, y) is the digital image mask formula for the sinusoidal form: m (x, y) = cos [2 |/] 0 (xcosθ+ysinθ)+α](ii) a Theta is the rotation angle of the sine stripe, alpha is the initial phase, f 0 Carrier frequency equal to cut-off frequency f c ；
S4, integrating the image before modulation to obtain a modulated image;
integrating the image before modulation at the sampling point to obtain a modulated image formula; the modulated image formula is:
wherein the content of the first and second substances,
represents the convolution, m (x) 0 ,y 0 ) Is a point (x) 0 ,y 0 ) A digital mask of (a), h il (x 0 ,y 0 ) Indicating the point of illumination (x) 0 ,y 0 ) Point spread function of (c), s (x) 0 ,y 0 ) Is (x) 0 ,y 0 ) Sample reflectivity of (b), h de (x 0 ,y 0 ) Is (x) 0 ,y 0 ) The light intensity distribution of (b);
and S5, carrying out Fourier transform processing on the modulated image, and generating a super-resolution image based on the high-frequency component.
2. A laser scanning microscopic imaging apparatus based on virtual modulation imaging, which is used for executing the laser scanning microscopic imaging method based on virtual modulation imaging according to claim 1, and comprises:
the descanning image acquisition module is used for acquiring a descanning image of the sample reflected fluorescence which is imaged on an image plane after passing through the scanning system;
the unscanned image acquisition module is used for calculating the light intensity distribution of the unscanned image and acquiring an unscanned image according to the light intensity distribution and the preset sample reflectivity;
the image mask processing module is used for carrying out image mask processing on the unscanned image to obtain an image before modulation;
the integral processing module is used for carrying out integral processing on the image before modulation to obtain a modulated image;
and the Fourier transform processing module is used for carrying out Fourier transform processing on the modulated image and generating a super-resolution image based on high-frequency components.
3. A laser scanning microscopic imaging system based on virtual modulation imaging is characterized by comprising a laser scanning microscope and a computer, wherein the computer is used for executing the laser scanning microscopic imaging method based on virtual modulation imaging according to claim 1.
4. The laser scanning microscopic imaging system based on virtual modulation imaging according to claim 3, characterized in that the laser scanning microscope comprises a laser, a collimating lens, a beam splitter, a two-dimensional scanning galvanometer, a first lens, a second lens, a reflector, an objective lens, a third lens and a camera device; the first lens and the second lens form a lens pair to expand;
laser emitted by the laser sequentially passes through the collimating lens, the beam splitter, the two-dimensional scanning galvanometer, the first lens, the second lens, the reflector and the objective lens and then reaches a sample, and fluorescence on the surface of the sample is excited and a fluorescence signal is reflected;
the fluorescence signal sequentially passes through the objective lens, the second lens, the first lens, the two-dimensional scanning galvanometer, the beam splitter and the third lens and then is imaged on the camera device; wherein, the directions of the laser light entering from the two-dimensional scanning galvanometer and the emergent fluorescent signal are on the same straight line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010510316.0A CN111665227B (en) | 2020-06-05 | 2020-06-05 | Laser scanning microscopic imaging method, device and system based on virtual modulation imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010510316.0A CN111665227B (en) | 2020-06-05 | 2020-06-05 | Laser scanning microscopic imaging method, device and system based on virtual modulation imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111665227A CN111665227A (en) | 2020-09-15 |
| CN111665227B true CN111665227B (en) | 2023-02-03 |
Family
ID=72386936
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010510316.0A Active CN111665227B (en) | 2020-06-05 | 2020-06-05 | Laser scanning microscopic imaging method, device and system based on virtual modulation imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111665227B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113624358B (en) * | 2021-07-14 | 2023-09-26 | 中国电子科技集团公司第十三研究所 | Three-dimensional displacement compensation method and control device for photothermal reflection microscopic thermal imaging |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008152011A (en) * | 2006-12-18 | 2008-07-03 | Lasertec Corp | Confocal microscope and method for picking up confocal image |
| CN106415354A (en) * | 2014-05-21 | 2017-02-15 | 浜松光子学株式会社 | Microscope device and image acquisition method |
| CN108387562A (en) * | 2018-02-02 | 2018-08-10 | 中国科学院上海光学精密机械研究所 | The adjusting method of pin hole axial position in confocal microscope system |
| CN111095074A (en) * | 2017-07-24 | 2020-05-01 | 密歇根大学董事会 | 3-axis side-view confocal fluorescence microscopy endoscope |
-
2020
- 2020-06-05 CN CN202010510316.0A patent/CN111665227B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008152011A (en) * | 2006-12-18 | 2008-07-03 | Lasertec Corp | Confocal microscope and method for picking up confocal image |
| CN106415354A (en) * | 2014-05-21 | 2017-02-15 | 浜松光子学株式会社 | Microscope device and image acquisition method |
| CN111095074A (en) * | 2017-07-24 | 2020-05-01 | 密歇根大学董事会 | 3-axis side-view confocal fluorescence microscopy endoscope |
| CN108387562A (en) * | 2018-02-02 | 2018-08-10 | 中国科学院上海光学精密机械研究所 | The adjusting method of pin hole axial position in confocal microscope system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111665227A (en) | 2020-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8705172B2 (en) | Microscopy method and microscope with enhanced resolution | |
| Squire et al. | Three dimensional image restoration in fluorescence lifetime imaging microscopy | |
| US8019136B2 (en) | Optical sectioning microscopy | |
| US8532398B2 (en) | Methods and apparatus for optical segmentation of biological samples | |
| Kubitscheck et al. | Lateral diffusion measurement at high spatial resolution by scanning microphotolysis in a confocal microscope | |
| US8705043B2 (en) | Height measurement by correlating intensity with position of scanning object along optical axis of a structured illumination microscope | |
| JP2018515744A (en) | Optical measuring device and process | |
| CN109712072B (en) | Fringe illumination Fourier domain iterative updating super-resolution microscopic imaging method based on total internal reflection | |
| JP2003227796A (en) | Method and arrangement for grasping sample by depth decomposition | |
| CN112345523B (en) | Open-field phase microscopic imaging device and method based on spiral phase plate | |
| CN111665227B (en) | Laser scanning microscopic imaging method, device and system based on virtual modulation imaging | |
| CN111024659B (en) | Multi-image reconstruction microscopic imaging method and device based on parallel detection | |
| NL2008873C2 (en) | Method and apparatus for multiple points of view three-dimensional microscopy. | |
| JP2915919B2 (en) | Laser scanning fluorescence microscope | |
| NL2018386B1 (en) | Structured illumination scanning microscopy | |
| US11885947B2 (en) | Lattice light sheet microscope and method for tiling lattice light sheet in lattice light sheet microscope | |
| CN113218926B (en) | Light modulation assembly, fluorescence microscope and chip detection system and method | |
| WO2017213171A1 (en) | Optical information detection apparatus and microscope system | |
| CN214374304U (en) | Composite super-resolution imaging device | |
| CN114740008A (en) | Super-resolution wafer defect detection system | |
| JP4207467B2 (en) | Microscope illumination device | |
| US20230281922A1 (en) | Non-Transitory Storage Medium Storing Thereon Optical Aberration Correction Program and Non-Transitory Storage Medium Storing Thereon Optical Wavefront Estimation | |
| US20230296515A1 (en) | Method for single molecule localization microscopy | |
| US20220236549A1 (en) | Phase-sensitive single molecule localization microscopy | |
| CN110989313B (en) | Holographic microscopic imaging device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |