CN110793633B - Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers - Google Patents
Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers Download PDFInfo
- Publication number
- CN110793633B CN110793633B CN201911115073.4A CN201911115073A CN110793633B CN 110793633 B CN110793633 B CN 110793633B CN 201911115073 A CN201911115073 A CN 201911115073A CN 110793633 B CN110793633 B CN 110793633B
- Authority
- CN
- China
- Prior art keywords
- optical fiber
- fluorescence
- light
- sample
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000013307 optical fiber Substances 0.000 title claims abstract description 77
- 238000003384 imaging method Methods 0.000 title claims abstract description 48
- 238000004364 calculation method Methods 0.000 title claims abstract description 9
- 239000000835 fiber Substances 0.000 claims abstract description 34
- 239000011159 matrix material Substances 0.000 claims abstract description 30
- 238000009826 distribution Methods 0.000 claims abstract description 16
- 230000005284 excitation Effects 0.000 claims description 26
- 230000008878 coupling Effects 0.000 claims description 19
- 238000010168 coupling process Methods 0.000 claims description 19
- 238000005859 coupling reaction Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 15
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 230000000295 complement effect Effects 0.000 claims description 3
- 230000001678 irradiating effect Effects 0.000 claims description 3
- 229910044991 metal oxide Inorganic materials 0.000 claims description 3
- 150000004706 metal oxides Chemical class 0.000 claims description 3
- 239000004065 semiconductor Substances 0.000 claims description 3
- 238000000034 method Methods 0.000 abstract description 13
- 238000011084 recovery Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 abstract description 4
- 102100021792 Gamma-sarcoglycan Human genes 0.000 abstract description 2
- 101000616435 Homo sapiens Gamma-sarcoglycan Proteins 0.000 abstract description 2
- 230000008859 change Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000000701 chemical imaging Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000002292 fluorescence lifetime imaging microscopy Methods 0.000 description 2
- 238000000799 fluorescence microscopy Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000684 flow cytometry Methods 0.000 description 1
- 238000001917 fluorescence detection Methods 0.000 description 1
- 238000002073 fluorescence micrograph Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 239000002086 nanomaterial Substances 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000004557 single molecule detection Methods 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
- G01J3/0218—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using optical fibers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/10—Arrangements of light sources specially adapted for spectrometry or colorimetry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2803—Investigating the spectrum using photoelectric array detector
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/44—Raman spectrometry; Scattering spectrometry ; Fluorescence spectrometry
- G01J3/4406—Fluorescence spectrometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/6402—Atomic fluorescence; Laser induced fluorescence
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/02—Details
- G01J3/10—Arrangements of light sources specially adapted for spectrometry or colorimetry
- G01J2003/102—Plural sources
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
- G01J2003/2826—Multispectral imaging, e.g. filter imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
- G01N2021/0106—General arrangement of respective parts
- G01N2021/0112—Apparatus in one mechanical, optical or electronic block
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N2021/6417—Spectrofluorimetric devices
- G01N2021/6419—Excitation at two or more wavelengths
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N2021/6484—Optical fibres
Landscapes
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Biochemistry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Optics & Photonics (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
The invention provides a single-pixel multispectral calculation imaging system and an imaging method based on bundled fibers, which can realize stronger anti-interference capability, have very few hardware consumption resources and have less requirements on texts. The invention provides an observation matrix which adopts the combination of a bundled optical fiber and an image sensor (CCD) to replace a DMDA new method of variation. One branch of the bundled optical fibers is connected with the sensor to collect signals, and the other branches are connected with the modulatable light source. Different light intensity distributions are formed on the irradiated target by different on-off states of the light sources, and the light intensity distributions are observation matrixesSince the light source can simply achieve a modulation of 10-100KHz, the frequency of change of the observation matrix can also reach this rate. After the system is built, the CCD is adopted to observe the matrixAnd collecting and recording frame by frame, establishing a recovery model, and then not collecting in the actual observation process. The method can realize high-speed single-pixel imaging.
Description
Technical Field
The invention relates to the technical field of optical computational imaging, in particular to a single-pixel multispectral computational imaging system and a single-pixel multispectral computational imaging method based on bundled fibers.
Background
The current high-sensitivity rapid fluorescence detection technology mainly comprises a single molecule detection technology, a time resolution technology and a super-resolution measurement technology. Among the imaging techniques for high temporal resolution, Fluorescence Lifetime Imaging Microscopy (FLIM) is the technology used more in the field at present. However, the nano-displacement scanning platform contained in the nano-displacement scanning platform has poor stability and complex scanning process, not only the manufacturing cost is increased, but also the testing time of the nano-material and the biomacromolecule is greatly prolonged, so the success rate is also obviously influenced. Therefore, it is currently difficult to simultaneously perform a correlation spectral analysis that achieves high temporal resolution and an observed object.
Modern digital cameras capture images using an array of photodetectors, while single-pixel imaging reconstructs the image by sampling the scene using a series of masks and correlating the distribution of these masks to the corresponding intensities measured using a single-pixel detector. While not performing as well as digital cameras in traditional visible imaging, single-pixel imaging has proven advantageous in non-conventional applications.
As a new imaging mode, the single-pixel camera is a typical application for exploring data sparsity. The system uses a set of observation matricesAnd the corresponding sampling value Y is used as algorithm input through a matrix relationAnd solving the image X by adopting an underdetermined equation solving method.
In 2006, RICE university applied the compressed sensing theory to imaging systems, and proposed a single-pixel camera. The camera utilizes a digital micromirror array to perform linear sampling on an optical image, measures a sampling value through a single detector element, and reconstructs a target image by utilizing a reconstruction algorithm, thereby realizing single-pixel imaging. The single-pixel camera integrates image acquisition and compression, saves storage resources and transmission bandwidth, and is suitable for the field of non-visible light which cannot be shot by the traditional method. Fig. 1 is a schematic diagram of imaging of a single-pixel camera and an image restoration effect.
In 2014, a high-TIME-resolution multispectral SINGLE-PHOTON IMAGING algorithm based on a compressed sensing algorithm is provided in a patent "TIME-RESOLVED SINGLE-PHOTON OR ULTRA-WEAK LIGHT MULTI-DIMENSIONAL IMAGING SPECTRUM SYSTEM ANDMETHOD" of the research center of space science and application of the Chinese academy of sciences. The implementation method is as shown in fig. 2, after a light source 1 irradiates a sample, the light passes through an optical filter or an attenuation sheet 2, an optical imaging component 3, a Spatial Light Modulator (SLM) or a Digital Micromirror (DMD)4, a concave mirror 5, a grating light splitting element 6, a light converging and collecting component (multi-wavelength switchable) 7, an entry point or an array single photon detector 9, a multi-channel measuring instrument 10 and a high-precision time measuring instrument 11 process detector data, the data enters a data packet memory 14, and calculation is performed through a compressed sensing correlation algorithm 15. However, the system is based on the random matrix generation of compressed sensing by a Spatial Light Modulator (SLM) or a Digital Micromirror (DMD), and although the single-photon sensor can achieve the response rate of picosecond level, the system is limited by the response rate of the SLM and the DMD (about 100 Hz and 300 Hz).
At present, most of researches on a single-pixel camera adopt a Digital Micromirror Device (DMD) to set an observation matrixAnd carrying out overturning change, and simultaneously collecting a corresponding sampling value Y by using a photomultiplier tube (PMT) or an avalanche diode (APD) of a high-sensitivity sensor. For high-speed imaging applications, factors related to the imaging frame frequency mainly include the observation matrix frequency, the imaging resolution and the image compression ratio, and the observation matrix frame frequency becomes a main factor restricting the imaging frame frequency when the imaging resolution and the image compression ratio are the same. At present, the frame frequency of a commonly used DMD is about 100-300Hz, and a 20KHz high frame frequency DMD is also provided, but the price is high, the control data flow is huge, and the use condition is high.
Disclosure of Invention
In view of this, the invention provides a single-pixel multispectral calculation imaging system and an imaging method based on bundled fibers, which can realize stronger anti-interference capability, have very few hardware consumption resources and have less requirements on texts.
In order to achieve the purpose, the invention provides a single-pixel multispectral calculation imaging system based on bundled optical fibers, which is characterized by comprising an optical fiber spectrometer, multimode optical fibers, an optical fiber coupling light source group, a second collimating lens, a beam splitter, an objective lens, a sample stage, a second color filter, an image sensor, signal processing equipment and a computer;
the optical fiber coupling light source group sequentially emits exciting light according to the fixed sequence, the exciting light of each time is emitted out through the multimode optical fiber, is collimated by the second collimating lens, passes through the beam splitter and is focused and irradiated on a sample on the sample stage through the objective lens, and structured light with a spatial distribution form is formed;
the sample emits fluorescence under the excitation of the structural light; after the fluorescence is focused by the objective lens, the fluorescence is transmitted and reflected at the beam splitter, part of the fluorescence is reflected and penetrates through the second color filter and then is detected on the image sensor, and the other part of the fluorescence is transmitted and then is converged into the multimode optical fiber through the second collimating lens; the fluorescence entering the multimode optical fiber is emitted from the corresponding optical fiber bundle and is detected by the optical fiber spectrometer; the signal processing equipment carries out photoelectric conversion on the fluorescence signals collected by the fiber spectrometer and the image sensor and then transmits the fluorescence signals to the computer.
The optical fiber spectrometer is replaced by a whole body consisting of a point or array single-photon detector, an aperture diaphragm, a converging lens, a first color filter and a first collimating lens;
the fluorescence entering the multimode fiber is emitted from the corresponding fiber bundle, is collimated by a first collimating lens, is filtered by a first color filter, is converged by a converging lens, and is detected by a point or array single-photon detector after being filtered by an aperture diaphragm space; the signal processing equipment carries out photoelectric conversion on the fluorescent signals collected by the point or array single-photon detector and the image sensor and then transmits the fluorescent signals to the computer.
The point or array single photon detector adopts a photomultiplier or an avalanche diode.
The multimode optical fiber adopts one-to-many fanout optical fiber bundles, and the fanout number is more than or equal to 7.
The optical fiber coupling light source group adopts a modulatable light source, the intensity and the opening and closing state of the light source are controlled through duty ratio modulation, and the number of the light sources is equal to the number of the fan-out of the multimode optical fibers minus the number of the point or array single photon detectors.
The optical fiber coupling light source group adopts a laser light source or an LED light source.
Among them, the image sensor employs a charge coupled device or a complementary metal oxide semiconductor.
The invention also provides an imaging method, which adopts the single-pixel multispectral calculation imaging system based on the bundled optical fiber and comprises the following steps:
step 1, placing a uniform standard fluorescent sample on a sample table, forming different excitation light combinations by each light source in an optical fiber coupling light source group under different opening and closing states and output power conditions, forming a fixed sequence with the length of N by the different excitation light combinations, sequentially emitting excitation light by the optical fiber coupling light source group according to the fixed sequence, emitting the excitation light through a multimode optical fiber each time, collimating the excitation light through a second collimating lens, passing through a beam splitter, focusing and irradiating the excitation light on the sample table through an objective lens, and forming structured light with a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; the fluorescence is focused by the objective lens, reflected at the beam splitter, penetrates through the second color filter and is detected on the image sensor to form a two-dimensional image corresponding to the secondary exciting light, the two-dimensional image is imaged and transmitted to a computer through signal processing equipment to be reconstructed into a row vector, the length of the row vector is the total pixel number M of the image, and the row vector corresponding to all exciting light combinations is recorded as an observation matrix phi with N rows and M columns according to the emission sequence;
step 2, placing a target sample on a sample stage, emitting exciting light by the optical fiber coupling light source group according to a fixed sequence in sequence, wherein each time the exciting light is emitted by the multimode optical fiber, is collimated by the second collimating lens, passes through the beam splitter and is focused and irradiated on the sample stage by the objective lens to form structured light with a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; after the fluorescence is focused by the objective lens, the fluorescence is transmitted at the beam splitter, the fluorescence is converged by the second collimating lens to enter the multimode optical fiber, the fluorescence entering the multimode optical fiber is emitted from the corresponding optical fiber bundle, the fluorescence intensity value corresponding to the exciting light is detected on the optical fiber spectrometer, and the fluorescence intensity value corresponding to all exciting light combinations is recorded as a column vector Y of N rows according to the emission sequence;
and 3, recording a column vector reconstructed by the sample image to be restored as X, wherein the column vector Y and the observation matrix phi and X satisfy the following relation:
Y=φ·X
where · is a matrix multiplication;
solving an approximate value X' of the X by adopting an underdetermined equation;
and reversely reconstructing the X' into a two-dimensional image matrix S, and taking the S as a sample image.
The optical fiber spectrometer is replaced by a whole body consisting of a point or array single-photon detector, an aperture diaphragm, a converging lens, a first color filter and a first collimating lens;
the fluorescence entering the multimode fiber is emitted from the corresponding fiber bundle, is collimated by a first collimating lens, is filtered by a first color filter, is converged by a converging lens, and is detected by a point or array single-photon detector after being filtered by an aperture diaphragm space; the signal processing equipment carries out photoelectric conversion on the fluorescent signals collected by the point or array single-photon detector and the image sensor and then transmits the fluorescent signals to the computer.
Has the advantages that:
the invention provides an observation matrix which adopts the combination of a bundled optical fiber and an image sensor (CCD) to replace a DMDA new method of variation. One branch of the bundled optical fibers is connected with the sensor to collect signals, and the other branches are connected with the modulatable light source. Different light intensity distributions are formed on the irradiated target by different on-off states of the light sources, and the light intensity distributions are observation matrixesThe observation matrix is simple in that the light source can simply realize a modulation of 10-100KHzThe rate may also be reached by varying the frequency. After the system is built, the CCD is adopted to observe the matrixAnd collecting and recording frame by frame, establishing a recovery model, and then not collecting in the actual observation process. The method can realize high-speed single-pixel imaging.
Drawings
FIG. 1 is a schematic diagram of a single-pixel camera and a recovery result;
FIG. 2 is a time-resolved single-photon or ultra-low-light multi-dimensional imaging spectroscopy system;
FIG. 3 is a schematic diagram of a single-pixel multi-spectral computational imaging system based on bundled fibers according to the present invention.
The system comprises a 1-1-point or array single photon detector, a 1-2-aperture diaphragm, a 1-3-convergent lens, a 1-4-first color filter (switchable by a rotating wheel), a 1-5-first collimating lens, a 1-6-multimode optical fiber, a 1-7-optical fiber coupling light source group (switchable by multiple wavelengths), a 1-8-second collimating lens, a 1-9-beam splitter, a 1-10-objective lens, a 1-11-sample stage, a 1-12-second color filter (switchable by a rotating wheel), a 1-13-image sensor, a 1-14-signal processing device and a 1-15-computer.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The frequency of single frame turnover of DMD (digital micromirror) is generally 100-200Hz, and the frame frequency of common images is 30-50Hz through 5-10 picture reconstruction. The bandwidth of a single-pixel sensor such as a PMT (photomultiplier tube) or an APD (avalanche photo diode) can reach 10KHz-100KHz, the bandwidth range can also be reached after the light source is modulated, 3KHz imaging and modulation rate are adopted in the current embodiment, 64 pictures are adopted for reconstruction in an algorithm, and the frame frequency can reach 470 Hz. The high-speed fluorescence imaging method has better application prospect in the fields of biological observation, flow cytometry and the like.
Based on the basis, the invention adopts the fiber spectrometer to replace a point or array single photon detector, and can realize continuous spectrum imaging.
As shown in FIG. 3, the single-pixel multispectral computational imaging system based on bundled fibers of the invention comprises: the system comprises an optical fiber spectrometer, multimode optical fibers 1-6, an optical fiber coupling light source group (switchable between multiple wavelengths) 1-7, second collimating lenses 1-8, beam splitters 1-9, objective lenses 1-10, sample stages 1-11, second color filters (switchable by rotating wheels) 1-12, image sensors 1-13, signal processing equipment 1-14 and a computer 1-15;
changing the on-off state and intensity of each light source in the optical fiber coupling light source group (switchable among multiple wavelengths) 1-7, and forming exciting light with different spatial distribution forms on the sample; collecting exciting light with different space distribution forms through an image sensor 1-13, and constructing a recovery model in a computer 1-15; the fluorescence intensity is collected by the point or array single photon detector 1-1 and substituted into the recovery model, and then the microscopic imaging recovery can be completed.
Specifically, the fiber-coupled light source groups (multi-wavelength switchable) 1-7 emit excitation light according to a fixed sequence; exciting light is emitted out through the multimode optical fiber 1-6, collimated through the second collimating lens 1-8, passes through the beam splitter 1-9, and is focused and irradiated on a sample on the sample stage 1-11 through the objective lens 1-10 to form structured light in a space distribution form; the sample emits fluorescence under the excitation of the structural light; after being focused by the objective lens 1-10, the fluorescence is transmitted and reflected at the beam splitter 1-9, part of the fluorescence is reflected and penetrates through the second color filter (which can be switched by a rotating wheel) 1-12 and then is detected on the image sensor 1-13, and the other part of the fluorescence is transmitted and then is converged by the second collimating lens 1-8 to enter the multimode optical fiber 1-6;
the fluorescence entering the multimode optical fibers 1-6 is emitted from the corresponding optical fiber bundle and is detected by the optical fiber spectrometer; the signal processing equipment 1-14 carries out photoelectric conversion on the fluorescence signals collected by the fiber spectrometer and the image sensor 1-13 and then transmits the fluorescence signals to the computer 1-15.
In order to realize better sensitivity, the fiber spectrometer can be replaced by a point or array single photon detector 1-1, an aperture diaphragm 1-2, a converging lens 1-3, a first color filter 1-4 (switchable by a rotating wheel) and a first collimating lens 1-5. The fluorescence entering the multimode optical fiber 1-6 is emitted from the corresponding optical fiber bundle, is collimated by a first collimating lens 1-5, is filtered by a first color filter (which can be switched by a rotating wheel) 1-4, is converged by a converging lens 1-3, and is detected by a point or array single-photon detector 1-1 after being spatially filtered by an aperture diaphragm 1-2; the signal processing equipment 1-14 carries out photoelectric conversion on the fluorescent signals collected by the point or array single photon detector 1-1 and the image sensor 1-13 and then transmits the signals to the computer 1-15.
The point or array single photon detector 1-1 adopts a photomultiplier tube (PMT) or an avalanche diode (APD).
The multimode fibers 1-6 adopt one-to-many fanout fiber bundles, and the fanout number is more than or equal to 1-7.
The optical fiber coupling light source group (switchable among multiple wavelengths) 1-7 adopts light sources which can be modulated at high speed, such as laser light sources, LED light sources and the like, the intensity and the open-close state of the light sources are controlled through duty ratio modulation, and the number of the light sources is equal to the number of the fan-out number of the multimode optical fibers 1-6 minus the number of the point or array single photon detectors 1-1.
The image sensors 1 to 13 employ Charge Coupled Devices (CCDs) or Complementary Metal Oxide Semiconductors (CMOS).
The invention also provides an imaging method of the single-pixel multispectral calculation imaging system based on the bundled optical fibers, which comprises the following steps:
step 1, placing uniform standard fluorescent samples on sample tables 1-11, and forming different combinations of all light sources in optical fiber coupling light source groups (switchable among multiple wavelengths) 1-7 according to different opening and closing states and output power conditions to form a fixed sequence, wherein the sequence length is N. The optical fiber coupling light source group (switchable among multiple wavelengths) 1-7 emits exciting light according to a fixed sequence, the exciting light is emitted out through the multimode optical fiber 1-6, is collimated by the second collimating lens 1-8, passes through the beam splitter 1-9 and then is focused and irradiated on a sample on the sample stage 1-11 through the objective lens 1-10, and structured light with a spatial distribution form is formed; the sample emits fluorescence under the excitation of the structural light; after being focused by an objective lens 1-10, fluorescence is reflected at a beam splitter 1-9, is detected on an image sensor 1-13 after penetrating through a second color filter (capable of being switched by a rotating wheel) 1-12, is imaged and transmitted to a computer 1-15 through a signal processing device 1-14, the two-dimensional image is recorded as one line in a two-dimensional observation matrix after being reconstructed, the length of the line is the total pixel number M of the image, and all fluorescence images are imaged and stored in the computer 1-15 through the image sensor 1-13 and are reconstructed into the observation matrixThe matrix is composed of N rows and M columns;
step 2, placing a target sample on a sample table 1-11, emitting exciting light by an optical fiber coupling light source group (switchable among multiple wavelengths) 1-7 according to a fixed sequence, emitting the exciting light through a multimode optical fiber 1-6, collimating the exciting light by a second collimating lens 1-8, passing through a beam splitter 1-9, and then focusing and irradiating the sample on the sample table 1-11 through an objective lens 1-10 to form structured light in a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; after being focused by an objective lens 1-10, fluorescence is transmitted at a beam splitter 1-9, is converged by a second collimating lens 1-8 and enters a multimode optical fiber 1-6, the fluorescence entering the multimode optical fiber 1-6 is emitted from a corresponding optical fiber beam, is collimated by a first collimating lens 1-5, is filtered by a first color filter (which can be switched by a rotating wheel) 1-4, is converged by a converging lens 1-3, and is detected by a point or array single photon detector 1-1 after being spatially filtered by an aperture diaphragm 1-2; collecting the point or array single photon detector 1-1 according to a fixed sequence to obtain a fluorescence intensity column vector Y with the length of N;
where · is the matrix multiplication. The above formula is the fluorescence intensity column vector Y and the observation matrixFor the purpose of knowing, an underdetermined equation solving algorithm is adopted to obtain an approximate value X' of X. Reconstructing the one-dimensional vector X' into a two-dimensional image matrix S, wherein S is a sample image; because the observation matrix is acquired by the CCD, the observation matrix can be accurately acquired for the single-waveband system of fluorescence imagingInformation, and the CCD collects an observation matrix due to the existence of various illumination conditions such as scattering, reflection, excitation and the like in a visible light all-band systemCannot be accurately used as recovery information, so the method can only be applied to multispectral imaging.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A single-pixel multispectral calculation imaging system based on bundled fibers is characterized by comprising a fiber spectrometer, multimode fibers (1-6), a fiber coupling light source group (1-7), a second collimating lens (1-8), a beam splitter (1-9), an objective lens (1-10), a sample table (1-11), a second color filter (1-12), an image sensor (1-13), signal processing equipment (1-14) and a computer (1-15);
the optical fiber coupling light source group (1-7) sequentially emits exciting light according to the fixed sequence, each exciting light is emitted out through the multimode optical fiber (1-6), is collimated by the second collimating lens (1-8), passes through the beam splitter (1-9) and is focused and irradiated on a sample on the sample table (1-11) through the objective lens (1-10), and structured light with a spatial distribution form is formed;
the sample emits fluorescence under the excitation of the structural light; after being focused by the objective lens (1-10), the fluorescence is transmitted and reflected at the beam splitter (1-9), part of the fluorescence is reflected and penetrates through the second color filter (1-12) and then is detected on the image sensor (1-13), and the other part of the fluorescence is transmitted and then is converged into the multimode optical fiber (1-6) through the second collimating lens (1-8); the fluorescence entering the multimode optical fibers (1-6) is emitted from the corresponding optical fiber bundle and is detected by an optical fiber spectrometer; the signal processing equipment (1-14) carries out photoelectric conversion on the fluorescence signals collected by the fiber spectrometer and the image sensor (1-13) and then transmits the fluorescence signals to the computer (1-15).
2. The single-pixel multispectral computational imaging system based on bundled fibers according to claim 1, wherein the fiber spectrometer is replaced by a point or array single-photon detector (1-1), an aperture stop (1-2), a converging lens (1-3), a first color filter (1-4) and a first collimating lens (1-5);
the fluorescence entering the multimode optical fiber (1-6) is emitted from the corresponding optical fiber bundle, is collimated by the first collimating lens (1-5), is filtered by the first color filter (1-4), is converged by the converging lens (1-3), and is detected by the point or array single-photon detector (1-1) after being spatially filtered by the aperture diaphragm (1-2); the signal processing equipment (1-14) performs photoelectric conversion on fluorescence signals collected by the point or array single photon detector (1-1) and the image sensor (1-13) and then transmits the fluorescence signals to the computer (1-15).
3. The single-pixel multi-spectral computational imaging system based on bundled optical fibers according to claim 2, characterized in that the point or array single photon detectors (1-1) employ photomultiplier tubes or avalanche diodes.
4. The single pixel multi-spectral computational imaging system based on bundled fibers according to claim 1 or 2, wherein the multimode fibers (1-6) use one-turn multi-fanout fiber bundles, the fanout number being greater than or equal to 7.
5. The single-pixel multispectral computational imaging system based on bundled fibers according to claim 2, wherein the fiber-coupled light source groups (1-7) use modulatable light sources, the intensity and the on-off state of the light sources are controlled by duty ratio modulation, and the number of the light sources is equal to the number of fan-outs of the multimode fibers (1-6) minus the number of point or array single-photon detectors (1-1).
6. The single-pixel multi-spectral computational imaging system based on bundled fibers according to claim 1 or 5, wherein the fiber-coupled light source groups (1-7) employ laser light sources or LED light sources.
7. The bundled fiber-based single-pixel multi-spectral computational imaging system according to claim 1 or 2, wherein the image sensors (1-13) employ charge-coupled devices or complementary metal oxide semiconductors.
8. An imaging method, wherein the bundled-fiber-based single-pixel multi-spectral computational imaging system of claim 1 is used, comprising the steps of:
step 1, placing a uniform standard fluorescent sample on a sample table (1-11), forming different excitation light combinations by using different on-off states and output power conditions of light sources in an optical fiber coupling light source group (1-7), forming a fixed sequence with the length of N by using the different excitation light combinations, sequentially emitting excitation light by the optical fiber coupling light source group (1-7) according to the fixed sequence, enabling the excitation light to be emitted out through a multimode optical fiber (1-6) every time, collimating the excitation light by a second collimating lens (1-8), passing through a beam splitter (1-9), and then focusing and irradiating the sample on the sample table (1-11) through an objective lens (1-10) to form structured light with a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; the fluorescence is focused by an objective lens (1-10), reflected at a beam splitter (1-9), penetrates through a second color filter (1-12), and is detected on an image sensor (1-13) to form a two-dimensional image corresponding to the secondary excitation light, the two-dimensional image is imaged and transmitted to a computer (1-15) through a signal processing device (1-14) to be reconstructed into a row vector, the length of the row vector is the total pixel number M of the image, and the row vector corresponding to the combination of all the excitation lights is recorded as an observation matrix phi with N rows and M columns according to the emission sequence;
step 2, placing a target sample on a sample table (1-11), sequentially emitting exciting light by an optical fiber coupling light source group (1-7) according to a fixed sequence, wherein each time of exciting light is emitted out through a multimode optical fiber (1-6), collimated by a second collimating lens (1-8), passes through a beam splitter (1-9), and then is focused and irradiated on the sample table (1-11) through an objective lens (1-10) to form structured light with a space distribution form; the sample emits fluorescence under the excitation of the structural light; after being focused by the objective lens (1-10), the fluorescence is transmitted at the beam splitter (1-9), is converged by the second collimating lens (1-8) and enters the multimode optical fiber (1-6), the fluorescence entering the multimode optical fiber (1-6) is emitted from the corresponding optical fiber bundle, the fluorescence is detected on the optical fiber spectrometer to form a fluorescence intensity value corresponding to the excitation light, and the fluorescence intensity values corresponding to all the excitation light combinations are recorded as a column vector Y of N rows according to the emission sequence;
and 3, recording a column vector reconstructed by the sample image to be restored as X, wherein the column vector Y and the observation matrix phi and X satisfy the following relation:
Y=φ·X
where · is a matrix multiplication;
solving an approximate value X' of the X by adopting an underdetermined equation;
and reversely reconstructing the X' into a two-dimensional image matrix S, and taking the S as a sample image.
9. The imaging method according to claim 8, characterized in that the fiber optic spectrometer is replaced by an integral body consisting of a point or array single photon detector (1-1), an aperture stop (1-2), a converging lens (1-3), a first color filter (1-4) and a first collimating lens (1-5);
the fluorescence entering the multimode optical fiber (1-6) is emitted from the corresponding optical fiber bundle, is collimated by the first collimating lens (1-5), is filtered by the first color filter (1-4), is converged by the converging lens (1-3), and is detected by the point or array single-photon detector (1-1) after being spatially filtered by the aperture diaphragm (1-2); the signal processing equipment (1-14) performs photoelectric conversion on fluorescence signals collected by the point or array single photon detector (1-1) and the image sensor (1-13) and then transmits the fluorescence signals to the computer (1-15).
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911115073.4A CN110793633B (en) | 2019-11-14 | 2019-11-14 | Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911115073.4A CN110793633B (en) | 2019-11-14 | 2019-11-14 | Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110793633A CN110793633A (en) | 2020-02-14 |
CN110793633B true CN110793633B (en) | 2020-10-27 |
Family
ID=69444583
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911115073.4A Active CN110793633B (en) | 2019-11-14 | 2019-11-14 | Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110793633B (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111272725A (en) * | 2020-04-03 | 2020-06-12 | 吴建明 | Hazardous article on-site detector based on fluorescence spectrum analysis |
CN111698405B (en) * | 2020-06-05 | 2021-09-03 | 中国航空工业集团公司洛阳电光设备研究所 | Parallel compression sensing imaging device |
CN111721720B (en) * | 2020-06-29 | 2023-09-01 | 北京深测科技有限公司 | Spectral imaging system |
CN111721718B (en) * | 2020-06-29 | 2023-09-01 | 北京深测科技有限公司 | Spectral imaging method and system |
CN112904362A (en) * | 2021-01-18 | 2021-06-04 | 中山大学 | Single photon detection imaging integrated load system and control method |
CN113063744A (en) * | 2021-03-22 | 2021-07-02 | 华东理工大学 | Micro-fluidic device and imaging detection device |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103115680B (en) * | 2013-01-24 | 2014-11-12 | 中国科学院空间科学与应用研究中心 | Super sensitive spectrograph and spectrum detection method |
CN103340601A (en) * | 2013-06-27 | 2013-10-09 | 中国科学院自动化研究所 | Multi-spectral imaging system and method based on endoscope |
CN104678716B (en) * | 2015-02-09 | 2017-05-03 | 中国科学院半导体研究所 | Laser direct writing photolithographic system combined with single quantum dot locating function and method thereof |
WO2018005623A1 (en) * | 2016-06-28 | 2018-01-04 | The Regents Of The University Of California | Fast two-photon imaging by diffracted swept-laser excitation |
GB201707239D0 (en) * | 2017-05-05 | 2017-06-21 | Univ Edinburgh | Optical system and method |
CN110389121A (en) * | 2019-07-24 | 2019-10-29 | 中国科学院半导体研究所 | Multifocal Structured Illumination fluoroscopic imaging systems |
-
2019
- 2019-11-14 CN CN201911115073.4A patent/CN110793633B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN110793633A (en) | 2020-02-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110793633B (en) | Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers | |
US20230349812A1 (en) | Fluorescence Imaging Flow Cytometry With Enhanced Image Resolution | |
Han et al. | Imaging technologies for flow cytometry | |
CN108474728B (en) | Multi-modality fluorescence imaging flow cytometry system | |
US7046359B2 (en) | System and method for dynamic chemical imaging | |
CN104054266B (en) | A kind of time resolution single photon or pole low light level multiplanar imaging spectroscopic system and method | |
EP0449597B1 (en) | Highly directional optical system and optical sectional image forming apparatus employing the same | |
US20130126755A1 (en) | Method and device for simultaneous multi-channel and multi-method acquisition of synchronized parameters in cross-system fluorescence lifetime applications | |
CN103091297B (en) | Super-resolution microscope method and device based on random fluorescence bleaching | |
Brydegaard et al. | Versatile multispectral microscope based on light emitting diodes | |
CN113624666B (en) | Stream type imaging system based on dot matrix laser scanning | |
WO2015164844A1 (en) | Super resolution microscopy | |
CN111971606B (en) | Time-resolved imaging method with high spatial resolution | |
US11231323B2 (en) | Time-resolved hyper-spectral single-pixel imaging | |
Rodríguez et al. | Dual-mode optical microscope based on single-pixel imaging | |
CA2571473A1 (en) | Method and apparatus for dark field chemical imaging | |
JP2011513740A (en) | Time-resolved spectroscopic analysis method and system using photon mixing detector | |
CN104614318A (en) | Rapid super-resolution micro-imaging method and device | |
US20110017915A1 (en) | Drift scanner for rare cell detection | |
US20130250088A1 (en) | Multi-color confocal microscope and imaging methods | |
Michalet et al. | High-throughput single-molecule fluorescence spectroscopy using parallel detection | |
CN116249891A (en) | Method for detecting emitted light, detection device and laser scanning microscope | |
EP3827278B1 (en) | Device and method for detecting time-resolved optical data | |
CN212410444U (en) | Image scanning microscopic imaging system | |
WO2022026053A1 (en) | Multiplexed sensor network using swept source raman spectroscopy |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |