CN110793633B - Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers - Google Patents

Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers Download PDF

Info

Publication number
CN110793633B
CN110793633B CN201911115073.4A CN201911115073A CN110793633B CN 110793633 B CN110793633 B CN 110793633B CN 201911115073 A CN201911115073 A CN 201911115073A CN 110793633 B CN110793633 B CN 110793633B
Authority
CN
China
Prior art keywords
optical fiber
fluorescence
light
sample
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911115073.4A
Other languages
Chinese (zh)
Other versions
CN110793633A (en
Inventor
李晓琼
樊云龙
钱成
杨春华
冷坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN201911115073.4A priority Critical patent/CN110793633B/en
Publication of CN110793633A publication Critical patent/CN110793633A/en
Application granted granted Critical
Publication of CN110793633B publication Critical patent/CN110793633B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G01J3/0205—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
    • G01J3/0218—Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using optical fibers
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G01J3/10—Arrangements of light sources specially adapted for spectrometry or colorimetry
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2803—Investigating the spectrum using photoelectric array detector
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/44—Raman spectrometry; Scattering spectrometry ; Fluorescence spectrometry
    • G01J3/4406—Fluorescence spectrometry
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01—Arrangements or apparatus for facilitating the optical investigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/6402—Atomic fluorescence; Laser induced fluorescence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • G01J3/10—Arrangements of light sources specially adapted for spectrometry or colorimetry
    • G01J2003/102—Plural sources
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G01J2003/2826—Multispectral imaging, e.g. filter imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01—Arrangements or apparatus for facilitating the optical investigation
    • G01N2021/0106—General arrangement of respective parts
    • G01N2021/0112—Apparatus in one mechanical, optical or electronic block
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N2021/6417—Spectrofluorimetric devices
    • G01N2021/6419—Excitation at two or more wavelengths
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64—Fluorescence; Phosphorescence
    • G01N21/645—Specially adapted constructive features of fluorimeters
    • G01N2021/6484—Optical fibres

Landscapes

  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Biochemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

The invention provides a single-pixel multispectral calculation imaging system and an imaging method based on bundled fibers, which can realize stronger anti-interference capability, have very few hardware consumption resources and have less requirements on texts. The invention provides an observation matrix which adopts the combination of a bundled optical fiber and an image sensor (CCD) to replace a DMD
Figure DDA0002273815150000011
A new method of variation. One branch of the bundled optical fibers is connected with the sensor to collect signals, and the other branches are connected with the modulatable light source. Different light intensity distributions are formed on the irradiated target by different on-off states of the light sources, and the light intensity distributions are observation matrixes
Figure DDA0002273815150000012
Since the light source can simply achieve a modulation of 10-100KHz, the frequency of change of the observation matrix can also reach this rate. After the system is built, the CCD is adopted to observe the matrix
Figure DDA0002273815150000013
And collecting and recording frame by frame, establishing a recovery model, and then not collecting in the actual observation process. The method can realize high-speed single-pixel imaging.

Description

Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers
Technical Field
The invention relates to the technical field of optical computational imaging, in particular to a single-pixel multispectral computational imaging system and a single-pixel multispectral computational imaging method based on bundled fibers.
Background
The current high-sensitivity rapid fluorescence detection technology mainly comprises a single molecule detection technology, a time resolution technology and a super-resolution measurement technology. Among the imaging techniques for high temporal resolution, Fluorescence Lifetime Imaging Microscopy (FLIM) is the technology used more in the field at present. However, the nano-displacement scanning platform contained in the nano-displacement scanning platform has poor stability and complex scanning process, not only the manufacturing cost is increased, but also the testing time of the nano-material and the biomacromolecule is greatly prolonged, so the success rate is also obviously influenced. Therefore, it is currently difficult to simultaneously perform a correlation spectral analysis that achieves high temporal resolution and an observed object.
Modern digital cameras capture images using an array of photodetectors, while single-pixel imaging reconstructs the image by sampling the scene using a series of masks and correlating the distribution of these masks to the corresponding intensities measured using a single-pixel detector. While not performing as well as digital cameras in traditional visible imaging, single-pixel imaging has proven advantageous in non-conventional applications.
As a new imaging mode, the single-pixel camera is a typical application for exploring data sparsity. The system uses a set of observation matrices
Figure GDA0002623882790000011
And the corresponding sampling value Y is used as algorithm input through a matrix relation
Figure GDA0002623882790000012
And solving the image X by adopting an underdetermined equation solving method.
In 2006, RICE university applied the compressed sensing theory to imaging systems, and proposed a single-pixel camera. The camera utilizes a digital micromirror array to perform linear sampling on an optical image, measures a sampling value through a single detector element, and reconstructs a target image by utilizing a reconstruction algorithm, thereby realizing single-pixel imaging. The single-pixel camera integrates image acquisition and compression, saves storage resources and transmission bandwidth, and is suitable for the field of non-visible light which cannot be shot by the traditional method. Fig. 1 is a schematic diagram of imaging of a single-pixel camera and an image restoration effect.
In 2014, a high-TIME-resolution multispectral SINGLE-PHOTON IMAGING algorithm based on a compressed sensing algorithm is provided in a patent "TIME-RESOLVED SINGLE-PHOTON OR ULTRA-WEAK LIGHT MULTI-DIMENSIONAL IMAGING SPECTRUM SYSTEM ANDMETHOD" of the research center of space science and application of the Chinese academy of sciences. The implementation method is as shown in fig. 2, after a light source 1 irradiates a sample, the light passes through an optical filter or an attenuation sheet 2, an optical imaging component 3, a Spatial Light Modulator (SLM) or a Digital Micromirror (DMD)4, a concave mirror 5, a grating light splitting element 6, a light converging and collecting component (multi-wavelength switchable) 7, an entry point or an array single photon detector 9, a multi-channel measuring instrument 10 and a high-precision time measuring instrument 11 process detector data, the data enters a data packet memory 14, and calculation is performed through a compressed sensing correlation algorithm 15. However, the system is based on the random matrix generation of compressed sensing by a Spatial Light Modulator (SLM) or a Digital Micromirror (DMD), and although the single-photon sensor can achieve the response rate of picosecond level, the system is limited by the response rate of the SLM and the DMD (about 100 Hz and 300 Hz).
At present, most of researches on a single-pixel camera adopt a Digital Micromirror Device (DMD) to set an observation matrix
Figure GDA0002623882790000021
And carrying out overturning change, and simultaneously collecting a corresponding sampling value Y by using a photomultiplier tube (PMT) or an avalanche diode (APD) of a high-sensitivity sensor. For high-speed imaging applications, factors related to the imaging frame frequency mainly include the observation matrix frequency, the imaging resolution and the image compression ratio, and the observation matrix frame frequency becomes a main factor restricting the imaging frame frequency when the imaging resolution and the image compression ratio are the same. At present, the frame frequency of a commonly used DMD is about 100-300Hz, and a 20KHz high frame frequency DMD is also provided, but the price is high, the control data flow is huge, and the use condition is high.
Disclosure of Invention
In view of this, the invention provides a single-pixel multispectral calculation imaging system and an imaging method based on bundled fibers, which can realize stronger anti-interference capability, have very few hardware consumption resources and have less requirements on texts.
In order to achieve the purpose, the invention provides a single-pixel multispectral calculation imaging system based on bundled optical fibers, which is characterized by comprising an optical fiber spectrometer, multimode optical fibers, an optical fiber coupling light source group, a second collimating lens, a beam splitter, an objective lens, a sample stage, a second color filter, an image sensor, signal processing equipment and a computer;
the optical fiber coupling light source group sequentially emits exciting light according to the fixed sequence, the exciting light of each time is emitted out through the multimode optical fiber, is collimated by the second collimating lens, passes through the beam splitter and is focused and irradiated on a sample on the sample stage through the objective lens, and structured light with a spatial distribution form is formed;
the sample emits fluorescence under the excitation of the structural light; after the fluorescence is focused by the objective lens, the fluorescence is transmitted and reflected at the beam splitter, part of the fluorescence is reflected and penetrates through the second color filter and then is detected on the image sensor, and the other part of the fluorescence is transmitted and then is converged into the multimode optical fiber through the second collimating lens; the fluorescence entering the multimode optical fiber is emitted from the corresponding optical fiber bundle and is detected by the optical fiber spectrometer; the signal processing equipment carries out photoelectric conversion on the fluorescence signals collected by the fiber spectrometer and the image sensor and then transmits the fluorescence signals to the computer.
The optical fiber spectrometer is replaced by a whole body consisting of a point or array single-photon detector, an aperture diaphragm, a converging lens, a first color filter and a first collimating lens;
the fluorescence entering the multimode fiber is emitted from the corresponding fiber bundle, is collimated by a first collimating lens, is filtered by a first color filter, is converged by a converging lens, and is detected by a point or array single-photon detector after being filtered by an aperture diaphragm space; the signal processing equipment carries out photoelectric conversion on the fluorescent signals collected by the point or array single-photon detector and the image sensor and then transmits the fluorescent signals to the computer.
The point or array single photon detector adopts a photomultiplier or an avalanche diode.
The multimode optical fiber adopts one-to-many fanout optical fiber bundles, and the fanout number is more than or equal to 7.
The optical fiber coupling light source group adopts a modulatable light source, the intensity and the opening and closing state of the light source are controlled through duty ratio modulation, and the number of the light sources is equal to the number of the fan-out of the multimode optical fibers minus the number of the point or array single photon detectors.
The optical fiber coupling light source group adopts a laser light source or an LED light source.
Among them, the image sensor employs a charge coupled device or a complementary metal oxide semiconductor.
The invention also provides an imaging method, which adopts the single-pixel multispectral calculation imaging system based on the bundled optical fiber and comprises the following steps:
step 1, placing a uniform standard fluorescent sample on a sample table, forming different excitation light combinations by each light source in an optical fiber coupling light source group under different opening and closing states and output power conditions, forming a fixed sequence with the length of N by the different excitation light combinations, sequentially emitting excitation light by the optical fiber coupling light source group according to the fixed sequence, emitting the excitation light through a multimode optical fiber each time, collimating the excitation light through a second collimating lens, passing through a beam splitter, focusing and irradiating the excitation light on the sample table through an objective lens, and forming structured light with a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; the fluorescence is focused by the objective lens, reflected at the beam splitter, penetrates through the second color filter and is detected on the image sensor to form a two-dimensional image corresponding to the secondary exciting light, the two-dimensional image is imaged and transmitted to a computer through signal processing equipment to be reconstructed into a row vector, the length of the row vector is the total pixel number M of the image, and the row vector corresponding to all exciting light combinations is recorded as an observation matrix phi with N rows and M columns according to the emission sequence;
step 2, placing a target sample on a sample stage, emitting exciting light by the optical fiber coupling light source group according to a fixed sequence in sequence, wherein each time the exciting light is emitted by the multimode optical fiber, is collimated by the second collimating lens, passes through the beam splitter and is focused and irradiated on the sample stage by the objective lens to form structured light with a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; after the fluorescence is focused by the objective lens, the fluorescence is transmitted at the beam splitter, the fluorescence is converged by the second collimating lens to enter the multimode optical fiber, the fluorescence entering the multimode optical fiber is emitted from the corresponding optical fiber bundle, the fluorescence intensity value corresponding to the exciting light is detected on the optical fiber spectrometer, and the fluorescence intensity value corresponding to all exciting light combinations is recorded as a column vector Y of N rows according to the emission sequence;
and 3, recording a column vector reconstructed by the sample image to be restored as X, wherein the column vector Y and the observation matrix phi and X satisfy the following relation:
Y＝φ·X
where · is a matrix multiplication;
solving an approximate value X' of the X by adopting an underdetermined equation;
and reversely reconstructing the X' into a two-dimensional image matrix S, and taking the S as a sample image.
The optical fiber spectrometer is replaced by a whole body consisting of a point or array single-photon detector, an aperture diaphragm, a converging lens, a first color filter and a first collimating lens;
the fluorescence entering the multimode fiber is emitted from the corresponding fiber bundle, is collimated by a first collimating lens, is filtered by a first color filter, is converged by a converging lens, and is detected by a point or array single-photon detector after being filtered by an aperture diaphragm space; the signal processing equipment carries out photoelectric conversion on the fluorescent signals collected by the point or array single-photon detector and the image sensor and then transmits the fluorescent signals to the computer.
Has the advantages that:
the invention provides an observation matrix which adopts the combination of a bundled optical fiber and an image sensor (CCD) to replace a DMD
Figure GDA0002623882790000051
A new method of variation. One branch of the bundled optical fibers is connected with the sensor to collect signals, and the other branches are connected with the modulatable light source. Different light intensity distributions are formed on the irradiated target by different on-off states of the light sources, and the light intensity distributions are observation matrixes
Figure GDA0002623882790000053
The observation matrix is simple in that the light source can simply realize a modulation of 10-100KHzThe rate may also be reached by varying the frequency. After the system is built, the CCD is adopted to observe the matrix
Figure GDA0002623882790000052
And collecting and recording frame by frame, establishing a recovery model, and then not collecting in the actual observation process. The method can realize high-speed single-pixel imaging.
Drawings
FIG. 1 is a schematic diagram of a single-pixel camera and a recovery result;
FIG. 2 is a time-resolved single-photon or ultra-low-light multi-dimensional imaging spectroscopy system;
FIG. 3 is a schematic diagram of a single-pixel multi-spectral computational imaging system based on bundled fibers according to the present invention.
The system comprises a 1-1-point or array single photon detector, a 1-2-aperture diaphragm, a 1-3-convergent lens, a 1-4-first color filter (switchable by a rotating wheel), a 1-5-first collimating lens, a 1-6-multimode optical fiber, a 1-7-optical fiber coupling light source group (switchable by multiple wavelengths), a 1-8-second collimating lens, a 1-9-beam splitter, a 1-10-objective lens, a 1-11-sample stage, a 1-12-second color filter (switchable by a rotating wheel), a 1-13-image sensor, a 1-14-signal processing device and a 1-15-computer.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The frequency of single frame turnover of DMD (digital micromirror) is generally 100-200Hz, and the frame frequency of common images is 30-50Hz through 5-10 picture reconstruction. The bandwidth of a single-pixel sensor such as a PMT (photomultiplier tube) or an APD (avalanche photo diode) can reach 10KHz-100KHz, the bandwidth range can also be reached after the light source is modulated, 3KHz imaging and modulation rate are adopted in the current embodiment, 64 pictures are adopted for reconstruction in an algorithm, and the frame frequency can reach 470 Hz. The high-speed fluorescence imaging method has better application prospect in the fields of biological observation, flow cytometry and the like.
Based on the basis, the invention adopts the fiber spectrometer to replace a point or array single photon detector, and can realize continuous spectrum imaging.
As shown in FIG. 3, the single-pixel multispectral computational imaging system based on bundled fibers of the invention comprises: the system comprises an optical fiber spectrometer, multimode optical fibers 1-6, an optical fiber coupling light source group (switchable between multiple wavelengths) 1-7, second collimating lenses 1-8, beam splitters 1-9, objective lenses 1-10, sample stages 1-11, second color filters (switchable by rotating wheels) 1-12, image sensors 1-13, signal processing equipment 1-14 and a computer 1-15;
changing the on-off state and intensity of each light source in the optical fiber coupling light source group (switchable among multiple wavelengths) 1-7, and forming exciting light with different spatial distribution forms on the sample; collecting exciting light with different space distribution forms through an image sensor 1-13, and constructing a recovery model in a computer 1-15; the fluorescence intensity is collected by the point or array single photon detector 1-1 and substituted into the recovery model, and then the microscopic imaging recovery can be completed.
Specifically, the fiber-coupled light source groups (multi-wavelength switchable) 1-7 emit excitation light according to a fixed sequence; exciting light is emitted out through the multimode optical fiber 1-6, collimated through the second collimating lens 1-8, passes through the beam splitter 1-9, and is focused and irradiated on a sample on the sample stage 1-11 through the objective lens 1-10 to form structured light in a space distribution form; the sample emits fluorescence under the excitation of the structural light; after being focused by the objective lens 1-10, the fluorescence is transmitted and reflected at the beam splitter 1-9, part of the fluorescence is reflected and penetrates through the second color filter (which can be switched by a rotating wheel) 1-12 and then is detected on the image sensor 1-13, and the other part of the fluorescence is transmitted and then is converged by the second collimating lens 1-8 to enter the multimode optical fiber 1-6;
the fluorescence entering the multimode optical fibers 1-6 is emitted from the corresponding optical fiber bundle and is detected by the optical fiber spectrometer; the signal processing equipment 1-14 carries out photoelectric conversion on the fluorescence signals collected by the fiber spectrometer and the image sensor 1-13 and then transmits the fluorescence signals to the computer 1-15.
In order to realize better sensitivity, the fiber spectrometer can be replaced by a point or array single photon detector 1-1, an aperture diaphragm 1-2, a converging lens 1-3, a first color filter 1-4 (switchable by a rotating wheel) and a first collimating lens 1-5. The fluorescence entering the multimode optical fiber 1-6 is emitted from the corresponding optical fiber bundle, is collimated by a first collimating lens 1-5, is filtered by a first color filter (which can be switched by a rotating wheel) 1-4, is converged by a converging lens 1-3, and is detected by a point or array single-photon detector 1-1 after being spatially filtered by an aperture diaphragm 1-2; the signal processing equipment 1-14 carries out photoelectric conversion on the fluorescent signals collected by the point or array single photon detector 1-1 and the image sensor 1-13 and then transmits the signals to the computer 1-15.
The point or array single photon detector 1-1 adopts a photomultiplier tube (PMT) or an avalanche diode (APD).
The multimode fibers 1-6 adopt one-to-many fanout fiber bundles, and the fanout number is more than or equal to 1-7.
The optical fiber coupling light source group (switchable among multiple wavelengths) 1-7 adopts light sources which can be modulated at high speed, such as laser light sources, LED light sources and the like, the intensity and the open-close state of the light sources are controlled through duty ratio modulation, and the number of the light sources is equal to the number of the fan-out number of the multimode optical fibers 1-6 minus the number of the point or array single photon detectors 1-1.
The image sensors 1 to 13 employ Charge Coupled Devices (CCDs) or Complementary Metal Oxide Semiconductors (CMOS).
The invention also provides an imaging method of the single-pixel multispectral calculation imaging system based on the bundled optical fibers, which comprises the following steps:
step 1, placing uniform standard fluorescent samples on sample tables 1-11, and forming different combinations of all light sources in optical fiber coupling light source groups (switchable among multiple wavelengths) 1-7 according to different opening and closing states and output power conditions to form a fixed sequence, wherein the sequence length is N. The optical fiber coupling light source group (switchable among multiple wavelengths) 1-7 emits exciting light according to a fixed sequence, the exciting light is emitted out through the multimode optical fiber 1-6, is collimated by the second collimating lens 1-8, passes through the beam splitter 1-9 and then is focused and irradiated on a sample on the sample stage 1-11 through the objective lens 1-10, and structured light with a spatial distribution form is formed; the sample emits fluorescence under the excitation of the structural light; after being focused by an objective lens 1-10, fluorescence is reflected at a beam splitter 1-9, is detected on an image sensor 1-13 after penetrating through a second color filter (capable of being switched by a rotating wheel) 1-12, is imaged and transmitted to a computer 1-15 through a signal processing device 1-14, the two-dimensional image is recorded as one line in a two-dimensional observation matrix after being reconstructed, the length of the line is the total pixel number M of the image, and all fluorescence images are imaged and stored in the computer 1-15 through the image sensor 1-13 and are reconstructed into the observation matrix
Figure GDA0002623882790000085
The matrix is composed of N rows and M columns;
step 2, placing a target sample on a sample table 1-11, emitting exciting light by an optical fiber coupling light source group (switchable among multiple wavelengths) 1-7 according to a fixed sequence, emitting the exciting light through a multimode optical fiber 1-6, collimating the exciting light by a second collimating lens 1-8, passing through a beam splitter 1-9, and then focusing and irradiating the sample on the sample table 1-11 through an objective lens 1-10 to form structured light in a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; after being focused by an objective lens 1-10, fluorescence is transmitted at a beam splitter 1-9, is converged by a second collimating lens 1-8 and enters a multimode optical fiber 1-6, the fluorescence entering the multimode optical fiber 1-6 is emitted from a corresponding optical fiber beam, is collimated by a first collimating lens 1-5, is filtered by a first color filter (which can be switched by a rotating wheel) 1-4, is converged by a converging lens 1-3, and is detected by a point or array single photon detector 1-1 after being spatially filtered by an aperture diaphragm 1-2; collecting the point or array single photon detector 1-1 according to a fixed sequence to obtain a fluorescence intensity column vector Y with the length of N;
step 3, recording the column vector reconstruction form of the two-dimensional image matrix to be restored as X, and then recording the fluorescence intensity column vector Y and the observation matrix
Figure GDA0002623882790000081
Satisfies the following relationship with the image to be restored X:
Figure GDA0002623882790000086
where · is the matrix multiplication. The above formula is the fluorescence intensity column vector Y and the observation matrix
Figure GDA0002623882790000082
For the purpose of knowing, an underdetermined equation solving algorithm is adopted to obtain an approximate value X' of X. Reconstructing the one-dimensional vector X' into a two-dimensional image matrix S, wherein S is a sample image; because the observation matrix is acquired by the CCD, the observation matrix can be accurately acquired for the single-waveband system of fluorescence imaging
Figure GDA0002623882790000083
Information, and the CCD collects an observation matrix due to the existence of various illumination conditions such as scattering, reflection, excitation and the like in a visible light all-band system
Figure GDA0002623882790000084
Cannot be accurately used as recovery information, so the method can only be applied to multispectral imaging.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. A single-pixel multispectral calculation imaging system based on bundled fibers is characterized by comprising a fiber spectrometer, multimode fibers (1-6), a fiber coupling light source group (1-7), a second collimating lens (1-8), a beam splitter (1-9), an objective lens (1-10), a sample table (1-11), a second color filter (1-12), an image sensor (1-13), signal processing equipment (1-14) and a computer (1-15);
the optical fiber coupling light source group (1-7) sequentially emits exciting light according to the fixed sequence, each exciting light is emitted out through the multimode optical fiber (1-6), is collimated by the second collimating lens (1-8), passes through the beam splitter (1-9) and is focused and irradiated on a sample on the sample table (1-11) through the objective lens (1-10), and structured light with a spatial distribution form is formed;
the sample emits fluorescence under the excitation of the structural light; after being focused by the objective lens (1-10), the fluorescence is transmitted and reflected at the beam splitter (1-9), part of the fluorescence is reflected and penetrates through the second color filter (1-12) and then is detected on the image sensor (1-13), and the other part of the fluorescence is transmitted and then is converged into the multimode optical fiber (1-6) through the second collimating lens (1-8); the fluorescence entering the multimode optical fibers (1-6) is emitted from the corresponding optical fiber bundle and is detected by an optical fiber spectrometer; the signal processing equipment (1-14) carries out photoelectric conversion on the fluorescence signals collected by the fiber spectrometer and the image sensor (1-13) and then transmits the fluorescence signals to the computer (1-15).
2. The single-pixel multispectral computational imaging system based on bundled fibers according to claim 1, wherein the fiber spectrometer is replaced by a point or array single-photon detector (1-1), an aperture stop (1-2), a converging lens (1-3), a first color filter (1-4) and a first collimating lens (1-5);
the fluorescence entering the multimode optical fiber (1-6) is emitted from the corresponding optical fiber bundle, is collimated by the first collimating lens (1-5), is filtered by the first color filter (1-4), is converged by the converging lens (1-3), and is detected by the point or array single-photon detector (1-1) after being spatially filtered by the aperture diaphragm (1-2); the signal processing equipment (1-14) performs photoelectric conversion on fluorescence signals collected by the point or array single photon detector (1-1) and the image sensor (1-13) and then transmits the fluorescence signals to the computer (1-15).
3. The single-pixel multi-spectral computational imaging system based on bundled optical fibers according to claim 2, characterized in that the point or array single photon detectors (1-1) employ photomultiplier tubes or avalanche diodes.
4. The single pixel multi-spectral computational imaging system based on bundled fibers according to claim 1 or 2, wherein the multimode fibers (1-6) use one-turn multi-fanout fiber bundles, the fanout number being greater than or equal to 7.
5. The single-pixel multispectral computational imaging system based on bundled fibers according to claim 2, wherein the fiber-coupled light source groups (1-7) use modulatable light sources, the intensity and the on-off state of the light sources are controlled by duty ratio modulation, and the number of the light sources is equal to the number of fan-outs of the multimode fibers (1-6) minus the number of point or array single-photon detectors (1-1).
6. The single-pixel multi-spectral computational imaging system based on bundled fibers according to claim 1 or 5, wherein the fiber-coupled light source groups (1-7) employ laser light sources or LED light sources.
7. The bundled fiber-based single-pixel multi-spectral computational imaging system according to claim 1 or 2, wherein the image sensors (1-13) employ charge-coupled devices or complementary metal oxide semiconductors.
8. An imaging method, wherein the bundled-fiber-based single-pixel multi-spectral computational imaging system of claim 1 is used, comprising the steps of:
step 1, placing a uniform standard fluorescent sample on a sample table (1-11), forming different excitation light combinations by using different on-off states and output power conditions of light sources in an optical fiber coupling light source group (1-7), forming a fixed sequence with the length of N by using the different excitation light combinations, sequentially emitting excitation light by the optical fiber coupling light source group (1-7) according to the fixed sequence, enabling the excitation light to be emitted out through a multimode optical fiber (1-6) every time, collimating the excitation light by a second collimating lens (1-8), passing through a beam splitter (1-9), and then focusing and irradiating the sample on the sample table (1-11) through an objective lens (1-10) to form structured light with a spatial distribution form; the sample emits fluorescence under the excitation of the structural light; the fluorescence is focused by an objective lens (1-10), reflected at a beam splitter (1-9), penetrates through a second color filter (1-12), and is detected on an image sensor (1-13) to form a two-dimensional image corresponding to the secondary excitation light, the two-dimensional image is imaged and transmitted to a computer (1-15) through a signal processing device (1-14) to be reconstructed into a row vector, the length of the row vector is the total pixel number M of the image, and the row vector corresponding to the combination of all the excitation lights is recorded as an observation matrix phi with N rows and M columns according to the emission sequence;
step 2, placing a target sample on a sample table (1-11), sequentially emitting exciting light by an optical fiber coupling light source group (1-7) according to a fixed sequence, wherein each time of exciting light is emitted out through a multimode optical fiber (1-6), collimated by a second collimating lens (1-8), passes through a beam splitter (1-9), and then is focused and irradiated on the sample table (1-11) through an objective lens (1-10) to form structured light with a space distribution form; the sample emits fluorescence under the excitation of the structural light; after being focused by the objective lens (1-10), the fluorescence is transmitted at the beam splitter (1-9), is converged by the second collimating lens (1-8) and enters the multimode optical fiber (1-6), the fluorescence entering the multimode optical fiber (1-6) is emitted from the corresponding optical fiber bundle, the fluorescence is detected on the optical fiber spectrometer to form a fluorescence intensity value corresponding to the excitation light, and the fluorescence intensity values corresponding to all the excitation light combinations are recorded as a column vector Y of N rows according to the emission sequence;
and 3, recording a column vector reconstructed by the sample image to be restored as X, wherein the column vector Y and the observation matrix phi and X satisfy the following relation:
Y＝φ·X
where · is a matrix multiplication;
solving an approximate value X' of the X by adopting an underdetermined equation;
and reversely reconstructing the X' into a two-dimensional image matrix S, and taking the S as a sample image.
9. The imaging method according to claim 8, characterized in that the fiber optic spectrometer is replaced by an integral body consisting of a point or array single photon detector (1-1), an aperture stop (1-2), a converging lens (1-3), a first color filter (1-4) and a first collimating lens (1-5);
the fluorescence entering the multimode optical fiber (1-6) is emitted from the corresponding optical fiber bundle, is collimated by the first collimating lens (1-5), is filtered by the first color filter (1-4), is converged by the converging lens (1-3), and is detected by the point or array single-photon detector (1-1) after being spatially filtered by the aperture diaphragm (1-2); the signal processing equipment (1-14) performs photoelectric conversion on fluorescence signals collected by the point or array single photon detector (1-1) and the image sensor (1-13) and then transmits the fluorescence signals to the computer (1-15).
CN201911115073.4A 2019-11-14 2019-11-14 Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers Active CN110793633B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911115073.4A CN110793633B (en) 2019-11-14 2019-11-14 Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911115073.4A CN110793633B (en) 2019-11-14 2019-11-14 Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers

Publications (2)

Publication Number Publication Date
CN110793633A CN110793633A (en) 2020-02-14
CN110793633B true CN110793633B (en) 2020-10-27

Family

ID=69444583

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911115073.4A Active CN110793633B (en) 2019-11-14 2019-11-14 Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers

Country Status (1)

Country Link
CN (1) CN110793633B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111272725A (en) * 2020-04-03 2020-06-12 吴建明 Hazardous article on-site detector based on fluorescence spectrum analysis
CN111698405B (en) * 2020-06-05 2021-09-03 中国航空工业集团公司洛阳电光设备研究所 Parallel compression sensing imaging device
CN111721720B (en) * 2020-06-29 2023-09-01 北京深测科技有限公司 Spectral imaging system
CN111721718B (en) * 2020-06-29 2023-09-01 北京深测科技有限公司 Spectral imaging method and system
CN112904362A (en) * 2021-01-18 2021-06-04 中山大学 Single photon detection imaging integrated load system and control method
CN113063744A (en) * 2021-03-22 2021-07-02 华东理工大学 Micro-fluidic device and imaging detection device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103115680B (en) * 2013-01-24 2014-11-12 中国科学院空间科学与应用研究中心 Super sensitive spectrograph and spectrum detection method
CN103340601A (en) * 2013-06-27 2013-10-09 中国科学院自动化研究所 Multi-spectral imaging system and method based on endoscope
CN104678716B (en) * 2015-02-09 2017-05-03 中国科学院半导体研究所 Laser direct writing photolithographic system combined with single quantum dot locating function and method thereof
WO2018005623A1 (en) * 2016-06-28 2018-01-04 The Regents Of The University Of California Fast two-photon imaging by diffracted swept-laser excitation
GB201707239D0 (en) * 2017-05-05 2017-06-21 Univ Edinburgh Optical system and method
CN110389121A (en) * 2019-07-24 2019-10-29 中国科学院半导体研究所 Multifocal Structured Illumination fluoroscopic imaging systems

Also Published As

Publication number Publication date
CN110793633A (en) 2020-02-14

Similar Documents

Publication Publication Date Title
CN110793633B (en) Single-pixel multispectral calculation imaging system and imaging method based on bundled optical fibers
US20230349812A1 (en) Fluorescence Imaging Flow Cytometry With Enhanced Image Resolution
Han et al. Imaging technologies for flow cytometry
CN108474728B (en) Multi-modality fluorescence imaging flow cytometry system
US7046359B2 (en) System and method for dynamic chemical imaging
CN104054266B (en) A kind of time resolution single photon or pole low light level multiplanar imaging spectroscopic system and method
EP0449597B1 (en) Highly directional optical system and optical sectional image forming apparatus employing the same
US20130126755A1 (en) Method and device for simultaneous multi-channel and multi-method acquisition of synchronized parameters in cross-system fluorescence lifetime applications
CN103091297B (en) Super-resolution microscope method and device based on random fluorescence bleaching
Brydegaard et al. Versatile multispectral microscope based on light emitting diodes
CN113624666B (en) Stream type imaging system based on dot matrix laser scanning
WO2015164844A1 (en) Super resolution microscopy
CN111971606B (en) Time-resolved imaging method with high spatial resolution
US11231323B2 (en) Time-resolved hyper-spectral single-pixel imaging
Rodríguez et al. Dual-mode optical microscope based on single-pixel imaging
CA2571473A1 (en) Method and apparatus for dark field chemical imaging
JP2011513740A (en) Time-resolved spectroscopic analysis method and system using photon mixing detector
CN104614318A (en) Rapid super-resolution micro-imaging method and device
US20110017915A1 (en) Drift scanner for rare cell detection
US20130250088A1 (en) Multi-color confocal microscope and imaging methods
Michalet et al. High-throughput single-molecule fluorescence spectroscopy using parallel detection
CN116249891A (en) Method for detecting emitted light, detection device and laser scanning microscope
EP3827278B1 (en) Device and method for detecting time-resolved optical data
CN212410444U (en) Image scanning microscopic imaging system
WO2022026053A1 (en) Multiplexed sensor network using swept source raman spectroscopy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant