CN116609305A - Linear illumination modulation multicolor imaging system - Google Patents
Linear illumination modulation multicolor imaging system Download PDFInfo
- Publication number
- CN116609305A CN116609305A CN202310411763.4A CN202310411763A CN116609305A CN 116609305 A CN116609305 A CN 116609305A CN 202310411763 A CN202310411763 A CN 202310411763A CN 116609305 A CN116609305 A CN 116609305A
- Authority
- CN
- China
- Prior art keywords
- image
- light
- images
- mixed
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005286 illumination Methods 0.000 title claims abstract description 78
- 238000003384 imaging method Methods 0.000 title claims abstract description 77
- 230000003287 optical effect Effects 0.000 claims description 29
- 230000008030 elimination Effects 0.000 claims description 21
- 238000003379 elimination reaction Methods 0.000 claims description 21
- 238000013519 translation Methods 0.000 claims description 12
- 238000007493 shaping process Methods 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 7
- 238000012546 transfer Methods 0.000 claims description 6
- 238000004091 panning Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 17
- 238000000034 method Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000009826 distribution Methods 0.000 description 7
- 238000000799 fluorescence microscopy Methods 0.000 description 7
- 238000006073 displacement reaction Methods 0.000 description 6
- 102000034287 fluorescent proteins Human genes 0.000 description 6
- 108091006047 fluorescent proteins Proteins 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000012632 fluorescent imaging Methods 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 238000000295 emission spectrum Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 210000001519 tissue Anatomy 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000003463 organelle Anatomy 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0071—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by measuring fluorescence emission
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0073—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by tomography, i.e. reconstruction of 3D images from 2D projections
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/6428—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0032—Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0064—Optical details of the image generation multi-spectral or wavelength-selective arrangements, e.g. wavelength fan-out, chromatic profiling
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/008—Details of detection or image processing, including general computer control
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/6428—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
- G01N2021/6439—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" with indicators, stains, dyes, tags, labels, marks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N2021/6463—Optics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Engineering & Computer Science (AREA)
- Optics & Photonics (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Immunology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biochemistry (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Chemical Kinetics & Catalysis (AREA)
- General Engineering & Computer Science (AREA)
- Radiology & Medical Imaging (AREA)
- Theoretical Computer Science (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
The application discloses a line illumination modulation multicolor imaging system, which belongs to the technical field of multicolor imaging and comprises: the line illumination modulation module comprises m monochromatic light sources and a modulation light path, and the wavelength of light beams emitted by each monochromatic light source is different; each light beam passes through a modulation light path to form a linear light spot focused on the focal plane of the objective lens, the illumination intensity of the linear light spot on the focal plane of the objective lens is Gaussian distributed in a first direction, the position parameters of each linear light spot are different, and the linear light spots of m channels are overlapped to form m-color linear illumination light; the imaging module is used for continuously scanning and imaging along a first direction by adopting a multi-element detector with n rows of pixels to obtain mixed images under illumination of at least m Zhang Zaixian illumination, each mixed image corresponds to one row of pixels, and each mixed image contains signals of m channels; and the image demodulation module is used for demodulating monochromatic images from the mixed images, wherein each monochromatic image corresponds to a signal of one channel. The present application enables simultaneous polychromatic imaging on a single detector.
Description
Technical Field
The application relates to the technical field of multicolor imaging, in particular to a line illumination modulation multicolor imaging system.
Background
Different structures in biological tissues are selectively marked by using fluorophores with different colors, multicolor fluorescent signals are obtained simultaneously by using multicolor microscopic imaging technology, and the spatial relationship and interaction among cells, organelles and molecules in the biological tissues can be better analyzed.
Current polychromatic imaging methods typically use dichroic mirrors to spatially separate the signals of each channel and pass them to different black and white cameras for detection, but this approach has some problems: for fluorophores with emission spectra overlapping each other, fluorescent signals of different channels cannot be completely separated by the dichroic mirror, so crosstalk between channels can exist; detecting by adopting a plurality of black-and-white cameras, wherein images of all channels are offset in the transverse direction and have differences of detection focal planes in the axial direction, so that complex registration processing is required to be carried out on original images; each channel requires a separate black and white camera, limited by the size and complexity of the system, and the number of channels for multicolor imaging is limited; the manufacturing cost of the system increases with the number of black and white cameras.
Disclosure of Invention
In order to overcome the problems of the existing multicolor imaging method, the embodiment of the application provides a line illumination modulation multicolor imaging system. The technical scheme is as follows:
a line illumination modulated multicolor imaging system comprising:
the line illumination modulation module comprises m monochromatic light sources and a modulation light path, and the wavelength of light beams emitted by each monochromatic light source is different; each light beam passes through the modulation light path to form a linear light spot focused on the focal plane of the objective lens, the illumination intensity of the linear light spot on the focal plane of the objective lens is Gaussian distributed in a first direction, the first direction is perpendicular to the extending direction of the linear light spot, the position parameters of each linear light spot are different, the linear light spots of m channels are overlapped to form m-color linear illumination light, and m is more than or equal to 2 and is a positive integer;
the imaging module is used for continuously scanning and imaging along a first direction by adopting a multi-element detector with n rows of pixels to obtain at least m mixed images under the irradiation of m-color line illumination light, each mixed image corresponds to one row of pixels, each mixed image comprises signals of m channels, and n is more than or equal to m and is a positive integer;
and the image demodulation module is used for demodulating a monochromatic image from the mixed image by utilizing a frequency domain demodulation algorithm or a spatial domain demodulation algorithm, wherein the monochromatic image corresponds to a signal of one channel.
Further, the modulation optical path includes a shaping optical path for shaping the light beam into a line light beam, a position adjusting optical path for adjusting a position parameter of each line light spot, and a projection optical path for superposing the line light spots to form m-color line illumination light.
Further, the shaping light path comprises a first dichroic mirror group for combining the m light sources, and a first beam expander, a second beam expander and a cylindrical lens which are sequentially arranged along the beam transmission direction after beam combination, wherein the first dichroic mirror group comprises m-1 dichroic mirrors.
Further, the position adjusting light path is located between the second beam expander and the cylindrical lens, and comprises a second dichroic mirror group, a third dichroic mirror group, a first reflecting mirror and a second reflecting mirror, wherein the second dichroic mirror group comprises m-1 dichroic mirrors, the third dichroic mirror group comprises m-1 dichroic mirrors, each time a light beam passes through one dichroic mirror in the second dichroic mirror group, the separated light path passes through the corresponding dichroic mirror in the third dichroic mirror group to be combined, and the residual light beam after passing through the second dichroic mirror group enters the third dichroic mirror group to be combined through the first reflecting mirror and the second reflecting mirror.
Further, the position adjustment optical path further includes a third mirror and a fourth mirror disposed on the optical path between the third dichroic mirror group and the cylindrical lens.
Further, the distance between the position parameters of each line light spot is larger than or equal to the width of each pixel of the multi-element detector corresponding to the object space.
Further, the imaging module includes:
a scanning unit for continuously scanning and imaging along a first direction by adopting a multi-element detector with n rows of pixels, wherein n is more than or equal to m;
an image block acquisition unit that acquires a stripe image block of an i-th pixel row in each frame image of one sample obtained in time sequence;
a splicing unit for sequentially splicing the strip image blocks of the ith pixel row in each frame of image of one sample to obtain a mixed image of the ith pixel row, i epsilon n; at least m times are repeated to obtain m mixed images.
Further, the image demodulation module is configured to demodulate m monochrome images from m hybrid images by using a frequency domain demodulation algorithm, where each monochrome image corresponds to a signal of a channel, and specifically includes:
the Fourier transform unit is used for carrying out Fourier transform on the original spatial domain images of the m mixed images to obtain m mixed frequency domain images;
the first demodulation unit is used for carrying out linear elimination operation on m mixed frequency domain images by combining an effective optical transfer function of a corresponding pixel row as a demodulation coefficient thereof to demodulate m single-color frequency domain images, wherein each single-color frequency domain image comprises a signal of one channel;
and the Fourier inverse transformation unit is used for carrying out Fourier inverse transformation on the m Zhang Shanse frequency domain image to obtain m monochromatic images.
Further, the image demodulation module is used for demodulating 1 single-color image from m mixed images by using a spatial demodulation algorithm through translational alignment and linear elimination operation, wherein the single-color image corresponds to a signal of a target channel by eliminating signals of m-1 non-target channels in sequence.
Further, the image demodulation module includes:
a first loop control unit for setting the value of the channel counter k to 1, and repeatedly executing the second demodulation unit, wherein the value of the channel counter is increased by 1 every time the second demodulation unit is executed until the value of the channel counter k is equal to m-1;
a second demodulation unit for setting the value of the image counter p to 1 and repeatedly executing the operation by the following submodules until the value of the image counter is equal to m-k;
a translation subunit for mixing the images I k Panning it with the blended image I k+p The signals of the k channels in (a) substantially overlap;
an elimination subunit for mixing the images I k And mix image I k+p Performing linear elimination operation to eliminate mixed image I k+p The signal of the k channel in (c) and is still denoted as I k+p ；
A counting subunit for adding 1 to the value of the image counter p.
The technical scheme provided by the embodiment of the application has the beneficial effects that at least:
the line illumination modulation multicolor imaging system provided by the embodiment of the application adopts a single multi-element detector to carry out scanning imaging, reduces the cost and complexity of the system compared with an imaging method which needs to use a plurality of cameras, adopts a plurality of pixel rows to carry out detection imaging, ensures that images among all channels are in a natural registration relationship, does not need to carry out additional registration processing, and realizes multicolor imaging of multiple channels. In fluorescence imaging, the line illumination modulation multicolor imaging system utilizes illumination light with different wavelengths to excite fluorescent signals with different colors differently, so that the problem of emission spectrum crosstalk is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a functional block diagram of a line illumination modulated multicolor imaging system of the present application;
FIG. 2 is a schematic diagram of the optical path structure of the line illumination modulated multicolor imaging of the present application;
FIG. 3 is a schematic view of the optical path structure of another line illumination modulated multicolor imaging of the present application;
FIG. 4 is a schematic view of the line illumination light distribution of the present application;
FIG. 5 is a schematic diagram of a sample imaging acquisition process of the present application;
fig. 6 is a functional block diagram of an image demodulation module of the present application;
FIG. 7 is a flow chart of a frequency domain demodulation algorithm of the multi-channel signal of the present application;
fig. 8 is a functional block diagram of another image demodulation module of the present application;
fig. 9 is a flowchart of the spatial demodulation algorithm of the multi-channel signal according to the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. In addition, the technical features of the embodiments of the present application described below may be combined with each other as long as they do not collide with each other.
As shown in fig. 1, an embodiment of the present application provides a line illumination modulated multicolor imaging system, including a line illumination modulation module 10, an imaging module 20, and an image demodulation module 30.
The linear illumination modulation module 10 comprises m (m is larger than or equal to 2, m is a positive integer) monochromatic light sources and a modulation light path, wherein the wavelength of light beams emitted by each monochromatic light source is different, each light beam forms a linear light spot focused on the focal plane of the objective lens through the modulation light path, the illumination intensity of the linear light spot on the focal plane of the objective lens is Gaussian distributed in a first direction, and the first direction is perpendicular to the extending direction of the linear light spot. The position parameters of the line light spots formed after the different laser beams are modulated are different, and the line light spots of m channels are overlapped to form m-color line illumination light.
The imaging module 20 is configured to perform continuous scanning imaging along a first direction by using a multi-element detector with n rows of pixels, so as to obtain at least m mixed images under illumination of the m color lines, where each mixed image corresponds to a row of pixels, each mixed image includes signals of m channels, and n is greater than or equal to m and is a positive integer.
The image demodulation module 30 is configured to demodulate monochrome images from the mixed images by using a frequency domain demodulation algorithm or a spatial domain demodulation algorithm, where each monochrome image corresponds to a signal of a channel.
The embodiment of the application provides a line illumination modulation multicolor imaging system. The scanning imaging is carried out by adopting a single multi-element detector, compared with an imaging method which needs to use a plurality of cameras, the cost and the complexity of the system are reduced, meanwhile, the detection imaging is carried out by adopting a plurality of pixel rows, the images among all channels are in a natural registration relationship, no additional registration processing is needed, and multi-channel multicolor imaging is realized. In fluorescence imaging, the line illumination modulation multicolor imaging system utilizes illumination light with different wavelengths to excite fluorescent signals with different colors differently, so that the problem of emission spectrum crosstalk is avoided.
The monochromatic Light source in the linear illumination modulation multicolor imaging system refers to a Light source with a certain wavelength, and can be either a monochromatic laser Light source or a monochromatic LED (Light-emitting diode) Light source or a monochromatic Light source obtained by combining a broad spectrum Light source with a narrow wave filter. In addition, the line illumination modulation multicolor imaging system can be suitable for fluorescence microscopic imaging and can also be suitable for multiple scenes of non-fluorescence microscopic imaging (namely, the illumination wavelength is the same as the detection wavelength), and the application is not limited to the above.
Further, the modulation optical path includes a shaping optical path for shaping the laser beam into line beams, a position adjusting optical path for adjusting a position parameter of each line spot, and a projection optical path for superposing the line spots to form m-color line illumination light.
Referring to fig. 2, for the optical path structure of the line illumination modulation polychromatic imaging in fluorescence microscopy imaging of the present application, the shaping optical path includes first dichroic mirror groups 3.1, 3.2, … …, 3 (m-1) for combining m laser light sources, and first beam expander 4, second beam expander 5 and cylindrical lens 12 sequentially disposed along the beam transmission direction after combining. The position adjusting light path is located between the second beam expander 5 and the cylindrical lens 12, and comprises second dichroic mirror groups 6.1, 6.2, … …, 6 (m-1), third dichroic mirror groups 7.1, 7.2, … …, 7 (m-1), a first reflecting mirror 8 and a second reflecting mirror 9, wherein each light beam is separated into a light path through one dichroic mirror in the second dichroic mirror groups, the separated light paths are combined through the corresponding dichroic mirrors in the third dichroic mirror groups, and the residual light beams after passing through the second dichroic mirror groups enter the third dichroic mirror groups through the first reflecting mirror 8 and the second reflecting mirror 9 to be combined. The projection light path includes an illumination cylindrical lens 13, an objective lens 14 and a fourth dichroic mirror 17 (in non-fluorescent imaging, a combination device consisting of a half-reflecting and half-transmitting beam splitter or a polarizing beam splitter and a quarter-glass, or other devices capable of reflecting illumination light and transmitting detection light with the same wavelength as the illumination light is needed), and is used for superposing the linear light spots to form polychromatic linear illumination light. The line illumination modulated polychromatic imaging system further comprises an imaging light path, in particular comprising an objective lens 14, a fourth dichroic mirror 17, an emission filter 18 (which is not required in non-fluorescent imaging), a detection cylinder 19 and a multiplexed detector 20.
Specifically, the first dichroic mirror groups 3.1, 3.2, … …, 3 (m-1) are arranged along the transmission direction of laser beams, the second laser light source 2.1, the third laser light source 2.2, … …, the m-th laser light source 2 (m-1) emit laser beams, which are converged into the laser beam emitted by the first laser light source 1 through one dichroic mirror in the first dichroic mirror group, to form beam combination beams, and are formed into beam combination beams by the first beam expander 4 and the second beam expander 5, then the second dichroic mirror groups 6.1, 6.2, … …, 6 (m-1) are utilized to separate illumination beams of different channels from each other and respectively enter different light paths, the light beams separated by the second dichroic mirror groups 6.1 are combined after passing through the third dichroic mirror groups 7.1, the separated residual beams are combined after passing through the second dichroic mirror groups 6.1, the separated by the second dichroic mirror groups 6.2, and the combined beams are combined after passing through the third dichroic mirror groups 7.2, 7.1 are combined after the separated by the second dichroic mirror groups, the separated by the second dichroic mirror groups 6.2, and the residual beams after the second dichroic mirror groups are combined by the second dichroic mirror groups 6.1, and the separated by the second dichroic mirror groups 7.2, and the residual beams after the light beams separated by the second dichroic mirror groups are combined by the second dichroic mirror groups 6.1, the second dichroic mirror groups are combined by the second mirror groups after the second laser beams separated by the second laser beams are separated by the second dichroic mirror groups 6.1, the residual beams after the second laser beams separated by the m-1.1, and the residual beams are separated by the second laser beams by the m-1, respectively passing through the second laser beam 2 (m-1, and the second beam 2, and the laser beam after the residual beams are separated by the first dichroic mirror 2.1 and the laser beam after the first and the laser beam are separated by the laser beam and the first beam and the laser beam respectively each respectively. The combined light beam is shaped into a linear light beam through a cylindrical lens 12, and the linear light beam is projected onto a focal plane of the objective lens through an illumination cylindrical lens 13, a fourth dichroic mirror 17 and the objective lens 14, so as to excite a fluorescent signal of a sample 15 on an electric translation stage 16, and the fluorescent signal sequentially passes through the objective lens 14, the fourth dichroic mirror 17, an emission filter 18 and a detection cylindrical lens 19, and finally is received by a multi-element detector 20 for imaging.
In some embodiments, the position-adjusting optical path further includes a third mirror 10 and a fourth mirror 11 disposed between the third dichroic mirror group 7 and the cylindrical lens 12.
Specifically, the position of the linear light spot reflected from the device to be adjusted in the x direction can be adjusted by adjusting the angle of any one or more of the second dichroic mirror group 6, the third dichroic mirror group 7, the first reflecting mirror 8 and the second reflecting mirror 9 in the position adjustment light path, and the angle of the third reflecting mirror 10 or the fourth reflecting mirror 11 can be adjusted to simultaneously adjust the positions of the linear light spots of all channels in the x direction. Preferably, the distance between the position parameters of each line light spot is larger than or equal to the corresponding width of the single pixel of the multi-element detector in the object space.
In other embodiments, the modulated light path may be configured to directly generate line beams of different wavelengths from linear LEDs of different emission wavelengths, and then projected onto the focal plane of the objective lens after passing through the projected light path. The position parameters of the light spot of the objective Jiao Mianxian can be changed by adjusting the positions of the different linear LEDs.
The wavelength of the laser beams emitted by each laser source is different, the position parameters of the line light spots focused on the focal plane of the objective lens by each laser beam through the modulation light path are different, and the line light spots on m channels can have certain dislocation in the x direction through the accurate adjustment of the position adjustment light path, so that the purpose of separating the positions of the line light spots of different channels in the x direction is achieved.
For better understanding of the present application, an optical path structure of the line illumination modulation polychromatic imaging in fluorescence microscopy imaging of the present application will be described below by taking the case of two laser light sources as an example, and as shown in fig. 3, there are only 2 laser light sources in the optical path structure, so there is only one dichroic mirror in the first dichroic mirror group for combining the laser light beams in the shaping optical path, and only one dichroic mirror is required for the second and third dichroic mirror groups in the position adjustment optical path.
Specifically, the laser beams emitted by the laser sources of the first laser source 1 and the second laser source 2 are combined through the first dichroic mirror 3, and are expanded through the first beam expander 4 and the second beam expander 5, then the illumination beams of different channels are separated by the second dichroic mirror 6 and enter two optical paths respectively, one optical path passes through the second dichroic mirror 6 and the third dichroic mirror 7, the other optical path passes through the second dichroic mirror 6, the first reflecting mirror 8, the second reflecting mirror 9 and the third dichroic mirror 7 in sequence, then the illumination beams of the two optical paths are combined again through the third dichroic mirror 7, finally the combined illumination beams are shaped into linear beams through the cylindrical lens 12, and are projected onto the focal plane of the objective lens through the illumination cylindrical lens 13, the fourth dichroic mirror 17 and the objective lens 14, so as to excite fluorescent signals of the sample 15 on the electric translation stage 16, and the fluorescent signals pass through the objective lens 14, the fourth dichroic mirror 17, the emission filter 18 and the detection cylindrical lens 19 in sequence, and finally the multiplexed detector 20 are received for imaging.
Fig. 4 is a schematic diagram of the line illumination beam distribution of the present application. In some embodiments, as shown in fig. 4, the line spot extension direction is the y-direction, and the first direction is the x-direction perpendicular to the line spot extension direction. The imaging area of the multi-element detector is n pixel rows arranged along the x direction, the illumination intensities of the line illumination light of the two channels are distributed in a Gaussian mode in the x direction, and the space between the position parameters of the line light spots of the two channels is 1 pixel.
The multi-element detector in this embodiment may be an area CCD (Charge-coupled device) or an area CMOS (Complementary Metal Oxide Semiconductor) camera having Sub-array or ROI (Region of interest) functions, or a linear CCD or linear CMOS camera having an area mode function may be used.
Further, referring to fig. 1, the imaging module 20 includes:
a scanning unit 201 for scanning and imaging continuously in a first direction using a multi-element detector having n rows of pixels, n being equal to or greater than m;
an image block acquisition unit 202 that acquires a stripe image block of an i-th line pixel in each frame image of one sample obtained in time series;
the stitching unit 203 sequentially stitches the stripe image blocks of the ith row of pixels in each frame of image of one sample to obtain a mixed image of the ith row of pixels, i e n.
Fig. 5 is a schematic diagram of a sample imaging acquisition process according to the present application, and the function of the imaging module 20 is described below in conjunction with fig. 5. In order to facilitate subsequent image demodulation, the imaging area of the camera in the embodiment is n pixel rows arranged along the x direction, and the movement direction of the sample is also along the x direction, so that the sample can be scanned and imaged under different pixels, multicolor imaging of m channels is realized, and the number of channels of multicolor imaging is widened.
Referring to fig. 5 (a), during an imaging acquisition, a scanning unit 201 continuously scans an image in the x-direction using a multi-element detector having n rows of pixels. Specifically, the electric translation stage 16 drives the sample 15 to move at a uniform speed along the x-direction, and the exposure time of a single frame of the camera is equal to the time of moving the sample by one pixel in the corresponding width of the object space. If the image corresponding to any pixel row in one frame of image is set as one stripe image block, a plurality of stripe image blocks corresponding to the pixel row in a plurality of frames of images are sequentially and continuously imaged for each part of the sample.
Referring to fig. 5 (b), the sequentially imaged stripe image blocks obtained in time sequence are spliced to obtain a mixed image corresponding to each pixel row, and each mixed image contains m fluorescent signals of channels. Specifically, the i-th line pixel line is selected from n pixel lines, and the image block acquisition unit 202 acquires a stripe image block of the i-th line pixel in each frame image of one sample obtained in time series. The stitching unit 203 sequentially stitches the stripe image blocks of the ith row of pixels in each frame of the image of one sample to obtain a mixed image of the ith row of pixels. Selecting different pixel rows, repeating for at least m times to obtain m mixed images of different pixel rows.
The demodulation principle and demodulation process in the case of fluorescence imaging will be mainly described below. The demodulation principle and the demodulation process of non-fluorescent imaging and the demodulation principle and the demodulation process of fluorescent imaging are basically the same, the main difference is only the difference of detection wavelengths, so that the effective PSF of the system is slightly different and can be ignored, and the method is still applicable.
Similar to the imaging principle of a line scanning confocal microscope, the blended image corresponding to each pixel row in the method is a convolution of the effective PSF (Point spread function ) of the system, which is the product of the illumination system PSF and the detection system PSF, and the fluorescent signal distribution on the sample.
Taking the fluorescence signal of a single channel as an example, the effective PSF of the jth channel in the ith pixel row is:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the lighting system PSF->The PSF, i of the detection system represents the serial number of each pixel row on the imaging area, i epsilon n, j represents the serial number of fluorescent protein, and also represents the channel serial number, x and y represent the transverse coordinates of the object space, the directions are shown in figures 2 and 3, the x direction is the movement direction of the sample, the y direction is the direction perpendicular to the movement direction of the sample, b i,j Representing the distance between the ith pixel row and the jth channel line spot location parameter.
Thus, the signal of the ith pixel row corresponding to the jth channel included in the resulting blended image may be expressed as
Wherein f j (x, y) represents the relative concentration distribution of the jth fluorescent protein on the sample, and "×" represents the convolution operation. As can be seen from equation (2), the fluorescence signal of the sample is co-modulated by the illumination system PSF and the detection system PSF, whereas the detection system PSF of each pixel of the camera is the same, so the variability of the modulation of the line illumination modulation module is mainly reflected on the illumination system PSF.
Under polychromatic illumination, the mixed image contains m channel fluorescent signals. Because of the dislocation of the linear light spots of different channels, for the same pixel row, the linear illumination modulation to which the sample fluorescent signals of different channels are subjected is different, and the mixed image of the ith pixel row can be expressed as:
I i (x,y)＝h i,1 (x,y)*f 1 (x,y)+h i,2 (x,y)*f 2 (x,y) (3)
similarly, the blended image for the i+1th pixel row can be expressed as:
I i+1 (x,y)＝h i+1,1 (x,y)*f 1 (x,y)+h i+1,2 (x,y)*f 2 (x,y) (4)
based on the principle, the mixed image is processed through the frequency domain demodulation algorithm and the spatial domain demodulation algorithm, so that m single-color images can be demodulated from m Zhang Hunge images, each single-color image corresponds to a channel fluorescence signal, and channels corresponding to the single-color images are different.
In some embodiments, a frequency domain demodulation algorithm is used, and a linear elimination operation is performed on the mixed image in the frequency domain by taking the optical transfer function as a modulation coefficient thereof, so that a monochromatic image is demodulated. Referring to fig. 6, the image demodulation module 30 includes:
a fourier transform unit 301, configured to perform fourier transform on the original spatial domain images of the m mixed images, to obtain m mixed frequency domain images;
a first demodulation unit 302, configured to perform linear elimination operation on m mixed frequency domain images by combining an effective optical transfer function of a corresponding pixel row as a demodulation coefficient thereof, so as to demodulate m single-color frequency domain images, where each single-color frequency domain image includes a fluorescent signal of a channel;
the inverse fourier transform unit 303 is configured to perform inverse fourier transform on the m Zhang Shanse frequency domain image, so as to obtain m single-color images.
As shown in fig. 7, the flow of the frequency domain demodulation algorithm for the multichannel fluorescent signal specifically includes:
the method comprises the steps of performing Fourier transform on an original spatial domain image of an m Zhang Hunge image through a Fourier transform unit 301 to obtain m mixed frequency domain images, then performing linear elimination operation on the m Zhang Hunge frequency domain images through a first demodulation unit 302 by combining an effective optical transfer function of a corresponding pixel row as a demodulation coefficient of the mixed frequency domain image, demodulating m single-color frequency domain images, wherein the single-color frequency domain images comprise fluorescent signals of one channel, and finally performing Fourier inverse transform on the m Zhang Shanse frequency domain images through an Fourier inverse transform unit 303 to obtain m single-color images, wherein each single-color image comprises fluorescent signals of one channel.
In other embodiments, s (n.gtoreq.s > m) blended images may also be selected for demodulating m-channel monochromatic images. Specifically, the 1 st to m Zhang Hunge th images (denoted as 1 st group mixed images) are selected from s Zhang Li, the 1 st group of m-th channel monochrome images are demodulated according to the above-mentioned frequency domain demodulation algorithm flow, then the (m+1) th image in the s Zhang Hunge th image is substituted for any 1 st image in the 1 st group of images and denoted as 2 nd group mixed images, then the (m+2) th channel monochrome images in the 2 nd group of m-th channels are demodulated according to the above-mentioned frequency domain demodulation algorithm flow, and then the (m+2) th image in the s mixed images is substituted for any 1 st image … … in the 2 nd group of images, and the cycle is performed until the (s-m+1) th channel monochrome images are demodulated. And finally, adding up the images with the same channel serial numbers in the demodulated s-m+1 group of images to obtain the monochromatic images with m channels and higher signal to noise ratio.
The following will specifically describe a case of two channels. Performing fourier transform on the spatial domain images of the formula (3) and the formula (4), and obtaining frequency domain images of the mixed image of the ith pixel row and the (i+1) th pixel row, wherein the frequency domain images are respectively as follows:
O i (u,v)＝H i,1 (u,v)×F 1 (u,v)+H i,2 (u,v)×F 2 (u,v) (5)
O i+1 (u,v)＝H i+1,1 (u,v)×F 1 (u,v)+H i+1,2 (u,v)×F 2 (u,v) (6)
wherein H is i,1 (u,v)、H i+1,1 (u,v)、H i,2 (u,v)、H i+1,2 (u,v)、F 1 (u, v) and F 2 (u, v) each represents h i,1 (x,y)、h i+1,1 (x,y)、h i,2 (x,y)、h i+1,2 (x,y)、f 1 (x, y) and f 2 Fourier transform result of (x, y), H i,1 (u,v)、H i+1,1 (u,v)、H i,2 (u, v) and H i+1,2 (u, v) can be obtained by theoretical simulation or experimentally measured.
Alternatively, a theoretical simulation mode may be adopted, specifically: h is calculated by combining a PSF theoretical expression and related parameters of an actual system through a formula (1) i,1 (x,y)、h i+1,1 (x,y)、h i,2 (x, y) and h i+1,2 (x, y) and then fourier transforming them respectively.
Alternatively, experimental measurement methods may be adopted, specifically: measuring effective PSFh of different channels in different pixel rows i,1 (x,y)、h i+1,1 (x,y)、h i,2 (x, y) and h i+1,2 (x, y) and then fourier transforming them respectively.
Thus, in the formula (5) and the formula (6), only F 1 (u, v) and F 2 (u, v) the two unknowns, and the single-color frequency domain images of the channel 1 and the channel 2 can be demodulated by performing linear elimination operation on the mixed frequency domain images. Specifically, the method comprises the following steps:
further, for monochromatic frequency domain image F 1 (u, v) and F 2 (u, v) inverse fourier transform, i.e. single color images of channel 1 and channel 2 can be demodulated.
Similarly, when the number of channels m >2, m linear equations can be listed according to the equation (5) and the equation (6), each equation contains m unknown quantities, and the m linear equations can be solved to demodulate the monochromatic images of m channels.
In other embodiments, spatial demodulation algorithms may also be employed. The spatial demodulation algorithm is an approximate demodulation algorithm, uses illumination intensity as a modulation coefficient thereof, and performs linear elimination operation on a mixed image in a spatial domain to finally realize multicolor demodulation. In the formula (1), the effective PSF obtained by multiplying the illumination PSF and the detection PSF in gaussian distribution can still be regarded as gaussian distribution approximately, and the intensity thereof decays rapidly with increasing distance from the center, so that only the contribution of the center region of the effective PSF to imaging can be considered, and the intensity values of the regions around the effective PSF are ignored, and the effective PSF can be expressed approximately as an impulse function. In addition, because there is a certain displacement between the illumination PSF and the detection PSF, the center of the effective PSF is not fixed at the origin any more, and a certain displacement is generated, and the displacement direction and the size of the effective PSF are changed along with the change of the displacement direction and the size between the illumination PSF and the detection PSF.
Taking the effective PSF of channel 1 at the ith pixel row as an example, it can be expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,the x-direction coordinate corresponding to the maximum value of the effective PSF intensity is represented, and δ (x, y) represents the unit impulse function.
The same principle can be obtained:
order the
And (3) bringing the formula (9) -the formula (16) into the formula (3) and the formula (4), thereby obtaining the following formula:
wherein L is i,1 、L i,2 、L i+1,1 、L i+1,2 Andmay be measured by simulation or experiment.
Alternatively, a simulation mode may be adopted, specifically: calculating each effective PSFh by combining a PSF theoretical expression and related parameters of the system through a formula (1) i,1 (x,y)、h i+1,1 (x,y)、h i,2 (x, y) and h i+1,2 (x, y), the intensity value of the strongest point of each effective PSF center is L i,1 、L i,2 、L i+1,1 、L i+1,2 The displacement of the strongest point of each effective PSF center relative to the origin is
Alternatively, experimental measurements may be used, specifically: independently starting a light source of the channel 1, scanning a sample along the x direction, collecting a sample image by the ith pixel row and the (i+1) th pixel row of the camera, and calculating the distance between the two images along the x axis to obtainAnd calculating the ratio of the average gray values between the two images, namely L i,1 /L i+1,1 . By following the above measurement procedure, the +.>And L i,2 /L i+1,2 Due to->And->And (3) withCan be converted into the same variable through translation and are mutually equivalent. Therefore, the equivalent in equation (17) and equation (18) is that there are only two unknownsThe variable can eliminate the signal of one channel by carrying out translation transformation and linear elimination on the mixed image, and a monochromatic image only comprising the signal of a single channel is demodulated.
Based on the principle, when the method is realized, a spatial demodulation algorithm is adopted to sequentially eliminate fluorescent signals of m-1 non-target channels from m Zhang Hunge images through translational alignment and linear elimination operation, so that 1 single-color image corresponding to only one target channel is demodulated.
Specifically, referring to fig. 8, the image demodulation module 30 includes:
a first loop control unit 304, configured to set the value of the channel counter k to 1, and repeatedly execute the second demodulation unit, where each execution controls the value of the channel counter to be increased by 1 until the value of the channel counter k is equal to m-1;
a second demodulation unit 305 for setting the value of the image counter p to 1 and repeatedly executing by the following submodules until the value of the image counter is equal to m-k;
a translation subunit 3051 for mixing the images I k Panning it with the blended image I k+p The fluorescence signals of the k channels in (a) substantially overlap;
an degerming subunit 3052 for mixing images I k And mix image I k+p Performing linear elimination operation to eliminate mixed image I k+p Fluorescent signal of k channel in (a);
a counting sub-unit 3053 for adding 1 to the value of the image counter p.
As shown in FIG. 9, the flow of the spatial demodulation algorithm of the multichannel fluorescent signal of the application is shown. The specific process is as follows:
setting the value of a channel counter k to 1, setting the value of an image counter p to 1, translating the mixed image corresponding to the 1 st pixel row, enabling the fluorescent signal of the 1 st channel in the mixed image corresponding to the 2 nd pixel row to basically overlap with the fluorescent signal of the 1 st channel in the mixed image, performing linear elimination operation on the two mixed images, eliminating the fluorescent signal of the 1 st channel in the mixed image corresponding to the 2 nd pixel row, and still recording the mixed image as the mixed image corresponding to the 2 nd pixel row. At this time, the value of the image counter p is incremented by 1. And then shifting the mixed image corresponding to the 1 st pixel row again to enable the fluorescent signal of the 1 st channel in the mixed image corresponding to the 3 rd pixel row to basically overlap with the fluorescent signal of the 1 st channel in the mixed image, performing linear elimination operation again, eliminating the fluorescent signal of the 1 st channel in the mixed image corresponding to the 3 rd pixel row, and still recording the mixed image as the mixed image corresponding to the 3 rd pixel row. At this time, the value of the image counter p is further increased by 1. And repeating the steps until the value of the image counter p is equal to m-1, and finishing the linear elimination operation of the mixed image corresponding to the 1 st pixel row and the m Zhang Hunge image to obtain m-1 mixed images with the fluorescent signals of the 1 st channel eliminated.
Further, the value of the channel counter k is increased by 1, the value of the image counter p is reset to be 1, the steps are repeated, the translation transformation and the linear elimination operation of the mixed image containing the fluorescent signals of the 2 nd channel are carried out until the value of the image counter p is equal to m-2, and m-2 mixed images with the fluorescent signals of the 1 st channel and the fluorescent signals of the 2 nd channel eliminated are obtained.
And executing the steps of adding 1 to the value of the channel counter k and resetting the value of the image counter p to be 1 again, and repeating the processes of the mixed image translation transformation and the linear elimination operation until the value of the image counter p is equal to m-k.
The steps are repeated until the value of the channel counter k is equal to m-1, and finally 1 monochromatic image only containing the fluorescent signal of the m-th channel is obtained.
The m channel numbers are artificially defined herein, i.e. any one of the m fluorescent proteins can be regarded as the m-th channel. The channel number of each fluorescent protein can be adjusted according to the difference of the demodulated target fluorescent protein, and the demodulated target fluorescent protein can be regarded as the mth channel, and then demodulated according to the above-described demodulation process.
Based on the above, by selecting different mixed images, different non-target channel fluorescent signals can be eliminated, and a monochrome image containing only any of the target channel fluorescent signals can be obtained.
In other embodiments, s (n.gtoreq.s > m) blended images may also be selected for demodulating m-channel monochromatic images. Specifically, the 1 st to m Zhang Hunge th images (denoted as 1 st group mixed images) are selected from s Zhang Li, the 1 st group of m-th channel monochrome images are demodulated according to the above-mentioned spatial demodulation algorithm flow, then the (m+1) th image in the s Zhang Hunge th image is substituted for any 1 st image in the 1 st group of images and denoted as 2 nd group mixed images, then the (m+2) th channel monochrome images in the 2 nd group of m-th channels are demodulated according to the above-mentioned spatial demodulation algorithm flow, and then the (m+2) th image in the s mixed images is substituted for any 1 st image … … in the 2 nd group of images, and the cycle is performed until the (s-m+1) th channel monochrome images are demodulated. And finally, adding up the images with the same channel serial numbers in the demodulated s-m+1 group of images to obtain the monochromatic images with m channels and higher signal to noise ratio.
Likewise, taking the case of two channels as an example, only image g of a single channel signal for channel 1 and channel 2 is included 1 (x, y) and g 2 (x, y) can be expressed as:
wherein in the formula (19)And->The translation along the x direction can realize the mutual conversion, and the translation distance is:
due toAnd->The symbols are identical, the values are very small and close, so that the value D is close to 0 and therefore negligible +.>And->Misalignment between, can be considered as:
is of the same kind
Further, substituting the formula (22) and the formula (23) into the formula (19) and the formula (20) can obtain:
from this, it can be seen that the demodulation result g j (x, y) and the original fluorescence signal f j Except for a certain displacement between (x, y), only a constant coefficient factor is different, and the constant coefficient factor only changes the whole gray value of the image, so the demodulation result g can be considered j And (x, y) is the actual distribution result of the signals of each channel on the sample.
Synchronous multicolor imaging is realized on a single camera, images among all channels are in a natural registration relationship, additional registration processing is not needed, and a monochromatic image containing a single-channel fluorescence signal can be obtained by processing a mixed image through a demodulation algorithm, so that the complexity of a system is greatly reduced.
The foregoing is only illustrative of the present application and is not to be construed as limiting thereof, but rather as various modifications, equivalent arrangements, improvements, etc., within the spirit and principles of the present application.
Claims (10)
1. A line illumination modulated multicolor imaging system, the line illumination modulated multicolor imaging system comprising:
the line illumination modulation module comprises m monochromatic light sources and a modulation light path, and the wavelength of light beams emitted by each monochromatic light source is different; each light beam passes through the modulation light path to form a linear light spot focused on the focal plane of the objective lens, the illumination intensity of the linear light spot on the focal plane of the objective lens is Gaussian distributed in a first direction, the first direction is perpendicular to the extending direction of the linear light spot, the position parameters of each linear light spot are different, the linear light spots of m channels are overlapped to form m-color linear illumination light, and m is more than or equal to 2 and is a positive integer;
the imaging module is used for continuously scanning and imaging along a first direction by adopting a multi-element detector with n rows of pixels to obtain at least m mixed images under the irradiation of m-color line illumination light, each mixed image corresponds to one row of pixels, each mixed image comprises signals of m channels, and n is more than or equal to m and is a positive integer;
and the image demodulation module is used for demodulating a monochromatic image from the mixed image by utilizing a frequency domain demodulation algorithm or a spatial domain demodulation algorithm, wherein the monochromatic image corresponds to a signal of one channel.
2. The line illumination modulated multicolor imaging system of claim 1 wherein the modulation light path comprises a shaping light path for shaping a light beam into a line beam, a position adjustment light path for adjusting a position parameter of each line spot, and a projection light path for superimposing the line spots to form m-color line illumination light.
3. The line illumination modulated multicolor imaging system according to claim 2, characterized in that the shaping optical path comprises a first dichroic mirror group for combining m light sources, and a first beam expander, a second beam expander, and a cylindrical lens sequentially arranged in a beam transfer direction after combining, the first dichroic mirror group comprising m-1 dichroic mirrors.
4. The line illumination modulated multicolor imaging system of claim 3 wherein the position-adjusted light path is located between the second beam expander and the cylindrical lens and comprises a second dichroic mirror group, a third dichroic mirror group, a first mirror and a second mirror, the second dichroic mirror group comprising m-1 dichroic mirrors, the third dichroic mirror group comprising m-1 dichroic mirrors, each light beam passing through one of the second dichroic mirrors being separated by a light path, the separated light paths being combined by a corresponding dichroic mirror of the third dichroic mirror group, and the remaining light beams after passing through the second dichroic mirror group being combined by the first mirror and the second mirror entering the third dichroic mirror group.
5. The line illumination modulated multicolor imaging system of claim 4 wherein the position-adjusted light path further comprises a third mirror and a fourth mirror disposed in the light path between the third dichroic mirror group and the cylindrical lens.
6. The line illumination modulated multicolor imaging system of claim 2 wherein a spacing between positional parameters of each of the line spots is greater than or equal to a corresponding width of individual pixels of the multi-element detector in object space.
7. The line illumination modulated multicolor imaging system of claim 1 wherein the imaging module comprises:
a scanning unit for continuously scanning and imaging along a first direction by adopting a multi-element detector with n rows of pixels, wherein n is more than or equal to m;
an image block acquisition unit that acquires a stripe image block of an i-th pixel row in each frame image of one sample obtained in time sequence;
and the splicing unit is used for sequentially splicing the strip image blocks of the ith pixel row in each frame of image of one sample to obtain a mixed image of the ith pixel row, i epsilon n.
8. The line illumination modulated multicolor imaging system according to any of claims 1-7, wherein the image demodulation module is configured to demodulate m monochrome images from m said hybrid images by using a frequency domain demodulation algorithm, each monochrome image corresponding to a signal of a channel, and specifically comprises:
the Fourier transform unit is used for carrying out Fourier transform on the original spatial domain images of the m mixed images to obtain m mixed frequency domain images;
the first demodulation unit is used for carrying out linear elimination operation on m mixed frequency domain images by combining an effective optical transfer function of a corresponding pixel row as a demodulation coefficient thereof to demodulate m single-color frequency domain images, wherein each single-color frequency domain image comprises a signal of one channel;
and the Fourier inverse transformation unit is used for carrying out Fourier inverse transformation on the m Zhang Shanse frequency domain image to obtain m monochromatic images.
9. The line illumination modulated multicolor imaging system according to any of claims 1-7, wherein the image demodulation module is configured to sequentially cancel signals of m-1 non-target channels from m mixed images by translational alignment and linear elimination operation using a spatial demodulation algorithm to demodulate 1 single-color image corresponding to the signal of the target channel.
10. The line illumination modulated multicolor imaging system of claim 9 wherein the image demodulation module comprises:
a first loop control unit for setting the value of the channel counter k to 1, and repeatedly executing the second demodulation unit, wherein the value of the channel counter is increased by 1 every time the second demodulation unit is executed until the value of the channel counter k is equal to m-1;
a second demodulation unit for setting the value of the image counter p to 1 and repeatedly executing the operation by the following submodules until the value of the image counter is equal to m-k;
a translation subunit for mixing the images I k Panning it with the blended image I k+p The signals of the k channels in (a) substantially overlap;
an elimination subunit for mixing the images I k And mix image I k+p Performing linear elimination operation to eliminate mixed image I k+p The signal of the k channel in (c) and is still denoted as I k+p ；
A counting subunit for adding 1 to the value of the image counter p.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310411763.4A CN116609305A (en) | 2023-04-17 | 2023-04-17 | Linear illumination modulation multicolor imaging system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310411763.4A CN116609305A (en) | 2023-04-17 | 2023-04-17 | Linear illumination modulation multicolor imaging system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116609305A true CN116609305A (en) | 2023-08-18 |
Family
ID=87678946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310411763.4A Pending CN116609305A (en) | 2023-04-17 | 2023-04-17 | Linear illumination modulation multicolor imaging system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116609305A (en) |
-
2023
- 2023-04-17 CN CN202310411763.4A patent/CN116609305A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11971355B2 (en) | Fluorescence observation apparatus and fluorescence observation method | |
| CN110178069B (en) | Microscope apparatus, method and system | |
| US11269171B2 (en) | Spectrally-resolved scanning microscope | |
| JP2005292839A (en) | Slit confocal microscope and its operation method | |
| US20180120575A1 (en) | Devices, systems and methods for illuminating and imaging objects | |
| US7645971B2 (en) | Image scanning apparatus and method | |
| CN108519329A (en) | A kind of line co-focusing imaging device of multi-channel scanning and detection | |
| JP2019520721A (en) | Color calibration and demonstration of digital pathology | |
| JPH11211668A (en) | Method and apparatus for detection of defect | |
| US6590612B1 (en) | Optical system and method for composing color images from chromatically non-compensated optics | |
| CA3106449A1 (en) | Scanning microscope using pulsed illumination | |
| JP2023517677A (en) | A High Throughput Snapshot Spectral Encoding Device for Fluorescence Spectral Microscopy | |
| CN116609305A (en) | Linear illumination modulation multicolor imaging system | |
| US20180084231A1 (en) | Machine vision spectral imaging | |
| CN116539575A (en) | Line illumination modulation polychrome tomography system | |
| JPH01282515A (en) | Beam scanning type optical microscope | |
| CN116593434A (en) | Line illumination modulation double-color tomography system | |
| JP2022530113A (en) | Hybrid imaging products and hybrid endoscope systems | |
| AU2019373533B8 (en) | High-throughput optical sectioning imaging method and imaging system | |
| US11842555B2 (en) | Signal acquisition apparatus, signal acquisition system, and signal acquisition method | |
| US20220373777A1 (en) | Subpixel line scanning | |
| JP7494490B2 (en) | Signal Acquisition Device, Signal Acquisition System, and Signal Acquisition Method | |
| WO2022080189A1 (en) | Biological specimen detection system, microscope system, fluorescence microscope system, biological specimen detection method, and program | |
| JP2002539497A (en) | Optical system without color correction for color image synthesis | |
| JPH02192276A (en) | Picture input and output device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |