CN107941775B - Multispectral Microscopic Imaging System - Google Patents

Abstract

The invention discloses a multispectral microscopic imaging system, which comprises: the microscope is used for amplifying and imaging the microscopic sample to an image surface leading-out port; the modulation mask is positioned on an output image surface of the microscope to restrict the range of a post-stage imaging field of view and is added with random mask modulation; the optical filter is positioned behind the field diaphragm to perform narrow-band filtering on illumination information from the microscopic sample and restrict the imaging spectral range of the system; the dispersion unit is used for spatially spreading the spectral information of each point on the sample; and the image sensor is used for recording the sample image modulated by the previous stage so as to reconstruct various mark images on one or two sheets through dispersion and mask modulation. The system can reconstruct various marked images by only adopting one or two images, improves the imaging speed, improves the image resolution, reduces the damage of laser to photosensitive samples and biological samples, and has simple structure and low cost.

Description

Multispectral Microscopic Imaging System

technical field

The invention relates to the technical field of computational photography, in particular to a multispectral microscope imaging system.

Background technique

Remote sensing imaging, life science and material science research have put forward higher requirements for multispectral imaging. The observation target or scene can emit signals with different spectral curves because different regions contain different materials or fluorescently labeled proteins. Multispectral imaging is to use these spectral response differences to calculate and reconstruct various materials or fluorescent protein imaging, and the main difficulty is that the spectral response curves of different materials or labeled proteins may overlap, which makes it impossible to simply add filters of different spectral bands. Enables separation of material or fluorescent marker imaging.

At present, in the macroscopic field, multispectral imaging mainly adds filters of different spectral bands to the imaging optical path for multiple imaging or uses multiple dichroic mirrors of different spectral bands to image multiple spectral bands at the same time, and then uses these spectral imaging Stacks, computationally reconstructed images of different materials or markers. In the field of microscopy imaging, there are multispectral imaging based on widefield microscopy, multispectral microscopy imaging based on confocal microscopy and light sheet illumination. Multispectral imaging based on wide-field microscope, using monochromatic or polychromatic excitation at the illumination end, adding filters of different spectral bands to the imaging optical path for multiple imaging or multiple dichroic mirrors of different spectral bands for simultaneous imaging, using these spectral Imaging stack calculations reconstruct images of different materials or markers, reducing the effect of overlapping spectral response curves. The multispectral microscopy imaging based on confocal microscopy scans the fluorescent sample point by point, adds a dispersive device at the acquisition end, and uses multiple PMTs (photomultiplier tubes, photomultiplier tubes) to record the spectral information of the point. Multispectral microscopy based on confocal and light sheet illumination, using line scanning light sheet illumination, adding a dispersive device to the imaging light path, and then recording the fluorescence spectrum excited on the entire line with a camera. The spectral information of the entire field of view is obtained by point-by-point scanning or line-by-line scanning by confocal microscopy, and then various labeled structures are calculated and reconstructed.

However, there are still deficiencies in related technologies. For example, multispectral imaging based on confocal microscopy requires strong excitation light, which is easy to cause fluorescence bleaching, and the strong light irradiation may damage sample cells or tissue structures; another example, the imaging process requires Scanning or multiple shots have low time efficiency, which makes it suitable for photosensitive samples and biological samples, and the imaging speed is slow and cannot record dynamic processes, which needs to be solved.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

Therefore, the purpose of the present invention is to provide a multi-spectral microscopic imaging system, which can improve the imaging speed, improve the image resolution, reduce the damage of laser light to photosensitive samples and biological samples, and has a simple structure and low cost.

In order to achieve the above purpose, an embodiment of the present invention proposes a multi-spectral microscopic imaging system, including: a microscope, which is used to amplify and image a microscopic sample to an image plane outlet; a modulation mask, the modulation mask is located in the The output image surface of the microscope is used to constrain the range of the imaging field of view of the subsequent stage, and random mask modulation is added; the filter is located behind the field diaphragm to control the illumination from the microscopic sample The information is narrow-band filtered to constrain the imaging spectral range of the system; the dispersion unit is used to spatially expand the spectral information of each point on the sample; the image sensor is used to record the sample image modulated by the previous stage to pass the dispersion and masking. Membrane modulation reconstructs a variety of labeled images on one or two sheets.

The multi-spectral microscope imaging system of the embodiment of the present invention can realize single or multiple exposure collection of data under a single camera, and can restore a single-color mark image, and realize that only one or two images can be used to reconstruct various kinds of images. The image is marked, thereby improving the imaging speed, improving the image resolution, reducing the damage of the laser to the photosensitive sample and the biological sample, and the structure is simple and the cost is low.

In addition, the multispectral microscopy imaging system according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Further, in an embodiment of the present invention, it further includes: a reconstruction module, which obtains an image through the dispersion and mask modulation, and reconstructs each fluorescent protein-labeled tissue of the sample from the spectral response curve of the labeled fluorescent protein. structure.

Further, in an embodiment of the present invention, the microscope may be a wide-field fluorescence microscope.

Further, in an embodiment of the present invention, the modulation mask is located on the image plane, and the dispersion unit is located on the Fourier plane.

Further, in an embodiment of the present invention, the Fourier plane at the dispersion unit and the image plane at the image sensor are realized by a lens or a 4f system composed of lenses.

Further, in an embodiment of the present invention, the 4f system is used for adding dispersion modulation and imaging magnification adjustment.

Further, in an embodiment of the present invention, the filter is located after the first-stage lens of the 4f system on the imaging optical path, or directly added inside the microscope or at other positions in the optical path to constrain the imaging spectrum. scope.

Further, in an embodiment of the present invention, the random mask adjusts the lateral position of the mask by means of translation.

Further, in an embodiment of the present invention, the dispersion unit is an Amoxie dispersion prism.

Further, in an embodiment of the present invention, the image sensor is a SCMOS monochrome sensor.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic structural diagram of a multispectral microscope imaging system according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a multispectral microscope imaging system according to a specific embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

The following describes the multispectral microscope imaging system proposed according to the embodiments of the present invention with reference to the accompanying drawings.

FIG. 1 is a schematic structural diagram of a multispectral microscope imaging system according to an embodiment of the present invention.

As shown in FIG. 1 , the multispectral microscope imaging system 10 includes: a microscope 100 , a modulation mask 200 , a filter 300 , a dispersion unit 400 and an image sensor 500 .

Among them, the microscope 100 is used for enlarging and imaging the microscopic sample to the image plane outlet. The modulation mask 200 is located on the output image plane of the microscope 100 to constrain the field of view of the post-stage imaging and add random mask modulation. Optical filter 300 is located behind the field diaphragm to narrow-band filter the illumination information from the microscopic sample, constraining the imaging spectral range of the system. The dispersion unit 400 is used to spatially expand the spectral information of each point on the sample. The image sensor 500 is used to record the pre-modulated sample images, so as to reconstruct various marking images in one or two sheets through dispersion and mask modulation. The system 10 in the embodiment of the present invention can reconstruct a variety of marked images by using only one or two images, improve imaging speed, improve image resolution, reduce laser damage to photosensitive samples and biological samples, and has a simple structure and low cost. low.

Wherein, in an embodiment of the present invention, the system 10 in the embodiment of the present invention further includes: a reconstruction module. Wherein, the reconstruction module obtains the image through dispersion and mask modulation, and reconstructs the tissue structure labeled by each fluorescent protein of the sample with the spectral response curve of the labeled fluorescent protein.

It can be understood that, in the embodiment of the present invention, images of different markers can be calculated and reconstructed by using multiple images obtained by a single shot or by translating the mask, combined with the spectral response curve of each fluorescent marker. The calculation and reconstruction process of the reconstruction module can be implemented on a hardware system such as a common PC or a workstation.

That is to say, the multispectral microscope imaging system 10 of the embodiment of the present invention mainly includes two parts: image acquisition and calculation reconstruction.

The image acquisition part consists of a microscope 100 , a modulation mask 200 , a filter 300 , a dispersion unit 400 and an image sensor 500 . The microscope 100 can realize fluorescence excitation and magnification imaging of multi-labeled samples. The modulation mask 200 adds random binary modulation to the image plane. The dispersion unit 400 expands the spectral information of each point on the image plane in space on the Fourier plane of the 4f system, and the 4f system realizes the adjustment of the magnification of the imaging system and the adjustment of the dispersion range.

The calculation and reconstruction part uses multiple images obtained by a single shot or a translation mask, that is, the images of the multi-labeled samples can be obtained by a single shot or two shots, and the imaging speed is fast. Combined with the spectral response curve of each fluorescent marker, it can be Computationally reconstructed images of different markers. The calculation and reconstruction process of the reconstruction module can be implemented on a hardware system such as a common PC or a workstation.

Further, in one embodiment of the present invention, the microscope 100 may be a wide-field fluorescence microscope.

It can be understood that the microscope 100 in the embodiment of the present invention is a wide-field fluorescence microscope, so as to realize the first-level magnification of the fluorescent sample, and the sample image surface is exported from the outlet to facilitate post-processing.

Further, in an embodiment of the present invention, the modulation mask 200 is located on the image plane, and the dispersion unit is located on the Fourier plane.

Further, in an embodiment of the present invention, the Fourier plane at the dispersion unit 400 and the image plane at the image sensor 500 are implemented by a lens or a 4f system composed of lenses.

Further, in one embodiment of the present invention, the 4f system is used to add dispersion modulation and imaging magnification adjustment.

Further, in an embodiment of the present invention, the filter 300 is located after the first-stage lens of the 4f system on the imaging optical path, or is directly added inside the microscope 100 or other positions in the optical path to constrain the imaging spectral range.

It can be understood that the optical filter 300 in the embodiment of the present invention may be located behind the first-stage lens of the 4f system behind the modulation mask 200, and the optical filter 300 is used to constrain the spectral range of the optical signal from the sample. That is to say, after passing through the filter 300, the optical signal from the sample can only pass within a specific bandwidth range, thereby limiting the range of the latter-stage dispersion. It should be noted that, in the actual imaging optical path, the filter 300 is not strictly limited, that is, the position is adjustable.

Among them, the 4f system can be two lenses or lenses with different focal lengths, and the rear focal plane of the front lens and the front focal plane of the rear lens coincide. If the image plane of the imaging optical path is located on the front focal plane of the front-stage lens, the enlarged or reduced image plane will be obtained at the back focal plane of the rear-stage lens.

Further, in an embodiment of the present invention, the random mask can adjust the lateral position of the mask by means of translation.

It can be understood that the modulation mask 200 is located on the microscopic output image plane, and the modulation mask 200 is used to limit the field of view and add random mask modulation, wherein the random mask can adjust the lateral position of the mask by means of translation. .

Optionally, in an embodiment of the present invention, the dispersive unit is an Amoxie dispersion prism.

Optionally, in an embodiment of the present invention, the image sensor 500 may be a SCMOS monochrome sensor.

It can be understood that the image sensor 500 can be located on the back focal plane of the second lens of the 4f system, and the image sensor 500 is used to record the imaging results of mask modulation and dispersion modulation, wherein the image sensor 500 adopts a SCMOS single color camera.

To sum up, in the embodiment of the present invention, a microscope objective lens can be used to add binary random mask modulation to the output sample fluorescence image surface, the spectral range can be restricted by the filter 300, and then dispersion modulation can be added to the Fourier surface, using 4f The system adjusts the magnification and dispersion range, and the image sensor 500 records the modulated image. The system 10 of the embodiment of the present invention has low excitation light source power, can be applied to the imaging of photosensitive samples and biological samples, reduces fluorescent bleaching and damage to samples, has fast imaging speed, and high image reconstruction resolution.

For example, as shown in FIG. 2 , in an embodiment of the present invention, the system includes: a microscope 100 , a modulation mask 200 , a filter 300 , a dispersion unit 400 and an image sensor 500 .

Specifically, as shown in FIG. 2 , the microscopic sample 107 is located on the focal plane of the microscope objective lens 106 of the microscope 100 ; the excitation light emitted by the laser light source 104 is reflected by the dichroic mirror 105 and then focused by the objective lens 106 to excite the sample 107 fluorescence signal. The fluorescent signal passes through the objective lens 106, passes through the dichroic mirror 105, and then passes through the mirror 103 and the tube mirror 102 to be imaged to the image plane outlet 101; a random mask is placed on the image plane for modulation to limit the field of view and add modulation ; Further, the image is connected to the dispersion prism 400 through the first lens of the 4f system and the filter 300; the optical signal after dispersion is imaged to the image sensor 500 on the back focal plane through the second lens of the 4f system.

The above-mentioned microscope 100 can be either a traditional commercial microscope, an inverted microscope or an upright microscope according to different applications, and is not limited to the inverted microscope shown in FIG. It is known to those skilled in the art, and will not be described in detail here.

In addition, for the multi-labeled fluorescent samples collected in the embodiment of the present invention with different modulation imaging processing, the single-labeled spectral response curve can be used to reconstruct the single-labeled imaging. Firstly, an overcomplete sparse representation dictionary is generated by imaging training of monochromatic labeled non-dispersive samples; the imaging process is regarded as a convolution process of the ground truth image and the labeled spectral response; by calculating the sparse representation coefficient of the dispersive image on the product of the dispersion kernel and the dictionary, Revert out to single-color marker imaging.

It should be noted that the matching of the numerical apertures of the systems at all levels, the numerical aperture provided by the objective lens itself, and at the same time it is necessary to select an appropriate dispersion kernel and the size of the over-trained dictionary.

According to the multi-spectral microscopy imaging system proposed in the embodiment of the present invention, it is possible to collect data with a single or multiple exposures under a single camera, to restore a single-color mark image, and to realize that only one or two images can be used to reconstruct the image. A variety of marked images are used to improve imaging speed, improve image resolution, reduce laser damage to photosensitive samples and biological samples, and have simple structure and low cost.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " Rear, Left, Right, Vertical, Horizontal, Top, Bottom, Inner, Outer, Clockwise, Counterclockwise, Axial, The orientations or positional relationships indicated by "radial direction", "circumferential direction", etc. are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the indicated devices or elements. It must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be construed as a limitation of the present invention.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

In the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between the two elements, unless otherwise specified limit. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

In the present invention, unless otherwise expressly specified and limited, a first feature "on" or "under" a second feature may be in direct contact between the first and second features, or the first and second features indirectly through an intermediary touch. Also, the first feature being "above", "over" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is level higher than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature has a lower level than the second feature.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (8)

1. a multispectral microscope imaging system, is characterized in that, comprises: Microscope, used to magnify and image the microscopic sample to the image plane exit; A modulation mask, the modulation mask is located on the output image plane of the microscope to constrain the range of the imaging field of view of the subsequent stage, and random mask modulation is added, wherein the random mask adjusts the lateral position of the mask by means of translation ; field diaphragm; an optical filter, the optical filter is located behind the field diaphragm to perform narrow-band filtering on the illumination information from the microscopic sample to constrain the imaging spectral range of the system; a dispersion unit for spatially expanding the spectral information of each point on the sample; and Image sensor for recording pre-modulated sample images to reconstruct a variety of marked images in one or two pieces through dispersion and mask modulation; The reconstruction module obtains an image through the dispersion and mask modulation, and reconstructs the fluorescent protein-labeled tissue structure of the sample based on the spectral response curve of the labeled fluorescent protein; wherein, each multi-labeled sample can be obtained through a single shot. imaging. 2 . The multispectral microscope imaging system according to claim 1 , wherein the microscope is a wide-field fluorescence microscope. 3 . 3 . The multispectral microscope imaging system according to claim 1 , wherein the modulation mask is located on the image plane, and the dispersion unit is located on the Fourier plane. 4 . 4 . The multispectral microscope imaging system according to claim 3 , wherein the Fourier plane at the dispersion unit and the image plane at the image sensor are realized by a 4f system composed of lenses or lenses. 5 . 5 . The multispectral microscope imaging system according to claim 4 , wherein the 4f system is used for adding dispersion modulation and imaging magnification adjustment. 6 . 6 . The multispectral microscope imaging system according to claim 5 , wherein the filter is located behind the first-stage lens of the 4f system on the imaging optical path, or directly added to the inside of the microscope or other optical paths. 7 . position to constrain the imaging spectral range. 7 . The multispectral microscope imaging system according to claim 1 , wherein the dispersion unit is an Amoxie dispersion prism. 8 . 8 . The multispectral microscope imaging system according to claim 1 , wherein the image sensor is a SCMOS monochromatic sensor. 9 .

Images