CN113341553A - Fourier laminated microscopic color imaging method - Google Patents

Fourier laminated microscopic color imaging method Download PDF

Info

Publication number
CN113341553A
CN113341553A CN202110606617.8A CN202110606617A CN113341553A CN 113341553 A CN113341553 A CN 113341553A CN 202110606617 A CN202110606617 A CN 202110606617A CN 113341553 A CN113341553 A CN 113341553A
Authority
CN
China
Prior art keywords
color
resolution
image
rgb
low
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110606617.8A
Other languages
Chinese (zh)
Other versions
CN113341553B (en
Inventor
赵巨峰
张培伟
崔光茫
吴小辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110606617.8A priority Critical patent/CN113341553B/en
Publication of CN113341553A publication Critical patent/CN113341553A/en
Application granted granted Critical
Publication of CN113341553B publication Critical patent/CN113341553B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/06—Means for illuminating specimens
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Optics & Photonics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

The invention discloses a Fourier laminated microscopic color imaging method, which comprises the following steps: controlling the lighting of LED lamps with different angles in the RGB three-color two-dimensional LED programmable matrix; the emergent wave of the sample to be detected is transmitted to a single-color CMOS to obtain a gray low-resolution intensity image; repeating the steps S1-S3 to obtain a plurality of low-resolution intensity images; synthesizing the acquired three-color low-resolution gray level image into a color low-resolution intensity image of an RGB model and converting the color low-resolution intensity image into an image of an l alpha beta model; processing and fusing parameters in the l alpha beta model to obtain a color high-resolution image of the l alpha beta model; and converting the color high-resolution image of the l alpha beta model into a color high-resolution image of an RGB model. The color image with high resolution can be obtained, so that the gray image is converted into the color image, the impression of the imaging result is improved, and the imaging time is greatly shortened.

Description

Fourier laminated microscopic color imaging method
Technical Field
The invention relates to the technical field of image processing, in particular to a Fourier laminated microscopic color imaging method.
Background
In the biomedical fields such as digital pathology, hematology and the like, there is an increasing demand for examination of pathological cells using a microscope, but there is a problem that the resolution of observation and the field of view are not compatible when observation is performed using a conventional microscope.
The Fourier Ptychographic Microscopy (FPM) is an emerging computational imaging technique that can effectively solve the problem that the resolution and the field of view observed in the conventional microscopic imaging field cannot be compatible. The technology combines the concepts of phase retrieval and synthetic aperture, and acquires low-resolution images of the sample under different angles of illumination on a microscope platform, wherein the low-resolution images of different angles correspond to different sample frequency spectrum information of frequency domains. And then, iterating the series of low-resolution images in a frequency domain by using the ideas of phase restoration and synthetic aperture, and finally restoring the gray level image of the sample with high resolution and large field of view by expanding the frequency domain. Information carried by the gray-scale image is far inferior to that of a color image, and the color microscopic picture plays an important role in pathological cell observation, medical research and other conditions.
There are data showing that the conventional color fourier stack imaging technique: the method needs to collect low-resolution images of a sample under the irradiation of red, green and blue (RGB) three kinds of LED lamps respectively, then carries out high-resolution image reconstruction on the images respectively, and then synthesizes the images into a high-resolution color image, and the method has the main defects that: the time required for lighting the three-color LED lamps during acquisition is three times that of the conventional FPM technology, the data size obtained is also three times larger, and the time required for reconstruction is also three times, so that the reconstruction efficiency is greatly affected. And because light sources with different wavelengths are needed, the chromatic aberration generated by the objective lens or the sleeve lens also affects the quality of the picture.
Therefore, a fourier stacked microscopy color imaging method is proposed herein, which mainly converts the acquired low resolution image from RGB color space to l α β color space, where the l component represents luminance information and the α β component represents color information. And then reconstructing the component I of the image of the component l alpha beta to obtain a high-resolution component I image. The obtained high-resolution l component image needs to be synthesized into a high-resolution l α β picture with the α β component image, but since the α β component picture is of a low resolution and is not matched with the l component picture, a low-resolution to high-resolution conversion model is further proposed herein to convert the low-resolution α β component picture into the high-resolution α β component picture. And then combining the l component image obtained by reconstruction and the alpha beta component image with high resolution to obtain an l alpha beta image, and converting the l alpha beta color space into an RGB color space to obtain a color image with high resolution. The colorization algorithm provided by the invention enables the gray level image to be converted into the color image, and the impression of the imaging result is improved.
Chinese patent document CN111610623A discloses a "fourier-stack-based microscopic imaging method". The method comprises the steps of lighting lamp beads in an annular LED area bright field area, and collecting bright field images; and (3) sequentially lightening lamp beads in a dark field area of the annular LED area array, and collecting 12 dark field images. Initializing an image information frequency spectrum by using the collected bright field image; carrying out constraint by using the shot bright field image, and recovering bright field spectrum information of the image; carrying out constraint by using the shot dark field image, and recovering dark field frequency spectrum information of the image; the constraint process is repeated until the image converges. The technical scheme finally recovers the gray-scale image with high resolution and large visual field of the sample, and the information carried by the gray-scale image is far inferior to that of a color image, so that the color microscopic image plays an important role in pathological cell observation, medical research and other conditions.
Disclosure of Invention
The invention mainly solves the technical problems that the efficiency of acquiring and reconstructing images is low and the image quality is affected by chromatic aberration generated by adopting light sources with different wavelengths, objective lenses or sleeve lenses, and provides a Fourier laminated micro-color imaging method.
The technical problem of the invention is mainly solved by the following technical scheme: the invention comprises the following steps:
s1, controlling the lighting of LED lamps with different angles in the RGB three-color two-dimensional LED programmable matrix;
s2, transmitting the light lightened by the RGB three-color two-dimensional LED programmable matrix to a sample plane to be detected and transmitting the light;
s3, transmitting the emergent wave of the sample to be detected to a single-color CMOS to obtain a gray low-resolution intensity image;
s4 repeating the steps S1-S3, and respectively lighting LEDs at different angles on the RGB three-color LED lamps to obtain a plurality of low-resolution intensity images;
s5, synthesizing the acquired three-color low-resolution gray level image into a color low-resolution intensity image of the RGB model;
s6, converting the synthesized RGB model image into a l alpha beta model image;
s7, processing and fusing the parameters in the l alpha beta model to obtain a color high-resolution image of the l alpha beta model;
s8 converts the color high-resolution image of the l α β model into a color high-resolution image of an RGB model.
Preferably, in step S1, the LED lamps of the RGB three-color two-dimensional LED programmable matrix are controlled to be red lamps, and then the LED lamps at different angles are sequentially turned on from the middle to the periphery in a "loop" shape, and the emitted light wave is recorded as light wave
Figure BDA0003087541150000031
Wherein the subscript r denotes red light, kxi,kyiRepresenting wavelengths in the x and y directions, respectively; after the red light is finished, the LED light of the RGB three-color two-dimensional LED programmable matrix is controlled to be green, and the light wave is recorded as
Figure BDA0003087541150000032
Wherein the subscript g represents a green lamp; after the green LED lamp is lighted, the LED lamp of the RGB three-color two-dimensional LED programmable matrix is controlled to be blue, and the light wave is recorded as
Figure BDA0003087541150000033
Wherein the subscript b denotes a blue lamp.
Preferably, the propagation process of step S2 specifically includes:
s2.1, transmitting light lightened by the RGB three-color two-dimensional LED programmable matrix to a plane of a sample to be detected, and moving the frequency spectrum of the sample to be detected;
s2.2, emergent light of a sample to be detected enters a microscope objective, namely Fourier change is carried out;
s2.3, emergent light passing through the microscope objective enters a pupil aperture, and the low-pass filtering is performed on the frequency spectrum of the sample in a Fourier domain;
and S2.4, emergent light passing through the pupil aperture only contains low-frequency partial frequency spectrum information, and then secondary Fourier change is carried out through the sleeve lens.
Preferably, in step S2.1, the sample to be measured is represented by a space function O (x, y), light wave
Figure BDA0003087541150000041
The light is transmitted to a sample O (x, y) to be measured through a space distance to generate an emergent wave
Figure BDA0003087541150000042
S2.2 emergent wave of sample to be measured
Figure BDA0003087541150000043
Through a microscope objective, Fourier transformation is carried out
Figure BDA0003087541150000044
S2.3, the light wave passing through the microscope objective is low-pass filtered by a pupil function P (x, y) of a pupil diaphragm, so that the light wave passing through the pupil only comprises a frequency spectrum in a cut-off frequency range;
s2.4, emergent light of the pupil aperture only contains low-frequency partial frequency spectrum information, then the second Fourier change is carried out through the sleeve lens, and the result is recorded as Or,fil(x,y)。
Preferably, the gray low-resolution intensity image of step S3 is recorded as Ir,nOr Ig,nOr Ib,nWhere r denotes a red light beat, g denotes a blue light beat, b denotes a green light beat, n denotes the second light in the Led matrix, Ir,n＝|Or,fil(x,y)|2。
Preferably, in step S4, a 15 × 15 RGB three-color two-dimensional LED programmable matrix is used, and there are 225 angular LED lamps in total, so that 225 × 3 low-resolution intensity pictures based on red, green and blue lamps are obtained.
Preferably, the step S6 is to synthesize the image I of RGB modelnLow resolution image I converted into l alpha beta modeln,lαβThe method comprises the following specific steps;
s6.1 converts the synthesized RGB image into the LMS color space, represented by the responses of the three cones of the human eye, by the following formula:
Figure BDA0003087541150000051
s6.2, data in the LMS space are dispersed, so that the LMS space is further converted into a logarithmic space with a base 10; therefore, the data distribution is converged, and the psychophysical research result of color perception of human beings is met.
Figure BDA0003087541150000052
S6.3 is converted from the LMS space to l alpha beta space, wherein l is a first principal component, alpha is a second principal component, and beta is a third principal component.
Figure BDA0003087541150000053
The conversion from RGB to l α β space is accomplished through the above three steps.
Preferably, the step S7 specifically includes:
s7.1, only l in the l alpha beta model is reconstructed according to an iterative algorithm, and an image containing high-resolution intensity is reconstructed, wherein the method specifically comprises the following steps: first, in the air space, the intensity information of the sample is Il,nMake a replacement
Figure BDA0003087541150000054
Then the data are reversely transmitted to a sample plane to be detected, the sample function is updated in the Fourier domain according to the support domain constraint, and a high-resolution picture I can be reconstructed after multiple updatesgp；
S7.2, performing resolution filling on the α β component using a high resolution conversion model to fill the low resolution α β component picture into a high resolution α β component picture, specifically including: based on the already reconstructed high-resolution image Igp,nAnd a low resolution picture I without reconstructionlp,1Blind recovery h (x, y) is expressed as:
Figure BDA0003087541150000061
wherein
Figure BDA0003087541150000062
Is a fidelity term, r1,r2Is a parameter of regularization;
the calculation method for solving the above equation is: low resolution picture I with initially given l componentl,1And high resolution picture IgpThen calculates the expression until the condition is satisfied
Figure BDA0003087541150000063
Then the expression converges, and after the transformation function h (x, y) is obtained, the expression:
ugp(x,y)＝ulp(x,y)*h(x,y)
substituting the low-resolution alpha beta component picture into the formula, and solving the high-resolution alpha beta component picture;
and S7.3, fusing the alpha beta and the reconstructed high-resolution l component image to obtain a color high-resolution image of the l alpha beta model.
Preferably, the step S8 converts the color high resolution image of the l α β model into the color high resolution image of the RGB model according to the following formula:
s8.1 conversion from l α β space to LMS logarithmic space
Figure BDA0003087541150000064
S8.2 conversion from LMS logarithmic space to LMS linear space
Figure BDA0003087541150000065
S8.3 conversion from LMS space to RGB space
Figure BDA0003087541150000071
And obtaining a color picture based on Fourier laminated microscopic imaging reconstruction through the formula.
Preferably, the Fourier laminated micro color imaging device comprises an RGB three-color two-dimensional LED programmable matrix, a glass slide, a sample to be detected, a microscope objective, a pupil aperture, a sleeve lens and a monochromatic CMOS camera which are sequentially arranged from bottom to top, LED lamps for respectively displaying three colors of red, green and blue are uniformly arranged on the surface of the RGB three-color two-dimensional LED programmable matrix, and the RGB three-color two-dimensional LED programmable matrix and the monochromatic CMOS camera are connected with a computer. The RGB three-color two-dimensional LED programmable matrix is used for illuminating a sample to be detected, the pupil aperture is mainly in a Fourier domain, when the frequency spectrum of the sample to be detected is expanded by illuminating light at different angles, the pupil function of the pupil aperture performs low-pass filtering on the frequency spectrum, the pupil aperture is used as a support domain constraint in the reconstruction process, so that the reconstruction can be converged under the following intensity constraint, the complementary metal oxide semiconductor CMOS is an optical sensor and is mainly used for recording a microscopic image of the image, the image quality is high.
The invention has the beneficial effects that: the method comprises the steps of converting an obtained low-resolution image from an RGB color space to an l alpha beta color space, then reconstructing an l component of the l alpha beta image to obtain a high-resolution l component image, combining the l component image obtained through reconstruction and the high-resolution alpha beta component image to obtain an l alpha beta image, and converting the l alpha beta color space to the RGB color space to obtain a high-resolution color image, so that the gray image is converted into the color image, the impression of an imaging result is improved, and the imaging time is greatly shortened.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a diagram of an apparatus for Fourier stacked microscopy imaging of the present invention.
Fig. 3 is a diagram of an R, G, B three-color two-dimensional LED programmable matrix of the present invention.
Fig. 4 is a flow chart of a fourier stacked microscopy imaging grayscale image colorization of the present invention.
In the figure, a RGB three-color two-dimensional LED programmable matrix 1, a glass slide 2, a sample to be detected 3, a microscope objective 4, a pupil aperture 5, a sleeve lens 6, a monochromatic CMOS camera 7, a central LED8 and an LED 9 are lighted.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
Example (b): in the fourier stacked micro color imaging method of this embodiment, as shown in fig. 2, the fourier stacked micro color imaging apparatus includes an RGB three-color two-dimensional LED programmable matrix 1, a glass slide 2, a sample 3 to be measured, a microscope objective 4, a pupil aperture 5, a sleeve lens 6, and a monochrome CMOS camera 7, which are sequentially arranged from bottom to top.
As shown in fig. 3, 15 × 15 LED lamps with 225 angles are uniformly arranged on the surface of the RGB three-color two-dimensional LED programmable matrix 1, the LED lamps can respectively display three colors of red, green and blue, and the RGB three-color two-dimensional LED programmable matrix 1 and the monochrome CMOS camera 7 are connected to a computer. The LED lamps with different angles are lighted in a 'loop' shaped LED lighting mode 9 from the center LED8 to the periphery in sequence. The RGB three-color two-dimensional LED programmable matrix is used for illuminating a sample to be detected, the pupil aperture is mainly in a Fourier domain, when the frequency spectrum of the sample to be detected is expanded by illuminating light at different angles, the pupil function of the pupil aperture performs low-pass filtering on the frequency spectrum, the pupil aperture is used as a support domain constraint in the reconstruction process, so that the reconstruction can be converged under the following intensity constraint, the complementary metal oxide semiconductor CMOS is an optical sensor and is mainly used for recording a microscopic image of the image, the image quality is high.
As shown in fig. 1, a fourier stacked microscopy color imaging method comprises the following steps:
s1 controls the lighting of the LED lamps with different angles in the RGB three-color two-dimensional LED programmable matrix 1. Firstly controlling an LED lamp of an RGB three-color two-dimensional LED programmable matrix 1 to be a red lamp, then sequentially lightening LED lamps with different angles from the middle to the periphery in a 'return' shape mode, and recording that light waves are emitted
Figure BDA0003087541150000091
Wherein the subscript r denotes red light, kxi,kyiRepresenting wavelengths in the x and y directions, respectively(ii) a After the red light is finished, the LED light of the RGB three-color two-dimensional LED programmable matrix 1 is controlled to be green, and the light wave is recorded as
Figure BDA0003087541150000092
Wherein the subscript g represents a green lamp; after the green LED lamp is lighted, the LED lamp of the RGB three-color two-dimensional LED programmable matrix 1 is controlled to be blue, and the light wave is recorded as
Figure BDA0003087541150000093
Wherein the subscript b denotes a blue lamp.
The light lightened by the S2 RGB three-color two-dimensional LED programmable matrix 1 is transmitted to the plane of the sample 3 to be measured and transmitted, and the transmission process specifically comprises the following steps:
s2.1, transmitting the light lightened by the RGB three-color two-dimensional LED programmable matrix 1 to the plane of a sample 3 to be detected, and moving the frequency spectrum of the sample 3 to be detected; the sample 3 to be measured is represented by a spatial function O (x, y), light wave
Figure BDA0003087541150000094
The light is transmitted to a sample to be measured through a space distance to generate 3O (x, y) interaction, and emergent waves are generated
Figure BDA0003087541150000095
S2.2, emergent light of a sample 3 to be detected enters a microscope objective 4, namely Fourier change is carried out; emergent wave of sample 3 to be measured
Figure BDA0003087541150000096
Through the microscope objective 4, Fourier transformation is carried out
Figure BDA0003087541150000097
S2.3, emergent light passing through the microscope objective 4 enters a pupil aperture 5, and the low-pass filtering is performed on the frequency spectrum of the sample in a Fourier domain; the light wave passing through the microscope objective 4 is low-pass filtered by the pupil function P (x, y) of the pupil diaphragm 5, so that the light wave passing through the pupil contains only a spectrum in the cut-off frequency range.
S2.4, emergent light passing through the pupil diaphragm 5 only contains low-frequency part frequency spectrum information, then secondary Fourier change is carried out through the sleeve lens 6, emergent light passing through the pupil diaphragm 5 only contains low-frequency part frequency spectrum information, secondary Fourier change is carried out through the sleeve lens 6, and the result is recorded as Or,fil(x,y)。
S3 the emergent wave of the sample 3 to be measured is transmitted to the single-color CMOS to obtain the gray low-resolution intensity image. The gray low resolution intensity image is denoted as Ir,nOr Ig,nOr Ib,nWhere r denotes a red light beat, g denotes a blue light beat, b denotes a green light beat, n denotes the second light in the Led matrix, Ir,n＝|Or,fil(x,y)|2. S4 repeats steps S1 to S3, and turns on LEDs of different angles for the RGB three-color LED lamps, respectively, thereby acquiring a plurality of low-resolution intensity images. Using 15 × 15 RGB three-color two-dimensional LED programmable matrix 1, there are 225 angular LED lamps in total, so 225 × 3 low-resolution intensity pictures based on red, green, and blue lamps, respectively, are acquired.
S5, synthesizing the acquired three-color low-resolution gray level image into a color low-resolution intensity image of the RGB model;
s6 image I of synthesized RGB modelnLow resolution image I converted into l alpha beta modeln,lαβThe method comprises the following specific steps;
s6.1 converts the synthesized RGB image into the LMS color space, represented by the responses of the three cones of the human eye, by the following formula:
Figure BDA0003087541150000101
s6.2, data in the LMS space are dispersed, so that the LMS space is further converted into a logarithmic space with a base 10; therefore, the data distribution is converged, and the psychophysical research result of color perception of human beings is met.
Figure BDA0003087541150000102
S6.3 is converted from the LMS space to l alpha beta space, wherein l is a first principal component, alpha is a second principal component, and beta is a third principal component.
Figure BDA0003087541150000111
The conversion from RGB to l α β space is accomplished through the above three steps.
S7 processes and fuses the parameters in the l α β model to obtain a color high resolution image of the l α β model, which specifically includes:
s7.1, only l in the l alpha beta model is reconstructed according to an iterative algorithm, and an image containing high-resolution intensity is reconstructed, wherein the method specifically comprises the following steps: first, in the air space, the intensity information of the sample is Il,nMake a replacement
Figure BDA0003087541150000112
Then the data are reversely transmitted to a 3 plane of a sample to be detected, a sample function is updated in a Fourier domain according to the support domain constraint, and a high-resolution picture I can be reconstructed after multiple updatesgp；
S7.2, performing resolution filling on the α β component using a high resolution conversion model to fill the low resolution α β component picture into a high resolution α β component picture, specifically including: based on the already reconstructed high-resolution image Igp,nAnd a low resolution picture I without reconstructionlp,1Blind recovery h (x, y) is expressed as:
Figure BDA0003087541150000113
wherein
Figure BDA0003087541150000114
Is a fidelity term, r1,r2Is a parameter of regularization;
the calculation method for solving the above equation is: low for initially given l componentResolution picture Il,1And high resolution picture IgpThen calculates the expression until the condition is satisfied
Figure BDA0003087541150000115
Then the expression converges, and after the transformation function h (x, y) is obtained, the expression:
ugp(x,y)＝ulp(x,y)*h(x,y)
substituting the low-resolution alpha beta component picture into the formula, and solving the high-resolution alpha beta component picture;
and S7.3, fusing the alpha beta and the reconstructed high-resolution l component image to obtain a color high-resolution image of the l alpha beta model.
S8 converts the color high resolution image of the l α β model into a color high resolution image of an RGB model, which specifically includes:
s8.1 conversion from l α β space to LMS logarithmic space
Figure BDA0003087541150000121
S8.2 conversion from LMS logarithmic space to LMS linear space
Figure BDA0003087541150000122
S8.3 conversion from LMS space to RGB space
Figure BDA0003087541150000123
And obtaining a color picture based on Fourier laminated microscopic imaging reconstruction through the formula.
A method for colorizing a gray image based on Fourier laminated microscopic imaging relates to the technical fields of computational imaging, microscopic imaging and the like, and mainly obtains a high-resolution image containing color information by synthesizing, converting, reconstructing and converting collected RGB low-resolution gray images. The invention provides a new solution for colorizing the gray level image of the Fourier laminated microscopic imaging, reduces the colorizing time, enables medical staff to analyze pathological cells more quickly and makes diagnosis more clearly.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Although terms like high resolution conversion models are used more here, the possibility of using other terms is not excluded. These terms are used merely to more conveniently describe and explain the nature of the present invention; they are to be construed as being without limitation to any additional limitations that may be imposed by the spirit of the present invention.

Claims (10)

1. A fourier stacked microscopy color imaging method, comprising the steps of:
s1, controlling the lighting of LED lamps with different angles in the RGB three-color two-dimensional LED programmable matrix (1);
the light lightened by the S2 RGB three-color two-dimensional LED programmable matrix (1) is transmitted to the plane of the sample (3) to be measured and transmitted;
s3, transmitting the emergent wave of the sample (3) to be detected to a single-color CMOS to obtain a gray low-resolution intensity image;
s4 repeating the steps S1-S3, and respectively lighting LEDs at different angles on the RGB three-color LED lamps to obtain a plurality of low-resolution intensity images;
s5, synthesizing the acquired three-color low-resolution gray level image into a color low-resolution intensity image of the RGB model;
s6, converting the synthesized RGB model image into a l alpha beta model image;
s7, processing and fusing the parameters in the l alpha beta model to obtain a color high-resolution image of the l alpha beta model;
s8 converts the color high-resolution image of the l α β model into a color high-resolution image of an RGB model.
2. The Fourier stacked microscopy color imaging method according to claim 1, wherein the step S1 comprises controlling the LED lamps of the RGB three-color two-dimensional LED programmable matrix (1) to be red lamps, and then sequentially lighting the LED lamps with different angles in a 'loop' shape from the middle to the periphery, and recording the light wave as light wave
Figure FDA0003087541140000011
Wherein the subscript r denotes red light, kxi,kyiRepresenting wavelengths in the x and y directions, respectively; after the red light is finished, the LED light of the RGB three-color two-dimensional LED programmable matrix (1) is controlled to be green, and the light wave is recorded as
Figure FDA0003087541140000012
Wherein the subscript g represents a green lamp; after the green LED lamp is lighted, the LED lamp of the RGB three-color two-dimensional LED programmable matrix (1) is controlled to be blue, and the light wave is recorded as
Figure FDA0003087541140000013
Wherein the subscript b denotes a blue lamp.
3. The fourier transform stack microscopy color imaging method according to claim 1, wherein the step S2 propagation process specifically comprises:
s2.1, transmitting the light lightened by the RGB three-color two-dimensional LED programmable matrix (1) to the plane of a sample (3) to be detected, and moving the frequency spectrum of the sample (3) to be detected;
s2.2, emergent light of a sample (3) to be detected enters a microscope objective (4), namely Fourier change is carried out;
s2.3, emergent light passing through the microscope objective (4) enters a pupil diaphragm (5), and the low-pass filtering is performed on the frequency spectrum of the sample in a Fourier domain;
and S2.4, emergent light passing through the pupil aperture (5) only contains low-frequency partial frequency spectrum information, and then secondary Fourier change is carried out through the sleeve lens (6).
4. A fourier stack microscopy colour imaging method as claimed in claim 1, characterized in that step S2.1 the sample (3) to be examined is represented by a spatial function O (x, y), light wave
Figure FDA0003087541140000021
Transmitted to a sample (3) to be measured through a space distance to generate an emergent wave through interaction
Figure FDA0003087541140000022
S2.2 outgoing wave of sample (3) to be measured
Figure FDA0003087541140000023
Through the microscope objective (4), Fourier change is carried out
Figure FDA0003087541140000024
S2.3, the light wave passing through the microscope objective (4) is low-pass filtered by a pupil function P (x, y) of a pupil diaphragm (5), so that the light wave passing through the pupil only comprises a frequency spectrum in a cut-off frequency range;
s2.4, the emergent light of the pupil aperture (5) only contains low-frequency partial frequency spectrum information, then the second Fourier change is carried out through the sleeve lens (6), and the result is recorded as Or,fil(x,y)。
5. The method of claim 1, wherein said step S3 is performed by recording a gray low resolution intensity image as Ir,nOr Ig,nOr Ib,nWhere r denotes a red light beat, g denotes a blue light beat, b denotes a green light beat, n denotes the second light in the Led matrix, Ir,n＝|Or,fil(x,y)|2。
6. A fourier stacked microscopy color imaging method according to claim 1, characterized in that step S4 uses a 15 x 15 RGB three-color two-dimensional LED programmable matrix (1) with a total of 225 angular LED lamps, thus obtaining 225 x 3 low resolution intensity pictures based on red, green and blue lamps respectively.
7. The fourier-stacked microscopy color imaging method as claimed in claim 1, wherein the step S6 synthesizes an image I of the RGB modelnLow resolution image I converted into l alpha beta modeln,lαβThe method comprises the following specific steps;
s6.1 converts the synthesized RGB image into the LMS color space, represented by the responses of the three cones of the human eye, by the following formula:
Figure FDA0003087541140000031
s6.2, data in the LMS space are dispersed, so that the LMS space is further converted into a logarithmic space with a base 10;
Figure FDA0003087541140000032
s6.3 is converted from the LMS space to l alpha beta space, wherein l is a first principal component, alpha is a second principal component, and beta is a third principal component.
Figure FDA0003087541140000033
The conversion from RGB to l α β space is accomplished through the above three steps.
8. The fourier transform stack microscopy color imaging method according to claim 7, wherein the step S7 specifically comprises:
s7.1, only l in the l alpha beta model is reconstructed according to an iterative algorithm, and an image containing high-resolution intensity is reconstructed, wherein the method specifically comprises the following steps: first, in the air space, the intensity information of the sample is Il,nMake a replacement
Figure FDA0003087541140000034
Then the data are reversely transmitted to the plane of a sample (3) to be detected, the sample function is updated in the Fourier domain according to the support domain constraint, and a high-resolution picture I can be reconstructed after multiple updatesgp；
S7.2, performing resolution filling on the α β component using a high resolution conversion model to fill the low resolution α β component picture into a high resolution α β component picture, specifically including: based on the already reconstructed high-resolution image Igp,nAnd a low resolution picture I without reconstructionlp,1Blind recovery h (x, y) is expressed as:
Figure FDA0003087541140000041
wherein
Figure FDA0003087541140000042
Is a fidelity term, r1,r2Is a parameter of regularization;
the calculation method for solving the above equation is: low resolution picture I with initially given l componentl,1And high resolution picture IgpThen calculates the expression until the condition is satisfied
Figure FDA0003087541140000043
Then the expression converges, and after the transformation function h (x, y) is obtained, the expression:
ugp(x,y)＝ulp(x,y)*h(x,y)
substituting the low-resolution alpha beta component picture into the formula, and solving the high-resolution alpha beta component picture;
and S7.3, fusing the alpha beta and the reconstructed high-resolution l component image to obtain a color high-resolution image of the l alpha beta model.
9. The fourier transform stacked microscopy color imaging method as claimed in claim 8, wherein the step S8 is to convert the color high resolution image of the l α β model into the color high resolution image of the RGB model by the following formula:
s8.1 conversion from l α β space to LMS logarithmic space
Figure FDA0003087541140000044
S8.2 conversion from LMS logarithmic space to LMS linear space
Figure FDA0003087541140000051
S8.3 conversion from LMS space to RGB space
Figure FDA0003087541140000052
And obtaining a color picture based on Fourier laminated microscopic imaging reconstruction through the formula.
10. The Fourier laminated micro-color imaging method according to claim 1, wherein the Fourier laminated micro-color imaging device comprises an RGB three-color two-dimensional LED programmable matrix (1), a glass slide (2), a sample to be detected (3), a microscope objective (4), a pupil aperture (5), a sleeve lens (6) and a monochromatic CMOS camera (7) which are sequentially arranged from bottom to top, LED lamps for respectively displaying three colors of red, green and blue are uniformly arranged on the surface of the RGB three-color two-dimensional LED programmable matrix (1), and the RGB three-color two-dimensional LED programmable matrix (1) and the monochromatic CMOS camera (7) are connected with a computer.
CN202110606617.8A 2021-05-27 2021-05-27 Fourier laminated microscopic color imaging method Active CN113341553B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110606617.8A CN113341553B (en) 2021-05-27 2021-05-27 Fourier laminated microscopic color imaging method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110606617.8A CN113341553B (en) 2021-05-27 2021-05-27 Fourier laminated microscopic color imaging method

Publications (2)

Publication Number Publication Date
CN113341553A true CN113341553A (en) 2021-09-03
CN113341553B CN113341553B (en) 2022-09-20

Family

ID=77473848

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110606617.8A Active CN113341553B (en) 2021-05-27 2021-05-27 Fourier laminated microscopic color imaging method

Country Status (1)

Country Link
CN (1) CN113341553B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110222062A1 (en) * 2008-02-01 2011-09-15 Palo Alto Research Center Incorporated Analyzers with time variation based on color-coded spatial modulation
US20140118529A1 (en) * 2012-10-30 2014-05-01 California Institute Of Technology Fourier Ptychographic Imaging Systems, Devices, and Methods
CN105911692A (en) * 2016-06-24 2016-08-31 南京理工大学 Optimal system parameter selection method for Fourier laminated micro imaging system
CN105976315A (en) * 2016-04-26 2016-09-28 清华大学深圳研究生院 Microscopic imaging method based on partial Fourier space
EP3232244A1 (en) * 2016-04-11 2017-10-18 Nederlandse Organisatie voor toegepast- natuurwetenschappelijk onderzoek TNO Microscopic imaging system and method of forming a microscopic image of an object
CN108550108A (en) * 2017-09-28 2018-09-18 武汉大学 A kind of Fourier's lamination image method for reconstructing minimized based on phase iteration
US20200209604A1 (en) * 2017-08-04 2020-07-02 Nanjing University Of Science And Technology Programmable annular led illumination-based high efficiency quantitative phase microscopy imaging method
US20210112187A1 (en) * 2017-03-24 2021-04-15 Nanjing University Of Science And Technology High-illumination numerical aperture-based large field-of-view high-resolution microimaging device and a method for iterative reconstruction

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110222062A1 (en) * 2008-02-01 2011-09-15 Palo Alto Research Center Incorporated Analyzers with time variation based on color-coded spatial modulation
US20140118529A1 (en) * 2012-10-30 2014-05-01 California Institute Of Technology Fourier Ptychographic Imaging Systems, Devices, and Methods
EP3232244A1 (en) * 2016-04-11 2017-10-18 Nederlandse Organisatie voor toegepast- natuurwetenschappelijk onderzoek TNO Microscopic imaging system and method of forming a microscopic image of an object
CN105976315A (en) * 2016-04-26 2016-09-28 清华大学深圳研究生院 Microscopic imaging method based on partial Fourier space
CN105911692A (en) * 2016-06-24 2016-08-31 南京理工大学 Optimal system parameter selection method for Fourier laminated micro imaging system
US20210112187A1 (en) * 2017-03-24 2021-04-15 Nanjing University Of Science And Technology High-illumination numerical aperture-based large field-of-view high-resolution microimaging device and a method for iterative reconstruction
US20200209604A1 (en) * 2017-08-04 2020-07-02 Nanjing University Of Science And Technology Programmable annular led illumination-based high efficiency quantitative phase microscopy imaging method
CN108550108A (en) * 2017-09-28 2018-09-18 武汉大学 A kind of Fourier's lamination image method for reconstructing minimized based on phase iteration

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
孙佳崇等: "傅里叶叠层显微成像技术：理论、发展和应用", 《光学学报》 *
李通等: "基于角度照明优化的傅里叶叠层显微成像方法", 《激光与光电子学进展》 *

Also Published As

Publication number Publication date
CN113341553B (en) 2022-09-20

Similar Documents

Publication Publication Date Title
CN106896489B (en) Frequency domain based on wavelength multiplexing pieces microscopic system and its method together
CN107209123B (en) Multi-aperture Fourier ptychographic and fluorescence imaging
JP7087198B2 (en) Hybrid spectrum imager
IL272907A (en) Microscopy of a tissue sample using structured illumination
CA3124052C (en) Systems and methods for image processing
JP2022529366A (en) Systems and methods for deep learning-based color holographic microscopes
WO2016107334A1 (en) Coloured three-dimensional tomography micro-imaging system and method based on structural illumination
US20150103401A1 (en) Reference color slide for use in color correction of transmission-microscope slides
JP2014137558A (en) Imaging device, imaging system and image processing method
CN115032196B (en) Full-scribing high-flux color pathological imaging analysis instrument and method
CN108648144B (en) FPM high-resolution color image reconstruction method based on deep learning algorithm
CN109407297A (en) A kind of multi-mode light field micro imaging method based on programmable LED array illumination
CN113568156A (en) Spectral microscopic imaging device and implementation method
CN109443535A (en) A kind of underwater microspectrum imaging device and method based on fast illuminated spectrum camera
Zhou et al. Accelerated Fourier ptychographic diffraction tomography with sparse annular LED illuminations
Zhou et al. Single-exposure 3D label-free microscopy based on color-multiplexed intensity diffraction tomography
CN113341553B (en) Fourier laminated microscopic color imaging method
US20220198653A1 (en) System and method for enhancing visualization of an eye image
CN111610623B (en) Fourier stack-based microscopic imaging method
CN113917677A (en) Three-dimensional super-resolution light sheet microscopic imaging method and microscope
CN113296259A (en) Super-resolution imaging method and device based on aperture modulation subsystem and deep learning
WO2022050109A1 (en) Image processing device, image processing method, and image processing system
CN115587961A (en) Cell imaging method based on multi-exposure image fusion technology
JP2016212412A (en) Method of generating contrast image of object structure, and related devices
US20210149177A1 (en) System and method for phase contrast microscopy imaging

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant