CN113066170A - Differential interference phase contrast imaging method based on deep learning - Google Patents
Differential interference phase contrast imaging method based on deep learning Download PDFInfo
- Publication number
- CN113066170A CN113066170A CN202110410841.XA CN202110410841A CN113066170A CN 113066170 A CN113066170 A CN 113066170A CN 202110410841 A CN202110410841 A CN 202110410841A CN 113066170 A CN113066170 A CN 113066170A
- Authority
- CN
- China
- Prior art keywords
- image
- differential interference
- bright field
- neural network
- phase contrast
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 24
- 238000013135 deep learning Methods 0.000 title claims abstract description 10
- 238000013528 artificial neural network Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims abstract description 8
- 238000011084 recovery Methods 0.000 claims abstract description 7
- 238000003062 neural network model Methods 0.000 claims description 10
- 230000003287 optical effect Effects 0.000 claims description 6
- 230000001427 coherent effect Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 claims description 2
- 229910000673 Indium arsenide Inorganic materials 0.000 claims 1
- RPQDHPTXJYYUPQ-UHFFFAOYSA-N indium arsenide Chemical compound [In]#[As] RPQDHPTXJYYUPQ-UHFFFAOYSA-N 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 9
- 238000000034 method Methods 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 abstract description 2
- 210000004027 cell Anatomy 0.000 description 12
- 238000010586 diagram Methods 0.000 description 4
- 229910052736 halogen Inorganic materials 0.000 description 4
- 150000002367 halogens Chemical class 0.000 description 3
- 210000001519 tissue Anatomy 0.000 description 3
- 238000002135 phase contrast microscopy Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 210000000805 cytoplasm Anatomy 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 125000001475 halogen functional group Chemical group 0.000 description 1
- 210000003463 organelle Anatomy 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
Abstract
The invention discloses a differential interference phase contrast imaging method based on deep learning, which can process a common bright field image into a differential interference phase contrast image, realize the pseudo three-dimensional effect of an object and enhance the display effect of a phase type object. The method comprises the steps of acquiring a data set by simultaneously acquiring a plurality of common bright field images of phase type samples and differential interference images corresponding to the common bright field images, and learning a differential interference phase contrast imaging mode through a constructed neural network. After training is finished, the pseudo three-dimensional effect of the object can be realized only by one in-focus intensity map, the calculation speed is high, the recovery accuracy is high, meanwhile, the observation capability of a phase type object is enhanced by combining a common bright field microscope, and the functions of the conventional bright field microscope are expanded.
Description
Technical Field
The invention relates to the field of optics, in particular to the field of microscopic imaging.
Background
The ordinary bright field microscope is a most commonly used microscopic instrument, helps people to effectively observe the microscopic world, and in the actual observation and imaging of biological tissue cells, the problem that most cells are tiny and transparent, the optical absorption coefficient of cytoplasm and most organelles is small, so that the amplitude of light waves after passing through the cells cannot be obviously changed, but the phase part changes greatly, after passing through a detector, only the intensity information (the square of the amplitude) of a light beam can be directly observed and recorded, the phase information is lost, and the bright field microscope is difficult to directly observe the cells. It is conventional practice to restore the visualization of the clear cells by converting phase changes into intensity changes. Phase contrast microscopy is based on the principle of spatial filtering and greatly improves the resolution of transparent cells under a bright field microscope by installing a coated glass substrate on the microscope. But also introduces a 'halo' effect, which hampers the observation of the clear cells. Differential interference phase contrast microscopy utilizes a Wollaston prism to form two beams of light and generate proper transverse shearing component and longitudinal separation quantity, thereby realizing the pseudo three-dimensional relief effect of transparent cells and enhancing the display of the cells.
Disclosure of Invention
The invention provides a pseudo three-dimensional effect imaging method based on deep learning, which is high in calculation speed, can realize the pseudo three-dimensional effect of an object by only one in-focus intensity map, and can be combined with a traditional common bright field microscope to improve the observation capability of transparent tissue cells.
Technical scheme
The invention is technically characterized by comprising two stages of training and recovering, and is divided into the following steps:
a. the training phase comprises the following steps:
s1, collecting differential interference phase contrast images of training samples by using a microscopic imaging system and recording the images as InWherein n is 1,2,3,4 … k;
s2, collecting a common bright field in-focus image of a training sample by using a microscopic imaging system, and recording the image as pinWherein n is 1,2,3,4 … k;
s3, establishing a neural network model, determining network model parameters, taking a common bright field in-focus image of a training sample as input of the neural network, taking a corresponding differential interference phase contrast image as output of the neural network model, and training the neural network model;
b. the recovery phase comprises the following steps:
s4, collecting a common bright field in-focus image of the sample to be detected by using a common bright field microscope;
s5, inputting the bright field of the sample to be detected into the trained neural network in the focal image to obtain a differential interference phase contrast image of the sample to be detected;
the microscopic imaging system of step S1 is an optical microscope or the like with differential interference function, and the light source thereof may be a coherent light source or a partially coherent light source.
The training sample of step S1 can be any phase-type object that can be used for imaging, including biological tissue, biological cells, industrial elements, etc.
And the differential interference phase contrast image of the acquired training sample in the step S1 is a three-dimensional relief image.
The microscopic imaging system of the step S2 is a normal optical microscope.
The neural network model in step S3 may be any end-to-end neural network model for image conversion, such as an Unet model based on a convolutional neural network, and the frame for constructing the neural network may be a pitoch, tenserflow, and the like, and the network only needs to be trained once, and then can realize a similar differential interference phase contrast image for the sample to be measured indefinitely.
Advantageous effects
After the construction and training of the whole neural network are completed, the differential interference phase contrast image of the sample can be recovered only by shooting an in-focus image by a common bright field microscope and inputting the in-focus image into the trained neural network, so that the three-dimensional relief effect is realized, and the observation of transparent cells is enhanced. The method can be combined with a common bright field microscope at low cost, realizes the differential interference phase contrast function, expands the functions of the conventional bright field microscope, has rapid recovery process and high recovery accuracy, and can realize real-time synchronous imaging.
Description of the drawings:
FIG. 1 is a flow chart of an imaging method of differential interference phase contrast image based on depth learning;
FIG. 2 is a block diagram of a U-net used in an embodiment;
fig. 3 is an optical path diagram of a microscope for acquiring a differential interference phase contrast image, wherein the differential interference phase contrast function is realized in advance.
In fig. 1: the solid part is the training phase and the dashed part is the recovery phase.
In fig. 2: in the network structure, the down-sampling process uses a convolution network with a residual error network, the up-sampling process uses a transposed convolution network with a residual error grid, and the size of all convolution kernels is 3x 3.
In fig. 3: the system comprises a 1-halogen lamp light source, a 2-polarizer, a 3-Wollaston prism, a 4-convergent lens, a 5-sample, a 6-objective lens, a 7-Wollaston prism, an 8-analyzer and a 9-CCD camera.
Detailed Description
The invention will now be further described with reference to the examples, and the accompanying drawings:
example 1: a differential interference phase contrast optical path for implementing the method is shown in fig. 3, and comprises: the device comprises a halogen lamp light source 1, a polarizer 2, a Wollaston prism 3, a converging lens 4, a sample 5, an objective lens 6, a Wollaston prism 7, an analyzer 8 and a CCD camera 9. The halogen lamp is divided into two beams of orthogonal polarized light through the Wollaston prism after passing through the polarizer, the light is collimated through the lens mirror, the light carries phase gradient information of a sample after passing through the sample, the light is combined through the objective lens and the Wollaston prism, an interference process is completed through the analyzer, and finally a differential interference phase contrast image is recorded by the CCD camera. The halogen lamp can be replaced by an LED light source, the magnification of the microscope objective is determined by actual conditions, and the transverse position of the Wollaston prism is used for adjusting the contrast of the differential interference pattern.
The working flow of the deep learning differential interference phase contrast imaging method is as follows:
and (3) executing a training phase: common bright field map pi for taking sample by microscopenSimultaneously switching to the differential interference phase contrast function to acquire the differential interference phase contrast image I of the sample corresponding to the differential interference phase contrast function one by onenChanging the field of view, repeating the above steps to obtain pi of k samplesnAnd InWhere n is 1,2,3,4 … k (5000). II common field diagramnAs input, the corresponding differential interference phase contrast diagram InMaking an output standard, and training the neural network model shown in fig. 2, wherein the network training parameters are as follows: batch size 64, Decay rate 0.95, Learning rate 0.0015, Epoch 80Shuffle, frequency 1/Epoch. The GPU used for network training is GTX 1080Ti, and the training time is 5 hours.
And (3) executing a recovery phase: after the training stage is executed once, the trained network can be used for recovering the common bright field pattern, and a corresponding differential interference phase contrast image can be obtained by inputting a bright field pattern. Only 0.005 second is needed to restore a normal bright field map of 128x128 pixels.
The result is restored to be close to the reality through the neural network.
Claims (5)
1. A differential interference phase contrast imaging method based on deep learning is characterized by comprising two stages of training and recovering and comprising the following steps:
a. the training phase comprises the following steps:
s1, collecting differential interference image lining image of training sample by using microscopic imaging system, and recordingAs InWherein n is 1,2,3,4 … k;
s2, collecting a common bright field in-focus image of a training sample by using a microscopic imaging system, and recording the image as pinWherein n is 1,2,3,4 … k;
s3, establishing a neural network model, determining network model parameters, and setting the common bright field of the training sample in a focal map IInAs input to the neural network, the corresponding differential interference image-lined image InAs the output of the neural network model, training it;
b. the recovery phase comprises the following steps:
s4, collecting the common bright field in-focus image pi of the sample to be detected by using a common bright field microscope0；
S5, enabling the bright field of the sample to be detected to be in a focal image pi0Inputting the trained neural network to obtain a differential interference image lining image I of the sample to be measured0。
2. The differential interference phase contrast imaging method based on deep learning of claim 1, characterized in that: the microscopic imaging system of step S1 is an optical microscope or the like with differential interference function, and the light source thereof may be a coherent light source or a partially coherent light source.
3. The differential interference phase contrast imaging method based on deep learning of claim 1, characterized in that: the training sample of step S1 can be any phase-type object that can be used for imaging, including biological tissue, biological cells, industrial elements, etc., and in principle, the number and type of samples are increased as much as possible.
4. The differential interference phase contrast imaging method based on deep learning of claim 1, characterized in that: the differential interference image lining image of the training sample of the step S1 is a three-dimensional relief image.
5. The differential interference phase contrast imaging method based on deep learning of claim 1, characterized in that: the neural network model in step S3 may be any end-to-end neural network model for image conversion, such as an Unet model based on a convolutional neural network, and the frame for constructing the neural network may be a pitoch, tenserflow, and the like, and the network only needs to be trained once, and then can realize a similar differential interference image lining image for the sample to be measured indefinitely.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110410841.XA CN113066170A (en) | 2021-04-15 | 2021-04-15 | Differential interference phase contrast imaging method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110410841.XA CN113066170A (en) | 2021-04-15 | 2021-04-15 | Differential interference phase contrast imaging method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113066170A true CN113066170A (en) | 2021-07-02 |
Family
ID=76566941
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110410841.XA Pending CN113066170A (en) | 2021-04-15 | 2021-04-15 | Differential interference phase contrast imaging method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113066170A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109581645A (en) * | 2018-11-22 | 2019-04-05 | 南京理工大学 | The micro imaging method of phase contrast and differential interference phase-contrast based on light intensity transmission equation |
| CN109685745A (en) * | 2019-01-02 | 2019-04-26 | 西北工业大学 | A kind of phase micro imaging method based on deep learning |
| CN111221118A (en) * | 2020-02-26 | 2020-06-02 | 南京理工大学 | Microscopic imaging method based on phase coding single lens |
-
2021
- 2021-04-15 CN CN202110410841.XA patent/CN113066170A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109581645A (en) * | 2018-11-22 | 2019-04-05 | 南京理工大学 | The micro imaging method of phase contrast and differential interference phase-contrast based on light intensity transmission equation |
| CN109685745A (en) * | 2019-01-02 | 2019-04-26 | 西北工业大学 | A kind of phase micro imaging method based on deep learning |
| CN111221118A (en) * | 2020-02-26 | 2020-06-02 | 南京理工大学 | Microscopic imaging method based on phase coding single lens |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Pan et al. | High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine | |
| CN106896489B (en) | Frequency domain based on wavelength multiplexing pieces microscopic system and its method together | |
| US9817224B2 (en) | Methods and systems for Fourier ptychographic imaging | |
| CN106842529B (en) | Quick three-dimensional micro imaging system | |
| CN101896856B (en) | Depth of field extension for optical tomography | |
| CN110262026B (en) | Aperture scanning Fourier ptychographic imaging | |
| Pech-Pacheco et al. | Diatom autofocusing in brightfield microscopy: a comparative study | |
| JP5090188B2 (en) | Microscope equipment | |
| CN109685745B (en) | Phase microscopic imaging method based on deep learning | |
| US20120081536A1 (en) | Microscope System, Microscopy Method and Storage Medium | |
| CN109459923B (en) | Holographic reconstruction algorithm based on deep learning | |
| US20080266655A1 (en) | Microscopy Arrangements and Approaches | |
| CN107966801A (en) | A kind of high speed Fourier lamination imaging device and reconstructing method based on ring illumination | |
| WO2019135778A1 (en) | Single-frame autofocusing using multi-led illumination | |
| Arpa et al. | Single lens off-chip cellphone microscopy | |
| US10775602B2 (en) | Microscopy method and apparatus for optical tracking of emitter objects | |
| CN107145052B (en) | Holographic microscopic imaging method based on digital interpolative and phase iteration | |
| JP2007510963A (en) | Digital imaging assembly and method thereof | |
| CN107110764A (en) | The system and associated method for controlling to analyze transparent sample using position | |
| JP2020514809A (en) | Four-dimensional multi-plane broadband imaging system based on non-reentrant second-order distorted (NRQD) grating and grating prism | |
| CN116183568B (en) | High-fidelity reconstruction method and device for three-dimensional structured light illumination super-resolution microscopic imaging | |
| CN112368625B (en) | Multi-point illuminator, confocal filter, confocal microscope, and method of operating same | |
| Plisson et al. | Technician or researcher? A visual answer | |
| CN113066170A (en) | Differential interference phase contrast imaging method based on deep learning | |
| CN115220211B (en) | Microscopic imaging system and method based on deep learning and light field imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210702 |
|
| WD01 | Invention patent application deemed withdrawn after publication |