CN109884018A - A kind of submicron order neural network based is without lens micro imaging method and system - Google Patents
A kind of submicron order neural network based is without lens micro imaging method and system Download PDFInfo
- Publication number
- CN109884018A CN109884018A CN201910220004.3A CN201910220004A CN109884018A CN 109884018 A CN109884018 A CN 109884018A CN 201910220004 A CN201910220004 A CN 201910220004A CN 109884018 A CN109884018 A CN 109884018A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- imaged
- dimensional
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The present invention discloses a kind of submicron order neural network based without lens micro imaging method and system, comprising: under the conditions of coherent illumination, by obtaining the corresponding hologram of interference fringe that object to be imaged generates without saturating microscope；Hologram based on object to be imaged carries out phase recovery, obtains the two dimensional amplitude image of the first resolution of object varying cross-section to be imaged and the two-dimensional phase image of first resolution；The two-dimensional phase image of the two dimensional amplitude image of first resolution and first resolution is input to trained generation confrontation network, the two dimensional amplitude image of second resolution and the two-dimensional phase image of second resolution are obtained, the second resolution is greater than first resolution.The present invention solves the problems, such as no lens microscope lack of resolution using neural network, can carry out resolution ratio reinforcement to arbitrary cell image, it allows to break through the limitation of Pixel size without lens imaging system, realizes submicron resolution.
Description
Technical field
The present invention relates to biomedical micro-imaging technique fields, more particularly, to a kind of Asia neural network based
Micron order is without lens micro imaging method and system.
Background technique
In biomedical detection process immediately, it is often necessary to analyze the morphology letter of the tiny samplers such as cell, microorganism
Fluoroscopic image information after breath and the biochemical reactions such as nucleic acid, antigen-antibody, and the acquisition of these information usually require to use it is aobvious
Micro- imaging device.Traditional optical microscopy is mainly made of light source, optical lens, photodetector three parts.Optical lens
Main function is that sample is carried out to optical amplifier and is focused it on photodetector to be imaged.But optical lens usually requires
It is used cooperatively with components such as optical tubes, aperture and focusing systems to obtain clearly image, considerably increases microscopical
Volume and complexity become microscope and are used for the big resistance that instant detection field must overcome.
Compared to conventional microscope, no lens microscope is as novel microscopic system, with its big visual field and simplicity
Characteristic obtains the favor of field of biomedicine.No lens imaging technology is by sample and charge coupled cell (charge-
Coupled device, CCD) or complementary metal conductor oxidate (Complementary Metal Oxide
Semiconductor, CMOS) photodetectors such as the chip skill that is in close contact, is imaged without optical element, directly to sample
Art.According to the difference of image-forming principle, no lens imaging technology is divided into two class of shadowgraph imaging and digital holographic imaging, these two types of without thoroughly
Mirror imaging technique respectively has advantage and disadvantage.Shade is the simplest without lens imaging system structure, it uses illumination of incoherent light, utilizes light
Along the optical principle of straightline propagation, the projection of sample is acquired by photodetector to obtain target information, gained image with it is original
Sample ratio is close to 1；Digital hologram uses coherent light illumination without lens imaging system, generates interference by the interventional procedures of light
Striped receives such interference fringe by photodetector to collect the space three-dimensional information of sample.
However, current imaging sensor can only receive intensity information and can not receive optical phase information, and in biology
In medical application, we usually need to observe the transparent or semitransparent sample such as cell, and the universal transmissivity of these samples is distributed phase
To uniform, strength information variation is smaller after light penetrates sample, therefore is only difficult the sample of acquisition high quality by the detection of light intensity
Image.Therefore, contact is not suitable for such detection without lens microscope yet.However, the change of sample refractive index affects light
Light path when across sample makes the phase information of light that significant change have occurred, therefore can be direct by the phase information of detection light
Accurately reflect properties of samples.The phase information of light is the key that reduction image high-resolution details.Digital hologram without lens at
The needs of the biomedical application of detection immediately as can meeting system, it is sent out using the light wave of reference light wave and object scatter
Raw coherent superposition generates interference fringe, these interference fringes will be become hologram by CMOS or CCD record, extensive by phase iteration
Double calculation method can restore former light field, and the amplitude by recovering and phase information obtain the physical aspect feature of sample.
Pixel ruler of the current these two types of resolution ratio without lens imaging system both limited by photodetector CMOS or CCD
It is very little.Theoretically smaller pixel dimension, the information that can be received are more small.Universal CMOS or CCD on the market at present
Pixel dimension in a μm magnitude, and the resolution ratio of ordinary optical microscope is normally close to diffraction limit, that is, hundred nano-scale.
In addition, the sensor surface that can collect the CCD or CMOS of color pattern has one layer due to the design reasons of photodetector
The size of Baeyer filter, each Baeyer unit is 3 × 3, and removing this Baeyer filter can be improved its resolution ratio, but opposite,
Optical sensor is lost the ability of acquisition color image.
Summary of the invention
In view of the drawbacks of the prior art, it is an object of the invention to solve the image point of existing no lens imaging technology acquisition
The low technical problem of resolution.
To achieve the above object, on the one hand, it is micro- without lens that the present invention provides a kind of submicron order neural network based
Imaging method includes the following steps:
Under the conditions of coherent illumination, the corresponding hologram of interference fringe that object to be imaged generates is obtained；
Hologram based on the object to be imaged carries out phase recovery, obtains the first of object varying cross-section to be imaged
The two dimensional amplitude image of resolution ratio and the two-dimensional phase image of first resolution；
The two-dimensional phase image of the two dimensional amplitude image of first resolution and first resolution is input to trained life
At confrontation network (generative adversarial networks, GAN), the two dimensional amplitude image of second resolution is obtained
With the two-dimensional phase image of second resolution, the second resolution is greater than first resolution.
Specifically, trained generation confrontation network G AN passes through to low-resolution image gathered in advance and corresponding height
Image in different resolution mapping training obtains, and closes for learning the mapping between the corresponding high-definition picture of low-resolution image
System, and the low-resolution image received is restored based on mapping relations, obtain corresponding high-definition picture.
Specifically, first resolution is low resolution, and second resolution is high-resolution, and general high-resolution is sub-micron
Grade.
Optionally, this further includes following steps without lens micro imaging method:
Multi-angle coherent illumination is carried out to the object to be imaged, obtains different angle object varying cross-section to be imaged
The two dimensional amplitude image of first resolution and the two-dimensional phase image of first resolution；
To the two dimensional amplitude image and first resolution of the first resolution of different angle object varying cross-section to be imaged
Two-dimensional phase image carry out biaxial compensation, obtain the spatial frequency information lost in multi-angle coherent illumination, recover at
As the 3-D image of object.
Optionally, training generates confrontation network as follows:
High-definition picture of the acquisition for the object of training in advance；
Fuzzy Processing is carried out to collected high-definition picture, obtains the corresponding low-resolution image for training；
High-definition picture low-resolution image corresponding with its for trained object forms training set；
It determining and generates the confrontation network architecture, it includes generator structure and discriminator structure that generations, which fights the network architecture,
The generator structure is used to generate the image detail in second resolution image, and the discriminator structure is for judging generator
The authenticity of the image detail of structural generation；
It is trained using the training set to the confrontation network architecture is generated, the corresponding height of study low-resolution image
Mapping relations between image in different resolution, obtain trained generation confrontation network, the training set be divided into training data and
Test data is generated the confrontation network architecture using training data and is trained study, assessed using test data, Zhi Daosuo
It states and generates the capable mapping established from low-resolution image to high-definition picture of the confrontation network architecture.
Optionally, Fuzzy Processing is carried out to collected high-definition picture, is realized especially by following formula:
Im=D (K*I)+N
Wherein, ImIt is the intensity distribution matrix of the high-definition picture of object, I indicates the strong of corresponding low-resolution image
Spend distribution matrix；K is the point spread function of optical system, is expressed as Gaussian convolution core；It * is the convolution between I and K；D is acted on
Convolution results indicate the discretization of camera sensor；N indicates additive white Gaussian noise.
On the other hand, the present invention provides a kind of submicron order neural network based without lens micro imaging system, comprising:
Lighting unit, for providing coherent illumination light；
Detection imaging unit, it is corresponding for obtaining the interference fringe that object to be imaged generates based on the coherent illumination light
Hologram；
Phase recovery unit carries out phase recovery for the hologram based on the object to be imaged, obtains object to be imaged
The two dimensional amplitude image of the first resolution of body varying cross-section and the two-dimensional phase image of first resolution；
High-resolution recovery unit, for by the two dimensional amplitude image of first resolution and the two-dimensional phase of first resolution
Image is input to trained generation confrontation network, obtains the two dimensional amplitude image of second resolution and the two dimension of second resolution
Phase image, the second resolution are greater than first resolution.
Optionally, this further includes mobile unit and 3-D image recovery unit without lens micro imaging system；
The mobile unit is used for portable lighting unit, to carry out multi-angle coherent illumination to the object to be imaged, obtains
To the two dimensional amplitude image of the first resolution of different angle object varying cross-section to be imaged and the two-dimensional phase of first resolution
Bit image；
The 3-D image recovery unit, for the first resolution to different angle object varying cross-section to be imaged
The two-dimensional phase image of two dimensional amplitude image and first resolution carries out biaxial compensation, loses in acquisition multi-angle coherent illumination
Spatial frequency information recovers the 3-D image of object to be imaged.
Optionally, this is without lens micro imaging system further include: generates confrontation network training unit；
The generation fights network training unit, for acquiring the high-definition picture of the object for training in advance；It is right
Collected high-definition picture carries out Fuzzy Processing, obtains the corresponding low-resolution image for training；For trained
The high-definition picture of object low-resolution image corresponding with its forms training set；It determines and generates the confrontation network architecture, it is described
Generating the confrontation network architecture includes generator structure and discriminator structure, and the generator structure is for generating second resolution figure
Image detail as in, the discriminator structure are used to differentiate the authenticity of the image detail of generator structural generation；Using institute
It states training set and is trained to the confrontation network architecture is generated, between the corresponding high-definition picture of study low-resolution image
Mapping relations, obtain trained generation confrontation network, the training set is divided into training data and test data, generates pair
The anti-network architecture is trained study using training data, is assessed using test data, until the generation fights network
Framework has the ability to establish the mapping from low-resolution image to high-definition picture.
Optionally, the generation fights network training unit, carries out Fuzzy Processing, tool to collected high-definition picture
Body is realized by following formula:
Im=D (K*I)+N
Wherein, ImIt is the intensity distribution matrix of the high-definition picture of object, I indicates the strong of corresponding low-resolution image
Spend distribution matrix；K is the point spread function of optical system, is expressed as Gaussian convolution core；It * is the convolution between I and K；D is acted on
Convolution results indicate the discretization of camera sensor；N indicates additive white Gaussian noise.
In general, through the invention it is contemplated above technical scheme is compared with the prior art, have below beneficial to effect
Fruit:
The present invention provides submicron order neural network based without lens micro imaging method and system, does not need any
Lens or high-power laser cell, overall architecture is simply portable, and high-resolution and big visual field can be achieved at the same time and can rebuild
The phase image of cell out.
Neural network is solved the problems, such as no lens microscope lack of resolution by the present invention, can be to arbitrary cell
Image carries out resolution ratio reinforcement, it allows to break through the limitation of Pixel size without lens imaging system, realizes submicron resolution.
The three-dimensional reconstruction to imaging cells, and the axial direction rebuild may be implemented in multi-angle illumination method proposed by the present invention
In micron dimension, this will allow us to distinguish the cell to overlap each other, can carry out to more highdensity cell sample resolution ratio
Imaging.
Submicron order neural network based proposed by the present invention is above-mentioned in solution without lens micro imaging method and system
Further applying in field of biomedicine is also achieved while problem.
Detailed description of the invention
Fig. 1 is submicron order neural network based provided by the invention without lens micro imaging method flow chart；
Fig. 2 is that generation provided by the invention fights neural network configuration diagram；
Fig. 3 is the imaging effect of no lens imaging system provided by the invention；
Fig. 4 is the flow chart of neural network training method provided by the invention；
Fig. 5 a be the slave hologram that provides of the specific embodiment of the invention recover without the original of Processing with Neural Network
Phase image；
Fig. 5 b is that the use that the specific embodiment of the invention provides mentions the final effect picture obtained after high-resolution method.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as in addition, technical characteristic involved in the various embodiments of the present invention described below
Not constituting a conflict with each other can be combined with each other.
The present invention is directed to disadvantages mentioned above or deficiency, proposes solution and provides corresponding system building method: being directed to
Not small enough the disadvantage of pixel dimension, the present invention bypass the limitation of Pixel size by way of software, utilize neural network, tool
Body, a kind of generation antagonism network G AN can be developed and used, to restore high-resolution from single low resolution measurement
Without lenticular image, to realize the resolution ratio of submicron order in entire sensor plane.Experimentally, our lensless system
Well-trained generation confrontation network G AN can be then inputted, with Real time Efficiency with fast Acquisition culture cell without lenticular image
Superresolution restoration is carried out, this imaging method based on deep learning can restore big visual field at a terrific speed (in 1 second)
Image, visual field area improves about 1.7 μm of resolution ratio substantially at 95 square millimeters, while need not change existing microscopical
Setting.
For the Baeyer filter of sensor surface, present applicant proposes the methods for removing it, it is possible thereby to obtain more high score
The grayscale image of the original light field of resolution.The ability of the acquisition color image lost because removing Baeyer filter, the application can be with
Restored by multi-wavelength illumination.
In addition, the application propose a kind of 3-D image that can construct imaging object without lens microscope system,
Pass through multi-angle illumination and carry out biaxial compensation to obtain the spatial frequency information lost in multi-angle illumination, which will allow
The 3-dimensional image for recovering the original is calculated by back projection.
Fig. 1 is submicron order neural network based provided by the invention without lens micro imaging method flow chart, including
Following steps:
S100, it is corresponding by obtaining the interference fringe that object to be imaged generates without saturating microscope under the conditions of coherent illumination
Hologram；
S200, the hologram based on the object to be imaged carry out phase recovery, obtain object varying cross-section to be imaged
First resolution two dimensional amplitude image and first resolution two-dimensional phase image；
The two-dimensional phase image of the two dimensional amplitude image of first resolution and first resolution is input to and trains by S300
GAN, obtain the two dimensional amplitude image of second resolution and the two-dimensional phase image of second resolution, the second resolution is big
In first resolution, the trained generation confrontation network G AN passes through to low-resolution image gathered in advance and corresponding
High-definition picture mapping training obtains, for learning the mapping between the corresponding high-definition picture of low-resolution image
Relationship, and the low-resolution image received is restored based on mapping relations, obtain corresponding high-definition picture.
The specific refinement of each step can be found in being discussed in detail for following embodiments.
It is an object of the invention to develop a kind of no lens microscope, resolution ratio, which can reach submicron order or even approach, spreads out
Emitter-base bandgap grading limit.
The present invention provides a kind of being concerned with microscopic system without lens for low cost, from top to bottom includes: lighting unit, filtering
Unit, detection imaging unit:
Common or narrowband LED light source can be used directly in lighting unit.Lighting unit further includes filter unit, filter
Wave cell illumination light is changed into coherent illumination light, including filter in spatial domain and frequency domain filtering, uses the small of 50um-100um magnitude
Bandwidth ± 10 or smaller optical filter progress bandpass filtering are placed below aperture as spatial filter in hole.
Note that aperture and optical filter here should be close to, and need suitably to adjust at a distance from lighting unit, to avoid because of the two
Between spacing generate additional diffraction light and cause to collect additional interference figure on detection imaging unit.
Detection imaging unit, common CMOS or CCD on the market, pixel dimension is in 2.2um or following.
Phase recovery unit carries out phase recovery for the hologram based on the object to be imaged, obtains object to be imaged
The two dimensional amplitude image of the first resolution of body varying cross-section and the two-dimensional phase image of first resolution.
High-resolution recovery unit, for by the two dimensional amplitude image of first resolution and the two-dimensional phase of first resolution
Image is input to trained generation confrontation network G AN, obtains the two dimensional amplitude image and second resolution of second resolution
Two-dimensional phase image, second resolution are greater than first resolution；The trained generation confrontation network G AN by adopting in advance
The low-resolution image of collection and the mapping training of corresponding high-definition picture obtain, corresponding for learning low-resolution image
High-definition picture between mapping relations, and the low-resolution image that receives is restored based on mapping relations, is obtained pair
The high-definition picture answered.
Mobile unit is used for portable lighting unit, to carry out multi-angle coherent illumination to the object to be imaged, obtains not
With the two dimensional amplitude image of the first resolution of angle object varying cross-section to be imaged and the two-dimensional phase bitmap of first resolution
Picture.
3-D image recovery unit, the two dimension for the first resolution to different angle object varying cross-section to be imaged
The two-dimensional phase image of magnitude image and first resolution carries out biaxial compensation, obtains the space lost in multi-angle coherent illumination
Frequency information recovers the 3-D image of object to be imaged.
GAN training unit, acquisition is used for the high-definition picture of the object of training in advance；To collected high resolution graphics
As carrying out Fuzzy Processing, the corresponding low-resolution image for training is obtained；High-definition picture for trained object
Low-resolution image corresponding with its forms training set；Determine that GAN framework, the GAN framework include generator structure and identification
Device structure, the generator structure are used to generate the image detail in second resolution image, and the discriminator structure is for sentencing
The authenticity of the image detail of other generator structural generation；GAN framework is trained using the training set, learns low resolution
Mapping relations between the corresponding high-definition picture of rate image, obtain trained GAN network, and the training set is divided
For training data and test data, GAN framework is trained study using training data, is assessed using test data, directly
It has the ability to establish the mapping from low-resolution image to high-definition picture to the GAN framework.
Culture dish is directly placed on CMOS or CCD, lighting source is opened, acquisition cell generates under coherent illumination
Interference fringe, this kind of interference image are known as in-line hologram.Then, dual image artifact is eliminated using iterative algorithm, and carries out phase
Bit recovery processing, so that it may the light intensity and phase information by cell are reconstructed, to obtain amplitude and phase image.
The present invention provides a kind of optimization method of resolution ratio that the image acquired is improved by deep learning network, including
Following steps:
Cell is sampled using high-resolution optical microscopy in advance, obtains a large amount of high-resolution cytological map
Picture.Downward sampling processing is carried out to collected high-definition picture, high-resolution image is obscured into the figure of low resolution
Then picture is trained, thus using this two groups of corresponding image collections as the training set for generating confrontation network (GAN network)
It establishes it and is mapped to the ability that full resolution pricture is rebuild from low resolution image.Dependent on such neural network, we can be from
Arbitrary low-resolution image recovers high-resolution image.
This method also needs following steps:
1, suitable cell original image is acquired in advance, and Fuzzy Processing is carried out for collected high-definition picture:
We are using image drop sampling model realization by high-resolution (High Resolution, HR) image to low
The transformation of resolution ratio (Low Resolution, LR) image, and the image of " LR-HR " is utilized to map to train GAN network, make it
With the ability from low explanation image restoring original high-resolution image.It should be noted that in order to ensure the model energy after training
It is enough accurately to restore the collected original image of no lens system, the image drop sampling model that we design allow for generating and
Without the very close low-resolution image of lens image.
The down-sampled model of traditional optical microscope system are as follows:
Im=D (K*I)+N
Wherein ImIt is the continuous actual strength distribution of sample to be imaged；K is the point spread function of optical system, is expressed as height
This convolution kernel；It * is the convolution between I and K；D acts on convolution results, indicates the discretization of camera sensor；N indicates that additivity is high
This white noise.In these parameters, white Gaussian noise is mainly as caused by the statistics thermal noise of CCD/CMOS sensor；ImIt is me
By optical microscopy obtain image, it is actually the down-sampled of I.This formula needs to optimize there are two parameter: convolution step
The size of middle Gaussian kernel and the variance of noise profile.
High power light object lens (X10) acquired image is approximately the actual consecutive image of sample by we, is then utilized
Different Sigma values carry out that gaussian filtering is down-sampled to image, and with the image comparison after down-sampled without camera lens original image come true
Fixed optimal Sigma value, i.e., the size of optimal gaussian mask.Later, we add different variances on the image after down-sampled
The white Gaussian noise of value, comparison determine optimal noise variance without camera lens original image.After the two key parameters have been determined, drop
Sampling model just can generate the low resolution image close enough with no camera lens original image.
2, the framework for generating confrontation network G AN is built using computer:
Referring to the framework of attached drawing 2:GAN network.
Part A is the structure of generator, and conv and resnet are the abbreviations of convolutional layer and rest network block.The ginseng of convolutional layer
Number is provided with " k-s-n " format, and wherein k is kernel size, and s is stride, and n is the quantity (i.e. the output channel of layer) of characteristic pattern.
The depth of each convolutional layer substantially indicates the quantity of its characteristic pattern, and lateral dimension indicates the size of its input.It share 16 it is residual
Block.
B is the structure of discriminator.Each convolutional layer in discriminator is convolutional layer, batch normalizing operation and ReLU activation letter
Several combinations.After arbitrary programming language realizes the framework of the neural network, the training set that step 1 is obtained is divided into two parts,
A part is used as training data, and another part is trained neural network and assesses as test data, until it has energy
Until power establishes the mapping from low resolution to high-definition picture.
3, it will need to propose high-resolution image input neural network, it will the image being more clear.Note that this
The source of image: initial cell must belong to a kind of with reference to certain in cell used in training process.
The present invention provides a kind of multi-angle illumination imaging method that can construct object 3-dimensional image, and this method is needed to this
The lighting unit without lens microscope system that application proposes is transformed.In particular it is required that guaranteeing source-sample away from constant
In the case of change lighting source angle, it is however generally that need within the scope of ± 50 ° portable lighting light source, the cell of different height
Projection will have different degrees of offset in CMOS plane, this series of images by include cell depth information.We
After being eliminated dual image using iterative algorithm to these original holograms and carried out phase recovery processing, they can be with reconstruction sample
The amplitude and phase image of the varying cross-section of volume.This solution requires also, as this numerical reconstruction process
As a result, we can distinguish the overlapping of cell without lens hologram, to increase the density for the cell that we can be used.
The purpose of the present invention is to provide a kind of low in cost, device it is easy it is portable, high-resolution and big can be achieved at the same time
Visual field, be able to achieve three-dimensional information real-time display micro imaging system, and propose on the basis of this system it is a variety of can be same
When use for improving the method for systemic resolution, make its performance close to optical microscopy, but avoid any because lens are led
The aberration of cause simultaneously retains every advantage without lens microscope.
No lens micro imaging method schematic diagram provided by the invention, comprising the following steps:
Shielding environment light, and power on for LED light source.
Filter unit is placed in front of LED unit, and adjusts LED light source as close as possible to the aperture of spatial filter；
Sample is placed on to the top of image processing unit；
Image processing unit is connected to computer, acquires image.
By above-mentioned steps, we obtain original image, eliminate dual image by iterative algorithm and carry out phase recovery processing
Can be restored from striped afterwards and obtain the cell image of original shooting, and can simultaneously using the application propose three kinds of schemes into
Row resolution ratio is strengthened.
The following are a specific embodiments provided by the invention:
Embodiment 1
The embodiment of the present invention provides one kind for proposing high-resolution Processing with Neural Network method, shows for reinforcing without lens
It is micro- that obtained final image is imaged.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and do not have to
It is of the invention in limiting.
The embodiment of the present invention has used imaging sensor (model: Aptina MT9P031), its spacing having a size of 2.2 μm,
Effective area is 5.7mm × 4.28mm, thin by what is cultivated on the sensor surface for the light sensor portion of no lens imaging
The near field optical signal of born of the same parents digitizes.It is first turned on LED light source, is filtered illumination, and sample is placed in sensor plane,
Digital picture obtained is transferred on computer by starting CMOS driving by USB data line.Iteration is used on computers
Algorithm eliminates the dual image artifact that interference generates, and after carrying out phase recovery processing according to Rayleigh-Suo Mofei diffraction formula, can be with
Obtain the cell pattern recovered from diffraction fringe.We are using the cell pattern that recovers as the input of neural network, to obtain
The output pattern for the resolution that secures satisfactory grades.Fig. 3 is the imaging effect of no lens imaging system provided by the invention, as shown in figure 3, figure is
Without the original hologram that relevant recovery algorithms are handled, the phase of available original objects after being restored with coherent algorithm
Figure and intensity map.
For this purpose, we, firstly the need of well-trained GAN network is created, train process such as attached drawing 4, it is specific as follows:
Several high-resolution (HR) image (attached drawing 4a, step of cell is obtained using traditional high magnification microscope first
1).By the image degradation model of transmission function of the reproduction without lens imaging process, low resolution (LR) image of simulation is generated,
Its image detail is similar to no lens microscope collected true experimental image (attached drawing 4a, step 2).It is with full resolution pricture
Target is modeled as inputting with low resolution image LR, enables and generates confrontation network (GAN) iterative learning from low-resolution image to its phase
The mapping for the high-resolution target answered, until the image point that the close enough optical microscopy of the output image quality of test can export
Resolution (attached drawing 4a, step 3).Later, this well-trained GAN can carry out the image that original no lens microscope obtains
Super-resolution is inferred.Experimentally, equipment can be with the (attached without lenticular image of fast Acquisition culture cell on our portable tablet
Fig. 4 b, step 1) are then input to well-trained GAN network, carry out superresolution restoration (attached drawing 4b, step with Real time Efficiency
It is rapid 2).The GAN network can quickly export super-resolution without lens image within the time less than one second.Therefore, this support
The contact image of GAN maintains the big visual field from non-unity magnification measurement, while having restored initially discrete by sensor pixel
Change destroyed high-resolution details.
Specifically, Fig. 5 a is the low resolution image without Processing with Neural Network, and Fig. 5 b is after Processing with Neural Network
Full resolution pricture, in culture dish Hela cell select three regions observed stage by stage, respectively after starting culture
3h, 6h, 9h, 12h, 15h, 18h, 21h after sampled and restored in real time, the image observed.It can be seen that and figure
The global pattern that 5a is provided compares, and is significantly improved by Fig. 5 b it is found that restoring resulting three block of cells resolution ratio, it is specific and
Speech is in sub-micrometer scale.
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to
The limitation present invention, any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should all include
Within protection scope of the present invention.
Claims (8)
1. a kind of submicron order neural network based is without lens micro imaging method, which comprises the steps of:
Under the conditions of coherent illumination, the corresponding hologram of interference fringe that object to be imaged generates is obtained；
Hologram based on the object to be imaged carries out phase recovery, and obtain object varying cross-section to be imaged first is differentiated
The two dimensional amplitude image of rate and the two-dimensional phase image of first resolution；
The two-dimensional phase image of the two dimensional amplitude image of first resolution and first resolution is input to trained generation pair
Anti- network obtains the two dimensional amplitude image of second resolution and the two-dimensional phase image of second resolution, the second resolution
Greater than first resolution.
2. no lens micro imaging method according to claim 1, which is characterized in that further include following steps:
Multi-angle coherent illumination is carried out to the object to be imaged, obtains the first of different angle object varying cross-section to be imaged
The two dimensional amplitude image of resolution ratio and the two-dimensional phase image of first resolution；
Two of two dimensional amplitude image and first resolution to the first resolution of different angle object varying cross-section to be imaged
It ties up phase image and carries out biaxial compensation, obtain the spatial frequency information lost in multi-angle coherent illumination, recover object to be imaged
The 3-D image of body.
3. no lens micro imaging method according to claim 1 or 2, which is characterized in that training life as follows
At confrontation network:
High-definition picture of the acquisition for the object of training in advance；
Fuzzy Processing is carried out to collected high-definition picture, obtains the corresponding low-resolution image for training；For
The high-definition picture of trained object low-resolution image corresponding with its forms training set；
It determines and generates the confrontation network architecture, it includes generator structure and discriminator structure that generations, which fights the network architecture, described
Generator structure is used to generate the image detail in second resolution image, and the discriminator structure is for judging generator structure
The authenticity of the image detail of generation；
It is trained using the training set to the confrontation network architecture is generated, the corresponding high-resolution of study low-resolution image
Mapping relations between rate image, obtain trained generation confrontation network, and the training set is divided into training data and test
Data are generated the confrontation network architecture using training data and are trained study, assessed using test data, until the life
It has the ability to establish the mapping from low-resolution image to high-definition picture at the confrontation network architecture.
4. no lens micro imaging method according to claim 3, which is characterized in that collected high-definition picture
Fuzzy Processing is carried out, is realized especially by following formula:
Im=D (K*I)+N
Wherein, ImIt is the intensity distribution matrix of the high-definition picture of object, I indicates the intensity point of corresponding low-resolution image
Cloth matrix；K is the point spread function of optical system, is expressed as Gaussian convolution core；It * is the convolution between I and K；D acts on convolution
As a result, indicating the discretization of camera sensor；N indicates additive white Gaussian noise.
5. a kind of submicron order neural network based is without lens micro imaging system characterized by comprising
Lighting unit, for providing coherent illumination light；
Detection imaging unit, for obtaining the corresponding holography of interference fringe that object to be imaged generates based on the coherent illumination light
Figure；
Phase recovery unit carries out phase recovery for the hologram based on the object to be imaged, obtains object to be imaged not
With the two dimensional amplitude image of the first resolution of cross section and the two-dimensional phase image of first resolution；
High-resolution recovery unit, for by the two-dimensional phase image of the two dimensional amplitude image of first resolution and first resolution
It is input to trained generation confrontation network, obtains the two dimensional amplitude image of second resolution and the two-dimensional phase of second resolution
Image, the second resolution are greater than first resolution.
6. no lens micro imaging system according to claim 5, which is characterized in that further include mobile unit and three-dimensional figure
As recovery unit；
The mobile unit is used for portable lighting unit, to carry out multi-angle coherent illumination to the object to be imaged, obtains not
With the two dimensional amplitude image of the first resolution of angle object varying cross-section to be imaged and the two-dimensional phase bitmap of first resolution
Picture；
The 3-D image recovery unit, the two dimension for the first resolution to different angle object varying cross-section to be imaged
The two-dimensional phase image of magnitude image and first resolution carries out biaxial compensation, obtains the space lost in multi-angle coherent illumination
Frequency information recovers the 3-D image of object to be imaged.
7. no lens micro imaging system according to claim 5 or 6, which is characterized in that further include: generate confrontation network
Training unit；
The generation fights network training unit, for acquiring the high-definition picture of the object for training in advance；To acquisition
The high-definition picture arrived carries out Fuzzy Processing, obtains the corresponding low-resolution image for training；For trained object
Corresponding with its low-resolution image of high-definition picture form training set；It determines and generates the confrontation network architecture, the generation
The confrontation network architecture includes generator structure and discriminator structure, and the generator structure is for generating in second resolution image
Image detail, the discriminator structure be used for differentiate generator structural generation image detail authenticity；Using the instruction
Practice collection and is trained to the confrontation network architecture is generated, reflecting between the corresponding high-definition picture of study low-resolution image
Relationship is penetrated, trained generation confrontation network is obtained, the training set is divided into training data and test data, generates confrontation net
Network framework is trained study using training data, is assessed using test data, until the generation fights the network architecture
It has the ability to establish the mapping from low-resolution image to high-definition picture.
8. no lens micro imaging system according to claim 7, which is characterized in that the generation fights network training list
Member carries out Fuzzy Processing to collected high-definition picture, realizes especially by following formula:
Im=D (K*I)+N
Wherein, ImIt is the intensity distribution matrix of the high-definition picture of object, I indicates the intensity point of corresponding low-resolution image
Cloth matrix；K is the point spread function of optical system, is expressed as Gaussian convolution core；It * is the convolution between I and K；D acts on convolution
As a result, indicating the discretization of camera sensor；N indicates additive white Gaussian noise.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910220004.3A CN109884018B (en) | 2019-03-22 | 2019-03-22 | Submicron lens-free microscopic imaging method and system based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910220004.3A CN109884018B (en) | 2019-03-22 | 2019-03-22 | Submicron lens-free microscopic imaging method and system based on neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109884018A true CN109884018A (en) | 2019-06-14 |
| CN109884018B CN109884018B (en) | 2020-09-18 |
Family
ID=66933599
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910220004.3A Active CN109884018B (en) | 2019-03-22 | 2019-03-22 | Submicron lens-free microscopic imaging method and system based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109884018B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110308547A (en) * | 2019-08-12 | 2019-10-08 | 青岛联合创智科技有限公司 | A kind of dense sample based on deep learning is without lens microscopic imaging device and method |
| CN111221118A (en) * | 2020-02-26 | 2020-06-02 | 南京理工大学 | Microscopic imaging method based on phase coding single lens |
| CN111652815A (en) * | 2020-05-26 | 2020-09-11 | 浙江大学 | Mask camera image restoration method based on deep learning |
| CN112130306A (en) * | 2020-09-17 | 2020-12-25 | 浙江大学山东工业技术研究院 | CMOS holographic microscopic imaging device and method applied to cell segmentation |
| CN112200726A (en) * | 2020-10-29 | 2021-01-08 | 陈根生 | Urinary sediment visible component detection method and system based on lens-free microscopic imaging |
| CN112461360A (en) * | 2020-10-26 | 2021-03-09 | 北京理工大学 | High-resolution single photon imaging method and system combined with physical noise model |
| CN112798504A (en) * | 2020-12-07 | 2021-05-14 | 西安电子科技大学 | Large-field-of-view high-throughput flow cytometry analysis system and analysis method |
| WO2021203291A1 (en) * | 2020-04-08 | 2021-10-14 | 深圳华大生命科学研究院 | Lens-free microscopic imaging system and method, and biochemical substance detection system and method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107101943A (en) * | 2017-05-18 | 2017-08-29 | 大连海事大学 | A kind of optofluidic is without lens holographic imaging activity of microalgae detection means and method |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| WO2018023039A1 (en) * | 2016-07-29 | 2018-02-01 | William Marsh Rice University | Lensless imaging device for microscopy and fingerprint biometric |
| CN108051930A (en) * | 2017-12-29 | 2018-05-18 | 南京理工大学 | Big visual field super-resolution dynamic phasing is without lens microscopic imaging device and reconstructing method |
| US20180247106A1 (en) * | 2017-02-22 | 2018-08-30 | University Of Connecticut | Systems and Methods for Cell Identification Using Lens-Less Imaging |
| CN108508588A (en) * | 2018-04-23 | 2018-09-07 | 南京大学 | A kind of multiple constraint information without lens holographic microphotography phase recovery method and its device |
-
2019
- 2019-03-22 CN CN201910220004.3A patent/CN109884018B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018023039A1 (en) * | 2016-07-29 | 2018-02-01 | William Marsh Rice University | Lensless imaging device for microscopy and fingerprint biometric |
| US20180247106A1 (en) * | 2017-02-22 | 2018-08-30 | University Of Connecticut | Systems and Methods for Cell Identification Using Lens-Less Imaging |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| CN107101943A (en) * | 2017-05-18 | 2017-08-29 | 大连海事大学 | A kind of optofluidic is without lens holographic imaging activity of microalgae detection means and method |
| CN108051930A (en) * | 2017-12-29 | 2018-05-18 | 南京理工大学 | Big visual field super-resolution dynamic phasing is without lens microscopic imaging device and reconstructing method |
| CN108508588A (en) * | 2018-04-23 | 2018-09-07 | 南京大学 | A kind of multiple constraint information without lens holographic microphotography phase recovery method and its device |
Non-Patent Citations (4)
| Title |
|---|
| HAO ZHANG ET AL.: "High -throughput, high-resolution deep learning microscopy based on registrationfree generative adversarial network", 《BIOMEDICAL OPTICS EXPRESS》 * |
| WEI LUO ET AL.: "Synthetic aperture-based on-chip microscopy", 《LIGHT：SCIENCE& APPLICATIONS》 * |
| XIONGCHAO CHEN ET AL.: "Generative adversarial network(GAN)enabled on-chip contact microscopy", 《BIORXIV A LICENSE》 * |
| 李聪慧等: "无透镜显微成像技术在即时检测中的应用进展", 《中国激光》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110308547A (en) * | 2019-08-12 | 2019-10-08 | 青岛联合创智科技有限公司 | A kind of dense sample based on deep learning is without lens microscopic imaging device and method |
| CN110308547B (en) * | 2019-08-12 | 2021-09-07 | 青岛联合创智科技有限公司 | Dense sample lens-free microscopic imaging device and method based on deep learning |
| CN111221118A (en) * | 2020-02-26 | 2020-06-02 | 南京理工大学 | Microscopic imaging method based on phase coding single lens |
| WO2021203291A1 (en) * | 2020-04-08 | 2021-10-14 | 深圳华大生命科学研究院 | Lens-free microscopic imaging system and method, and biochemical substance detection system and method |
| CN115004076A (en) * | 2020-04-08 | 2022-09-02 | 深圳华大生命科学研究院 | Lens-free microscopic imaging system and method and biochemical substance detection system and method |
| CN111652815A (en) * | 2020-05-26 | 2020-09-11 | 浙江大学 | Mask camera image restoration method based on deep learning |
| CN111652815B (en) * | 2020-05-26 | 2023-05-05 | 浙江大学 | Mask plate camera image restoration method based on deep learning |
| CN112130306A (en) * | 2020-09-17 | 2020-12-25 | 浙江大学山东工业技术研究院 | CMOS holographic microscopic imaging device and method applied to cell segmentation |
| CN112461360A (en) * | 2020-10-26 | 2021-03-09 | 北京理工大学 | High-resolution single photon imaging method and system combined with physical noise model |
| CN112461360B (en) * | 2020-10-26 | 2021-10-12 | 北京理工大学 | High-resolution single photon imaging method and system combined with physical noise model |
| CN112200726A (en) * | 2020-10-29 | 2021-01-08 | 陈根生 | Urinary sediment visible component detection method and system based on lens-free microscopic imaging |
| CN112798504A (en) * | 2020-12-07 | 2021-05-14 | 西安电子科技大学 | Large-field-of-view high-throughput flow cytometry analysis system and analysis method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109884018B (en) | 2020-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109884018A (en) | A kind of submicron order neural network based is without lens micro imaging method and system | |
| de Haan et al. | Deep-learning-based image reconstruction and enhancement in optical microscopy | |
| Zeng et al. | Deep learning for digital holography: a review | |
| Pan et al. | High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine | |
| CN104285175B (en) | The method and apparatus that individual particle positioning is carried out using wavelet analysis | |
| Ren et al. | Fringe pattern improvement and super-resolution using deep learning in digital holography | |
| CN102473286B (en) | Method and system for super-resolution signal reconstruction | |
| Li et al. | Deep DIH: single-shot digital in-line holography reconstruction by deep learning | |
| JP2007510963A (en) | Digital imaging assembly and method thereof | |
| CN110378473A (en) | Method and device is chromatographed based on deep learning and the phase of random pattern | |
| CN115144371A (en) | Lensless Fourier laminated diffraction tomography microscopic imaging method based on wavelength scanning | |
| Lu et al. | Virtual-scanning light-field microscopy for robust snapshot high-resolution volumetric imaging | |
| Li et al. | Fast confocal microscopy imaging based on deep learning | |
| Astratov et al. | Roadmap on Label‐Free Super‐Resolution Imaging | |
| Guo et al. | Lensfree auto-focusing imaging using nuclear norm of gradient | |
| Wijesinghe et al. | Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams | |
| CN112200726B (en) | Urinary sediment visible component detection method and system based on lensless microscopic imaging | |
| Li et al. | Deep adversarial network for super stimulated emission depletion imaging | |
| CN110989155B (en) | Lens-free microscopic imaging device based on optical filter array and reconstruction method | |
| CN115220211B (en) | Microscopic imaging system and method based on deep learning and light field imaging | |
| Chen et al. | Superresolution microscopy imaging based on full-wave modeling and image reconstruction | |
| Wu et al. | Three-dimensional identification of microorganisms using a digital holographic microscope | |
| Gil et al. | Segmenting quantitative phase images of neurons using a deep learning model trained on images generated from a neuronal growth model | |
| Liu | Optical phase space measurements and applications to 3D imaging and light scattering | |
| Haputhanthri et al. | Towards ultrafast quantitative phase imaging via differentiable microscopy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |