CN109724950A - Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second - Google Patents
Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second Download PDFInfo
- Publication number
- CN109724950A CN109724950A CN201711037932.3A CN201711037932A CN109724950A CN 109724950 A CN109724950 A CN 109724950A CN 201711037932 A CN201711037932 A CN 201711037932A CN 109724950 A CN109724950 A CN 109724950A
- Authority
- CN
- China
- Prior art keywords
- image
- super
- resolution
- module
- dynamic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second.The present invention relates to a kind of technique for taking of dynamic super-resolution fluorescence micro-image, comprising: (1) a kind of fluorescent microscopic imaging device can be carried out super-resolution imaging and low resolution imaging, and the field of view of two kinds of imaging modes observation, object plane are identical；The device can handle the sampling frame per second of low resolution image and feedback control super resolution image in real time；(2) a kind of dynamic super-resolution fluorescence micro image shooting method of device is corresponded to, it include: to acquire low resolution image with fixed high-speed sampling frame per second to be handled in real time, image dynamic change rate is analyzed by the way of calculating light stream, image dynamic change rate is converted to the acquisition time interval of super resolution image, by the sampling frame per second of feedback loop real-time control super resolution image, it is allowed to adapt to sample dynamic change rate.The present invention extends the shooting time to sample for realizing the super-resolution shooting to the quick dynamic changing process of sample as far as possible.
Description
Technical field
The present invention relates to super-resolution fluorescence technical field of imaging, more particularly to a kind of dynamic super-resolution fluorescence micro-image
Technique for taking.
Background technique
Traditional fluorescence microscope (such as Both wide field illumination, scanning copolymerization coke, total internal reflection illumination fluorescence microscope) is limited to
Abbe diffraction limit, spatial resolution cannot be better than 200nm, can not differentiate more fine structure.In recent ten years
A variety of fluorescence imaging methods for breaking through diffraction limit, referred to as super-resolution imaging are developed, as stimulated emission depletion is imaged
(STED), reversible saturation/switch optical transition imaging (RESOLFT), photoactivation position micro-imaging (PALM/FPALM), random light
Learn reconstruct micro-imaging (STORM), structured illumination micro-imaging (SIM, saturated structures illumination SSIM, nonlinear organization illumination
NSIM) etc.；These super-resolution imaging technologies needed than conventional fluorescent imaging method higher intensity laser lighting (such as STED,
SSIM, NSIM etc.), or repeatedly ON/OFF fluorescent molecule (such as RESOLFT, PALM/FPALM, STORM) is needed, it is serious to add
Acute fluorescent bleach and phototoxicity to sample；Therefore, these imaging methods with space hyperresolution are applied to living thin
When the imaging of born of the same parents' dynamic process, the number of image frames that can be continuously shot is far fewer than conventional fluorescent imaging method.
However for slowly varying image, do not need very high sampling frame per second and shot, just it is observed that shooting
The complete procedure of period sample dynamic change；In fact, according to Nyquist sampling thheorem, it should make sampling frame per second when shooting
Match with the rate of change of image, can either observe the complete procedure of sample dynamic change during shooting in this way, and can
Extend shooting time as far as possible, is unlikely to be bleached fluorescent molecule too early because of the sampling frame per second of high speed.
Summary of the invention
In view of the above-mentioned problems, the present invention provides a kind of technique for taking of dynamic super-resolution fluorescence micro-image, purpose
It is to realize in the shooting process of super-resolution imaging the frame per second of automatic adjustment sampling in real time, adapts it to, match sample and move
The rate of state variation.
To achieve the goals above, this invention takes following technical schemes:
A kind of fluorescent microscopic imaging device, the fluorescent microscopic imaging device are functionally divided into following sections:
Sample stage is observed sample for carrying；
Super-resolution imaging module, the module include a series of optical elements, image device and light source, use super-resolution imaging
When module is imaged, image has high spatial resolution, can break through Abbe diffraction limit；
Low resolution imaging module, the module include a series of optical elements, image device and light source, use low resolution imaging
When module is imaged, image has low spatial resolution, cannot break through Abbe diffraction limit；
Control module is shot for controlling super-resolution imaging module and low resolution imaging module, including control two
Light source and optical element in module；
Image recording and image processing module, for recording image and carrying out image procossing, analysis image dynamic change speed
Rate；
Feedback loop module, link control module and image recording and image processing module, according to image dynamic change
The signal of rate output automatic adjustment sampling frame per second.
The super-resolution imaging module and low resolution imaging module, the two share the same microcobjective when being imaged, and
Position of identical, the observed object plane of observed field of view in sample is also identical；
The super-resolution imaging module and low resolution imaging module, exposure dose when low resolution imaging on sample are far weaker than
Exposure dose when super-resolution imaging on sample is smaller to fluorescent bleach and phototoxicity caused by sample；
The super-resolution imaging module and low resolution imaging module, the two can be independently imaged.
It is basic corresponding to a kind of dynamic super-resolution fluorescence micro image shooting method of the fluorescent microscopic imaging device
Concept is as follows:
Fluorescent bleach and phototoxicity of the low resolution imaging module with lower exposure dose irradiating sample, when imaging to sample
It is smaller, and the sampling frame per second of low resolution imaging module fixation with higher, the speed for real-time monitoring sample dynamic change
Rate；The low resolution original image of acquisition is sent into image recording and image processing module carries out image procossing, analyzes and is continuously shot
The low resolution image of multiframe (two frames or more) dynamic change rate, feedback loop module becomes according to low resolution image dynamic
The rate of change exports corresponding signal to control module, and the output signal is used to adjust the sampling frame per second of super resolution image, when
The rate of sample dynamic change is faster, and the sampling frame per second of super resolution image is higher, conversely, the rate of sample dynamic change is slower,
The sampling frame per second of super resolution image is lower；Control module according to the signal received in real time super-resolution imaging and it is low differentiate at
It does and switches as between；Super-resolution imaging module shoots sample with the sampling frame per second that dynamic adjusts, whenever super-resolution imaging
When sample is imaged in module, low resolution imaging module meeting break-off, after the completion of frame super resolution image acquisition, low point
It distinguishes that image-forming module can resume work, continues to monitor；The above shooting process persistently carries out until sample is bleached or acquires
Enough frame numbers；Super resolution image collected and low resolution image are all recorded in image recording and image processing module.
Further, the dynamic super-resolution fluorescence micro image shooting method realizes that process needs following link:
(1) adjusting, correction and preparation before shooting to fluorescent microscopic imaging device；(2) real-time image procossing in shooting process；
(3) timing control in shooting process；(4) image recording；It is specific as follows:
1. adjusting, correction and preparation before shooting to fluorescent microscopic imaging device
1.1 adjust the optical path in calibration super-resolution imaging module and low resolution imaging module, the visual field area for observing the two
Position of identical, the observed object plane in domain in sample is also identical.
1.2 collected original images need to first pass through image preprocessing step before further using, and remember in image
The method that the setting each step of image preprocessing can be used in record and image processing module, each pre-treatment step and its method are such as
Under:
1.2.1 low resolution image picture element interpolation
When the low size (horizontal pixel number, longitudinal pixel number) for differentiating original image is less than the size of super-resolution original image
When, need to be inserted into it pixel to reach (horizontal pixel number, longitudinal pixel number are all identical) identical with super resolution image size, and
Interpolation is carried out in the location of pixels newly inserted, the image smoothing after making interpolation.
1.2.2 image registration
Registration described here, sample object plane maps to the pattern distortion as generating when plane when being to correct imaging；
Sample using chessboard grid pattern, or comprising a large amount of known reference point coordinates marks the image of acquisition as standard sample
It is fixed, correcting image distortion, make on mapped image the relative position of each point on sample the relative position of script it is identical；To mark
Quasi- sample obtains mapping the rectification function after completing registration, and the formal acquired image that shoots just uses the mapping the rectification function to carry out
Image registration.
1.2.3 pixel value is corrected
The purpose of pixel value correction be exactly by adjusting the pixel value of position each on image, make on object plane fluorescent brightness and
The corresponding relationship of pixel value is in each location of pixels uniformity on image；It is carried out using the standard sample of known fluorescent brightness
Pixel value correction；Pixel value the rectification function is obtained after completing correction to standard sample, the formal acquired image that shoots just uses
The pixel value the rectification function is corrected.
1.2.4 noise is filtered out
It is described to filter out noise, it is therefore an objective to filter out intrinsic shot when including: noise, the fluorescence dim light emission of lighting source and make an uproar
Picture noise caused by electronic noises of image-forming components such as sound, the flashing of fluorescent molecule, CCD/CMOS/APD/PMT etc.；Using equal
The methods of value filtering, gaussian filtering, median filtering, bilateral filtering filter out noise to image.
2. real-time image procossing in shooting process
Scan picture is carried out for low resolution image, is completed by image recording and image processing module, mesh
Be the rate for analyzing image dynamic change in real time during shooting, the specific steps are as follows:
2.1 image preprocessing
As it was noted above, each step of image preprocessing is provided with before shooting in image recording and image processing module
Each frame image of rapid processing method, low resolution imaging module acquisition is sent into image recording and image processing module in real time
Carry out image preprocessing；After completing image preprocessing, next step could be continued.
2.2 fluorescent bleaches correction
During being continuously shot, fluorescent molecule can be bleached, and image can become that darkness deepens, by acquired image
Luminance standardization processing is done, i.e. fluorescent bleach is corrected, antidote are as follows: do the brightness of the low resolution image of each frame at standardization
Reason, make the pixel value in non-zero pixel region on unit area and the low resolution image that acquires for the first time in unit in non-zero pixel region
Pixel value on area is identical；After completing fluorescent bleach correction, next step could be continued.
2.3 analysis image dynamic change rates
The fixed detection window of one width is set, the low resolution image that N frame is continuously shot can be accommodated in detection window,
Middle N is the frame number of setting, and N also represents the width of detection window；The adjacent image of every two frame, calculates in comparison detection window
The rate of the image dynamic change in detection window is measured in light stream on image with light stream, and the light stream is vector, is illustrated
Movement of the angle point (characteristic point) in the two field pictures of front and back on image；
Analyzing image dynamic change rate, specific step is as follows:
(1) when the number of image frames for including in detection window is less than N, image is successively read in by the time sequencing of acquisition, directly
Number of image frames in detection window reaches N；
(2) first frame image and the second frame image in comparison detection window, calculate light stream；And so on, compare kth frame
+ 1 frame image of image and kth, calculates light stream, until completeer N-1 frame image and nth frame image；N-1 light is calculated altogether
Stream；
(3) after counting N-1 optical flow computation, the sum of light stream vector, including the light stream vector that mould is 0 also counts；Remember again
Record the information of each light stream vector, the information of each light stream vector includes: that starting point coordinate, terminal point coordinate, the light stream vector are related to
Two field pictures frame number；The sum of light stream vector and the information of light stream vector generate data packet<dynamic-change information>together；
(4) data packet<dynamic-change information>is sent to feedback loop module；Then in the method for first in first out,
The frame image of moment earliest will be acquired in detection window and removes detection window, then is read in the new image of a frame and entered detecting window
Mouthful, the acquisition moment of the new images supplemented follows the nth frame image of script in detection window closely；
(5) whenever acquiring super resolution image, all images in detection window are emptied, are acquired to a frame super resolution image
After, detection window reads in low resolution image again, low after the image read at first at this time i.e. super resolution image acquisition
Resolution imaging module first frame image collected；
(6) step (1)~(5) are repeated in entire shooting process, continue and analyze in real time image dynamic change rate,
One<dynamic-change information>data packet of every generation, is soon sent to feedback loop module.
3. the timing control in shooting process
The timing control, refers to when control super-resolution imaging module and low resolution imaging module are acquiring image；Instead
After feedback return circuit module receives data packet<dynamic-change information>, it is converted into the adjusting letter of super resolution image sampling frame per second
Number, then by adjustment signal input control module, generate the timing control signal of super resolution image and the acquisition of low resolution image, timing
Detailed process is as follows for control:
(1) the sampling frame per second for presetting low resolution imaging module is fl, i.e., the low resolution image acquisition of adjacent two frame when
Between between be divided into tl=1/fl；With fixed sampling frame per second flThe low resolution image of continuous collecting is low only when acquiring super resolution image
Resolution imaging module can suspend acquisition, and after acquiring to a frame super resolution image, low resolution imaging module can be again extensive immediately
Second mining collection；
(2) a function T (), referred to as time interval function, the input variable of the function are set in feedback loop module
For data packet<dynamic-change information>, output variable is the acquisition time interval t of adjacent two frames super resolution images, due to sampling
Frame per second fsWith acquisition time interval reciprocal relation each other: fs=1/ts, so it is equivalent to adjust acquisition time interval by T () function
Frame per second is sampled in adjusting；Time interval function T () has the property that
1. the size of all light stream vectors is all to swear in 0, data packet without light stream in data packet<dynamic-change information>
When measuring or being inputted without data packet, the output t of T ()s=t0, t0Indicate the default value at super resolution image acquisition time interval；
t0It is also the upper limit of T () function-output；
2. limiting the lower limit t of T () function-outputs_min, make ts_min≥N*tl, wherein N can be accommodated low for detection window
The frame number of resolution image, tlFor the acquisition time interval of low resolution image；
3. the interior light stream vector of data packet<dynamic-change information>increases, tsValue decline；Light stream vector reduces in data packet, ts
Value rises；tsVariation range in section [ts_min, t0] on；
(3) it enables t indicate the time in entire shooting process, shoots initial time t=0；Enable taIndicate that acquiring a certain frame surpasses
At the time of resolution image, t is enabledbAt the time of indicating estimated acquisition next frame super resolution image；taThe super resolution image at moment has acquired
First<dynamic-change information>the data packet input time interval function T () generated after finishing, obtains the output valve t of functions=
t1, i.e., t at the time of estimated next time acquires super resolution imageb=ta+t1；
(4) in taTo tbPeriod in, subsequent<dynamic-change information>data packet, each data packet can be received one by one
T () function, obtained function-output t will be inputteds=ti(wherein i indicates<dynamic-change information>data packet received
Number), work as ti< t1When, it is contemplated that t at the time of acquisition super resolution image next timebBy ta+t1It is changed to ta+ti；Work as ti≥t1When, in advance
T at the time of meter acquisition next time super resolution imagebKeep ta+t1It is constant；This process persistently carries out, and for any j > i, works as tj< ti
When, it is contemplated that t at the time of acquisition super resolution image next timebBy ta+tiIt is changed to ta+tj, work as tj≥tiWhen, it is contemplated that acquisition next time oversubscription
T at the time of distinguishing imagebKeep ta+tiIt is constant；
(5) as t >=tbWhen, stop reading in new<dynamic-change information>data packet, and start super-resolution imaging mould immediately
Block acquires the super resolution image of a new frame；When acquiring super resolution image, suspend the acquisition of low resolution imaging module, to super-resolution
After Image Acquisition, low resolution imaging module can be restored to acquire again immediately, and low resolution image collected is read into detection
Window is for analyzing image dynamic change rate；
(6) step (3)~(5) are repeated, until completing entire shooting process.
4. image recording
All super resolution images and low resolution image acquired in shooting process is all stored in image recording and image
In processing module, and at the time of will record the every frame super resolution image of lower acquisition and low resolution image, wherein shooting initial time
It is 0.Can carry out post-processing to the image of record after the completion of shooting will own using the super resolution image of shooting as priori knowledge
Low resolution image be converted to super resolution image, make the observation to sample dynamic process with more integrality and continuity；It is related to
The part of image post-processing is not belonging to the contents of the present invention, does not further spread out.
The above is links needed for realizing dynamic super-resolution fluorescence micro image shooting method, wherein link
(2) timing control in the scan picture in shooting process and link (3) shooting process is combined together, and realizes oversubscription
Distinguish the automatic adjusument of image sampling frame per second.Super-resolution imaging module described in technical solution of the present invention, can be used STED,
A variety of super-resolution imaging methods published such as RESOLFT, PALM, FPALM, STORM, SIM, SSIM, NSIM；Skill of the present invention
It is micro- that Both wide field illumination micro-imaging, total internal reflection illumination (TIRF) can be used in low resolution imaging module described in art scheme
Imaging, scanning confocal microscopic image, laser layer shine the imaging methods such as micro- (light sheet/SPIM) imaging, and used
Low resolution imaging method have the sampling frame per second of high speed.
Due to the adoption of the above technical solution, the invention has the following advantages that
(1) the sampling frame per second of super resolution image has the ability of automatic adjusument, in the slowly varying process of observation sample
When, it is unlikely to be bleached fluorescent molecule too early because of the sampling frame per second of high speed, can extends as far as possible to sample dynamic process
Shooting time；
(2) low resolution imaging module can have the sampling frame per second of high speed, the quick dynamic process of sample can be responded, by anti-
Feedback adjusts the sampling frame per second of super resolution image in time, carries out super-resolution imaging to quick dynamic process；
(3) super-resolution imaging module can use STED, RESOLFT, PALM, FPALM, STORM, SIM, SSIM, NSIM etc.
A variety of super-resolution imaging methods make the good compatibility of scheme.
Detailed description of the invention
Fig. 1 is the structural block diagram of fluorescent microscopic imaging device of the present invention；
Fig. 2 is the implementation process of super resolution image sampling frame per second automatic adjusument of the present invention；
Fig. 3 is the light path schematic diagram of one embodiment provided by the invention.
Specific embodiment
In order to make those skilled in the art more fully understand technical solution of the present invention, below in conjunction with embodiment and reference
Invention is further described in detail for attached drawing.It should be appreciated that the skill described herein that the examples are only for explaining the invention
Art scheme, described embodiment are only a part of the embodiment of the present invention, are based on technical solution of the present invention, those skilled in the art
Member's other embodiments obtained without making creative work, all should belong to the scope of the present invention it
It is interior.
The general configuration of the optical path of embodiment is as shown in figure 3, the present embodiment is suitable for the sample of thinner thickness, Both wide field illumination
When, the fluorescence signal other than object plane can be ignored.In this embodiment, super-resolution imaging is carried out using STED, use is general
Logical Both wide field illumination carries out low resolution imaging.The element that super-resolution imaging module includes has: lens group, beam spot scans module, anti-
Penetrate mirror, dichroic mirror 1/2, bandpass filter, optical fiber, detector, phase plate, exciting light, loss light；Low resolution imaging module includes
Element have: dichroic mirror 3, lens 1/2, CCD camera, Both wide field illumination light source；The shared element of two modules are as follows: object lens, beam splitting
Piece.Super-resolution imaging module and low resolution imaging module have been separated by beam splitting chip.Low resolution imaging module uses and falls the formula of penetrating
Kohler illumination, light source is imaged on the back focal plane of object lens, makes the uniform intensity being radiated on sample.Swashed by control module control
It shines, the opening and closing of loss light, Both wide field illumination light source, can make to switch between super-resolution imaging and low resolution imaging.With
CCD makes low resolution imaging have the sampling frame per second of high speed entire view field imaging.Exciting light and loss are needed when super-resolution imaging
The point by point scanning on sample of the hot spot of light, then collects fluorescence, finally constitutes entire image.The optical path of the embodiment is through toning
Section, is overlapped super-resolution imaging and the object plane of low resolution imaging, and the two field of view is identical.
The specific implementing procedure of the present embodiment is as follows:
1. adjusting, correction and preparation before shooting to fluorescent microscopic imaging device
1.1 adjust the optical path in calibration super-resolution imaging module and low resolution imaging module, the visual field area for observing the two
Position of identical, the observed object plane in domain in sample is also identical；
1.2 are arranged the method that each step of image preprocessing can be used in image recording and image processing module, comprising:
(1) low resolution image picture element interpolation, (2) image registration, the correction of (3) pixel value, (4) filter out noise.
2. shooting and recording the image of acquisition
It enables t indicate the time in entire shooting process, shoots initial time t=0；When starting shooting, first in t=0
It carves and acquires first frame super resolution image, be recorded in image recording and image processing module；Then it is switched to by control module low
Resolution image acquisition, the sampling frame per second of low resolution image are fixed as fl；The low resolution original image of each frame of acquisition all first passes through
Image preprocessing step and fluorescent bleach rectification step, are then successively read in detection window again, when the image in detection window
When frame number reaches the value N of setting, the dynamic change rate of image in detection window at this time is analyzed using the method for calculating light stream, and
It generates data packet<dynamic-change information>；<dynamic-change information>data packet is sent to feedback loop module, feedback loop mould
Time interval function T () has been preset in block, for by<dynamic-change information>data packet be converted to super resolution image acquisition when
Between be spaced ts, for first data packet, the t that is converted tosValue be expressed as t1, i.e., estimated acquisition next time super resolution image when
Carving is t1；If t >=t at this time1, stop acquiring low resolution image, empty detection window, and acquire the oversubscription of the second frame immediately
It distinguishes image, after acquiring to super resolution image, restores the acquisition of low resolution image again；Conversely, if t < t at this time1, visit
It surveys window to continue to read in the low resolution image of a frame, and the frame image acquired at first in script window is removed, then analyze N at this time
The dynamic change rate of frame image generates and sends second<dynamic-change information>data packet to feedback loop module, through T
() function is converted to ts=t2, work as t2< t1When, it is contemplated that t is advanced at the time of acquisition super resolution image next time2, otherwise still protect
It holds as t1；As shown in Fig. 2, above-mentioned " analysis image rate of change --- sending<dynamic-change information>data packet --- T () function
Be converted to acquisition time interval ts" process repeat, for any j > i, work as tj< ti, it is contemplated that acquisition next time super-resolution figure
As at the time of be changed to tj, otherwise it is maintained as ti；Until t >=ti, complete the acquisition of the second frame super resolution image.
Above procedure is to realize the automatic adjusument of super resolution image sampling frame per second, constantly heavy during shooting
Multiple above step, the super resolution image of frame number needed for acquiring, or until sample can not be further continued for shooting by fluorescent bleach.Entirely
All super resolution images and low resolution image acquired in shooting process, which all record, is stored in image recording and image processing module
In, and record acquire every frame image at the time of.
The representation of time interval function T () is not unique, such as can be expressed as form:
Wherein, N is the number of image frames that detection window can accommodate；flFor the sampling frame per second of low resolution image；t0For the super of default
Resolution image samples frame per second；VmFor the mould of the maximum light stream vector of modulus value in<dynamic-change information>data packet；VthIt is set for one
The fixed threshold value for being greater than 0；ε () is jump function, works as Vm-VthWhen >=0, ε ()=1, otherwise ε ()=0；M is < dynamic change letter
The sum of light stream vector in breath > data packet；ViFor the mould of i-th of light stream vector in<dynamic-change information>data packet；∑iVi/M
It indicates to calculate average value to the mould of all light stream vectors；Two factor alphas, β >=0.
As shown from the above formula, when all light stream vectors are all 0, it is meant that image does not change, and obtains ts=t0,
Use the super resolution image acquisition time interval of default namely tsThe maximum value of value；With the increase of light stream vector, tsGradually
Reduce, when light stream vector is very big, ts→N/fl, super resolution image is acquired using the smallest time interval；When maximum light stream
Vector is more than the threshold value V of settingthWhen, it is meant that even if other light stream vectors all very littles (even 0) at this time, image integrally becomes
In the case where changing less, there is also the faster dynamic changes in part, therefore are also required to reduce ts, specific adjustment amount is depending on being
The size of number α；β×∑iVi/ M is used to measure the situation of change of image entirety, tsAdjustment amount depend on factor beta size.
Claims (6)
1. a kind of fluorescent microscopic imaging device, which is characterized in that the fluorescent microscopic imaging device is functionally divided into as follows
Several parts:
(1) sample stage is observed sample for carrying；
(2) super-resolution imaging module, the module include a series of optical elements, image device and light source, use super-resolution imaging
When module is imaged, image has high spatial resolution, can break through Abbe diffraction limit；
(3) low resolution imaging module, the module include a series of optical elements, image device and light source, use low resolution imaging
When module is imaged, image has low spatial resolution, cannot break through Abbe diffraction limit；
(4) control module is shot for controlling super-resolution imaging module and low resolution imaging module, including two moulds of control
Light source and optical element in block；
(5) image recording and image processing module, for recording image and carrying out image procossing, analysis image dynamic change speed
Rate；
(6) feedback loop module, link control module and image recording and image processing module, according to the speed of image dynamic change
The signal of rate output automatic adjustment sampling frame per second；
The super-resolution imaging module and low resolution imaging module share the same microcobjective, and optical path is after overregulating,
The imaging of two modules visual field is identical, object plane position is identical in sample；Other components in entire fluorescent microscopic imaging device,
It include: that optical element, image device, light source etc. can also be shared by two modules；
The super-resolution imaging module and low resolution imaging module, each self energy are independently imaged sample；Two modules carry out
It is required when imaging with the fluorescent molecule in light source activation sample, exposure dose when low resolution imaging on sample is far weaker than oversubscription
Exposure dose when distinguishing imaging on sample keeps low resolution imaging smaller to fluorescent bleach and phototoxicity caused by sample.
2. a kind of dynamic super-resolution fluorescence micro image shooting method using fluorescent microscopic imaging device, which is characterized in that institute
It is as follows to state dynamic super-resolution fluorescence micro image shooting method:
Low resolution imaging module with lower exposure dose irradiating sample, when imaging to the fluorescent bleach of sample and phototoxicity compared with
It is small, and the sampling frame per second of low resolution imaging module fixation with higher, the rate for real-time monitoring sample dynamic change；
The low resolution original image of acquisition is sent into image recording and image processing module and carries out image procossing, analyze be continuously shot it is more
The rate of the dynamic change of the low resolution image of frame (two frames or more), feedback loop module is according to low resolution image dynamic change
Rate exports corresponding signal to control module, and the output signal is used to adjust the sampling frame per second of super resolution image, works as sample
The rate of dynamic change is faster, and the sampling frame per second of super resolution image is higher, conversely, the rate of sample dynamic change is slower, oversubscription
Distinguish that the sampling frame per second of image is lower；Control module according to the signal received in real time super-resolution imaging and low resolution imaging it
Between do and switch；Super-resolution imaging module shoots sample with the sampling frame per second that dynamic adjusts, whenever super-resolution imaging module
When sample is imaged, low resolution imaging module can break-off, after the completion of frame super resolution image acquisition, it is low differentiate at
As module can resume work, continue to monitor；The above shooting process persistently carries out until sample is bleached or acquires enough
Frame number；Super resolution image collected and low resolution image are all recorded in image recording and image processing module；
After the original image refers to that image device receives the fluorescence signal from sample, the image of direct acquisition and recording, without
Cross image procossing.
3. a kind of dynamic super-resolution fluorescence micro image shooting method using fluorescent microscopic imaging device, which is characterized in that real
The existing dynamic super-resolution fluorescence micro image shooting method needs following link:
(1) adjusting, correction and preparation before shooting to fluorescent microscopic imaging device
1.1) optical path in calibration super-resolution imaging module and low resolution imaging module, the field of view phase for observing the two are adjusted
It is also identical with, position of the observed object plane in sample；
1.2) each step meeting of image preprocessing is set in the image recording of fluorescent microscopic imaging device and image processing module
The method used；
Above step 1.1), 1.2) execute in order；
(2) real-time image procossing in shooting process
Scan picture is carried out for low resolution image, is completed by image recording and image processing module, it is therefore an objective to
Analyze the rate of image dynamic change in real time during shooting, the specific steps are as follows:
2.1) image preprocessing
Using pre-set image pre-processing method in image recording and image processing module, in shooting process to low resolution
Each frame image of image-forming module acquisition carries out image preprocessing in real time；After completing image preprocessing, it could continue next
Step；
2.2) fluorescent bleach is corrected
The low resolution image of every frame does fluorescent bleach correction, antidote again after image preprocessing are as follows: by the low resolution of each frame
Standardization is done in the brightness of image, the low resolution figure for making the pixel value in non-zero pixel region on unit area and acquiring for the first time
Pixel value as in non-zero pixel region on unit area is identical；After completing fluorescent bleach correction, next step could be continued
Suddenly；
2.3) image dynamic change rate is analyzed
The fixed detection window of one width is set, the low resolution image that N frame is continuously shot can be accommodated in detection window, wherein N
For the frame number of setting, N also represents the width of detection window；The adjacent image of every two frame, calculates figure in comparison detection window
As upper light stream, the rate of the image dynamic change in detection window is measured with light stream, the light stream is vector, illustrates figure
As movement of the upper angle point (characteristic point) in the two field pictures of front and back；After image dynamic change rate analysis, reflection is generated
The data packet of image dynamic change rate, is then forwarded to feedback loop module；
(3) timing control in shooting process
The timing control, refers to when control super-resolution imaging module and low resolution imaging module are acquiring image；It feeds back to
After road module receives the data packet of analysis image dynamic change rate, it is converted into the adjusting of super resolution image sampling frame per second
Signal, then by adjustment signal input control module, generate the timing control signal of super resolution image and the acquisition of low resolution image；
(4) image recording
All super resolution images and low resolution image acquired in shooting process is all stored in image recording and image procossing
In module, and at the time of will record the every frame super resolution image of lower acquisition and low resolution image, wherein shooting initial time is 0.
4. dynamic super-resolution fluorescence micro image shooting method as claimed in claim 3, which is characterized in that the step 1.2)
Image preprocessing in step 2.1), includes the following steps:
(1) low resolution image picture element interpolation
When the low size (horizontal pixel number, longitudinal pixel number) for differentiating original image is less than the size of super-resolution original image,
Need to be inserted into it pixel to reach (horizontal pixel number, longitudinal pixel number are all identical) identical with super resolution image size, and
The location of pixels newly inserted carries out interpolation, the image smoothing after making interpolation；
(2) image registration
Registration described here, sample object plane maps to the pattern distortion as generating when plane when being to correct imaging；It uses
Chessboard grid pattern, or the sample comprising a large amount of known reference point coordinates demarcate the image of acquisition as standard sample, rectify
Positive image distortion, make on mapped image the relative position of each point on sample the relative position of script it is identical；To standard sample
Product obtain mapping the rectification function after completing registration, and the formal acquired image that shoots just uses the mapping the rectification function to carry out image
Registration；
(3) pixel value is corrected
The purpose of pixel value correction is exactly to make fluorescent brightness and image on object plane by adjusting the pixel value of position each on image
The corresponding relationship of upper pixel value is in each location of pixels uniformity；Pixel is carried out using the standard sample of known fluorescent brightness
Value correction；Pixel value the rectification function is obtained after completing correction to standard sample, the formal acquired image that shoots just uses the picture
Plain value the rectification function is corrected；
(4) noise is filtered out
It is described to filter out noise, it is therefore an objective to filter out intrinsic shot noise when including: noise, the fluorescence dim light emission of lighting source,
Picture noise caused by electronic noise of the image-forming components such as flashing, the CCD/CMOS/APD/PMT of fluorescent molecule etc.；Use mean value
The methods of filtering, gaussian filtering, median filtering, bilateral filtering filter out noise to image.
5. dynamic super-resolution fluorescence micro image shooting method as claimed in claim 3, which is characterized in that the step 2.3)
Analyzing image dynamic change rate, specific step is as follows:
(1) the low resolution image that N frame is continuously shot can be accommodated by being arranged in detection window, when the number of image frames for including in detection window
When less than N, image is successively read in by the time sequencing of acquisition, until the number of image frames in detection window reaches N；
(2) first frame image and the second frame image in comparison detection window, calculate light stream；And so on, compare kth frame image
With+1 frame image of kth, light stream is calculated, until completeer N-1 frame image and nth frame image；N-1 light stream is calculated altogether；
(3) after counting N-1 optical flow computation, the sum of light stream vector, including the light stream vector that mould is 0 also counts；It re-records every
The information of a light stream vector, the information of each light stream vector include: two that starting point coordinate, terminal point coordinate, the light stream vector are related to
The frame number of frame image；The sum of light stream vector and the information of light stream vector generate data packet<dynamic-change information>together；
(4) data packet<dynamic-change information>is sent to feedback loop module；Then it in the method for first in first out, will visit
It surveys the frame image of acquisition moment earliest in window and removes detection window, then read in the new image of a frame and enter detection window, institute
The acquisition moment of the new images of supplement follows the nth frame image of script in detection window closely；
(5) whenever acquiring super resolution image, all images in detection window are emptied, are finished to frame super resolution image acquisition
Afterwards, detection window reads in low resolution image again, after the image read at first at this time i.e. super resolution image acquisition, low resolution
Image-forming module first frame image collected；
(6) step (1)~(5) are repeated in entire shooting process, continue and analyze image dynamic change rate, every life in real time
At<dynamic-change information>data packet, it is soon sent to feedback loop module.
6. dynamic super-resolution fluorescence micro image shooting method as claimed in claim 3, which is characterized in that the link (3)
Timing control in shooting process, detailed process is as follows:
(1) the sampling frame per second for presetting low resolution imaging module is f1, i.e., the time interval of the low resolution image acquisition of adjacent two frame
For t1=1/f1；With fixed sampling frame per second f1The low resolution image of continuous collecting, only when acquiring super resolution image, it is low differentiate at
As module can suspend acquisition, to frame super resolution image acquisition after, low resolution imaging module can be restored to acquire again immediately；
(2) a function T (), referred to as time interval function are set in feedback loop module, and the input variable of the function is point
Data packet<dynamic-change information>generated after analysis image dynamic change rate, output variable are adjacent two frames super resolution image
Acquisition time interval ts, due to sampling frame per second fsWith acquisition time interval reciprocal relation each other: fs=1/ts, so passing through T ()
Function adjusts acquisition time interval and is equal to adjusting sampling frame per second；Time interval function T () has the property that
1. in data packet<dynamic-change information>size of all light stream vectors be all in 0, data packet without light stream vector or
When person does not have data packet input, the output t of T ()s=t0, t0Indicate the default value at super resolution image acquisition time interval；t0It is also
The upper limit of T () function-output；
2. limiting the lower limit t of T () function-outputs_min, make ts_min≥N*t1, wherein N is the low resolution that detection window can accommodate
The frame number of image, t1For the acquisition time interval of low resolution image；
3. the interior light stream vector of data packet<dynamic-change information>increases, tsValue decline；Light stream vector reduces in data packet, tsIn value
It rises；tsVariation range in section [ts_min, t0] on；
(3) it enables t indicate the time in entire shooting process, shoots initial time t=0；Enable taIt indicates to acquire a certain frame super-resolution figure
At the time of picture, t is enabledbAt the time of indicating estimated acquisition next frame super resolution image；taIt is raw after the super resolution image acquisition at moment
At first<dynamic-change information>data packet input time interval function T (), obtain the output valve t of functions=t1, i.e., in advance
T at the time of meter acquisition next time super resolution imageb=ta+t1；
(4) in taTo tbPeriod in, subsequent<dynamic-change information>data packet can be received one by one, each data packet can
Input T () function, obtained function-output ts=ti(wherein i indicates the volume of<dynamic-change information>data packet received
Number), work as ti< t1When, it is contemplated that t at the time of acquisition super resolution image next timebBy ta+t1It is changed to ta+ti；Work as ti≥t1When, it is contemplated that
T at the time of acquisition super resolution image next timebKeep ta+t1It is constant；This process persistently carries out, and for any j > i, works as tj< tiWhen,
It is expected that t at the time of acquisition super resolution image next timebBy ta+tiIt is changed to ta+tj, work as tj≥tiWhen, it is contemplated that acquisition next time super-resolution
T at the time of imagebKeep ta+tiIt is constant；
(5) as t >=tbWhen, stop reading in new<dynamic-change information>data packet, and start super-resolution imaging module immediately, adopt
Collect the super resolution image of a new frame；When acquiring super resolution image, suspends the acquisition of low resolution imaging module, adopted to super resolution image
After collection, low resolution imaging module can be restored to acquire again immediately, and low resolution image collected is read into detection window use
In analysis image dynamic change rate；
(6) step (3)~(5) are repeated, until completing entire shooting process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711037932.3A CN109724950A (en) | 2017-10-27 | 2017-10-27 | Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711037932.3A CN109724950A (en) | 2017-10-27 | 2017-10-27 | Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109724950A true CN109724950A (en) | 2019-05-07 |
Family
ID=66292345
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711037932.3A Pending CN109724950A (en) | 2017-10-27 | 2017-10-27 | Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109724950A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378853A (en) * | 2019-07-11 | 2019-10-25 | Oppo广东移动通信有限公司 | Depth map treating method and apparatus |
| CN110956607A (en) * | 2019-07-11 | 2020-04-03 | 中国科学院国家天文台 | Method and device for testing ultimate signal-to-noise ratio and stability of ultimate signal-to-noise ratio |
| CN111258264A (en) * | 2020-02-24 | 2020-06-09 | 北京龙鼎源科技股份有限公司 | Filtering method and device for field noise, storage medium and processor |
| CN112162079A (en) * | 2020-09-09 | 2021-01-01 | 中国科学院过程工程研究所 | Unattended testing system device and testing method for thermophysical parameters of melt |
| CN112268879A (en) * | 2020-09-17 | 2021-01-26 | 中国科学院生物物理研究所 | Fluorescence microscopic imaging system |
| CN113099108A (en) * | 2021-03-11 | 2021-07-09 | 广州安凯微电子股份有限公司 | Control method and device of camera equipment |
| CN114112007A (en) * | 2021-10-09 | 2022-03-01 | 昆明嘉和科技股份有限公司 | Data processing method for intelligent wireless vibration node of mobile equipment |
-
2017
- 2017-10-27 CN CN201711037932.3A patent/CN109724950A/en active Pending
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378853A (en) * | 2019-07-11 | 2019-10-25 | Oppo广东移动通信有限公司 | Depth map treating method and apparatus |
| CN110956607A (en) * | 2019-07-11 | 2020-04-03 | 中国科学院国家天文台 | Method and device for testing ultimate signal-to-noise ratio and stability of ultimate signal-to-noise ratio |
| CN110378853B (en) * | 2019-07-11 | 2021-03-26 | Oppo广东移动通信有限公司 | Depth map processing method and device |
| CN110956607B (en) * | 2019-07-11 | 2023-04-07 | 中国科学院国家天文台 | Method and device for testing ultimate signal-to-noise ratio and stability of ultimate signal-to-noise ratio |
| CN111258264A (en) * | 2020-02-24 | 2020-06-09 | 北京龙鼎源科技股份有限公司 | Filtering method and device for field noise, storage medium and processor |
| CN111258264B (en) * | 2020-02-24 | 2021-06-15 | 北京龙鼎源科技股份有限公司 | Filtering method and device for field noise, storage medium and processor |
| CN112162079A (en) * | 2020-09-09 | 2021-01-01 | 中国科学院过程工程研究所 | Unattended testing system device and testing method for thermophysical parameters of melt |
| CN112268879A (en) * | 2020-09-17 | 2021-01-26 | 中国科学院生物物理研究所 | Fluorescence microscopic imaging system |
| CN112268879B (en) * | 2020-09-17 | 2022-08-26 | 中国科学院生物物理研究所 | Fluorescence microscopic imaging system |
| CN113099108A (en) * | 2021-03-11 | 2021-07-09 | 广州安凯微电子股份有限公司 | Control method and device of camera equipment |
| CN114112007A (en) * | 2021-10-09 | 2022-03-01 | 昆明嘉和科技股份有限公司 | Data processing method for intelligent wireless vibration node of mobile equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109724950A (en) | Dynamic super-resolution fluorescence imaging technique with adaptively sampled frame per second | |
| US10602087B2 (en) | Image acquisition device, and imaging device | |
| AU703583B2 (en) | Method and apparatus for rapid capture of focused microscopic images | |
| CN110225248A (en) | Image-pickup method and device, electronic equipment, computer readable storage medium | |
| JP5289756B2 (en) | Microscope observation system | |
| US20050056767A1 (en) | Image focusing in fluorescent imaging | |
| CN106210520B (en) | A kind of automatic focusing electronic eyepiece and system | |
| JP2008020498A (en) | Automatic focus detection device | |
| CN107219207B (en) | Automatic focusing method of CCD biochip fluorescence scanner | |
| DE60223693T2 (en) | STANDBY GRADING DEVICE AND STILL IMAGING METHOD | |
| CN103079032A (en) | High-definition capillary electrophoresis gel imaging system | |
| CN105592258B (en) | Auto focusing method and device | |
| CN110502942A (en) | Scanning means and its light filling lamp control method | |
| CN107431764A (en) | The control method of camera, camera main body and camera | |
| CN110049240A (en) | Camera control method, device, electronic equipment and computer readable storage medium | |
| WO2016101092A1 (en) | Digital microscope and focusing method thereof | |
| JP2012120132A (en) | Imaging apparatus and program | |
| CN101509998B (en) | Automatic focusing method and microscope applying the method | |
| CN109724951A (en) | A kind of dynamic super-resolution fluorescence imaging technique | |
| CA3106449A1 (en) | Scanning microscope using pulsed illumination | |
| JP2005215712A (en) | Imaging device for counting individual number and control means | |
| CN105319725B (en) | Super-resolution imaging method for fast moving objects | |
| US11256078B2 (en) | Continuous scanning for localization microscopy | |
| JP2008197144A (en) | Photographing apparatus, and focus control method and control unit for photographic lens | |
| JP5222779B2 (en) | License plate recognition device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190507 |
|
| WD01 | Invention patent application deemed withdrawn after publication |