CN1534323A - Method of making biological microscope imaging technique reach increasing 3D field depth and resolution degree - Google Patents

Abstract

A method for increasing the focus depth and resolution of the image formed by biomicroscope includes using gel to embed a specimen in it for fixing the specimen in 3D space, scanning the front surface and back surface of specimen, and combining the scanned front and back images in 3D mode. A method for combining said images are also disclosed.

Description

The biological microscope image technology reaches the method that increases by the 3 dimension depth of field and resolution

Technical field

The present invention relates to a kind of method, especially the biological microscope image technology reaches the method that increases by the 3 dimension depth of field and resolution.

Background technology

Known confocal microscope is by removing the noise that non-focal plane produces, to obtain the high-res micro-imaging of different depth in sample.Its cardinal principle can be divided into three steps and illustrate: at first laser can become single luminous point via object lens focusing, shines the certain depth of sample single-point; In addition, the light that is reflected or dispersed by focus also can become single light beam via object lens focusing, fully by the pin hole (pinhole aperture) before the image detector; At last, other noise photons that following non-focus produced that reach on focus then are blocked around this pin hole down, thereby guarantee the accuracy of detector acquisition focal plane signal, and then reach the micro-imaging (as shown in Figure 1) of the different depth high-res of being wished to get.Therefore, pin hole is more little, just can remove many more noises, and obtains single-minded and micro-imaging clearly.

The microscope that compares confocal microscope and traditional type, the former has more advantage significantly: in the luminescence microscope of traditional type, if will observe the image of sheet biological tissue in Z-direction, only can observe the interior limited focal plane scope of institute's use object lens depth of field with limiting to, exceeded this scope, the light of non-focal plane will have a strong impact on the light of focal plane, cause contrast to slacken and image fog, and, if the sample of observing can distribute multiple fluorescent simultaneously, the image of the various single fluorescents of then wishing to get all will mix the noise that other spectrum fluorescents are arranged; The confocal microscope then is specially designed to these sheet fluorescent samples (as tissue of biological cells), the function of its optical cross section can significantly lower conventional microscope the non-focal plane noise that can't remove, on the sample of multiple fluorescent, also can separate the fluorescent message of different spectral ranges definitely, and obtain the clear micro-imaging of the different depth of multiple fluorescent sheet tissue.

At present for increasing the depth of field that microscope is swept figure, employed method has two kinds, one for to utilize two microscope stands to be located at the front of sample and the position of reverse side, after having calculated relative position, do the conjugation capture from positive with reverse side respectively, the image thickness of gained approximately be the separate unit microscope the twice of getable image group thickness, but this way increases required hardware cost widely.It is two for the technology of utilizing multiphoton microscope increases the thickness of image group, and its hardware cost also can be higher, and if cooperate multiphoton microscope to use the present invention, the available depth of field is about 2 times of the independent use multiphoton microscope depth of field.

Summary of the invention

The present invention is the use that cooperates sample embedding gel, two image groups is done the image fit after being the positive and negative figure of sweeping, but the depth of field that increases the capture scope is to reach the method that improves the microscope image analytic degree.The present invention is main but be not limited to the stereopsis fit of biology microscope image.

The present invention is in order to obtain the darker stereopsis of thickness, be sample to be done figure is swept in the front and reverse side is swept figure, because sample is lived by sample embedding gel sets at three dimensions, so the obtained image in before the positive and negative figure of sweeping overturns and upset back only can occur in the translation on the XY plane, difference with the Z axle being the axle center rotation can not occur in the rotation at ψ angle in the three dimensions; After finding the lap position on the Z axle, get before the upset and each a slice image in the overlapped part after the upset, calculate before the upset and the obtained translational movement of image on the XY plane in upset back, with be the rotation amount in axle center with the Z axle, sweep the image group of figure as reference with the front again, adjusting reverse side and sweep the Z axle bed mark of figure image, the translational movement on XY plane, and is the rotation amount in axle center with the Z axle, and two groups of continuous images can be rebuild becomes one group of complete stereopsis.It determines that positive and negative sweep the lap position of figure image group on the Z axle, be to utilize the fast fourier conversion method to dwindle the scope that to seek the lap position on the Z axle, use the notion of Sobel rim detection, find out the zone of edge variation maximum in the image, utilize this zone to use the method for relevant matches again, judge reverse side and sweep in the figure, sweep the most similar image of image A chosen among the figure (its flow process as shown in Figure 8), to judge the lap position on the positive and negative figure of the sweeping Z axle to the front.In case the lap position of figure image group on the Z axle swept in the decision positive and negative, re-use the fast fourier conversion method and find out the translation of X, Y direction and the angle of rotating with the Z axle, to adjust the position of upper and lower layer image group, to make the image that draws is complete stereopsis.

The present invention is in order to obtain the darker stereopsis of thickness, is sample is done figure is swept in the front and reverse side is swept figure; As shown in Figure 2, wherein the thick line rectangle frame is for just, the anti-figure image group overlapping areas of sweeping, since sample by the embedding gel sets in three dimensions, just make, the obtained image in before the anti-figure of sweeping overturns and upset back only can occur in the translation on the XY plane, with the difference that with the Z axle is the axle center rotation, just utilize, the anti-figure of sweeping stereopsis overlapped part is found out the lap position on the Z axle, get that figure is swept in the front and reverse side is swept each a slice image in the figure overlapping region, figure is swept in the calculating front and reverse side is swept the translational movement of the obtained image of figure on the XY plane, with be the rotation amount in axle center with the Z axle, the image group of sweeping figure with the front is as reference, adjust reverse side and sweep the Z axle bed mark of figure image, the translational movement on XY plane, with be the rotation amount in axle center with the Z axle, so just can obtain one group of complete stereopsis.Whole microscope is positive and negative sweeps figure image fit system flowchart as shown in Figure 3.

Technology and the skill below introducing among the present invention to be used:

1, the fast fourier transformation applications is in the image fit

Using fast fourier to be transformed in the image fit is the phase relation of utilizing the conversion of Fu Li leaf.If two images only have the difference of translation, promptly as image f ₂Be f ₁Translation (x ₀, y ₀) the result, then two is image related suc as formula (1).

Formula (1) gets formula (2), wherein displacement relation (x between the two after doing the Fu Shi conversion ₀, y ₀),

f ₂(x，y)＝f ₁(x-x ₀，y-y ₀) (1)

$F_2(\mathrm{\ξ},\mathrm{\η})=e^{-j2\mathrm{\π}({\mathrm{\ξx}}_0+\mathrm{\η}{y}_0)}\×F_1(\mathrm{\ξ},\mathrm{\η})---\left(2\right)$

Only appear at phase term e ^{-j2 π (ξ x0+ η y0)}In.

Phase term can be calculated by formula (3), wherein F ^*Conjugate complex number for F.

With the phase term e in the formula (3) ^{-j2 π (ξ x0+ η y0)}After doing anti-Fu Shi conversion, by the result of calculation of anti-Fu's formula conversion, its pulse position can determine (x ₀, y ₀).Formula (4) is Fu Shi conversion and the right relation of anti-Fu Shi conversion.

$\frac{F_2(\mathrm{\ξ},\mathrm{\η}){F_1}^{*}(\mathrm{\ξ},\mathrm{\η})}{\left|F_2(\mathrm{\ξ},\mathrm{\η}){F_1}^{*}(\mathrm{\ξ},\mathrm{\η})\right|}=e^{-j2\mathrm{\π}({\mathrm{\ξx}}_0+{\mathrm{\ηy}}_0)}---\left(3\right)$

$\mathrm{\δ}(x-x_0,y-y_0)\⇔e^{-j2\mathrm{\π}({\mathrm{\ξx}}_0+{\mathrm{\ηy}}_0)}---\left(4\right)$

When two images rotate with translation simultaneously suc as formula (5).

f ₂(x，y)＝f ₁(xcosθ ₀+ysinθ ₀-x ₀，-xsinθ ₀+ycosθ ₀-y) (5)

After formula (5) is got fast fourier conversion, two images in the relativeness of frequency domain suc as formula (6).

$F_2(\mathrm{\ξ},\mathrm{\η})=e^{-j2\mathrm{\π}({\mathrm{\ξx}}_0+{\mathrm{\ηy}}_0)}\×---\left(6\right)$

$F_1({\mathrm{\ξ}\cos \mathrm{\θ}}_0+\mathrm{\η}{\sin \mathrm{\θ}}_0,-{\mathrm{\ξ}\sin \mathrm{\θ}}_0+{\mathrm{\η}\cos \mathrm{\θ}}_0)$

Formula (6) is got its amplitude M1, M2 as the formula (7), wherein M ₁, M ₂Be respectively F ₁, F ₂Amplitude.Observation type (7), M ₂Be M ₁Rotated θ ₀The result of degree is to M ₁, M ₂Get formula (8) after doing the polar coordinates conversion.

M ₂(ξ，η)＝M ₁(ξcosθ ₀+ηsinθ ₀，-ξsinθ ₀+ηcosθ ₀) (7)

M ₁(ρ，θ)＝M ₂(ρ，θ-θ ₀) (8)

Observation type (8) becomes the relation of translation between M1 and the M2, its translational movement is θ ₀, utilize the phase relation of above-mentioned Fu Li leaf conversion just can solve θ ₀Next f1 is rotated θ ₀After f " ₁, f " ₁With f ₂Between the relation of surplus translation only just, utilize the phase relation of Fu Li leaf conversion just can learn (x again ₀, y ₀), the process of whole execution please refer to shown in Figure 4.The high frequency that image is done after the Fu Shi conversion partly comprises the key character of original figure, thus before doing the polar coordinates conversion, add Hi-pass filter, the accuracy of raising angle assessment, its specification is suc as formula (9), wherein D ₀=0.3.

2.Sobel rim detection

The present invention uses the Sobel operator to do rim detection, after the Sobel operator is finished rim detection to image f, each pixel on the image f can obtain the gradient intensity size | f|, Fig. 5 (B) finishes gradient intensity image after the rim detection for the Sobel operator to Fig. 5 (A), its gradient intensity value big more (bright more on the image), representing this place is tangible more edge, it is the big place of the bright variation of image, the present invention utilizes the gradient intensity image after image f finishes rim detection, seeks the zone that has limbus to change in the image f.Set a suitably big or small block, seek among the gradient intensity figure of image f in block | the position of f| summation maximum, this position is zone such as the Fig. 5 (C) that has limbus to change in the image f, utilize this zone to serve as a model, highlight the otherness of all the other images to be compared and model room, use in order to the follow-up relevant matches of doing.

3. relevant matches method

(s t), can find out to model w the most similar zone in image f suc as formula the correlation coefficient r of (10) gained to utilize the method for relevant matches.W (x, y) model (template) the image f of M * N (x, right by a left side in y), from top to bottom mobile use formula (10) obtains r (s, t) related coefficient, wherein s=0,1,2 ... M-1, t=0,1,2, N-1, T are image f and model w overlapping areas, and promptly formula (2-1) is only carried out in the overlapping region, (x y) represents image f and model w overlapping areas average brightness to f, and w represents model w average brightness.

As Fig. 6, wherein f (x, initial point y) is positioned at the upper left corner, w (x, initial point y) is positioned at its center, is positioned at f (x, y) Nei any position (s, t) all use formula (10) obtain a correlation coefficient r (s, t), and the highest position of similarity numerical value promptly is f (x, y) in w (x, y) the most similar zone, correlation coefficient r (s, t) span be-1≤r (s, t)≤1.

$r(s,t)=\frac{\underset{\mathrm{xy}\∈T}{\mathrm{\Σ}}\mathrm{\Σ}[f(x,y)-\stackrel{\‾}{f}(x,y)][w(x-s,y-t)-\stackrel{\‾}{w}]}{{\left\{\underset{\mathrm{xy}\∈T}{\mathrm{\Σ}}\mathrm{\Σ}{[f(x,y)-\stackrel{\‾}{f}(x,y)]}^2\underset{\mathrm{xy}\∈T}{\mathrm{\Σ}}\mathrm{\Σ}{[w(x-s,y-t)-\stackrel{\‾}{w}]}^2\right\}}^{1/2}}---\left(10\right)$

4. the lap position of figure image group on the Z axle swept in definite positive and negative

Determine that the lap position on the Z axle is exactly to find identical image in the two image groups, to determine the position of overlapping region.The first step is to utilize above-mentioned fast fourier conversion method to calculate that resulting peak value (peak) makes a decision in the process of the anglec of rotation between two images, because when the difference between the image (for example: image A and image B fairly similar) of two fairly similars is rotation and translation, resulting peak value can be when changing image B into image C (image A and image C dissmilarity) peak value of doing identical computing gained to come much higherly, so reverse side sweep in the figure image group to the front sweep several more similar images of figure image A be positioned at the most outstanding peak value near, be positioned at the thick line rectangle frame as the most outstanding peak value of Fig. 7, (define these several images herein and be " image group K " near sweeping the 11st of figure so several images more similar to image A are positioned at reverse side, chat with convenient hereinafter stating), utilize this idea to find out lap position on the rough location Z axle, dwindle the image sheet number that to compare, to reduce the calculated amount of postorder.Yet these several images all might be that reverse side is swept the most similar image of image A of sweeping figure among the figure to the front.Calculate and sweep each image of image group K selected among the figure and the translational movement of image A on the XY plane and the angle of rotating from the negative with the Z axle, adjust the angle of its translational movement and rotation, utilize the notion of the above-mentioned Sobel rim detection of putting forward again, find out the zone of edge variation maximum in the image, utilize this zone to use the method for relevant matches, judge reverse side and sweep Tu Nei and the positive the most similar image of sweeping among the figure of image A, the lap position of figure image group on the Z axle swept in the decision positive and negative, and its process as shown in Figure 8.At last, after having located the lap position on the Z axle, get that figure is swept in the front and reverse side is swept each a slice image in the figure overlapping region, calculate and positively to sweep that figure and reverse side are swept the translational movement of the obtained image of figure on the XY plane and with the rotation amount of Z axle, the image group of sweeping figure with the front is as reference, adjusting reverse side and sweep the Z axle bed mark of figure image, the translational movement on XY plane, and is the rotation amount in axle center with the Z axle.

Description of drawings

Fig. 1 shows that the present invention utilizes laser to become single luminous point via object lens focusing, shines the certain depth of sample single-point;

Fig. 2 shows that the present invention does the front with sample and sweeps the synoptic diagram that figure and reverse side are swept figure;

Fig. 3 shows the positive and negative figure of the sweeping image of the whole microscope of the present invention fit system flowchart;

Fig. 4 is for showing that the present invention does the process that Fu Shi changes whole execution;

Fig. 5 (A) makees the image of rim detection for showing the present invention;

Fig. 5 (B) finishes gradient intensity image after the rim detection for showing the Sobel operator to Fig. 5 (A);

The zone of Fig. 5 (C) for showing that the Sobel operator is finished has limbus to change in the image f after the rim detection;

Fig. 6 utilizes the related coefficient of the method gained of relevant matches for showing the present invention, can find out to model w the most similar zone in image f;

Fig. 7 sweeps in the figure image group the position of outstanding peak value for showing the present invention at reverse side;

Fig. 8 shows that the present invention determines that positive and negative sweep the process flow diagram of the lap position of figure image group on the Z axle;

Fig. 9 sweeps the exterior view of figure for showing the biological specimen positive and negative;

Figure 10 is for showing the complete stereopsis after the positive and negative figure of sweeping of biological specimen does the image fit.

Embodiment

First embodiment

Fig. 9 sweeps the exterior view of figure for the biological specimen positive and negative, each of getting that the image in bottom of sweeping figure in the front and reverse side sweep figure is opened image, utilize the fast fourier conversion method to calculate that resulting peak value (peak) makes a decision in the process of the anglec of rotation between two images, find out the front and sweep image in bottom of figure and sweep the approximate location (as shown in Figure 7) of figure image group on the Z axle with respect to reverse side.Utilize the notion of the above-mentioned Sobel rim detection of putting forward again, find out edge variation is bigger in the image zone such as the thick line rectangle frame of Fig. 5 (C) and be the bigger zone of edge variation in the image.Utilize this zone to use the method for relevant matches, judge reverse side and sweep Tu Nei and the positive the most similar image of sweeping among the figure of image A.The lap position of figure image group on the Z axle swept in the decision positive and negative.Lap position flow process system on whole definite Z axle as shown in Figure 8.Behind the lap position of determining on the Z axle, get that figure is swept in the front and reverse side is swept each a slice image in the figure overlapping region, figure is swept in the calculating front and reverse side is swept the translational movement of the obtained image of figure on the XY plane, with the rotation amount (as shown in Figure 4) that with the Z axle is the axle center, sweep the image group of figure as reference with the front again, adjusting reverse side and sweep the Z axle bed mark of figure image, the translational movement on XY plane, and is the rotation amount in axle center with the Z axle, so just can obtain one group of complete stereopsis.Figure 10 positive and negative figure of sweeping of biological specimen for this reason does complete stereopsis behind the image fit, and its thickness is about the front and sweeps 2 times of figure.

Second embodiment

The about 160 μ m of the brain thickness of fruit bat indicate green fluorescent albumen with cranial nerve cell, and it can obtain complete 3D brain image to utilize 488nm laser deexcitation.But we can find clearly that the image of laser scanning gained is just fuzzy more to the bottom more.Its main cause is that biological specimen has light absorptive, and the energy of exciting light or the energy of radiating light are absorbed by sample, are very unclear and cause obtained stereopsis below the degree of depth at certain.The method of utilizing the present invention to carry only need be scanned up to the degree of depth just over half brain and obtain the front and sweep figure and reverse side and sweep and do the image fit after the clear stereopsis group of figure again and can obtain complete and 3D brain image clearly.

The 3rd embodiment

Generally doing the scanning of confocal micro-imaging is biological tissue's thin slice to be embedded in remove to carry out micro-imaging scanner uni record in the glycerine.Cooperate method of the present invention, biological tissue's sheet thickness of sweeping can be retouched and two times can be increased near original biological tissue thickness, promptly earlier biological tissue is used sample embedding gel sets behind three dimensions, utilize the confocal microscope to be scanned up to the degree of depth just over half brain and obtain the front and sweep figure and reverse side and sweep and do the image fit after the clear stereopsis group of figure again and can obtain complete and 3D biological tissue image clearly, its stereopsis thickness is about and generally is embedded in two times that carry out micro-imaging scanning gained image thickness in the glycerine.

The 4th embodiment

Generally doing the scanning of confocal micro-imaging is biological tissue's thin slice to be embedded in remove to carry out micro-imaging scanner uni record in the glycerine.Use the FocusClear clarification to organize to be embedded in MountClear again if change ^TMThen can deepen greatly to retouch the depth of field of sweeping.If cooperate method of the present invention again, the stereopsis thickness of gained can double again, is embedded in MountClear again for single FocusClear of cooperation clarifies to organize ^TMTwo times of method.

The 5th embodiment

For the image of obtaining the full brain of mouse must be done the brain that becomes mouse the vibrations section earlier, the thin slice that traditional practice is cut into 100-200 sheet 10-20 μ m for the brain of will about 4mm thick one-tenth mouse is embedded in and removes to carry out micro-imaging scanner uni record in the glycerine.Be embedded in MountClear again if use the FocusClear clarification to organize ^TM, then can scan the sheet of about 200 μ m.If utilize method of the present invention can obtain then that 400 μ m sheets are in-house knows image again, a complete one-tenth mouse brain then only need be cut into about 10 400 thick tissue of μ m, so utilize method of the present invention not only can effectively increase the depth of field of stereopsis, and can effectively reduce number of slices as the full brain that becomes mouse in the big biological tissue of thickness.

The 6th embodiment

Cooperate the use of multiphoton microscope, the method for utilizing the present invention to carry is utilized multiphoton microscope to do the front earlier and is swept figure and sweep figure with reverse side and do the image fit again, and the thickness (depth of field) that can obtain stereopsis is about the twice of single use multiphoton microscope.That is to say that an about 4mm of complete one-tenth mouse brain only need be cut into the sheet of 5 800 μ m, is embedded in MountClear ^TMIn, positive and negative utilize the present invention to do the image fit after sweeping figure again, can obtain the micro-imaging of complete brain.

Claims (5)

1, a kind of biological microscope image technology reaches the method that increases by the 3 dimension depth of field and resolution, it is characterized in that, be to scan one or more groups 3 dimension image more from the negative from one or more groups 3 dimension image of sample front scan, cooperate X, the translation of Y direction, the rotation of Z axle and, improve people's living condition the fabric texture number of slices and increase the biological image depth of field and resolution to reach to reduce with two or more sets one group of synthetic complete 3 dimension image technology that double the degree of depth and large volume of 3 dimension images covers of positive counter-scanning.

2, biological microscope image technology according to claim 1 reaches the method that increases by the 3 dimension depth of field and resolution, it is characterized in that, it is the sample that will be fixed on behind the three dimensions, respectively from just, reverse side scans, integrate mathematical methods such as the known fast fourier conversion of utilization then, just judge, the anti-figure gained image group of sweeping is behind the lap position on the Z axle, the image group of sweeping figure with the front is as reference, adjust reverse side and sweep the Z axle bed mark of figure image, the translational movement on XY plane, with be the rotation amount in axle center with the Z axle, about the merging, about and positive and negative image, and merge the fit method heavily cover image, to obtain the darker and complete microscope stereopsis of the depth of field.

3, biological microscope image technology according to claim 1 reaches the method that increases by the 3 dimension depth of field and resolution, it is characterized in that it can use single or the capture of multi-wavelength excitation light auxiliary microscope, to improve the resolution of stereopsis.

4, biological microscope image technology according to claim 1 reaches the method that increases by the 3 dimension depth of field and resolution, it is characterized in that, its multiphoton microscope of can arranging in pairs or groups uses to improve the resolution of its stereopsis.

5, biological microscope image technology according to claim 1 reaches the method that increases by the 3 dimension depth of field and resolution, it is characterized in that, it can be applied to but be not limited to the combination of confocal micro-imaging and reach and increase the thick stitch clarity and the degree of depth.

Images