WO2005031645A1 - Enhancement of spatial resolution of imaging systems by means of defocus - Google Patents

Enhancement of spatial resolution of imaging systems by means of defocus Download PDF

Info

Publication number
WO2005031645A1
WO2005031645A1 PCT/AU2004/001342 AU2004001342W WO2005031645A1 WO 2005031645 A1 WO2005031645 A1 WO 2005031645A1 AU 2004001342 W AU2004001342 W AU 2004001342W WO 2005031645 A1 WO2005031645 A1 WO 2005031645A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
blurred
deblurred
defocused
deconvolution
Prior art date
Application number
PCT/AU2004/001342
Other languages
French (fr)
Inventor
Timur Eugenievich Gureyev
Andrew Wesley Stevenson
Original Assignee
Commonwealth Scientific And Industrial Research Organisation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2003905370A external-priority patent/AU2003905370A0/en
Application filed by Commonwealth Scientific And Industrial Research Organisation filed Critical Commonwealth Scientific And Industrial Research Organisation
Publication of WO2005031645A1 publication Critical patent/WO2005031645A1/en

Links

Classifications

    • G06T5/73

Definitions

  • the present invention relates to enhancement of spatial resolution of an imaging system by means of defocus particularly, but not exclusively, for use in imaging and lithography.
  • the first class includes 'incoherent aberrations' such as the blurring of an image due to the finite size of the source of light ('penumbral blurring') and finite detector resolution.
  • the second type includes 'coherent aberrations' such as e.g. the Fresnel diffraction effects ('diffraction fringes'). Both classes of aberrations usually degrade the performance of the optical system and, in particular, degrade the spatial resolution.
  • the present invention proposes that in certain cases it is possible to make the two types of aberrations at least partially cancel each other's effect and thereby provide a means for improving the overall performance of an optical system.
  • the present invention considers the specific problem of deconvolution of a noisy image given complete or partial knowledge of a point-spread function (PSF) of the imaging system.
  • PSF point-spread function
  • Such a problem is encountered in a variety of scientific and industrial applications, e.g. in imaging using visible light, electrons, X-rays, etc (see e.g. [1]).
  • the deconvolution problem is a difficult one due to its mathematical ill- posedness [2].
  • the latter means in particular that small errors (e.g. noise) in the measured data may result in strong artefacts in the reconstructed (deconvolved) image.
  • a large number of methods have been developed over the years that implement various regularization techniques for more robust deconvolution of noisy data [2, 3].
  • Equation (6) reflects the simple fact that at the level of second-order Taylor approximations, the changes in an image due to convolution with an arbitrary PSF are proportional to the width of the PSF and to the local curvature of the ideal image [5] (for example, it is easy to see that image regions with zero curvature do not change as a result of convolution, apart from possible trivial shift and multiplication by a constant factor).
  • eq.(6) may not provide an exact deconvolution, it possesses two important advantages over some more conventional deconvolution formulae (such as e.g. Wiener deconvolution). Firstly, eq.(6) does not require the precise knowledge of the PSF, but only its width.
  • eq.(6) is 'local' in the sense that the value of the deconvolved image at a point (x, y) depends only on the values of the blurred image and its second derivatives at the same point. These two properties can be particularly valuable in the cases where the knowledge of the PSF and/or of the blurred image is incomplete.
  • Deconvolution techniques based on eq.(6) have been extensively studied in [5]. We use eq.(6) as a starting point for the development of a new deconvolution methodology.
  • a method of deconvolution for producing a deblurred image of an object from an imaging system which produces penumbral and other instrumental blurring using quantitative optimisation of counteractive affects of Fresnel diffraction and the penumbral blurring.
  • the present invention provides a method of deconvolution applied to images affected by penumbral and other instrumental blurring, wherein a deblurred image is obtained by a linear combination of a defocused image and the original in-focus blurred image.
  • the deblurred image is obtained by means of a suitable numerical processing of a single slightly defocused image.
  • the deblurred image is acquired directly (no numerical processing is required for the deblurring) as a defocused image at the "special defocus distance".
  • the special defocus distance is provided in terms of a function of the PSF properties, wavelength of the radiation and the materials of the sample.
  • the above-described methodology improves the performance (spatial resolution) of imaging systems by means of at least partial compensation of the inherent image blurring due to the point-spread function of the system.
  • the method is useful in imaging but may also have application to enhancing spatial resolution in optical, electron or X-ray lithography.
  • Figure 1 is an 'Ideal' 256x256 pixel image used in numerical experiments
  • Figure 2 is a blurred image obtained by the convolution of the 'ideal' image from Figure 1 with a 3x3 pixel wide Gaussian PSF and small amount (1%) of added Gaussian noise;
  • Figure 3 is a deblurred image obtained using numerical (software) deconvolution from data in Figure 2;
  • Figure 4 is a deblurred image obtained from a single noisy blurred defocused image;
  • Figure 5(a). is an experimental in-line X-ray image of an edge of a Polyethylene sheet
  • Figure 5(b). is a cross-section of image intensity through the edge of the sheet
  • Figure 6(a) is a reconstructed distribution of the projected thickness of Polyethylene
  • Figure 6(b) is a cross-section of the reconstructed distribution of the projected thickness of Polyethylene
  • Figure 7(a) is a deblurred distribution of the projected thickness of Polyethylene obtained in accordance with eq.(2);
  • Figure 7(b) is a cross-section of the deblurred thickness distribution through the edge of the sheet.
  • I z (x,y) I(x,y) -a l(x,y) , (7)
  • n l- ⁇ -i ⁇ is the complex refractive index of the material.
  • D z I Z *P
  • the operator * denotes convolution
  • the blurred image D(x, y) is defined by eq.(l) and we assumed that the difference between the PSFs in the two planes is negligible, i.e. ⁇ I Z * (P Z -P) ⁇ « ⁇ I Z *P ⁇ .
  • eqs.(7) and (8), as presented above, are valid in the case of a quasi-plane incident wave. Their generalization to the case of a quasi-spherical incident wave is quite straightforward and can be found e.g. in [6].
  • Equations (7) and (8) show that the defocused image of a single-material object represents a linear combination of the in-focus image and its Laplacian. Isolating V 2 D(x,y) in eq.(8) and substituting it into eq.(6) we obtain
  • a deblurred in-focus image can be obtained from a single defocused image using the 'phase retrieval' procedure, i.e. solving eq.(8) for the in-focus image:
  • I(x,y) [(b 0 - b 2 la)(l - a ⁇ y l + b 2 l ]D z (x,y) . (11)
  • Equation (11) allows one to obtain a deblurred in-focus image from a single defocused image. It can be demonstrated that the main software component of the deblurring method described by eq.(l l), i.e. the application of the operator [1 - ⁇ V 2 ] -1 to the blurred defocused image, is very stable with respect to image noise. It can be computed using fast and efficient numerical algorithms, such as the Fast Fourier transform. The robustness of this deblurring method can be understood as a consequence of the fact that coherent defocused images have more spectral energy (and therefore better signal-to-noise ratio) in high-order Fourier components compared to the in-focus images.
  • a b 2 /b 0 .
  • This condition describes a specific match between the parameters of the imaging system (represented by the width of the Gaussian PSF, radiation wavelength and the defocus distance) and the fundamental characteristic ( ⁇ / ⁇ ) of the complex refractive index of the sample that is required for the optimal deblurring by means of defocus.
  • eq.(16) is more sensitive to low-frequency noise in the experimental data, and, therefore, is less stable than eq.(ll).
  • eq.(16) contains an additional term, b 2 /b 0 , which takes into account the blurring of the phase-contrast image due to the PSF of the imaging system.
  • the relative RMS error of the resultant image with respect to the ideal image from Fig.l was equal to 1.6 % (compared to 2.1 % for the noise-free blurred image). Note that in this noise-free case one can still achieve a much better result by using a simple Wiener deconvolution [5]. However, when we applied eq.(6) to the blurred image with noise (from Fig.2), the result (shown in Fig.3) turned out to be very noisy. The relative RMS error of the image in Fig.3 was 6 % compared to only 2.3 % for the raw noisy blurred image (Fig.2).
  • Fig.5(b) which displays a cross-sectional profile through the edge in Fig.5(a) averaged over the height of the image.
  • the source size in the direction orthogonal to the edge was equal to ⁇ 7 micron, which determined the approximate width of the PSF in this direction.
  • Figure 6(a) unlike Fig.5(a), displays the expected projected thickness profile of the sample blurred by the PSF.
  • eq.(l 1) to the image in Fig.5(a) with the result shown in Fig.7(a).
  • Fig.6(a) but the edge in Fig.7(a) is much sharper than in Fig.6(a) confirming the presence of a clear deblurring effect due to the partial compensation of the PSF-induced spread.
  • the general nature of the deblurring effect can be easily visualized by considering the profile in Fig.7(b) to be a weighted sum of the profiles from Fig.5(b) and 6(b). The compensation of PSF-induced blurring by the Fresnel diffraction effect is evident.
  • the spatial resolution estimated from Fig.7(b) is equal to ⁇ 5 micron, which is smaller than the source size.
  • the method combines in a non-trivial way a novel method for image deconvolution as described in [6], with the TIE approximation for the Fresnel diffraction.
  • the method allows one to improve the performance of some imaging systems without modifying the imaging hardware.
  • the method utilises a new mathematical deconvolution technique which to our knowledge has not been known previously, and combines it with a particular description of Fresnel diffraction based on the TIE.
  • the combination of the two special methods and their application to the problem of quantitative image deblurring is non-trivial and new. It allows one to overcome a well-known and very difficult problem existing in the image deblurring field, namely the sensitivity of deconvolution methods to noise in the image data. This sensitivity is overcome in the proposed new method by performing the deconvolution partially in hardware before the statistical noise affects the collected image data.
  • Another valuable feature of the method is in its ability to operate successfully given only very limited amount of information about the point-spread function of the imaging system, namely just its average width. A single defocused image will in some cases be sufficient for the application of the method. Finally, the proposed method is quantitative in nature and at the same time allows, unlike many popular deconvolution methods, for very efficient and rapid implementation, so it can be used 'on-line' and in realtime regimes.
  • the abovedescribed method has many applications in a variety of imaging systems and in lithography. So far, successful performance of the method under the conditions of X-ray 'in-line' imaging of 'homogeneous' objects has been demonstrated. However, many other possible modifications and applications of the method are envisaged. For example, the method may be applied to imaging systems using different forms of radiation (including visible light as used in photographic cameras, microscopes, etc) and matter waves (e.g. electron microscopy). Applicability of the proposed method to imaging using reflected (rather than transmitted) radiation is another possibility (e.g. in optical photography). Other potential applications include confocal microscopy and tomography.
  • the method can be viewed as a hardware (or combined software- hardware) implementation of second-order deconvolution which relies on the knowledge of the width of the PSF.
  • the proposed method is quantitative in nature and should be viewed as a special variant of image deconvolution.
  • the method is quite different from most conventional techniques for image sharpening as it is based on the actual physics of image formation and does not arbitrarily change the true information content of the image.
  • the invention also relates to an apparatus for performing one of the deconvolution methods of the invention, including imaging system hardware and/or a computer product with computer readable program code for performing the method.

Abstract

A new method for rapid deblurring (partial deconvolution) is proposed which is based on the physics of coherent image formation. The method uses slightly out-of-focus images in a way that makes the effects of Fresnel diffraction counteract the blurring due to the point-spread function. The resultant deblurring improves the spatial resolution in the images and is comparatively insensitive to noise.

Description

ENHANCEMENT OF SPATIAL RESOLUTION OF IMAGING SYSTEMS BY MEANS OF DEFOCUS
Field of the Invention
The present invention relates to enhancement of spatial resolution of an imaging system by means of defocus particularly, but not exclusively, for use in imaging and lithography.
Background of the Invention
Virtually every imaging system suffers to some extent from two classes of aberrations. The first class includes 'incoherent aberrations' such as the blurring of an image due to the finite size of the source of light ('penumbral blurring') and finite detector resolution. The second type includes 'coherent aberrations' such as e.g. the Fresnel diffraction effects ('diffraction fringes'). Both classes of aberrations usually degrade the performance of the optical system and, in particular, degrade the spatial resolution. The present invention proposes that in certain cases it is possible to make the two types of aberrations at least partially cancel each other's effect and thereby provide a means for improving the overall performance of an optical system. The general concept of using hardware solutions to reduce optical aberrations has previously successfully been applied in, for example, construction of achromatic optical systems by combining optical elements with opposite chromatic aberrations. A similar approach also led to the definition and use of Scherzer defocus in electron microscopy [4, page 298].
The present invention considers the specific problem of deconvolution of a noisy image given complete or partial knowledge of a point-spread function (PSF) of the imaging system. Such a problem is encountered in a variety of scientific and industrial applications, e.g. in imaging using visible light, electrons, X-rays, etc (see e.g. [1]). It is also known that the deconvolution problem is a difficult one due to its mathematical ill- posedness [2]. The latter means in particular that small errors (e.g. noise) in the measured data may result in strong artefacts in the reconstructed (deconvolved) image. A large number of methods have been developed over the years that implement various regularization techniques for more robust deconvolution of noisy data [2, 3]. It has been demonstrated that accurate deconvolution can be achieved in the presence of suitable additional (a priori) information that helps to constrain the set of all possible solutions, thus eliminating some reconstruction artefacts [3]. However, deconvolution methods that 5 make effective use of a priori knowledge are often iterative, computationally intensive and may occasionally diverge. Therefore, a need still exists for rapid, simple and robust methods capable of providing at least partial deconvolution of isy images, particularly when the PSF is not known precisely.
iv, We have recently suggested a method for describing the effect of image blurring due to the finite point-spread function (PSF) of an optical system in the form of a Taylor series with the coefficients proportional to the integral moments of the PSF. That approach allowed us to prove in particular that in the case of symmetric PSFs, deblurring of an image can be achieved by subtracting from the blurred image its appropriately weighted Laplacian. The
15 required Laplacian of the image can in principle be calculated numerically, but this operation is known to be very sensitive to any measurement noise present in the image, as demonstrated below:
Let the measured image, D(x, y) , be described by the convolution of the 'ideal' image, 0 I(x, y) , (corresponding to the delta-function PSF) with the actual PSF of the imaging system, P(x,y):
D(x,y) = l(x - x',y - y')P(x', y')ώc'dy' . (1) 5 In reality, the measured image usually contains some noise (e.g. due to the photon counting statistics). Therefore, any useful deconvolution method, i.e. a method for finding the 'ideal' image I(x, y) from eq.(l), must be able to cope with noisy input data.
Following [5], let us expand the ideal signal, I(x-x y-y'), in eq.(l) into the Taylor series at 0 point (x, y) and exchange the order of integration and summation: D(x,y) = ∑ mndm xd;i(x,y) ,, (2) m=0 n=0
where the coefficients amn are proportional to the moments of the PSF,
Figure imgf000004_0001
Here we have assumed that the function I(x, y) is infinitely differentiable, the PSF has finite moments of all orders and the series eq.(2) converge with respect to a chosen functional metric. We then construct a solution to eq.(l) in the form of a formal series (Ansatz),
I(x,y) = ∑∑bkldx kd! yD(x,y) (4) k=0 1=0
with unknown coefficients bu- Substituting eq.(2) into eq.(4) and equating the coefficients of the partial derivatives of the ideal signal I, we obtain the following recursive expressions for coefficients b t,
"kl
Figure imgf000004_0002
where k, I = 0, 1, 2, ..., and < is the Kronecker delta. Sufficient conditions for the convergence of the formal series eq.(4) with coefficients defined by eq.(5) can be found in [5]. We are interested primarily in a second-order approximation that can be derived from the general deconvolution procedure or formula defined by eqs.(4)-(5). In what follows, we consider PSFs with their 'centre of gravity' coinciding with the origin of coordinates (this can always be achieved by an appropriate shift of the origin of coordinates). Then the first-order moments of the PSF are equal to zero, i.e. aw = a0l = 0. We also consider at first only isotropic PSFs with their second-order moments satisfying the relationships a20 = a02 ≡ a2 and an = 0 . h this case, as it follows from eqs.(4)-(5), the second-order partial deconvolution can be expressed by a simple formula
I(x, y) = b0D(x, y) - b2ΨD(x, y) , (6)
where V ≡ dx + dy , b0 = l/a00 and b2 = a2 l a0Q . This formula states that for any symmetric PSF, a partial deconvolution (deblurring) of an image can be achieved by subtracting an appropriately weighted Laplacian of the original (blurred) image from the image itself. General sufficient conditions for the validity (accuracy) of eq.(6) have been presented in [5]. In the important special case of an isotropic Gaussian PSF P(x, y) with standard deviation σ these validity conditions can be expressed in a simple form as the requirement \ dx 2k dy'D(x) |< C(l/σ)2(t+,) , that should hold for all integers /ς/=l, 2, 3, ..., and some constant C. In other words, the characteristic length of variation of the measured image intensity should be larger than σ . Equation (6) reflects the simple fact that at the level of second-order Taylor approximations, the changes in an image due to convolution with an arbitrary PSF are proportional to the width of the PSF and to the local curvature of the ideal image [5] (for example, it is easy to see that image regions with zero curvature do not change as a result of convolution, apart from possible trivial shift and multiplication by a constant factor). Although eq.(6) may not provide an exact deconvolution, it possesses two important advantages over some more conventional deconvolution formulae (such as e.g. Wiener deconvolution). Firstly, eq.(6) does not require the precise knowledge of the PSF, but only its width. Secondly, unlike the general deconvolution expressions, eq.(6) is 'local' in the sense that the value of the deconvolved image at a point (x, y) depends only on the values of the blurred image and its second derivatives at the same point. These two properties can be particularly valuable in the cases where the knowledge of the PSF and/or of the blurred image is incomplete. The inherent ill-posedness of the deconvolution problem, eq.(l), appears in eq.(6) in the form of the instability of numerical differentiation required for evaluations of the image Laplacian. Deconvolution techniques based on eq.(6) have been extensively studied in [5]. We use eq.(6) as a starting point for the development of a new deconvolution methodology.
Summary of the Invention
In accordance with the invention, there is provided a method of deconvolution for producing a deblurred image of an object from an imaging system which produces penumbral and other instrumental blurring, using quantitative optimisation of counteractive affects of Fresnel diffraction and the penumbral blurring.
In one particular aspect, the present invention provides a method of deconvolution applied to images affected by penumbral and other instrumental blurring, wherein a deblurred image is obtained by a linear combination of a defocused image and the original in-focus blurred image. In a second preferred aspect of the invention the deblurred image is obtained by means of a suitable numerical processing of a single slightly defocused image. In a third aspect of the invention the deblurred image is acquired directly (no numerical processing is required for the deblurring) as a defocused image at the "special defocus distance". The special defocus distance is provided in terms of a function of the PSF properties, wavelength of the radiation and the materials of the sample.
The above-described methodology improves the performance (spatial resolution) of imaging systems by means of at least partial compensation of the inherent image blurring due to the point-spread function of the system. The method is useful in imaging but may also have application to enhancing spatial resolution in optical, electron or X-ray lithography.
In yet other aspects, there is provided an apparatus and/or a computer product for performing one of the above-described methods and an image produced by one of the methods. Brief Description of the Drawings
The invention is described in more detail with reference to the accompanying drawings, in which:
Figure 1 is an 'Ideal' 256x256 pixel image used in numerical experiments;
Figure 2 is a blurred image obtained by the convolution of the 'ideal' image from Figure 1 with a 3x3 pixel wide Gaussian PSF and small amount (1%) of added Gaussian noise;
Figure 3 is a deblurred image obtained using numerical (software) deconvolution from data in Figure 2; Figure 4 is a deblurred image obtained from a single noisy blurred defocused image;
Figure 5(a). is an experimental in-line X-ray image of an edge of a Polyethylene sheet;
Figure 5(b). is a cross-section of image intensity through the edge of the sheet;
Figure 6(a) is a reconstructed distribution of the projected thickness of Polyethylene;
Figure 6(b) is a cross-section of the reconstructed distribution of the projected thickness of Polyethylene;
Figure 7(a) is a deblurred distribution of the projected thickness of Polyethylene obtained in accordance with eq.(2); and
Figure 7(b) is a cross-section of the deblurred thickness distribution through the edge of the sheet.
Detailed Description
In order to derive a method of deconvolution according to the present invention, we firstly consider a special case of an object consisting predominantly of a single material [6]. Let such an object be illuminated by a monochromatic plane wave with wavenumber h=2π/λ (λ is the wavelength) propagating along the optic axis z. We denote by I(x, y) the distribution of transmitted intensity in the plane z=0 immediately after the object. The intensity distribution in a slightly 'defocused' image, Iz(x, y) (z is the defocus distance) can be described by the special form of the Transport of Intensity equation (TIE) [6],
Iz(x,y) = I(x,y) -a l(x,y) , (7)
where a = zδ l(2kβ) and n = l-δ -iβ is the complex refractive index of the material.
The region of validity of eq.(7) is approximately defined by the inequality < h2 , where h is the size of the smallest resolvable feature present in the in-focus image. In the presence of a non-trivial PSF eq.(7) changes to
Dz (x, y) = D(x, y) - aΨD(x, y) , (8)
where Dz = IZ *P , the operator * denotes convolution, the blurred image D(x, y) is defined by eq.(l) and we assumed that the difference between the PSFs in the two planes is negligible, i.e. \ IZ * (PZ -P) \ «\ IZ *P \ . Note that eqs.(7) and (8), as presented above, are valid in the case of a quasi-plane incident wave. Their generalization to the case of a quasi-spherical incident wave is quite straightforward and can be found e.g. in [6]. hi the latter case, however, geometrical expansion associated with free-space propagation needs to be taken into account not only with respect to the image Dz , but also for the PSF. In other words, all functions defined on the 'defocused' plane z have to be rescaled in accordance with magnification relative to the 'focal' plane z=0. This conventional procedure has been applied in the experimental example presented later.
Equations (7) and (8) show that the defocused image of a single-material object represents a linear combination of the in-focus image and its Laplacian. Isolating V2D(x,y) in eq.(8) and substituting it into eq.(6) we obtain
I(x, y) = (b0 - b2 1 a)D(x, y) + (b2 1 a)Dz (x, y) . (9) We see that deblurring of an image of a single-material object can be achieved by adding a properly weighted defocused image to the original blurred one. Such 'hardware' deblurring does not suffer from the numerical instability intrinsic to eq.(6), as the Laplacian of the blurred image is now obtained through hardware by means of defocus.
Furthermore, a deblurred in-focus image can be obtained from a single defocused image using the 'phase retrieval' procedure, i.e. solving eq.(8) for the in-focus image:
D(x,y) = [l- V2T1Dz(x,y) . (10)
Substituting eq.(lθ) into eq.(9) we derive
I(x,y) = [(b0 - b2 la)(l - aΨyl + b2 l ]Dz(x,y) . (11)
Equation (11) allows one to obtain a deblurred in-focus image from a single defocused image. It can be demonstrated that the main software component of the deblurring method described by eq.(l l), i.e. the application of the operator [1 - αV2]-1 to the blurred defocused image, is very stable with respect to image noise. It can be computed using fast and efficient numerical algorithms, such as the Fast Fourier transform. The robustness of this deblurring method can be understood as a consequence of the fact that coherent defocused images have more spectral energy (and therefore better signal-to-noise ratio) in high-order Fourier components compared to the in-focus images.
Finally, the phase retrieval procedure can be made redundant if the defocus distance is chosen in such a way that a = b2 /b0. In this case the first of the two additive terms on the right-hand side of eq.(9) disappears, and we arrive at a simple equation:
I(x, y) = b0Dz (x, y) , when z = zd ≡ 2kβa2 l(δa) . (12) Therefore, the blurred defocused image at the defocus distance zd is approximately equal (apart from an overall multiplicative constant) to the deblurred in-focus image. The deblurring is achieved because at the defocus distance zd the Fresnel diffraction effects optimally compensate the blurring due to the PSF. This is analogous to Scherzer defocus defined in electron microscopy as a distance at which the defocus optimally compensates the spherical aberration of an electron microscope [4]. In the case of an isotropic Gaussian PSF with standard deviation σ, one obtains a2 1 a = σ2 12 and, hence, zd G ss - kσ2 β I δ .
Introducing the Fresnel number, NF ≡ kσ2 1 z, corresponding to the feature size σ, we see that the optimal defocus distance zd corresponds to the condition
NF = δ lβ . (13)
This condition describes a specific match between the parameters of the imaging system (represented by the width of the Gaussian PSF, radiation wavelength and the defocus distance) and the fundamental characteristic (δ / β) of the complex refractive index of the sample that is required for the optimal deblurring by means of defocus.
So far the deblurring method based on eqs.(9), (11) or (12) has been developed only for monochromatic incident radiation (with wavenumber k), isotropic PSFs (for which a20 = a02 ) and transmission images of objects consisting of a single material (it is actually sufficient that the ratio of real and imaginary parts of the projected complex refractive index of the object, n(x, y) ≡ \n(x,y,z)dz , is the same for all x and y, i.e.
Ken(x,y)/Im.n(x,y) = constant [6]). If polychromatic incident radiation is used, the deblurring method can still be applied with the refractive index averaged with respect to the spectral distribution of the incident radiation [7]. In the case of anisotropic PSFs partial isotropic deblurring can still be achieved with a2 ≡ (a20 + a02)/2. Alternatively, anisotropic deblurring can be achieved in a preferred direction, e.g. in the x-direction when one sets a2 ≡ a20 . The experimental test results presented below were obtained using a polychromatic spatially anisotropic source. A similar procedure of 'hardware' deblurring can be applied to slightly defocused images of pure phase (non-absorbing) samples. In this case I(x, y) = I0 = constant ,
D(x, y) = D0 = I0a , and instead of eq.(8) we have:
Dz(x, y) = D0[l - b0(z /k)V2φ(x, y) *P(x,y)] , (14)
where φ(x, y) is the phase distribution in the transmitted beam in the object plane, z=0 [4, page 61]. Solving eq.(14) for φ * P , (φ *P)(x, y) = k/(zb0) V~2[l - Dz(x,y)f D0] and substituting the result into the second-order deconvolution formula for the phase (a direct analogue of eq.(6)),
φ(x,y) ≤ bQ(φ *P)(x, y) - b2 (φ *P)(x,y) , (15)
we derive the following analogue of eq.(ll) which allows one to obtain the deblurred phase distribution in the plane z=0 from a single blurred defocused image:
φ(x,y) = (k lz)(b2 /b0 - V-2)[Dz(x,y)/D0 - l] . (16)
We should note that eq.(16) is more sensitive to low-frequency noise in the experimental data, and, therefore, is less stable than eq.(ll). Compared to the usual TIE-based phase retrieval formula for a pure phase object [8, eq.(7)], eq.(16) contains an additional term, b2 /b0 , which takes into account the blurring of the phase-contrast image due to the PSF of the imaging system.
We tested the deblurring methods based on eqs.(6), (9), (11) and (12) using 256x256 pixel 'Lena' image shown in Fig.l. Figure 2 depicts a blurred image obtained by the convolution of the 'ideal' image from Fig.l with a 3x3 pixel wide Gaussian PSF and with 1% pseudorandom Poisson noise. Before attempting to deconvolve the noisy image from Fig.2, we first calculated the Laplacian of the noise-free blurred image and added it with an appropriate weight to the blurred image itself to obtain a deblurred image in accordance with eq.(6). The relative RMS error of the resultant image with respect to the ideal image from Fig.l was equal to 1.6 % (compared to 2.1 % for the noise-free blurred image). Note that in this noise-free case one can still achieve a much better result by using a simple Wiener deconvolution [5]. However, when we applied eq.(6) to the blurred image with noise (from Fig.2), the result (shown in Fig.3) turned out to be very noisy. The relative RMS error of the image in Fig.3 was 6 % compared to only 2.3 % for the raw noisy blurred image (Fig.2). This demonstrates the desirability of collecting the data of the Laplacian experimentally in order to avoid the noise-amplification effect inherent in numerical differentiation involved in eq.(6). We have verified the robustness of the proposed deblurring procedure by applying eqs.(9), (11) and (12) for deblurring of the noisy image from Fig.2. For this purpose we assumed that the images had the physical size of 256x256 micron, the wavelength was λ=0.1 nm and δ/β=10 (these conditions correspond to the case of hard X-ray microscopy). We calculated the blurred noisy defocused images at 2=l cm for use with eq.(9) and (11), and at the 'optimal' propagation distance z <3auss = ι 4 cm for eq.(i2). We convolved all the defocused images with the 3x3 pixel wide Gaussian PSF and 'added' 1% Poisson noise to simulate realistic blurred noisy images. The relative RMS error of the deblurred images obtained from these blurred noisy images using eqs.(9), (11) and (12) was found to be 2.1%. 2.0% and 1.8%, respectively, i.e. substantially better than the result obtained above with eq.(6). The deblurred image obtained using eq.(12) is presented in Fig.4. This image clearly displays some fine details not visible in the blurred image, Fig.2. The image in Fig.4 is also much less noisy compared to Fig.3 confirming the robustness of the proposed deblurring technique. We have further tested the stability of deconvolution using eq.(12) by applying it to images with different levels of noise. For this purpose we 'added' 1%, 3%, 5% and 10% of Poisson noise to the previously simulated blurred defocused image at the 'optimal' propagation distance zd""ss = 1.4 cm. The relative RMS error between these noisy blurred defocused images and the 'ideal' image from Fig.l was found to be equal to 1.8%, 3.4%, 5.2% and 10.1% respectively, which confirms the excellent stability properties of the proposed 'deblur by defocus' procedure. We have also tested the deblurring method based on eq.(ll) using an experimental in-line X-ray image of an edge of an approximately 100 micron thick Polyethylene sheet (Fig.5(a)) collected using a laboratory microfocus source with a tungsten target operated at 30keN. The sample was located at a distance of 4 cm from the source, with the detector located 196 cm from the sample. The spatial resolution of the detector referred to the object plane was ~2 micron. One can see typical phase-contrast effect (Fresnel fringe) near the edge of the Polyethylene sheet in Fig.5(b) which displays a cross-sectional profile through the edge in Fig.5(a) averaged over the height of the image. The source size in the direction orthogonal to the edge was equal to ~7 micron, which determined the approximate width of the PSF in this direction. For comparison, we present in Fig.6(a) a 'phase-retrieved' version of the image from Fig.5(a) obtained using eq.(lθ) with the wavelength 2=0.09 nm (average for the incident X-ray spectrum). Figure 6(a), unlike Fig.5(a), displays the expected projected thickness profile of the sample blurred by the PSF. We also applied eq.(l 1) to the image in Fig.5(a) with the result shown in Fig.7(a). One can see that the latter image still displays the correct projected density profile as in
Fig.6(a), but the edge in Fig.7(a) is much sharper than in Fig.6(a) confirming the presence of a clear deblurring effect due to the partial compensation of the PSF-induced spread. The general nature of the deblurring effect can be easily visualized by considering the profile in Fig.7(b) to be a weighted sum of the profiles from Fig.5(b) and 6(b). The compensation of PSF-induced blurring by the Fresnel diffraction effect is evident. The spatial resolution estimated from Fig.7(b) is equal to ~5 micron, which is smaller than the source size. Note that this deblurring technique worked quite well despite the experimental image being rather noisy (noise level was ~7.6 %). In fact, as it can be seen from a comparison of Fig.5 with Fig.7, the deblurred image is less noisy (~7.1 %) than the original one, which is quite a remarkable result for a deconvolution technique.
Accordingly, the method combines in a non-trivial way a novel method for image deconvolution as described in [6], with the TIE approximation for the Fresnel diffraction.
Although it has been known for some time that defocused images collected using incident illumination with high spatial coherence may help to reveal edges and interfaces via improved contrast, to our knowledge no one has yet attempted to use this effect to quantitatively cancel the blurring due to the PSF.
The method allows one to improve the performance of some imaging systems without modifying the imaging hardware. The method utilises a new mathematical deconvolution technique which to our knowledge has not been known previously, and combines it with a particular description of Fresnel diffraction based on the TIE. The combination of the two special methods and their application to the problem of quantitative image deblurring is non-trivial and new. It allows one to overcome a well-known and very difficult problem existing in the image deblurring field, namely the sensitivity of deconvolution methods to noise in the image data. This sensitivity is overcome in the proposed new method by performing the deconvolution partially in hardware before the statistical noise affects the collected image data. Another valuable feature of the method is in its ability to operate successfully given only very limited amount of information about the point-spread function of the imaging system, namely just its average width. A single defocused image will in some cases be sufficient for the application of the method. Finally, the proposed method is quantitative in nature and at the same time allows, unlike many popular deconvolution methods, for very efficient and rapid implementation, so it can be used 'on-line' and in realtime regimes.
As may be appreciated, the abovedescribed method has many applications in a variety of imaging systems and in lithography. So far, successful performance of the method under the conditions of X-ray 'in-line' imaging of 'homogeneous' objects has been demonstrated. However, many other possible modifications and applications of the method are envisaged. For example, the method may be applied to imaging systems using different forms of radiation (including visible light as used in photographic cameras, microscopes, etc) and matter waves (e.g. electron microscopy). Applicability of the proposed method to imaging using reflected (rather than transmitted) radiation is another possibility (e.g. in optical photography). Other potential applications include confocal microscopy and tomography.
In conclusion, we have presented a new method for rapid and robust deblurring of some types of images. The method can be viewed as a hardware (or combined software- hardware) implementation of second-order deconvolution which relies on the knowledge of the width of the PSF. In that respect the proposed method is quantitative in nature and should be viewed as a special variant of image deconvolution. The method is quite different from most conventional techniques for image sharpening as it is based on the actual physics of image formation and does not arbitrarily change the true information content of the image. We have developed several variants of the new deblurring method, including one that does not require any image processing at all, provided that the image is collected at a specific defocus distance at which Fresnel diffraction optimally compensates the PSF-induced blurring. We have tested the proposed deblurring techniques on simulated and experimental images with realistic amounts of noise.
The invention also relates to an apparatus for performing one of the deconvolution methods of the invention, including imaging system hardware and/or a computer product with computer readable program code for performing the method.
REFERENCES
[1] J.C.Dainty and R.Shaw, Image science (Academic Press, London, 1974). [2] A.N.Tichonov and V.Arsenin, Solutions of ill-posed problems (Wiley, New- York, 1977).
[3] CRNogel, Computational methods for inverse problems (SIAM, Philadelphia, 2002). [4] J.M.Cowley, Diffraction physics, 3rd revised edition (Elsevier, Amsterdam, 1995). [5] T.E.Gureyev, YaJ.Νesterets, A.W.Stevenson, and S.W.Wilkins, Appl.Optics 42, 6488 (2003).
[6] D.Paganin, S.C.Mayo, T.E.Gureyev, P.R.Miller, and S.W.Wilkins, J. Microscopy 206, 33 (2002). [7] T.E.Gureyev, D.M.Paganin, A.W.Stevenson, S.C.Mayo, and S.W.Wilkins, accepted for publication in Phys.Rev.Lett. [8] M.R.Teague, Deterministic phase retrieval: a Green's function solution, J.Opt.Soc.Am. 73 1434-1441 (1983). [9] T.E.Gureyev, and A.W.Stevenson, Provisional patent no. 2003905370, priority date October 2003.

Claims

The claims:
1. A method of deconvolution for producing a deblurred image of an object from an imaging system which produces penumbral and other instrumental blurring, using quantitative optimisation of counteractive effects of Fresnel diffraction and the penumbral blurring.
2. A method as claimed in claim 1, wherein the deblurred image is obtained by a linear combination of a defocused blurred image and an in-focus blurred image.
3. A method of deconvolution applied to images affected by penumbral and other instrumental blurring, wherein a deblurred image is obtained by a linear combination of a defocused blurred image and an in-focus blurred image.
4. A method as claimed in claim 2 or 3, wherein the deblurred in-focus image is calculated according to the following:
I(x, y) = (b0 - b2 1 a)D(x, y) + (b2 1 ά)Dz (x, y) where: I(x, y) = intensity distribution of the deblurred image bQ = l/α00 and a is a coefficient proportional to a moment of the point- spread function. ™2 2 00 ' = 2δ/(2kβ) , where k = 2πlλ and δ εtndβ are related by n = \ - δ -iβ , the complex refractive index of the object material. D(x, y) — blurred in-focus image; and Dz(x,y) = blurred defocused image.
5. A method as claimed in claim 1, wherein the deblurred image is obtained by numerical processing of a single defocused image.
6. A method of deconvolution applied to images affected by penumbral and other instrumental blurring, wherein a deblurred image is obtained by numerical processing of a single defocused image.
7. A method as claimed in claim 5 or 6, wherein the numerical processing includes applying the following operator to the defocused image:
I(x, y) ≤ [(b0 - b2 1 a)(l - a )~ + b2 1 a]Dz (x, y) where: 7(x, y) = intensity distribution of the deblurred in-focus image b0 = 1 / a00 and aQ0 is a coefficient proportional to a moment of the point-spread function. b2 = a2 /a00 ; a = 2δ/(2kβ) , where k = 2πlλ and δ an.ά.β are related by n = l -δ -iβ , the complex refractive index of the object material. D(x, y) = blurred in-focus image; and Dz (x, y) = blurred defocussed image.
8. A method as claimed in claim 1, wherein the deblurred image is obtained at a predetermined defocus distance.
9. A method of deconvolution, wherein a deblurred image is obtained at a predetermined defocus distance.
10. A method as claimed in claim 8 or 9, wherein the defocus distance (za) is determined with reference to the relationship: z = zd ≡ 2kβa2 f(δa00) .
11. A method as claimed in claim 1, wherein the method is applied assuming an isotropic point-spread function.
12. A method as claimed in claim 1, wherein the method is applied assuming an anisotropic point-spread function.
13. A method as claimed in claim 11 or 12, wherein the method is applied for monochromatic radiation.
14. A method as claimed in claim 11 or 12, wherein the method is applied for polychromatic radiation, and the method includes averaging a refractive index with respect to the spectral distribution of the incident radiation.
15. A method as claimed in claim 13 or 14, wherein the radiation is highly coherent radiation.
16. A method as claimed in claim 13 or 14, wherein the radiation is partially coherent radiation.
17. A method as claimed in any one of claims 11 to 16, wherein the object is assumed to be of a single material composition.
18. A method as claimed in any one of claims 11 to 16, wherein the object is assumed to be of a multiple material composition.
19. A method of deconvolution, including obtaining a deblurred phase distribution of a pure phase (non-absorbing) object from a single blurred defocused image, in accordance with: φ(x,y) = (klz)(b2 /b0 - V-2)[Dz(x,y)/D0 - l] where: φ(x, v) is the phase distribution in a transmitted beam in the object plane, z = 0 ; z = defocused plane distance from the object; b0 = \lam,b2 - a2 1 a2^ with a2 and aQ0 being coefficients proportional to moments of the point-spread function v2 =aXs Dz (x, y) = blurred defocussed image at position z; and D0 = 70α00 , with I0 = constant.
20. An image formed by a method as claimed in any one of .claims 1 to 19.
21. Apparatus adapted to perform a method as claimed in any one of claims 1 to 19.
22. A computer program product, including a computer useable medium having computer readable program code for performing a method as claimed in any of claims 1 to 19.
PCT/AU2004/001342 2003-10-02 2004-09-30 Enhancement of spatial resolution of imaging systems by means of defocus WO2005031645A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2003905370A AU2003905370A0 (en) 2003-10-02 Enhancement of spatial resolution of imaging systems by means of defocus
AU2003905370 2003-10-02

Publications (1)

Publication Number Publication Date
WO2005031645A1 true WO2005031645A1 (en) 2005-04-07

Family

ID=34382677

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2004/001342 WO2005031645A1 (en) 2003-10-02 2004-09-30 Enhancement of spatial resolution of imaging systems by means of defocus

Country Status (1)

Country Link
WO (1) WO2005031645A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007080130A2 (en) * 2006-01-13 2007-07-19 Micronic Laser Systems Ab Apparatuses, methods and computer programs for artificial resolution enhancement in optical systems
ES2291129A1 (en) * 2006-08-03 2008-02-16 Consejo Superior De Investigaciones Cientificas Rivet manufactured for non-metallic materials
US7860333B2 (en) 2007-01-09 2010-12-28 University Of Utah Research Foundation Systems and methods for deblurring data corrupted by shift variant blurring
WO2014013021A1 (en) 2012-07-20 2014-01-23 Carl Zeiss Ag Method and apparatus for image reconstruction
US10009587B1 (en) 2017-08-14 2018-06-26 Christie Digital Systems Usa, Inc. Real-time spatial-based resolution enhancement using shifted superposition

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0466252A2 (en) * 1990-07-03 1992-01-15 The Research Foundation Of State University Of New York A method and apparatus for restoring convolution degraded images and signals
US6433325B1 (en) * 1999-08-07 2002-08-13 Institute Of Microelectronics Apparatus and method for image enhancement

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0466252A2 (en) * 1990-07-03 1992-01-15 The Research Foundation Of State University Of New York A method and apparatus for restoring convolution degraded images and signals
US6433325B1 (en) * 1999-08-07 2002-08-13 Institute Of Microelectronics Apparatus and method for image enhancement

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007080130A2 (en) * 2006-01-13 2007-07-19 Micronic Laser Systems Ab Apparatuses, methods and computer programs for artificial resolution enhancement in optical systems
WO2007080130A3 (en) * 2006-01-13 2008-04-03 Micronic Laser Systems Ab Apparatuses, methods and computer programs for artificial resolution enhancement in optical systems
JP2009523241A (en) * 2006-01-13 2009-06-18 マイクロニック レーザー システムズ アクチボラゲット APPARATUS, METHOD, AND COMPUTER PROGRAM FOR IMPROVING ARTIFICIAL RESOLUTION IN OPTICAL SYSTEM
ES2291129A1 (en) * 2006-08-03 2008-02-16 Consejo Superior De Investigaciones Cientificas Rivet manufactured for non-metallic materials
WO2008020109A1 (en) * 2006-08-03 2008-02-21 Consejo Superior De Investigaciones Científicas Method for restoration of images which are affected by imperfections, device for implementation of this, and the corresponding applications
US7860333B2 (en) 2007-01-09 2010-12-28 University Of Utah Research Foundation Systems and methods for deblurring data corrupted by shift variant blurring
US8218889B2 (en) 2007-01-09 2012-07-10 University Of Utah Research Foundation Systems and methods for deblurring data corrupted by shift variant blurring
WO2014013021A1 (en) 2012-07-20 2014-01-23 Carl Zeiss Ag Method and apparatus for image reconstruction
DE102012106584A1 (en) 2012-07-20 2014-01-23 Carl Zeiss Ag Method and apparatus for image reconstruction
US9516242B2 (en) 2012-07-20 2016-12-06 Carl Zeiss Ag Method and apparatus for image reconstruction
DE102012106584B4 (en) * 2012-07-20 2021-01-07 Carl Zeiss Ag Method and device for image reconstruction
US10009587B1 (en) 2017-08-14 2018-06-26 Christie Digital Systems Usa, Inc. Real-time spatial-based resolution enhancement using shifted superposition

Similar Documents

Publication Publication Date Title
Orieux et al. Bayesian estimation for optimized structured illumination microscopy
US7978403B2 (en) Imaging interferometric microscopy
Reu et al. Camera system resolution and its influence on digital image correlation
Zuo et al. Transport-of-intensity phase imaging using Savitzky-Golay differentiation filter-theory and applications
Becker et al. Deconvolution of light sheet microscopy recordings
US20150100278A1 (en) Systems and methods for quantitative phase imaging with partially coherent illumination
Roels et al. An overview of state‐of‐the‐art image restoration in electron microscopy
Chakraborty et al. Source diversity for transport of intensity phase imaging
Conchello et al. Enhanced three-dimensional reconstruction from confocal scanning microscope images. II. Depth discrimination versus signal-to-noise ratio in partially confocal images
Li et al. Efficient assessment method of on-board modulation transfer function of optical remote sensing sensors
Gureyev et al. Image deblurring by means of defocus
Midoh et al. Accuracy improvement of phase estimation in electron holography using noise reduction methods
Otón et al. XTEND: Extending the depth of field in cryo soft X-ray tomography
US8653454B2 (en) Electron-beam image reconstruction
Lee et al. Deep learning for high-resolution and high-sensitivity interferometric phase contrast imaging
Huang et al. Exploring an optimal wavelet-based filter for cryo-ET imaging
Chakraborty et al. Optical convolution for quantitative phase retrieval using the transport of intensity equation
EP3830628A1 (en) Device and process for capturing microscopic plenoptic images with turbulence attenuation
WO2005031645A1 (en) Enhancement of spatial resolution of imaging systems by means of defocus
Paganin et al. Phase contrast, phase retrieval and aberration balancing in shift-invariant linear imaging systems
Gunjala et al. Extreme ultraviolet microscope characterization using photomask surface roughness
Eljarrat et al. Multi-focus TIE algorithm including partial spatial coherence and overlapping filters
Becker Improving the signal-to-noise ratio in incoherent imaging
Wu et al. Phase diversity technique with sparse regularization in liquid crystal adaptive optics system
JPH11258057A (en) Complex signal detecting method, complex microscope and complex diffraction device

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

DPEN Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed from 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase