JP4917404B2 - Phase object visualization method and microscope system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce information on phase distribution from an observation image of a phase object in a light viewing field microscope and to reconstruct a differential interference image or a phase difference image or the like by using the information on the phase distribution. <P>SOLUTION: Two images before and behind the vicinity of the focusing position of an optical system are acquired from a sample having the phase distribution, a difference image and a sum image are calculated from the acquired images, and the difference image is divided by the sum image so as to calculate a normalized image, then performing deconvolution of the normalized image by the optical responsiveness characteristic of the optical system, whereby a phase distribution image is reproduced. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to a method for visualizing a phase object with a microscope, and more particularly to reproduction of accurate phase information.

  Ordinary living cells are substantially colorless and transparent, and are difficult to observe with a bright field microscope. Therefore, when observing a living cell, devise such as staining the cell. Alternatively, microscopes incorporating special devices that detect the topological properties of living cells have been used. Examples thereof include a phase contrast microscope and a differential interference microscope.

  Moreover, the minute unevenness formed on the semiconductor substrate or the metal surface is a typical practical example of the differential interference microscope observed using the phase difference of the surface reflected light. The phase difference of the reflected light produced by minute irregularities is visualized by a differential interference microscope.

However, phase contrast microscopes and differential interference microscopes have to incorporate a special device into the microscope, and there is a desire to observe the phase difference by ordinary bright field observation. As a solution, there is a method of visualizing a phase object by defocusing as described in Patent Document 1. Patent Document 1 discloses a method in which a contrast correlated to a phase distribution of a phase object is generated by shifting the focal position from the in-focus position, and the phase object is visualized.
JP 2005-173288 A

  However, in the above-mentioned Patent Document 1, although a phase object is visualized, a method for restoring information on a phase distribution of the phase object is not shown.

  In the present invention, phase distribution information is reproduced from an observation image of a phase object in a bright field microscope. Then, using this phase distribution information, a differential interference image, a phase difference image, and the like are reproduced.

An object of the present invention is to obtain an image of the two front and rear focus near the position of the optical system from a sample having a phase distribution, the image difference image calculated from the images acquired difference image in this longitudinal 2 places Divide by the calculated sum image , or calculate the normalized image by dividing the difference image by the in-focus position image acquired at the in-focus position, and the normalized image is given by the following equation of the optical system : This is accomplished by reconstructing the phase distribution image by deconvolution with a dynamic response characteristic.

here,

Is TCC (Transmission Cross Coefficient).

According to the present invention, phase distribution information can be reproduced from an observation image of a phase object in a bright field microscope.
And the height of the unevenness | corrugation of a sample can be calculated using this phase distribution image. That is, the present invention can measure irregularities using a normal bright field microscope.

  Furthermore, using this phase distribution information, a differential interference image, a phase difference image, and the like can be generated. This means that differential interference observation and phase difference observation can be performed without using a special microscope such as a differential interference microscope or a phase contrast microscope.

  First, in the following, a theoretical explanation will be given on the method of visualizing the phase object and the irregularities on the substrate surface by shifting the focus, and the problems of this method will be explained. Thereafter, a solution to this problem in the present invention will be described.

  FIG. 1 is a diagram for explaining the principle, and schematically shows an object point, an imaging optical system entrance pupil, an exit pupil, and an imaging plane. As shown in FIG. 1, when the observation object is placed at the in-focus position, the light emitted from the observation object point spreads in a spherical wave shape as indicated by the solid line and enters the entrance pupil of the imaging optical system. The light that enters the entrance pupil becomes spherical-wave-like convergent light from the exit pupil of the imaging optical system, and is condensed on the imaging surface to form an image. At this time, since there is no optical path (phase) difference between the light beams transmitted through the imaging optical system, no blur occurs in the image.

  However, when the object point is moved to the position indicated by the dotted line, the light from the object point enters the entrance pupil in a spherical wave shape. The light that has entered the entrance pupil changes to spherical wave light that travels from the exit pupil toward the image plane after movement. When observed on the image plane after movement, no optical path (phase) difference occurs in the light, but when observed on the original imaging plane without moving the observation point, an optical path (phase) difference occurs in each light beam.

  In the present invention, by utilizing this point, the observation object is arranged at a position shifted from the in-focus position of the imaging optical system, thereby generating a phase difference in each light beam transmitted through the imaging optical system. Yes.

In FIG. 1, the phase difference between the light beam on the optical axis of the imaging optical system and the maximum NA (numerical aperture) light beam is the largest.
Therefore, by observing the observation object at a position outside the in-focus position of the imaging optical system, the amount of deviation (defocus amount) from the in-focus position between the light transmitted through the observation object and the diffracted light can be accommodated. It is possible to generate a phase difference amount.

  As a result, this amount of phase difference functions equivalent to the phase film used in the phase difference observation method, etc., and can provide an image contrast proportional to the phase distribution, and visualize the unevenness on the phase object and the substrate. be able to.

  When observing a phase object, the image contrast proportional to the phase distribution of the phase object is proportional to the phase amount of the observation object and the phase difference amount provided between transmitted light and diffracted light. The angle between the light diffracted by the observation object and the transmitted light varies depending on the shape of the observation object. When the angle between the diffracted light and the transmitted light changes, the amount of phase difference generated between the two light beams varies even with the same defocus amount. Therefore, a better image contrast can be obtained by changing the defocus amount for each cell to be observed.

  Further, the sign of the phase difference of each light beam generated by defocusing changes depending on whether the observation object is shifted to the near point side or the far point side from the in-focus position of the imaging optical system. That is, an image observed by shifting an observation object such as a phase object from the in-focus position of the imaging optical system to the near point side and an image observed by shifting to the far point side are obtained corresponding to the phase distribution of the observation object. The image contrast is also reversed.

  Therefore, an image component that is not affected by the phase difference given by defocusing can be separated by performing an inter-image calculation on the image captured by shifting to the near point side and the image captured by shifting to the far point side. Further, by performing the difference calculation for each pixel of the two images, the image contrast of the image component corresponding to the phase distribution of the observation object is doubled.

In the following, the logical background of the above phenomenon will be described.
Since the microscope illumination method is partially coherent illumination, its imaging intensity distribution is expressed by the following equation.

here

Is TCC (Transmission Cross Coefficient) and is given below.

here

Represents the Fourier transform of the complex amplitude distribution of the object,

Represents the intensity distribution of the pupil of the illumination optical system,

Represents the pupil function of the imaging optical system.
In the following, for the sake of simplicity, a one-dimensional model having no change in the y-axis direction is considered. In other words, the image intensity distribution is

And TCC

And simplified.
If the phase distribution of the phase object is φ (x) and the transmittance is 1 for simplicity, the complex amplitude distribution o (x) of the object and its Fourier transform O (f) are It is represented by

Here, Φ (f) is a Φ (x) Fourier transform of the phase distribution.
By substituting this into equation (3), the following is obtained approximately.

Get.
Here, the first term in the above formula is a term whose sign does not change by defocus, and the second term is a term whose sign changes by defocus. To show that, consider the effect of defocus on T (f, 0) + T (0, -f) and T (f, 0) -T (0, -f).

  In general, the pupil function when aberration W (ξ) occurs due to defocusing, etc. is

Given in. here

It is. This is substituted into equation (4). At this time, assuming that the aberration W (ξ) is rotationally symmetric (for example, the aberration due to defocus is rotationally symmetric),

Get.
On the other hand, the wavefront aberration due to the defocus of the defocus amount z uses the pupil coordinates ξ,

It is expressed. That is, T (f, 0) + T (0, -f) is an even function with respect to the defocus amount z, and T (f, 0) -T (0, -f) is an odd function with respect to the defocus amount z. It is. This indicates that the first term of the formula (5) is a term whose sign does not change by defocus, and the second term is a term whose sign does not change by defocus.

  Also, due to this property, if the image intensity distribution of defocus amount z and defocus amount -z is I (x; z) and I (x; -z), respectively, the image intensity distribution of the difference image is

Thus, an image in which an element proportional to the phase distribution is emphasized twice is obtained.
FIG. 2 is an example of visualization of the phase distribution by the above method. This image shows a defocused image (FIGS. 2 (a) and 2 (b)) of a lattice pattern composed of grooves of the same height manufactured on a silicon wafer using a semiconductor manufacturing technique and a difference image I of the defocused image (x; z) -I (x; -z). The shooting conditions were 20 × objective lens, 9 mm focal length, and 0.45 NA. When σ = aperture stop diameter / pupil diameter of objective lens = 0.3, the image was taken with a defocus amount of 8 μm.

  As understood from FIG. 2C, the degree of visualization regarding the height of the groove differs depending on the width of the groove. In particular, the contrast is strong in a portion where the width of the grating is narrow, so that it seems that the heights of the grooves are different.

  The reason why such a problem occurs is that T (f, 0; z) -T (0, -f; z) in the equation (6) becomes a coefficient depending on the spatial frequency, and the phase distribution (Fourier transform) Φ ( This is because it depends on f). As a result, in the example of FIG. 2, the visualization depending on the width of the groove has been performed.

In order to solve the above problem, the present invention considers a method of restoring only the phase distribution φ (x) from the equation (6).
here,

If the inverse Fourier transform is otf (x, z),

It becomes.
here,

Is determined by the characteristics of the optical system (NA of the objective lens, focal length, illumination wavelength) and the amount of defocus.
That is, if the OTF of the optical system used for image acquisition is calculated in advance, the phase distribution φ (x) of the phase object is deconvolved by otf (x, z).

By phase distribution

Is calculated.
In the above description, a model in which the phase distribution change of the sample is limited to one dimension has been described, but the same argument holds when the phase distribution change of the sample is two-dimensional. At this time,

Can be used.
As can be seen from the above discussion, in the present invention, the OTF is acquired with high accuracy, which is related to the phase distribution restoration accuracy. Therefore, it is desirable that the intensity distribution Q (ξ) of the pupil of the illumination optical system is constant. Furthermore, it is desirable to use light having a single wavelength or a narrow wavelength range as illumination light. Of course, a method is also conceivable in which the above-described invention is implemented for each of the three primary color illuminations, and a color image is restored from the obtained image.

Hereinafter, a phase object visualization method based on the above theory will be described with a specific example. FIG. 3 is a flowchart showing an embodiment of the present invention.
First, two images I (x; z) and I (x; -z) before and after the in-focus position are acquired (S1). At this time, the defocus amount from the in-focus position is desirably equal.

  Next, a difference image I (x; z) -I (x; -z) and a sum image I (x; z) + I (x; -z) are calculated from the two images (S2, S3). Then, the difference image is normalized with the sum image (S4). That means

Calculate
Next, the difference image normalized with the sum image is subjected to Fourier transform (S5). Thereafter, the result of the Fourier transform is divided by the inverse Fourier transform otf (x, z) of the optical response characteristic (S6). At this time, the optical response characteristic OTF is determined by the characteristics of the microscope optical system (the objective lens NA, the focal length, and the illumination wavelength) and the amount of defocusing, so this amount may be calculated in advance.

Finally, a phase distribution image is reproduced by performing inverse Fourier transform on the above result (S7).
Moreover, a modification can be considered for implementation of this invention. In the normalization step (S4), the focused image I (x; 0) can be used instead of the sum image I (x; z) + I (x; −z). In other words,

Instead of calculating

Calculate
The effect of using a focused image instead of a sum image in the implementation of the present invention is that an object other than a phase object (or an object having both a property as a phase object and a property other than a phase object) Occurs when in the viewing field. Defocused images I (x; z) and I (x; -z) are simply blurred images for objects other than phase objects. For example, the dust seen on the specimens (a) and (b) of FIG. Since these do not show phenomena peculiar to phase objects such as contrast reversal, they are made difficult to see in FIG. 2C by creating a difference image I (x; z) -I (x; -z). .

  However, in the sum image I (x; z) + I (x; −z), the influence other than the phase object is enhanced and visualized as a blurred image. At this time, normalization with reduced influence of the blurred image can be performed by using the focused image I (x; 0) instead of the sum image I (x; z) + I (x; −z).

  As an application of the above-described visualization method, a differential interference image can be generated from the phase distribution image obtained by the present invention. The differential interference image here is an image obtained when the target sample is photographed with a differential interference microscope. That is, according to the present invention, a differential interference image can be created using a normal bright field microscope without using a differential interference microscope.

  The relationship between the phase distribution image and the differential interference image is established by convolution of the MTF of the differential interference microscope, as described in Patent Document 2 and Non-Patent Document 1. Here, the MTF of the differential interference microscope is

Where Δ is the share,

It is. If the Fourier transform of this MTF is psfD (x), the differential interference image is

Given in.
Japanese Patent No. 3523725 Optics Communications 260 (2006) 117-126 Note that in equation (9), M (f) and M (0) are factors by the optical system (differential interference microscope), and thus contribute essentially to the conversion to differential interference images. Factors that are

It is. That is, a differential interference image is obtained by convolving the phase distribution with an amount including this factor (Fourier transform).
FIG. 6 shows a differential interference image generated by the above method. For comparison, the figure shows the deconvolution according to the present invention and the non-convolution. FIG. 6A shows a differential interference image generated by regarding the expression (6) as a phase distribution, and FIG. 6B shows a phase distribution φ (x) as a result of applying the deconvolution of the present invention. A differential interference image is generated. As is clear when the two images are compared, the concave and convex portions of the grating are reproduced in a form close to reality in the differential interference image to which the deconvolution according to the present invention is applied.

  The phase distribution φ (x) according to the present invention can also be applied to generate a phase difference image. The phase difference image here is an image obtained when the target sample is photographed with a phase contrast microscope. That is, according to the present invention, a phase contrast image can be created using a normal bright field microscope without using a phase contrast microscope.

  The relationship between the phase distribution image and the phase difference image is also formed by convolution of the MTF of the phase contrast microscope, as described in Patent Document 3. Here, the MTF of the phase contrast microscope is

Is given. Here, Pa (ξ) and Pb (ξ) are quantities determined by an optical system that satisfies the relationship of pupil function P (ξ) and P (ξ) = Pa (ξ) + Pa (ξ). In other words, if this MTF Fourier transform is psfD (x), the phase difference image is

Given in.
Patent No. 3540352 Here, a microscope for carrying out the present invention will be described with reference to FIG. Although this figure is a schematic diagram of an epi-illumination type upright microscope, the present invention is not limited to this microscope configuration. Depending on the sample to be observed, an inverted cubic microscope or a transmission illumination type microscope can be used freely.

  In FIG. 4, the illumination light emitted from the light source 1 passes through the illumination optical system 2, turns back the optical path by the half mirror 3, enters the objective lens 4, and illuminates the sample (phase object) 5. The observation light emitted from the sample which is a phase object is magnified by the objective lens 4, passes through the half lens 3, and forms an image on the light receiving surface of the image sensor 8 such as a CCD camera by the imaging lens 7. The captured image is transferred to the computer 9, where a series of processes shown in FIG. 3 such as a sum image calculation, a normalization calculation, a Fourier transform calculation, and a deconvolution in OTF are performed. The visualized image of the phase object obtained as a result is displayed on the display on the computer 9.

  In order to implement the present invention, it is necessary to acquire two images before and after the focal point. Therefore, it is desirable that the focal position is changed by moving the stage 6 or the objective lens 4 in conjunction with the image acquisition by the image sensor 8.

  These series of processes may be implemented by software on a computer or may be implemented as hardware. Further, instead of being configured in the form of an independent computer, the microscope system may be configured integrally.

Finally, the effects of the above-described method will be described based on examples.
FIG. 5 shows an image of a lattice pattern made of a groove having a step of 50 nm manufactured on a silicon wafer using a semiconductor manufacturing technique in the same manner as FIG. The imaging conditions were 20 × objective lens, 9 mm focal length, and 0.45 NA, and σ = aperture aperture diameter / pupil diameter of objective lens = 0.5. At this time, (a) is an image at the in-focus position, (b) is a +2 μm defocused image, and (b) is a −2 μm defocused image.

  The graph shown in FIG. 6 is a graph of optical response characteristics used when reproducing a phase distribution image. This graph is calculated using the formula (8) from the focal length of 9 mm, NA of 0.45, and the defocus amount of 2 μm, which are the conditions for photographing FIG.

  FIG. 7 illustrates the effect of reproducing the phase distribution information according to the present invention. FIG. 7A shows an arc tangent of the normalized difference image of the image shown in FIG. 5 and visualized as a phase distribution image. On the other hand, FIG. 7B shows a phase distribution image visualized by executing deconvolution according to the present invention. Note that the normalization method used at this time is normalization using a sum image.

  FIG. 8 is a graph showing the cross section of the height of the grating obtained from the two pieces of phase distribution information in order to easily understand the difference between the phase distribution images of FIG. (A) corresponds to the phase distribution information of FIG. 7 (a), and (b) corresponds to the phase distribution information of FIG. 7 (b). In both figures, the horizontal axis represents the pixels in the cross section in FIG. 7, and the vertical axis represents the phase distribution converted into height units (μm) (that is, the phase difference expressed in wavelength). As can be seen from the figure, it can be seen that the grating height calculated from the phase distribution information subjected to deconvolution according to the present invention is closer to the actual grating height.

  FIG. 9 shows an embodiment of generating a differential interference image (DIC image), which is an application of the present invention. (A) is obtained by convolving the MTF of the differential interference microscope with the phase distribution image of FIG. 7 (a), and (b) is the phase distribution image of FIG. 7 (b) with the differential interference microscope. It was obtained by convolving MTF.

  FIG. 10 also shows an image of a grating pattern composed of grooves having a step of 50 nm. The imaging conditions were 20 × objective lens, 9 mm focal length, and 0.45 NA, and σ = aperture aperture diameter / pupil diameter of objective lens = 0.5. At this time, (a) is an image at the in-focus position, (b) is a defocus image of +4 μm, and (b) is a defocus image of −4 μm.

  The graph shown in FIG. 11 is a graph of optical response characteristics used when reproducing the phase distribution image. This graph is calculated using the formula (8) from the focal length of 9 mm, NA of 0.45, and the defocus amount of 4 μm, which are the conditions for photographing FIG.

  FIG. 12 illustrates the effect of reproducing the phase distribution information according to the present invention. FIG. 12A shows an arc tangent of the normalized difference image of the image shown in FIG. 10 and is visualized as a phase distribution image. On the other hand, FIG. 12B shows a phase distribution image visualized by executing deconvolution according to the present invention. Note that the normalization method used at this time is normalization using a focused image.

  FIG. 13 is a graph showing the cross section of the grating height obtained from two pieces of phase distribution information in order to make it easy to understand the difference between the phase distribution images of FIG. (A) corresponds to the phase distribution information of FIG. 12 (a), and (b) corresponds to the phase distribution information of FIG. 12 (b). In both figures, the horizontal axis represents the pixels of the cross section in FIG. 12, and the vertical axis represents the phase distribution converted into height units (μm) (that is, the phase difference expressed in wavelength). As can be seen from the figure, it can be seen that the grating height calculated from the phase distribution information subjected to deconvolution according to the present invention is closer to the actual grating height.

  FIG. 14 shows an embodiment of generating a differential interference image (DIC image), which is an application of the present invention. (A) is obtained by convolving the MTF of the differential interference microscope with the phase distribution image of FIG. 12 (a), and (b) is the phase distribution image of FIG. 12 (b) with the differential interference microscope. It was obtained by convolving MTF.

  FIG. 15 also shows an image of a grating pattern composed of grooves having a step of 50 nm. The imaging conditions were 20 × objective lens, 9 mm focal length, and 0.45 NA, and σ = aperture aperture diameter / pupil diameter of objective lens = 0.75. At this time, (a) is an image at the in-focus position, (b) is a defocus image of +4 μm, and (b) is a defocus image of −4 μm.

  The graph shown in FIG. 16 is a graph of optical response characteristics used when reproducing a phase distribution image. This graph is calculated using the formula (8) from the focal length of 9 mm, NA of 0.45, and the defocus amount of 4 μm, which are the conditions for photographing FIG.

  FIG. 17 illustrates the effect of reproducing the phase distribution information according to the present invention. FIG. 17A shows an arc tangent of the normalized difference image of the image shown in FIG. 15 and visualized as a phase distribution image. On the other hand, FIG. 17B shows a phase distribution image visualized by executing deconvolution according to the present invention. Note that the normalization method used at this time is normalization using a focused image.

  FIG. 18 is a graph showing the cross section of the lattice height obtained from the two pieces of phase distribution information in order to easily understand the difference between the phase distribution images of FIG. (A) corresponds to the phase distribution information of FIG. 17 (a), and (b) corresponds to the phase distribution information of FIG. 17 (b). In both figures, the horizontal axis represents the pixels of the cross section in FIG. 17, and the vertical axis represents the phase distribution converted into height units (μm) (that is, the phase difference expressed in wavelength). As can be seen from the figure, it can be seen that the grating height calculated from the phase distribution information subjected to deconvolution according to the present invention is closer to the actual grating height.

  FIG. 19 shows an embodiment of generating a differential interference image (DIC image), which is an application of the present invention. (A) is obtained by convolving the MTF of the differential interference microscope with the phase distribution image of FIG. 17 (a), and (b) is the phase distribution image of FIG. 17 (b) with the differential interference microscope. It was obtained by convolving MTF.

  As described above, according to the present invention, the phase distribution of the phase object can be acquired without incorporating a special device, and a differential interference image or a phase difference image can be generated based on the phase distribution. .

It is a figure explaining the phase difference (optical path difference) at the time of defocusing. It is the defocus image before and behind a focus, and its difference image. It is a flowchart showing the Example of this invention. It is a structural example of the microscope utilized for implementation of this invention. It is a focused image and a defocused image of the sample used in the embodiment of the present invention (1). It is an optical response function used in the embodiment of the present invention (1). It is a phase distribution image which shows the effect of implementation of this invention (1). It is a cross-sectional image of phase distribution which shows the effect of implementation of this invention (1). It is a differential interference image which shows the application example of implementation of this invention (1). It is a focused image and a defocused image of the sample used in the embodiment of the present invention (2). It is an optical response function used in the embodiment of the present invention (2). It is a phase distribution image which shows the effect of implementation of this invention (2). It is a cross-sectional image of a phase distribution showing the effect of implementation of the present invention (2). It is a differential interference image which shows the application example of implementation of this invention (2). It is a focused image and defocused image of the sample used in the embodiment of the present invention (3). It is an optical response function used in the embodiment of the present invention (3). It is a phase distribution image which shows the effect of implementation of this invention (3). It is a cross-sectional image of phase distribution which shows the effect of implementation of the present invention (3). It is a differential interference image which shows the application example of implementation of this invention (3).

Claims (5)

Acquire images from a sample with a phase distribution at two locations before and after the in-focus position of the optical system.
The difference image is calculated from the image,
Divide the difference image by the sum image calculated from the images acquired at the two front and rear positions, or calculate the normalized image by dividing the difference image by the in-focus position image acquired at the in-focus position ;
A phase object visualization method, wherein a phase distribution image is reproduced by deconvolution of the normalized image with an optical response characteristic given by the following expression of the optical system.
here,
Is TCC (Transmission Cross Coefficient). The method of calculating the uneven | corrugated height of a sample from the said phase distribution image obtained by the visualization method of the phase object of Claim 1 . A method for generating a differential interference image from the phase distribution image obtained by the phase object visualization method according to claim 1 . A method for generating a phase difference image from the phase distribution image obtained by the phase object visualization method according to claim 1 . Imaging means for acquiring images from a sample having a phase distribution at two positions before and after the in-focus position of the optical system;
Computing means for calculating a difference image from the image ;
The normalized image is calculated by dividing the difference image by the sum image calculated from the images acquired at the two front and rear positions , or by dividing the difference image by the in-focus position image acquired at the in-focus position by the imaging means. Normalization means to
A microscope system comprising deconvolution means for deconvolving the normalized image with an optical response characteristic given by the following expression of the optical system.
here,
Is TCC (Transmission Cross Coefficient).