JP2022096690A - Image processing method, apparatus, and program - Google Patents

Abstract

To enable detection of a point defect.SOLUTION: The present image processing method comprises the steps of: (A) creating an image of a grid by predetermined filter processing for each direction of a predetermined orthogonal axis or for each direction of an orthogonal axis in a case where the image is rotated such that a principal direction of the grid in a surface of a sample is parallel to any of orthogonal axes according to the principal direction, from the image of the sample; (B) calculating a sampling moire fringe from the image of the grid for each of the above mentioned directions, and calculating a phase distribution of a virtual one-dimensional moire fringe in a case where a pitch is defined as a virtual designated pitch on the basis of the sampling moire fringe; and (C) creating a two-dimensional multiplication moire fringe that is obtained by synthesizing a luminance distribution of the moire fringe based on a phase distribution in which the phase distribution is multiplied by a predetermined number for each of the above directions.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing technique for detecting defects in a material.

In order to guarantee the performance and life of semiconductor devices, there is a strong demand for the establishment of techniques for precisely controlling atomic-level lattice defects that occur during manufacturing. In addition, it is very important to evaluate the point defects such as atomic vacancies introduced during metal processing and the line defects such as dislocations to elucidate the mechanism from plastic deformation to fracture. Therefore, if the influence of the manufacturing process or processing process on the density and distribution of lattice defects can be quantitatively evaluated, the optimum manufacturing or processing process technology for improving the performance of devices and metal materials can be established.

Until now, the detection and evaluation of individual lattice defects was mainly performed by visual observation of high-magnification captured images using a transmission electron microscope (TEM) or atomic force microscope (AFM). It took time and effort, and it was extremely difficult to detect defects at the atomic level efficiently. Several commercial inspection devices such as wafer surface analysis systems have been developed, but these devices and software are expensive.

Since crystalline structures and periodic structures such as graphene, nanoparticles, nanoparticles, nanorods, and nanotubes contain diffraction gratings, optical techniques by image processing are suitable for low-cost defect detection. Existing optical methods include Fourier Transform (FT), Windowed Fourier Transform (WFT), Scanning Moire Method, Geometric Moire Method, Digital Moire Method, and Sampling Moire Method (SM). Moire. For example, Patent Documents 2 and 3) and the like exist.

In order to detect the dislocation distribution of the atomic structure in a wide field of view, a Fourier Transform Filtered Sampling Moire (FTF-SM) technology (for example, Patent Document 1) has been proposed, and GaN and Si have been proposed. Dislocations and strain distributions can be visualized from TEM images of semiconductors. The sampling moire method (for example, Non-Patent Document 1) and the digital phase shift moire method (Non-Patent Document 2) are used for detecting dislocation distribution and interface.

In these existing optical methods, the lattice arrangements in different directions of the crystal structure and the periodic structure were analyzed separately. Therefore, although line defects such as dislocations can be easily detected, it is difficult to detect point defects such as vacancies.

Japanese Unexamined Patent Publication No. 2018-109593 Japanese Unexamined Patent Publication No. 2019-66369 Japanese Unexamined Patent Publication No. 2009-264852

Wang Q, Ri S, Xia P, Liu Z., "Automatic detection of defect positions including interface dislocations and strain measurement in Ge / Si heterostructure from moire phase processing of TEM image", Optics and Lasers in Engineering, 2020, 129: 106077 .. Zhu Y, Wen H, Zhang H, Liu Z., "Application of digital phase shifting moire method in interface and dislocation location recognition and real strain characterization from HRTEM images", Optics Express, 2019, 27 (25): 36990-37002.

Therefore, an object of the present invention is to provide an image processing technique capable of detecting a point defect as one aspect.

The image processing method according to the present invention is either (A) from the image of the sample, for each direction of a predetermined orthogonal axis, or depending on the main direction of the lattice on the surface of the sample, the main direction is the orthogonal axis. A step of generating a grid image by a predetermined filter process for each direction of the orthogonal axis when the image is rotated so as to be parallel to the above, and (B) sampling moire fringes from the grid image in each of the above directions. The step of calculating the phase distribution of the virtual one-dimensional moire fringes when the pitch is set to a virtual designated pitch based on the sampling moire fringes, and (C) the phase distribution is predeterminedly multiplied in each of the above directions. It includes a step of generating a two-dimensional doubling moiré fringe obtained by synthesizing the brightness distribution of the moiré fringe by the phase distribution.

According to one aspect, point defects can be detected.

FIG. 1 is a diagram for showing a configuration example of a defect detection system according to the present embodiment. FIG. 2 is a diagram for explaining the main direction and angle of the grid. FIG. 3 is a diagram showing a processing flow according to the present embodiment. FIG. 4 is a diagram showing the processing results of each stage in the present embodiment. FIG. 5 is a diagram for explaining rotation and trimming of an image. FIG. 6 is a diagram for explaining the sampling moire method. FIG. 7 is a diagram showing a processing flow according to the present embodiment. FIG. 8 is a diagram showing a processing flow of defect position coordinate detection processing. FIG. 9 is a diagram showing the processing results of each stage in the defect position coordinate detection processing. FIG. 10 is a diagram for explaining the first embodiment. FIG. 11 is a diagram for explaining the second embodiment. FIG. 12 is a diagram for explaining the second embodiment. FIG. 13 is a diagram for explaining the third embodiment. FIG. 14 is a diagram showing an example of a functional configuration of a computer.

FIG. 1 shows a defect detection system 100 according to an embodiment of the present invention. The defect detection system 100 includes an image pickup device 10 and a defect detection device 20. The image pickup apparatus 10 is, for example, a transmission electron microscope (TEM) or the like, and photographs a two-dimensional crystal or a periodic structure (called a lattice) appearing on the surface of the sample 1.

The defect detection device 20 stores the processing results of the data storage unit 21 for storing the image data taken by the image pickup device 10 and the data in the process of processing, the image processing unit 23 for executing the processing described below, and the image processing unit 23. It has an output unit 25 for outputting to an output device (for example, a display device, a printing device, another computer, etc.). The image processing unit 23 includes a preprocessing unit 231, a rotation processing unit 232, a virtual moire generation unit 233, a two-dimensional photomultiplier moire generation unit 234, and a defect position detection unit 235.

The preprocessing unit 231 executes a process of generating a main grid image (for example, FIGS. 6 (a), 4 (c), and (d)) from an image taken by the image pickup apparatus 10 for each main grid direction. The rotation processing unit 232 executes a process of rotating or trimming the image as needed. The virtual moiré generation unit 233 generates sampling moiré fringes (for example, FIG. 6B) from each main grid image by the sampling moiré method, and calculates the phase distribution of the sampling moiré fringes (for example, FIG. 6C). , The phase distribution of the grid is calculated from the phase distribution of the sampling moire fringes, and the phase distribution of the virtual moire (for example, FIGS. 4 (e) and 4 (f)) is calculated from the phase distribution of the grid. The virtual moiré is a virtual one-dimensional moiré fringe when the pitch is set to a virtual designated pitch based on the sampling moiré fringe. The virtual moiré may also be referred to as a digital moiré.

The two-dimensional doubling moire generation unit 234 generates two-dimensional doubling moire fringes (for example, FIG. 4 (g)) from the phase distribution of virtual moiré in each main lattice direction, and the phase distribution of the two-dimensional doubling moire fringes. (For example, FIG. 4 (h)) is calculated. The two-dimensional doubling moiré fringe is the sum of the luminance distributions of the moiré fringes due to the phase distribution obtained by multiplying the phase distribution of the virtual moiré by a predetermined value for each main direction. The defect position detection unit 235 calculates the strain distribution of the grid (for example, FIGS. 9A and 9B) from the phase distribution of the grid for each main direction, and integrates the strain distribution of the grid for each main direction (for example). FIG. 9C) is generated, and a process of detecting defect position coordinates from the image is executed.

In the present embodiment, by introducing a two-dimensional photomultiplier fringe that enlarges the original two-dimensional microstructure, it is possible to detect a point defect of the lattice. If the atomic arrangement of the crystal structure is regarded as a regular lattice, the lattice defects including the point defects deform a part of the ordered lattice, so that the moire pattern generated from the atomic arrangement also changes according to the point defects. It is something that makes use of that. By using virtual moiré, line defects can also be detected.

First, the principal direction r and the angle θ of the observed lattice will be described with reference to FIG. The grid is usually composed of a main grid in two or three directions. When the x-axis and the y-axis are defined as shown in FIG. 2A, the normal direction of, for example, an atomic train (thick line) arranged at a pitch _pn is the main direction rn of the main lattice _n , and the main lattice n The angle θ _n of is the angle formed by the main direction _rn counterclockwise from the x-axis. As schematically shown in FIG. 2B, the grids composed of the main grids in two directions are the main grid 1 having the principal direction r ₁ and the angle θ ₁ , and the main grid r ₂ and the angle θ ₂ . Determined by the main grid 2. As schematically shown in FIG. 3C, the grid composed of the main grids in three directions has a main grid 1 having a principal direction r ₁ and an angle θ ₁ , and a main grid having a principal direction r ₂ and an angle θ ₂ . It is determined by any two of the grid 2, the principal direction r ₃ , and the main grid 3 having an angle θ ₃ . Hereinafter, a grid composed of two main directions will be described as an example, but even if the grid is composed of three or more main grids, it may be for any two main directions or for each combination of the two main directions. It should be processed to.

Next, the processing contents of the defect detection device 20 will be described with reference to FIGS. 3 to 9.

First, the preprocessing unit 231 reads out the image of the sample 1 taken by the image pickup apparatus 10 from the data storage unit 21 (FIG. 3: step S1). Noise reduction processing may be performed on the image of the sample 1 by the image pickup apparatus 10 or the preprocessing unit 231. In this example, as shown in FIG. 4A, the image pickup apparatus 10 captures a grid containing two point defects surrounded by a circle and one line defect surrounded by a square. Therefore, it is assumed that a photographed image (also referred to as a grid image) as shown in FIG. 4 (b) can be obtained. In this example, the main grid r ₁ is substantially parallel to the x-axis, and the main grid r ₂ is substantially parallel to the y-axis. An example is shown in which the luminance value of the image is 256 gradations.

さらに、主方向ｒ_ｎのピッチｐ_ｎ（ｘ，ｙ）を、以下のように算出する。このようなピッチｐ_ｎ（ｘ，ｙ）から得られる評価領域におけるピッチの平均値は、所定のフィルタ処理やサンプリングモアレ縞の生成などで用いられる。なお、角度及びピッチについては、厳密な値でなくても良いので、外部から入力することで与えるようにしても良い。

Next, the preprocessing unit 231 calculates the angles and pitches of the principal directions 1 and 2 from the captured image (step S3). Various methods can be adopted for this calculation, and for example, a method using a two-dimensional Fourier transform and a two-dimensional inverse Fourier transform shown in Patent Document 2 may be used. To outline this method, a two-dimensional Fourier transform and a two-dimensional inverse Fourier transform are combined to calculate the phase _{φFT_n} (x, y) of the lattice for each main direction rn ( _n = 1, 2), and the angular distribution. θ _n (x, y) is calculated as follows. The average angle in the evaluation region obtained from the angle distribution θ _n (x, y) is used for image rotation by the rotation processing unit 232.

Further, the pitch _pn (x, y) of the principal direction _rn is calculated as follows. The average value of the pitches in the evaluation region obtained from such pitches _pn (x, y) is used in a predetermined filter process, generation of sampling moire fringes, and the like. Since the angle and pitch do not have to be exact values, they may be given by inputting from the outside.

Next, the rotation processing unit 232 rotates the image as necessary based on the angle distribution θ _n (x, y) in the principal directions 1 and 2 (step S5). In the case shown in FIG. 4 (b), the main grid r ₁ is substantially parallel to the horizontal x-axis, and the main grid r ₂ is substantially parallel to the vertical y-axis, so that rotation is not necessary. For example, if the principal direction is within a range of about 10 degrees with the x-axis or y-axis, it is not necessary to rotate. In this case, one principal direction is the x-axis and the other principal direction is the y-axis. On the other hand, when the principal directions r1 and r2 _are tilted from the x-axis and the y-axis as shown in FIG. 5 (a), _one of them is parallel to the x-axis as shown in FIG. 5 (b). Rotate the other so that it is parallel to the y-axis. At this time, the rotation angle is determined by the difference between the average angle in the evaluation region obtained from the angle distribution θ _n (x, y) and the x-axis or the y-axis. However, it is not necessary to rotate at exactly this rotation angle. When rotated, the image is processed as a minimum rectangular size image that surrounds the rotated image. Since the image is the smallest rectangle that surrounds the rotated image, it has a margin added.

Next, the preprocessing unit 231 performs a predetermined filter process on the captured image or the captured image after rotation in each main direction to generate a main grid image (step S7). Specifically, for the principal direction r ₁ at an angle parallel to or close to the x-axis, the [2p ₁ + 1,1] size averaging filter or median filter (p ₁ is the average value of the pitches in the principal direction r ₁ ). ) Is applied to generate _a principal lattice image in the principal direction r1. On the other hand, for the principal direction r ₂ at an angle parallel to or close to the y-axis, an averaging filter or a median filter of [1, 2p ₂ + 1] size is applied (p ₂ is the average value of the pitches in the principal direction r ₂ ). This generates a principal lattice image in the principal direction _r2 . The averaging filter and the median filter are examples of low-pass filters. The filter size is an example and may be slightly different.

When step S7 is executed for the captured image as shown in FIG. 4 (b), the main grid image in the _main direction r1 as shown in FIG. 4 (c) and the main direction r as shown in FIG. 4 (d). ₂ main grid images are obtained. When the captured image is rotated, a main grid image in the main direction r1 as shown in FIG. 5 (c) and _a main grid image in the main direction r2 as shown in _FIG . 5 (d) are obtained. Be done. Also in FIGS. 5 (c) and 5 (d), the added margin is maintained.

Then, the virtual moiré generation unit 233 generates sampling moiré fringes by the sampling moiré method for each main grid image (step S9). Although the sampling moiré method is well known, the outline of the part related to the present embodiment will be described.

ここでＡ_１（ｘ，ｙ）は主格子１の振幅、φ_１（ｘ，ｙ）は主格子１の位相、Ｂ_１（ｘ，ｙ）は背景輝度、ｐ_１（ｘ，ｙ）は主格子１のピッチを表す。なお、以下では、主方向ｒ_１を例に示すが、「１」を「２」と表せば、主方向ｒ_２の場合を表す。また、ｘのみを用いている部分についてはｙに置換すれば良い。 For example, in the case of a main grid image as shown in FIG. 4 (c), the main direction r ₁ is parallel to the x-axis, and the luminance I ₁ (x, y) of the main grid 1 is expressed by the following equation.

Here, A ₁ (x, y) is the amplitude of the main grid 1, φ ₁ (x, y) is the phase of the main grid 1, B ₁ (x, y) is the background luminance, and p ₁ (x, y) is the main. Represents the pitch of lattice 1. In the following, the principal direction r ₁ is shown as an example, but if "1" is expressed as "2", the case of the principal direction r ₂ is represented. Further, the portion using only x may be replaced with y.

ここでｋはｋ段階目のサンプリングモアレ縞を示しており、φ_ｍ１（ｘ，ｙ）は、ｋ＝０において主方向ｒ_１の主格子画像から生成されるサンプリングモアレ縞の位相である。さらに、Ｔ_１は正の整数であって、主格子１のピッチの平均値に近いサンプリングピッチである。例えば評価領域におけるピッチの平均値がｐ_１＝１０．４の場合には、Ｔ_１＝１０を採用する。 In the sampling moiré method, T _{1 pixel downsampling and lattice image brightness interpolation are performed on the main lattice image, and the same T 1 pixel} downsampling and lattice image brightness interpolation are performed while shifting the start point by 1 pixel. T _One -step phase-shifted sampling moiré fringes are generated. FIG. 6 (a) is the same as FIG. 4 (c), whereas the sampling moire method produces sampling moire fringes as shown in FIG. 6 (b). The luminance distribution _Im1 (x, y, k) of this sampling moire fringe is expressed as follows.

Here, k indicates the sampling moire fringes at the kth stage, and φ _m1 (x, y) is the phase of the sampling moire fringes generated from the main lattice image in the main direction r1 at _k = 0. Further, T ₁ is a positive integer and has a sampling pitch close to the average value of the pitches of the main grid 1. For example, when the average value of the pitches in the evaluation region is p ₁ = 10.4, T ₁ = 10 is adopted.

すなわち、Ｉ_ｍ１（ｘ，ｙ，ｋ）が得られれば、この式から位相分布φ_ｍ１（ｘ，ｙ）が計算される。図６（ｂ）のようなサンプリングモアレ縞から、図６（ｃ）に示すようなサンプリングモアレ縞の位相分布が得られる。 Next, the virtual moiré generation unit 233 calculates the phase distribution from the sampling moiré fringes generated in each main direction (step S11). For example, the phase distribution φ _m1 (x, y) of the sampling moiré fringes in the _main direction r1 is calculated as follows by the phase shift method using the discrete Fourier transform (DFT) algorithm.

That is, if _{Im1 (x, y, k) is obtained, the phase distribution φ m1 (} x, y) is calculated from this equation. From the sampling moiré fringes as shown in FIG. 6 (b), the phase distribution of the sampling moiré fringes as shown in FIG. 6 (c) can be obtained.

Then, the virtual moiré generation unit 233 calculates the phase distribution of the main grid from the phase distribution of the sampling moiré fringes for each main direction (step S13). Specifically, the phase distribution φ ₁ (x, y) of the main grid in the main direction r ₁ is calculated according to the following equation.

The method of calculating the phase distribution of the main lattice from the sampling moire fringes is not only the method based on DFT, but also the fast Fourier transform (FFT), the windowed Fourier transform (WFT), and the wavelet transform. Other spatial phase analysis methods such as the method may be adopted.

ここで、Ｎ_１は、主格子１のピッチｐ_１に近く且つそれより大きい正の実数である指定値である。このようなＮ_１を用いるのは、以下の処理で生成する２次元増倍モアレ縞を元格子の分布特徴と同一にするためである。 Further, the virtual moiré generation unit 233 calculates the phase distribution of the virtual moiré from the phase distribution of the main grid for each main direction (step S15). The _phase distribution φ _dm1 (x, y) of the virtual moiré in the principal direction r1 is calculated according to the following equation.

Here, N ₁ is a designated value which is a positive real number close to and larger than the pitch p ₁ of the main grid 1. The reason for using such N ₁ is to make the two-dimensional photomultiplier moire fringes generated by the following processing the same as the distribution characteristics of the original lattice.

As mentioned above. Generating sampling moiré fringes with the integer closest to the grid pitch as the sampling pitch improves the accuracy of calculating the phase distribution of the sampling moiré fringes, but if the sampling pitch is smaller than the grid pitch, the moiré tends to be distorted and the grid. The distorted tendency of is reversed. Therefore, a virtual pitch N ₁ , which is a real number larger than the pitch of the lattice, is selected so that the distorted tendency of the moiré is the same as the distorted tendency of the lattice, and the phase distribution of the virtual moiré is calculated. Conversely, if the sampling pitch is too close to the grid pitch, the moiré spacing is too wide to easily identify the distorted characteristics of the moiré, so the _phase of the virtual moiré using the real virtual pitch N1. Is generated, and the visualization effect of the expansion phenomenon of the distorted features of the lattice is arbitrarily adjusted.

ここで、Ｎ_２は、主格子２のピッチｐ_２に近く且つそれより大きい正の実数である指定値である。 The processing up to this point is the processing described in Patent Document 1. For the principal direction r ₂ , x in the equations (3), (4), (6) and (7) may be set to y. Further, the subscript "1" may be changed to "2". The phase distribution φ _dm2 (x, y) of the virtual moiré is calculated by the following formula.

Here, N ₂ is a designated value which is a positive real number close to and larger than the pitch p ₂ of the main grid 2.

In the example of FIG. 4, as shown in FIG. 4 (e), the _phase distribution of the virtual moire in the principal direction r1 is obtained. Here, distortion due to line defects appears in the part surrounded by the square. On the other hand, as shown in _FIG . 4 (f), the phase distribution of the virtual moire in the principal direction r2 can be obtained. The phase distribution of the virtual moiré is (-π, + π]. In the processing up to this point, the distortion due to the line defect can be grasped, but the point defect cannot be detected.

Further, in the example of FIG. 5, the phase distribution of the virtual moiré in the principal direction r1 is obtained as shown in FIG. 5 (e), and the _phase distribution of the virtual moiré in the principal direction r2 is obtained as shown in _FIG . 5 (f). A phase distribution is obtained. The margins that were also present in FIGS. 5 (c) and 5 (d) are maintained in FIGS. 5 (e) and 5 (f).

The process shifts to the process of FIG. 7 via the terminal A. Here, when the rotation processing unit 232 is rotating in step S5, the rotation processing unit 232 performs reverse rotation at the same angle as the rotation in step S5 and trims the margins (step S17). In the example of FIG. 5 (e), when the reverse rotation is performed, an image as shown in FIG. 5 (g) is obtained. Further, in the example of FIG. 5 (f), when the reverse rotation is performed, an image as shown in FIG. 5 (h) is obtained. Further, by trimming the margins, an image as shown in FIG. 5 (i) can be obtained from the image of FIG. 5 (g). Further, from the image of FIG. 5 (h), an image as shown in FIG. 5 (j) can be obtained. This makes it possible to obtain an image of the same size as the original image. The image to be the target of reverse rotation and trimming may include not only the phase distribution of the virtual moiré but also other images after rotation.

ここで、Ｍ_１は、主方向ｒ_１における１次元モアレ縞の密度に対する増倍率であり、Ｍ_２は、主方向ｒ_２における１次元モアレ縞の密度に対する増倍率であり、共に正の実数である。 Then, the two-dimensional doubling moire generation unit 234 generates two-dimensional doubling moire fringes from the phase distribution of the virtual moiré (step S19). Since the moire fringes can be regarded as an enlargement phenomenon of the original lattice, two-dimensional moire fringes for simultaneously expanding the lattice in two directions, that is, two-dimensional multiplied moire fringes are generated. Furthermore, in order to emphasize slight deformation due to lattice defects, etc., a virtual multiplication of moire is also performed as necessary. That is, the luminance distribution _{Im_2D} of the two-dimensional photomultiplier moire fringes is calculated by the following formula.

Here, M ₁ is an multiplication factor for the density of one-dimensional moire fringes in the main direction r ₁ , and M ₂ is an multiplication factor for the density of one-dimensional moire fringes in the main direction r ₂ , both of which are positive real numbers. be.

M ₁ φ _dm 1 (x, y) represents the phase of the one-dimensional multiplied moire fringes in the principal direction r ₁ , and M ₂ φ _dm2 (x, y) represents the one-dimensional augmentation in the principal direction r ₂ . It represents the phase of double moire fringes. cos [M ₁ φ _dm 1 (x, y)] is the luminance distribution of the one-dimensional multiplied moire fringes in the main direction r ₁ , and cos [M ₂ φ _dm2 (x, y)] is the luminance distribution in the main direction r ₂ . It is a luminance distribution of one-dimensional multiplied moire fringes. Therefore, the equation (9) represents the average of the luminance values of the one-dimensional doubling moire fringes in the two main directions, and this is the luminance distribution of the two-dimensional doubling moire fringes. In the example of FIG. 4, two-dimensional doubling moire fringes as shown in FIG. 4 (g) can be obtained. By generating the two-dimensional doubling moire fringes in this way, it becomes possible to detect both the point defects surrounded by circles and the line defects surrounded by squares. It should be noted that the equation (9) shows an example showing the composition of the luminance distribution of the one-dimensional multiplied moire fringes in each main direction, and may be a sum instead of the average.

バーチャルモアレの位相分布は（－π，＋π］であったが、２次元増倍モアレ縞の位相分布は、（０，＋π］となる。図４の例では、図４（ｈ）に示すような位相分布が得られるようになる。この位相分布でも、丸で囲まれた点欠陥も四角で囲まれた線欠陥も検出することが出来るようになる。 Further, the two-dimensional doubling moire generation unit 234 calculates the phase distribution from the two-dimensional doubling moire fringes (step S21). The phase distribution φ _{m_2D} (x, y) of the two-dimensional doubling moire fringes is calculated by the following equation.

The phase distribution of the virtual moiré was (-π, + π], but the phase distribution of the two-dimensional photomultiplier fringes is (0, + π]. In the example of FIG. 4, as shown in FIG. 4 (h). A phase distribution can be obtained. Even with this phase distribution, it becomes possible to detect point defects surrounded by circles and line defects surrounded by squares.

Since the phase distribution of the two-dimensional multiplied moire fringes is an image in which the phase of the two-dimensional lattice is enlarged (amplified), the enlargement (amplification) rate with respect to the lattice pitch may be adjusted according to the ease of visualization. The enlargement ratio of defect visualization is expressed by the following formula.

Here, p ₁ (x, y) represents the pitch distribution in the principal direction r ₁ , and p ₂ (x, y) represents the pitch distribution in the principal direction r ₂ . d _m1 is the interval between the photomultiplier moire fringes in the main direction r1 and d _m2 is the _interval between the _{photomultiplier} moire fringes in the main direction r2. Then, M _p1 is the enlargement ratio of the multiplied moire fringes with respect to the lattice pitch of the principal direction r ₁ , and M _p2 is the enlargement ratio of the multiplied moire fringes with respect to the lattice pitch of the main direction r ₂ . From these equations, it can be seen that the magnification of defect visualization can be arbitrarily adjusted for the spacing of the two-dimensional multiplied moire fringes by adjusting M ₁ and N ₁ , M ₂ and N ₂ .

In order to retain the original characteristics of the lattice when expanding the lattice structure, it is desirable to set the enlargement ratios of the photomultiplier moiré fringes with respect to the lattice pitch to be the same or similar in the two main directions. That is, it is preferable that M _p1 = M _p2 or M _p1 ≈ M _p2 . Any M ₁ , M ₂ , N ₁ , N ₂ can be selected as long as M _p 1 and M _{p 2} can be equal or nearly equal. Typically, assuming that the _average value p1 of the pitches in the principal direction _r1 and the _average value _p2 of the pitches in the principal direction r2, the enlargement ratio of the multiplied moire fringes with respect to the lattice pitch before the multiplication is equal to that, that is, Set to | N ₁ / (N ₁ -p ₁ ) | = | N ₂ / (N ₂ -p ₂ ) | and select the same multiplication factor, that is, M ₁ = M ₂ . More specifically, N ₁ is about 1.05 to 1.2 times that of p ₁ , N ₂ is about 1.05 to 1.2 times that of p ₂ , and M ₁ = M ₂ = 3 to 6. preferable. For example, based on such a relationship, multiples of N ₁ and N ₂ with respect to the average value of the pitch, M ₁ and M ₂ may be set in advance and adjusted in some cases.

After that, the defect position detection unit 235 executes the defect position coordinate detection process (step S23). This process will be described with reference to FIGS. 8 and 9. The distorted part of the pattern in the two-dimensional doubling moire fringe or its phase distribution is the same as the position of the defect in the original image. From this, it becomes possible to detect point defects and line defects by comparing with a pattern without defects. The automatic detection can be performed by various image processing methods. Here, as an example, a method of multiplying the strain distributions of the two main grids and detecting the position coordinates of the defect by the binarized image processing of the result will be described.

ここで、φ_１（ｘ，ｙ）は、（６）式で算出される主格子１の位相分布である。主格子２については、同じ式でφ_２（ｘ，ｙ）により算出される。 First, the defect position detection unit 235 calculates the angular distribution of the main grids 1 and 2 from the phase distributions of the main grids 1 and 2 (step S31). For example, the angle distribution θ ₁ (x, y) for the main grid 1 is calculated by the following equation.

Here, φ ₁ (x, y) is the phase distribution of the main grid 1 calculated by Eq. (6). The main grid 2 is calculated by φ ₂ (x, y) using the same equation.

主格子２については、同じ式でφ_２（ｘ，ｙ）及びθ_２（ｘ，ｙ）により算出される。 Further, the defect position detection unit 235 calculates the respective pitch distributions from the angular distributions of the main grids 1 and 2 (step S33). For example, the pitch distribution p ₁ (x, y) for the main grid 1 is calculated by the following equation.

The main grid 2 is calculated by φ ₂ (x, y) and θ ₂ (x, y) using the same equation.

主格子２については、同じ式でｐ_２（ｘ，ｙ）及びｐ_{２＿Ａｖｇ}により算出される。例えば、図９（ａ）に示すような主格子１のひずみ分布が得られ、図９（ｂ）に示すような主格子２のひずみ分布が得られる。 Further, the defect position detection unit 235 calculates the strain distribution for each of the main grids 1 and 2 from the pitch distribution and the pitch average value (step S35). The pitch average value _{p1_Avg} is calculated from the pitch distribution in the evaluation region. The strain distribution of the main grid 1 is calculated by the following formula.

The main grid 2 is calculated by p ₂ (x, y) and p _{2_Avg} with the same formula. For example, the strain distribution of the main grid 1 as shown in FIG. 9 (a) can be obtained, and the strain distribution of the main grid 2 as shown in FIG. 9 (b) can be obtained.

図９の例では、図９（ｃ）に示すような画像が生成される。 Then, the defect position detection unit 235 integrates the strain distributions of the main grids 1 and 2 to generate an image for detection (step S37). Since the spots that become point defects have the characteristic that the strains of the two main grids are large or small at the same time, the strain distributions of the two main grids are used at the same time to detect the defective spots. Various methods can be adopted to integrate the strain distributions, for example, multiplying the strain distributions of the two main grids is performed. In the case of product, it is calculated by the following formula.

In the example of FIG. 9, an image as shown in FIG. 9 (c) is generated.

Then, the defect position detection unit 235 binarizes the generated image and detects the defect position coordinates (step S39). The threshold value for binarization is set in advance based on the experimental results and the like. In addition, the threshold value may be adjusted as necessary. For example, a pixel having a luminance value larger than the threshold value (large distortion) is represented by white, and a pixel having a luminance value equal to or lower than the threshold value (small distortion) is represented by black. Then, in the example of FIG. 9, the image as shown in FIG. 9D is obtained. Large strains are detected in three places. The process returns to the caller's process.

Returning to the description of the process of FIG. 7, the output unit 25 executes defect position display on the two-dimensional double moire fringe or its phase distribution (step S25). For example, when a circle mark is overlaid on a two-dimensional double moire fringe at a defect position as shown in FIG. 9 (d), the display as shown in FIG. 9 (e) is obtained. Not only line defects but also point defects can be properly grasped. It should be noted that the two-dimensional doubling moire fringes or the phase distribution itself thereof may be output.

Regarding the integration of the strain distribution in step S37, not only the product but also the absolute value of the product (Equation (17)), the sum (Equation (18)), and the absolute value of the sum ((19)) as shown below. )) May be used.

Further, the two-dimensional multiplied moire fringe or the distorted portion of the phase distribution thereof may be detected by using another image processing method or an image recognition algorithm.

By adopting such a configuration, it is possible to detect both point defects and line defects in a wide field of view by moire. It is possible to analyze crystals or periodic structures on any scale including periodic structure distributions such as squares, rectangles, triangles, or hexagons. Moreover, since the analysis can be performed from the image of the single-shot sample, it can be easily applied to the image taken during the dynamic test. Furthermore, since the moiré phase analysis is performed, it is resistant to random noise of the camera and has high measurement accuracy.

This embodiment is used for defect detection of periodic structures of materials and structures on various scales in fields such as semiconductor devices, automobiles, aerospace, biomedicine, and materials science. Objects that can be analyzed include semiconductors, metals, composite materials, biomaterials, and the like.
Typical applications in the industrial field mainly include:
(1) Elucidation of the material damage mechanism by detecting point defects such as vacancies, substituted atoms, and interstitial atoms in the crystal structure of materials such as metals and semiconductors, line defects such as dislocations and slips, and heterophase atomic clusters. It can be used to improve the process of reducing the defect rate.
(2) For improving the manufacturing process by detecting non-uniformity, defects, deviations, interfaces, and measuring strain distribution in periodic structures such as graphene, nanoparticles, nanoparticles, nanorods, nanotubes, microsheaves, and nanoimprint molds. Available.
(3) It can be used for displacement, displacement, and strain measurement of a structure having a periodic structure under an external force (a pattern is attached to the surface or an artificial pattern is present on the surface).

[Example 1] Simulation verification of point defect detection FIG. 10 shows the results of simulation and verification of point defect detection by moire analysis. First, an intersecting grid having the principal direction r ₁ as the x direction and r ₂ as the y direction was generated with 532 × 428 pixels. The pitch of the two main grids is about 10.4 pixels. Three types of point defects, vacancies, substitution atoms, and interstitial atoms, were introduced into the two-dimensional lattice arrangement (Fig. 10 (a)). In order to simulate a TEM image, FIG. 10 (b) in which Gaussian noise with σ = 50% (noise amplitude is 50% of the lattice amplitude) was superimposed was used as the image to be analyzed.

The main lattice in the x-direction and the y-direction is separated from the analysis image by a low-pass filter, and the one-dimensional sampling moire phase in the x-direction and the y-direction is calculated by the equation (5) using the sampling pitch T ₁ = T ₂ = 10 pixels. Each was calculated. Using virtual pitches N ₁ and N ₂ larger than the pitch of the main grid, the phase distributions of the one-dimensional virtual moiré in the x and y directions were calculated by Eqs. (7) and (8), respectively. As an example, FIG. 10 (c) shows a one-dimensional virtual moire phase in the y direction when the virtual pitch N ₂ = 12 pixels. From such a one-dimensional virtual phase distribution, it is difficult to detect and identify point defects.

On the other hand, when the virtual pitch N ₁ = N ₂ = 12 pixels and the multiplication factor for the density of the one-dimensional virtual moire fringes is M ₁ = M ₂ = 6, the 2 obtained by the equations (10) and (11) can be obtained. The phase distribution of the dimensionally multiplied moire fringes is as shown in FIG. 10 (d). It was confirmed that the point defects surrounded by circles could be clearly detected, and that their types could be visually identified.

[Example 2] Simulation of adjusting the enlargement ratio of the two-dimensional doubling moire fringes The enlargement ratio of the interval of the two-dimensional doubling moire fringes with respect to the _two -dimensional lattice pitch is set to a virtual pitch (N) in the _main directions r1 and r2. ₁ and N ₂ ) and can be arbitrarily adjusted according to the multiplication factor (M ₁ and M ₂ ) with respect to the density of the one-dimensional moire fringes. For the grid of FIG. 10B, FIG. 11 shows the phase distribution of two-dimensional photomultiplier moire fringes at different magnifications. In FIG. 11, the two-dimensional magnification moire fringes when the virtual pitch N ₁ = N ₂ is changed to 11, 11.5, 12 pixels and the magnification M ₁ = M ₂ is changed to 1, 3, 5 respectively. The phase distribution of is shown. In this example, a setting of M ₁ = M ₂ = 5, N ₁ = N ₂ = 11.5 or 12 pixels is more likely to find point defects than other combinations. With such an appropriate combination of N ₁ , N ₂ , M ₁ , and M ₂ , point defects can be easily detected.

In FIG. 11, an example of M ₁ = M ₂ and N ₁ = N ₂ is shown, but in FIG. 12, two-dimensional photomultiplier moire fringes are shown when M _x and My and N _x and N _y are different _values . Shows the phase distribution.

FIG. 12A shows an image of a periodic structure in which point defects and noise are introduced. The pitch in the x direction is 10.4 pixels, and the pitch in the y direction is 1.05 times the pitch in the x direction. Since there is a slight angle between the principal directions r ₁ and the _x direction and there is no rotation of the image during analysis, the enlargement ratio parameters of this example are called M _x and My, N _x and N _y . And.

In the case of a grid in which the pitch in the x direction and the pitch in the y direction are different as shown in FIG. 12 (a), M _x and My, and N _x and N _y are multiplied by the same value as shown in _FIG . 12 (b). In the phase distribution, the aspect ratio changes greatly, making it difficult to find the location of point defects. Therefore, as shown in FIG. 12 (c), it is different when the virtual pitch in the y direction is not 1.05 times the virtual pitch in the x direction (that is, N _x = N _y = 12 pixels). By selecting the multiplication factor (M _x = 3, _{My =} 6) for the density of the one-dimensional moiré fringes, the magnification ratio of the doubling moiré fringes with respect to the lattice pitch can be the same magnification in both directions.

Further, as shown in FIG. 12 (d), when the virtual pitch in the y direction is 1.05 times the virtual pitch in the x direction (that is, N _x = 12 pixels, N _y = 12.6 pixels). If the multiplication factor with respect to the density of the one-dimensional moire fringes is M _x = My, the enlargement ratios of the augmented moiré fringes with respect to the lattice pitch in the x direction and the _y direction are substantially the same. With these adjustment methods, even if the pitch of the object in the x direction and the pitch in the y direction are different, it becomes possible to cope with it.

[Example 3] Detection and verification of atomic vacancy from TEM photograph of Si single crystal sample The two-dimensional atomic structure is enlarged from the TEM photograph (FIG. 13 (a)) of the Si single crystal structure in which defects are introduced by electron beam irradiation. Two-dimensional doubling moire fringes were generated, and the phase distribution of the two-dimensional doubling moire fringes was calculated (FIG. 13 (b)). From the singular points in the phase pattern of the two-dimensional multiplied moire fringes, the point defects of the Si crystal lattice could be detected. When the enlarged view of the TEM image of the detected point defect position was observed, the existence of atomic vacancies was confirmed, and the effectiveness of the two-dimensional photomultiplier moire method in the present embodiment was confirmed.

Although the embodiments of the present invention have been described above, the present invention is not limited thereto. For example, the processing flow is an example, and as long as the processing result does not change, the order of the steps may be changed or a plurality of steps may be executed in parallel. Further, the functional configuration example of FIG. 1 is also an example, and may not match the program module configuration. Further, the defect detection device 20 may be mounted on one computer or may be mounted on a plurality of computers. Further, the image pickup device 10 and the defect detection device 20 may be integrated or may be provided at a remote location.

The defect detection device 20 described above is a computer device, and as shown in FIG. 14, a memory 2501, a CPU (Central Processing Unit) 2503, a hard disk drive (HDD) 2505, and a display device. The display control unit 2507 connected to the 2509, the drive device 2513 for the removable disk 2511, the input device 2515, and the communication control unit 2517 for connecting to the network are connected by a bus 2519. The HDD may be a storage device such as a solid state drive (SSD). The operating system (OS) and the application program for carrying out the processing according to the embodiment of the present invention are stored in the HDD 2505 and read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. .. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing contents of the application program to perform a predetermined operation. Further, the data in the middle of processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In an embodiment of the technique, the application program for performing the processes described above is stored and distributed on a computer-readable removable disk 2511 and installed from the drive device 2513 to the HDD 2505. It may be installed in the HDD 2505 via a network such as the Internet and a communication control unit 2517. Such a computer device realizes various functions as described above by organically cooperating with the hardware such as the CPU 2503 and the memory 2501 described above and the program such as the OS and the application program. ..

The data used by executing the processing as described above is stored in a storage device such as a memory 2501 or an HDD 2505 regardless of whether the data is in the middle of processing or is the processing result.

The embodiments described above can be summarized as follows.

In the image processing method according to the present embodiment, (A) from the image of the sample, the main direction is the orthogonal axis in each direction of the predetermined orthogonal axis or according to the main direction of the lattice on the surface of the sample. A step of generating a grid image by a predetermined filter process for each direction of the orthogonal axis when the image is rotated so as to be parallel to any of them, and (B) sampling moire from the grid image in each of the above directions. A step of calculating the fringes and calculating the phase distribution of the virtual one-dimensional moiré fringes when the pitch is a virtual specified pitch based on the sampling moire fringes, and (C) a predetermined phase distribution for each of the above directions. It includes a step of generating a two-dimensional doubling moire fringe obtained by synthesizing the brightness distribution of the moire fringes due to the doubled phase distribution.

By executing such image processing, it becomes possible to detect point defects. In addition, line defects can be detected. It should be noted that the directions include not only the case where they completely match but also the case where they are approximate even if they do not completely match.

The image processing method may further include (D) a step of calculating the phase distribution of the two-dimensional doubled moire fringes from the two-dimensional doubled moire fringes. Even with such a phase distribution, point defects can be detected. In addition, line defects can be detected.

The image processing method includes (E) a step of calculating the strain distribution of the grid in each of the directions from the phase distribution of the grid in the direction calculated based on the sampling moire fringes in each of the directions, and (F) each of the directions. It may further include a step of detecting defect position coordinates from an image in which the strain distribution of the grid is integrated. This makes it possible to automatically detect the positions of point defects and line defects.

The image processing method may further include (G) a step of displaying the detected defect position coordinates on the image of the two-dimensional multiplied moire fringes. It becomes easier to recognize the defect position in the two-dimensional photomultiplier moire fringe.

The image processing method includes a step of calculating the strain distribution of the grid in each direction from the phase distribution of the grid in the direction calculated based on the sampling moire fringes in each direction, and a grid in each direction. The step of detecting the defect position coordinates from the image in which the strain distribution of the above is integrated and the step of displaying the detected defect position coordinates on the image showing the phase distribution of the two-dimensional doubling moire fringes are further included. Is also good. It becomes easier to recognize the defect position in the phase distribution of the two-dimensional photomultiplier moire fringes.

The steps for calculating the strain distribution are (e1) the step of calculating the angular distribution of the grid in each direction from the phase distribution of the grid in the grid direction calculated based on the sampling moire fringes in each direction, and (e1). e2) For each of the above directions, a step of calculating the grid pitch distribution from the grid angle distribution and (e3) for each of the above directions may include a step of calculating the grid strain distribution from the grid pitch distribution. The strain distribution can be obtained relatively easily, but the strain distribution may be obtained by another method.

The steps for detecting the defect position coordinates are (f1) a step of integrating the strain distributions of the lattice in each direction by a predetermined method to generate an image, and (f2) a predetermined step in the generated image. It may include a step of specifying the position coordinates of the pixel of the luminance value having a predetermined relationship with the threshold value. The predetermined method is realized by, for example, a product, an absolute value of the product, a sum, an absolute value of the sum, and the like. The predetermined relationship is that the higher the luminance value is, the larger the strain is, the larger the distortion is.

A program for causing a computer to execute the image processing method described above can be created, and the program is stored in various storage media.

Further, the device that executes the image processing method as described above may be realized by one computer or by a plurality of computers, and may be realized by a combination of them in an image processing system or an image processing system. It shall be simply called a system.

100 Defect detection system 10 Imaging device 20 Defect detection device 21 Data storage unit 23 Image processing unit 25 Output unit 231 Preprocessing unit 232 Rotation processing unit 233 Virtual moire generation unit 235 Two-dimensional magnification moire generation unit 235 Defect position detection unit

Claims (9)

From the image of the sample, rotate the image so that the principal direction is parallel to any of the orthogonal axes in each direction of the predetermined orthogonal axis or according to the principal direction of the lattice on the surface of the sample. In this case, a step of generating a grid image by a predetermined filter process for each direction of the orthogonal axis, and
A step of calculating sampling moire fringes from the image of the grid for each of the directions and calculating the phase distribution of virtual one-dimensional moire fringes when the pitch is set to a virtual designated pitch based on the sampling moire fringes. ,
A step of generating a two-dimensional photomultiplier moire fringe obtained by synthesizing the luminance distribution of the moire fringe by the phase distribution obtained by multiplying the phase distribution by a predetermined direction in each direction.
Image processing methods performed by the computer, including. The image processing method according to claim 1, further comprising a step of calculating the phase distribution of the two-dimensional doubled moire fringes from the two-dimensional doubled moire fringes. A step of calculating the strain distribution of the grid for each direction from the phase distribution of the grid in the direction calculated based on the sampling moire fringes for each direction.
A step of detecting defect position coordinates from an image in which the strain distribution of the grid in each direction is integrated, and
The image processing method according to claim 1, further comprising. The image processing method according to claim 3, further comprising a step of displaying the detected defect position coordinates on the image of the two-dimensional doubled moire fringes. A step of calculating the strain distribution of the grid for each direction from the phase distribution of the grid in the direction calculated based on the sampling moire fringes for each direction.
A step of detecting defect position coordinates from an image in which the strain distribution of the grid in each direction is integrated, and
A step of displaying the detected defect position coordinates on an image showing the phase distribution of the two-dimensional photomultiplier fringes, and
The image processing method according to claim 2, further comprising. The step of calculating the strain distribution is
A step of calculating the angular distribution of the grid for each direction from the phase distribution of the grid in the grid direction calculated based on the sampling moire fringes for each direction.
A step of calculating the pitch distribution of the grid from the angle distribution of the grid for each of the directions,
A step of calculating the strain distribution of the grid from the pitch distribution of the grid in each of the directions,
The image processing method according to claim 3 or 5. The step of detecting the defect position coordinates is
A step of integrating the strain distributions of the grid in each direction by a predetermined method to generate an image, and
In the generated image, a step of specifying the position coordinates of a pixel having a luminance value having a predetermined relationship with a predetermined threshold value and a step of specifying the position coordinates.
The image processing method according to claim 3 or 5. A program for causing a computer to execute the image processing method according to any one of claims 1 to 7. From the image of the sample, rotate the image so that the principal direction is parallel to any of the orthogonal axes in each direction of the predetermined orthogonal axis or according to the principal direction of the lattice on the surface of the sample. In this case, a means for generating a grid image by a predetermined filter process in each direction of the orthogonal axis, and
As a means for calculating sampling moire fringes from the image of the grid for each of the directions and calculating the phase distribution of virtual one-dimensional moire fringes when the pitch is a virtual designated pitch based on the sampling moire fringes. ,
A means for generating a two-dimensional photomultiplier moire fringe obtained by synthesizing a luminance distribution of a moire fringe with a phase distribution obtained by multiplying the phase distribution by a predetermined direction in each direction.
Image processing system with.

Images