WO2023238384A1

WO2023238384A1 - Device and method for observing sample

Info

Publication number: WO2023238384A1
Application number: PCT/JP2022/023459
Authority: WO
Inventors: 晟伊藤; 敦宮本; 直明近藤; 洋彦木附
Original assignee: 株式会社日立ハイテク
Priority date: 2022-06-10
Filing date: 2022-06-10
Publication date: 2023-12-14
Also published as: TW202349338A

Abstract

A problem exists in the prior art in which it is difficult to perform learning such that images of different image quality are estimated for individual regions by a single loss function when the ideal image quality varies by region of an estimated image. In this method and device for observing a sample, a first learning image and a second learning image corresponding to the first learning image are acquired, estimation processing parameters for an estimation engine for estimating the second learning image from the first learning image are learned using the first learning image and the second learning image, an estimated image estimated from the first learning image and the second learning image corresponding to the first learning image are divided into areas Ri (i = 1 to N, and N is the number of areas) during learning of the estimation processing parameters, and the loss of pixel groups Pi from the second learning image and pixels groups Qi from the estimated image which are included in each of the areas Ri is learned using a loss function Fi for performing evaluation by means of a predetermined standard.

Description

Sample observation device and method

The present invention relates to a method and apparatus for imaging a sample such as a semiconductor wafer using a charged particle microscope or the like to obtain an image for sample observation, and also relates to a method and apparatus for estimating a higher quality image from the captured image. It is related to the device. A device and a method are provided for estimating images whose image quality differs depending on the region when learning the correspondence between the first learning image and the second learning image in advance by a machine learning method.

In semiconductor wafer manufacturing, it is important to quickly start up the manufacturing process and move quickly to a high-yield mass production system in order to secure profits. For this purpose, various inspection devices, observation devices, and measurement devices are introduced into production lines.

A sample observation device is a device that takes a high-resolution image of the defect location on a wafer based on the defect location coordinates (coordinate information indicating the location of the defect on the sample) output by the inspection device, and outputs the image. A sample observation device using a scanning electron microscope (SEM) (hereinafter referred to as review SEM) is widely used. Automation of observation work is desired on semiconductor mass production lines, and the review SEM is equipped with a function to perform automatic defect image collection processing (ADR: Automatic Defect Review), which automatically collects images at defect positions within a sample.

There are many types of circuit pattern structures formed on semiconductor wafers, and the defects that occur also vary in type and location. It is important to output. Therefore, an image for sample observation is obtained by increasing the visibility using image processing technology for a raw image obtained by converting a signal obtained from a detector of a review SEM into an image. As a way to improve visibility, many methods have been proposed in which the correspondence between images of different image quality is learned in advance, and when an image similar to one image quality is input, the image of the other image quality is estimated. . For example, Japanese Patent Application Publication No. 2018-137275 (Patent Document 1) discloses a method of estimating a high magnification image from a low magnification image by learning in advance the relationship between an image captured at a low magnification and an image captured at a high magnification. It is described in.

Japanese Patent Application Publication No. 2018-137275

In sample observation equipment, it is important to output images with high visibility in order to observe defects, circuit patterns, etc., and it is possible to apply image processing to captured images to improve image visibility. It is being said. One method is to estimate a high-quality image using a captured image as an input by learning the relationship between a captured image and a high-quality image in advance using a machine learning method. Generally, in machine learning methods, the loss between an estimated image and a high-quality image is calculated using a single predefined loss function, and estimation processing parameters are updated to reduce the loss. However, during observation, the desired image quality may vary depending on the observer, as the desired area to observe differs depending on the observer. In this case, it is difficult to learn to estimate images with different image quality for each region using a single loss function.

In order to solve the above-mentioned problems, the present invention provides a visual inspection method and a visual inspection system having the following features.
That is, a sample observation method and apparatus,
obtaining a first learning image and a second learning image corresponding to the first learning image;
learning estimation processing parameters of an estimation engine that estimates a second learning image from the first learning image using the first learning image and the second learning image;
In learning the estimation processing parameters, the estimated image estimated from the first learning image and the second learning image corresponding to the first learning image are divided into regions Ri (i=1 to N, N: number of regions). death,
Learning is performed using a loss function Fi that evaluates the loss between the pixel group Pi of the second learning image and the pixel group Qi of the estimated image included in each region Ri using a predetermined standard.

According to the present invention, it is possible to change the loss calculation method in learning of the estimation engine for each region, and the performance of the image estimation engine is improved.

FIG. 3 is a diagram showing an example of a learning processing sequence of an estimation engine according to the present invention. FIG. 2 is a diagram for explaining details of the area division process of the embodiment shown in FIG. 1; 3 is a diagram illustrating an image of a three-layer circuit pattern according to Example 1. FIG. According to Example 1, the estimated image and the second learning image corresponding to the first learning image are divided into a defective region R1'' and a normal region R2'' without defects, and in R1'', the loss function F1'' is In R2'', the estimation processing parameters are learned using the loss function F2''. 1 is a diagram illustrating a configuration in which a label is attached to each pattern and a background part using layout information acquired from design data, and a label image is acquired according to the first embodiment; FIG. 3 is a diagram for explaining a loss function Fi according to Example 1. FIG. The loss function Fi is defined as a weighted sum of elemental losses calculated by a plurality of elemental loss functions fij (j = 1 to M, M is the number of elemental losses), and the elemental loss value is calculated for each region Ri of the second learning image. change the weight wij of FIG. 3 is a diagram showing an example of a Graphical User Interface (GUI) according to the present embodiment. FIG. 2 is a diagram showing an example of a neural network having a three-layer structure according to the present embodiment. FIG. 7 is a diagram illustrating a first learning image captured using one additional frame and a second learning image captured using 64 additional frames, according to the present embodiment.

Embodiments for implementing the present invention will be described below with reference to the drawings. The embodiments described below do not limit the claimed invention, and all of the elements and combinations thereof described in the embodiments are essential to the solution of the invention. Not necessarily.

This embodiment is a sample observation method and apparatus, in which a first learning image and a second learning image corresponding to the first learning image are acquired, and the first learning image and the second learning image are The estimation processing parameters of the estimation engine that estimates the second training image from the first training image are learned using the training image, and in learning the estimation processing parameters, the estimated image estimated from the first training image and the corresponding The second learning image corresponding to the first learning image is divided into regions Ri (i=1 to N, N: number of regions), and the pixel group Pi of the second learning image included in each region Ri and the estimated image This is an embodiment of a sample observation method and apparatus that learns using a loss function Fi that evaluates the loss with a pixel group Qi based on a predetermined standard.
(1)
That is, in Example 1, a first learning image and a second learning image corresponding to the first learning image are acquired,
learning estimation processing parameters of an estimation engine that estimates a second learning image from the first learning image using the first learning image and the second learning image;
In learning the estimation processing parameters, the estimated image estimated from the first learning image and the second learning image corresponding to the first learning image are divided into regions Ri (i=1 to N, N: number of regions). death,
This is an example of a configuration in which learning is performed using a loss function Fi that evaluates the loss between the pixel group Pi of the second learning image and the pixel group Qi of the estimated image included in each region Ri using a predetermined criterion.

　Additional information about this feature. Since the estimation engine updates the estimation parameters based on the loss, the image quality of the image output by the estimation engine after learning changes depending on the loss function used during learning. Conventional methods apply a single loss function to the entire image and learn the parameters of the estimation engine, making it difficult to learn to estimate images with different image quality depending on the region. In this embodiment, by changing the loss function for each region, it is possible to estimate images with different image quality for each region.

Hereinafter, the features of this embodiment will be explained in more detail.

FIG. 1 shows an example of the learning processing sequence of the estimation engine according to this embodiment. In the figure, a sample is imaged and a plurality of pairs of a first learning image (100) and a second learning image (101) are acquired.

Here, when acquiring the first learning image and the second learning image, one or more of the image resolution, the number of added frames, and the focus position are changed so that the second learning image becomes the first learning image. Acquire images with higher quality than the training images. For example, as shown in FIG. 9, a first learning image (900) captured with the number of addition frames of 1 and a second learning image (901) captured with the number of addition frames of 64 may be used.

Next, the first learning image and the second learning image are aligned to obtain an aligned first learning image (103) and an aligned second learning image (106). Note that in alignment, normalized correlation, pixel difference, or the like may be used as an evaluation value, and alignment may be performed based on the position where the evaluation value is maximum or minimum. Furthermore, if the image resolutions of the first learning image and the second learning image are different, alignment may be performed after aligning the image resolutions by linear interpolation or the like before alignment.

The aligned first learning image (103) is input to the estimation engine (104) to obtain an estimated image (105). Region division processing (107) is applied to the aligned second learning image (106) to obtain regions R1 (108) to region RN (109). Next, for each region Ri (i=1 to N, N: number of regions), using a different loss function Fi, a pixel group Pi of pixels of the aligned second learning image 106 and a pixel group of pixels of the estimated image 105 are calculated. Calculate the loss with group Qi.

That is, in region R1, loss is calculated using loss function F1, and in region RN, loss is calculated using loss function FN different from F1. The loss of the entire image is calculated (112) by adding up the losses (110, 111) for each region, and the estimation processing parameters of the estimation engine (104) are updated based on this loss.

Note that various existing machine learning methods can be used as the estimation engine (104), such as a deep neural network.

As one method using a deep neural network, a convolutional neural network described in Non-Patent Document 1 may be used. Specifically, a neural network having a three-layer structure as shown in FIG. 8 may be used. Here, Y is the input image, F1(Y) and F2(Y) are intermediate data, and F(Y) is the estimation result. Note that the intermediate data and final results are calculated using Equations 1 to 3. However, "*" represents a convolution operation. Here, W1 is n1 filters of size c0×f1×f1, c0 represents the number of channels of the input image, and f1 represents the size of the spatial filter. An n1-dimensional feature map is obtained by convolving the input image with a filter of size c0×f1×f1 n1 times. B1 is an n1-dimensional vector and is a bias component corresponding to n1 filters. Similarly, W2 is a filter of size n1×f2×f2, B2 is a vector of n2 dimensions, W3 is a filter of size n2×f3×f3, and B3 is a vector of c3 dimensions.
F1(Y)=max(0, W1*Y+B1)...(Formula 1)
F2(Y)=max(0, W2*F1(Y)+B2)...(Formula 2)
F(Y)=W3*F2(Y)+B3...(Formula 3)
Among these, c0 and c3 are values determined by the number of channels of the first learning image and the second learning image. Further, f1, f2, n1, and n2 are hyperparameters determined by the user before the learning sequence, and may be set to, for example, f1=9, f2=5, n1=128, and n2=64. The parameters to be adjusted through learning by the estimation engine (104) are W1, W2, W3, B1, B2, and B3.

Note that there are various methods for region division depending on the purpose. Specific examples of the area division method include the following.
(A1) Division into edge areas and non-edge areas (A2) Division into upper layer pattern areas and lower layer pattern areas (A3) Division into defect areas and normal (non-defect) areas (A4) Label the design data in advance. , Region division based on labels In this specification, (A1) to (A4) described above have been explained as examples of region division, but region division is not limited to this, and can be performed according to the purpose or user specification. It is possible to divide the area by any method based on the above.
(2)
In images for sample observation, it is important to estimate strong contrast and fine features in edge areas such as the outline of circuit patterns, while on the other hand, it is important to estimate defects and circuit patterns in flat areas without irregularities (non-edge areas). In order to avoid erroneous recognition, it is important that the estimated image has weak contrast or that small elements such as noise have been removed.

To solve this problem, in this embodiment, in addition to the feature (1) described above, in the region segmentation process (104), a brightness gradient image is obtained by applying a differential filter to the second learning image, and the brightness gradient image is Using the gradient image and the edge determination threshold, the first learning image, the corresponding estimated image, and the second learning image are divided into the edge region R1 and non-edge region R2 of the circuit pattern (A1), and the loss is calculated in R1. It is characterized in that the estimation processing parameters are learned using the function F1 and the loss function F2 in R2.

This feature will be supplemented using FIG. 2.
In the figure, a brightness gradient image (203) is obtained by applying a differential filter (202) to a second learning image (201). Various methods can be used for the differential filter, but for example, the sum of the absolute value of the difference between the target pixel and the pixel on the right, and the absolute value of the difference between the target pixel and the pixel adjacent below, is used as the differential filter result. You can also use it as Next, in edge determination processing (205), the brightness gradient image (203) is binarized using the brightness gradient image (203) and the edge determination threshold (204), and the edge region R1 (206) is and a non-edge region R2 (207). Note that the edge determination threshold value (204) may be set by the user or may be set according to a predetermined rule. Due to the above characteristics, learning is performed using different loss functions for edge regions and non-edge regions, and it becomes possible to perform learning to estimate images of different image quality for edge regions and non-edge regions, respectively.
(3)
In images for sample observation, when a sample has a multilayer circuit pattern, it may be necessary to estimate an image with high contrast in a specific layer. To solve this problem, in this embodiment, in addition to the features described in (1) and (2), in the region segmentation process (107), the second learning image and the layer determination threshold are used to The estimated image and second learning image corresponding to are divided into regions R1' of the first layer pattern to region RN' of the Nth layer pattern, and Ri' (i=1 to N, N: number of regions) is a loss function. Estimated processing parameters are learned using Fi'.

This feature (3) will be supplementarily explained using an image taken of a three-layer circuit pattern shown in FIG. 3 as an example.
By inputting the second learning image (301) and the layer determination threshold (302) to the layer determination process (303), the 1st layer pattern region R1' (304), the 2nd layer pattern region R2' (305) and the 3rd layer pattern region R2' (305) It is divided into layer pattern regions R3' (305).

For example, when a semiconductor wafer having a multilayer circuit pattern is imaged using a charged particle microscope, the pattern in the upper layer is generally brighter, and the pattern in the lower layer is darker. Therefore, in the layer determination process, it is possible to separate the luminance of the second learning image (301) by converting it into N values based on the layer determination threshold (302). With this feature, by learning using a different loss function for each region divided into each layer, it is possible to learn to estimate images with different image quality depending on the layer, that is, images with high contrast in a specific layer. becomes.
(4)
In images for sample observation, when a defect exists in a sample, visibility of the defect area is important, and an image with high contrast of the defect may be required. Furthermore, it is important to estimate an image with less noise so as to avoid erroneously recognizing a defect in an area without a defect, and the items that are considered important in image estimation may differ depending on the presence or absence of a defect.

To solve this problem, in this embodiment, in addition to the above-mentioned characteristics, a first learning image and a second learning image are acquired based on defect coordinate information,
Acquire a reference image that corresponds to the second learning image and does not include defects,
A defective area in the second learning image is detected using the second learning image and the reference image, and based on the detected area, the estimated image and the second learning image corresponding to the first learning image are converted to the defective area R1. It is characterized in that the process is divided into '' and a normal region R2'' without defects, and the estimation processing parameters are learned using the loss function F1'' for R1'' and the loss function F2'' for R2''.

This feature (4) will be supplemented using FIG. 4.
A second learning image (401) and a region where a circuit pattern similar to the second learning image is formed are imaged to obtain a reference image (402), and the second learning image and the reference image are compared. A defective area is detected (403), and the second learning image is divided into a defective area R1'' (404) and a defect-free area R2'' (405).

This feature makes it possible to perform learning using different loss functions in the defective region R1'' and the defect-free region R2'', and learn to estimate an image with high defect visibility and low noise in the defect-free region. It becomes possible to do so.
(5)
In images for sample observation, it is sometimes necessary to estimate images with high contrast within a specific pattern. To solve this problem, in addition to the above-mentioned features, the present invention further adds a label to the design data, aligns either the first learning image or the second learning image with the design data, and calculates the position. The second learning image is divided into regions Ri (i=1 to N, N: number of regions) based on the matching result and the label, and estimation processing is performed using the loss function Fi''' in the region Ri'''. Equipped with the feature of learning parameters.

Here, design data refers to layout data on the sample to be observed. For example, if the sample is a semiconductor, the design data is data in which edge information of a designed shape of a semiconductor circuit pattern is written as coordinate data.

This feature (5) will be supplemented using FIG. 5.
Using the layout information (501) acquired from the design data, labels are attached to each pattern and background (502), and a label image is acquired (503). In the example of FIG. 5, three types of labels are provided. Further, the design data (501) and the second learning image (504) are compared to obtain a position (506) corresponding to the second learning image (505). Regions R1''' to R3' are divided by performing region division processing (507) on the second learning image (504) based on the design data (506) and label image (503) corresponding to the second learning image. ''(508-510) is obtained.

In the corresponding position acquisition process, the contour information of the pattern may be acquired by extracting edges from the second learning image, for example, and the corresponding position may be searched for by comparing it with design data.
With this feature, by learning using a different loss function for each region corresponding to the label given to the design data, it becomes possible to learn to estimate an image with a high contrast of a specific pattern.
(6)
There are various factors involved in image quality, such as brightness and contrast, and it is difficult to express it using a single loss function. Therefore, in this embodiment, in addition to the feature described in (1), the loss function Fi is a weighted sum of elemental losses calculated by a plurality of elemental loss functions fij (j = 1 to M, M is the number of elemental losses). It is characterized in that the weight wij of the element loss value is changed for each region Ri of the second learning image.

This feature (6) will be supplemented using FIG. 6. Although the explanation will be given using the loss F1 (P1, Q1) as an example, the present invention can be applied to any loss Fi (Pi, Qi).
Using the estimated image (600) and the second learning image as input, element loss functions F11 to F1M (602 to 603) are calculated. The weights w11 to w1M (604) of each element loss indicating which element loss is considered important and each element loss (602 to 603) are input, and the loss F1 (P1, Q1) is calculated (605).

Here, the loss F1 (P1, Q1) is calculated by a weighted sum of each element loss shown in the following formula.
F1 (P1, Q1) = w11f11 (P1, Q1) + w12f12 (P1, Q1) +
...+w1Mf1M (P1, Q1)
The element loss function is a method of calculating the loss (element loss) corresponding to each element of image quality. For example, in the loss function Fi, the element loss function regarding brightness is the absolute square error (Pi-Qi)^2, contrast The element loss function for the brightness gradient is expressed as the absolute square error of the brightness gradient (Pi'-Qi')^2. Note that Pi' is the brightness gradient of the pixel group Pi, and Qi' is the brightness gradient of the pixel group Qi.

Due to the above characteristics, by calculating and learning for each region Ri a weighted sum of multiple element losses calculated using multiple element loss functions, it is possible to learn to estimate an image that reflects multiple elements related to image quality. becomes possible.
(7)
FIG. 7 shows an example of a Graphical User Interface (GUI) when implementing this embodiment.

With this GUI, it is possible to set a determination threshold for region division and specify the importance level of each region.
Region segmentation is performed on the second learning image (700), and the defect region (701), edge region (702), non-edge region (703), first layer region (704), and second layer region (705) are Is displayed. Further, it is possible to set a threshold value used for region division (706, 707). It is also possible to set each area (708, 710, 712) and the degree of importance of each element in each area (709, 711, 713). By setting the weight (604) of each element loss function based on the importance level set here, it is possible to reflect it on the loss.

Note that the present invention is not limited to the embodiments described above, and further includes various modifications. For example, the above-described embodiments and modified examples have been described in detail to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described.

Claims

A sample observation method and apparatus, comprising:
obtaining a first learning image and a second learning image corresponding to the first learning image;
learning estimation processing parameters of an estimation engine that estimates a second learning image from the first learning image using the first learning image and the second learning image;
In learning the estimation processing parameters, the estimated image estimated from the first learning image and the second learning image corresponding to the first learning image are divided into regions Ri (i=1 to N, N: number of regions). death,
Learning using a loss function Fi that evaluates the loss between the pixel group Pi of the second learning image and the pixel group Qi of the estimated image included in each region Ri using a predetermined standard.
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to claim 1, wherein when capturing the first learning image and the second learning image,
The second learning image is characterized in that the quality is higher than the first learning image by changing one or more of the image resolution, the number of added frames, and the focus position.
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to claim 2,
The estimation engine is characterized by using a convolutional neural network,
The method is characterized in that the estimation processing parameters are updated by error backpropagation processing so that the loss calculated by the loss function is reduced.
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to claim 3,
A brightness gradient image is obtained by applying a differential filter to the second learning image, and the estimated image estimated from the first learning image and the second The training image is divided into an edge region R1 and a non-edge region R2 of the circuit pattern, and the estimation processing parameters are learned using the loss function F1 in R1 and the loss function F2 in R2.
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to claim 3,
Using the second learning image and the layer determination threshold, the estimated image estimated from the first learning image and the second learning image are converted to the region R1' of the first layer pattern to the region RN' of the Nth layer pattern. In Ri', the estimation processing parameters are learned using the loss function Fi' (i = 1 to N, N: number of regions).
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to claim 3,
Obtaining a first learning image and a second learning image based on the defect coordinate information,
Acquire a reference image that corresponds to the second learning image and does not include defects,
A defective area in the second learning image is detected using the second learning image and the reference image, and based on the detected area, the estimated image estimated from the first learning image and the second learning image are used as the defective area. Divide into R1'' and a normal region R2'' with no defects, and learn the estimation processing parameters using the loss function F1'' for R1'' and the loss function F2'' for R2''.
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to claim 3,
Assign labels to design data,
, align either the first learning image or the second learning image with the design data,
Divide the second learning image into regions Ri''' (i=1 to N, N: number of regions) based on the alignment result and the label,
In the region Ri''', the estimation processing parameters are learned using the loss function Fi'''.
A method and apparatus for observing a sample, characterized in that:
The method and apparatus for observing a sample according to any one of claims 3 to 7,
The loss function Fi is defined as a weighted sum of elemental losses calculated by a plurality of elemental loss functions fij (j=1 to M, M: number of elemental losses), and has different weights for each region Ri of the second learning image. A method and apparatus for observing a sample, characterized by comprising the following.