WO2024069701A1

WO2024069701A1 - Model generating method and defect inspection system

Info

Publication number: WO2024069701A1
Application number: PCT/JP2022/035715
Authority: WO
Inventors: 大地稲富; 貴裕浦野; 淳二山本; 展明広瀬; 野央波多; 洋憲櫻井
Original assignee: 株式会社日立ハイテク
Priority date: 2022-09-26
Filing date: 2022-09-26
Publication date: 2024-04-04

Abstract

The present invention has: a first step for reading defect candidate images used for training deep learning models; a second step for assigning labels to a portion of the defect candidate images to create teaching images and evaluation images; a third step for, after updating the parameters for each of a plurality of the deep learning models using teaching images, evaluating the classification accuracy using the evaluation images, and selecting the best model with the highest classification accuracy among the plurality of deep learning models; and a fourth step for classifying defect candidate images that have not been assigned a label using the best model, and select a portion of the defect candidate images for assigning labels on the basis of the DOI-likeness according to the best model, and executes the second step and the third step again after execution of the fourth step.

Description

Model generation method and defect inspection system

The present invention relates to a technology that uses a deep learning model to classify defect candidates into defects of interest (DOI) and nuisance.

Semiconductor devices are manufactured by subjecting wafers made of silicon and other materials to multiple processes to form fine circuits. In the manufacturing process of semiconductor devices, visual inspections may be performed before and after each process in order to improve and stabilize yields. Among the equipment used for visual inspection, equipment used for visual inspection of wafers after circuit patterns have been formed detects defects such as pattern defects or foreign objects based on reference images and inspection images obtained by shining lamp light, laser light, or electron beams on areas corresponding to two patterns that were originally formed to have the same shape. Specifically, the difference between the reference image and the inspection image is calculated, and areas where the difference is greater than a separately determined threshold value are detected as defect candidates.

In this type of inspection, the lower the threshold value, the smaller the defects that can be detected. However, lowering the threshold value will result in many false reports due to errors during image capture, roughness, minute differences in the pattern, or differences in brightness due to uneven film thickness. Such false reports are called nuisance. In contrast, defects that customers want detected are called DOIs (Defects of Interest).

Patent Document 1 discloses that in order to eliminate false reports due to patterns, a threshold value is set to detect defect candidates based on the signal variation calculated for each area in the chip. In addition, examples of technology for removing nuisance from defect candidates are disclosed in Patent Documents 2 and 3, for example.

JP 2000-105203 A JP 2012-112915 A JP 2014-149177 A

In the conventional technologies disclosed in Patent Documents 2 and 3, nuisance contained in defect candidates is analyzed and removal is achieved by designing features to identify the nuisance. Therefore, if the type of wafer to be inspected is changed or the process is changed, and the nuisance to be removed changes, it is necessary to redesign the features.

Designing features requires several weeks to several months of work, so it takes time to respond to changes in what is to be removed. On the other hand, applying deep learning methods to distinguish between DOI and Nuisance makes it easier to respond to changes in what is to be removed. When applying deep learning methods, classification rules are automatically generated by teaching example images and training the deep learning model, eliminating the need to design features to identify Nuisance.

However, when using deep learning to classify defect candidate images into DOI and Nuisance, in order to enable classification with high accuracy, it is necessary to design an optimal deep learning model for each type of image to be classified. Here, the image type is a combination of an image of the background pattern and an image of the defect candidate. In addition, it is necessary to prepare several thousand labeled images as training data for each class (in this case, DOI and Nuisance), and preparing the training data takes time and effort.

The model generation method according to one embodiment of the present invention is a model generation method for generating a deep learning model that classifies defect candidate images into DOI and Nuisance, and includes a first step of preparing a plurality of deep learning models with different hyperparameters in advance and reading the defect candidate images to be used for training the deep learning models, a second step of labeling some of the defect candidate images to create teaching images and evaluation images, a third step of evaluating the classification accuracy using the evaluation images after updating the parameters of each of the plurality of deep learning models using the teaching images, and selecting the best model with the highest classification accuracy from the plurality of deep learning models, and a fourth step of classifying the defect candidate images that have not been labeled using the best model, and selecting a portion of the defect candidate images to be labeled based on the DOI-likeness according to the best model, and after executing the fourth step, executing the second and third steps again.

　According to the present invention, even if the Nuisance to be removed changes, a deep learning model capable of removing Nuisance can be obtained in a short time, so that highly sensitive defect inspection can always be performed. Other issues and novel features will become apparent from the description of this specification and the accompanying drawings.

1 is a diagram illustrating an example of a configuration of a defect inspection device. 2 is an example of a hardware configuration of an information processing device. 2 is a diagram illustrating an example of a configuration of an image acquisition unit. 2 is an example of an inspection target and an acquired image of the defect inspection device. 2 is an outline of an inspection operation of the defect inspection device. 1 is an example of a deep learning model generation process. 13 is another example of a deep learning model generation process. 1 is an example of a configuration of a defect inspection system. 1 is an example of a timing chart of a deep learning model generation process. 13 is an example of an operation screen. 13 is an example of an operation screen. 13 is an example of an operation screen. 13 is an example of an operation screen. 13 is an example of an operation screen.

The following describes in detail an embodiment of the present invention with reference to the drawings. In all drawings used to explain the embodiment, the same components are generally designated by the same reference numerals, and repeated explanations will be omitted.

1 shows an example of the configuration of the defect inspection device 100 of this embodiment. The defect inspection device of this embodiment is a device that detects defects present on the surface of a sample based on a signal obtained by irradiating the sample with electromagnetic waves such as light or a charged particle beam such as an electron beam. For example, it includes a bright-field inspection device that irradiates the sample with light and detects defects based on the reflected light, a dark-field inspection device that irradiates the sample with light and detects defects based on the scattered light, and an electron beam inspection device (including a device called a review SEM) that detects defects based on secondary electrons obtained by irradiating the sample with an electron beam. The defect inspection device 100 of this embodiment is configured with an image acquisition unit 200, a control unit 102, a calculation unit 103, a storage unit 104, an input/output unit 105, and a communication unit 106 as main functional blocks. The input/output unit 105 is connected to an input/output device 110. The input/output device 110 includes input devices such as a keyboard and a pointing device, and a display device such as a display. The communication unit 106 is connected to a model generation device 610. The image acquisition unit 200 acquires inspection image data of the semiconductor wafer. The calculation unit 103 extracts defect candidate images based on the feature amount of the image transferred from the image acquisition unit 200, performs nuisance removal processing by deep learning described later, and transmits data on the remaining defect candidates (hereinafter referred to as defect candidate data) to the control unit 102. The defect candidate data includes, for example, image data, coordinates on the sample indicating the position where the image was acquired, evaluation value (DOI likeness), and other information. The control unit 102 stores the received defect candidate data in the storage unit 104, transmits data of the received defect candidate data that can be used to create inspection conditions and check results to the input/output unit 105, and transmits data used to generate a model for nuisance removal described later to the communication unit 106. The input/output unit 105 processes the received defect candidate data into a form that can be visually confirmed by humans and transmits it to the input/output device 110. The input/output device 110 outputs the received defect candidate data on a screen. The communication unit 106 transmits the defect candidate data to the model generation device 610.

The functional blocks of the control unit 102, the calculation unit 103, the memory unit 104, the input/output unit 105, and the communication unit 106 of the defect inspection device 100 are realized by the information processing device 101. The information processing device 101 includes a processor (CPU) 121, a memory 122, a storage device 123, an input/output port 124, a network interface 125, and a bus 126 as shown in FIG. 2A. The processor 121 functions as a functional unit (functional block) that provides a predetermined function by executing processing according to a program loaded in the memory 122. The storage device 123 stores data and programs used in the functional unit. For the storage device 123, a non-volatile storage medium such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) is used. The input/output port 124 is connected to the input/output device 110 and executes the exchange of signals between the information processing device 101 and the input/output device 110. The network interface 125 enables communication with other information processing devices via a network. The other information processing devices include a model generating device 610. These components of the information processing device 101 are communicatively connected to each other via a bus 126.

FIG. 2B shows an example of the configuration of the image acquisition unit 200 of the defect inspection device 100. Here, an example of an optical inspection device that uses light to detect defects is shown. The image acquisition unit 200 is composed of a stage 210, an illumination optical system 220, a detection optical system 230, an image sensor 240, and a signal processing unit 250. The sample 211 is an object to be inspected, such as a semiconductor wafer. The stage 210 mounts the sample 211 and is capable of moving within the XY plane, rotating (θ), and moving in the Z direction.

The illumination optical system 220 irradiates the sample 211 with light 221. When the light 221 hits the sample 211, reflected light 222 and scattered light 223 are generated from the sample 211.

The detection optical system 230 directs the reflected light 222 or scattered light 223 toward the imaging surface of the image sensor 240. The image sensor 240 captures the scattered light 223. The detection optical system 230 may include a spatial filter that cuts out light resulting from a pattern that is repeated at a constant cycle. The image sensor 240 transmits an imaging signal to the signal processing unit 250.

The signal processing unit 250 processes the imaging signal received from the image sensor 240 to generate an observation image of the surface of the sample 211. An observation image obtained by irradiating the sample 211 with light 221 from diagonally above is called a dark-field image. Note that a method may also be used in which the observation image is obtained by capturing reflected light 222 from the sample 211 above the sample 211. An observation image obtained by this method is called a bright-field image.

Figure 3 shows a schematic plan view of the sample 211. When the sample 211 is a patterned semiconductor wafer, a circuit pattern of a semiconductor chip is formed on the surface of the sample 211. Each semiconductor chip before being separated from the semiconductor wafer is called a die. The dies have the same circuit pattern. Although there is no particular limitation on the inspection method, here, an image of the entire surface of the semiconductor wafer is taken and divided into images of a predetermined size. Hereinafter, this divided image is called an observation image. Since the dies D301 to D304 in Figure 3 have the same circuit pattern, if there is no defect, the observation images of the regions P305 to P308 occupying the same coordinates on the die will be the same. On the other hand, if, for example, only the observation image of region P305 among the observation images of regions P305 to P308 has a large brightness value, there is a possibility that, for example, a foreign object exists at the position of region P305. Therefore, the observation image of region P305 is extracted as a defect candidate image. Here, we have explained an example of extracting defect candidate images using the difference in luminance (brightness) of the observed images as a feature, but the feature is not limited to this, and multiple feature values can also be used.

FIG. 4 is a diagram showing an overview of the inspection operation of the defect inspection device 100. When the inspection starts, the sample 211 is loaded into the defect inspection device 100 (S400). Next, the image acquisition unit 200 captures the observation image (S401). Next, the calculation unit 103 performs a feature calculation process on the acquired observation image, such as calculating the difference by comparing the brightness of the observation image of an area occupying the same coordinates on the die (S402), and extracts defect candidates by comparing with a separately calculated threshold value and performing an operation such as a filter process using the feature (S403). In this embodiment, further, using the deep learning model generated by the model generation device 610, nuisance is removed from the defect candidates extracted in step S403 to identify the defect (S404), and the inspection result is output (S405).

An example of the deep learning model generation process will be described using the flow in FIG. 5A. This process is executed by the model generation device 610. First, the defect inspection device 100 reads the defect candidate image acquired by executing the flow in FIG. 4 (S500). If multiple defect inspection devices each acquire defect candidate images, the defect candidate images from those defect inspection devices may be read. The defect candidate image may be a defect candidate image extracted by inspection at normal inspection sensitivity, or a defect candidate image extracted by inspection with extremely increased inspection sensitivity. In the latter case, the defect candidate image extracted in step S403 will contain a large amount of Nuisance, but this embodiment includes a step (S404) of removing Nuisance using a deep learning model, so the inspection results are not degraded.

Next, about 10 images are randomly selected from the loaded defect candidate images (S501). Alternatively, in order to efficiently proceed with model learning (described later), images may be selected so that the image features of the selected defect candidate images are sparse. For example, if the image feature is the brightness of the defect candidate images, the defect candidate images are selected so that they include a wide range of defect candidate images from high brightness to low brightness. The selected defect candidate images are displayed on the GUI of the input/output device 110 (S502), and a user operation is accepted (S503). Next, a label of "DOI" or "Nuisance" is assigned to each defect candidate image according to the user's operation, and classification is performed (S504). It is determined whether the number of images labeled "DOI" and the number of images labeled "Nuisance" are each equal to or greater than a prescribed number (S505). The prescribed number is set to, for example, 5 or more. If the specified number is not met, the process returns to the selection of images to be displayed (S501), and if the specified number is met, the defect candidate images are divided into training images and evaluation images (S506). This division is performed so that the proportions of each class (images labeled "DOI" and images labeled "Nuisance") contained in the training images and evaluation images are equal.

Next, a deep learning model for classifying DOI and Nuisance is trained. As mentioned above, the optimal network configuration of the deep learning model may differ for each image type. For this reason, multiple models are prepared in advance in the model generation device 610. For example, when using a CNN (Convolutional Neural Network) for judgment, multiple CNNs with different hyperparameters are prepared. Multiple CNNs are trained using training images, and the CNN that provides the most accurate judgment is selected as the model for classifying DOI and Nuisance for that image type. This makes it possible to handle a variety of image types. The flow will be explained below.

First, the parameters of a number of models prepared in advance are updated (trained) using the training image (S507), and the evaluation image is classified using these models (S508). The models prepared in advance may be models that have been previously trained using other test results, or may be untrained models, or may include both. The model outputs, for example, the probability that an input image is DOI (hereinafter referred to as DOI-likeness), and a threshold value that can best classify the evaluation image into DOI and Nuisance is also recorded. The DOI-likeness may be determined based on the probability of Nuisance. Next, the model with the highest performance when classifying the evaluation image is selected as the best model (S509). For example, the accuracy of the model can be evaluated using the AUC (Area Under Curve) of the ROC curve as an index of the classification accuracy of the model. Next, it is determined whether the classification accuracy (e.g., AUC) of the selected best model reaches a target value set in advance (S510). The target value may be changed by the user.

If the target value is not reached, all images that are not labeled by the best model, i.e., not classified as "DOI" or "Nuisance", are classified (S513), and several images each with the highest DOI-likeness (images with the highest DOI-likeness or close to it), images with the DOI-likeness closest to the threshold recorded in step S508 (images with the DOI-likeness at or close to the threshold), and images with the lowest DOI-likeness (images with the lowest DOI-likeness or close to it) are selected (S514). Note that the selection method is not limited to this, and several images may be selected so that they are evenly spaced when sorted in order of DOI-likeness determined by the best model. Images are selected based on some criteria using DOI-likeness. Alternatively, images may be selected based on image features. In either case, images are selected to cover the spread of DOI-likeness or image features in all images and to avoid arbitrary bias. In deep learning, inference results for types of images that have not been learned are ambiguous. As described above, by selecting images to be used as training images so as to cover the entire variability of the image type, and repeating the process of training using the created training images, the accuracy of the model can be efficiently improved. After the selection of the display image (S514) is completed, the image is displayed (S502), an operation is accepted (S503), and a label is assigned (S504), and then the process returns to dividing the image data (S506).

On the other hand, if the classification accuracy of the best model reaches a preset target value, the best model is determined as a deep learning model that classifies the image types of defect candidate images into DOI and Nuisance. Furthermore, all images that are not labeled with the best model may be classified (S511). Images whose DOI-likeness determined by the best model exceeds the threshold value recorded in step S508 may be output as DOI, and which defect candidate images have been determined to be DOI may be stored in a memory unit (S512). The classification of DOI and Nuisance is performed by comparing the probability that the model outputs a DOI (DOI-likeness) with a threshold value, and the threshold value used is the value recorded in step S508, but the user may adjust the threshold value.

Another example of the deep learning model generation process will be described using the flow in Figure 5B. The same parts as in the flow in Figure 5A will not be described. The difference from the flow in Figure 5A is that it includes an accuracy judgment by the user (S520). Even if it is determined that the classification performance of the evaluation images using the best model does not reach the target, if the user looks at the images during labeling (S504) and feels that the classification accuracy is high enough, they can perform an operation to indicate completion, and the process can proceed to classifying all images using the best model (S511) and outputting the classification results (S512). In step S520, the classification accuracy of the best model may be displayed on the operation screen, and the user may make a judgment by looking at the numerical value.

FIG. 6 shows an example of the system configuration of a defect inspection system 600 of this embodiment. The defect inspection system 600 has one or more inspection devices 100-1 to 100-3, a model generation device 610, and an input/output device 620. The input/output device 620 includes input devices such as a keyboard and a pointing device, and a display device such as a display. The functional blocks of the communication unit 611, control unit 612, calculation unit 613, storage unit 614, and input/output unit 615 of the model generation device 610 are realized by an information processing device. The hardware configuration of the information processing device is the same as that of the information processing device 101 shown in FIG. 2A, so repeated explanations will be omitted.

When generating a model, defect candidate images are acquired using inspection devices 100-1 to 3. At this time, one inspection device or some or all of a group of inspection devices may be used. Furthermore, the sample inspected by the inspection device may be, for example, the same semiconductor wafer, or different semiconductor wafers on which dies with the same circuit pattern are formed. Inspection devices 100-1 to 3 transmit information on the defect candidates they have generated to communication unit 611 of model generation device 610 via a network. Using the transmitted defect candidate images, model generation device 610 generates a deep learning model that inspection device 100 uses in step S404 of defect inspection, following the flow shown in FIG. 5A or FIG. 5B.

The calculation unit 613 uses the received defect candidate image to train one or more untrained models or pre-trained models stored in the storage unit 614. As described above, the storage unit 614 stores a plurality of deep learning models with different hyperparameters in advance. At this time, in order to obtain a labeled image necessary for model training, the calculation unit 613 transmits the defect candidate image selected as a labeling target from among the unlabeled images to the input/output unit 615. The input/output unit 615 displays the received defect candidate image on the display device of the input/output device 620 (S502). The input device of the input/output device 620 accepts a user's operation and transmits a signal for labeling to the input/output unit 615. The input/output unit 615 uses the received signal to label the defect candidate image and transmits the label to the calculation unit 613 and the storage unit 614. The calculation unit 613 updates the parameters of the deep learning model using the received labeled image (S507). Each time the parameter update is completed, the calculation unit 613 may evaluate the model using the labeled image stored in the storage unit 614 and check the accuracy. When the user performs an operation to complete learning (Yes in S510 or S520), the control unit 612 transmits the best model at that time to each of the inspection devices 100-1 to 100-3 via the communication unit 611.

The timing for updating the model used by the inspection devices 100-1 to 100-3 in step S404 is arbitrary. For example, it may be triggered by the results of an accuracy evaluation performed by the calculation unit 613 using the labeled images stored in the memory unit 614. The inspection devices 100-1 to 100-3 use the received trained model to remove nuisance from the results of subsequent inspections and identify defects. The received deep learning model may be used repeatedly.

FIG. 7 shows the user's work (labeling (S504)) and the processing of the model generating device 610 in the deep learning model generation processing shown in FIG. 5A or FIG. 5B, and the transition of the number of labeled images accompanying these processes. As explained in the flow of the deep learning model generation processing, the model generating device 610 updates the parameters of multiple deep learning models using labeled images (training images) obtained by the user's labeling work. Since it takes time to update the parameters (S507) and to infer the DOI-likeness of all unlabeled images (S513) necessary to select the next training image, if each operation is performed in series, there is a risk of a waiting time occurring between the end of the user's work (S504) and the time when the model generating device 610 performs these processes and the next labeling target image is displayed (S502). In order to reduce the occurrence of such a waiting time, it is preferable to execute the user's work and the processing of the model generating device in parallel, as shown in the timing chart of FIG. 7.

After the user's first labeling operation 701 is completed and the model generating device 610 has completed the parameter update operation 702 and the inference operation 703, an inference operation 705 is executed in parallel with the user's second labeling operation 704 to select the next image to be labeled. The

inference operations

703 and 705 are inferences based on the same model, so the same inference results are obtained. However, since the image that was the subject of the second labeling operation 704 is not subject to inference in the inference operation 705, an image that has not yet been subjected to labeling processing can be selected as the image to be labeled by the inference operation 705. The next image to be labeled selected by the inference operation 705 is displayed as soon as the user's second labeling operation 704 is completed, so that the user can move on to the third labeling operation 706. In addition, the next parameter update operation 707 and inference operation 708 are started in parallel at the same time that the third labeling operation 706 is started.

In this way, by performing the initial inference twice in the repeated labeling and learning/inference, it becomes possible for the labeling work by the user and the learning/inference processing of the device to be performed in parallel. This makes it possible to hide the time required for the processing of the model generating device 610 and improve processing efficiency. Note that instead of performing the initial inference twice, two sets of target images for the labeling work may be selected for the initial inference.

FIGS. 8A-D show an example of the operation screen of this embodiment. By making the operation screen a highly operable GUI, the time required for teaching can be shortened. At least several images 801-804 to be labeled and corresponding label selection buttons 811-814 and an operation completion button group 830 are displayed in a pop-up window 800, which is the operation screen. The pop-up window 800 may display an image switching control button group 820 shown in FIG. 8A. Alternatively, the image display method adjustment button group 840 shown in FIG. 8B may be displayed. Alternatively, the division boundary selection buttons 860-1-5 shown in FIG. 8D may be displayed. The operation of this operation screen will be described below with reference to FIG. 8A-D.

The images to be labeled selected in steps S501 and S514 are displayed on the operation screen in order of their DOI resemblance. The number of images to be displayed is about ten to several tens of images. If all images cannot be displayed at once in the pop-up window 800, a scroll operation is accepted so that the displayed images to be labeled can be switched. Label selection buttons 811 to 814 corresponding to the images to be labeled 801 to 804 are arranged near the displayed images to be labeled, and the label to be assigned to each image to be labeled can be selected by clicking the mouse pointer 810 on the buttons. The label selection buttons can be radio buttons as shown in FIG. 8 (here, DOI is represented as "A" and Nuisance as "B"), with the initial state being DOI or Nuisance, and the image to which the opposite label is to be assigned can be clicked with the mouse pointer 810, or they can be check boxes so that only DOI or Nuisance can be clicked. As shown in FIG. 8C, a rectangular selection can be performed by dragging the mouse pointer 810, and the labels of the images inside the rectangular area can be inverted. Alternatively, as shown in FIG. 8D, clicking one of the division boundary selection buttons 860-1 to 860-5 with the mouse pointer 810 may invert the labels of all images displayed to the left or right of that boundary. In this case, if an individual label selection button 811 to 814 is clicked before clicking the division boundary selection button 860-1 to 860-5, processing may be performed that does not invert the labels of that image.

When an operation completion button 831 located in the group of operation completion buttons 830 is clicked with the mouse pointer 810, a label specified by the user is assigned to the displayed images 801-804 to be labeled, they are saved, and the next image to be labeled is displayed. Also, a teaching completion button 832 may be placed in the group of operation completion buttons 830. Clicking this button with the mouse pointer 810 may end the labeling process and allow the user to proceed to the next process.

As shown in FIG. 8A, defect candidate images 801-1 to 804-1 to be labeled and corresponding reference die images 801-2 to 804-2 may be displayed alternately. The defect candidate images are grouped based on coordinate information on the die, and the reference die image is selected from the observation images of the area occupying the same coordinates on the die (see FIG. 3). Image switching may be stopped by clicking an image switching on/off button 822 arranged in the image switching control button group 820 shown in FIG. 8A with the mouse pointer 810. Also, the displayed images may be switched at the time input into a switching speed adjustment box 821 arranged in the image switching control button group 820 or set by the button.

Also, if the brightness of the images 801 to 804 to be labeled displayed in the pop-up window 800 can be adjusted, it becomes easier for the user to make a judgment. The image quality of the images to be labeled can be adjusted by the group of image display method adjustment buttons 840 shown in FIG. 8B. For example, the group of image display method adjustment buttons 840 may have a box 841 for setting the maximum brightness and a box 842 for setting the minimum brightness. Brightness may be adjusted by setting pixels equal to or higher than the tone set as the maximum brightness as white and pixels equal to or lower than the tone set as the minimum brightness as black, and allocating pixel values of the tones in between evenly between the maximum tone and the minimum tone on the display of the input/output device 620.

Note that in the examples of Figures 8A to 8D, the images to be labeled are displayed side-by-side, but they may also be displayed side-by-side vertically.

Figure 9 shows an example of the configuration of another operation screen. In the left area 910 of the pop-up window 900, a group of labeling target images 911-1 to 4 that are inferred to have a high DOI-likeness are displayed, and in the right area 920, a group of labeling target images 921-1 to 4 that are inferred to have a low DOI-likeness are displayed. Images that have been incorrectly classified can be selected and corrected by clicking with the mouse pointer 810 on the arrows 912-1 to 4 or arrows 922-1 to 4 that correspond to the images.

The present invention is not limited to the above-described embodiment, but includes various modifications. For example, the above-described embodiment has been described in detail to make the present invention easier to understand, and is not necessarily limited to having all of the configurations described.

For example, in the embodiment, an inspection device that detects defects on a semiconductor wafer on which a pattern is formed is used as an example for explanation, but nuisance can also be removed in an inspection device that detects defects on a semiconductor wafer on which no pattern is formed (hereinafter referred to as a surface inspection device). A surface inspection device is a device that detects defects such as foreign matter and scratches on the surface by applying light such as a laser to the surface of a sample such as a wafer. When inspecting the surface of a wafer using a surface inspection device, the sample is fixed on a rotating stage or a stage that moves in the XYZ directions using, for example, a vacuum chuck or a holding device, and light is applied to the sample surface while moving the sample, and the reflected light and scattered light generated at that time are observed. Foreign matter and scratches on the sample surface are detected by detecting the difference between when the surface of the sample is as expected (defect-free state) and when foreign matter or scratches are present. The surface inspection device detects defects by comparing the signal waveforms obtained, and can remove nuisance using a trained model. For classifying the data to train the model, data obtained by converting waveform information into an image may be used, the waveform information may be used as is, coordinate information on the sample may be used, or a combination of these pieces of information may be used.

In the embodiment, an example was described in which the generation of a model for removing nuisance was performed by a model generation device connected to the defect inspection device, but the model may also be generated by installing model generation software in the information processing device of the defect inspection device.

100: defect inspection device, 101: information processing device, 102: control unit, 103: calculation unit, 104: memory unit, 105: input/output unit, 106: communication unit, 110: input/output device, 121: processor (CPU), 122: memory, 123: storage device, 124: input/output port, 125: network interface, 126: bus, 200: image acquisition unit, 210: stage, 211: sample, 220: illumination optical system, 221: light, 222: reflected light, 223: scattered light, 230: detection optical system, 240: image sensor, 250: signal processing unit, 600: defect inspection system, 610: model generation device, 611: communication unit, 612: control unit, 613: calculation unit, 614: memory unit, 615: input/output unit, 620: input/output device, 701, 70 4, 706: labeling operation, 702, 707: parameter update processing, 703, 705, 708: inference processing, 800, 900: pop-up window, 801, 802, 803, 804, 911, 921: image to be labeled, 810: mouse pointer, 811, 812, 813, 814: label selection button, 820: image switching control button group, 821: switching speed adjustment box, 822: image switching on/off button, 830: operation completion button group, 831: operation completion button, 832: teaching completion button, 840: image display method adjustment button group, 841, 842: box, 832: teaching completion button, 860: division boundary selection button, 910: left side area, 920: right side area, 912, 922: arrows.

Claims

A model generation method for generating a deep learning model that classifies defect candidate images into DOI and Nuisance,
Multiple deep learning models with different hyperparameters are prepared in advance,
A first step of reading defect candidate images for use in training a deep learning model;
a second step of labeling a portion of the defect candidate image to generate a teaching image and an evaluation image;
a third step of evaluating classification accuracy using the evaluation image after updating parameters for each of the plurality of deep learning models using the teaching image, and selecting a best model having the highest classification accuracy from among the plurality of deep learning models;
a fourth step of classifying the defect candidate images that have not been labeled by the best model, and selecting a portion of the defect candidate images to be labeled based on a DOI likelihood according to the best model;
A model generating method comprising the steps of: executing the fourth step, and then executing the second step and the third step again.
In claim 1,
If the best model selected in the third step does not satisfy a desired classification accuracy, the fourth step is performed following the third step;
A model generation method in which, if the best model selected in the third step satisfies the desired classification accuracy, the best model is a deep learning model that classifies the image type of the defect candidate image into DOI and Nuisance.
In claim 1,
In the third step, a threshold value of DOI-likeness at which the best model classifies the evaluation image into DOI and Nuisance is recorded;
A model generation method in which the defect candidate images to be labeled in the fourth step are selected so as to include images whose DOI-likeness evaluated by the best model is at or near the threshold value, images whose DOI-likeness evaluated by the best model is at or near the highest value, and images whose DOI-likeness evaluated by the best model is at or near the lowest value.
In claim 1,
A model generation method in which, when the second to fourth steps are repeatedly executed, the third step and the second step are executed in parallel for a portion of the defect candidate images selected by the fourth step.
In claim 1,
In the second step after the execution of the fourth step, a window is displayed on a display device to allow a user to select whether a part of the defect candidate image is a DOI or a Nuisance;
A model generation method in which defect candidate images displayed in the window are displayed in order of DOI likelihood evaluated by the best model.
In claim 5,
A model generating method in which the window is provided with a button for selecting whether the defect candidate image displayed in the window is a DOI or a Nuisance.
In claim 6,
A model generation method in which division boundary selection buttons are provided in the window between the defect candidate images displayed in the window, and by specifying one of the division boundary selection buttons, one of the defect candidate images divided by the division boundary selection button is selected as DOI, and the other defect candidate image is selected as Nuisance.
In claim 1,
In the second step after the execution of the fourth step, a window is displayed on a display device to allow a user to specify whether a part of the defect candidate image is a DOI or a Nuisance;
A model generation method in which the defect candidate images displayed in the window are divided into a group evaluated by the best model as having a high DOI likelihood and a group evaluated as having a low DOI likelihood.
A defect inspection system including a defect inspection device and a model generation device connected to the defect inspection device and configured to generate a deep learning model for classifying a defect candidate image acquired by the defect inspection device into a DOI and a Nuisance,
The defect inspection apparatus includes an image acquisition unit that acquires an observation image, and an information processing device that calculates a feature amount of the observation image to extract a defect candidate image, and identifies, from among the extracted defect candidate images, a defect candidate image that is classified as a DOI using the deep learning model, as a defect;
The model generation device includes a storage unit in which a plurality of deep learning models having different hyperparameters are stored in advance, and a calculation unit that executes first to fourth steps;
The first step includes reading defect candidate images to be used for training a deep learning model;
The second step is to label a part of the defect candidate image to generate a teaching image and an evaluation image;
The third step includes updating parameters of each of the deep learning models using the training image, evaluating classification accuracy using the evaluation image, and selecting a best model having the highest classification accuracy from among the deep learning models;
The fourth step classifies the defect candidate images that have not been labeled by the best model, and selects a portion of the defect candidate images to be labeled based on the DOI likelihood by the best model;
A defect inspection system, comprising: a step of executing the second step and the third step again after the fourth step is executed.
In claim 9,
If the best model selected in the third step does not satisfy a desired classification accuracy, the calculation unit executes the fourth step following the third step,
A defect inspection system characterized in that, if the best model selected in the third step satisfies the desired classification accuracy, the best model is determined as the deep learning model to be used in the defect inspection device.
In claim 9,
A defect inspection system, wherein the defect candidate image includes an image of a background pattern and an image of the defect candidate.
In claim 9,
A defect inspection system, characterized in that the image acquisition unit acquires the observed image based on a signal obtained by irradiating an inspection target sample with an electromagnetic wave or a charged particle beam.