WO2023188803A1

WO2023188803A1 - Automatic defect classification device

Info

Publication number: WO2023188803A1
Application number: PCT/JP2023/003386
Authority: WO
Inventors: 辰彦坪井
Original assignee: 東レエンジニアリング株式会社; 東レエンジニアリング先端半導体Ｍｉテクノロジー株式会社
Priority date: 2022-03-30
Filing date: 2023-02-02
Publication date: 2023-10-05
Also published as: JP2023147373A

Abstract

Provided is an automatic defect classification device that facilitates correction of errors in classification categories assigned in advance by a user and enables improvement in the accuracy of automatic defect classification, even when there are many defect images constituting training data.　The automatic defect classification device is for generating, by using training data in which classification categories clearly indicating associations of a plurality of defect images with defect classifications have been assigned to the defect images in advance by the user, classifiers corresponding to the defect images and for, by using the classifiers, automatically assigning the classification categories to defect images of which classification categories are unknown. The automatic defect classification device comprises: a training data input unit that receives input of training data; a processing unit that performs a predetermined process on the plurality of defect images; and a display unit that displays the defect images. The processing unit causes the display unit to perform display in a changed display format of defect portions of the defect images in accordance with the classification categories which have been assigned by the classifiers and with which the defect images are associated.

Description

Automatic defect classification device

The present invention creates a classification standard using training data consisting of a plurality of defect images to which a classification category has been assigned by the user, and automatically assigns a classification category to a defective image whose classification category is unknown based on the classification standard. The present invention relates to an automatic defect classification device for determining.

For example, a semiconductor device is formed on a single semiconductor wafer by layering a large number of semiconductor device circuits (that is, repeated external patterns of device chips), and then is singulated into individual chip components. Components are packaged and shipped individually as electronic components or incorporated into electrical products.

Before each chip component is separated into pieces, a single semiconductor wafer is held and film formation, exposure/development, etching, smoothing processing, etc. are repeatedly performed, and during this process, the formation of Defect detection and defect type classification are performed for device chips that have been developed (for example, Patent Document 1).

Japanese Patent Application Publication No. 2007-071586

Classification standards are created using training data to automatically determine classification categories for defective images with unknown classification categories. Granted. Therefore, if there is an error in the user's assignment of classification categories to the training data, the classification accuracy will decrease.

On the other hand, the accuracy of automatic classification improves as the number of multiple defective images that make up the training data increases, but since it involves visual inspection, it becomes harder to notice mistakes as the number of images increases, so it is difficult to improve classification accuracy. was difficult.

Therefore, the present invention has been made in view of the above problems, and
The purpose of the present invention is to provide an automatic defect classification device that can easily correct errors in classification categories assigned in advance by a user and improve the accuracy of automatic defect classification even if there are many defect images that constitute training data. do.

In order to solve the above problems, one aspect of the present invention is as follows:
A classifier for defective images is generated using training data in which a classification category that clearly indicates which defect classification the defective image belongs to is assigned in advance by the user to each of the plurality of defective images, and the classifier is used to generate a classifier for the defective image. In an automatic defect classification device that automatically assigns a classification category to a defect image whose classification category is unknown,
a teacher data input section for inputting teacher data;
a processing unit that performs predetermined processing on the plurality of defect images;
Equipped with a display section that displays defect images,
The processing section changes the display format of the defective part of the defective image on the display section according to the classification category to which the defective image belongs, assigned by the classifier, and causes the defective image to be displayed.

Moreover, in order to solve the above-mentioned problem, another aspect according to the present invention,
A classifier for defective images is generated using training data in which a classification category that clearly indicates which defect classification the defective image belongs to is assigned in advance by the user to each of the plurality of defective images, and the classifier is used to generate a classifier for the defective image. In an automatic defect classification device that automatically assigns a classification category to a defect image whose classification category is unknown,
a teacher data input section for inputting teacher data;
a processing unit that performs predetermined processing on the plurality of defect images;
Equipped with a display section that displays defect images,
The processing unit changes and displays the display format of the defective part of the defective image on the display unit in accordance with the classification accuracy of the classification category to which the defective image belongs, assigned by the classifier.

Even if there are many defect images that constitute the training data, it is possible to easily correct errors in the classification categories assigned in advance by the user and improve the accuracy of automatic defect classification.

It is an image diagram showing an example of a defect image in a form embodying the present invention. 1 is a schematic diagram showing an overall configuration including an example of a form embodying the present invention. FIG. 1 is a schematic diagram showing a main part of an example of a form embodying the present invention. FIG. 3 is an image diagram showing an example of a display unit in a form embodying the present invention. FIG. 2 is a flow diagram showing an example of a form of embodying the present invention. FIG. 3 is an image diagram showing an example of a display unit in a form embodying the present invention. FIG. 3 is an image diagram showing an example of a display unit in a form embodying the present invention. It is an image figure which shows another example in the form which embodies this invention.

Embodiments for carrying out the present invention will be described below with reference to the drawings.

FIG. 1 is an image diagram showing an example of a defect image in an embodiment of the present invention.
FIGS. 1A to 1D respectively illustrate defect images P1 to P4 including defects X1 to X4. Here, the classification categories of defects X1 to X4 are respectively A defect, B defect, C defect, and D defect.
On the other hand, images P1' to P4' without defects are respectively illustrated in FIGS. 1E to 1H for reference.

FIG. 2 is a schematic diagram showing the overall configuration including an example of a form embodying the present invention.
FIG. 2 shows the overall configuration of an inspection system 100 including the automatic defect classification device 1 according to the present invention.

The inspection system 100 captures an inspection image P for inspecting an inspection object W, inspects the presence, number, position, defect type, etc. of defects X in the inspection image P, and outputs the inspection results.
Specifically, the inspection system 100 includes, in addition to the automatic defect classification device 1, an imaging section 110, a transport section 120, a defect detection section 130, a control section 140, and the like.

The transport unit 110 holds the inspection target W and moves it at a predetermined speed or stops it at a predetermined location.
Specifically, the transport section 110 includes a workpiece holding section that holds the inspection object W, an XYθ stage section that moves and rotates the inspection object W in the horizontal direction, and the like.

The imaging unit 120 captures an image of a moving or stationary inspection object W, and outputs an inspection image P.
Specifically, the imaging section 120 includes an optical system main body section 121 including a lens barrel and a lens, an illumination section 122, and an imaging camera 123.
More specifically, the imaging unit 120 may be one that captures an image in a wide viewing range using a low-magnification lens using a monochrome camera or the like, or one that captures an image in a narrow viewing range using a high-magnification lens using a color camera or the like.

The defect detection unit 130 acquires the inspection image P output from the imaging unit 120 and detects the defect X in the inspection image P.
Specifically, the defect detection unit 130 inspects whether or not the defect X is included in the inspection image P. If the defect X is included, the defect detection unit 130 includes the number, position, and area of the defect X in the defect image Px. It is configured to add and output defect information such as (that is, inspection results). Note that the defect detection unit 130 can also output a learning defect image Pt, the details of which will be described later.
More specifically, the defect detection unit 130 can be exemplified by one that performs inspection based on a black-and-white image captured with a low-magnification lens, or one that performs inspection based on a color image captured with a high-magnification lens.

The control unit 140 centrally controls the inspection system 100. Specifically, the control unit 140 is connected to the automatic defect classification device 1, the transport unit 110, the imaging unit 120, the defect detection unit 130, etc., and inputs and outputs control signals to each unit to detect desired defects. It is configured to cause the automatic defect classification device 1 to acquire the image Px.

FIG. 3 is a schematic diagram showing a main part of an example of a form embodying the present invention. FIG. 3 shows a schematic diagram of an automatic defect classification device 1 according to the present invention.

The automatic defect classification device 1 classifies the type of defect X present in the inspection target W.
Specifically, the automatic defect classification device 1 uses the training data T to generate a classifier for the defect image, and uses the classifier to automatically assign a classification category to the defect image Px whose classification category is unknown. It is intended to give. The training data T includes a plurality of pre-obtained learning defect images Pt (so-called learning images), each of which is given a classification category in advance by the user to specify which defect classification these defect images Pt belong to. It is something.
More specifically, the automatic defect classification device 1 includes a defect image input section GI, a teacher data input section 2, an operation input section 3, a storage section 4, a processing section 5, a display section 6, a result output section RO, etc. It consists of a computer (hardware) and its execution program (software), etc.
The type of defect X in the defect image Px detected by the defect detection unit 130 is determined and a classification category is assigned.

The defect image input section GI is for inputting the defect image Px.
Specifically, the defect image input unit GI acquires the defect image Px output from the defect detection unit 130 or the like. Note that the defect image input unit GI can also acquire an additional learning defect image Pt output from the defect detection unit 130 or the like.

The teacher data input section 2 is for inputting the teacher data T.
Specifically, the teacher data input unit 2 acquires the teacher data T output from an external device, a host computer, or the like. More specifically, the teacher data input unit 2 acquires the teacher data T via a recording medium, a communication line, or the like.
Note that the teacher data T stores a plurality of defect images Pt and the classification categories of the defects X included in these defect images Pt, respectively, in association with each other.

The operation input unit 3 accepts cursor (also referred to as pointer) operations and numerical inputs by the user.
Specifically, the operation input section 3 is used to select an image displayed on the display section 6, select/change setting items, and input numerical values.
Note that the defect image input section GI, teacher data input section 2, and operation input section 3 are configured as part of the input section DI of the computer.

The storage unit 4 stores teacher data T, defect images Pt, Px, programs used for each process, and the like.
Specifically, the storage unit 4 is composed of a computer memory or an auxiliary storage device (SSD, HDD, etc.).

The processing unit 5 performs predetermined processing on the plurality of defect images Pt and unknown defect images Px. Furthermore, the processing section 5 changes the display format of the defective part Q of the defective image Pt on the display section 6 and displays it.
Specifically, the processing unit 5 evaluates the display format of the defective part Q of the defective image Pt on the display unit 6 based on a predetermined procedure registered in advance, an operation by the user (an example will be described below), etc. The defect image is changed and displayed according to the classification category to which the defect image belongs, assigned by the classifier Bt.
More specifically, the processing unit 5 includes a teacher data evaluation classifier generation unit 51, a feature value calculation unit 52, a classification accuracy calculation unit 53, a classification category determination unit 54, a defective part display change unit 55, and an automatic defect classifier. It includes a generation section 57, a defect category classification section 59, etc., and is composed of a computer (hardware) and its execution program (software).

The teacher data evaluation classifier generation unit 51 generates a plurality of defect classifiers included in the teacher data T before generating an automatic defect classifier Ba for assigning a classification category to a defect image Px whose classification category is unknown. This is to generate an evaluation classifier Bt for evaluating the classification category assigned to the defect image Pt.
Specifically, the teacher data evaluation classifier generation unit 51 performs machine learning based on the teacher data T and generates an evaluation classifier Bt including a decision tree or the like.
More specifically, the teacher data evaluation classifier generation unit 51 generates an evaluation classifier Bt that includes a model (so-called random forest) that generates a plurality of decision trees and determines their prediction results by majority vote or the like. generate.

The feature amount calculation unit 52 performs feature amount calculation processing for each of the plurality of defect images Pt included in the teacher data T based on the evaluation classifier Bt.
Specifically, the feature amount calculation unit 52 calculates the appearance frequency of the feature amount passing through each classification category of the prediction result of the decision tree in the evaluation classifier Bt generated by the teacher data evaluation classifier generation unit 51. The weight of each feature for each classification category is calculated by normalizing the appearance frequency. Then, all of the plurality of defective images Pt included in the training data T are divided into groups for each classification category, and from the feature vectors of the image group of each divided classification category, the degree of coincidence of each feature amount for each classification category (i.e., The acceptance criteria (for example, the range of normal values) for determining the classification accuracy (classification accuracy) are calculated and determined.

The classification accuracy calculation unit 53 sets, for each of the plurality of defect images Pt included in the training data T, the feature amount of the defect image Pt and the classification category that is the same or different from that given to the defect image Pt. The degree of agreement (that is, the classification accuracy) of each feature quantity with each acceptance criterion is calculated.
Specifically, the classification accuracy calculation unit 53 calculates the feature vector of each defect image Pt for each same classification category classified by the user based on the feature amount of each defect image Pt calculated by the feature amount calculation unit 52. Statistical processing is performed to determine whether or not each feature quantity component falls within a normal value range, and the classification accuracy is calculated.

The classification category determination unit 54 determines the classification category to which the defect image Pt assigned by the evaluation classifier Bt belongs.
Specifically, the classification category determining unit 54 determines for each defective image Pt to which classification category it is appropriate for the defective image Pt to belong, based on the classification accuracy calculated by the classification accuracy calculating unit 53.

The defective part display changing unit 55 changes the display format of the defective part Q of the defective image Pt.
Specifically, the defective part display changing unit 55 performs image processing on the defective image Pt based on preset parameters and procedures, and changes the display format of the defective part Q for each classification category to which the defective image Pt belongs. Perform the process to change.
For example, if the classification category to which the defect image Pt belongs is A defect, the defect part Q is changed to "green", if it is B defect, it is changed to "yellow", and if it is C defect, it is changed to "red". .
It should be noted that the user can set in advance which defect should be displayed in which color, and can change it as appropriate.

The automatic defect classifier generation unit 57 generates an automatic defect classifier Ba for assigning a classification category to the defect image Px.
Specifically, the automatic defect classifier generation unit 57 has the same configuration as the teacher data evaluation classifier generation unit 51, and machine learning is performed based on the improved teacher data T' to generate a decision tree. An automatic defect classifier Ba including the following is generated.

The defect category classification unit 59 assigns a classification category to a defect image Px whose classification category is unknown.
Specifically, the defect category classification unit 59 assigns a classification category to the defect image Px whose classification category is unknown, based on the automatic defect classifier Ba.

The display unit 6 displays a plurality of defect images Pt, classification categories assigned to these defect images Pt, and other information regarding the defect images.
Specifically, the display section 6 is composed of a video monitor, a touch panel, etc. (that is, a part of the output section DO of the computer), and displays the defective image Pt etc. that have been subjected to predetermined processing in the processing section 5. It will be done. More specifically, the display section 6 constitutes a user interface U in combination with the above-described operation input section 3.

FIG. 4 is an image diagram showing an example of a display unit in a form embodying the present invention.
FIG. 4 shows an example of the image diagram GA displayed on the display unit 6.

The display section 6 includes a defect image display area G1, an image display setting area G2, a display item setting area G3, a display order setting area G4, a classification information display area G5, etc., and displays display items, parameters, etc. according to user operations. You can change or check the information.

The defect image display area G1 is an area that displays the defect image Pt (image number 001 in this example) that is selected for display. Specifically, the right end of the defect image display area G1 is provided with a vertical scroll bar L11 for vertically scrolling the displayed defect image Pt, and up/down movement buttons B11 and B12, and the lower end is provided with a vertical scroll bar L11 that vertically scrolls the displayed defect image Pt. A horizontal scroll bar L12 for horizontally scrolling the defect image Pt and left/right movement buttons B13 and B14 are provided so that parts that cannot be displayed (that is, hidden) within the defect image display area G1 can be confirmed. It is composed of

The image display setting area G2 is an area for changing and setting the display magnification and brightness of the defect image Pt displayed in the defect image display area G1.
Specifically, the currently set display magnification (×1 in this example) is displayed in the image display setting area G2. Then, when button D21 is pressed, other display magnifications are displayed as a drop-down list. Then, by moving or clicking the cursor C in the drop-down list to select the magnification to be displayed (for example, ×2, ×5, entire image, etc.), the changed display magnification is displayed.
Furthermore, a brightness setting value display area G21 and a slide switch L21 are arranged in the image display setting area G2.
The current brightness setting value is displayed in the brightness setting value display area G21.
The slide switch L21 is for changing the brightness setting value. By placing the cursor C on the slide switch L21 and dragging it to the left or right, the current brightness setting value is changed, and the brightness of the defective image Pt displayed in the defective image display area G1 is changed. Ru.

The display item setting area G3 is an area for setting whether to change the display format of the defective part Q of the defective image Pt displayed in the defective image display area G1.
Specifically, a box button B3 is arranged in the display item setting area G3. Then, each time the box button B3 is clicked, the inside of the box button B3 is alternately switched between white and black, and the display format of the defective part Q of the defective image Pt is changed in accordance with the switching. More specifically, when the box button B3 is white, the defective part Q is displayed as the defective image Pt. On the other hand, when the box button B3 is black, the display format of the defective part Q of the defective image P is changed by applying the present invention, which will be described in detail below, and is displayed with a color or a pattern.

The display order setting area G4 is an area for setting the order of defect images Pt to be displayed in the defect image display area G1.
Specifically, the display order setting area G4 is configured to display multiple display orders registered in advance (for example, data list ascending order, data list descending order, category ascending order, category descending order, classification accuracy ascending order, classification accuracy descending order, etc.). One method is selected and set, and the defect image Pt is displayed using that method.
Note that the data list ascending order/descending order is a method of displaying the defective images Pt in ascending order/descending order based on the serial number assigned to the defective image Pt.
Moreover, the category ascending order/descending order is a method of displaying the defective images Pt in ascending order/descending order based on the classification category assigned to the defective image Pt.
Further, the classification accuracy ascending/descending order is a method of displaying in ascending/descending order based on the classification accuracy (degree of matching) of the classification category assigned to the defective image Pt.

For example, the currently set order (in this example, ascending data list order) is displayed in the display standard setting area G4, but if you press button D41, other display orders will be displayed as a drop-down list. Ru. Then, by moving or clicking the cursor C in the drop-down list to select the desired display order (for example, ascending order by category), the changed display order will be displayed, and the defects in the defect image display area G1 will be displayed. The images Pt are sorted and displayed in ascending order of category.
Further, the display order setting area G4 includes an image number display area G41, a 10-sheet back button B41, a 1-sheet back button B42, a 1-sheet forward button B43, a 10-sheet forward button 44, etc. Each time the cursor C is placed on the button and clicked, the image number displayed in the image number display area G41 is changed, and the defective image Pt of that image number is displayed in the defective image display area G1.

The classification information display area G5 is an area for displaying classification information regarding the image selected by moving or clicking the cursor C (hereinafter referred to as the selected image) among the defect images Pt displayed in the defect image display area G1. be. Specifically, in the classification information display area G5, an MDC display area G51, an MDC candidate display area G52, a classification accuracy display area G53, a classification change destination selection area G54, a change button B53, etc. are arranged.

The MDC display area G51 is an area that displays the classification category assigned by the user to the selected image. For example, "NG-A" indicating defect A is displayed in the MDC display area G51.

The MDC candidate display area G52 is an area that displays reclassification candidate categories for the selected image. Specifically, the classification category with the second highest classification accuracy is displayed in the MDC candidate display area G52 as a reclassification candidate category.

The classification accuracy display area G53 displays the classification accuracy (that is, the degree of coincidence) of the classification category to which the defect image Pt displayed in the defect image display area G1 belongs. Specifically, each classification category and classification accuracy are displayed in a table format in the classification accuracy display area G53.
Further, at the right end of the classification accuracy display area G53, a vertical scroll bar L51 for vertically scrolling multiple classification categories and up/down movement buttons B51 and B52 are provided, so that they cannot be displayed all at once in the classification accuracy display area G53. It is configured so that other (that is, hidden) classification categories and classification accuracy can be checked.

The classification category change destination selection area G54 is an area for selecting which classification category to change to when changing the classification category assigned to the selected image. For example, the currently assigned classification category (NG-A in this example) is displayed, but if you press button D51, other classification categories (NG-B, C, D,...) will drop. Displayed as a down list. Then, by moving or clicking the cursor C in the drop-down list and selecting the classification category you wish to change (for example, NG-C), the classification category to be changed will be displayed (However, at this point The teacher data T has not been changed yet).

The change button B53 is a switch button for deciding to change the classification category. By pressing the change button B53, the classification category displayed in the classification category change destination selection area G54 is changed to the teacher data T as the classification category of the selected image.

Note that buttons B4 and B5 are arranged at the lower right of the image diagram GA.
When the button B4 is pressed, the parameter changes, etc. are saved, the displayed window is closed, the teacher data T is improved, and the improved teacher data T' is generated.
On the other hand, when button B5 is pressed, the parameter changes etc. are not saved and the displayed window is closed (that is, no improvement is made to the teacher data T and no teacher data T' is generated).

The result output unit RO outputs the classification results.
Specifically, for a defect image Px with an unknown classification category acquired by the defect image input unit GI, a classification category automatically assigned based on the automatic defect classifier Ba (that is, a classification result) is transmitted to an external device. It is output to a device, host computer, etc.
More specifically, the result output section RO is constituted by a part of the output section DO of the computer.

[Processing flow]
FIG. 5 is a flow diagram illustrating an example of an embodiment of the present invention. FIG. 5 shows a flow for improving the accuracy of automatic defect classification by using the automatic defect classification device 1 according to the present invention to correct teacher data T containing errors in the classification categories assigned by the user. .

First, teacher data T output from an external device, host computer, etc. is input from the teacher data input section 2 (step s1). Specifically, the teacher data T is output from an external device, a host computer, etc. based on a user's instruction.

Next, the teacher data evaluation classifier generation unit 51 generates an evaluation classifier Bt based on the teacher data T (step s2). Specifically, when the user presses the "execute" button displayed on any screen, machine learning based on the current training data T is started, and the process of generating the evaluation classifier Bt is performed. .

Next, the feature amount calculation unit 52 performs feature amount calculation processing for each of the plurality of defect images Pt included in the teacher data T based on the evaluation classifier Bt (step s3).

Next, in the classification accuracy calculation unit 53, for each of the plurality of defect images Pt included in the teacher data T, the feature amount of the defect image Pt and the same and different classification categories as those assigned to the defect image Pt are calculated. The degree of coincidence (that is, the classification accuracy) of each of the feature amounts set with the acceptance criteria is calculated (step s4).

Next, the classification category determining unit 54 determines, for each of the plurality of defective images Pt, the classification category to which the defective image Pt assigned by the evaluation classifier Bt belongs (step s5).

Next, the display format of the defect site Q of the defect image Pt on the display unit 6 is changed and displayed (step s6).

After this, the user confirms the appropriateness of the classification category of the defective image Pt, and determines whether it is necessary to change (that is, modify) the classification category included in the teacher data T (step s7).

If necessary, the classification category is changed for one or more defective images Pt (step s8), and the training data T is improved to generate improved training data T'.

Thereafter, the teacher data evaluation classifier generation unit 51 performs machine learning (relearning) based on the improved teacher data T' and determines whether to regenerate the evaluation classifier Bt (step s9). If relearning is necessary, the steps s2 to s9 described above are repeated, and if relearning is not necessary, the series of flows is ended.

Thereafter, machine learning is performed using the automatic defect classifier generation unit 57 based on the latest (that is, improved) teacher data T', and an automatic defect classifier Ba is generated. Thereafter, in the automatic defect classification device 1 according to the present invention, the defect category classification unit 59 uses the automatic defect classifier Ba to assign a classification category to the defect image Px whose classification category is unknown.

Since the automatic defect classification device 1 according to the present invention has such a configuration, the processing section 5 performs predetermined processing on the plurality of defect images Pt, and the defective part Q of the defective image Pt on the display section 6 is displayed. The display format of can be changed and displayed.
Therefore, even if there are many defect images Pt that constitute the training data T, it is possible to easily correct errors in the classification categories assigned by the user and improve the accuracy of automatic defect classification.

[About display mode]
In embodying the present invention, the processing unit 5 is not limited to the above-described configuration, and is provided with a single image display mode and an image list display mode, and in the image list display mode, any defective image Pt selected by the user can be displayed. may be configured to be displayed in a single image display mode.

The "single image display mode" is a mode in which one image selected from a plurality of defect images Pt classified into the same classification category is displayed on the display unit 6. Note that the image diagram GA displayed on the display unit 6 illustrated in FIG. 4 corresponds to display in the "single image display mode", so detailed explanation will be omitted.
On the other hand, the "image list display mode" is a mode in which a plurality of defect images Pt classified into the same classification category are arranged in a matrix and displayed on the display unit 6.

FIG. 6 is an image diagram showing an example of a display unit in a form embodying the present invention.
FIG. 6 shows an example of an image diagram GB displayed on the display unit 6 in the "image list display mode".

When switching the display mode, buttons B1 and B2 are arranged above the defect image display area G1 of the display section 6. Button B1 is used to switch and display the defect image display area G1 to the "single image display mode". On the other hand, button B2 is used to switch and display the defect image display area G1 to the "image list display mode".
Specifically, when button B2 is pressed in the state of image diagram GA in "single image display mode" as shown in FIG. 4, image diagram GB in "image list display mode" as shown in FIG. When the button B1 is pressed, the image is switched to the "single image display mode" image view GA as shown in FIG.

In the "image list display mode", the defect image display area G1 is an area that displays a plurality of defect images Pt and classification categories assigned to the defect images Pt.
Specifically, a plurality of defect images Pt are displayed in a matrix (also referred to as a tile) in the defect image display area G1, and classification categories are displayed below the defect images Pt. Further, at the right end of the defect image display area G1, a vertical scroll bar L11 for vertically scrolling the plurality of defect images Pt and up/down movement buttons B11, B12 are provided, and at the lower end, the plurality of defect images Pt are displayed. A horizontal scroll bar L12 for horizontal scrolling and left/right movement buttons B13 and B14 are provided so that other defective images Pt that cannot be displayed (that is, hidden) within the defective image display area G1 can be confirmed. It is configured.

The "image list display mode" is based on the display order set in the display order setting area G4 (for example, data list ascending order, data list descending order, category ascending order, category descending order, classification accuracy ascending order, classification accuracy descending order, etc.). , a plurality of defect images Pt are displayed.
Specifically, if the data list is in ascending/descending order, a plurality of defective images Pt are arranged and displayed in ascending/descending order from the upper left end to the right side and the bottom based on the serial number assigned to the defective image Pt. .
On the other hand, in ascending/descending category order, a plurality of defective images Pt are arranged and displayed in ascending/descending order from the upper left end to the right side and the bottom, based on the classification category assigned to the defective image Pt.
On the other hand, if the classification accuracy is in ascending/descending order, a plurality of defective images Pt are arranged in ascending/descending order from the upper left corner to the right side and the bottom, based on the classification accuracy (degree of coincidence) of the classification category assigned to the defective image Pt. are arranged and displayed.
In addition, in the "image list display mode", the image number of the defective image Pt selected by the user (for example, the user moves the cursor C on the defective image Pt and single-clicks it) is displayed in the image number display area G41. .

[About switching operation]
When there is a "single image display mode" and an "image list display mode", the processing unit 5 can select any defective image Pt selected by the user if the user performs the following operations in the "image list display mode". Switch to "single image display mode" and display.

Specifically, in the state of the image diagram GB in the "image list display mode" as shown in FIG. By moving the cursor C and double-clicking, the "single image display mode" is displayed for the defective image Pt.
Alternatively, the user can move the cursor C over the defective image Pt (in this example, the second from the left on the top row) that is desired to be enlarged and single-click it to enter the image selection state, and then press the button B1. The defective image Pt is displayed in "single image display mode".

FIG. 7 is an image diagram showing an example of a display unit in a form embodying the present invention.
FIG. 7 illustrates an image diagram GC in which an arbitrary defect image Pt (in this example, image number 002) selected by the user is displayed in the defect image display area G1 of the display unit 6 in "single image display mode". has been done.

With such a configuration, it is possible to view the colors and patterns of the defective parts Q of multiple defective images Pt from a bird's-eye view in the image list display mode, or to display noteworthy defect images Pt in the single image display mode. This is preferable because it makes it possible to judge the appropriateness of the classification category.
For example, in the image list display mode, it is useful for the user to compare defect images Pt in which defective parts Q have the same display format (color, pattern, etc.) and judge whether it is appropriate that the same classification category is assigned. On the other hand, in the single screen display mode, while the defect images Pt are sent one by one, the defect parts Q displayed in the same display format (color, pattern, etc.) are unnatural (the defect part Q is located far away). It is possible to check whether there is a defective part Q (for example, if the defective part Q is not present where it should be).

Therefore, even if there are many defect images Pt that constitute the training data T, it is possible to easily correct errors in the classification categories assigned in advance by the user, and improve the accuracy of automatic defect classification.

Note that in embodying the present invention, such switching of display modes is not an essential function, and the defect image Pt may be displayed in the same display method as either the single screen display mode or the image list display mode described above. Also good. Even in this case, errors in the classification category can be corrected more easily than in the case where the display format of the defective part Q is not changed (conventional display), and the accuracy of automatic defect classification can be improved.

In the above description, the processing unit 5 has a configuration in which the display format of the defective part Q of the defective image on the display unit 6 is changed and displayed according to the classification category to which the defective image Pt belongs, assigned by the classifier.

However, instead of or in addition to such a configuration, the processing unit 5 changes the display format of the defective part Q of the defective image on the display unit 6 to the format to which the defective image Pt assigned by the classifier belongs. The configuration may be such that the information is changed and displayed depending on the classification accuracy of the classification category.

FIG. 8 is an image diagram showing an example of a display unit in a mode embodying the present invention.
FIGS. 8(a) to 8(c) each show an example of a defective part Q of the defective image Pt displayed in the defective image display area G1 of the display unit 6.
The defect images Pt shown in FIGS. 8(a) to 8(c) all belong to the same classification category (eg, classified as defect A), but have different classification accuracies.

In this example, the hatching (that is, the pattern) of the defective part Q is changed and displayed according to the classification accuracy of the classification category, but the color of the defective part Q may be changed. For example, if the classification accuracy is high, it is displayed in "green", if the classification accuracy is low, it is displayed in "red", and if it is medium, it is displayed in "yellow".

With such a configuration, the classification accuracy is low even when checking the defective images Pt one by one in the single screen display mode, or when checking multiple defective images Pt at once in the image list display mode. This is preferable because it allows the user to recognize the object and then determine whether or not it is necessary to modify the classification category.

1 Automatic defect classification device DI input section GI defect image input section 2 Teacher data input section 3 Operation input section 4 Storage section 5 Processing section 6 Display section DO Output section RO Result output section U User interface 51 Classifier generation section for teacher data evaluation 52 Feature value calculation unit 53 Classification accuracy calculation unit
54 Classification category judgment section
55 Defective part display change section
57 Automatic defect classifier generation section 59 Defect category classification section G1 Defect image display area G2 Image display setting area
G3 Display item setting area
G4 Display order setting area
G5 Classification information display area
P Inspection image Pt Defect image for learning (classification category already assigned)
Px inspection image (classification category unknown)
T Teacher data (before improvement)
T' Teacher data (after improvement)
Ba Automatic defect classifier Bt Classifier for evaluation

Claims

A classifier for defective images is generated using training data in which a classification category that clearly indicates which defect classification the defective image belongs to is assigned in advance by the user to each of the plurality of defective images, and the classifier is used to generate a classifier for the defective image. In an automatic defect classification device that automatically assigns a classification category to a defect image whose classification category is unknown,
a teacher data input unit for inputting the teacher data;
a processing unit that performs predetermined processing on the plurality of defect images;
comprising a display unit that displays the defect image;
The processing unit is configured to change and display a display format of a defective part of the defective image on the display unit in accordance with the classification category to which the defective image belongs, assigned by the classifier. Classification device.
A classifier for defective images is generated using training data in which a classification category that clearly indicates which defect classification the defective image belongs to is assigned in advance by the user to each of the plurality of defective images, and the classifier is used to generate a classifier for the defective image. In an automatic defect classification device that automatically assigns a classification category to a defect image whose classification category is unknown,
a teacher data input unit for inputting the teacher data;
a processing unit that performs predetermined processing on the plurality of defect images;
comprising a display unit that displays the defect image;
The processing section is characterized in that the display format of the defective part of the defective image on the display section is changed and displayed in accordance with the classification accuracy of the classification category to which the defective image belongs, assigned by the classifier. , automatic defect classifier.
The processing unit includes:
a single image display mode in which the display unit displays a selected one of the plurality of defective images classified into the same classification category;
an image list display mode in which a plurality of defective images classified into the same classification category are arranged in a matrix and displayed on the display unit;
The automatic defect classification apparatus according to Claim 1 or Claim 2, characterized in that any defect image selected by the user in the image list display mode is displayed in the single image display mode.